2024-05-09T01:48:25.942 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T01:48:25.942 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/postgres}} --verbose --archive /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698978 --owner scheduled_pdonnell@teuthology --name pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698978/orig.config.yaml 2024-05-09T01:48:26.004 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T01:48:26.090 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698978 branch: wip-pdonnell-testing-20240508.183908-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7698978' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240508.183908-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.183908-debug sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1722 sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.183908-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNpPt5pvoTf/QUE3Qw7ZhDe8wN2XJgFEpK1iYetqP9rrPORLdmXVn1sAO22do/R5jTdzlq51Vy4ya01FYXPT/7k= smithi059.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJyAe1aVc8WSlgbVJJ7Ub0JuYXRjw3AL9ZyvspIsG8GCBZkWkTSKBbWyxHXAl9e+4H1RfVdjEp/BJvQTeSS0MJM= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDGgvxeWvVzeUKEEa25+F50/1JBbqMVJYDjjJ3nQkJNHO0on/xoW7+ll3NmvvKah/H4GZ5cAoSiKu7cOAeKs8kg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_22:06:20 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T01:48:26.090 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa; will attempt to use it 2024-05-09T01:48:26.091 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks 2024-05-09T01:48:26.091 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T01:48:26.092 INFO:teuthology.task.internal:Checking packages... 2024-05-09T01:48:26.117 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c9da4049f638c0f84bdd9b4e33c42e86046ac554' 2024-05-09T01:48:26.117 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T01:48:26.117 INFO:teuthology.packaging:ref: None 2024-05-09T01:48:26.117 INFO:teuthology.packaging:tag: None 2024-05-09T01:48:26.117 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.183908-debug 2024-05-09T01:48:26.117 INFO:teuthology.packaging:sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:48:26.117 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240508.183908-debug 2024-05-09T01:48:26.341 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3594.gc9da4049 2024-05-09T01:48:26.343 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T01:48:26.351 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T01:48:26.351 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T01:48:26.402 INFO:teuthology.task.internal:Saving configuration 2024-05-09T01:48:26.423 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T01:48:26.435 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T01:48:26.458 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698978', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 01:43:27.034149', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNpPt5pvoTf/QUE3Qw7ZhDe8wN2XJgFEpK1iYetqP9rrPORLdmXVn1sAO22do/R5jTdzlq51Vy4ya01FYXPT/7k='} 2024-05-09T01:48:26.479 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698978', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 01:43:27.032911', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJyAe1aVc8WSlgbVJJ7Ub0JuYXRjw3AL9ZyvspIsG8GCBZkWkTSKBbWyxHXAl9e+4H1RfVdjEp/BJvQTeSS0MJM='} 2024-05-09T01:48:26.499 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698978', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 01:43:27.031654', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDGgvxeWvVzeUKEEa25+F50/1JBbqMVJYDjjJ3nQkJNHO0on/xoW7+ll3NmvvKah/H4GZ5cAoSiKu7cOAeKs8kg='} 2024-05-09T01:48:26.499 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T01:48:26.506 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-09T01:48:26.506 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-09T01:48:26.506 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-09T01:48:26.506 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T01:48:26.645 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f16f0c270d0>, signals=[15]) 2024-05-09T01:48:26.645 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T01:48:26.652 INFO:teuthology.task.internal:Opening connections... 2024-05-09T01:48:26.652 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T01:48:26.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:48:26.730 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi059.front.sepia.ceph.com 2024-05-09T01:48:26.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:48:26.804 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-05-09T01:48:26.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:48:26.880 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T01:48:26.887 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-05-09T01:48:26.906 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-05-09T01:48:26.907 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-05-09T01:48:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-05-09T01:48:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-05-09T01:48:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-05-09T01:48:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-05-09T01:48:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-05-09T01:48:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-05-09T01:48:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T01:48:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-05-09T01:48:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-05-09T01:48:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T01:48:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-05-09T01:48:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T01:48:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T01:48:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T01:48:26.966 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-05-09T01:48:26.991 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2024-05-09T01:48:27.009 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2024-05-09T01:48:27.009 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2024-05-09T01:48:27.066 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2024-05-09T01:48:27.066 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="9" 2024-05-09T01:48:27.066 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2024-05-09T01:48:27.067 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2024-05-09T01:48:27.067 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="9" 2024-05-09T01:48:27.067 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el9" 2024-05-09T01:48:27.067 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T01:48:27.067 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2024-05-09T01:48:27.067 INFO:teuthology.orchestra.run.smithi059.stdout:LOGO="fedora-logo-icon" 2024-05-09T01:48:27.067 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T01:48:27.067 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2024-05-09T01:48:27.067 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T01:48:27.067 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T01:48:27.067 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T01:48:27.067 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2024-05-09T01:48:27.082 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-05-09T01:48:27.099 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-05-09T01:48:27.099 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-05-09T01:48:27.156 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-05-09T01:48:27.156 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-05-09T01:48:27.156 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-05-09T01:48:27.156 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-05-09T01:48:27.156 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-05-09T01:48:27.157 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-05-09T01:48:27.157 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T01:48:27.157 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-05-09T01:48:27.157 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-05-09T01:48:27.157 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T01:48:27.157 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-05-09T01:48:27.157 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T01:48:27.157 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T01:48:27.157 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T01:48:27.157 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-05-09T01:48:27.181 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T01:48:27.192 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T01:48:27.219 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T01:48:27.219 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-05-09T01:48:27.221 DEBUG:teuthology.orchestra.run.smithi059:> test '!' -e /home/ubuntu/cephtest 2024-05-09T01:48:27.224 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-05-09T01:48:27.239 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T01:48:27.252 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T01:48:27.252 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-05-09T01:48:27.279 DEBUG:teuthology.orchestra.run.smithi059:> test -z $(ls -A /var/lib/ceph) 2024-05-09T01:48:27.281 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-05-09T01:48:27.375 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T01:48:27.451 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T01:48:27.464 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-09T01:48:27.464 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-09T01:48:27.464 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-09T01:48:27.464 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T01:48:27.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:48:27.481 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-05-09T01:48:27.538 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-437.el9.x86_64 2024-05-09T01:48:27.539 INFO:teuthology.task.kernel:Running kernel on smithi019: 5.14.0-437.el9.x86_64 2024-05-09T01:48:27.539 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2024-05-09T01:48:29.893 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-05-09T01:48:32.169 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-05-09T01:48:36.478 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 9.8 MB/s | 6.2 MB 00:00 2024-05-09T01:48:38.770 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-05-09T01:48:40.464 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-09T01:48:45.797 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-05-09T01:48:47.903 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T01:48:47.903 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T01:48:48.011 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-05-09T01:48:48.020 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-05-09T01:48:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-05-09T01:48:48.020 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-05-09T01:48:48.020 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-05-09T01:48:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T01:48:48.020 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-05-09T01:48:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T01:48:48.020 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T01:48:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T01:48:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:48:48.021 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-05-09T01:48:48.021 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-05-09T01:48:48.021 INFO:teuthology.orchestra.run.smithi019.stdout:Install 4 Packages 2024-05-09T01:48:48.021 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:48:48.021 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 78 M 2024-05-09T01:48:48.022 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 123 M 2024-05-09T01:48:48.022 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-05-09T01:48:48.173 INFO:teuthology.orchestra.run.smithi019.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 5.4 MB/s | 484 kB 00:00 2024-05-09T01:48:48.674 INFO:teuthology.orchestra.run.smithi019.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 26 MB/s | 15 MB 00:00 2024-05-09T01:48:49.207 INFO:teuthology.orchestra.run.smithi019.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 30 MB/s | 34 MB 00:01 2024-05-09T01:48:49.633 INFO:teuthology.orchestra.run.smithi019.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 19 MB/s | 28 MB 00:01 2024-05-09T01:48:49.633 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-05-09T01:48:49.633 INFO:teuthology.orchestra.run.smithi019.stdout:Total 48 MB/s | 78 MB 00:01 2024-05-09T01:48:50.344 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-05-09T01:48:50.737 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-05-09T01:48:50.737 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-05-09T01:48:51.714 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-05-09T01:48:51.714 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-05-09T01:48:54.274 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-05-09T01:48:54.923 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T01:48:54.999 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T01:48:57.814 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T01:48:58.066 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T01:49:03.137 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T01:49:03.294 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T01:49:08.388 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T01:49:39.539 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T01:49:39.579 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T01:49:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T01:49:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T01:49:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T01:49:39.959 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T01:49:40.900 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T01:49:40.901 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:49:40.901 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-05-09T01:49:40.901 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T01:49:40.901 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T01:49:40.901 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T01:49:40.901 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T01:49:40.901 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:49:40.901 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-05-09T01:49:41.206 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T01:49:41.281 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T01:49:41.281 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T01:49:41.281 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T01:49:41.281 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T01:49:41.282 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T01:49:41.282 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-09T01:49:41.282 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi019.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T01:49:41.282 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T01:49:41.333 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T01:49:41.333 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T01:49:41.334 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -qi grub2-tools 2024-05-09T01:49:41.365 INFO:teuthology.orchestra.run.smithi019.stdout:Name : grub2-tools 2024-05-09T01:49:41.365 INFO:teuthology.orchestra.run.smithi019.stdout:Epoch : 1 2024-05-09T01:49:41.365 INFO:teuthology.orchestra.run.smithi019.stdout:Version : 2.06 2024-05-09T01:49:41.365 INFO:teuthology.orchestra.run.smithi019.stdout:Release : 68.el9 2024-05-09T01:49:41.365 INFO:teuthology.orchestra.run.smithi019.stdout:Architecture: x86_64 2024-05-09T01:49:41.365 INFO:teuthology.orchestra.run.smithi019.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T01:49:41.365 INFO:teuthology.orchestra.run.smithi019.stdout:Group : Unspecified 2024-05-09T01:49:41.365 INFO:teuthology.orchestra.run.smithi019.stdout:Size : 8264254 2024-05-09T01:49:41.365 INFO:teuthology.orchestra.run.smithi019.stdout:License : GPLv3+ 2024-05-09T01:49:41.365 INFO:teuthology.orchestra.run.smithi019.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T01:49:41.366 INFO:teuthology.orchestra.run.smithi019.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T01:49:41.366 INFO:teuthology.orchestra.run.smithi019.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T01:49:41.366 INFO:teuthology.orchestra.run.smithi019.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T01:49:41.366 INFO:teuthology.orchestra.run.smithi019.stdout:Packager : builder@centos.org 2024-05-09T01:49:41.366 INFO:teuthology.orchestra.run.smithi019.stdout:Vendor : CentOS 2024-05-09T01:49:41.366 INFO:teuthology.orchestra.run.smithi019.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T01:49:41.366 INFO:teuthology.orchestra.run.smithi019.stdout:Summary : Support tools for GRUB. 2024-05-09T01:49:41.366 INFO:teuthology.orchestra.run.smithi019.stdout:Description : 2024-05-09T01:49:41.366 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:49:41.366 INFO:teuthology.orchestra.run.smithi019.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T01:49:41.366 INFO:teuthology.orchestra.run.smithi019.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T01:49:41.366 INFO:teuthology.orchestra.run.smithi019.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T01:49:41.366 INFO:teuthology.orchestra.run.smithi019.stdout:hardware devices. 2024-05-09T01:49:41.366 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:49:41.367 INFO:teuthology.orchestra.run.smithi019.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T01:49:41.367 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T01:49:41.368 INFO:teuthology.task.kernel:Updating grub on smithi019 to boot 5.14.0-444.el9.x86_64 2024-05-09T01:49:41.368 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T01:49:42.163 INFO:teuthology.orchestra.run.smithi019.stderr:Generating grub configuration file ... 2024-05-09T01:49:43.509 INFO:teuthology.orchestra.run.smithi019.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T01:49:43.541 INFO:teuthology.orchestra.run.smithi019.stderr:done 2024-05-09T01:49:43.543 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-05-09T01:49:43.607 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.7WI5nFdR49 2024-05-09T01:49:43.607 DEBUG:teuthology.orchestra.run.smithi019:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7WI5nFdR49 2024-05-09T01:49:43.673 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.7WI5nFdR49 2024-05-09T01:49:43.810 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.7WI5nFdR49 is 6KB 2024-05-09T01:49:43.858 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.7WI5nFdR49 2024-05-09T01:49:43.873 DEBUG:teuthology.orchestra.run.smithi019:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T01:49:43.936 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T01:49:43.936 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T01:49:43.936 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T01:49:43.936 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T01:49:43.937 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T01:49:44.090 DEBUG:teuthology.orchestra.run.smithi019:> sudo shutdown -r now 2024-05-09T01:50:14.093 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-09T01:50:14.093 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-09T01:50:14.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:50:32.482 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-05-09T01:50:41.483 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-09T01:50:41.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:51:41.487 DEBUG:teuthology.orchestra.remote:timed out 2024-05-09T01:51:53.487 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-09T01:51:53.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:51:53.813 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-05-09T01:51:54.198 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-09T01:51:54.198 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-09T01:51:54.198 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-05-09T01:51:54.215 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-444.el9.x86_64 2024-05-09T01:51:54.215 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T01:51:54.215 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T01:51:54.215 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T01:51:55.216 DEBUG:teuthology.parallel:result is None 2024-05-09T01:51:55.217 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T01:51:55.223 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T01:51:55.223 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T01:51:55.226 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T01:51:55.229 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T01:51:55.247 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T01:51:55.277 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T01:51:55.327 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T01:51:55.327 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T01:51:55.329 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T01:51:55.332 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T01:51:55.459 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T01:51:55.465 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T01:51:55.466 DEBUG:teuthology.orchestra.run.smithi059:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T01:51:55.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:51:55.484 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T01:51:55.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:51:55.500 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T01:51:55.502 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T01:51:55.527 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T01:51:55.561 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T01:51:55.562 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T01:51:55.595 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T01:51:55.599 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T01:51:55.616 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T01:51:55.706 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T01:51:55.707 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T01:51:55.714 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T01:51:55.714 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T01:51:55.717 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T01:51:55.750 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T01:51:55.779 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T01:51:55.787 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T01:51:55.787 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T01:51:55.789 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T01:51:55.819 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T01:51:55.836 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T01:51:55.887 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T01:51:55.942 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T01:51:56.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T01:51:56.070 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:51:56.070 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T01:51:56.138 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T01:51:56.185 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T01:51:56.241 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T01:51:56.327 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T01:51:56.369 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:51:56.369 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T01:51:56.436 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T01:51:56.477 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T01:51:56.534 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T01:51:56.616 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T01:51:56.660 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:51:56.660 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T01:51:56.728 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-05-09T01:51:56.730 DEBUG:teuthology.orchestra.run.smithi059:> sudo service rsyslog restart 2024-05-09T01:51:56.733 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-05-09T01:51:56.767 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T01:51:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T01:51:56.803 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T01:51:57.253 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T01:51:57.261 INFO:teuthology.task.internal:Starting timer... 2024-05-09T01:51:57.261 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T01:51:57.326 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T01:51:57.379 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-09T01:51:57.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-05-09T01:51:57.456 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-05-09T01:51:57.458 DEBUG:teuthology.orchestra.run.smithi059:> sudo service auditd rotate 2024-05-09T01:51:57.526 INFO:teuthology.orchestra.run.smithi059.stdout:Rotating logs: 2024-05-09T01:51:57.528 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-05-09T01:51:57.583 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-05-09T01:51:57.585 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T01:51:57.586 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-05-09T01:51:57.620 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-05-09T01:51:57.620 DEBUG:teuthology.orchestra.run.smithi059:> /usr/sbin/getenforce 2024-05-09T01:51:57.644 INFO:teuthology.orchestra.run.smithi059.stdout:Permissive 2024-05-09T01:51:57.645 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-05-09T01:51:57.666 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-05-09T01:51:57.667 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi059.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-05-09T01:51:57.667 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T01:51:57.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:51:57.696 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T01:51:57.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:51:57.726 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T01:51:57.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:51:57.754 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T01:51:57.755 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-05-09T01:51:57.783 DEBUG:teuthology.orchestra.run.smithi059:> sudo /usr/sbin/setenforce permissive 2024-05-09T01:51:57.809 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-05-09T01:51:57.834 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T01:51:57.844 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-09T01:51:57.849 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-09T01:51:57.988 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T01:51:58.003 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T01:51:58.005 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi059.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T01:56:27.642 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi059.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-05-09T01:56:27.643 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-09T01:56:27.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:56:27.720 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-05-09T01:56:27.798 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-09T01:56:27.798 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-09T01:56:27.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:56:27.875 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-05-09T01:56:27.956 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-09T01:56:27.956 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-05-09T01:56:27.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T01:56:28.035 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-05-09T01:56:28.109 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-05-09T01:56:28.110 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T01:56:28.120 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T01:56:28.120 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T01:56:28.121 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T01:56:28.124 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T01:56:28.124 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T01:56:28.126 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T01:56:28.126 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T01:56:28.154 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T01:56:28.158 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T01:56:28.170 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T01:56:28.173 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T01:56:28.176 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T01:56:28.189 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T01:56:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-05-09T01:56:28.199 INFO:teuthology.orchestra.run.smithi059.stderr:sudo: ntpd: command not found 2024-05-09T01:56:28.209 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-05-09T01:56:28.211 INFO:teuthology.orchestra.run.smithi059.stdout:506 Cannot talk to daemon 2024-05-09T01:56:28.211 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-05-09T01:56:28.221 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-05-09T01:56:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T01:56:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T01:56:28.234 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T01:56:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T01:56:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T01:56:28.248 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T01:56:28.295 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-05-09T01:56:28.299 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T01:56:28.299 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-05-09T01:56:28.299 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:56:28.299 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:56:28.299 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:56:28.299 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:56:28.300 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-05-09T01:56:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-05-09T01:56:28.303 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T01:56:28.303 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-05-09T01:56:28.304 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:56:28.304 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:56:28.304 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:56:28.304 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:56:28.305 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T01:56:28.305 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-05-09T01:56:28.306 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:56:28.306 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:56:28.306 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:56:28.306 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:56:28.306 INFO:teuthology.run_tasks:Running task install... 2024-05-09T01:56:28.315 DEBUG:teuthology.task.install:project ceph 2024-05-09T01:56:28.315 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'}} 2024-05-09T01:56:28.315 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'} 2024-05-09T01:56:28.315 INFO:teuthology.task.install:Using flavor: default 2024-05-09T01:56:28.321 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T01:56:28.321 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T01:56:28.322 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T01:56:28.322 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:56:28.324 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T01:56:28.324 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:56:28.325 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T01:56:28.325 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:56:28.482 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T01:56:28.482 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T01:56:28.487 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T01:56:28.487 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T01:56:28.561 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T01:56:28.561 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T01:56:28.594 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T01:56:28.595 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:56:28.595 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T01:56:28.619 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T01:56:28.619 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T01:56:28.664 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T01:56:28.664 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:56:28.664 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T01:56:28.669 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T01:56:28.670 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:56:28.670 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T01:56:28.684 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T01:56:28.691 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T01:56:28.691 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T01:56:28.698 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T01:56:28.698 DEBUG:teuthology.orchestra.run.smithi059:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T01:56:28.757 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T01:56:28.759 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T01:56:28.765 DEBUG:teuthology.orchestra.run.smithi059:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T01:56:28.820 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-05-09T01:56:28.821 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-05-09T01:56:28.835 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T01:56:28.840 DEBUG:teuthology.orchestra.run.smithi059:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T01:56:28.901 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-05-09T01:56:28.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-05-09T01:56:28.904 INFO:teuthology.orchestra.run.smithi059.stdout:check_obsoletes = 1 2024-05-09T01:56:28.906 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum clean all 2024-05-09T01:56:29.048 INFO:teuthology.orchestra.run.smithi196.stdout:54 files removed 2024-05-09T01:56:29.076 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T01:56:29.143 INFO:teuthology.orchestra.run.smithi059.stdout:54 files removed 2024-05-09T01:56:29.154 INFO:teuthology.orchestra.run.smithi019.stdout:54 files removed 2024-05-09T01:56:29.171 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T01:56:29.182 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T01:56:29.861 INFO:teuthology.orchestra.run.smithi059.stdout:ceph packages for x86_64 203 kB/s | 84 kB 00:00 2024-05-09T01:56:29.945 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 178 kB/s | 84 kB 00:00 2024-05-09T01:56:30.042 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 119 kB/s | 84 kB 00:00 2024-05-09T01:56:30.423 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-05-09T01:56:30.470 INFO:teuthology.orchestra.run.smithi059.stdout:ceph noarch packages 21 kB/s | 12 kB 00:00 2024-05-09T01:56:30.545 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-05-09T01:56:30.896 INFO:teuthology.orchestra.run.smithi059.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-05-09T01:56:30.912 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-05-09T01:56:31.007 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 3.5 kB/s | 1.9 kB 00:00 2024-05-09T01:56:31.788 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.1 MB 00:00 2024-05-09T01:56:33.762 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.1 MB 00:02 2024-05-09T01:56:35.115 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 8.1 MB 00:04 2024-05-09T01:56:35.780 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-05-09T01:56:40.181 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.2 MB 00:00 2024-05-09T01:56:42.590 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-05-09T01:56:43.866 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-05-09T01:56:48.645 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-05-09T01:56:49.225 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-09T01:56:50.606 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-05-09T01:56:51.644 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T01:56:51.645 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T01:56:51.656 INFO:teuthology.orchestra.run.smithi019.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T01:56:51.657 INFO:teuthology.orchestra.run.smithi019.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T01:56:51.658 INFO:teuthology.orchestra.run.smithi019.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T01:56:51.659 INFO:teuthology.orchestra.run.smithi019.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T01:56:51.661 INFO:teuthology.orchestra.run.smithi019.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T01:56:51.663 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T01:56:51.664 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T01:56:51.729 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-05-09T01:56:51.740 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-05-09T01:56:51.741 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-05-09T01:56:51.741 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-05-09T01:56:51.741 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-05-09T01:56:51.741 INFO:teuthology.orchestra.run.smithi019.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T01:56:51.741 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T01:56:51.741 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T01:56:51.741 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T01:56:51.741 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T01:56:51.741 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T01:56:51.741 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T01:56:51.742 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T01:56:51.742 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T01:56:51.742 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T01:56:51.742 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T01:56:51.742 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T01:56:51.742 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T01:56:51.742 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 763 k 2024-05-09T01:56:51.742 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T01:56:51.742 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T01:56:51.742 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T01:56:51.742 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T01:56:51.742 INFO:teuthology.orchestra.run.smithi019.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T01:56:51.743 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T01:56:51.743 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T01:56:51.743 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T01:56:51.743 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T01:56:51.743 INFO:teuthology.orchestra.run.smithi019.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T01:56:51.743 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T01:56:51.743 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T01:56:51.743 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T01:56:51.743 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T01:56:51.743 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T01:56:51.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 162 k 2024-05-09T01:56:51.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 321 k 2024-05-09T01:56:51.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 301 k 2024-05-09T01:56:51.744 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T01:56:51.744 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T01:56:51.744 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T01:56:51.744 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T01:56:51.744 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T01:56:51.744 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-05-09T01:56:51.744 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T01:56:51.744 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T01:56:51.744 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T01:56:51.744 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T01:56:51.744 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T01:56:51.744 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T01:56:51.744 INFO:teuthology.orchestra.run.smithi019.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T01:56:51.744 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T01:56:51.745 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-05-09T01:56:51.745 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T01:56:51.745 INFO:teuthology.orchestra.run.smithi019.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T01:56:51.745 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T01:56:51.745 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T01:56:51.745 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T01:56:51.745 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T01:56:51.745 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T01:56:51.745 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T01:56:51.745 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T01:56:51.745 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T01:56:51.745 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T01:56:51.745 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T01:56:51.745 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T01:56:51.746 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T01:56:51.746 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T01:56:51.746 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T01:56:51.746 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T01:56:51.746 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T01:56:51.746 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T01:56:51.746 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T01:56:51.746 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T01:56:51.746 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T01:56:51.746 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T01:56:51.746 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T01:56:51.746 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T01:56:51.746 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T01:56:51.746 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T01:56:51.747 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T01:56:51.747 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T01:56:51.747 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T01:56:51.747 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T01:56:51.747 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T01:56:51.747 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T01:56:51.747 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T01:56:51.747 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T01:56:51.748 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T01:56:51.748 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T01:56:51.748 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T01:56:51.748 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T01:56:51.748 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T01:56:51.748 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T01:56:51.748 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T01:56:51.748 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T01:56:51.748 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T01:56:51.748 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T01:56:51.749 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T01:56:51.749 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T01:56:51.749 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T01:56:51.749 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T01:56:51.749 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T01:56:51.749 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T01:56:51.749 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T01:56:51.749 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T01:56:51.750 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T01:56:51.750 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T01:56:51.750 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T01:56:51.750 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T01:56:51.750 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T01:56:51.750 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T01:56:51.750 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T01:56:51.750 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T01:56:51.750 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T01:56:51.750 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T01:56:51.750 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T01:56:51.750 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T01:56:51.751 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T01:56:51.751 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T01:56:51.751 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T01:56:51.751 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T01:56:51.751 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T01:56:51.751 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T01:56:51.751 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T01:56:51.751 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T01:56:51.751 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T01:56:51.751 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T01:56:51.751 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T01:56:51.751 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T01:56:51.751 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T01:56:51.751 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T01:56:51.752 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T01:56:51.752 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T01:56:51.752 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T01:56:51.752 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T01:56:51.752 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T01:56:51.752 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T01:56:51.752 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T01:56:51.752 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T01:56:51.752 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T01:56:51.752 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T01:56:51.752 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T01:56:51.752 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T01:56:51.752 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T01:56:51.753 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T01:56:51.753 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T01:56:51.753 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T01:56:51.753 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T01:56:51.753 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T01:56:51.753 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T01:56:51.753 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T01:56:51.753 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T01:56:51.753 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T01:56:51.753 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T01:56:51.753 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T01:56:51.753 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T01:56:51.753 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T01:56:51.753 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T01:56:51.753 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T01:56:51.754 INFO:teuthology.orchestra.run.smithi019.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T01:56:51.754 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T01:56:51.754 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T01:56:51.754 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-05-09T01:56:51.754 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T01:56:51.754 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:56:51.754 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-05-09T01:56:51.754 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-05-09T01:56:51.754 INFO:teuthology.orchestra.run.smithi019.stdout:Install 148 Packages 2024-05-09T01:56:51.754 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 8 Packages 2024-05-09T01:56:51.754 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:56:51.755 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 219 M 2024-05-09T01:56:51.755 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-05-09T01:56:52.568 INFO:teuthology.orchestra.run.smithi019.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-05-09T01:56:52.793 INFO:teuthology.orchestra.run.smithi019.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 3.7 MB/s | 863 kB 00:00 2024-05-09T01:56:52.852 INFO:teuthology.orchestra.run.smithi019.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 2.5 MB/s | 147 kB 00:00 2024-05-09T01:56:53.227 INFO:teuthology.orchestra.run.smithi019.stdout:(4/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 6.3 MB/s | 5.4 MB 00:00 2024-05-09T01:56:53.277 INFO:teuthology.orchestra.run.smithi019.stdout:(5/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 5.3 MB/s | 2.3 MB 00:00 2024-05-09T01:56:53.403 INFO:teuthology.orchestra.run.smithi019.stdout:(6/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 8.6 MB/s | 1.5 MB 00:00 2024-05-09T01:56:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 6.7 MB/s | 4.9 MB 00:00 2024-05-09T01:56:55.345 INFO:teuthology.orchestra.run.smithi019.stdout:(8/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x86 9.1 MB/s | 18 MB 00:01 2024-05-09T01:56:55.487 INFO:teuthology.orchestra.run.smithi019.stdout:(9/156): ceph-radosgw-19.0.0-3594.gc9da4049.el9 6.8 MB/s | 10 MB 00:01 2024-05-09T01:56:55.512 INFO:teuthology.orchestra.run.smithi019.stdout:(10/156): ceph-selinux-19.0.0-3594.gc9da4049.el 151 kB/s | 25 kB 00:00 2024-05-09T01:56:55.580 INFO:teuthology.orchestra.run.smithi019.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 3.3 MB/s | 224 kB 00:00 2024-05-09T01:56:55.646 INFO:teuthology.orchestra.run.smithi019.stdout:(12/156): cephfs-shell-19.0.0-3594.gc9da4049.el 407 kB/s | 27 kB 00:00 2024-05-09T01:56:55.721 INFO:teuthology.orchestra.run.smithi019.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 422 kB/s | 32 kB 00:00 2024-05-09T01:56:55.822 INFO:teuthology.orchestra.run.smithi019.stdout:(14/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 7.4 MB/s | 756 kB 00:00 2024-05-09T01:56:55.897 INFO:teuthology.orchestra.run.smithi019.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 2.1 MB/s | 158 kB 00:00 2024-05-09T01:56:55.973 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.2 MB 00:01 2024-05-09T01:56:55.980 INFO:teuthology.orchestra.run.smithi019.stdout:(16/156): librados-devel-19.0.0-3594.gc9da4049. 1.5 MB/s | 127 kB 00:00 2024-05-09T01:56:56.004 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 6.2 MB 00:03 2024-05-09T01:56:56.097 INFO:teuthology.orchestra.run.smithi019.stdout:(17/156): libradosstriper1-19.0.0-3594.gc9da404 4.2 MB/s | 497 kB 00:00 2024-05-09T01:56:56.289 INFO:teuthology.orchestra.run.smithi019.stdout:(18/156): ceph-common-19.0.0-3594.gc9da4049.el9 5.0 MB/s | 20 MB 00:03 2024-05-09T01:56:56.398 INFO:teuthology.orchestra.run.smithi019.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 418 kB/s | 45 kB 00:00 2024-05-09T01:56:56.515 INFO:teuthology.orchestra.run.smithi019.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 1.1 MB/s | 135 kB 00:00 2024-05-09T01:56:56.624 INFO:teuthology.orchestra.run.smithi019.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 1.5 MB/s | 162 kB 00:00 2024-05-09T01:56:56.765 INFO:teuthology.orchestra.run.smithi019.stdout:(22/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 7.7 MB/s | 5.2 MB 00:00 2024-05-09T01:56:56.799 INFO:teuthology.orchestra.run.smithi019.stdout:(23/156): python3-rados-19.0.0-3594.gc9da4049.e 1.8 MB/s | 321 kB 00:00 2024-05-09T01:56:56.824 INFO:teuthology.orchestra.run.smithi019.stdout:(24/156): python3-rbd-19.0.0-3594.gc9da4049.el9 5.0 MB/s | 301 kB 00:00 2024-05-09T01:56:56.866 INFO:teuthology.orchestra.run.smithi019.stdout:(25/156): python3-rgw-19.0.0-3594.gc9da4049.el9 1.4 MB/s | 99 kB 00:00 2024-05-09T01:56:56.892 INFO:teuthology.orchestra.run.smithi019.stdout:(26/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 1.3 MB/s | 86 kB 00:00 2024-05-09T01:56:56.967 INFO:teuthology.orchestra.run.smithi019.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 2.2 MB/s | 169 kB 00:00 2024-05-09T01:56:57.026 INFO:teuthology.orchestra.run.smithi019.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 587 kB/s | 34 kB 00:00 2024-05-09T01:56:57.092 INFO:teuthology.orchestra.run.smithi019.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 2.1 MB/s | 142 kB 00:00 2024-05-09T01:56:57.293 INFO:teuthology.orchestra.run.smithi019.stdout:(30/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 7.1 MB/s | 3.0 MB 00:00 2024-05-09T01:56:57.463 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 16 kB 00:00 2024-05-09T01:56:57.568 INFO:teuthology.orchestra.run.smithi019.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 7.4 MB/s | 3.5 MB 00:00 2024-05-09T01:56:57.635 INFO:teuthology.orchestra.run.smithi019.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 4.0 MB/s | 274 kB 00:00 2024-05-09T01:56:57.657 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-05-09T01:56:57.702 INFO:teuthology.orchestra.run.smithi019.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 740 kB/s | 49 kB 00:00 2024-05-09T01:56:57.761 INFO:teuthology.orchestra.run.smithi019.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3594.gc 282 kB/s | 16 kB 00:00 2024-05-09T01:56:57.836 INFO:teuthology.orchestra.run.smithi019.stdout:(35/156): ceph-volume-19.0.0-3594.gc9da4049.el9 3.6 MB/s | 273 kB 00:00 2024-05-09T01:56:57.978 INFO:teuthology.orchestra.run.smithi019.stdout:(36/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 5.3 MB/s | 763 kB 00:00 2024-05-09T01:56:58.062 INFO:teuthology.orchestra.run.smithi019.stdout:(37/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 219 kB/s | 18 kB 00:00 2024-05-09T01:56:58.345 INFO:teuthology.orchestra.run.smithi019.stdout:(38/156): ceph-mgr-diskprediction-local-19.0.0- 7.0 MB/s | 7.4 MB 00:01 2024-05-09T01:56:58.921 INFO:teuthology.orchestra.run.smithi019.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 94 kB/s | 80 kB 00:00 2024-05-09T01:56:58.994 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-09T01:56:59.346 INFO:teuthology.orchestra.run.smithi019.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 827 kB/s | 803 kB 00:00 2024-05-09T01:56:59.384 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-09T01:56:59.729 INFO:teuthology.orchestra.run.smithi019.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 242 kB/s | 195 kB 00:00 2024-05-09T01:57:00.413 INFO:teuthology.orchestra.run.smithi019.stdout:(42/156): ceph-test-19.0.0-3594.gc9da4049.el9.x 8.8 MB/s | 43 MB 00:04 2024-05-09T01:57:00.438 INFO:teuthology.orchestra.run.smithi019.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 232 kB/s | 253 kB 00:01 2024-05-09T01:57:01.080 INFO:teuthology.orchestra.run.smithi019.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 930 kB/s | 1.2 MB 00:01 2024-05-09T01:57:01.147 INFO:teuthology.orchestra.run.smithi019.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 145 kB/s | 106 kB 00:00 2024-05-09T01:57:01.181 INFO:teuthology.orchestra.run.smithi019.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 182 kB/s | 135 kB 00:00 2024-05-09T01:57:01.822 INFO:teuthology.orchestra.run.smithi019.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 168 kB/s | 125 kB 00:00 2024-05-09T01:57:01.956 INFO:teuthology.orchestra.run.smithi019.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 265 kB/s | 215 kB 00:00 2024-05-09T01:57:02.374 INFO:teuthology.orchestra.run.smithi019.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 257 kB/s | 107 kB 00:00 2024-05-09T01:57:02.631 INFO:teuthology.orchestra.run.smithi019.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 475 kB/s | 689 kB 00:01 2024-05-09T01:57:02.648 INFO:teuthology.orchestra.run.smithi019.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 78 kB/s | 21 kB 00:00 2024-05-09T01:57:02.907 INFO:teuthology.orchestra.run.smithi019.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 127 kB/s | 33 kB 00:00 2024-05-09T01:57:03.282 INFO:teuthology.orchestra.run.smithi019.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 482 kB/s | 313 kB 00:00 2024-05-09T01:57:03.474 INFO:teuthology.orchestra.run.smithi019.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 571 kB/s | 943 kB 00:01 2024-05-09T01:57:03.532 INFO:teuthology.orchestra.run.smithi019.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 73 kB/s | 18 kB 00:00 2024-05-09T01:57:03.658 INFO:teuthology.orchestra.run.smithi019.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 61 kB/s | 11 kB 00:00 2024-05-09T01:57:03.995 INFO:teuthology.orchestra.run.smithi019.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 136 kB/s | 45 kB 00:00 2024-05-09T01:57:04.109 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-09T01:57:04.109 INFO:teuthology.orchestra.run.smithi019.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 278 kB/s | 160 kB 00:00 2024-05-09T01:57:04.657 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-09T01:57:05.417 INFO:teuthology.orchestra.run.smithi019.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 465 kB/s | 662 kB 00:01 2024-05-09T01:57:05.451 INFO:teuthology.orchestra.run.smithi019.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 434 kB/s | 582 kB 00:01 2024-05-09T01:57:05.684 INFO:teuthology.orchestra.run.smithi019.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 95 kB/s | 25 kB 00:00 2024-05-09T01:57:05.793 INFO:teuthology.orchestra.run.smithi019.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 139 kB/s | 47 kB 00:00 2024-05-09T01:57:06.152 INFO:teuthology.orchestra.run.smithi019.stdout:(63/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 943 kB/s | 3.0 MB 00:03 2024-05-09T01:57:06.368 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T01:57:06.368 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T01:57:06.379 INFO:teuthology.orchestra.run.smithi196.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T01:57:06.380 INFO:teuthology.orchestra.run.smithi196.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T01:57:06.381 INFO:teuthology.orchestra.run.smithi196.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T01:57:06.382 INFO:teuthology.orchestra.run.smithi196.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T01:57:06.383 INFO:teuthology.orchestra.run.smithi196.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T01:57:06.385 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T01:57:06.387 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T01:57:06.410 INFO:teuthology.orchestra.run.smithi019.stdout:(64/156): openblas-0.3.21-2.el9.x86_64.rpm 136 kB/s | 35 kB 00:00 2024-05-09T01:57:06.444 INFO:teuthology.orchestra.run.smithi019.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 290 kB/s | 188 kB 00:00 2024-05-09T01:57:06.447 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-09T01:57:06.463 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-05-09T01:57:06.463 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-05-09T01:57:06.463 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-05-09T01:57:06.464 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-09T01:57:06.464 INFO:teuthology.orchestra.run.smithi196.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T01:57:06.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T01:57:06.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T01:57:06.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T01:57:06.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T01:57:06.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T01:57:06.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T01:57:06.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T01:57:06.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T01:57:06.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T01:57:06.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T01:57:06.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T01:57:06.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T01:57:06.465 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 763 k 2024-05-09T01:57:06.465 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T01:57:06.465 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T01:57:06.465 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T01:57:06.465 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T01:57:06.465 INFO:teuthology.orchestra.run.smithi196.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T01:57:06.465 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T01:57:06.465 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T01:57:06.465 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T01:57:06.465 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T01:57:06.465 INFO:teuthology.orchestra.run.smithi196.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T01:57:06.465 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T01:57:06.465 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T01:57:06.465 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T01:57:06.466 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T01:57:06.466 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T01:57:06.466 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 162 k 2024-05-09T01:57:06.466 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 321 k 2024-05-09T01:57:06.466 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 301 k 2024-05-09T01:57:06.466 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T01:57:06.466 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T01:57:06.466 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T01:57:06.466 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T01:57:06.466 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T01:57:06.466 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-05-09T01:57:06.466 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T01:57:06.466 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T01:57:06.467 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T01:57:06.467 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T01:57:06.467 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T01:57:06.467 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T01:57:06.467 INFO:teuthology.orchestra.run.smithi196.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T01:57:06.467 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T01:57:06.467 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-05-09T01:57:06.467 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T01:57:06.467 INFO:teuthology.orchestra.run.smithi196.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T01:57:06.467 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T01:57:06.467 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T01:57:06.467 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T01:57:06.467 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T01:57:06.467 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T01:57:06.468 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T01:57:06.468 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T01:57:06.468 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T01:57:06.468 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T01:57:06.468 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T01:57:06.468 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T01:57:06.468 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T01:57:06.468 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T01:57:06.468 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T01:57:06.468 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T01:57:06.468 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T01:57:06.468 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T01:57:06.468 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T01:57:06.468 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T01:57:06.468 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T01:57:06.469 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T01:57:06.469 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T01:57:06.469 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T01:57:06.469 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T01:57:06.469 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T01:57:06.469 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T01:57:06.469 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T01:57:06.469 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T01:57:06.469 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T01:57:06.469 INFO:teuthology.orchestra.run.smithi196.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T01:57:06.469 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T01:57:06.469 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T01:57:06.470 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T01:57:06.470 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T01:57:06.470 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T01:57:06.470 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T01:57:06.470 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T01:57:06.470 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T01:57:06.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T01:57:06.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T01:57:06.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T01:57:06.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T01:57:06.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T01:57:06.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T01:57:06.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T01:57:06.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T01:57:06.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T01:57:06.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T01:57:06.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T01:57:06.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T01:57:06.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T01:57:06.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T01:57:06.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T01:57:06.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T01:57:06.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T01:57:06.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T01:57:06.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T01:57:06.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T01:57:06.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T01:57:06.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T01:57:06.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T01:57:06.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T01:57:06.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T01:57:06.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T01:57:06.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T01:57:06.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T01:57:06.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T01:57:06.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T01:57:06.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T01:57:06.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T01:57:06.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T01:57:06.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T01:57:06.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T01:57:06.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T01:57:06.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T01:57:06.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T01:57:06.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T01:57:06.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T01:57:06.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T01:57:06.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T01:57:06.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T01:57:06.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T01:57:06.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T01:57:06.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T01:57:06.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T01:57:06.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T01:57:06.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T01:57:06.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T01:57:06.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T01:57:06.474 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T01:57:06.474 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T01:57:06.474 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T01:57:06.474 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T01:57:06.474 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T01:57:06.474 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T01:57:06.474 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T01:57:06.474 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T01:57:06.474 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T01:57:06.474 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T01:57:06.474 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T01:57:06.474 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T01:57:06.474 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T01:57:06.474 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T01:57:06.475 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T01:57:06.475 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T01:57:06.475 INFO:teuthology.orchestra.run.smithi196.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T01:57:06.475 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T01:57:06.475 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T01:57:06.475 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-05-09T01:57:06.475 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T01:57:06.475 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:57:06.475 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-09T01:57:06.475 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-05-09T01:57:06.475 INFO:teuthology.orchestra.run.smithi196.stdout:Install 148 Packages 2024-05-09T01:57:06.476 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 8 Packages 2024-05-09T01:57:06.476 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:57:06.476 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 219 M 2024-05-09T01:57:06.477 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-09T01:57:06.603 INFO:teuthology.orchestra.run.smithi019.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 318 kB/s | 292 kB 00:00 2024-05-09T01:57:07.018 INFO:teuthology.orchestra.run.smithi059.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T01:57:07.019 INFO:teuthology.orchestra.run.smithi059.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T01:57:07.030 INFO:teuthology.orchestra.run.smithi059.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T01:57:07.031 INFO:teuthology.orchestra.run.smithi059.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T01:57:07.031 INFO:teuthology.orchestra.run.smithi059.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T01:57:07.032 INFO:teuthology.orchestra.run.smithi059.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T01:57:07.034 INFO:teuthology.orchestra.run.smithi059.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T01:57:07.036 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T01:57:07.037 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T01:57:07.101 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-09T01:57:07.120 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-05-09T01:57:07.120 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-05-09T01:57:07.120 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-05-09T01:57:07.120 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-09T01:57:07.120 INFO:teuthology.orchestra.run.smithi059.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T01:57:07.120 INFO:teuthology.orchestra.run.smithi059.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T01:57:07.121 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T01:57:07.121 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T01:57:07.121 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T01:57:07.121 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T01:57:07.121 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T01:57:07.121 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T01:57:07.121 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T01:57:07.121 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T01:57:07.121 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T01:57:07.121 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T01:57:07.121 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T01:57:07.121 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 763 k 2024-05-09T01:57:07.121 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T01:57:07.121 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T01:57:07.122 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T01:57:07.122 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T01:57:07.122 INFO:teuthology.orchestra.run.smithi059.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T01:57:07.122 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T01:57:07.122 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T01:57:07.122 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T01:57:07.122 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T01:57:07.122 INFO:teuthology.orchestra.run.smithi059.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T01:57:07.122 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T01:57:07.122 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T01:57:07.122 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T01:57:07.122 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T01:57:07.122 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T01:57:07.122 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 162 k 2024-05-09T01:57:07.123 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 321 k 2024-05-09T01:57:07.123 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 301 k 2024-05-09T01:57:07.123 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T01:57:07.123 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T01:57:07.123 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T01:57:07.123 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T01:57:07.123 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T01:57:07.123 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrading: 2024-05-09T01:57:07.123 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T01:57:07.123 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T01:57:07.123 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T01:57:07.123 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T01:57:07.123 INFO:teuthology.orchestra.run.smithi059.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T01:57:07.123 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T01:57:07.124 INFO:teuthology.orchestra.run.smithi059.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T01:57:07.124 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T01:57:07.124 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-05-09T01:57:07.124 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T01:57:07.124 INFO:teuthology.orchestra.run.smithi059.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T01:57:07.124 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T01:57:07.124 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T01:57:07.124 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T01:57:07.124 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T01:57:07.124 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T01:57:07.124 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T01:57:07.124 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T01:57:07.124 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T01:57:07.124 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T01:57:07.125 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T01:57:07.125 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T01:57:07.125 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T01:57:07.125 INFO:teuthology.orchestra.run.smithi059.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T01:57:07.125 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T01:57:07.125 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T01:57:07.125 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T01:57:07.125 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T01:57:07.125 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T01:57:07.125 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T01:57:07.125 INFO:teuthology.orchestra.run.smithi059.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T01:57:07.125 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T01:57:07.125 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T01:57:07.125 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T01:57:07.125 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T01:57:07.126 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T01:57:07.126 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T01:57:07.126 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T01:57:07.126 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T01:57:07.126 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T01:57:07.126 INFO:teuthology.orchestra.run.smithi059.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T01:57:07.126 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T01:57:07.126 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T01:57:07.126 INFO:teuthology.orchestra.run.smithi059.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T01:57:07.126 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T01:57:07.126 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T01:57:07.126 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T01:57:07.126 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T01:57:07.126 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T01:57:07.127 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T01:57:07.127 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T01:57:07.127 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T01:57:07.127 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T01:57:07.127 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T01:57:07.127 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T01:57:07.127 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T01:57:07.127 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T01:57:07.127 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T01:57:07.127 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T01:57:07.127 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T01:57:07.127 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T01:57:07.127 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T01:57:07.127 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T01:57:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T01:57:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T01:57:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T01:57:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T01:57:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T01:57:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T01:57:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T01:57:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T01:57:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T01:57:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T01:57:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T01:57:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T01:57:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T01:57:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T01:57:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T01:57:07.129 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T01:57:07.129 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T01:57:07.129 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T01:57:07.129 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T01:57:07.129 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T01:57:07.129 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T01:57:07.129 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T01:57:07.129 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T01:57:07.129 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T01:57:07.129 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T01:57:07.129 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T01:57:07.129 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T01:57:07.129 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T01:57:07.130 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T01:57:07.130 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T01:57:07.130 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T01:57:07.130 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T01:57:07.130 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T01:57:07.130 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T01:57:07.130 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T01:57:07.130 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T01:57:07.130 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T01:57:07.130 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T01:57:07.130 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T01:57:07.130 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T01:57:07.130 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T01:57:07.130 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T01:57:07.131 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T01:57:07.131 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T01:57:07.131 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T01:57:07.131 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T01:57:07.131 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T01:57:07.131 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T01:57:07.131 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T01:57:07.131 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T01:57:07.131 INFO:teuthology.orchestra.run.smithi059.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T01:57:07.131 INFO:teuthology.orchestra.run.smithi059.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T01:57:07.131 INFO:teuthology.orchestra.run.smithi059.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T01:57:07.131 INFO:teuthology.orchestra.run.smithi059.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T01:57:07.131 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T01:57:07.131 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T01:57:07.131 INFO:teuthology.orchestra.run.smithi059.stdout:Installing weak dependencies: 2024-05-09T01:57:07.132 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T01:57:07.132 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:57:07.132 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-09T01:57:07.132 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-05-09T01:57:07.132 INFO:teuthology.orchestra.run.smithi059.stdout:Install 148 Packages 2024-05-09T01:57:07.132 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade 8 Packages 2024-05-09T01:57:07.132 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:57:07.132 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 219 M 2024-05-09T01:57:07.133 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-09T01:57:07.360 INFO:teuthology.orchestra.run.smithi196.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-05-09T01:57:08.302 INFO:teuthology.orchestra.run.smithi196.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 916 kB/s | 863 kB 00:00 2024-05-09T01:57:08.425 INFO:teuthology.orchestra.run.smithi059.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-05-09T01:57:08.502 INFO:teuthology.orchestra.run.smithi196.stdout:(3/156): ceph-common-19.0.0-3594.gc9da4049.el9. 14 MB/s | 20 MB 00:01 2024-05-09T01:57:08.527 INFO:teuthology.orchestra.run.smithi196.stdout:(4/156): ceph-immutable-object-cache-19.0.0-359 651 kB/s | 147 kB 00:00 2024-05-09T01:57:08.827 INFO:teuthology.orchestra.run.smithi196.stdout:(5/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 7.0 MB/s | 2.3 MB 00:00 2024-05-09T01:57:08.987 INFO:teuthology.orchestra.run.smithi019.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 682 kB/s | 1.6 MB 00:02 2024-05-09T01:57:09.101 INFO:teuthology.orchestra.run.smithi059.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 1.2 MB/s | 863 kB 00:00 2024-05-09T01:57:09.292 INFO:teuthology.orchestra.run.smithi059.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 767 kB/s | 147 kB 00:00 2024-05-09T01:57:09.294 INFO:teuthology.orchestra.run.smithi019.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.6 MB/s | 4.4 MB 00:02 2024-05-09T01:57:09.453 INFO:teuthology.orchestra.run.smithi196.stdout:(6/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 7.8 MB/s | 4.9 MB 00:00 2024-05-09T01:57:09.495 INFO:teuthology.orchestra.run.smithi196.stdout:(7/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 1.6 MB/s | 1.5 MB 00:00 2024-05-09T01:57:09.570 INFO:teuthology.orchestra.run.smithi196.stdout:(8/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-05-09T01:57:09.679 INFO:teuthology.orchestra.run.smithi196.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 232 kB/s | 25 kB 00:00 2024-05-09T01:57:09.861 INFO:teuthology.orchestra.run.smithi019.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 235 kB/s | 133 kB 00:00 2024-05-09T01:57:10.354 INFO:teuthology.orchestra.run.smithi019.stdout:(70/156): openblas-openmp-0.3.21-2.el9.x86_64.r 1.2 MB/s | 4.7 MB 00:03 2024-05-09T01:57:10.434 INFO:teuthology.orchestra.run.smithi059.stdout:(4/156): ceph-common-19.0.0-3594.gc9da4049.el9. 8.5 MB/s | 20 MB 00:02 2024-05-09T01:57:10.568 INFO:teuthology.orchestra.run.smithi059.stdout:(5/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 1.8 MB/s | 2.3 MB 00:01 2024-05-09T01:57:10.685 INFO:teuthology.orchestra.run.smithi059.stdout:(6/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 6.0 MB/s | 1.5 MB 00:00 2024-05-09T01:57:10.785 INFO:teuthology.orchestra.run.smithi059.stdout:(7/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-05-09T01:57:10.870 INFO:teuthology.orchestra.run.smithi019.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 484 kB/s | 911 kB 00:01 2024-05-09T01:57:11.287 INFO:teuthology.orchestra.run.smithi019.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 210 kB/s | 87 kB 00:00 2024-05-09T01:57:11.845 INFO:teuthology.orchestra.run.smithi196.stdout:(10/156): ceph-radosgw-19.0.0-3594.gc9da4049.el 4.3 MB/s | 10 MB 00:02 2024-05-09T01:57:12.004 INFO:teuthology.orchestra.run.smithi019.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 626 kB/s | 1.0 MB 00:01 2024-05-09T01:57:12.087 INFO:teuthology.orchestra.run.smithi196.stdout:(11/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 6.7 MB/s | 18 MB 00:02 2024-05-09T01:57:12.112 INFO:teuthology.orchestra.run.smithi196.stdout:(12/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 839 kB/s | 224 kB 00:00 2024-05-09T01:57:12.179 INFO:teuthology.orchestra.run.smithi196.stdout:(13/156): cephfs-shell-19.0.0-3594.gc9da4049.el 294 kB/s | 27 kB 00:00 2024-05-09T01:57:12.221 INFO:teuthology.orchestra.run.smithi196.stdout:(14/156): libcephfs-devel-19.0.0-3594.gc9da4049 292 kB/s | 32 kB 00:00 2024-05-09T01:57:12.330 INFO:teuthology.orchestra.run.smithi196.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 1.4 MB/s | 158 kB 00:00 2024-05-09T01:57:12.430 INFO:teuthology.orchestra.run.smithi196.stdout:(16/156): librados-devel-19.0.0-3594.gc9da4049. 1.2 MB/s | 127 kB 00:00 2024-05-09T01:57:12.489 INFO:teuthology.orchestra.run.smithi196.stdout:(17/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 2.4 MB/s | 756 kB 00:00 2024-05-09T01:57:12.572 INFO:teuthology.orchestra.run.smithi196.stdout:(18/156): libradosstriper1-19.0.0-3594.gc9da404 3.4 MB/s | 497 kB 00:00 2024-05-09T01:57:12.656 INFO:teuthology.orchestra.run.smithi196.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 542 kB/s | 45 kB 00:00 2024-05-09T01:57:12.662 INFO:teuthology.orchestra.run.smithi019.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 371 kB/s | 244 kB 00:00 2024-05-09T01:57:12.739 INFO:teuthology.orchestra.run.smithi196.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 1.6 MB/s | 135 kB 00:00 2024-05-09T01:57:12.823 INFO:teuthology.orchestra.run.smithi196.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 1.9 MB/s | 162 kB 00:00 2024-05-09T01:57:12.915 INFO:teuthology.orchestra.run.smithi196.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 3.4 MB/s | 321 kB 00:00 2024-05-09T01:57:12.944 INFO:teuthology.orchestra.run.smithi059.stdout:(8/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 2.1 MB/s | 4.9 MB 00:02 2024-05-09T01:57:13.007 INFO:teuthology.orchestra.run.smithi196.stdout:(23/156): python3-rbd-19.0.0-3594.gc9da4049.el9 3.2 MB/s | 301 kB 00:00 2024-05-09T01:57:13.035 INFO:teuthology.orchestra.run.smithi059.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 275 kB/s | 25 kB 00:00 2024-05-09T01:57:13.091 INFO:teuthology.orchestra.run.smithi196.stdout:(24/156): python3-rgw-19.0.0-3594.gc9da4049.el9 1.2 MB/s | 99 kB 00:00 2024-05-09T01:57:13.158 INFO:teuthology.orchestra.run.smithi196.stdout:(25/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 1.3 MB/s | 86 kB 00:00 2024-05-09T01:57:13.161 INFO:teuthology.orchestra.run.smithi059.stdout:(10/156): ceph-radosgw-19.0.0-3594.gc9da4049.el 4.3 MB/s | 10 MB 00:02 2024-05-09T01:57:13.252 INFO:teuthology.orchestra.run.smithi059.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 2.4 MB/s | 224 kB 00:00 2024-05-09T01:57:13.336 INFO:teuthology.orchestra.run.smithi059.stdout:(12/156): cephfs-shell-19.0.0-3594.gc9da4049.el 324 kB/s | 27 kB 00:00 2024-05-09T01:57:13.411 INFO:teuthology.orchestra.run.smithi059.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 425 kB/s | 32 kB 00:00 2024-05-09T01:57:13.425 INFO:teuthology.orchestra.run.smithi196.stdout:(26/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 5.5 MB/s | 5.2 MB 00:00 2024-05-09T01:57:13.479 INFO:teuthology.orchestra.run.smithi019.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 1.6 MB/s | 5.8 MB 00:03 2024-05-09T01:57:13.500 INFO:teuthology.orchestra.run.smithi196.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 2.2 MB/s | 169 kB 00:00 2024-05-09T01:57:13.575 INFO:teuthology.orchestra.run.smithi196.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 458 kB/s | 34 kB 00:00 2024-05-09T01:57:13.578 INFO:teuthology.orchestra.run.smithi059.stdout:(14/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 4.4 MB/s | 756 kB 00:00 2024-05-09T01:57:13.621 INFO:teuthology.orchestra.run.smithi019.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 259 kB/s | 248 kB 00:00 2024-05-09T01:57:13.625 INFO:teuthology.orchestra.run.smithi196.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 2.8 MB/s | 142 kB 00:00 2024-05-09T01:57:13.684 INFO:teuthology.orchestra.run.smithi196.stdout:(30/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 5.8 MB/s | 3.0 MB 00:00 2024-05-09T01:57:13.762 INFO:teuthology.orchestra.run.smithi059.stdout:(15/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 5.8 MB/s | 18 MB 00:03 2024-05-09T01:57:13.799 INFO:teuthology.orchestra.run.smithi059.stdout:(16/156): libcephsqlite-19.0.0-3594.gc9da4049.e 727 kB/s | 158 kB 00:00 2024-05-09T01:57:13.805 INFO:teuthology.orchestra.run.smithi019.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 79 kB/s | 26 kB 00:00 2024-05-09T01:57:13.829 INFO:teuthology.orchestra.run.smithi059.stdout:(17/156): librados-devel-19.0.0-3594.gc9da4049. 1.9 MB/s | 127 kB 00:00 2024-05-09T01:57:13.955 INFO:teuthology.orchestra.run.smithi059.stdout:(18/156): libradosstriper1-19.0.0-3594.gc9da404 3.1 MB/s | 497 kB 00:00 2024-05-09T01:57:14.030 INFO:teuthology.orchestra.run.smithi019.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 47 kB/s | 19 kB 00:00 2024-05-09T01:57:14.047 INFO:teuthology.orchestra.run.smithi059.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 492 kB/s | 45 kB 00:00 2024-05-09T01:57:14.243 INFO:teuthology.orchestra.run.smithi196.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 5.7 MB/s | 3.5 MB 00:00 2024-05-09T01:57:14.305 INFO:teuthology.orchestra.run.smithi019.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 126 kB/s | 35 kB 00:00 2024-05-09T01:57:14.326 INFO:teuthology.orchestra.run.smithi196.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 3.2 MB/s | 274 kB 00:00 2024-05-09T01:57:14.393 INFO:teuthology.orchestra.run.smithi196.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 740 kB/s | 49 kB 00:00 2024-05-09T01:57:14.397 INFO:teuthology.orchestra.run.smithi059.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 387 kB/s | 135 kB 00:00 2024-05-09T01:57:14.452 INFO:teuthology.orchestra.run.smithi196.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3594.gc 283 kB/s | 16 kB 00:00 2024-05-09T01:57:14.489 INFO:teuthology.orchestra.run.smithi059.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 1.7 MB/s | 162 kB 00:00 2024-05-09T01:57:14.514 INFO:teuthology.orchestra.run.smithi019.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 243 kB/s | 172 kB 00:00 2024-05-09T01:57:14.577 INFO:teuthology.orchestra.run.smithi196.stdout:(35/156): ceph-volume-19.0.0-3594.gc9da4049.el9 2.1 MB/s | 273 kB 00:00 2024-05-09T01:57:14.614 INFO:teuthology.orchestra.run.smithi059.stdout:(22/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 6.6 MB/s | 5.2 MB 00:00 2024-05-09T01:57:14.640 INFO:teuthology.orchestra.run.smithi059.stdout:(23/156): python3-rados-19.0.0-3594.gc9da4049.e 2.1 MB/s | 321 kB 00:00 2024-05-09T01:57:14.681 INFO:teuthology.orchestra.run.smithi059.stdout:(24/156): python3-rbd-19.0.0-3594.gc9da4049.el9 4.4 MB/s | 301 kB 00:00 2024-05-09T01:57:14.715 INFO:teuthology.orchestra.run.smithi059.stdout:(25/156): python3-rgw-19.0.0-3594.gc9da4049.el9 1.3 MB/s | 99 kB 00:00 2024-05-09T01:57:14.727 INFO:teuthology.orchestra.run.smithi196.stdout:(36/156): ceph-mgr-diskprediction-local-19.0.0- 7.1 MB/s | 7.4 MB 00:01 2024-05-09T01:57:14.740 INFO:teuthology.orchestra.run.smithi059.stdout:(26/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 1.4 MB/s | 86 kB 00:00 2024-05-09T01:57:14.761 INFO:teuthology.orchestra.run.smithi196.stdout:(37/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 4.1 MB/s | 763 kB 00:00 2024-05-09T01:57:14.791 INFO:teuthology.orchestra.run.smithi059.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 3.3 MB/s | 169 kB 00:00 2024-05-09T01:57:14.795 INFO:teuthology.orchestra.run.smithi196.stdout:(38/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 273 kB/s | 18 kB 00:00 2024-05-09T01:57:14.833 INFO:teuthology.orchestra.run.smithi059.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 821 kB/s | 34 kB 00:00 2024-05-09T01:57:14.891 INFO:teuthology.orchestra.run.smithi059.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 2.4 MB/s | 142 kB 00:00 2024-05-09T01:57:15.006 INFO:teuthology.orchestra.run.smithi019.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 510 kB/s | 250 kB 00:00 2024-05-09T01:57:15.067 INFO:teuthology.orchestra.run.smithi059.stdout:(30/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 8.7 MB/s | 3.0 MB 00:00 2024-05-09T01:57:15.187 INFO:teuthology.orchestra.run.smithi196.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 189 kB/s | 80 kB 00:00 2024-05-09T01:57:15.306 INFO:teuthology.orchestra.run.smithi019.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 1.5 MB/s | 6.0 MB 00:04 2024-05-09T01:57:15.370 INFO:teuthology.orchestra.run.smithi196.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-05-09T01:57:15.562 INFO:teuthology.orchestra.run.smithi196.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-05-09T01:57:15.587 INFO:teuthology.orchestra.run.smithi196.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-05-09T01:57:15.600 INFO:teuthology.orchestra.run.smithi059.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 5.0 MB/s | 3.5 MB 00:00 2024-05-09T01:57:15.659 INFO:teuthology.orchestra.run.smithi059.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 4.6 MB/s | 274 kB 00:00 2024-05-09T01:57:15.701 INFO:teuthology.orchestra.run.smithi059.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 1.2 MB/s | 49 kB 00:00 2024-05-09T01:57:15.729 INFO:teuthology.orchestra.run.smithi196.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 751 kB/s | 106 kB 00:00 2024-05-09T01:57:15.731 INFO:teuthology.orchestra.run.smithi019.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 306 kB/s | 222 kB 00:00 2024-05-09T01:57:15.734 INFO:teuthology.orchestra.run.smithi059.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3594.gc 496 kB/s | 16 kB 00:00 2024-05-09T01:57:15.757 INFO:teuthology.orchestra.run.smithi019.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 171 kB/s | 77 kB 00:00 2024-05-09T01:57:15.784 INFO:teuthology.orchestra.run.smithi059.stdout:(35/156): ceph-volume-19.0.0-3594.gc9da4049.el9 5.4 MB/s | 273 kB 00:00 2024-05-09T01:57:15.838 INFO:teuthology.orchestra.run.smithi196.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-05-09T01:57:15.910 INFO:teuthology.orchestra.run.smithi059.stdout:(36/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 6.0 MB/s | 763 kB 00:00 2024-05-09T01:57:15.930 INFO:teuthology.orchestra.run.smithi196.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 3.3 MB/s | 1.2 MB 00:00 2024-05-09T01:57:15.955 INFO:teuthology.orchestra.run.smithi196.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-05-09T01:57:15.994 INFO:teuthology.orchestra.run.smithi059.stdout:(37/156): ceph-mgr-diskprediction-local-19.0.0- 8.0 MB/s | 7.4 MB 00:00 2024-05-09T01:57:16.010 INFO:teuthology.orchestra.run.smithi059.stdout:(38/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 182 kB/s | 18 kB 00:00 2024-05-09T01:57:16.039 INFO:teuthology.orchestra.run.smithi196.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-09T01:57:16.098 INFO:teuthology.orchestra.run.smithi019.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 124 kB/s | 42 kB 00:00 2024-05-09T01:57:16.189 INFO:teuthology.orchestra.run.smithi196.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 689 kB 00:00 2024-05-09T01:57:16.239 INFO:teuthology.orchestra.run.smithi196.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-05-09T01:57:16.273 INFO:teuthology.orchestra.run.smithi196.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-05-09T01:57:16.298 INFO:teuthology.orchestra.run.smithi196.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 367 kB/s | 21 kB 00:00 2024-05-09T01:57:16.340 INFO:teuthology.orchestra.run.smithi196.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 786 kB/s | 33 kB 00:00 2024-05-09T01:57:16.357 INFO:teuthology.orchestra.run.smithi019.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.5 MB/s | 5.1 MB 00:02 2024-05-09T01:57:16.365 INFO:teuthology.orchestra.run.smithi196.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-05-09T01:57:16.382 INFO:teuthology.orchestra.run.smithi019.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 431 kB/s | 269 kB 00:00 2024-05-09T01:57:16.408 INFO:teuthology.orchestra.run.smithi196.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-05-09T01:57:16.427 INFO:teuthology.orchestra.run.smithi059.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 185 kB/s | 80 kB 00:00 2024-05-09T01:57:16.449 INFO:teuthology.orchestra.run.smithi196.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-05-09T01:57:16.566 INFO:teuthology.orchestra.run.smithi196.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-05-09T01:57:16.592 INFO:teuthology.orchestra.run.smithi196.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-05-09T01:57:16.617 INFO:teuthology.orchestra.run.smithi196.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-05-09T01:57:16.651 INFO:teuthology.orchestra.run.smithi196.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-09T01:57:16.676 INFO:teuthology.orchestra.run.smithi196.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-05-09T01:57:16.692 INFO:teuthology.orchestra.run.smithi019.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 268 kB/s | 159 kB 00:00 2024-05-09T01:57:16.697 INFO:teuthology.orchestra.run.smithi059.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 732 kB/s | 195 kB 00:00 2024-05-09T01:57:16.701 INFO:teuthology.orchestra.run.smithi196.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 506 kB/s | 25 kB 00:00 2024-05-09T01:57:16.727 INFO:teuthology.orchestra.run.smithi196.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-05-09T01:57:16.727 INFO:teuthology.orchestra.run.smithi019.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 149 kB/s | 51 kB 00:00 2024-05-09T01:57:16.752 INFO:teuthology.orchestra.run.smithi196.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-09T01:57:16.755 INFO:teuthology.orchestra.run.smithi059.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-05-09T01:57:16.777 INFO:teuthology.orchestra.run.smithi196.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-09T01:57:16.803 INFO:teuthology.orchestra.run.smithi196.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-05-09T01:57:16.878 INFO:teuthology.orchestra.run.smithi059.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-05-09T01:57:16.912 INFO:teuthology.orchestra.run.smithi196.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 35 MB/s | 4.7 MB 00:00 2024-05-09T01:57:16.954 INFO:teuthology.orchestra.run.smithi059.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-05-09T01:57:16.979 INFO:teuthology.orchestra.run.smithi059.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-05-09T01:57:16.980 INFO:teuthology.orchestra.run.smithi196.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-05-09T01:57:17.020 INFO:teuthology.orchestra.run.smithi196.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-09T01:57:17.042 INFO:teuthology.orchestra.run.smithi019.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 153 kB/s | 54 kB 00:00 2024-05-09T01:57:17.053 INFO:teuthology.orchestra.run.smithi196.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-05-09T01:57:17.063 INFO:teuthology.orchestra.run.smithi059.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-05-09T01:57:17.071 INFO:teuthology.orchestra.run.smithi196.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-05-09T01:57:17.088 INFO:teuthology.orchestra.run.smithi059.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-09T01:57:17.129 INFO:teuthology.orchestra.run.smithi196.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-05-09T01:57:17.171 INFO:teuthology.orchestra.run.smithi196.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-05-09T01:57:17.172 INFO:teuthology.orchestra.run.smithi059.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-09T01:57:17.176 INFO:teuthology.orchestra.run.smithi019.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 341 kB/s | 279 kB 00:00 2024-05-09T01:57:17.246 INFO:teuthology.orchestra.run.smithi196.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.8 MB 00:00 2024-05-09T01:57:17.272 INFO:teuthology.orchestra.run.smithi059.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-05-09T01:57:17.296 INFO:teuthology.orchestra.run.smithi196.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-09T01:57:17.314 INFO:teuthology.orchestra.run.smithi059.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-05-09T01:57:17.317 INFO:teuthology.orchestra.run.smithi019.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 151 kB/s | 42 kB 00:00 2024-05-09T01:57:17.364 INFO:teuthology.orchestra.run.smithi059.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-05-09T01:57:17.372 INFO:teuthology.orchestra.run.smithi196.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-05-09T01:57:17.381 INFO:teuthology.orchestra.run.smithi059.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-05-09T01:57:17.400 INFO:teuthology.orchestra.run.smithi196.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-05-09T01:57:17.414 INFO:teuthology.orchestra.run.smithi196.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 613 kB/s | 26 kB 00:00 2024-05-09T01:57:17.414 INFO:teuthology.orchestra.run.smithi059.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 982 kB/s | 33 kB 00:00 2024-05-09T01:57:17.431 INFO:teuthology.orchestra.run.smithi196.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 572 kB/s | 19 kB 00:00 2024-05-09T01:57:17.441 INFO:teuthology.orchestra.run.smithi059.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-05-09T01:57:17.465 INFO:teuthology.orchestra.run.smithi196.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-05-09T01:57:17.474 INFO:teuthology.orchestra.run.smithi059.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 547 kB/s | 18 kB 00:00 2024-05-09T01:57:17.483 INFO:teuthology.orchestra.run.smithi196.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-05-09T01:57:17.526 INFO:teuthology.orchestra.run.smithi019.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 126 kB/s | 44 kB 00:00 2024-05-09T01:57:17.532 INFO:teuthology.orchestra.run.smithi196.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-09T01:57:17.532 INFO:teuthology.orchestra.run.smithi059.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-05-09T01:57:17.549 INFO:teuthology.orchestra.run.smithi059.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-05-09T01:57:17.575 INFO:teuthology.orchestra.run.smithi059.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-05-09T01:57:17.591 INFO:teuthology.orchestra.run.smithi059.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-05-09T01:57:17.607 INFO:teuthology.orchestra.run.smithi196.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-05-09T01:57:17.633 INFO:teuthology.orchestra.run.smithi059.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-09T01:57:17.657 INFO:teuthology.orchestra.run.smithi196.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-09T01:57:17.667 INFO:teuthology.orchestra.run.smithi059.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 7.5 MB/s | 582 kB 00:00 2024-05-09T01:57:17.699 INFO:teuthology.orchestra.run.smithi196.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-05-09T01:57:17.700 INFO:teuthology.orchestra.run.smithi059.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-05-09T01:57:17.751 INFO:teuthology.orchestra.run.smithi059.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-09T01:57:17.766 INFO:teuthology.orchestra.run.smithi196.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-05-09T01:57:17.792 INFO:teuthology.orchestra.run.smithi196.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-05-09T01:57:17.793 INFO:teuthology.orchestra.run.smithi059.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-05-09T01:57:17.803 INFO:teuthology.orchestra.run.smithi019.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 130 kB/s | 36 kB 00:00 2024-05-09T01:57:17.817 INFO:teuthology.orchestra.run.smithi196.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-09T01:57:17.827 INFO:teuthology.orchestra.run.smithi059.stdout:(64/156): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 35 kB 00:00 2024-05-09T01:57:17.842 INFO:teuthology.orchestra.run.smithi196.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-09T01:57:17.843 INFO:teuthology.orchestra.run.smithi059.stdout:(65/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 121 kB/s | 25 kB 00:00 2024-05-09T01:57:17.868 INFO:teuthology.orchestra.run.smithi196.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-09T01:57:17.893 INFO:teuthology.orchestra.run.smithi196.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-09T01:57:17.935 INFO:teuthology.orchestra.run.smithi196.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-09T01:57:17.952 INFO:teuthology.orchestra.run.smithi059.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-05-09T01:57:17.977 INFO:teuthology.orchestra.run.smithi196.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-05-09T01:57:17.993 INFO:teuthology.orchestra.run.smithi019.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 327 kB/s | 221 kB 00:00 2024-05-09T01:57:18.011 INFO:teuthology.orchestra.run.smithi059.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 26 MB/s | 4.4 MB 00:00 2024-05-09T01:57:18.019 INFO:teuthology.orchestra.run.smithi196.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-05-09T01:57:18.061 INFO:teuthology.orchestra.run.smithi059.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-09T01:57:18.069 INFO:teuthology.orchestra.run.smithi196.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 722 kB/s | 36 kB 00:00 2024-05-09T01:57:18.103 INFO:teuthology.orchestra.run.smithi059.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 9.7 MB/s | 911 kB 00:00 2024-05-09T01:57:18.111 INFO:teuthology.orchestra.run.smithi196.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-05-09T01:57:18.128 INFO:teuthology.orchestra.run.smithi059.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-09T01:57:18.152 INFO:teuthology.orchestra.run.smithi019.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 135 kB/s | 47 kB 00:00 2024-05-09T01:57:18.161 INFO:teuthology.orchestra.run.smithi196.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-05-09T01:57:18.187 INFO:teuthology.orchestra.run.smithi059.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-05-09T01:57:18.203 INFO:teuthology.orchestra.run.smithi196.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-05-09T01:57:18.229 INFO:teuthology.orchestra.run.smithi059.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-05-09T01:57:18.253 INFO:teuthology.orchestra.run.smithi196.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-09T01:57:18.304 INFO:teuthology.orchestra.run.smithi196.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-09T01:57:18.304 INFO:teuthology.orchestra.run.smithi059.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-05-09T01:57:18.346 INFO:teuthology.orchestra.run.smithi196.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-05-09T01:57:18.346 INFO:teuthology.orchestra.run.smithi059.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-05-09T01:57:18.429 INFO:teuthology.orchestra.run.smithi196.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-05-09T01:57:18.446 INFO:teuthology.orchestra.run.smithi059.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2024-05-09T01:57:18.472 INFO:teuthology.orchestra.run.smithi059.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-05-09T01:57:18.495 INFO:teuthology.orchestra.run.smithi019.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 169 kB/s | 58 kB 00:00 2024-05-09T01:57:18.497 INFO:teuthology.orchestra.run.smithi059.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-05-09T01:57:18.522 INFO:teuthology.orchestra.run.smithi059.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-05-09T01:57:18.548 INFO:teuthology.orchestra.run.smithi059.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-09T01:57:18.573 INFO:teuthology.orchestra.run.smithi059.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-05-09T01:57:18.623 INFO:teuthology.orchestra.run.smithi059.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-09T01:57:18.646 INFO:teuthology.orchestra.run.smithi196.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-05-09T01:57:18.700 INFO:teuthology.orchestra.run.smithi059.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-05-09T01:57:18.724 INFO:teuthology.orchestra.run.smithi059.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-09T01:57:18.749 INFO:teuthology.orchestra.run.smithi059.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-09T01:57:18.755 INFO:teuthology.orchestra.run.smithi196.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-05-09T01:57:18.766 INFO:teuthology.orchestra.run.smithi059.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-05-09T01:57:18.791 INFO:teuthology.orchestra.run.smithi059.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-05-09T01:57:18.817 INFO:teuthology.orchestra.run.smithi059.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-09T01:57:18.818 INFO:teuthology.orchestra.run.smithi019.stdout:(98/156): socat-1.7.4.1-5.el9.x86_64.rpm 369 kB/s | 305 kB 00:00 2024-05-09T01:57:18.842 INFO:teuthology.orchestra.run.smithi059.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-09T01:57:18.859 INFO:teuthology.orchestra.run.smithi059.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-05-09T01:57:18.876 INFO:teuthology.orchestra.run.smithi059.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-05-09T01:57:18.909 INFO:teuthology.orchestra.run.smithi059.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-05-09T01:57:18.919 INFO:teuthology.orchestra.run.smithi019.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 220 kB/s | 94 kB 00:00 2024-05-09T01:57:18.943 INFO:teuthology.orchestra.run.smithi059.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-05-09T01:57:18.985 INFO:teuthology.orchestra.run.smithi059.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-05-09T01:57:19.019 INFO:teuthology.orchestra.run.smithi059.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-05-09T01:57:19.097 INFO:teuthology.orchestra.run.smithi196.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-05-09T01:57:19.156 INFO:teuthology.orchestra.run.smithi196.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 772 kB/s | 309 kB 00:00 2024-05-09T01:57:19.178 INFO:teuthology.orchestra.run.smithi019.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 178 kB/s | 64 kB 00:00 2024-05-09T01:57:19.180 INFO:teuthology.orchestra.run.smithi196.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-05-09T01:57:19.210 INFO:teuthology.orchestra.run.smithi059.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-05-09T01:57:19.228 INFO:teuthology.orchestra.run.smithi059.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 227 kB/s | 47 kB 00:00 2024-05-09T01:57:19.248 INFO:teuthology.orchestra.run.smithi196.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-05-09T01:57:19.261 INFO:teuthology.orchestra.run.smithi019.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-05-09T01:57:19.261 INFO:teuthology.orchestra.run.smithi059.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-09T01:57:19.278 INFO:teuthology.orchestra.run.smithi059.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-09T01:57:19.286 INFO:teuthology.orchestra.run.smithi019.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 122 kB/s | 45 kB 00:00 2024-05-09T01:57:19.303 INFO:teuthology.orchestra.run.smithi059.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-05-09T01:57:19.320 INFO:teuthology.orchestra.run.smithi059.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-05-09T01:57:19.323 INFO:teuthology.orchestra.run.smithi196.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 649 kB/s | 49 kB 00:00 2024-05-09T01:57:19.346 INFO:teuthology.orchestra.run.smithi059.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-05-09T01:57:19.371 INFO:teuthology.orchestra.run.smithi059.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 442 kB/s | 22 kB 00:00 2024-05-09T01:57:19.398 INFO:teuthology.orchestra.run.smithi196.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 898 kB/s | 67 kB 00:00 2024-05-09T01:57:19.413 INFO:teuthology.orchestra.run.smithi059.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-05-09T01:57:19.465 INFO:teuthology.orchestra.run.smithi196.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-09T01:57:19.490 INFO:teuthology.orchestra.run.smithi196.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-05-09T01:57:19.557 INFO:teuthology.orchestra.run.smithi196.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-05-09T01:57:19.579 INFO:teuthology.orchestra.run.smithi019.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-05-09T01:57:19.599 INFO:teuthology.orchestra.run.smithi196.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-05-09T01:57:19.604 INFO:teuthology.orchestra.run.smithi019.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-05-09T01:57:19.625 INFO:teuthology.orchestra.run.smithi196.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 650 kB/s | 43 kB 00:00 2024-05-09T01:57:19.647 INFO:teuthology.orchestra.run.smithi059.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.5 MB/s | 309 kB 00:00 2024-05-09T01:57:19.667 INFO:teuthology.orchestra.run.smithi196.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 481 kB/s | 32 kB 00:00 2024-05-09T01:57:19.680 INFO:teuthology.orchestra.run.smithi059.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 587 kB/s | 19 kB 00:00 2024-05-09T01:57:19.687 INFO:teuthology.orchestra.run.smithi019.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-05-09T01:57:19.692 INFO:teuthology.orchestra.run.smithi196.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-05-09T01:57:19.706 INFO:teuthology.orchestra.run.smithi059.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-05-09T01:57:19.739 INFO:teuthology.orchestra.run.smithi059.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 787 kB/s | 26 kB 00:00 2024-05-09T01:57:19.751 INFO:teuthology.orchestra.run.smithi196.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-05-09T01:57:19.773 INFO:teuthology.orchestra.run.smithi059.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-05-09T01:57:19.776 INFO:teuthology.orchestra.run.smithi196.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-05-09T01:57:19.815 INFO:teuthology.orchestra.run.smithi059.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-05-09T01:57:19.821 INFO:teuthology.orchestra.run.smithi019.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-05-09T01:57:19.835 INFO:teuthology.orchestra.run.smithi196.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-05-09T01:57:19.852 INFO:teuthology.orchestra.run.smithi196.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 468 kB/s | 35 kB 00:00 2024-05-09T01:57:19.865 INFO:teuthology.orchestra.run.smithi059.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-05-09T01:57:19.879 INFO:teuthology.orchestra.run.smithi019.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-05-09T01:57:19.910 INFO:teuthology.orchestra.run.smithi196.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-09T01:57:19.969 INFO:teuthology.orchestra.run.smithi196.stdout:(122/156): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-05-09T01:57:19.994 INFO:teuthology.orchestra.run.smithi196.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-05-09T01:57:20.082 INFO:teuthology.orchestra.run.smithi196.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 95 kB/s | 11 kB 00:00 2024-05-09T01:57:20.088 INFO:teuthology.orchestra.run.smithi019.stdout:(108/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.8 MB/s | 16 MB 00:03 2024-05-09T01:57:20.099 INFO:teuthology.orchestra.run.smithi059.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.5 MB/s | 839 kB 00:00 2024-05-09T01:57:20.103 INFO:teuthology.orchestra.run.smithi196.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-05-09T01:57:20.115 INFO:teuthology.orchestra.run.smithi019.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 208 kB/s | 49 kB 00:00 2024-05-09T01:57:20.153 INFO:teuthology.orchestra.run.smithi196.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 326 kB/s | 23 kB 00:00 2024-05-09T01:57:20.178 INFO:teuthology.orchestra.run.smithi196.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-05-09T01:57:20.189 INFO:teuthology.orchestra.run.smithi019.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-05-09T01:57:20.221 INFO:teuthology.orchestra.run.smithi196.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 295 kB/s | 20 kB 00:00 2024-05-09T01:57:20.241 INFO:teuthology.orchestra.run.smithi059.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-05-09T01:57:20.291 INFO:teuthology.orchestra.run.smithi059.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 875 kB/s | 43 kB 00:00 2024-05-09T01:57:20.335 INFO:teuthology.orchestra.run.smithi059.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 772 kB/s | 32 kB 00:00 2024-05-09T01:57:20.346 INFO:teuthology.orchestra.run.smithi196.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 371 kB/s | 46 kB 00:00 2024-05-09T01:57:20.371 INFO:teuthology.orchestra.run.smithi196.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-05-09T01:57:20.375 INFO:teuthology.orchestra.run.smithi059.stdout:(115/156): python3-certifi-2021.10.8-2.el9.noar 362 kB/s | 15 kB 00:00 2024-05-09T01:57:20.421 INFO:teuthology.orchestra.run.smithi196.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-05-09T01:57:20.430 INFO:teuthology.orchestra.run.smithi019.stdout:(111/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-05-09T01:57:20.442 INFO:teuthology.orchestra.run.smithi059.stdout:(116/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-09T01:57:20.447 INFO:teuthology.orchestra.run.smithi196.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-05-09T01:57:20.497 INFO:teuthology.orchestra.run.smithi196.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-09T01:57:20.522 INFO:teuthology.orchestra.run.smithi196.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-05-09T01:57:20.531 INFO:teuthology.orchestra.run.smithi019.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 152 kB/s | 67 kB 00:00 2024-05-09T01:57:20.551 INFO:teuthology.orchestra.run.smithi059.stdout:(117/156): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-05-09T01:57:20.564 INFO:teuthology.orchestra.run.smithi196.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-09T01:57:20.589 INFO:teuthology.orchestra.run.smithi196.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-05-09T01:57:20.631 INFO:teuthology.orchestra.run.smithi196.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-05-09T01:57:20.642 INFO:teuthology.orchestra.run.smithi059.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-05-09T01:57:20.665 INFO:teuthology.orchestra.run.smithi196.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-09T01:57:20.681 INFO:teuthology.orchestra.run.smithi019.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.7 MB/s | 839 kB 00:00 2024-05-09T01:57:20.684 INFO:teuthology.orchestra.run.smithi059.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 853 kB/s | 35 kB 00:00 2024-05-09T01:57:20.699 INFO:teuthology.orchestra.run.smithi196.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 884 kB/s | 59 kB 00:00 2024-05-09T01:57:20.706 INFO:teuthology.orchestra.run.smithi019.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 248 kB/s | 43 kB 00:00 2024-05-09T01:57:20.732 INFO:teuthology.orchestra.run.smithi019.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-05-09T01:57:20.732 INFO:teuthology.orchestra.run.smithi196.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 533 kB/s | 36 kB 00:00 2024-05-09T01:57:20.766 INFO:teuthology.orchestra.run.smithi196.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 764 kB/s | 51 kB 00:00 2024-05-09T01:57:20.774 INFO:teuthology.orchestra.run.smithi019.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-05-09T01:57:20.776 INFO:teuthology.orchestra.run.smithi059.stdout:(120/156): libarrow-9.0.0-11.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-05-09T01:57:20.799 INFO:teuthology.orchestra.run.smithi019.stdout:(117/156): python3-cachetools-4.2.4-1.el9.noarc 274 kB/s | 32 kB 00:00 2024-05-09T01:57:20.802 INFO:teuthology.orchestra.run.smithi059.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-05-09T01:57:20.808 INFO:teuthology.orchestra.run.smithi196.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-09T01:57:20.833 INFO:teuthology.orchestra.run.smithi196.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-05-09T01:57:20.857 INFO:teuthology.orchestra.run.smithi019.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-05-09T01:57:20.860 INFO:teuthology.orchestra.run.smithi059.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-09T01:57:20.883 INFO:teuthology.orchestra.run.smithi019.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-05-09T01:57:20.892 INFO:teuthology.orchestra.run.smithi196.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-09T01:57:20.894 INFO:teuthology.orchestra.run.smithi059.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 320 kB/s | 11 kB 00:00 2024-05-09T01:57:20.917 INFO:teuthology.orchestra.run.smithi019.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-05-09T01:57:20.917 INFO:teuthology.orchestra.run.smithi196.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-05-09T01:57:20.927 INFO:teuthology.orchestra.run.smithi059.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 535 kB/s | 18 kB 00:00 2024-05-09T01:57:20.942 INFO:teuthology.orchestra.run.smithi019.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 420 kB/s | 35 kB 00:00 2024-05-09T01:57:20.961 INFO:teuthology.orchestra.run.smithi059.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 695 kB/s | 23 kB 00:00 2024-05-09T01:57:20.967 INFO:teuthology.orchestra.run.smithi196.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-09T01:57:20.968 INFO:teuthology.orchestra.run.smithi019.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-09T01:57:21.011 INFO:teuthology.orchestra.run.smithi059.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 390 kB/s | 19 kB 00:00 2024-05-09T01:57:21.018 INFO:teuthology.orchestra.run.smithi019.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-05-09T01:57:21.026 INFO:teuthology.orchestra.run.smithi196.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-09T01:57:21.043 INFO:teuthology.orchestra.run.smithi019.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-05-09T01:57:21.045 INFO:teuthology.orchestra.run.smithi059.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 591 kB/s | 20 kB 00:00 2024-05-09T01:57:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-05-09T01:57:21.110 INFO:teuthology.orchestra.run.smithi019.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 347 kB/s | 23 kB 00:00 2024-05-09T01:57:21.160 INFO:teuthology.orchestra.run.smithi019.stdout:(127/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-05-09T01:57:21.177 INFO:teuthology.orchestra.run.smithi019.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-05-09T01:57:21.202 INFO:teuthology.orchestra.run.smithi019.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-05-09T01:57:21.220 INFO:teuthology.orchestra.run.smithi059.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.6 MB/s | 2.0 MB 00:00 2024-05-09T01:57:21.254 INFO:teuthology.orchestra.run.smithi059.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-05-09T01:57:21.261 INFO:teuthology.orchestra.run.smithi019.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-09T01:57:21.301 INFO:teuthology.orchestra.run.smithi019.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 469 kB/s | 46 kB 00:00 2024-05-09T01:57:21.304 INFO:teuthology.orchestra.run.smithi059.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-09T01:57:21.338 INFO:teuthology.orchestra.run.smithi059.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-05-09T01:57:21.355 INFO:teuthology.orchestra.run.smithi059.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-09T01:57:21.389 INFO:teuthology.orchestra.run.smithi059.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 491 kB/s | 16 kB 00:00 2024-05-09T01:57:21.395 INFO:teuthology.orchestra.run.smithi019.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 4.4 MB/s | 1.0 MB 00:00 2024-05-09T01:57:21.412 INFO:teuthology.orchestra.run.smithi019.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 383 kB/s | 58 kB 00:00 2024-05-09T01:57:21.431 INFO:teuthology.orchestra.run.smithi059.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-05-09T01:57:21.437 INFO:teuthology.orchestra.run.smithi019.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-05-09T01:57:21.462 INFO:teuthology.orchestra.run.smithi019.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-05-09T01:57:21.464 INFO:teuthology.orchestra.run.smithi059.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 852 kB/s | 28 kB 00:00 2024-05-09T01:57:21.487 INFO:teuthology.orchestra.run.smithi019.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-09T01:57:21.490 INFO:teuthology.orchestra.run.smithi059.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 896 kB/s | 90 kB 00:00 2024-05-09T01:57:21.507 INFO:teuthology.orchestra.run.smithi059.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 737 kB/s | 31 kB 00:00 2024-05-09T01:57:21.513 INFO:teuthology.orchestra.run.smithi019.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-05-09T01:57:21.540 INFO:teuthology.orchestra.run.smithi059.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-05-09T01:57:21.546 INFO:teuthology.orchestra.run.smithi019.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-05-09T01:57:21.572 INFO:teuthology.orchestra.run.smithi019.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-09T01:57:21.574 INFO:teuthology.orchestra.run.smithi059.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.1 MB/s | 36 kB 00:00 2024-05-09T01:57:21.597 INFO:teuthology.orchestra.run.smithi019.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 704 kB/s | 59 kB 00:00 2024-05-09T01:57:21.608 INFO:teuthology.orchestra.run.smithi059.stdout:(140/156): python3-typing-extensions-4.4.0-2.el 1.5 MB/s | 51 kB 00:00 2024-05-09T01:57:21.626 INFO:teuthology.orchestra.run.smithi196.stdout:(148/156): ceph-test-19.0.0-3594.gc9da4049.el9. 3.6 MB/s | 43 MB 00:11 2024-05-09T01:57:21.631 INFO:teuthology.orchestra.run.smithi019.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 427 kB/s | 36 kB 00:00 2024-05-09T01:57:21.633 INFO:teuthology.orchestra.run.smithi059.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-05-09T01:57:21.656 INFO:teuthology.orchestra.run.smithi019.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 610 kB/s | 51 kB 00:00 2024-05-09T01:57:21.675 INFO:teuthology.orchestra.run.smithi059.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-05-09T01:57:21.693 INFO:teuthology.orchestra.run.smithi196.stdout:(149/156): librados2-19.0.0-3594.gc9da4049.el9. 4.6 MB/s | 3.3 MB 00:00 2024-05-09T01:57:21.706 INFO:teuthology.orchestra.run.smithi019.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-05-09T01:57:21.723 INFO:teuthology.orchestra.run.smithi019.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 973 kB/s | 90 kB 00:00 2024-05-09T01:57:21.766 INFO:teuthology.orchestra.run.smithi019.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-05-09T01:57:21.767 INFO:teuthology.orchestra.run.smithi059.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-05-09T01:57:21.790 INFO:teuthology.orchestra.run.smithi019.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-05-09T01:57:21.792 INFO:teuthology.orchestra.run.smithi059.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-05-09T01:57:21.809 INFO:teuthology.orchestra.run.smithi059.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 478 kB/s | 20 kB 00:00 2024-05-09T01:57:21.815 INFO:teuthology.orchestra.run.smithi019.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-09T01:57:21.818 INFO:teuthology.orchestra.run.smithi196.stdout:(150/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 3.9 MB/s | 3.1 MB 00:00 2024-05-09T01:57:21.851 INFO:teuthology.orchestra.run.smithi059.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-05-09T01:57:21.924 INFO:teuthology.orchestra.run.smithi019.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-05-09T01:57:22.160 INFO:teuthology.orchestra.run.smithi196.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 122 kB/s | 42 kB 00:00 2024-05-09T01:57:22.299 INFO:teuthology.orchestra.run.smithi019.stdout:(149/156): librados2-19.0.0-3594.gc9da4049.el9. 6.5 MB/s | 3.3 MB 00:00 2024-05-09T01:57:22.350 INFO:teuthology.orchestra.run.smithi019.stdout:(150/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 5.8 MB/s | 3.1 MB 00:00 2024-05-09T01:57:22.476 INFO:teuthology.orchestra.run.smithi059.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 2.4 MB/s | 1.6 MB 00:00 2024-05-09T01:57:22.502 INFO:teuthology.orchestra.run.smithi196.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 727 kB/s | 248 kB 00:00 2024-05-09T01:57:22.543 INFO:teuthology.orchestra.run.smithi059.stdout:(148/156): librados2-19.0.0-3594.gc9da4049.el9. 4.8 MB/s | 3.3 MB 00:00 2024-05-09T01:57:22.544 INFO:teuthology.orchestra.run.smithi196.stdout:(153/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.5 MB/s | 2.3 MB 00:00 2024-05-09T01:57:22.644 INFO:teuthology.orchestra.run.smithi196.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.9 MB/s | 1.8 MB 00:00 2024-05-09T01:57:22.835 INFO:teuthology.orchestra.run.smithi059.stdout:(149/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 8.6 MB/s | 3.1 MB 00:00 2024-05-09T01:57:22.944 INFO:teuthology.orchestra.run.smithi196.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 1.3 MB 00:00 2024-05-09T01:57:22.975 INFO:teuthology.orchestra.run.smithi019.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 67 kB/s | 42 kB 00:00 2024-05-09T01:57:23.050 INFO:teuthology.orchestra.run.smithi019.stdout:(152/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.1 MB/s | 2.3 MB 00:01 2024-05-09T01:57:23.342 INFO:teuthology.orchestra.run.smithi019.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.8 MB/s | 1.8 MB 00:01 2024-05-09T01:57:23.435 INFO:teuthology.orchestra.run.smithi059.stdout:(150/156): ceph-test-19.0.0-3594.gc9da4049.el9. 4.2 MB/s | 43 MB 00:10 2024-05-09T01:57:23.777 INFO:teuthology.orchestra.run.smithi059.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 122 kB/s | 42 kB 00:00 2024-05-09T01:57:23.784 INFO:teuthology.orchestra.run.smithi019.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 307 kB/s | 248 kB 00:00 2024-05-09T01:57:23.903 INFO:teuthology.orchestra.run.smithi059.stdout:(152/156): NetworkManager-1.47.5-1.el9.x86_64.r 1.7 MB/s | 2.3 MB 00:01 2024-05-09T01:57:24.028 INFO:teuthology.orchestra.run.smithi059.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 11 MB/s | 1.3 MB 00:00 2024-05-09T01:57:24.110 INFO:teuthology.orchestra.run.smithi019.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-05-09T01:57:24.136 INFO:teuthology.orchestra.run.smithi059.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 692 kB/s | 248 kB 00:00 2024-05-09T01:57:24.186 INFO:teuthology.orchestra.run.smithi196.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.5 MB/s | 2.4 MB 00:01 2024-05-09T01:57:24.192 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-09T01:57:24.193 INFO:teuthology.orchestra.run.smithi196.stdout:Total 12 MB/s | 219 MB 00:17 2024-05-09T01:57:24.236 INFO:teuthology.orchestra.run.smithi059.stdout:(155/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.3 MB/s | 1.8 MB 00:01 2024-05-09T01:57:24.426 INFO:teuthology.orchestra.run.smithi019.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-05-09T01:57:24.431 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-05-09T01:57:24.431 INFO:teuthology.orchestra.run.smithi019.stdout:Total 6.7 MB/s | 219 MB 00:32 2024-05-09T01:57:24.912 INFO:teuthology.orchestra.run.smithi059.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.7 MB/s | 2.4 MB 00:00 2024-05-09T01:57:24.917 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-09T01:57:24.918 INFO:teuthology.orchestra.run.smithi059.stdout:Total 12 MB/s | 219 MB 00:17 2024-05-09T01:57:25.344 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-09T01:57:25.493 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-09T01:57:25.493 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-09T01:57:25.640 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-05-09T01:57:25.809 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-05-09T01:57:25.809 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-05-09T01:57:26.097 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-09T01:57:26.265 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-09T01:57:26.266 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-09T01:57:27.251 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-09T01:57:27.251 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-09T01:57:28.012 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-05-09T01:57:28.012 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-05-09T01:57:28.144 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-09T01:57:28.144 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-09T01:57:33.351 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-09T01:57:33.440 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T01:57:33.478 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T01:57:33.531 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T01:57:33.566 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T01:57:33.591 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T01:57:33.634 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T01:57:33.960 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T01:57:34.076 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:57:34.148 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-05-09T01:57:34.183 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:57:34.210 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:57:34.248 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T01:57:34.284 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:57:34.287 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T01:57:34.314 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T01:57:34.344 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-09T01:57:34.349 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T01:57:34.390 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T01:57:34.415 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T01:57:34.417 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T01:57:34.456 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T01:57:34.466 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T01:57:34.468 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T01:57:34.510 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T01:57:34.511 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T01:57:34.539 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:57:34.543 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T01:57:34.568 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T01:57:34.612 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T01:57:34.642 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:57:34.673 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T01:57:34.719 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T01:57:34.741 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T01:57:34.761 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T01:57:34.882 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T01:57:34.941 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:57:34.949 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T01:57:35.008 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T01:57:35.070 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:57:35.077 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:57:35.079 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T01:57:35.092 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:57:35.185 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:57:35.206 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:57:35.214 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T01:57:35.255 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T01:57:35.298 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T01:57:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T01:57:35.453 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T01:57:35.478 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:57:35.488 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:57:35.521 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:57:35.571 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:57:35.599 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T01:57:35.603 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:57:35.630 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T01:57:35.675 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:57:35.693 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T01:57:35.723 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T01:57:35.748 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T01:57:35.764 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T01:57:35.814 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T01:57:35.841 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:57:35.873 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:57:35.931 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:57:35.951 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T01:57:35.959 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T01:57:35.978 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T01:57:36.004 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T01:57:36.021 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T01:57:36.038 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T01:57:36.043 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T01:57:36.062 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T01:57:36.102 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T01:57:36.110 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T01:57:36.137 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T01:57:36.181 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T01:57:36.213 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T01:57:36.242 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T01:57:36.253 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T01:57:36.253 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T01:57:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T01:57:36.313 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T01:57:36.364 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T01:57:36.403 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T01:57:36.436 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:57:36.455 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T01:57:36.501 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T01:57:36.506 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:57:36.530 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T01:57:36.566 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T01:57:36.608 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T01:57:36.638 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T01:57:36.682 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T01:57:36.704 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:57:36.708 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T01:57:36.743 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T01:57:36.752 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T01:57:36.815 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T01:57:36.827 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T01:57:36.836 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:57:36.860 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T01:57:36.896 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T01:57:36.918 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:57:36.965 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T01:57:36.971 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T01:57:37.020 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T01:57:37.030 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T01:57:37.080 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T01:57:37.113 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T01:57:37.121 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T01:57:37.140 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T01:57:37.219 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T01:57:37.227 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T01:57:37.256 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T01:57:37.280 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T01:57:37.292 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T01:57:37.309 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T01:57:37.354 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T01:57:37.359 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T01:57:37.408 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T01:57:37.432 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T01:57:37.451 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T01:57:37.464 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T01:57:37.483 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T01:57:37.500 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T01:57:37.511 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T01:57:37.528 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T01:57:37.546 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T01:57:37.583 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T01:57:37.600 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T01:57:37.632 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T01:57:37.635 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T01:57:37.643 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T01:57:37.664 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T01:57:37.671 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T01:57:37.699 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T01:57:37.713 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T01:57:37.720 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T01:57:37.752 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T01:57:37.777 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T01:57:37.792 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T01:57:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T01:57:37.862 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T01:57:37.864 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T01:57:37.890 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T01:57:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T01:57:37.948 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T01:57:37.965 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T01:57:38.012 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T01:57:38.014 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T01:57:38.092 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T01:57:38.104 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T01:57:38.186 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T01:57:38.239 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T01:57:38.256 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T01:57:38.289 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T01:57:38.292 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T01:57:38.334 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T01:57:38.380 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T01:57:38.416 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T01:57:38.459 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T01:57:38.465 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T01:57:38.551 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T01:57:38.586 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T01:57:38.634 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T01:57:38.667 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T01:57:38.696 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T01:57:38.713 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T01:57:38.833 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T01:57:38.868 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T01:57:38.877 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T01:57:38.882 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T01:57:38.912 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T01:57:38.937 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T01:57:38.953 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T01:57:38.956 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T01:57:38.990 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T01:57:39.032 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T01:57:39.033 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T01:57:39.048 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T01:57:39.071 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T01:57:39.087 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T01:57:39.094 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T01:57:39.122 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T01:57:39.126 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T01:57:39.152 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T01:57:39.168 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T01:57:39.177 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T01:57:39.205 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T01:57:39.241 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T01:57:39.251 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T01:57:39.290 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T01:57:39.291 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T01:57:39.324 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T01:57:39.350 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T01:57:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T01:57:39.379 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T01:57:39.410 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T01:57:39.417 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T01:57:39.445 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T01:57:39.445 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T01:57:39.487 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T01:57:39.507 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T01:57:39.526 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T01:57:39.554 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T01:57:39.559 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T01:57:39.581 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T01:57:39.590 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T01:57:39.598 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T01:57:39.623 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T01:57:39.625 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T01:57:39.628 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T01:57:39.656 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T01:57:39.661 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T01:57:39.670 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T01:57:39.689 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T01:57:39.710 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T01:57:39.724 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T01:57:39.745 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T01:57:39.769 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T01:57:39.788 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T01:57:39.810 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T01:57:39.811 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T01:57:39.853 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T01:57:39.854 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T01:57:39.885 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T01:57:39.885 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T01:57:39.910 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T01:57:39.931 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T01:57:39.931 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T01:57:40.007 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T01:57:40.028 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T01:57:40.076 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T01:57:40.086 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T01:57:40.119 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T01:57:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T01:57:40.149 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T01:57:40.176 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T01:57:40.191 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T01:57:40.204 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T01:57:40.238 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T01:57:40.277 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T01:57:40.291 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T01:57:40.317 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T01:57:40.323 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T01:57:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T01:57:40.364 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T01:57:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T01:57:40.388 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T01:57:40.414 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T01:57:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T01:57:40.457 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T01:57:40.491 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T01:57:40.492 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T01:57:40.581 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T01:57:40.621 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T01:57:40.630 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T01:57:40.661 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T01:57:40.662 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T01:57:40.691 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T01:57:40.708 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T01:57:40.733 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T01:57:40.773 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T01:57:40.798 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T01:57:40.813 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T01:57:40.818 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T01:57:40.842 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T01:57:40.843 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T01:57:40.872 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T01:57:40.876 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T01:57:40.876 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T01:57:40.910 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T01:57:40.952 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T01:57:40.972 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T01:57:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T01:57:41.026 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T01:57:41.077 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T01:57:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T01:57:41.125 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T01:57:41.153 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T01:57:41.156 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T01:57:41.204 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T01:57:41.307 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T01:57:41.328 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T01:57:41.359 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T01:57:41.375 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T01:57:41.437 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T01:57:41.642 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T01:57:41.693 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T01:57:41.726 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T01:57:41.828 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T01:57:41.893 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T01:57:41.983 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T01:57:42.020 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T01:57:42.073 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T01:57:42.153 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T01:57:42.183 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T01:57:42.196 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T01:57:42.203 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T01:57:42.230 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T01:57:42.234 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T01:57:42.314 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T01:57:42.337 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T01:57:42.389 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T01:57:42.481 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T01:57:42.516 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T01:57:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T01:57:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T01:57:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T01:57:42.994 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T01:57:43.110 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T01:57:43.385 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T01:57:43.465 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T01:57:43.492 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T01:57:43.501 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T01:57:43.525 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T01:57:43.624 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T01:57:43.841 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T01:57:43.883 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T01:57:43.954 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T01:57:43.985 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T01:57:44.021 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T01:57:44.432 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T01:57:44.506 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T01:57:44.515 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T01:57:44.533 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T01:57:44.534 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T01:57:44.559 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T01:57:44.604 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T01:57:44.626 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T01:57:44.893 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T01:57:44.939 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T01:57:44.948 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T01:57:45.030 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T01:57:45.033 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T01:57:45.062 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T01:57:45.062 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T01:57:45.095 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T01:57:45.097 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T01:57:45.429 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T01:57:45.474 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T01:57:45.549 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T01:57:45.579 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T01:57:45.612 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T01:57:45.615 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T01:57:45.642 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T01:57:45.713 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T01:57:45.743 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T01:57:46.131 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T01:57:46.160 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T01:57:46.231 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T01:57:46.253 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T01:57:46.886 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:57:46.931 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:57:47.483 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:57:47.534 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T01:57:47.625 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T01:57:47.625 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T01:57:47.625 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T01:57:47.625 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:57:47.669 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T01:57:48.018 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:57:48.049 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:57:48.574 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:57:48.618 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:57:48.645 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:57:48.685 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T01:57:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T01:57:48.778 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T01:57:48.778 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T01:57:48.779 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:57:48.812 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T01:57:49.206 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T01:57:49.246 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T01:57:49.341 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T01:57:49.341 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T01:57:49.341 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T01:57:49.341 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:57:49.372 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T01:57:59.010 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T01:57:59.011 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-05-09T01:57:59.011 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-05-09T01:57:59.011 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-05-09T01:57:59.011 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-05-09T01:57:59.011 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-05-09T01:57:59.011 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-05-09T01:57:59.011 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-05-09T01:57:59.011 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:57:59.182 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T01:57:59.198 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T01:57:59.198 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:57:59.198 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T01:57:59.198 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:58:00.677 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T01:58:00.677 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-05-09T01:58:00.677 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-05-09T01:58:00.677 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-05-09T01:58:00.677 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-05-09T01:58:00.677 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-05-09T01:58:00.677 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-05-09T01:58:00.678 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-05-09T01:58:00.678 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:58:00.785 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T01:58:00.811 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T01:58:00.811 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:00.812 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T01:58:00.812 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T01:58:00.812 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T01:58:00.812 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:58:00.868 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T01:58:00.885 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T01:58:00.885 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:00.886 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T01:58:00.886 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:58:01.036 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T01:58:01.058 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T01:58:01.058 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:01.058 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T01:58:01.058 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T01:58:01.059 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T01:58:01.059 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:58:01.091 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T01:58:01.091 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /sys 2024-05-09T01:58:01.092 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /proc 2024-05-09T01:58:01.092 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /mnt 2024-05-09T01:58:01.092 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /var/tmp 2024-05-09T01:58:01.092 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /home 2024-05-09T01:58:01.092 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /root 2024-05-09T01:58:01.092 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /tmp 2024-05-09T01:58:01.092 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:58:01.272 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T01:58:01.291 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T01:58:01.291 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:01.291 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T01:58:01.291 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:58:01.520 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T01:58:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T01:58:01.543 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:01.543 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T01:58:01.543 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T01:58:01.543 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T01:58:01.543 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:58:01.641 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T01:58:01.690 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T01:58:01.745 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T01:58:01.930 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T01:58:01.987 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T01:58:02.055 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T01:58:02.143 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T01:58:02.274 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T01:58:02.337 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T01:58:02.369 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T01:58:02.397 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T01:58:02.529 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T01:58:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T01:58:02.558 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:02.558 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T01:58:02.558 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T01:58:02.558 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T01:58:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:58:02.797 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T01:58:02.821 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T01:58:02.821 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:02.821 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T01:58:02.821 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T01:58:02.821 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T01:58:02.821 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:58:02.841 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T01:58:02.871 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T01:58:02.907 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T01:58:02.923 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T01:58:02.951 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T01:58:02.951 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:02.951 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T01:58:02.951 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T01:58:02.951 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T01:58:02.951 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:58:02.964 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T01:58:02.993 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T01:58:03.182 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T01:58:03.206 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T01:58:03.206 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:03.206 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T01:58:03.206 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T01:58:03.206 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T01:58:03.206 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:58:03.306 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T01:58:03.330 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T01:58:03.330 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:03.330 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T01:58:03.330 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T01:58:03.331 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T01:58:03.331 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:58:03.436 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T01:58:03.495 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T01:58:03.558 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T01:58:03.683 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T01:58:03.706 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T01:58:03.707 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:03.707 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T01:58:03.707 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T01:58:03.707 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T01:58:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:58:03.752 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T01:58:03.807 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T01:58:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T01:58:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T01:58:03.888 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T01:58:03.936 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T01:58:03.977 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T01:58:04.033 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T01:58:04.105 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T01:58:04.132 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T01:58:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T01:58:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T01:58:04.186 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T01:58:04.223 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T01:58:04.250 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T01:58:04.265 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T01:58:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T01:58:04.507 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T01:58:04.551 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T01:58:04.591 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T01:58:04.598 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T01:58:04.618 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T01:58:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T01:58:04.665 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T01:58:04.729 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T01:58:04.755 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T01:58:05.040 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T01:58:05.062 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T01:58:05.067 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T01:58:05.096 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T01:58:05.099 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T01:58:05.165 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T01:58:05.189 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T01:58:05.206 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T01:58:05.378 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T01:58:05.406 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T01:58:05.429 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T01:58:05.429 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:05.429 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T01:58:05.429 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T01:58:05.429 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T01:58:05.429 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:58:05.473 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T01:58:05.484 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T01:58:05.582 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T01:58:05.629 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T01:58:05.847 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T01:58:05.882 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T01:58:06.288 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T01:58:06.325 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T01:58:06.559 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T01:58:06.585 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T01:58:06.607 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T01:58:06.607 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:06.607 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T01:58:06.607 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T01:58:06.607 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T01:58:06.607 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:58:06.660 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T01:58:06.681 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T01:58:06.682 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:06.682 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T01:58:06.682 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:58:06.727 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T01:58:06.749 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T01:58:06.750 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:06.750 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T01:58:06.750 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T01:58:06.750 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T01:58:06.750 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:58:06.896 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T01:58:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T01:58:07.043 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T01:58:07.061 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T01:58:07.083 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T01:58:07.083 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:07.084 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T01:58:07.084 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T01:58:07.084 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T01:58:07.084 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:58:07.214 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T01:58:07.251 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T01:58:07.276 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T01:58:07.276 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:07.277 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T01:58:07.277 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T01:58:07.277 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T01:58:07.277 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:58:07.327 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T01:58:07.338 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T01:58:07.339 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T01:58:07.367 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T01:58:07.441 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T01:58:07.485 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T01:58:07.495 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T01:58:07.658 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T01:58:07.686 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T01:58:07.710 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T01:58:07.710 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:07.710 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T01:58:07.711 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T01:58:07.711 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T01:58:07.711 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:58:07.761 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T01:58:07.772 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T01:58:07.874 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T01:58:07.928 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T01:58:08.494 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T01:58:08.538 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T01:58:08.562 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T01:58:08.563 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:08.563 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T01:58:08.563 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T01:58:08.563 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T01:58:08.563 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:58:08.605 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T01:58:08.629 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T01:58:08.629 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:08.629 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T01:58:08.629 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:58:08.681 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T01:58:08.706 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T01:58:08.706 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:08.706 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T01:58:08.706 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T01:58:08.706 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T01:58:08.706 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:58:08.911 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T01:58:08.947 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T01:58:08.971 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T01:58:08.971 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:08.971 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T01:58:08.971 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T01:58:08.971 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T01:58:08.971 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:58:09.014 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T01:58:09.032 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T01:58:09.038 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T01:58:09.038 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:09.038 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T01:58:09.038 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:58:09.056 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T01:58:09.056 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:09.056 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T01:58:09.056 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T01:58:09.056 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T01:58:09.056 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:58:09.090 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T01:58:09.114 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T01:58:09.114 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:09.114 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T01:58:09.114 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T01:58:09.114 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T01:58:09.114 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:58:09.440 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T01:58:09.464 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T01:58:09.464 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T01:58:09.464 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T01:58:09.464 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T01:58:09.464 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T01:58:09.464 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:58:11.331 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T01:58:11.372 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T01:58:11.413 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T01:58:11.445 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T01:58:11.473 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T01:58:11.651 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T01:58:11.693 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T01:58:11.723 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T01:58:11.753 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T01:58:11.808 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T01:58:12.394 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:58:12.430 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:58:12.450 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:58:13.272 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T01:58:13.360 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T01:58:13.409 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T01:58:13.498 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T01:58:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T01:58:13.572 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T01:58:13.572 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T01:58:13.640 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T01:58:13.673 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T01:58:13.673 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T01:58:13.686 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T01:58:13.728 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T01:58:13.741 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T01:58:13.766 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T01:58:13.768 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T01:58:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:58:13.771 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:58:13.800 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T01:58:13.827 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T01:58:13.959 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:58:13.959 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T01:58:13.969 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T01:58:14.011 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T01:58:14.032 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T01:58:14.052 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T01:58:14.052 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T01:58:14.083 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T01:58:14.111 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T01:58:14.115 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:58:14.190 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T01:58:14.220 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T01:58:14.250 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T01:58:14.288 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T01:58:14.305 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T01:58:14.348 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T01:58:14.378 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T01:58:14.409 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T01:58:14.446 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T01:58:14.905 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:58:14.944 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:58:14.965 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:58:15.085 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:58:15.118 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:58:15.139 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T01:58:15.865 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T01:58:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T01:58:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T01:58:16.050 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T01:58:16.106 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T01:58:16.135 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T01:58:16.161 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T01:58:16.162 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T01:58:16.169 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T01:58:16.216 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T01:58:16.254 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T01:58:16.296 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T01:58:16.296 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T01:58:16.311 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T01:58:16.351 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T01:58:16.377 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T01:58:16.377 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T01:58:16.380 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T01:58:16.405 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T01:58:16.414 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:58:16.414 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:58:16.470 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T01:58:16.503 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T01:58:16.503 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T01:58:16.587 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T01:58:16.599 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:58:16.599 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T01:58:16.612 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T01:58:16.617 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:58:16.617 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:58:16.683 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T01:58:16.750 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:58:16.814 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T01:58:16.814 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T01:58:16.905 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T01:58:16.971 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:58:19.780 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:58:19.780 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T01:58:19.780 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T01:58:19.780 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T01:58:19.780 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T01:58:19.780 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T01:58:19.780 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T01:58:19.780 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T01:58:19.780 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:58:19.780 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:58:19.780 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T01:58:19.781 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T01:58:19.781 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T01:58:19.781 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:58:19.781 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T01:58:19.781 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T01:58:19.781 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T01:58:19.781 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T01:58:19.781 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T01:58:19.781 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T01:58:19.781 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T01:58:19.781 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T01:58:19.781 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T01:58:19.781 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T01:58:19.781 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T01:58:19.782 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T01:58:19.782 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T01:58:19.782 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T01:58:19.782 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T01:58:19.782 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T01:58:19.793 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T01:58:19.793 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T01:58:19.793 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T01:58:19.794 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T01:58:19.794 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T01:58:19.794 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T01:58:19.794 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T01:58:19.794 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T01:58:19.794 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T01:58:19.794 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T01:58:19.794 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T01:58:19.794 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T01:58:19.795 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T01:58:19.795 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T01:58:19.795 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T01:58:19.795 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T01:58:19.795 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T01:58:19.795 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T01:58:19.795 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T01:58:19.795 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T01:58:19.795 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T01:58:19.795 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T01:58:19.795 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T01:58:19.795 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T01:58:19.795 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T01:58:19.795 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T01:58:19.796 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T01:58:19.796 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T01:58:19.796 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T01:58:19.796 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T01:58:19.796 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T01:58:19.796 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T01:58:19.796 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T01:58:19.796 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T01:58:19.796 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T01:58:19.796 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T01:58:19.796 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T01:58:19.796 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T01:58:19.796 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T01:58:19.797 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T01:58:19.797 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T01:58:19.797 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T01:58:19.797 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T01:58:19.797 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T01:58:19.797 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T01:58:19.797 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T01:58:19.797 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T01:58:19.797 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T01:58:19.797 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T01:58:19.797 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T01:58:19.797 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T01:58:19.797 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T01:58:19.797 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T01:58:19.798 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T01:58:19.798 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T01:58:19.798 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T01:58:19.798 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T01:58:19.798 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T01:58:19.798 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T01:58:19.798 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T01:58:19.798 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T01:58:19.798 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T01:58:19.798 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T01:58:19.798 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T01:58:19.798 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T01:58:19.798 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T01:58:19.798 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T01:58:19.799 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T01:58:19.799 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T01:58:19.799 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T01:58:19.799 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T01:58:19.799 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T01:58:19.799 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T01:58:19.799 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T01:58:19.799 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T01:58:19.799 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T01:58:19.799 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T01:58:19.799 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T01:58:19.799 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T01:58:19.799 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T01:58:19.799 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T01:58:19.799 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T01:58:19.800 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T01:58:19.800 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T01:58:19.800 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T01:58:19.800 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T01:58:19.800 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T01:58:19.800 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T01:58:19.800 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T01:58:19.800 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T01:58:19.800 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T01:58:19.800 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T01:58:19.800 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T01:58:19.800 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T01:58:19.800 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T01:58:19.800 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T01:58:19.801 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T01:58:19.801 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T01:58:19.801 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T01:58:19.801 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T01:58:19.801 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T01:58:19.801 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T01:58:19.801 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T01:58:19.801 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T01:58:19.801 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T01:58:19.801 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T01:58:19.801 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T01:58:19.801 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T01:58:19.801 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T01:58:19.801 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T01:58:19.802 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T01:58:19.802 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T01:58:19.802 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T01:58:19.802 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T01:58:19.802 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T01:58:19.802 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T01:58:19.802 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T01:58:19.802 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T01:58:19.802 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T01:58:19.802 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T01:58:19.802 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T01:58:19.802 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T01:58:19.802 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T01:58:19.802 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T01:58:19.802 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T01:58:19.803 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T01:58:19.803 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T01:58:19.803 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T01:58:19.803 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T01:58:19.803 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T01:58:19.803 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T01:58:19.803 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T01:58:19.803 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T01:58:19.803 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T01:58:22.333 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:58:22.333 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T01:58:22.333 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T01:58:22.333 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T01:58:22.333 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T01:58:22.333 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T01:58:22.333 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T01:58:22.333 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T01:58:22.333 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:58:22.333 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:58:22.334 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T01:58:22.334 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T01:58:22.334 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T01:58:22.334 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:58:22.334 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T01:58:22.334 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T01:58:22.334 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T01:58:22.334 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T01:58:22.334 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T01:58:22.334 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T01:58:22.335 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T01:58:22.335 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T01:58:22.335 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T01:58:22.335 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T01:58:22.335 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T01:58:22.335 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T01:58:22.335 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T01:58:22.335 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T01:58:22.335 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T01:58:22.335 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T01:58:22.335 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T01:58:22.335 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T01:58:22.335 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T01:58:22.335 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T01:58:22.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T01:58:22.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T01:58:22.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T01:58:22.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T01:58:22.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T01:58:22.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T01:58:22.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T01:58:22.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T01:58:22.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T01:58:22.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T01:58:22.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T01:58:22.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T01:58:22.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T01:58:22.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T01:58:22.336 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T01:58:22.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T01:58:22.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T01:58:22.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T01:58:22.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T01:58:22.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T01:58:22.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T01:58:22.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T01:58:22.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T01:58:22.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T01:58:22.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T01:58:22.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T01:58:22.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T01:58:22.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T01:58:22.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T01:58:22.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T01:58:22.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T01:58:22.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T01:58:22.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T01:58:22.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T01:58:22.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T01:58:22.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T01:58:22.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T01:58:22.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T01:58:22.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T01:58:22.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T01:58:22.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T01:58:22.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T01:58:22.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T01:58:22.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T01:58:22.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T01:58:22.339 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T01:58:22.340 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T01:58:22.340 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T01:58:22.340 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T01:58:22.340 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T01:58:22.340 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T01:58:22.340 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T01:58:22.340 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T01:58:22.340 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T01:58:22.340 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T01:58:22.340 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T01:58:22.340 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T01:58:22.340 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T01:58:22.340 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T01:58:22.340 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T01:58:22.341 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T01:58:22.341 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T01:58:22.341 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T01:58:22.341 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T01:58:22.341 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T01:58:22.341 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T01:58:22.341 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T01:58:22.341 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T01:58:22.341 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T01:58:22.341 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T01:58:22.341 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T01:58:22.341 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T01:58:22.341 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T01:58:22.341 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T01:58:22.342 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T01:58:22.342 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T01:58:22.342 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T01:58:22.342 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T01:58:22.342 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T01:58:22.342 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T01:58:22.342 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T01:58:22.342 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T01:58:22.342 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T01:58:22.342 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T01:58:22.342 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T01:58:22.342 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T01:58:22.342 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T01:58:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T01:58:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T01:58:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T01:58:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T01:58:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T01:58:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T01:58:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T01:58:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T01:58:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T01:58:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T01:58:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T01:58:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T01:58:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T01:58:22.343 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T01:58:22.344 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T01:58:22.344 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T01:58:22.344 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T01:58:22.344 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T01:58:22.344 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T01:58:22.344 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T01:58:22.344 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T01:58:22.344 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T01:58:22.344 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T01:58:22.344 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T01:58:22.344 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T01:58:22.344 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T01:58:22.344 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T01:58:22.344 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T01:58:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T01:58:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T01:58:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T01:58:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T01:58:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T01:58:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T01:58:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T01:58:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T01:58:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T01:58:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T01:58:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T01:58:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T01:58:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T01:58:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T01:58:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T01:58:22.359 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:58:22.359 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T01:58:22.360 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T01:58:22.360 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T01:58:22.360 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T01:58:22.360 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T01:58:22.360 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T01:58:22.360 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T01:58:22.360 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T01:58:22.360 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T01:58:22.360 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T01:58:22.360 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T01:58:22.360 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T01:58:22.360 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T01:58:22.360 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T01:58:22.360 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T01:58:22.361 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T01:58:22.361 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T01:58:22.361 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T01:58:22.361 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T01:58:22.361 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T01:58:22.361 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T01:58:22.361 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T01:58:22.361 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T01:58:22.361 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T01:58:22.361 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T01:58:22.361 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T01:58:22.361 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T01:58:22.361 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T01:58:22.361 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T01:58:22.362 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T01:58:22.362 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T01:58:22.362 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T01:58:22.362 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T01:58:22.362 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T01:58:22.362 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T01:58:22.362 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T01:58:22.362 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T01:58:22.362 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T01:58:22.362 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T01:58:22.362 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T01:58:22.362 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T01:58:22.362 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T01:58:22.362 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T01:58:22.363 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T01:58:22.363 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T01:58:22.363 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T01:58:22.363 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T01:58:22.363 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T01:58:22.363 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T01:58:22.363 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T01:58:22.363 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T01:58:22.363 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T01:58:22.363 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T01:58:22.363 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T01:58:22.363 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T01:58:22.363 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T01:58:22.363 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T01:58:22.364 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T01:58:22.364 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T01:58:22.364 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T01:58:22.364 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T01:58:22.364 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T01:58:22.364 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T01:58:22.364 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T01:58:22.364 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T01:58:22.364 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T01:58:22.364 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T01:58:22.364 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T01:58:22.364 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T01:58:22.364 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T01:58:22.364 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T01:58:22.365 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T01:58:22.365 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T01:58:22.365 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T01:58:22.365 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T01:58:22.365 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T01:58:22.365 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T01:58:22.365 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T01:58:22.365 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T01:58:22.365 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T01:58:22.365 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T01:58:22.365 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T01:58:22.365 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T01:58:22.365 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T01:58:22.365 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T01:58:22.365 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T01:58:22.366 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T01:58:22.366 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T01:58:22.366 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T01:58:22.366 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T01:58:22.366 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T01:58:22.366 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T01:58:22.366 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T01:58:22.366 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T01:58:22.366 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T01:58:22.366 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T01:58:22.366 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T01:58:22.366 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T01:58:22.366 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T01:58:22.367 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T01:58:22.367 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T01:58:22.367 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T01:58:22.367 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T01:58:22.367 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T01:58:22.367 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T01:58:22.367 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T01:58:22.367 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T01:58:22.367 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T01:58:22.367 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T01:58:22.367 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T01:58:22.367 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T01:58:22.367 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T01:58:22.367 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T01:58:22.368 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T01:58:22.368 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T01:58:22.368 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T01:58:22.368 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T01:58:22.368 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T01:58:22.368 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T01:58:22.368 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T01:58:22.368 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T01:58:22.368 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T01:58:22.368 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T01:58:22.368 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T01:58:22.368 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T01:58:22.368 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T01:58:22.368 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T01:58:22.369 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T01:58:22.369 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T01:58:22.369 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T01:58:22.369 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T01:58:22.369 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T01:58:22.369 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T01:58:22.369 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T01:58:22.369 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T01:58:22.369 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T01:58:22.369 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T01:58:22.369 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T01:58:22.369 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T01:58:22.369 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T01:58:22.369 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T01:58:22.369 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T01:58:22.370 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T01:58:22.370 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T01:58:22.370 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T01:58:22.370 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T01:58:22.370 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T01:58:22.370 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T01:58:22.370 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T01:58:22.370 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T01:58:22.370 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T01:58:22.370 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T01:58:22.370 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T01:58:22.370 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T01:58:22.370 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T01:58:22.370 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T01:58:22.371 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T01:58:22.371 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T01:58:22.371 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T01:58:22.371 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T01:58:22.371 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T01:58:22.371 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T01:58:22.371 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T01:58:22.522 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:58:22.522 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:58:22.522 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-05-09T01:58:22.522 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T01:58:22.522 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T01:58:22.522 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T01:58:22.522 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T01:58:22.522 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.523 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.523 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T01:58:22.523 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T01:58:22.523 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-09T01:58:22.523 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T01:58:22.523 INFO:teuthology.orchestra.run.smithi196.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T01:58:22.523 INFO:teuthology.orchestra.run.smithi196.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T01:58:22.523 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T01:58:22.523 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.523 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.523 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.523 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.523 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:22.524 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.524 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.524 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.524 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:22.524 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:22.524 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:22.524 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:22.524 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:22.524 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.524 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.525 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:22.525 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.525 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.525 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.525 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:22.525 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:22.525 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.525 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.525 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:22.525 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T01:58:22.525 INFO:teuthology.orchestra.run.smithi196.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T01:58:22.525 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T01:58:22.526 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T01:58:22.526 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T01:58:22.526 INFO:teuthology.orchestra.run.smithi196.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T01:58:22.526 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T01:58:22.526 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T01:58:22.526 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T01:58:22.526 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T01:58:22.526 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T01:58:22.526 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.526 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.526 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.526 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T01:58:22.526 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T01:58:22.527 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T01:58:22.527 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T01:58:22.527 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T01:58:22.527 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.527 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.527 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T01:58:22.527 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.527 INFO:teuthology.orchestra.run.smithi196.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T01:58:22.527 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T01:58:22.527 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T01:58:22.527 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T01:58:22.527 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T01:58:22.527 INFO:teuthology.orchestra.run.smithi196.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T01:58:22.527 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T01:58:22.528 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T01:58:22.528 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T01:58:22.528 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T01:58:22.528 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T01:58:22.528 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T01:58:22.528 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T01:58:22.528 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T01:58:22.528 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T01:58:22.528 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T01:58:22.528 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T01:58:22.528 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T01:58:22.528 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T01:58:22.528 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T01:58:22.528 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T01:58:22.529 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T01:58:22.529 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T01:58:22.529 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.529 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.529 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.529 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T01:58:22.529 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T01:58:22.529 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T01:58:22.529 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T01:58:22.529 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T01:58:22.529 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T01:58:22.529 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T01:58:22.530 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T01:58:22.530 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T01:58:22.530 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T01:58:22.530 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T01:58:22.530 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T01:58:22.530 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T01:58:22.530 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T01:58:22.530 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T01:58:22.530 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T01:58:22.530 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T01:58:22.530 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T01:58:22.530 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T01:58:22.530 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T01:58:22.530 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T01:58:22.531 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T01:58:22.531 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T01:58:22.531 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T01:58:22.531 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T01:58:22.531 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T01:58:22.531 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T01:58:22.531 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T01:58:22.531 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T01:58:22.531 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T01:58:22.531 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T01:58:22.531 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T01:58:22.531 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T01:58:22.531 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T01:58:22.531 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T01:58:22.532 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T01:58:22.532 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T01:58:22.532 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T01:58:22.532 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T01:58:22.532 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T01:58:22.532 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.532 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.532 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T01:58:22.532 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T01:58:22.532 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T01:58:22.532 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.532 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T01:58:22.532 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T01:58:22.532 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T01:58:22.532 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T01:58:22.533 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T01:58:22.533 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T01:58:22.533 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T01:58:22.533 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T01:58:22.533 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T01:58:22.533 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T01:58:22.533 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T01:58:22.533 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T01:58:22.533 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T01:58:22.533 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T01:58:22.534 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T01:58:22.534 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.534 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.534 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:22.534 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T01:58:22.534 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T01:58:22.534 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T01:58:22.534 INFO:teuthology.orchestra.run.smithi196.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T01:58:22.534 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T01:58:22.534 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T01:58:22.535 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T01:58:22.535 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T01:58:22.535 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-09T01:58:22.947 DEBUG:teuthology.parallel:result is None 2024-05-09T01:58:25.097 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:58:25.097 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:58:25.097 INFO:teuthology.orchestra.run.smithi059.stdout:Upgraded: 2024-05-09T01:58:25.097 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T01:58:25.097 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T01:58:25.097 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T01:58:25.097 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T01:58:25.097 INFO:teuthology.orchestra.run.smithi059.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.097 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.097 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T01:58:25.098 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T01:58:25.098 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-09T01:58:25.098 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T01:58:25.098 INFO:teuthology.orchestra.run.smithi059.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T01:58:25.098 INFO:teuthology.orchestra.run.smithi059.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T01:58:25.098 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T01:58:25.098 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.098 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.098 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.098 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.098 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.098 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.098 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.099 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.099 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.099 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.099 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.099 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.099 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.099 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.099 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.099 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.099 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.099 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.099 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.099 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.100 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.100 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.100 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.100 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.100 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T01:58:25.100 INFO:teuthology.orchestra.run.smithi059.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T01:58:25.100 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T01:58:25.100 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T01:58:25.100 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T01:58:25.100 INFO:teuthology.orchestra.run.smithi059.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T01:58:25.100 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T01:58:25.100 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T01:58:25.100 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T01:58:25.100 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T01:58:25.101 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T01:58:25.101 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.101 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.101 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.101 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T01:58:25.101 INFO:teuthology.orchestra.run.smithi059.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T01:58:25.101 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T01:58:25.101 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T01:58:25.101 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T01:58:25.101 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.101 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.101 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T01:58:25.101 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.102 INFO:teuthology.orchestra.run.smithi059.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T01:58:25.102 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T01:58:25.102 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T01:58:25.102 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T01:58:25.102 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T01:58:25.102 INFO:teuthology.orchestra.run.smithi059.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T01:58:25.102 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T01:58:25.102 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T01:58:25.102 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T01:58:25.102 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T01:58:25.102 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T01:58:25.102 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T01:58:25.102 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T01:58:25.102 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T01:58:25.103 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T01:58:25.103 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T01:58:25.103 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T01:58:25.103 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T01:58:25.103 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T01:58:25.103 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T01:58:25.103 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T01:58:25.103 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T01:58:25.103 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T01:58:25.103 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.103 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.103 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.103 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T01:58:25.103 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T01:58:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T01:58:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T01:58:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T01:58:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T01:58:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T01:58:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T01:58:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T01:58:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T01:58:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T01:58:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T01:58:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T01:58:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T01:58:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T01:58:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T01:58:25.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T01:58:25.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T01:58:25.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T01:58:25.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T01:58:25.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T01:58:25.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T01:58:25.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T01:58:25.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T01:58:25.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T01:58:25.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T01:58:25.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T01:58:25.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T01:58:25.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T01:58:25.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T01:58:25.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T01:58:25.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T01:58:25.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T01:58:25.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T01:58:25.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T01:58:25.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T01:58:25.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T01:58:25.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T01:58:25.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T01:58:25.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T01:58:25.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T01:58:25.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T01:58:25.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T01:58:25.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T01:58:25.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T01:58:25.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T01:58:25.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T01:58:25.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T01:58:25.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T01:58:25.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T01:58:25.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T01:58:25.108 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T01:58:25.108 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T01:58:25.108 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T01:58:25.108 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T01:58:25.108 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T01:58:25.108 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T01:58:25.108 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T01:58:25.108 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.108 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.108 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.108 INFO:teuthology.orchestra.run.smithi059.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T01:58:25.108 INFO:teuthology.orchestra.run.smithi059.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T01:58:25.108 INFO:teuthology.orchestra.run.smithi059.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T01:58:25.108 INFO:teuthology.orchestra.run.smithi059.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T01:58:25.108 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T01:58:25.109 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T01:58:25.109 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T01:58:25.109 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-09T01:58:25.109 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-09T01:58:25.158 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T01:58:25.159 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:58:25.159 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-05-09T01:58:25.159 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T01:58:25.159 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T01:58:25.159 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T01:58:25.159 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T01:58:25.159 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.159 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.159 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T01:58:25.159 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T01:58:25.159 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-05-09T01:58:25.159 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T01:58:25.159 INFO:teuthology.orchestra.run.smithi019.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T01:58:25.160 INFO:teuthology.orchestra.run.smithi019.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T01:58:25.160 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T01:58:25.160 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.160 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.160 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.160 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.160 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.160 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.160 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.160 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.160 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.160 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.160 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.160 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.161 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.161 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.161 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.161 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.161 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.161 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.161 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.161 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.161 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.161 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.161 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.161 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T01:58:25.161 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T01:58:25.162 INFO:teuthology.orchestra.run.smithi019.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T01:58:25.162 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T01:58:25.162 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T01:58:25.162 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T01:58:25.162 INFO:teuthology.orchestra.run.smithi019.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T01:58:25.162 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T01:58:25.162 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T01:58:25.162 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T01:58:25.162 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T01:58:25.162 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T01:58:25.162 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.162 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.162 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.162 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T01:58:25.163 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T01:58:25.163 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T01:58:25.163 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T01:58:25.163 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T01:58:25.163 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.163 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.163 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T01:58:25.163 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.163 INFO:teuthology.orchestra.run.smithi019.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T01:58:25.163 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T01:58:25.163 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T01:58:25.163 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T01:58:25.163 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T01:58:25.164 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T01:58:25.164 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T01:58:25.164 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T01:58:25.164 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T01:58:25.164 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T01:58:25.164 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T01:58:25.164 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T01:58:25.164 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T01:58:25.164 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T01:58:25.164 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T01:58:25.164 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T01:58:25.164 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T01:58:25.164 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T01:58:25.164 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T01:58:25.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T01:58:25.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T01:58:25.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T01:58:25.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T01:58:25.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T01:58:25.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T01:58:25.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T01:58:25.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T01:58:25.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T01:58:25.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T01:58:25.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T01:58:25.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T01:58:25.166 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T01:58:25.166 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T01:58:25.166 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T01:58:25.166 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T01:58:25.166 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T01:58:25.166 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T01:58:25.166 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T01:58:25.166 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T01:58:25.166 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T01:58:25.166 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T01:58:25.166 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T01:58:25.166 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T01:58:25.166 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T01:58:25.166 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T01:58:25.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T01:58:25.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T01:58:25.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T01:58:25.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T01:58:25.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T01:58:25.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T01:58:25.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T01:58:25.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T01:58:25.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T01:58:25.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T01:58:25.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T01:58:25.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T01:58:25.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T01:58:25.167 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T01:58:25.168 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T01:58:25.168 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T01:58:25.168 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T01:58:25.168 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T01:58:25.168 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.168 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.168 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T01:58:25.168 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T01:58:25.168 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T01:58:25.168 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.168 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T01:58:25.168 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T01:58:25.168 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T01:58:25.168 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T01:58:25.168 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T01:58:25.169 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T01:58:25.169 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T01:58:25.169 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T01:58:25.169 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T01:58:25.169 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T01:58:25.169 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T01:58:25.169 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T01:58:25.169 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T01:58:25.169 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T01:58:25.169 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T01:58:25.169 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.169 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.169 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T01:58:25.169 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T01:58:25.169 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T01:58:25.170 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T01:58:25.170 INFO:teuthology.orchestra.run.smithi019.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T01:58:25.170 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T01:58:25.170 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T01:58:25.170 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T01:58:25.170 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:58:25.170 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-05-09T01:58:25.498 DEBUG:teuthology.parallel:result is None 2024-05-09T01:58:25.553 DEBUG:teuthology.parallel:result is None 2024-05-09T01:58:25.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:58:25.780 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T01:58:25.803 INFO:teuthology.orchestra.run.smithi019.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T01:58:25.804 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T01:58:25.804 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T01:58:25.806 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:58:25.960 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T01:58:25.983 INFO:teuthology.orchestra.run.smithi059.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T01:58:25.984 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T01:58:25.984 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T01:58:25.986 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:58:26.208 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T01:58:26.233 INFO:teuthology.orchestra.run.smithi196.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T01:58:26.233 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T01:58:26.234 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T01:58:26.235 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T01:58:26.236 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:58:26.236 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T01:58:26.269 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:58:26.269 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T01:58:26.301 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:58:26.301 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T01:58:26.330 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T01:58:26.331 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:58:26.331 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T01:58:26.360 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T01:58:26.429 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:58:26.429 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T01:58:26.459 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T01:58:26.528 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:58:26.528 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T01:58:26.557 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T01:58:26.625 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T01:58:26.625 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:58:26.625 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T01:58:26.655 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T01:58:26.723 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:58:26.723 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T01:58:26.753 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T01:58:26.821 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:58:26.821 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T01:58:26.849 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T01:58:26.916 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T01:58:26.917 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:58:26.917 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T01:58:26.948 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T01:58:27.018 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T01:58:27.018 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T01:58:27.048 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T01:58:27.118 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T01:58:27.118 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T01:58:27.146 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T01:58:27.213 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-09T01:58:27.318 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-09T01:58:27.318 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:58:27.319 INFO:tasks.cephadm:Cluster fsid is a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T01:58:27.319 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-09T01:58:27.319 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.19', 'mon.b': '172.21.15.59', 'mon.c': '172.21.15.196'} 2024-05-09T01:58:27.319 INFO:tasks.cephadm:First mon is mon.a on smithi019 2024-05-09T01:58:27.319 INFO:tasks.cephadm:First mgr is x 2024-05-09T01:58:27.320 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-09T01:58:27.320 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname $(hostname -s) 2024-05-09T01:58:27.351 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname $(hostname -s) 2024-05-09T01:58:27.381 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-05-09T01:58:27.409 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-09T01:58:27.409 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:58:27.625 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79226/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3594-gc9da4049', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3594.gc9da4049'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-09 00:57:42.125322', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.183908-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79224/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3594-gc9da4049', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3594.gc9da4049'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 19:52:58.989794', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.183908-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-09T01:58:27.843 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240508.183908-debug, sha1 c9da4049f638c0f84bdd9b4e33c42e86046ac554 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:58:27.844 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm 2024-05-09T01:58:27.845 DEBUG:teuthology.orchestra.run.smithi019:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T01:58:28.272 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-r--r--. 1 ubuntu ubuntu 782864 May 9 01:58 /home/ubuntu/cephtest/cephadm 2024-05-09T01:58:28.272 DEBUG:teuthology.orchestra.run.smithi059:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T01:58:28.705 INFO:teuthology.orchestra.run.smithi059.stdout:-rw-r--r--. 1 ubuntu ubuntu 782864 May 9 01:58 /home/ubuntu/cephtest/cephadm 2024-05-09T01:58:28.705 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T01:58:29.125 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 782864 May 9 01:58 /home/ubuntu/cephtest/cephadm 2024-05-09T01:58:29.126 DEBUG:teuthology.orchestra.run.smithi019:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T01:58:29.146 DEBUG:teuthology.orchestra.run.smithi059:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T01:58:29.165 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T01:58:29.207 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 on all hosts... 2024-05-09T01:58:29.207 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T01:58:29.210 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T01:58:29.213 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T01:58:30.755 INFO:teuthology.orchestra.run.smithi059.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T01:58:30.756 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T01:58:30.839 INFO:teuthology.orchestra.run.smithi019.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T01:58:59.369 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-09T01:58:59.369 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T01:58:59.369 INFO:teuthology.orchestra.run.smithi059.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T01:58:59.369 INFO:teuthology.orchestra.run.smithi059.stdout: "repo_digests": [ 2024-05-09T01:58:59.369 INFO:teuthology.orchestra.run.smithi059.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T01:58:59.369 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-09T01:58:59.369 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-09T01:59:03.503 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T01:59:03.503 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T01:59:03.503 INFO:teuthology.orchestra.run.smithi019.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T01:59:03.503 INFO:teuthology.orchestra.run.smithi019.stdout: "repo_digests": [ 2024-05-09T01:59:03.503 INFO:teuthology.orchestra.run.smithi019.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T01:59:03.503 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T01:59:03.503 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T01:59:07.392 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-05-09T01:59:07.393 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T01:59:07.393 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T01:59:07.393 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-05-09T01:59:07.393 INFO:teuthology.orchestra.run.smithi196.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T01:59:07.393 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-05-09T01:59:07.393 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-05-09T01:59:07.418 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph 2024-05-09T01:59:07.453 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph 2024-05-09T01:59:07.487 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-05-09T01:59:07.514 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /etc/ceph 2024-05-09T01:59:07.544 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 777 /etc/ceph 2024-05-09T01:59:07.573 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-05-09T01:59:07.598 INFO:tasks.cephadm:Writing seed config... 2024-05-09T01:59:07.599 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-09T01:59:07.599 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-09T01:59:07.600 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-09T01:59:07.600 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-09T01:59:07.600 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-09T01:59:07.601 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-09T01:59:07.602 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-09T01:59:07.602 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-09T01:59:07.602 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-09T01:59:07.602 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-09T01:59:07.603 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-09T01:59:07.603 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-09T01:59:07.603 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-09T01:59:07.603 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-09T01:59:07.603 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-09T01:59:07.603 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-09T01:59:07.604 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-09T01:59:07.604 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-09T01:59:07.604 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-09T01:59:07.604 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-09T01:59:07.604 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-09T01:59:07.604 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-09T01:59:07.605 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-09T01:59:07.605 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-09T01:59:07.605 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-09T01:59:07.605 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-09T01:59:07.605 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-09T01:59:07.605 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-09T01:59:07.606 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-09T01:59:07.606 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-09T01:59:07.606 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-09T01:59:07.606 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-09T01:59:07.606 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-09T01:59:07.606 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-09T01:59:07.607 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-09T01:59:07.607 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-09T01:59:07.607 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-09T01:59:07.607 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-09T01:59:07.607 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-09T01:59:07.607 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-09T01:59:07.608 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-09T01:59:07.608 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-09T01:59:07.608 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-09T01:59:07.608 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-09T01:59:07.608 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-09T01:59:07.608 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-09T01:59:07.610 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T01:59:07.610 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-09T01:59:07.630 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a0611f3e-0da7-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-09T01:59:07.630 DEBUG:teuthology.orchestra.run.smithi019:mon.a> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.a.service 2024-05-09T01:59:07.674 DEBUG:teuthology.orchestra.run.smithi019:mgr.x> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mgr.x.service 2024-05-09T01:59:07.718 INFO:tasks.cephadm:Bootstrapping... 2024-05-09T01:59:07.718 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 -v bootstrap --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.19 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:59:07.920 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-05-09T01:59:07.920 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554', '-v', 'bootstrap', '--fsid', 'a0611f3e-0da7-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.19', '--skip-admin-label'] 2024-05-09T01:59:07.920 INFO:teuthology.orchestra.run.smithi019.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-09T01:59:07.921 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying podman|docker is present... 2024-05-09T01:59:07.948 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T01:59:07.948 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying lvm2 is present... 2024-05-09T01:59:07.948 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying time synchronization is in place... 2024-05-09T01:59:07.956 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T01:59:07.956 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T01:59:07.962 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T01:59:07.962 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-09T01:59:07.969 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-05-09T01:59:07.976 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-05-09T01:59:07.976 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-05-09T01:59:07.976 INFO:teuthology.orchestra.run.smithi019.stdout:Repeating the final host check... 2024-05-09T01:59:08.001 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T01:59:08.001 INFO:teuthology.orchestra.run.smithi019.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-09T01:59:08.001 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl is present 2024-05-09T01:59:08.001 INFO:teuthology.orchestra.run.smithi019.stdout:lvcreate is present 2024-05-09T01:59:08.008 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T01:59:08.008 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T01:59:08.014 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T01:59:08.014 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-09T01:59:08.021 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-05-09T01:59:08.028 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-05-09T01:59:08.028 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-05-09T01:59:08.028 INFO:teuthology.orchestra.run.smithi019.stdout:Host looks OK 2024-05-09T01:59:08.028 INFO:teuthology.orchestra.run.smithi019.stdout:Cluster fsid: a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T01:59:08.029 INFO:teuthology.orchestra.run.smithi019.stdout:Acquiring lock 139837608893888 on /run/cephadm/a0611f3e-0da7-11ef-bc97-c7b262605968.lock 2024-05-09T01:59:08.029 INFO:teuthology.orchestra.run.smithi019.stdout:Lock 139837608893888 acquired on /run/cephadm/a0611f3e-0da7-11ef-bc97-c7b262605968.lock 2024-05-09T01:59:08.029 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 3300 ... 2024-05-09T01:59:08.029 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 6789 ... 2024-05-09T01:59:08.029 INFO:teuthology.orchestra.run.smithi019.stdout:Base mon IP(s) is [172.21.15.19:3300, 172.21.15.19:6789], mon addrv is [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-05-09T01:59:08.032 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.19 metric 100 2024-05-09T01:59:08.032 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.19 metric 100 2024-05-09T01:59:08.035 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-09T01:59:08.035 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-09T01:59:08.035 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-09T01:59:08.037 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-09T01:59:08.038 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-09T01:59:08.038 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T01:59:08.038 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-09T01:59:08.038 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-05-09T01:59:08.038 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T01:59:08.038 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-05-09T01:59:08.039 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-05-09T01:59:08.039 INFO:teuthology.orchestra.run.smithi019.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-09T01:59:08.039 INFO:teuthology.orchestra.run.smithi019.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-09T01:59:08.040 INFO:teuthology.orchestra.run.smithi019.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T01:59:08.554 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 2024-05-09T01:59:08.555 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T01:59:08.555 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Getting image source signatures 2024-05-09T01:59:08.555 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:0411e0fd7a5d45f8ab31b23cec89f97c70494de39461f644c7cdc77de3b9cbca 2024-05-09T01:59:08.555 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-09T01:59:08.555 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying config sha256:db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 2024-05-09T01:59:08.555 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-09T01:59:09.559 INFO:teuthology.orchestra.run.smithi019.stdout:ceph: stdout ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T01:59:09.559 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph version: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T01:59:09.559 INFO:teuthology.orchestra.run.smithi019.stdout:Extracting ceph user uid/gid from container image... 2024-05-09T01:59:10.441 INFO:teuthology.orchestra.run.smithi019.stdout:stat: stdout 167 167 2024-05-09T01:59:10.442 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial keys... 2024-05-09T01:59:11.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQDuLTxmIX0QMBAAje7A+24910gUAUI/zoshnw== 2024-05-09T01:59:12.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQDvLTxmNM3OJhAA5m9QJusrqpp8amcjey+AmQ== 2024-05-09T01:59:13.037 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQDwLTxmr3A4HhAAL2ljh5/09TXQ51AHcCNlZw== 2024-05-09T01:59:13.038 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial monmap... 2024-05-09T01:59:15.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T01:59:15.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-09T01:59:15.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T01:59:15.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T01:59:15.637 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool for a [v2:172.21.15.19:3300,v1:172.21.15.19:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T01:59:15.637 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = quincy 2024-05-09T01:59:15.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: set fsid to a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T01:59:15.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T01:59:15.637 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T01:59:15.637 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mon... 2024-05-09T01:59:25.267 INFO:teuthology.orchestra.run.smithi019.stdout:create mon.a on 2024-05-09T01:59:25.763 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-09T01:59:26.044 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-09T01:59:26.601 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a0611f3e-0da7-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-a0611f3e-0da7-11ef-bc97-c7b262605968.target. 2024-05-09T01:59:26.601 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a0611f3e-0da7-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-a0611f3e-0da7-11ef-bc97-c7b262605968.target. 2024-05-09T01:59:26.893 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.a 2024-05-09T01:59:26.893 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-09T01:59:27.068 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a0611f3e-0da7-11ef-bc97-c7b262605968.target.wants/ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@.service. 2024-05-09T01:59:27.626 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T01:59:27.626 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-09T01:59:27.633 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T01:59:27.633 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-09T01:59:27.633 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-09T01:59:27.633 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T01:59:27.633 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon to start... 2024-05-09T01:59:27.634 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon... 2024-05-09T01:59:27.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:27 smithi019 systemd[1]: Started Ceph mon.a for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T01:59:28.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:28 smithi019 ceph-mon[29768]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T01:59:28.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:28 smithi019 ceph-mon[29768]: monmap epoch 1 2024-05-09T01:59:28.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:28 smithi019 ceph-mon[29768]: fsid a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T01:59:28.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:28 smithi019 ceph-mon[29768]: last_changed 2024-05-09T01:59:13.406931+0000 2024-05-09T01:59:28.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:28 smithi019 ceph-mon[29768]: created 2024-05-09T01:59:13.406931+0000 2024-05-09T01:59:28.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:28 smithi019 ceph-mon[29768]: min_mon_release 19 (squid) 2024-05-09T01:59:28.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:28 smithi019 ceph-mon[29768]: election_strategy: 1 2024-05-09T01:59:28.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:28 smithi019 ceph-mon[29768]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T01:59:28.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:28 smithi019 ceph-mon[29768]: fsmap 2024-05-09T01:59:28.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:28 smithi019 ceph-mon[29768]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T01:59:28.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:28 smithi019 ceph-mon[29768]: mgrmap e1: no daemons active 2024-05-09T01:59:28.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:28 smithi019 ceph-mon[29768]: from='client.? 172.21.15.19:0/2452342443' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-09T01:59:28.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout cluster: 2024-05-09T01:59:28.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout id: a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T01:59:28.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-09T01:59:28.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:59:28.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout services: 2024-05-09T01:59:28.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.552964s) 2024-05-09T01:59:28.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-09T01:59:28.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-09T01:59:28.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:59:28.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout data: 2024-05-09T01:59:28.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-09T01:59:28.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-09T01:59:28.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-09T01:59:28.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pgs: 2024-05-09T01:59:28.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:59:28.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.060+0000 7fd82c268700 1 Processor -- start 2024-05-09T01:59:28.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.060+0000 7fd82c268700 1 -- start start 2024-05-09T01:59:28.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.060+0000 7fd82c268700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82410a620 0x7fd82410ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:28.696 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.060+0000 7fd82c268700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd82407d6b0 con 0x7fd82410a620 2024-05-09T01:59:28.696 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.061+0000 7fd82a004700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82410a620 0x7fd82410ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:28.696 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.061+0000 7fd82a004700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82410a620 0x7fd82410ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44082/0 (socket says 172.21.15.19:44082) 2024-05-09T01:59:28.696 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.061+0000 7fd82a004700 1 -- 172.21.15.19:0/2266097113 learned_addr learned my addr 172.21.15.19:0/2266097113 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:28.696 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.061+0000 7fd82a004700 1 -- 172.21.15.19:0/2266097113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82410cf80 con 0x7fd82410a620 2024-05-09T01:59:28.696 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.061+0000 7fd82a004700 1 --2- 172.21.15.19:0/2266097113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82410a620 0x7fd82410ca40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd8200098a0 tx=0x7fd820009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3129db3d214a4c5c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:28.696 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.061+0000 7fd829002700 1 -- 172.21.15.19:0/2266097113 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd820004160 con 0x7fd82410a620 2024-05-09T01:59:28.696 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.061+0000 7fd829002700 1 -- 172.21.15.19:0/2266097113 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd820004300 con 0x7fd82410a620 2024-05-09T01:59:28.696 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.061+0000 7fd829002700 1 -- 172.21.15.19:0/2266097113 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8200036a0 con 0x7fd82410a620 2024-05-09T01:59:28.697 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.062+0000 7fd82c268700 1 -- 172.21.15.19:0/2266097113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82410a620 msgr2=0x7fd82410ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:28.697 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.062+0000 7fd82c268700 1 --2- 172.21.15.19:0/2266097113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82410a620 0x7fd82410ca40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd8200098a0 tx=0x7fd820009bb0 comp rx=0 tx=0).stop 2024-05-09T01:59:28.697 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.062+0000 7fd82c268700 1 -- 172.21.15.19:0/2266097113 shutdown_connections 2024-05-09T01:59:28.697 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.062+0000 7fd82c268700 1 --2- 172.21.15.19:0/2266097113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82410a620 0x7fd82410ca40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:28.697 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.062+0000 7fd82c268700 1 -- 172.21.15.19:0/2266097113 >> 172.21.15.19:0/2266097113 conn(0x7fd8241040e0 msgr2=0x7fd824106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:28.697 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.062+0000 7fd82c268700 1 -- 172.21.15.19:0/2266097113 shutdown_connections 2024-05-09T01:59:28.697 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.062+0000 7fd82c268700 1 -- 172.21.15.19:0/2266097113 wait complete. 2024-05-09T01:59:28.697 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.063+0000 7fd82c268700 1 Processor -- start 2024-05-09T01:59:28.697 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.063+0000 7fd82c268700 1 -- start start 2024-05-09T01:59:28.697 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.063+0000 7fd82c268700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82410a620 0x7fd82419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:28.697 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.063+0000 7fd82c268700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd82410d390 con 0x7fd82410a620 2024-05-09T01:59:28.697 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.063+0000 7fd82a004700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82410a620 0x7fd82419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:28.697 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.063+0000 7fd82a004700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82410a620 0x7fd82419d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44094/0 (socket says 172.21.15.19:44094) 2024-05-09T01:59:28.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.063+0000 7fd82a004700 1 -- 172.21.15.19:0/2452342443 learned_addr learned my addr 172.21.15.19:0/2452342443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:28.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.064+0000 7fd82a004700 1 -- 172.21.15.19:0/2452342443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82419dac0 con 0x7fd82410a620 2024-05-09T01:59:28.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.064+0000 7fd82a004700 1 --2- 172.21.15.19:0/2452342443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82410a620 0x7fd82419d580 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd820000c00 tx=0x7fd820004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:28.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.064+0000 7fd81b7fe700 1 -- 172.21.15.19:0/2452342443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd820003db0 con 0x7fd82410a620 2024-05-09T01:59:28.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.064+0000 7fd81b7fe700 1 -- 172.21.15.19:0/2452342443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd820011630 con 0x7fd82410a620 2024-05-09T01:59:28.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.064+0000 7fd82c268700 1 -- 172.21.15.19:0/2452342443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82419dd50 con 0x7fd82410a620 2024-05-09T01:59:28.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.064+0000 7fd81b7fe700 1 -- 172.21.15.19:0/2452342443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd820011950 con 0x7fd82410a620 2024-05-09T01:59:28.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.064+0000 7fd82c268700 1 -- 172.21.15.19:0/2452342443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8241a0960 con 0x7fd82410a620 2024-05-09T01:59:28.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.064+0000 7fd81b7fe700 1 -- 172.21.15.19:0/2452342443 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd82001a460 con 0x7fd82410a620 2024-05-09T01:59:28.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.064+0000 7fd81b7fe700 1 -- 172.21.15.19:0/2452342443 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd82001b780 con 0x7fd82410a620 2024-05-09T01:59:28.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.065+0000 7fd82c268700 1 -- 172.21.15.19:0/2452342443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd824064e20 con 0x7fd82410a620 2024-05-09T01:59:28.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.068+0000 7fd81b7fe700 1 -- 172.21.15.19:0/2452342443 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd82001b980 con 0x7fd82410a620 2024-05-09T01:59:28.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.142+0000 7fd82c268700 1 -- 172.21.15.19:0/2452342443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fd82402e8c0 con 0x7fd82410a620 2024-05-09T01:59:28.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.142+0000 7fd81b7fe700 1 -- 172.21.15.19:0/2452342443 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fd82001bb60 con 0x7fd82410a620 2024-05-09T01:59:28.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.143+0000 7fd82c268700 1 -- 172.21.15.19:0/2452342443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82410a620 msgr2=0x7fd82419d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:28.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.143+0000 7fd82c268700 1 --2- 172.21.15.19:0/2452342443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82410a620 0x7fd82419d580 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd820000c00 tx=0x7fd820004750 comp rx=0 tx=0).stop 2024-05-09T01:59:28.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.143+0000 7fd82c268700 1 -- 172.21.15.19:0/2452342443 shutdown_connections 2024-05-09T01:59:28.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.143+0000 7fd82c268700 1 --2- 172.21.15.19:0/2452342443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd82410a620 0x7fd82419d580 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:28.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.143+0000 7fd82c268700 1 -- 172.21.15.19:0/2452342443 >> 172.21.15.19:0/2452342443 conn(0x7fd8241040e0 msgr2=0x7fd824105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:28.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.143+0000 7fd82c268700 1 -- 172.21.15.19:0/2452342443 shutdown_connections 2024-05-09T01:59:28.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:28.143+0000 7fd82c268700 1 -- 172.21.15.19:0/2452342443 wait complete. 2024-05-09T01:59:28.699 INFO:teuthology.orchestra.run.smithi019.stdout:mon is available 2024-05-09T01:59:28.699 INFO:teuthology.orchestra.run.smithi019.stdout:Assimilating anything we can from ceph.conf... 2024-05-09T01:59:29.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:29 smithi019 ceph-mon[29768]: from='client.? 172.21.15.19:0/2640073247' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T01:59:29.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:29 smithi019 ceph-mon[29768]: from='client.? 172.21.15.19:0/2640073247' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-09T01:59:29.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:59:29.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T01:59:29.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T01:59:29.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-09T01:59:29.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-05-09T01:59:29.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T01:59:29.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T01:59:29.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T01:59:29.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T01:59:29.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:59:29.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T01:59:29.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T01:59:29.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:59:29.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T01:59:29.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T01:59:29.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T01:59:29.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T01:59:29.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T01:59:29.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T01:59:29.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.137+0000 7f7ff3f6a700 1 Processor -- start 2024-05-09T01:59:29.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.137+0000 7f7ff3f6a700 1 -- start start 2024-05-09T01:59:29.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.137+0000 7f7ff3f6a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108a00 0x7f7fec108e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:29.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.137+0000 7f7ff3f6a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec109400 con 0x7f7fec108a00 2024-05-09T01:59:29.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.138+0000 7f7ff1d06700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108a00 0x7f7fec108e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:29.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.138+0000 7f7ff1d06700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108a00 0x7f7fec108e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44098/0 (socket says 172.21.15.19:44098) 2024-05-09T01:59:29.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.138+0000 7f7ff1d06700 1 -- 172.21.15.19:0/2780869722 learned_addr learned my addr 172.21.15.19:0/2780869722 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:29.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.138+0000 7f7ff1d06700 1 -- 172.21.15.19:0/2780869722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec109bd0 con 0x7f7fec108a00 2024-05-09T01:59:29.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.138+0000 7f7ff1d06700 1 --2- 172.21.15.19:0/2780869722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108a00 0x7f7fec108e10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80098a0 tx=0x7f7fe8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ca66fa5d35a7a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:29.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.138+0000 7f7ff0d04700 1 -- 172.21.15.19:0/2780869722 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fe8004160 con 0x7f7fec108a00 2024-05-09T01:59:29.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.138+0000 7f7ff0d04700 1 -- 172.21.15.19:0/2780869722 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe8004300 con 0x7f7fec108a00 2024-05-09T01:59:29.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.138+0000 7f7ff0d04700 1 -- 172.21.15.19:0/2780869722 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fe80036a0 con 0x7f7fec108a00 2024-05-09T01:59:29.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.139+0000 7f7ff3f6a700 1 -- 172.21.15.19:0/2780869722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108a00 msgr2=0x7f7fec108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:29.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.139+0000 7f7ff3f6a700 1 --2- 172.21.15.19:0/2780869722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108a00 0x7f7fec108e10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80098a0 tx=0x7f7fe8009bb0 comp rx=0 tx=0).stop 2024-05-09T01:59:29.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.139+0000 7f7ff3f6a700 1 -- 172.21.15.19:0/2780869722 shutdown_connections 2024-05-09T01:59:29.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.139+0000 7f7ff3f6a700 1 --2- 172.21.15.19:0/2780869722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108a00 0x7f7fec108e10 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:29.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.139+0000 7f7ff3f6a700 1 -- 172.21.15.19:0/2780869722 >> 172.21.15.19:0/2780869722 conn(0x7f7fec1040e0 msgr2=0x7f7fec106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:29.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.139+0000 7f7ff3f6a700 1 -- 172.21.15.19:0/2780869722 shutdown_connections 2024-05-09T01:59:29.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.139+0000 7f7ff3f6a700 1 -- 172.21.15.19:0/2780869722 wait complete. 2024-05-09T01:59:29.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.140+0000 7f7ff3f6a700 1 Processor -- start 2024-05-09T01:59:29.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.140+0000 7f7ff3f6a700 1 -- start start 2024-05-09T01:59:29.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.140+0000 7f7ff3f6a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108a00 0x7f7fec19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:29.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.140+0000 7f7ff3f6a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec10a100 con 0x7f7fec108a00 2024-05-09T01:59:29.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.140+0000 7f7ff1d06700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108a00 0x7f7fec19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:29.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.140+0000 7f7ff1d06700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108a00 0x7f7fec19d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44106/0 (socket says 172.21.15.19:44106) 2024-05-09T01:59:29.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.140+0000 7f7ff1d06700 1 -- 172.21.15.19:0/2640073247 learned_addr learned my addr 172.21.15.19:0/2640073247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:29.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.141+0000 7f7ff1d06700 1 -- 172.21.15.19:0/2640073247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec19daf0 con 0x7f7fec108a00 2024-05-09T01:59:29.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.141+0000 7f7ff1d06700 1 --2- 172.21.15.19:0/2640073247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108a00 0x7f7fec19d5b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8000c00 tx=0x7f7fe8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:29.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.141+0000 7f7fe2ffd700 1 -- 172.21.15.19:0/2640073247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fe8003db0 con 0x7f7fec108a00 2024-05-09T01:59:29.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.141+0000 7f7fe2ffd700 1 -- 172.21.15.19:0/2640073247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe8011630 con 0x7f7fec108a00 2024-05-09T01:59:29.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.141+0000 7f7ff3f6a700 1 -- 172.21.15.19:0/2640073247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fec19dd80 con 0x7f7fec108a00 2024-05-09T01:59:29.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.141+0000 7f7fe2ffd700 1 -- 172.21.15.19:0/2640073247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fe8011950 con 0x7f7fec108a00 2024-05-09T01:59:29.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.141+0000 7f7ff3f6a700 1 -- 172.21.15.19:0/2640073247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fec1a0990 con 0x7f7fec108a00 2024-05-09T01:59:29.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.141+0000 7f7fe2ffd700 1 -- 172.21.15.19:0/2640073247 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7fe801a460 con 0x7f7fec108a00 2024-05-09T01:59:29.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.141+0000 7f7fe2ffd700 1 -- 172.21.15.19:0/2640073247 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7fe801b780 con 0x7f7fec108a00 2024-05-09T01:59:29.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.142+0000 7f7ff3f6a700 1 -- 172.21.15.19:0/2640073247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fec064e20 con 0x7f7fec108a00 2024-05-09T01:59:29.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.144+0000 7f7fe2ffd700 1 -- 172.21.15.19:0/2640073247 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7fe8031020 con 0x7f7fec108a00 2024-05-09T01:59:29.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.216+0000 7f7ff3f6a700 1 -- 172.21.15.19:0/2640073247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f7fec02dc40 con 0x7f7fec108a00 2024-05-09T01:59:29.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.220+0000 7f7fe2ffd700 1 -- 172.21.15.19:0/2640073247 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f7fe801b980 con 0x7f7fec108a00 2024-05-09T01:59:29.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.221+0000 7f7ff3f6a700 1 -- 172.21.15.19:0/2640073247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108a00 msgr2=0x7f7fec19d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:29.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.221+0000 7f7ff3f6a700 1 --2- 172.21.15.19:0/2640073247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108a00 0x7f7fec19d5b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8000c00 tx=0x7f7fe8004750 comp rx=0 tx=0).stop 2024-05-09T01:59:29.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.221+0000 7f7fe2ffd700 1 -- 172.21.15.19:0/2640073247 <== mon.0 v2:172.21.15.19:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7fe801a840 con 0x7f7fec108a00 2024-05-09T01:59:29.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.221+0000 7f7ff3f6a700 1 -- 172.21.15.19:0/2640073247 shutdown_connections 2024-05-09T01:59:29.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.221+0000 7f7ff3f6a700 1 --2- 172.21.15.19:0/2640073247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fec108a00 0x7f7fec19d5b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:29.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.221+0000 7f7ff3f6a700 1 -- 172.21.15.19:0/2640073247 >> 172.21.15.19:0/2640073247 conn(0x7f7fec1040e0 msgr2=0x7f7fec193d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:29.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.221+0000 7f7ff3f6a700 1 -- 172.21.15.19:0/2640073247 shutdown_connections 2024-05-09T01:59:29.889 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:29.221+0000 7f7ff3f6a700 1 -- 172.21.15.19:0/2640073247 wait complete. 2024-05-09T01:59:29.890 INFO:teuthology.orchestra.run.smithi019.stdout:Generating new minimal ceph.conf... 2024-05-09T01:59:30.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:30 smithi019 ceph-mon[29768]: from='client.? 172.21.15.19:0/2587694323' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"}]: dispatch 2024-05-09T01:59:30.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.342+0000 7f8958916700 1 Processor -- start 2024-05-09T01:59:30.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.342+0000 7f8958916700 1 -- start start 2024-05-09T01:59:30.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.342+0000 7f8958916700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8954109b00 0x7f8954109f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:30.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.342+0000 7f8958916700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f895410a500 con 0x7f8954109b00 2024-05-09T01:59:30.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.343+0000 7f895259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8954109b00 0x7f8954109f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:30.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.343+0000 7f895259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8954109b00 0x7f8954109f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44110/0 (socket says 172.21.15.19:44110) 2024-05-09T01:59:30.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.343+0000 7f895259c700 1 -- 172.21.15.19:0/1580417139 learned_addr learned my addr 172.21.15.19:0/1580417139 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:30.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.343+0000 7f895259c700 1 -- 172.21.15.19:0/1580417139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895410acd0 con 0x7f8954109b00 2024-05-09T01:59:30.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.343+0000 7f895259c700 1 --2- 172.21.15.19:0/1580417139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8954109b00 0x7f8954109f10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f89440098a0 tx=0x7f8944009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a278c710c783f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:30.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.343+0000 7f895159a700 1 -- 172.21.15.19:0/1580417139 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8944004160 con 0x7f8954109b00 2024-05-09T01:59:30.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.343+0000 7f895159a700 1 -- 172.21.15.19:0/1580417139 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8944004300 con 0x7f8954109b00 2024-05-09T01:59:30.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.343+0000 7f895159a700 1 -- 172.21.15.19:0/1580417139 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8944011af0 con 0x7f8954109b00 2024-05-09T01:59:30.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.344+0000 7f8958916700 1 -- 172.21.15.19:0/1580417139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8954109b00 msgr2=0x7f8954109f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:30.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.344+0000 7f8958916700 1 --2- 172.21.15.19:0/1580417139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8954109b00 0x7f8954109f10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f89440098a0 tx=0x7f8944009bb0 comp rx=0 tx=0).stop 2024-05-09T01:59:30.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.344+0000 7f8958916700 1 -- 172.21.15.19:0/1580417139 shutdown_connections 2024-05-09T01:59:30.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.344+0000 7f8958916700 1 --2- 172.21.15.19:0/1580417139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8954109b00 0x7f8954109f10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:30.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.344+0000 7f8958916700 1 -- 172.21.15.19:0/1580417139 >> 172.21.15.19:0/1580417139 conn(0x7f89541051e0 msgr2=0x7f8954107630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:30.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.344+0000 7f8958916700 1 -- 172.21.15.19:0/1580417139 shutdown_connections 2024-05-09T01:59:30.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.344+0000 7f8958916700 1 -- 172.21.15.19:0/1580417139 wait complete. 2024-05-09T01:59:30.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.345+0000 7f8958916700 1 Processor -- start 2024-05-09T01:59:30.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.345+0000 7f8958916700 1 -- start start 2024-05-09T01:59:30.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.345+0000 7f8958916700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8954109b00 0x7f895407f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:30.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.345+0000 7f8958916700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f895410e080 con 0x7f8954109b00 2024-05-09T01:59:30.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.345+0000 7f895259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8954109b00 0x7f895407f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:30.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.345+0000 7f895259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8954109b00 0x7f895407f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44114/0 (socket says 172.21.15.19:44114) 2024-05-09T01:59:30.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.345+0000 7f895259c700 1 -- 172.21.15.19:0/2587694323 learned_addr learned my addr 172.21.15.19:0/2587694323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:30.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.346+0000 7f895259c700 1 -- 172.21.15.19:0/2587694323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895407f820 con 0x7f8954109b00 2024-05-09T01:59:30.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.346+0000 7f895259c700 1 --2- 172.21.15.19:0/2587694323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8954109b00 0x7f895407f2e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f89440090e0 tx=0x7f8944011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.346+0000 7f89437fe700 1 -- 172.21.15.19:0/2587694323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89440039e0 con 0x7f8954109b00 2024-05-09T01:59:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.346+0000 7f89437fe700 1 -- 172.21.15.19:0/2587694323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8944003b80 con 0x7f8954109b00 2024-05-09T01:59:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.346+0000 7f8958916700 1 -- 172.21.15.19:0/2587694323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895407da70 con 0x7f8954109b00 2024-05-09T01:59:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.346+0000 7f89437fe700 1 -- 172.21.15.19:0/2587694323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8944024690 con 0x7f8954109b00 2024-05-09T01:59:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.346+0000 7f8958916700 1 -- 172.21.15.19:0/2587694323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895407df50 con 0x7f8954109b00 2024-05-09T01:59:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.347+0000 7f89437fe700 1 -- 172.21.15.19:0/2587694323 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8944022020 con 0x7f8954109b00 2024-05-09T01:59:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.347+0000 7f89437fe700 1 -- 172.21.15.19:0/2587694323 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f89440193c0 con 0x7f8954109b00 2024-05-09T01:59:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.347+0000 7f8958916700 1 -- 172.21.15.19:0/2587694323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8954064e20 con 0x7f8954109b00 2024-05-09T01:59:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.350+0000 7f89437fe700 1 -- 172.21.15.19:0/2587694323 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f894401a9c0 con 0x7f8954109b00 2024-05-09T01:59:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.420+0000 7f8958916700 1 -- 172.21.15.19:0/2587694323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"} v 0) -- 0x7f895407e1d0 con 0x7f8954109b00 2024-05-09T01:59:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.421+0000 7f89437fe700 1 -- 172.21.15.19:0/2587694323 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"}]=0 v2) ==== 126+0+175 (secure 0 0 0) 0x7f894401a430 con 0x7f8954109b00 2024-05-09T01:59:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.421+0000 7f8958916700 1 -- 172.21.15.19:0/2587694323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8954109b00 msgr2=0x7f895407f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.421+0000 7f8958916700 1 --2- 172.21.15.19:0/2587694323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8954109b00 0x7f895407f2e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f89440090e0 tx=0x7f8944011940 comp rx=0 tx=0).stop 2024-05-09T01:59:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.421+0000 7f8958916700 1 -- 172.21.15.19:0/2587694323 shutdown_connections 2024-05-09T01:59:30.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.421+0000 7f8958916700 1 --2- 172.21.15.19:0/2587694323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8954109b00 0x7f895407f2e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:30.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.421+0000 7f8958916700 1 -- 172.21.15.19:0/2587694323 >> 172.21.15.19:0/2587694323 conn(0x7f89541051e0 msgr2=0x7f8954106a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:30.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.422+0000 7f8958916700 1 -- 172.21.15.19:0/2587694323 shutdown_connections 2024-05-09T01:59:30.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:30.422+0000 7f8958916700 1 -- 172.21.15.19:0/2587694323 wait complete. 2024-05-09T01:59:30.966 INFO:teuthology.orchestra.run.smithi019.stdout:Restarting the monitor... 2024-05-09T01:59:31.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:30 smithi019 systemd[1]: Stopping Ceph mon.a for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T01:59:31.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:31 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a[29764]: 2024-05-09T01:59:31.241+0000 7f46a1bea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:59:31.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:31 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a[29764]: 2024-05-09T01:59:31.241+0000 7f46a1bea700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-09T01:59:31.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:31 smithi019 podman[30015]: 2024-05-09 01:59:31.477788854 +0000 UTC m=+0.346170101 container died 1a2e821721d85bb43c23335c5591623534c95189318e14f3cd551e27472357aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-09T01:59:32.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:31 smithi019 podman[30015]: 2024-05-09 01:59:31.774814234 +0000 UTC m=+0.643195473 container cleanup 1a2e821721d85bb43c23335c5591623534c95189318e14f3cd551e27472357aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-09T01:59:32.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:31 smithi019 bash[30015]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a 2024-05-09T01:59:33.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:33 smithi019 podman[30029]: 2024-05-09 01:59:33.524843297 +0000 UTC m=+2.050775897 container remove 1a2e821721d85bb43c23335c5591623534c95189318e14f3cd551e27472357aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-09T01:59:33.900 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:33 smithi019 systemd[1]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-09T01:59:33.900 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:33 smithi019 systemd[1]: Stopped Ceph mon.a for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T01:59:33.901 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:33 smithi019 systemd[1]: Starting Ceph mon.a for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T01:59:33.901 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:33 smithi019 podman[30131]: 2024-05-09 01:59:33.79827974 +0000 UTC m=+0.019239837 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:59:33.901 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:33 smithi019 podman[30131]: 2024-05-09 01:59:33.900689332 +0000 UTC m=+0.121649430 container create 4d8bf39103f3413dd54b9f9d782cc5312527d3e29548bf4a2826a12a87e3bcc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-05-09T01:59:34.151 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 podman[30131]: 2024-05-09 01:59:34.102894327 +0000 UTC m=+0.323854432 container init 4d8bf39103f3413dd54b9f9d782cc5312527d3e29548bf4a2826a12a87e3bcc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:59:34.152 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 podman[30131]: 2024-05-09 01:59:34.106157925 +0000 UTC m=+0.327118025 container start 4d8bf39103f3413dd54b9f9d782cc5312527d3e29548bf4a2826a12a87e3bcc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:59:34.152 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T01:59:34.152 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev), process ceph-mon, pid 2 2024-05-09T01:59:34.152 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: pidfile_write: ignore empty --pid-file 2024-05-09T01:59:34.152 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: load: jerasure load: lrc 2024-05-09T01:59:34.152 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: RocksDB version: 7.9.2 2024-05-09T01:59:34.152 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Git sha 0 2024-05-09T01:59:34.152 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Compile date 2024-05-08 19:33:14 2024-05-09T01:59:34.152 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: DB SUMMARY 2024-05-09T01:59:34.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: DB Session ID: P02F3WRF07M0QX0IZQKS 2024-05-09T01:59:34.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: CURRENT file: CURRENT 2024-05-09T01:59:34.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T01:59:34.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-09T01:59:34.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-09T01:59:34.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104712 ; 2024-05-09T01:59:34.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.error_if_exists: 0 2024-05-09T01:59:34.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.create_if_missing: 0 2024-05-09T01:59:34.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.paranoid_checks: 1 2024-05-09T01:59:34.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T01:59:34.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T01:59:34.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T01:59:34.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.env: 0x55b2a8534500 2024-05-09T01:59:34.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T01:59:34.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.info_log: 0x55b2a9f06020 2024-05-09T01:59:34.154 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T01:59:34.154 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.statistics: (nil) 2024-05-09T01:59:34.154 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.use_fsync: 0 2024-05-09T01:59:34.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_log_file_size: 0 2024-05-09T01:59:34.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T01:59:34.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T01:59:34.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T01:59:34.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T01:59:34.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.allow_fallocate: 1 2024-05-09T01:59:34.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T01:59:34.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T01:59:34.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.use_direct_reads: 0 2024-05-09T01:59:34.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T01:59:34.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T01:59:34.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.db_log_dir: 2024-05-09T01:59:34.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.wal_dir: 2024-05-09T01:59:34.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T01:59:34.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T01:59:34.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T01:59:34.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T01:59:34.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T01:59:34.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T01:59:34.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T01:59:34.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T01:59:34.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.write_buffer_manager: 0x55b2a9dd08c0 2024-05-09T01:59:34.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T01:59:34.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T01:59:34.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T01:59:34.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T01:59:34.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T01:59:34.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T01:59:34.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T01:59:34.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T01:59:34.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.unordered_write: 0 2024-05-09T01:59:34.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T01:59:34.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T01:59:34.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T01:59:34.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T01:59:34.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.row_cache: None 2024-05-09T01:59:34.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.wal_filter: None 2024-05-09T01:59:34.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T01:59:34.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T01:59:34.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.two_write_queues: 0 2024-05-09T01:59:34.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T01:59:34.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.wal_compression: 0 2024-05-09T01:59:34.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.atomic_flush: 0 2024-05-09T01:59:34.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T01:59:34.158 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T01:59:34.158 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T01:59:34.158 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.log_readahead_size: 0 2024-05-09T01:59:34.158 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T01:59:34.158 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T01:59:34.158 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T01:59:34.158 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T01:59:34.158 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T01:59:34.158 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T01:59:34.158 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T01:59:34.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_background_jobs: 2 2024-05-09T01:59:34.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_background_compactions: -1 2024-05-09T01:59:34.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_subcompactions: 1 2024-05-09T01:59:34.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T01:59:34.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T01:59:34.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T01:59:34.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T01:59:34.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T01:59:34.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T01:59:34.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T01:59:34.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T01:59:34.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_open_files: -1 2024-05-09T01:59:34.159 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T01:59:34.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T01:59:34.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T01:59:34.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T01:59:34.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_background_flushes: -1 2024-05-09T01:59:34.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Compression algorithms supported: 2024-05-09T01:59:34.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T01:59:34.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: kZSTD supported: 0 2024-05-09T01:59:34.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: kXpressCompression supported: 0 2024-05-09T01:59:34.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T01:59:34.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: kZlibCompression supported: 1 2024-05-09T01:59:34.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: kSnappyCompression supported: 1 2024-05-09T01:59:34.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: kLZ4Compression supported: 1 2024-05-09T01:59:34.160 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: kBZip2Compression supported: 0 2024-05-09T01:59:34.161 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T01:59:34.161 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T01:59:34.161 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-09T01:59:34.161 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T01:59:34.161 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T01:59:34.161 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.merge_operator: 2024-05-09T01:59:34.161 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compaction_filter: None 2024-05-09T01:59:34.161 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T01:59:34.161 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T01:59:34.161 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T01:59:34.161 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T01:59:34.161 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b2a9f061e0) 2024-05-09T01:59:34.161 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks: 1 2024-05-09T01:59:34.161 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T01:59:34.162 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T01:59:34.162 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_top_level_index_and_filter: 1 2024-05-09T01:59:34.162 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_type: 0 2024-05-09T01:59:34.162 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_index_type: 0 2024-05-09T01:59:34.162 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_shortening: 1 2024-05-09T01:59:34.162 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T01:59:34.162 INFO:journalctl@ceph.mon.a.smithi019.stdout: checksum: 4 2024-05-09T01:59:34.162 INFO:journalctl@ceph.mon.a.smithi019.stdout: no_block_cache: 0 2024-05-09T01:59:34.162 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache: 0x55b2a9e031f0 2024-05-09T01:59:34.162 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_name: BinnedLRUCache 2024-05-09T01:59:34.162 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_options: 2024-05-09T01:59:34.163 INFO:journalctl@ceph.mon.a.smithi019.stdout: capacity : 536870912 2024-05-09T01:59:34.163 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_shard_bits : 4 2024-05-09T01:59:34.163 INFO:journalctl@ceph.mon.a.smithi019.stdout: strict_capacity_limit : 0 2024-05-09T01:59:34.163 INFO:journalctl@ceph.mon.a.smithi019.stdout: high_pri_pool_ratio: 0.000 2024-05-09T01:59:34.163 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_compressed: (nil) 2024-05-09T01:59:34.163 INFO:journalctl@ceph.mon.a.smithi019.stdout: persistent_cache: (nil) 2024-05-09T01:59:34.163 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size: 4096 2024-05-09T01:59:34.163 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size_deviation: 10 2024-05-09T01:59:34.163 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_restart_interval: 16 2024-05-09T01:59:34.163 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_block_restart_interval: 1 2024-05-09T01:59:34.163 INFO:journalctl@ceph.mon.a.smithi019.stdout: metadata_block_size: 4096 2024-05-09T01:59:34.163 INFO:journalctl@ceph.mon.a.smithi019.stdout: partition_filters: 0 2024-05-09T01:59:34.163 INFO:journalctl@ceph.mon.a.smithi019.stdout: use_delta_encoding: 1 2024-05-09T01:59:34.163 INFO:journalctl@ceph.mon.a.smithi019.stdout: filter_policy: bloomfilter 2024-05-09T01:59:34.163 INFO:journalctl@ceph.mon.a.smithi019.stdout: whole_key_filtering: 1 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout: verify_compression: 0 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout: read_amp_bytes_per_bit: 0 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout: format_version: 5 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_index_compression: 1 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_align: 0 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout: max_auto_readahead_size: 262144 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout: prepopulate_block_cache: 0 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout: initial_auto_readahead_size: 8192 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compression: NoCompression 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T01:59:34.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.num_levels: 7 2024-05-09T01:59:34.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T01:59:34.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T01:59:34.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T01:59:34.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T01:59:34.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T01:59:34.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T01:59:34.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T01:59:34.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T01:59:34.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T01:59:34.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T01:59:34.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T01:59:34.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T01:59:34.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T01:59:34.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T01:59:34.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T01:59:34.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T01:59:34.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T01:59:34.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T01:59:34.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T01:59:34.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T01:59:34.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T01:59:34.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T01:59:34.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T01:59:34.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T01:59:34.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T01:59:34.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T01:59:34.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T01:59:34.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T01:59:34.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T01:59:34.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T01:59:34.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T01:59:34.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T01:59:34.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T01:59:34.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T01:59:34.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T01:59:34.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T01:59:34.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T01:59:34.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T01:59:34.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T01:59:34.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T01:59:34.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T01:59:34.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T01:59:34.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T01:59:34.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T01:59:34.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T01:59:34.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T01:59:34.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T01:59:34.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T01:59:34.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T01:59:34.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T01:59:34.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T01:59:34.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T01:59:34.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T01:59:34.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-09T01:59:34.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.inplace_update_support: 0 2024-05-09T01:59:34.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T01:59:34.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T01:59:34.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T01:59:34.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T01:59:34.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.bloom_locality: 0 2024-05-09T01:59:34.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.max_successive_merges: 0 2024-05-09T01:59:34.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T01:59:34.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T01:59:34.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T01:59:34.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T01:59:34.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.ttl: 2592000 2024-05-09T01:59:34.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T01:59:34.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T01:59:34.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T01:59:34.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.enable_blob_files: false 2024-05-09T01:59:34.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.min_blob_size: 0 2024-05-09T01:59:34.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T01:59:34.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T01:59:34.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T01:59:34.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T01:59:34.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T01:59:34.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T01:59:34.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T01:59:34.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T01:59:34.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-09T01:59:34.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-09T01:59:34.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6fc8384f-c522-4126-9649-1e749070810f 2024-05-09T01:59:34.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715219974147309, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-09T01:59:34.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-09T01:59:34.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715219974148519, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99939, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97821, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14028, "raw_average_key_size": 49, "raw_value_size": 90179, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715219974, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6fc8384f-c522-4126-9649-1e749070810f", "db_session_id": "P02F3WRF07M0QX0IZQKS", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-09T01:59:34.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715219974148606, "job": 1, "event": "recovery_finished"} 2024-05-09T01:59:34.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-09T01:59:34.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T01:59:34.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b2a9de7c00 2024-05-09T01:59:34.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: DB pointer 0x55b2aebda000 2024-05-09T01:59:34.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T01:59:34.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T01:59:34.172 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** DB Stats ** 2024-05-09T01:59:34.172 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T01:59:34.172 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T01:59:34.172 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T01:59:34.172 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T01:59:34.172 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T01:59:34.172 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T01:59:34.172 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T01:59:34.172 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-09T01:59:34.172 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-05-09T01:59:34.172 INFO:journalctl@ceph.mon.a.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T01:59:34.172 INFO:journalctl@ceph.mon.a.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T01:59:34.172 INFO:journalctl@ceph.mon.a.smithi019.stdout: L0 2/0 99.46 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T01:59:34.173 INFO:journalctl@ceph.mon.a.smithi019.stdout: Sum 2/0 99.46 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T01:59:34.173 INFO:journalctl@ceph.mon.a.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T01:59:34.173 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-09T01:59:34.173 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-05-09T01:59:34.173 INFO:journalctl@ceph.mon.a.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T01:59:34.173 INFO:journalctl@ceph.mon.a.smithi019.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T01:59:34.173 INFO:journalctl@ceph.mon.a.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 136.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T01:59:34.173 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-09T01:59:34.173 INFO:journalctl@ceph.mon.a.smithi019.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T01:59:34.173 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-09T01:59:34.173 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T01:59:34.173 INFO:journalctl@ceph.mon.a.smithi019.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T01:59:34.173 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T01:59:34.173 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T01:59:34.173 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T01:59:34.174 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T01:59:34.174 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative compaction: 0.00 GB write, 27.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T01:59:34.174 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval compaction: 0.00 GB write, 27.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T01:59:34.174 INFO:journalctl@ceph.mon.a.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T01:59:34.174 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache BinnedLRUCache@0x55b2a9e031f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-09T01:59:34.174 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-09T01:59:34.174 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-09T01:59:34.174 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T01:59:34.174 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: starting mon.a rank 0 at public addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] at bind addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T01:59:34.174 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: mon.a@-1(???) e1 preinit fsid a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T01:59:34.174 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: mon.a@-1(???).mds e1 new map 2024-05-09T01:59:34.174 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: mon.a@-1(???).mds e1 print_map 2024-05-09T01:59:34.175 INFO:journalctl@ceph.mon.a.smithi019.stdout: e1 2024-05-09T01:59:34.175 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T01:59:34.175 INFO:journalctl@ceph.mon.a.smithi019.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T01:59:34.175 INFO:journalctl@ceph.mon.a.smithi019.stdout: legacy client fscid: -1 2024-05-09T01:59:34.175 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-09T01:59:34.175 INFO:journalctl@ceph.mon.a.smithi019.stdout: No filesystems configured 2024-05-09T01:59:34.175 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T01:59:34.175 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T01:59:34.175 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T01:59:34.175 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T01:59:34.175 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-09T01:59:34.183 INFO:teuthology.orchestra.run.smithi019.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-09T01:59:34.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 bash[30131]: 4d8bf39103f3413dd54b9f9d782cc5312527d3e29548bf4a2826a12a87e3bcc3 2024-05-09T01:59:34.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 systemd[1]: Started Ceph mon.a for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T01:59:34.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T01:59:34.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: monmap epoch 1 2024-05-09T01:59:34.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: fsid a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T01:59:34.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: last_changed 2024-05-09T01:59:13.406931+0000 2024-05-09T01:59:34.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: created 2024-05-09T01:59:13.406931+0000 2024-05-09T01:59:34.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: min_mon_release 19 (squid) 2024-05-09T01:59:34.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: election_strategy: 1 2024-05-09T01:59:34.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T01:59:34.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: fsmap 2024-05-09T01:59:34.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T01:59:34.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:34 smithi019 ceph-mon[30151]: mgrmap e1: no daemons active 2024-05-09T01:59:35.233 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.608+0000 7fbd361e7700 1 Processor -- start 2024-05-09T01:59:35.233 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.608+0000 7fbd361e7700 1 -- start start 2024-05-09T01:59:35.234 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.608+0000 7fbd361e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3010a5e0 0x7fbd3010ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:35.234 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.608+0000 7fbd361e7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3007d6f0 con 0x7fbd3010a5e0 2024-05-09T01:59:35.234 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.609+0000 7fbd2f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3010a5e0 0x7fbd3010ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:35.234 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.609+0000 7fbd2f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3010a5e0 0x7fbd3010ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44130/0 (socket says 172.21.15.19:44130) 2024-05-09T01:59:35.234 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.609+0000 7fbd2f7fe700 1 -- 172.21.15.19:0/1824490220 learned_addr learned my addr 172.21.15.19:0/1824490220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:35.234 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.609+0000 7fbd2f7fe700 1 -- 172.21.15.19:0/1824490220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3010cf40 con 0x7fbd3010a5e0 2024-05-09T01:59:35.234 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.609+0000 7fbd2f7fe700 1 --2- 172.21.15.19:0/1824490220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3010a5e0 0x7fbd3010ca00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbd200098a0 tx=0x7fbd20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4f64bf3b848e47 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:35.234 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.609+0000 7fbd2e7fc700 1 -- 172.21.15.19:0/1824490220 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd20004160 con 0x7fbd3010a5e0 2024-05-09T01:59:35.234 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.610+0000 7fbd2e7fc700 1 -- 172.21.15.19:0/1824490220 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd20004300 con 0x7fbd3010a5e0 2024-05-09T01:59:35.234 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.610+0000 7fbd2e7fc700 1 -- 172.21.15.19:0/1824490220 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd20011af0 con 0x7fbd3010a5e0 2024-05-09T01:59:35.234 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.610+0000 7fbd361e7700 1 -- 172.21.15.19:0/1824490220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3010a5e0 msgr2=0x7fbd3010ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:35.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.610+0000 7fbd361e7700 1 --2- 172.21.15.19:0/1824490220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3010a5e0 0x7fbd3010ca00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbd200098a0 tx=0x7fbd20009bb0 comp rx=0 tx=0).stop 2024-05-09T01:59:35.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.610+0000 7fbd361e7700 1 -- 172.21.15.19:0/1824490220 shutdown_connections 2024-05-09T01:59:35.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.610+0000 7fbd361e7700 1 --2- 172.21.15.19:0/1824490220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3010a5e0 0x7fbd3010ca00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:35.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.610+0000 7fbd361e7700 1 -- 172.21.15.19:0/1824490220 >> 172.21.15.19:0/1824490220 conn(0x7fbd301040e0 msgr2=0x7fbd30106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:35.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.610+0000 7fbd361e7700 1 -- 172.21.15.19:0/1824490220 shutdown_connections 2024-05-09T01:59:35.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.610+0000 7fbd361e7700 1 -- 172.21.15.19:0/1824490220 wait complete. 2024-05-09T01:59:35.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.611+0000 7fbd361e7700 1 Processor -- start 2024-05-09T01:59:35.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.611+0000 7fbd361e7700 1 -- start start 2024-05-09T01:59:35.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.612+0000 7fbd361e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3010a5e0 0x7fbd3019de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:35.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.612+0000 7fbd361e7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3010df00 con 0x7fbd3010a5e0 2024-05-09T01:59:35.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.612+0000 7fbd2f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3010a5e0 0x7fbd3019de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:35.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.612+0000 7fbd2f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3010a5e0 0x7fbd3019de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44140/0 (socket says 172.21.15.19:44140) 2024-05-09T01:59:35.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.612+0000 7fbd2f7fe700 1 -- 172.21.15.19:0/2701472353 learned_addr learned my addr 172.21.15.19:0/2701472353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:35.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.612+0000 7fbd2f7fe700 1 -- 172.21.15.19:0/2701472353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3019e3c0 con 0x7fbd3010a5e0 2024-05-09T01:59:35.235 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.612+0000 7fbd2f7fe700 1 --2- 172.21.15.19:0/2701472353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3010a5e0 0x7fbd3019de80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009fb0 tx=0x7fbd20011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:35.236 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.612+0000 7fbd2cff9700 1 -- 172.21.15.19:0/2701472353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd20011bc0 con 0x7fbd3010a5e0 2024-05-09T01:59:35.236 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.612+0000 7fbd361e7700 1 -- 172.21.15.19:0/2701472353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3019e650 con 0x7fbd3010a5e0 2024-05-09T01:59:35.236 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.612+0000 7fbd2cff9700 1 -- 172.21.15.19:0/2701472353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd20024440 con 0x7fbd3010a5e0 2024-05-09T01:59:35.236 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.612+0000 7fbd2cff9700 1 -- 172.21.15.19:0/2701472353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2001a5c0 con 0x7fbd3010a5e0 2024-05-09T01:59:35.236 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.613+0000 7fbd361e7700 1 -- 172.21.15.19:0/2701472353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd301a1260 con 0x7fbd3010a5e0 2024-05-09T01:59:35.236 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.613+0000 7fbd2cff9700 1 -- 172.21.15.19:0/2701472353 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbd2001ac10 con 0x7fbd3010a5e0 2024-05-09T01:59:35.236 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.613+0000 7fbd2cff9700 1 -- 172.21.15.19:0/2701472353 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbd20019700 con 0x7fbd3010a5e0 2024-05-09T01:59:35.236 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.614+0000 7fbd361e7700 1 -- 172.21.15.19:0/2701472353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd30064e20 con 0x7fbd3010a5e0 2024-05-09T01:59:35.236 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.616+0000 7fbd2cff9700 1 -- 172.21.15.19:0/2701472353 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fbd2002ddf0 con 0x7fbd3010a5e0 2024-05-09T01:59:35.236 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.687+0000 7fbd361e7700 1 -- 172.21.15.19:0/2701472353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fbd301a18d0 con 0x7fbd3010a5e0 2024-05-09T01:59:35.236 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.689+0000 7fbd2cff9700 1 -- 172.21.15.19:0/2701472353 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fbd200245f0 con 0x7fbd3010a5e0 2024-05-09T01:59:35.236 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.690+0000 7fbd361e7700 1 -- 172.21.15.19:0/2701472353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3010a5e0 msgr2=0x7fbd3019de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:35.236 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.690+0000 7fbd361e7700 1 --2- 172.21.15.19:0/2701472353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3010a5e0 0x7fbd3019de80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009fb0 tx=0x7fbd20011890 comp rx=0 tx=0).stop 2024-05-09T01:59:35.236 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.690+0000 7fbd361e7700 1 -- 172.21.15.19:0/2701472353 shutdown_connections 2024-05-09T01:59:35.236 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.690+0000 7fbd361e7700 1 --2- 172.21.15.19:0/2701472353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3010a5e0 0x7fbd3019de80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:35.237 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.690+0000 7fbd361e7700 1 -- 172.21.15.19:0/2701472353 >> 172.21.15.19:0/2701472353 conn(0x7fbd301040e0 msgr2=0x7fbd30105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:35.237 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.690+0000 7fbd361e7700 1 -- 172.21.15.19:0/2701472353 shutdown_connections 2024-05-09T01:59:35.237 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:34.691+0000 7fbd361e7700 1 -- 172.21.15.19:0/2701472353 wait complete. 2024-05-09T01:59:35.237 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-09T01:59:35.259 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:59:35.259 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mgr... 2024-05-09T01:59:35.260 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-09T01:59:35.260 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-09T01:59:35.525 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mgr.x 2024-05-09T01:59:35.526 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-09T01:59:35.684 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a0611f3e-0da7-11ef-bc97-c7b262605968.target.wants/ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@.service. 2024-05-09T01:59:35.948 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:35 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2701472353' entity='client.admin' 2024-05-09T01:59:35.949 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:35 smithi019 podman[30370]: 2024-05-09 01:59:35.850609984 +0000 UTC m=+0.018479460 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T01:59:36.195 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T01:59:36.195 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-09T01:59:36.203 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T01:59:36.203 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-09T01:59:36.203 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-09T01:59:36.203 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T01:59:36.212 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T01:59:36.212 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-09T01:59:36.220 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T01:59:36.220 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-09T01:59:36.220 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-09T01:59:36.220 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-09T01:59:36.220 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr to start... 2024-05-09T01:59:36.220 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr... 2024-05-09T01:59:36.288 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:36 smithi019 podman[30370]: 2024-05-09 01:59:36.113920272 +0000 UTC m=+0.281789730 container init fb9ba98e5788fb69ea0f7a33100fd486a6d11f027435800bed710edb562aac05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-09T01:59:36.288 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:36 smithi019 podman[30370]: 2024-05-09 01:59:36.11671162 +0000 UTC m=+0.284581083 container start fb9ba98e5788fb69ea0f7a33100fd486a6d11f027435800bed710edb562aac05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:59:36.288 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:36 smithi019 bash[30370]: fb9ba98e5788fb69ea0f7a33100fd486a6d11f027435800bed710edb562aac05 2024-05-09T01:59:36.288 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:36 smithi019 systemd[1]: Started Ceph mgr.x for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T01:59:36.570 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:36 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:36.287+0000 7ff4fa5e2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T01:59:36.570 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:36 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:36.485+0000 7ff4fa5e2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T01:59:36.844 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:36 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:36.591+0000 7ff4fa5e2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T01:59:37.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:36 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3994069038' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T01:59:37.146 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:36 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:36.929+0000 7ff4fa5e2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T01:59:37.146 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:37 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:37.018+0000 7ff4fa5e2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T01:59:37.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:59:37.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-09T01:59:37.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "a0611f3e-0da7-11ef-bc97-c7b262605968", 2024-05-09T01:59:37.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T01:59:37.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T01:59:37.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T01:59:37.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T01:59:37.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:37.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T01:59:37.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T01:59:37.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-05-09T01:59:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:59:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T01:59:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T01:59:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:59:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-09T01:59:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T01:59:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T01:59:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T01:59:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T01:59:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T01:59:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T01:59:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T01:59:37.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T01:59:37.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T01:59:37.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T01:59:37.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:37.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T01:59:37.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T01:59:37.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T01:59:37.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T01:59:37.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T01:59:37.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T01:59:37.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T01:59:37.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T01:59:37.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T01:59:37.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:37.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T01:59:37.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:37.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T01:59:37.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T01:59:37.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:37.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T01:59:37.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T01:59:37.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T01:59:37.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T01:59:37.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T01:59:37.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T01:59:37.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T01:59:37.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:59:37.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T01:59:37.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:37.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T01:59:37.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:37.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T01:59:27.591975+0000", 2024-05-09T01:59:37.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T01:59:37.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:37.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T01:59:37.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-09T01:59:37.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.709+0000 7fa633370700 1 Processor -- start 2024-05-09T01:59:37.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.709+0000 7fa633370700 1 -- start start 2024-05-09T01:59:37.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.710+0000 7fa633370700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa62c10a620 0x7fa62c10ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:37.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.710+0000 7fa633370700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c07d690 con 0x7fa62c10a620 2024-05-09T01:59:37.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.710+0000 7fa63110c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa62c10a620 0x7fa62c10ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:37.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.710+0000 7fa63110c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa62c10a620 0x7fa62c10ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33848/0 (socket says 172.21.15.19:33848) 2024-05-09T01:59:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.710+0000 7fa63110c700 1 -- 172.21.15.19:0/3675124004 learned_addr learned my addr 172.21.15.19:0/3675124004 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.710+0000 7fa63110c700 1 -- 172.21.15.19:0/3675124004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c10cf80 con 0x7fa62c10a620 2024-05-09T01:59:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.710+0000 7fa63110c700 1 --2- 172.21.15.19:0/3675124004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa62c10a620 0x7fa62c10ca40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa6280098a0 tx=0x7fa628009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56478e3c0e793071 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.711+0000 7fa623fff700 1 -- 172.21.15.19:0/3675124004 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa628004160 con 0x7fa62c10a620 2024-05-09T01:59:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.711+0000 7fa623fff700 1 -- 172.21.15.19:0/3675124004 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa628004300 con 0x7fa62c10a620 2024-05-09T01:59:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.711+0000 7fa623fff700 1 -- 172.21.15.19:0/3675124004 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa628011af0 con 0x7fa62c10a620 2024-05-09T01:59:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.711+0000 7fa633370700 1 -- 172.21.15.19:0/3675124004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa62c10a620 msgr2=0x7fa62c10ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.711+0000 7fa633370700 1 --2- 172.21.15.19:0/3675124004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa62c10a620 0x7fa62c10ca40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa6280098a0 tx=0x7fa628009bb0 comp rx=0 tx=0).stop 2024-05-09T01:59:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.711+0000 7fa633370700 1 -- 172.21.15.19:0/3675124004 shutdown_connections 2024-05-09T01:59:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.711+0000 7fa633370700 1 --2- 172.21.15.19:0/3675124004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa62c10a620 0x7fa62c10ca40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.711+0000 7fa633370700 1 -- 172.21.15.19:0/3675124004 >> 172.21.15.19:0/3675124004 conn(0x7fa62c104040 msgr2=0x7fa62c1064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.712+0000 7fa633370700 1 -- 172.21.15.19:0/3675124004 shutdown_connections 2024-05-09T01:59:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.712+0000 7fa633370700 1 -- 172.21.15.19:0/3675124004 wait complete. 2024-05-09T01:59:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.712+0000 7fa633370700 1 Processor -- start 2024-05-09T01:59:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.712+0000 7fa633370700 1 -- start start 2024-05-09T01:59:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.713+0000 7fa633370700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa62c10a620 0x7fa62c19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.713+0000 7fa633370700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c10dee0 con 0x7fa62c10a620 2024-05-09T01:59:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.713+0000 7fa63110c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa62c10a620 0x7fa62c19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.713+0000 7fa63110c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa62c10a620 0x7fa62c19de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33860/0 (socket says 172.21.15.19:33860) 2024-05-09T01:59:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.713+0000 7fa63110c700 1 -- 172.21.15.19:0/3994069038 learned_addr learned my addr 172.21.15.19:0/3994069038 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.713+0000 7fa63110c700 1 -- 172.21.15.19:0/3994069038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c19e3c0 con 0x7fa62c10a620 2024-05-09T01:59:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.713+0000 7fa63110c700 1 --2- 172.21.15.19:0/3994069038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa62c10a620 0x7fa62c19de80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa6280117b0 tx=0x7fa628011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.713+0000 7fa6227fc700 1 -- 172.21.15.19:0/3994069038 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6280039c0 con 0x7fa62c10a620 2024-05-09T01:59:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.713+0000 7fa6227fc700 1 -- 172.21.15.19:0/3994069038 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa628003b60 con 0x7fa62c10a620 2024-05-09T01:59:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.713+0000 7fa633370700 1 -- 172.21.15.19:0/3994069038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62c19e650 con 0x7fa62c10a620 2024-05-09T01:59:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.713+0000 7fa6227fc700 1 -- 172.21.15.19:0/3994069038 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6280196a0 con 0x7fa62c10a620 2024-05-09T01:59:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.713+0000 7fa633370700 1 -- 172.21.15.19:0/3994069038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62c1a1260 con 0x7fa62c10a620 2024-05-09T01:59:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.714+0000 7fa6227fc700 1 -- 172.21.15.19:0/3994069038 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa628019ca0 con 0x7fa62c10a620 2024-05-09T01:59:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.714+0000 7fa6227fc700 1 -- 172.21.15.19:0/3994069038 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa628024a70 con 0x7fa62c10a620 2024-05-09T01:59:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.715+0000 7fa633370700 1 -- 172.21.15.19:0/3994069038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62c064e20 con 0x7fa62c10a620 2024-05-09T01:59:37.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.717+0000 7fa6227fc700 1 -- 172.21.15.19:0/3994069038 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa628043ec0 con 0x7fa62c10a620 2024-05-09T01:59:37.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.791+0000 7fa633370700 1 -- 172.21.15.19:0/3994069038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa62c1a1430 con 0x7fa62c10a620 2024-05-09T01:59:37.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.791+0000 7fa6227fc700 1 -- 172.21.15.19:0/3994069038 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa628032020 con 0x7fa62c10a620 2024-05-09T01:59:37.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.792+0000 7fa633370700 1 -- 172.21.15.19:0/3994069038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa62c10a620 msgr2=0x7fa62c19de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:37.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.792+0000 7fa633370700 1 --2- 172.21.15.19:0/3994069038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa62c10a620 0x7fa62c19de80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa6280117b0 tx=0x7fa628011b70 comp rx=0 tx=0).stop 2024-05-09T01:59:37.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.792+0000 7fa633370700 1 -- 172.21.15.19:0/3994069038 shutdown_connections 2024-05-09T01:59:37.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.792+0000 7fa633370700 1 --2- 172.21.15.19:0/3994069038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa62c10a620 0x7fa62c19de80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:37.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.792+0000 7fa633370700 1 -- 172.21.15.19:0/3994069038 >> 172.21.15.19:0/3994069038 conn(0x7fa62c104040 msgr2=0x7fa62c104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:37.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.792+0000 7fa633370700 1 -- 172.21.15.19:0/3994069038 shutdown_connections 2024-05-09T01:59:37.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:36.792+0000 7fa633370700 1 -- 172.21.15.19:0/3994069038 wait complete. 2024-05-09T01:59:37.425 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (1/15)... 2024-05-09T01:59:38.569 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:38.296+0000 7ff4fa5e2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T01:59:38.570 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:38.475+0000 7ff4fa5e2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T01:59:38.967 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:38.665+0000 7ff4fa5e2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T01:59:38.968 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:38.966+0000 7ff4fa5e2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T01:59:39.273 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:39 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:39.062+0000 7ff4fa5e2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T01:59:39.274 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:39 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:39.272+0000 7ff4fa5e2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T01:59:39.539 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:39 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:39.474+0000 7ff4fa5e2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T01:59:40.118 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:40 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1760209204' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T01:59:40.552 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:40 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:40.116+0000 7ff4fa5e2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T01:59:40.552 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:40 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:40.347+0000 7ff4fa5e2200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T01:59:40.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:59:40.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-09T01:59:40.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "a0611f3e-0da7-11ef-bc97-c7b262605968", 2024-05-09T01:59:40.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T01:59:40.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T01:59:40.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T01:59:40.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T01:59:40.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:40.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T01:59:40.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T01:59:40.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-05-09T01:59:40.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:59:40.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T01:59:40.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T01:59:40.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:59:40.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-09T01:59:40.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T01:59:40.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:40.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T01:59:40.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T01:59:40.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:40.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T01:59:40.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:40.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T01:59:40.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T01:59:40.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T01:59:40.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T01:59:40.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T01:59:40.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T01:59:40.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:40.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T01:59:40.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T01:59:40.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T01:59:40.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T01:59:40.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T01:59:40.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T01:59:40.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T01:59:40.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T01:59:40.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T01:59:40.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:40.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T01:59:40.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:40.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T01:59:40.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T01:59:40.572 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:40.572 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T01:59:40.572 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T01:59:40.572 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T01:59:40.572 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T01:59:40.572 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T01:59:40.572 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T01:59:40.572 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T01:59:40.572 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:59:40.572 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T01:59:40.572 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:40.572 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T01:59:40.572 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:40.572 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T01:59:27.591975+0000", 2024-05-09T01:59:40.572 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T01:59:40.573 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:40.573 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T01:59:40.573 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-09T01:59:40.573 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.877+0000 7f3dbcc0f700 1 Processor -- start 2024-05-09T01:59:40.573 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.877+0000 7f3dbcc0f700 1 -- start start 2024-05-09T01:59:40.573 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.878+0000 7f3dbcc0f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db810a610 0x7f3db810ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:40.573 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.878+0000 7f3dbcc0f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3db807ce20 con 0x7f3db810a610 2024-05-09T01:59:40.573 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.878+0000 7f3db659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db810a610 0x7f3db810ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:40.573 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.878+0000 7f3db659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db810a610 0x7f3db810ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33876/0 (socket says 172.21.15.19:33876) 2024-05-09T01:59:40.573 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.878+0000 7f3db659c700 1 -- 172.21.15.19:0/302172029 learned_addr learned my addr 172.21.15.19:0/302172029 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:40.573 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.878+0000 7f3db659c700 1 -- 172.21.15.19:0/302172029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db810cf70 con 0x7f3db810a610 2024-05-09T01:59:40.573 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.879+0000 7f3db659c700 1 --2- 172.21.15.19:0/302172029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db810a610 0x7f3db810ca30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3da80098a0 tx=0x7f3da8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3558830d36a6846c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:40.573 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.879+0000 7f3db559a700 1 -- 172.21.15.19:0/302172029 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3da8004160 con 0x7f3db810a610 2024-05-09T01:59:40.574 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.879+0000 7f3db559a700 1 -- 172.21.15.19:0/302172029 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3da8004300 con 0x7f3db810a610 2024-05-09T01:59:40.574 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.879+0000 7f3db559a700 1 -- 172.21.15.19:0/302172029 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3da8011af0 con 0x7f3db810a610 2024-05-09T01:59:40.574 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.880+0000 7f3dbcc0f700 1 -- 172.21.15.19:0/302172029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db810a610 msgr2=0x7f3db810ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:40.574 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.880+0000 7f3dbcc0f700 1 --2- 172.21.15.19:0/302172029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db810a610 0x7f3db810ca30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3da80098a0 tx=0x7f3da8009bb0 comp rx=0 tx=0).stop 2024-05-09T01:59:40.574 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.880+0000 7f3dbcc0f700 1 -- 172.21.15.19:0/302172029 shutdown_connections 2024-05-09T01:59:40.574 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.880+0000 7f3dbcc0f700 1 --2- 172.21.15.19:0/302172029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db810a610 0x7f3db810ca30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:40.574 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.880+0000 7f3dbcc0f700 1 -- 172.21.15.19:0/302172029 >> 172.21.15.19:0/302172029 conn(0x7f3db8104090 msgr2=0x7f3db81064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:40.574 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.880+0000 7f3dbcc0f700 1 -- 172.21.15.19:0/302172029 shutdown_connections 2024-05-09T01:59:40.574 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.880+0000 7f3dbcc0f700 1 -- 172.21.15.19:0/302172029 wait complete. 2024-05-09T01:59:40.574 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.881+0000 7f3dbcc0f700 1 Processor -- start 2024-05-09T01:59:40.574 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.881+0000 7f3dbcc0f700 1 -- start start 2024-05-09T01:59:40.574 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.881+0000 7f3dbcc0f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db810a610 0x7f3db819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:40.574 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.881+0000 7f3dbcc0f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3db810ded0 con 0x7f3db810a610 2024-05-09T01:59:40.574 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.882+0000 7f3db659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db810a610 0x7f3db819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:40.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.882+0000 7f3db659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db810a610 0x7f3db819deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33882/0 (socket says 172.21.15.19:33882) 2024-05-09T01:59:40.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.882+0000 7f3db659c700 1 -- 172.21.15.19:0/1760209204 learned_addr learned my addr 172.21.15.19:0/1760209204 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:40.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.882+0000 7f3db659c700 1 -- 172.21.15.19:0/1760209204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db819e3f0 con 0x7f3db810a610 2024-05-09T01:59:40.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.882+0000 7f3db659c700 1 --2- 172.21.15.19:0/1760209204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db810a610 0x7f3db819deb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3da80117b0 tx=0x7f3da8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:40.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.882+0000 7f3da77fe700 1 -- 172.21.15.19:0/1760209204 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3da80038b0 con 0x7f3db810a610 2024-05-09T01:59:40.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.882+0000 7f3dbcc0f700 1 -- 172.21.15.19:0/1760209204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db819e680 con 0x7f3db810a610 2024-05-09T01:59:40.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.882+0000 7f3da77fe700 1 -- 172.21.15.19:0/1760209204 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3da8003a50 con 0x7f3db810a610 2024-05-09T01:59:40.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.882+0000 7f3da77fe700 1 -- 172.21.15.19:0/1760209204 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3da8019570 con 0x7f3db810a610 2024-05-09T01:59:40.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.882+0000 7f3dbcc0f700 1 -- 172.21.15.19:0/1760209204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db81a1290 con 0x7f3db810a610 2024-05-09T01:59:40.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.883+0000 7f3da77fe700 1 -- 172.21.15.19:0/1760209204 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3da8019bb0 con 0x7f3db810a610 2024-05-09T01:59:40.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.883+0000 7f3da77fe700 1 -- 172.21.15.19:0/1760209204 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3da8024730 con 0x7f3db810a610 2024-05-09T01:59:40.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.883+0000 7f3dbcc0f700 1 -- 172.21.15.19:0/1760209204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db8064e20 con 0x7f3db810a610 2024-05-09T01:59:40.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.888+0000 7f3da77fe700 1 -- 172.21.15.19:0/1760209204 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3da802dde0 con 0x7f3db810a610 2024-05-09T01:59:40.575 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.963+0000 7f3dbcc0f700 1 -- 172.21.15.19:0/1760209204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3db81a14b0 con 0x7f3db810a610 2024-05-09T01:59:40.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.963+0000 7f3da77fe700 1 -- 172.21.15.19:0/1760209204 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3da8032030 con 0x7f3db810a610 2024-05-09T01:59:40.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.964+0000 7f3dbcc0f700 1 -- 172.21.15.19:0/1760209204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db810a610 msgr2=0x7f3db819deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:40.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.964+0000 7f3dbcc0f700 1 --2- 172.21.15.19:0/1760209204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db810a610 0x7f3db819deb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3da80117b0 tx=0x7f3da8011890 comp rx=0 tx=0).stop 2024-05-09T01:59:40.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.964+0000 7f3dbcc0f700 1 -- 172.21.15.19:0/1760209204 shutdown_connections 2024-05-09T01:59:40.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.964+0000 7f3dbcc0f700 1 --2- 172.21.15.19:0/1760209204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3db810a610 0x7f3db819deb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:40.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.964+0000 7f3dbcc0f700 1 -- 172.21.15.19:0/1760209204 >> 172.21.15.19:0/1760209204 conn(0x7f3db8104090 msgr2=0x7f3db8104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:40.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.964+0000 7f3dbcc0f700 1 -- 172.21.15.19:0/1760209204 shutdown_connections 2024-05-09T01:59:40.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:39.964+0000 7f3dbcc0f700 1 -- 172.21.15.19:0/1760209204 wait complete. 2024-05-09T01:59:40.576 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (2/15)... 2024-05-09T01:59:40.819 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:40 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:40.787+0000 7ff4fa5e2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T01:59:41.271 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:40 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:40.887+0000 7ff4fa5e2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T01:59:41.271 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:41 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:41.270+0000 7ff4fa5e2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T01:59:41.991 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:41 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:41.640+0000 7ff4fa5e2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T01:59:41.992 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:41 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:41.736+0000 7ff4fa5e2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T01:59:42.320 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:41 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:41.990+0000 7ff4fa5e2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T01:59:42.320 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:42 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:42.128+0000 7ff4fa5e2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T01:59:42.320 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:42 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:42.227+0000 7ff4fa5e2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T01:59:42.820 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:42 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:42.566+0000 7ff4fa5e2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T01:59:42.820 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:42 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:42.666+0000 7ff4fa5e2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T01:59:43.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:43 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/345308635' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T01:59:43.570 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:43 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:43.308+0000 7ff4fa5e2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T01:59:43.570 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:43 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:43.532+0000 7ff4fa5e2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T01:59:43.731 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:59:43.731 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-09T01:59:43.731 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "a0611f3e-0da7-11ef-bc97-c7b262605968", 2024-05-09T01:59:43.731 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T01:59:43.731 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T01:59:43.731 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T01:59:43.731 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T01:59:43.731 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:43.731 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T01:59:43.732 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T01:59:43.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T01:59:43.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T01:59:43.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T01:59:43.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T01:59:43.740 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:43.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T01:59:43.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T01:59:43.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T01:59:43.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T01:59:43.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T01:59:43.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T01:59:43.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T01:59:43.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T01:59:43.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T01:59:43.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:43.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T01:59:43.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:43.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T01:59:43.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T01:59:43.741 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:43.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T01:59:43.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T01:59:43.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T01:59:43.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T01:59:43.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T01:59:43.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T01:59:43.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T01:59:43.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:59:43.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T01:59:43.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:43.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T01:59:43.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:43.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T01:59:27.591975+0000", 2024-05-09T01:59:43.742 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T01:59:43.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:43.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T01:59:43.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-09T01:59:43.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.016+0000 7fc789a35700 1 Processor -- start 2024-05-09T01:59:43.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.016+0000 7fc789a35700 1 -- start start 2024-05-09T01:59:43.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.016+0000 7fc789a35700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc78410a690 0x7fc78410cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:43.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.016+0000 7fc789a35700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc78407cda0 con 0x7fc78410a690 2024-05-09T01:59:43.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.017+0000 7fc782ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc78410a690 0x7fc78410cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:43.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.017+0000 7fc782ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc78410a690 0x7fc78410cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33894/0 (socket says 172.21.15.19:33894) 2024-05-09T01:59:43.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.017+0000 7fc782ffd700 1 -- 172.21.15.19:0/766787322 learned_addr learned my addr 172.21.15.19:0/766787322 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:43.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.017+0000 7fc782ffd700 1 -- 172.21.15.19:0/766787322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78410cff0 con 0x7fc78410a690 2024-05-09T01:59:43.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.017+0000 7fc782ffd700 1 --2- 172.21.15.19:0/766787322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc78410a690 0x7fc78410cab0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc774009e80 tx=0x7fc77400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23339a3714d5604e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:43.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.017+0000 7fc781ffb700 1 -- 172.21.15.19:0/766787322 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc774017070 con 0x7fc78410a690 2024-05-09T01:59:43.743 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.017+0000 7fc781ffb700 1 -- 172.21.15.19:0/766787322 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc77400b8c0 con 0x7fc78410a690 2024-05-09T01:59:43.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.017+0000 7fc781ffb700 1 -- 172.21.15.19:0/766787322 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc774004490 con 0x7fc78410a690 2024-05-09T01:59:43.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.018+0000 7fc789a35700 1 -- 172.21.15.19:0/766787322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc78410a690 msgr2=0x7fc78410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:43.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.018+0000 7fc789a35700 1 --2- 172.21.15.19:0/766787322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc78410a690 0x7fc78410cab0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc774009e80 tx=0x7fc77400b330 comp rx=0 tx=0).stop 2024-05-09T01:59:43.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.018+0000 7fc789a35700 1 -- 172.21.15.19:0/766787322 shutdown_connections 2024-05-09T01:59:43.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.018+0000 7fc789a35700 1 --2- 172.21.15.19:0/766787322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc78410a690 0x7fc78410cab0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:43.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.018+0000 7fc789a35700 1 -- 172.21.15.19:0/766787322 >> 172.21.15.19:0/766787322 conn(0x7fc784104090 msgr2=0x7fc7841064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:43.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.019+0000 7fc789a35700 1 -- 172.21.15.19:0/766787322 shutdown_connections 2024-05-09T01:59:43.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.019+0000 7fc789a35700 1 -- 172.21.15.19:0/766787322 wait complete. 2024-05-09T01:59:43.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.019+0000 7fc789a35700 1 Processor -- start 2024-05-09T01:59:43.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.019+0000 7fc789a35700 1 -- start start 2024-05-09T01:59:43.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.020+0000 7fc789a35700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc78410a690 0x7fc78419de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:43.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.020+0000 7fc789a35700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc78410def0 con 0x7fc78410a690 2024-05-09T01:59:43.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.020+0000 7fc782ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc78410a690 0x7fc78419de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:43.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.020+0000 7fc782ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc78410a690 0x7fc78419de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33896/0 (socket says 172.21.15.19:33896) 2024-05-09T01:59:43.744 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.020+0000 7fc782ffd700 1 -- 172.21.15.19:0/345308635 learned_addr learned my addr 172.21.15.19:0/345308635 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:43.745 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.020+0000 7fc782ffd700 1 -- 172.21.15.19:0/345308635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78419e3b0 con 0x7fc78410a690 2024-05-09T01:59:43.745 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.020+0000 7fc782ffd700 1 --2- 172.21.15.19:0/345308635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc78410a690 0x7fc78419de70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc774004220 tx=0x7fc77400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:43.745 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.020+0000 7fc788a33700 1 -- 172.21.15.19:0/345308635 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc774020020 con 0x7fc78410a690 2024-05-09T01:59:43.745 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.021+0000 7fc788a33700 1 -- 172.21.15.19:0/345308635 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc774012430 con 0x7fc78410a690 2024-05-09T01:59:43.745 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.021+0000 7fc789a35700 1 -- 172.21.15.19:0/345308635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78419e640 con 0x7fc78410a690 2024-05-09T01:59:43.745 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.021+0000 7fc788a33700 1 -- 172.21.15.19:0/345308635 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc774017040 con 0x7fc78410a690 2024-05-09T01:59:43.745 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.021+0000 7fc789a35700 1 -- 172.21.15.19:0/345308635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7841a1250 con 0x7fc78410a690 2024-05-09T01:59:43.745 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.021+0000 7fc788a33700 1 -- 172.21.15.19:0/345308635 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc774023020 con 0x7fc78410a690 2024-05-09T01:59:43.745 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.021+0000 7fc788a33700 1 -- 172.21.15.19:0/345308635 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc77400f050 con 0x7fc78410a690 2024-05-09T01:59:43.745 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.022+0000 7fc789a35700 1 -- 172.21.15.19:0/345308635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc784064e20 con 0x7fc78410a690 2024-05-09T01:59:43.745 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.024+0000 7fc788a33700 1 -- 172.21.15.19:0/345308635 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc77402a020 con 0x7fc78410a690 2024-05-09T01:59:43.745 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.100+0000 7fc789a35700 1 -- 172.21.15.19:0/345308635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc7841a1810 con 0x7fc78410a690 2024-05-09T01:59:43.745 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.100+0000 7fc788a33700 1 -- 172.21.15.19:0/345308635 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc77400b860 con 0x7fc78410a690 2024-05-09T01:59:43.746 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.101+0000 7fc789a35700 1 -- 172.21.15.19:0/345308635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc78410a690 msgr2=0x7fc78419de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:43.746 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.101+0000 7fc789a35700 1 --2- 172.21.15.19:0/345308635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc78410a690 0x7fc78419de70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc774004220 tx=0x7fc77400bd50 comp rx=0 tx=0).stop 2024-05-09T01:59:43.746 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.101+0000 7fc789a35700 1 -- 172.21.15.19:0/345308635 shutdown_connections 2024-05-09T01:59:43.746 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.101+0000 7fc789a35700 1 --2- 172.21.15.19:0/345308635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc78410a690 0x7fc78419de70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:43.746 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.101+0000 7fc789a35700 1 -- 172.21.15.19:0/345308635 >> 172.21.15.19:0/345308635 conn(0x7fc784104090 msgr2=0x7fc7841048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:43.746 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.101+0000 7fc789a35700 1 -- 172.21.15.19:0/345308635 shutdown_connections 2024-05-09T01:59:43.746 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:43.101+0000 7fc789a35700 1 -- 172.21.15.19:0/345308635 wait complete. 2024-05-09T01:59:43.746 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (3/15)... 2024-05-09T01:59:44.451 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:44 smithi019 ceph-mon[30151]: Activating manager daemon x 2024-05-09T01:59:44.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:44 smithi019 ceph-mon[30151]: mgrmap e2: x(active, starting, since 0.00375326s) 2024-05-09T01:59:44.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:44 smithi019 ceph-mon[30151]: from='mgr.14100 172.21.15.19:0/289359752' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T01:59:44.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:44 smithi019 ceph-mon[30151]: from='mgr.14100 172.21.15.19:0/289359752' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T01:59:44.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:44 smithi019 ceph-mon[30151]: from='mgr.14100 172.21.15.19:0/289359752' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T01:59:44.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:44 smithi019 ceph-mon[30151]: from='mgr.14100 172.21.15.19:0/289359752' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:59:44.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:44 smithi019 ceph-mon[30151]: from='mgr.14100 172.21.15.19:0/289359752' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T01:59:44.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:44 smithi019 ceph-mon[30151]: Manager daemon x is now available 2024-05-09T01:59:44.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:44 smithi019 ceph-mon[30151]: from='mgr.14100 172.21.15.19:0/289359752' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T01:59:44.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:44 smithi019 ceph-mon[30151]: from='mgr.14100 172.21.15.19:0/289359752' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T01:59:44.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:44 smithi019 ceph-mon[30151]: from='mgr.14100 172.21.15.19:0/289359752' entity='mgr.x' 2024-05-09T01:59:44.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:44 smithi019 ceph-mon[30151]: from='mgr.14100 172.21.15.19:0/289359752' entity='mgr.x' 2024-05-09T01:59:44.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:44 smithi019 ceph-mon[30151]: from='mgr.14100 172.21.15.19:0/289359752' entity='mgr.x' 2024-05-09T01:59:45.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:45 smithi019 ceph-mon[30151]: mgrmap e3: x(active, since 1.00815s) 2024-05-09T01:59:46.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:46 smithi019 ceph-mon[30151]: mgrmap e4: x(active, since 2s) 2024-05-09T01:59:46.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:46 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2288491130' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T01:59:47.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:59:47.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-09T01:59:47.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "a0611f3e-0da7-11ef-bc97-c7b262605968", 2024-05-09T01:59:47.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T01:59:47.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T01:59:47.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T01:59:47.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T01:59:47.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:47.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T01:59:47.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T01:59:47.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-05-09T01:59:47.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:59:47.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T01:59:47.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T01:59:47.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:59:47.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-09T01:59:47.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T01:59:47.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:47.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T01:59:47.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T01:59:47.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:47.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T01:59:47.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:47.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T01:59:47.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T01:59:47.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T01:59:47.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T01:59:47.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T01:59:47.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T01:59:47.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:47.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T01:59:47.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T01:59:47.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T01:59:47.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T01:59:47.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T01:59:47.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T01:59:47.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T01:59:47.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T01:59:47.064 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T01:59:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:47.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T01:59:47.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T01:59:47.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T01:59:27.591975+0000", 2024-05-09T01:59:47.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T01:59:47.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-09T01:59:47.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T01:59:47.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-09T01:59:47.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.170+0000 7ff4ad4ab700 1 Processor -- start 2024-05-09T01:59:47.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.170+0000 7ff4ad4ab700 1 -- start start 2024-05-09T01:59:47.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.170+0000 7ff4ad4ab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4a810a5b0 0x7ff4a810c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:47.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.170+0000 7ff4ad4ab700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a807d890 con 0x7ff4a810a5b0 2024-05-09T01:59:47.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.170+0000 7ff4a6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4a810a5b0 0x7ff4a810c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:47.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.170+0000 7ff4a6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4a810a5b0 0x7ff4a810c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33976/0 (socket says 172.21.15.19:33976) 2024-05-09T01:59:47.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.170+0000 7ff4a6ffd700 1 -- 172.21.15.19:0/1984209147 learned_addr learned my addr 172.21.15.19:0/1984209147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:47.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.171+0000 7ff4a6ffd700 1 -- 172.21.15.19:0/1984209147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a810cf10 con 0x7ff4a810a5b0 2024-05-09T01:59:47.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.171+0000 7ff4a6ffd700 1 --2- 172.21.15.19:0/1984209147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4a810a5b0 0x7ff4a810c9d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff4980098a0 tx=0x7ff498009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771d473e69f0ed2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:47.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.171+0000 7ff4a5ffb700 1 -- 172.21.15.19:0/1984209147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff498004160 con 0x7ff4a810a5b0 2024-05-09T01:59:47.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.171+0000 7ff4a5ffb700 1 -- 172.21.15.19:0/1984209147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff498004300 con 0x7ff4a810a5b0 2024-05-09T01:59:47.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.171+0000 7ff4a5ffb700 1 -- 172.21.15.19:0/1984209147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff498011af0 con 0x7ff4a810a5b0 2024-05-09T01:59:47.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.172+0000 7ff4ad4ab700 1 -- 172.21.15.19:0/1984209147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4a810a5b0 msgr2=0x7ff4a810c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:47.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.172+0000 7ff4ad4ab700 1 --2- 172.21.15.19:0/1984209147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4a810a5b0 0x7ff4a810c9d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff4980098a0 tx=0x7ff498009bb0 comp rx=0 tx=0).stop 2024-05-09T01:59:47.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.172+0000 7ff4ad4ab700 1 -- 172.21.15.19:0/1984209147 shutdown_connections 2024-05-09T01:59:47.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.172+0000 7ff4ad4ab700 1 --2- 172.21.15.19:0/1984209147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4a810a5b0 0x7ff4a810c9d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:47.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.172+0000 7ff4ad4ab700 1 -- 172.21.15.19:0/1984209147 >> 172.21.15.19:0/1984209147 conn(0x7ff4a8104010 msgr2=0x7ff4a81064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:47.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.172+0000 7ff4ad4ab700 1 -- 172.21.15.19:0/1984209147 shutdown_connections 2024-05-09T01:59:47.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.172+0000 7ff4ad4ab700 1 -- 172.21.15.19:0/1984209147 wait complete. 2024-05-09T01:59:47.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.173+0000 7ff4ad4ab700 1 Processor -- start 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.173+0000 7ff4ad4ab700 1 -- start start 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.173+0000 7ff4ad4ab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4a810a5b0 0x7ff4a819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.173+0000 7ff4ad4ab700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a810ded0 con 0x7ff4a810a5b0 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.173+0000 7ff4a6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4a810a5b0 0x7ff4a819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.173+0000 7ff4a6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4a810a5b0 0x7ff4a819de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33986/0 (socket says 172.21.15.19:33986) 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.173+0000 7ff4a6ffd700 1 -- 172.21.15.19:0/2288491130 learned_addr learned my addr 172.21.15.19:0/2288491130 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.174+0000 7ff4a6ffd700 1 -- 172.21.15.19:0/2288491130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a819e390 con 0x7ff4a810a5b0 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.174+0000 7ff4a6ffd700 1 --2- 172.21.15.19:0/2288491130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4a810a5b0 0x7ff4a819de50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff4980117b0 tx=0x7ff498011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.174+0000 7ff48ffff700 1 -- 172.21.15.19:0/2288491130 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4980039c0 con 0x7ff4a810a5b0 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.174+0000 7ff4ad4ab700 1 -- 172.21.15.19:0/2288491130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a819e620 con 0x7ff4a810a5b0 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.174+0000 7ff48ffff700 1 -- 172.21.15.19:0/2288491130 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff498003b60 con 0x7ff4a810a5b0 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.174+0000 7ff48ffff700 1 -- 172.21.15.19:0/2288491130 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4980196a0 con 0x7ff4a810a5b0 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.174+0000 7ff4ad4ab700 1 -- 172.21.15.19:0/2288491130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a81a1230 con 0x7ff4a810a5b0 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.175+0000 7ff4ad4ab700 1 -- 172.21.15.19:0/2288491130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a8064e20 con 0x7ff4a810a5b0 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.175+0000 7ff48ffff700 1 -- 172.21.15.19:0/2288491130 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7ff498019960 con 0x7ff4a810a5b0 2024-05-09T01:59:47.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.175+0000 7ff48ffff700 1 --2- 172.21.15.19:0/2288491130 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7ff49003e4e0 0x7ff4900409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:47.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.176+0000 7ff48ffff700 1 -- 172.21.15.19:0/2288491130 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff498053ac0 con 0x7ff4a810a5b0 2024-05-09T01:59:47.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.176+0000 7ff4a67fc700 1 --2- 172.21.15.19:0/2288491130 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7ff49003e4e0 0x7ff4900409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:47.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.176+0000 7ff4a67fc700 1 --2- 172.21.15.19:0/2288491130 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7ff49003e4e0 0x7ff4900409f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff49c006fd0 tx=0x7ff49c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:47.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.181+0000 7ff48ffff700 1 -- 172.21.15.19:0/2288491130 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff498017e90 con 0x7ff4a810a5b0 2024-05-09T01:59:47.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.486+0000 7ff4ad4ab700 1 -- 172.21.15.19:0/2288491130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff4a81a1450 con 0x7ff4a810a5b0 2024-05-09T01:59:47.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.486+0000 7ff48ffff700 1 -- 172.21.15.19:0/2288491130 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff498023390 con 0x7ff4a810a5b0 2024-05-09T01:59:47.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.489+0000 7ff4ad4ab700 1 -- 172.21.15.19:0/2288491130 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7ff49003e4e0 msgr2=0x7ff4900409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:47.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.489+0000 7ff4ad4ab700 1 --2- 172.21.15.19:0/2288491130 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7ff49003e4e0 0x7ff4900409f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff49c006fd0 tx=0x7ff49c006e40 comp rx=0 tx=0).stop 2024-05-09T01:59:47.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.489+0000 7ff4ad4ab700 1 -- 172.21.15.19:0/2288491130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4a810a5b0 msgr2=0x7ff4a819de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:47.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.489+0000 7ff4ad4ab700 1 --2- 172.21.15.19:0/2288491130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4a810a5b0 0x7ff4a819de50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff4980117b0 tx=0x7ff498011b70 comp rx=0 tx=0).stop 2024-05-09T01:59:47.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.489+0000 7ff4ad4ab700 1 -- 172.21.15.19:0/2288491130 shutdown_connections 2024-05-09T01:59:47.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.489+0000 7ff4ad4ab700 1 --2- 172.21.15.19:0/2288491130 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7ff49003e4e0 0x7ff4900409f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:47.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.489+0000 7ff4ad4ab700 1 --2- 172.21.15.19:0/2288491130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4a810a5b0 0x7ff4a819de50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:47.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.489+0000 7ff4ad4ab700 1 -- 172.21.15.19:0/2288491130 >> 172.21.15.19:0/2288491130 conn(0x7ff4a8104010 msgr2=0x7ff4a81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:47.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.489+0000 7ff4ad4ab700 1 -- 172.21.15.19:0/2288491130 shutdown_connections 2024-05-09T01:59:47.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:46.489+0000 7ff4ad4ab700 1 -- 172.21.15.19:0/2288491130 wait complete. 2024-05-09T01:59:47.070 INFO:teuthology.orchestra.run.smithi019.stdout:mgr is available 2024-05-09T01:59:48.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:47 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2571000166' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T01:59:48.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:59:48.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T01:59:48.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T01:59:48.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-09T01:59:48.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T01:59:48.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T01:59:48.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T01:59:48.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T01:59:48.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:59:48.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T01:59:48.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T01:59:48.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-09T01:59:48.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T01:59:48.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T01:59:48.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T01:59:48.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T01:59:48.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T01:59:48.308 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T01:59:48.308 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.477+0000 7ff37ca72700 1 Processor -- start 2024-05-09T01:59:48.308 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.477+0000 7ff37ca72700 1 -- start start 2024-05-09T01:59:48.308 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.477+0000 7ff37ca72700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff378109af0 0x7ff378109f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:48.308 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.477+0000 7ff37ca72700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff37810a4f0 con 0x7ff378109af0 2024-05-09T01:59:48.308 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.478+0000 7ff37659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff378109af0 0x7ff378109f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:48.308 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.478+0000 7ff37659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff378109af0 0x7ff378109f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37020/0 (socket says 172.21.15.19:37020) 2024-05-09T01:59:48.308 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.478+0000 7ff37659c700 1 -- 172.21.15.19:0/2002129641 learned_addr learned my addr 172.21.15.19:0/2002129641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:48.308 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.478+0000 7ff37659c700 1 -- 172.21.15.19:0/2002129641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37810acc0 con 0x7ff378109af0 2024-05-09T01:59:48.308 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.478+0000 7ff37659c700 1 --2- 172.21.15.19:0/2002129641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff378109af0 0x7ff378109f00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3680098a0 tx=0x7ff368009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf6d344c3777a69 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:48.308 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.478+0000 7ff37559a700 1 -- 172.21.15.19:0/2002129641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff368004160 con 0x7ff378109af0 2024-05-09T01:59:48.309 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.478+0000 7ff37559a700 1 -- 172.21.15.19:0/2002129641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff368004300 con 0x7ff378109af0 2024-05-09T01:59:48.309 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.478+0000 7ff37559a700 1 -- 172.21.15.19:0/2002129641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff368011af0 con 0x7ff378109af0 2024-05-09T01:59:48.309 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.479+0000 7ff37ca72700 1 -- 172.21.15.19:0/2002129641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff378109af0 msgr2=0x7ff378109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:48.309 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.479+0000 7ff37ca72700 1 --2- 172.21.15.19:0/2002129641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff378109af0 0x7ff378109f00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3680098a0 tx=0x7ff368009bb0 comp rx=0 tx=0).stop 2024-05-09T01:59:48.309 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.479+0000 7ff37ca72700 1 -- 172.21.15.19:0/2002129641 shutdown_connections 2024-05-09T01:59:48.309 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.479+0000 7ff37ca72700 1 --2- 172.21.15.19:0/2002129641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff378109af0 0x7ff378109f00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:48.309 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.479+0000 7ff37ca72700 1 -- 172.21.15.19:0/2002129641 >> 172.21.15.19:0/2002129641 conn(0x7ff378105170 msgr2=0x7ff3781075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:48.309 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.479+0000 7ff37ca72700 1 -- 172.21.15.19:0/2002129641 shutdown_connections 2024-05-09T01:59:48.309 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.479+0000 7ff37ca72700 1 -- 172.21.15.19:0/2002129641 wait complete. 2024-05-09T01:59:48.309 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.480+0000 7ff37ca72700 1 Processor -- start 2024-05-09T01:59:48.309 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.480+0000 7ff37ca72700 1 -- start start 2024-05-09T01:59:48.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.480+0000 7ff37ca72700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff378109af0 0x7ff37807f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:48.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.480+0000 7ff37ca72700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff37810e070 con 0x7ff378109af0 2024-05-09T01:59:48.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.480+0000 7ff37659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff378109af0 0x7ff37807f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:48.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.480+0000 7ff37659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff378109af0 0x7ff37807f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37028/0 (socket says 172.21.15.19:37028) 2024-05-09T01:59:48.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.480+0000 7ff37659c700 1 -- 172.21.15.19:0/2571000166 learned_addr learned my addr 172.21.15.19:0/2571000166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:48.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.481+0000 7ff37659c700 1 -- 172.21.15.19:0/2571000166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37807f9c0 con 0x7ff378109af0 2024-05-09T01:59:48.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.481+0000 7ff37659c700 1 --2- 172.21.15.19:0/2571000166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff378109af0 0x7ff37807f480 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff3680090e0 tx=0x7ff368011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:48.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.481+0000 7ff3677fe700 1 -- 172.21.15.19:0/2571000166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3680039e0 con 0x7ff378109af0 2024-05-09T01:59:48.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.481+0000 7ff3677fe700 1 -- 172.21.15.19:0/2571000166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff368003b80 con 0x7ff378109af0 2024-05-09T01:59:48.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.481+0000 7ff37ca72700 1 -- 172.21.15.19:0/2571000166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37807dc10 con 0x7ff378109af0 2024-05-09T01:59:48.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.481+0000 7ff3677fe700 1 -- 172.21.15.19:0/2571000166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff368024690 con 0x7ff378109af0 2024-05-09T01:59:48.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.481+0000 7ff37ca72700 1 -- 172.21.15.19:0/2571000166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37807e0f0 con 0x7ff378109af0 2024-05-09T01:59:48.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.482+0000 7ff3677fe700 1 -- 172.21.15.19:0/2571000166 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7ff368022020 con 0x7ff378109af0 2024-05-09T01:59:48.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.482+0000 7ff37ca72700 1 -- 172.21.15.19:0/2571000166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff378064e20 con 0x7ff378109af0 2024-05-09T01:59:48.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.483+0000 7ff3677fe700 1 --2- 172.21.15.19:0/2571000166 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7ff36003e4e0 0x7ff3600409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:48.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.483+0000 7ff3677fe700 1 -- 172.21.15.19:0/2571000166 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff3680533d0 con 0x7ff378109af0 2024-05-09T01:59:48.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.483+0000 7ff375d9b700 1 --2- 172.21.15.19:0/2571000166 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7ff36003e4e0 0x7ff3600409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:48.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.483+0000 7ff375d9b700 1 --2- 172.21.15.19:0/2571000166 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7ff36003e4e0 0x7ff3600409f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff36c006fd0 tx=0x7ff36c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:48.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.488+0000 7ff3677fe700 1 -- 172.21.15.19:0/2571000166 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff3680172b0 con 0x7ff378109af0 2024-05-09T01:59:48.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.702+0000 7ff37ca72700 1 -- 172.21.15.19:0/2571000166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff37807e900 con 0x7ff378109af0 2024-05-09T01:59:48.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.704+0000 7ff3677fe700 1 -- 172.21.15.19:0/2571000166 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7ff368053130 con 0x7ff378109af0 2024-05-09T01:59:48.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.707+0000 7ff37ca72700 1 -- 172.21.15.19:0/2571000166 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7ff36003e4e0 msgr2=0x7ff3600409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:48.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.707+0000 7ff37ca72700 1 --2- 172.21.15.19:0/2571000166 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7ff36003e4e0 0x7ff3600409f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff36c006fd0 tx=0x7ff36c006e40 comp rx=0 tx=0).stop 2024-05-09T01:59:48.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.707+0000 7ff37ca72700 1 -- 172.21.15.19:0/2571000166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff378109af0 msgr2=0x7ff37807f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:48.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.707+0000 7ff37ca72700 1 --2- 172.21.15.19:0/2571000166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff378109af0 0x7ff37807f480 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff3680090e0 tx=0x7ff368011940 comp rx=0 tx=0).stop 2024-05-09T01:59:48.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.707+0000 7ff37ca72700 1 -- 172.21.15.19:0/2571000166 shutdown_connections 2024-05-09T01:59:48.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.707+0000 7ff37ca72700 1 --2- 172.21.15.19:0/2571000166 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7ff36003e4e0 0x7ff3600409f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:48.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.707+0000 7ff37ca72700 1 --2- 172.21.15.19:0/2571000166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff378109af0 0x7ff37807f480 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:48.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.707+0000 7ff37ca72700 1 -- 172.21.15.19:0/2571000166 >> 172.21.15.19:0/2571000166 conn(0x7ff378105170 msgr2=0x7ff3781059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:48.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.707+0000 7ff37ca72700 1 -- 172.21.15.19:0/2571000166 shutdown_connections 2024-05-09T01:59:48.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:47.707+0000 7ff37ca72700 1 -- 172.21.15.19:0/2571000166 wait complete. 2024-05-09T01:59:48.312 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling cephadm module... 2024-05-09T01:59:49.286 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:49 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3721802942' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-09T01:59:49.286 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:49 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: ignoring --setuser ceph since I am not root 2024-05-09T01:59:49.286 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:49 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: ignoring --setgroup ceph since I am not root 2024-05-09T01:59:49.286 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:49 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:49.285+0000 7fe22e12a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T01:59:49.569 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:49 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:49.486+0000 7fe22e12a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T01:59:49.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.744+0000 7fd0b78b7700 1 Processor -- start 2024-05-09T01:59:49.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.744+0000 7fd0b78b7700 1 -- start start 2024-05-09T01:59:49.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.744+0000 7fd0b78b7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b01089e0 0x7fd0b0108df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:49.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.744+0000 7fd0b78b7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b01093e0 con 0x7fd0b01089e0 2024-05-09T01:59:49.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.744+0000 7fd0b5653700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b01089e0 0x7fd0b0108df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:49.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.745+0000 7fd0b5653700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b01089e0 0x7fd0b0108df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37042/0 (socket says 172.21.15.19:37042) 2024-05-09T01:59:49.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.745+0000 7fd0b5653700 1 -- 172.21.15.19:0/3582068492 learned_addr learned my addr 172.21.15.19:0/3582068492 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:49.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.745+0000 7fd0b5653700 1 -- 172.21.15.19:0/3582068492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b0109bb0 con 0x7fd0b01089e0 2024-05-09T01:59:49.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.745+0000 7fd0b5653700 1 --2- 172.21.15.19:0/3582068492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b01089e0 0x7fd0b0108df0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0098a0 tx=0x7fd0ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c2d27894063d44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:49.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.745+0000 7fd0a7fff700 1 -- 172.21.15.19:0/3582068492 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0ac004160 con 0x7fd0b01089e0 2024-05-09T01:59:49.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.745+0000 7fd0a7fff700 1 -- 172.21.15.19:0/3582068492 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd0ac004300 con 0x7fd0b01089e0 2024-05-09T01:59:49.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.745+0000 7fd0a7fff700 1 -- 172.21.15.19:0/3582068492 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0ac011af0 con 0x7fd0b01089e0 2024-05-09T01:59:49.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.746+0000 7fd0b78b7700 1 -- 172.21.15.19:0/3582068492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b01089e0 msgr2=0x7fd0b0108df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:49.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.746+0000 7fd0b78b7700 1 --2- 172.21.15.19:0/3582068492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b01089e0 0x7fd0b0108df0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0098a0 tx=0x7fd0ac009bb0 comp rx=0 tx=0).stop 2024-05-09T01:59:49.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.746+0000 7fd0b78b7700 1 -- 172.21.15.19:0/3582068492 shutdown_connections 2024-05-09T01:59:49.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.746+0000 7fd0b78b7700 1 --2- 172.21.15.19:0/3582068492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b01089e0 0x7fd0b0108df0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:49.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.746+0000 7fd0b78b7700 1 -- 172.21.15.19:0/3582068492 >> 172.21.15.19:0/3582068492 conn(0x7fd0b0104040 msgr2=0x7fd0b01064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:49.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.746+0000 7fd0b78b7700 1 -- 172.21.15.19:0/3582068492 shutdown_connections 2024-05-09T01:59:49.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.746+0000 7fd0b78b7700 1 -- 172.21.15.19:0/3582068492 wait complete. 2024-05-09T01:59:49.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.747+0000 7fd0b78b7700 1 Processor -- start 2024-05-09T01:59:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.747+0000 7fd0b78b7700 1 -- start start 2024-05-09T01:59:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.747+0000 7fd0b78b7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b01089e0 0x7fd0b019dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.747+0000 7fd0b78b7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b010cf60 con 0x7fd0b01089e0 2024-05-09T01:59:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.747+0000 7fd0b5653700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b01089e0 0x7fd0b019dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.747+0000 7fd0b5653700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b01089e0 0x7fd0b019dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37044/0 (socket says 172.21.15.19:37044) 2024-05-09T01:59:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.747+0000 7fd0b5653700 1 -- 172.21.15.19:0/3721802942 learned_addr learned my addr 172.21.15.19:0/3721802942 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.747+0000 7fd0b5653700 1 -- 172.21.15.19:0/3721802942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b019e400 con 0x7fd0b01089e0 2024-05-09T01:59:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.748+0000 7fd0b5653700 1 --2- 172.21.15.19:0/3721802942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b01089e0 0x7fd0b019dec0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0090e0 tx=0x7fd0ac011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.748+0000 7fd0a67fc700 1 -- 172.21.15.19:0/3721802942 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0ac0039e0 con 0x7fd0b01089e0 2024-05-09T01:59:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.748+0000 7fd0a67fc700 1 -- 172.21.15.19:0/3721802942 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd0ac003b80 con 0x7fd0b01089e0 2024-05-09T01:59:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.748+0000 7fd0b78b7700 1 -- 172.21.15.19:0/3721802942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b019e690 con 0x7fd0b01089e0 2024-05-09T01:59:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.748+0000 7fd0a67fc700 1 -- 172.21.15.19:0/3721802942 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0ac024690 con 0x7fd0b01089e0 2024-05-09T01:59:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.748+0000 7fd0b78b7700 1 -- 172.21.15.19:0/3721802942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b01a12a0 con 0x7fd0b01089e0 2024-05-09T01:59:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.749+0000 7fd0a67fc700 1 -- 172.21.15.19:0/3721802942 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fd0ac022020 con 0x7fd0b01089e0 2024-05-09T01:59:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.749+0000 7fd0b78b7700 1 -- 172.21.15.19:0/3721802942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b0064e20 con 0x7fd0b01089e0 2024-05-09T01:59:49.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.749+0000 7fd0a67fc700 1 --2- 172.21.15.19:0/3721802942 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7fd09c03e530 0x7fd09c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:49.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.750+0000 7fd0a67fc700 1 -- 172.21.15.19:0/3721802942 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd0ac053490 con 0x7fd0b01089e0 2024-05-09T01:59:49.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.750+0000 7fd0b4e52700 1 --2- 172.21.15.19:0/3721802942 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7fd09c03e530 0x7fd09c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:49.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.750+0000 7fd0b4e52700 1 --2- 172.21.15.19:0/3721802942 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7fd09c03e530 0x7fd09c040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0006fd0 tx=0x7fd0a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:49.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:48.755+0000 7fd0a67fc700 1 -- 172.21.15.19:0/3721802942 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd0ac0249f0 con 0x7fd0b01089e0 2024-05-09T01:59:49.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:49.028+0000 7fd0b78b7700 1 -- 172.21.15.19:0/3721802942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fd0b002e5f0 con 0x7fd0b01089e0 2024-05-09T01:59:49.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:49.079+0000 7fd0a67fc700 1 -- 172.21.15.19:0/3721802942 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fd0b002e5f0 con 0x7fd0b01089e0 2024-05-09T01:59:49.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:49.082+0000 7fd0a67fc700 1 -- 172.21.15.19:0/3721802942 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fd0ac024c70 con 0x7fd0b01089e0 2024-05-09T01:59:49.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:49.082+0000 7fd0b78b7700 1 -- 172.21.15.19:0/3721802942 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7fd09c03e530 msgr2=0x7fd09c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:49.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:49.082+0000 7fd0b78b7700 1 --2- 172.21.15.19:0/3721802942 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7fd09c03e530 0x7fd09c040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0006fd0 tx=0x7fd0a0006e40 comp rx=0 tx=0).stop 2024-05-09T01:59:49.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:49.082+0000 7fd0b78b7700 1 -- 172.21.15.19:0/3721802942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b01089e0 msgr2=0x7fd0b019dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:49.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:49.082+0000 7fd0b78b7700 1 --2- 172.21.15.19:0/3721802942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b01089e0 0x7fd0b019dec0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0090e0 tx=0x7fd0ac011940 comp rx=0 tx=0).stop 2024-05-09T01:59:49.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:49.082+0000 7fd0b78b7700 1 -- 172.21.15.19:0/3721802942 shutdown_connections 2024-05-09T01:59:49.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:49.082+0000 7fd0b78b7700 1 --2- 172.21.15.19:0/3721802942 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7fd09c03e530 0x7fd09c040a40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:49.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:49.082+0000 7fd0b78b7700 1 --2- 172.21.15.19:0/3721802942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b01089e0 0x7fd0b019dec0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:49.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:49.082+0000 7fd0b78b7700 1 -- 172.21.15.19:0/3721802942 >> 172.21.15.19:0/3721802942 conn(0x7fd0b0104040 msgr2=0x7fd0b01944c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:49.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:49.083+0000 7fd0b78b7700 1 -- 172.21.15.19:0/3721802942 shutdown_connections 2024-05-09T01:59:49.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:49.083+0000 7fd0b78b7700 1 -- 172.21.15.19:0/3721802942 wait complete. 2024-05-09T01:59:49.920 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:49 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:49.594+0000 7fe22e12a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T01:59:50.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:50 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3721802942' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-09T01:59:50.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:50 smithi019 ceph-mon[30151]: mgrmap e5: x(active, since 5s) 2024-05-09T01:59:50.172 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:49 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:49.928+0000 7fe22e12a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T01:59:50.172 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:50 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:50.020+0000 7fe22e12a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T01:59:51.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-09T01:59:51.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-09T01:59:51.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T01:59:51.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T01:59:51.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T01:59:51.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-09T01:59:51.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.159+0000 7f853337f700 1 Processor -- start 2024-05-09T01:59:51.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.160+0000 7f853337f700 1 -- start start 2024-05-09T01:59:51.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.160+0000 7f853337f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c1089f0 0x7f852c108e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:51.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.160+0000 7f853337f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f852c1093f0 con 0x7f852c1089f0 2024-05-09T01:59:51.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.160+0000 7f853111b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c1089f0 0x7f852c108e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:51.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.160+0000 7f853111b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c1089f0 0x7f852c108e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37082/0 (socket says 172.21.15.19:37082) 2024-05-09T01:59:51.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.160+0000 7f853111b700 1 -- 172.21.15.19:0/3936206780 learned_addr learned my addr 172.21.15.19:0/3936206780 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:51.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.160+0000 7f853111b700 1 -- 172.21.15.19:0/3936206780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852c109bc0 con 0x7f852c1089f0 2024-05-09T01:59:51.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.161+0000 7f853111b700 1 --2- 172.21.15.19:0/3936206780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c1089f0 0x7f852c108e00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f85280098a0 tx=0x7f8528009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb99761e41eb396 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:51.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.161+0000 7f8523fff700 1 -- 172.21.15.19:0/3936206780 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8528004160 con 0x7f852c1089f0 2024-05-09T01:59:51.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.161+0000 7f8523fff700 1 -- 172.21.15.19:0/3936206780 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8528004300 con 0x7f852c1089f0 2024-05-09T01:59:51.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.161+0000 7f8523fff700 1 -- 172.21.15.19:0/3936206780 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8528011af0 con 0x7f852c1089f0 2024-05-09T01:59:51.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.162+0000 7f853337f700 1 -- 172.21.15.19:0/3936206780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c1089f0 msgr2=0x7f852c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:51.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.162+0000 7f853337f700 1 --2- 172.21.15.19:0/3936206780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c1089f0 0x7f852c108e00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f85280098a0 tx=0x7f8528009bb0 comp rx=0 tx=0).stop 2024-05-09T01:59:51.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.162+0000 7f853337f700 1 -- 172.21.15.19:0/3936206780 shutdown_connections 2024-05-09T01:59:51.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.162+0000 7f853337f700 1 --2- 172.21.15.19:0/3936206780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c1089f0 0x7f852c108e00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:51.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.162+0000 7f853337f700 1 -- 172.21.15.19:0/3936206780 >> 172.21.15.19:0/3936206780 conn(0x7f852c104090 msgr2=0x7f852c1064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:51.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.162+0000 7f853337f700 1 -- 172.21.15.19:0/3936206780 shutdown_connections 2024-05-09T01:59:51.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.162+0000 7f853337f700 1 -- 172.21.15.19:0/3936206780 wait complete. 2024-05-09T01:59:51.077 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.163+0000 7f853337f700 1 Processor -- start 2024-05-09T01:59:51.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.163+0000 7f853337f700 1 -- start start 2024-05-09T01:59:51.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.163+0000 7f853337f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c1089f0 0x7f852c1a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:51.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.163+0000 7f853337f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f852c07c130 con 0x7f852c1089f0 2024-05-09T01:59:51.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.163+0000 7f853111b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c1089f0 0x7f852c1a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:51.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.163+0000 7f853111b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c1089f0 0x7f852c1a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37096/0 (socket says 172.21.15.19:37096) 2024-05-09T01:59:51.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.163+0000 7f853111b700 1 -- 172.21.15.19:0/2493036125 learned_addr learned my addr 172.21.15.19:0/2493036125 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:51.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.163+0000 7f853111b700 1 -- 172.21.15.19:0/2493036125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852c1a0600 con 0x7f852c1089f0 2024-05-09T01:59:51.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.163+0000 7f853111b700 1 --2- 172.21.15.19:0/2493036125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c1089f0 0x7f852c1a00c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f85280090e0 tx=0x7f8528011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:51.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.164+0000 7f85227fc700 1 -- 172.21.15.19:0/2493036125 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85280039e0 con 0x7f852c1089f0 2024-05-09T01:59:51.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.164+0000 7f85227fc700 1 -- 172.21.15.19:0/2493036125 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8528003b80 con 0x7f852c1089f0 2024-05-09T01:59:51.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.164+0000 7f85227fc700 1 -- 172.21.15.19:0/2493036125 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85280246e0 con 0x7f852c1089f0 2024-05-09T01:59:51.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.164+0000 7f853337f700 1 -- 172.21.15.19:0/2493036125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f852c1a0890 con 0x7f852c1089f0 2024-05-09T01:59:51.078 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.164+0000 7f853337f700 1 -- 172.21.15.19:0/2493036125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f852c1a34a0 con 0x7f852c1089f0 2024-05-09T01:59:51.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.165+0000 7f853337f700 1 -- 172.21.15.19:0/2493036125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f852c064e20 con 0x7f852c1089f0 2024-05-09T01:59:51.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.165+0000 7f85227fc700 1 -- 172.21.15.19:0/2493036125 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f8528022020 con 0x7f852c1089f0 2024-05-09T01:59:51.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.165+0000 7f85227fc700 1 --2- 172.21.15.19:0/2493036125 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f851803e580 0x7f8518040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:51.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.165+0000 7f85227fc700 1 -- 172.21.15.19:0/2493036125 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8528053770 con 0x7f852c1089f0 2024-05-09T01:59:51.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.165+0000 7f853091a700 1 -- 172.21.15.19:0/2493036125 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f851803e580 msgr2=0x7f8518040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1829553843 2024-05-09T01:59:51.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.165+0000 7f853091a700 1 --2- 172.21.15.19:0/2493036125 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f851803e580 0x7f8518040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T01:59:51.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.170+0000 7f85227fc700 1 -- 172.21.15.19:0/2493036125 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f85280172b0 con 0x7f852c1089f0 2024-05-09T01:59:51.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.366+0000 7f853091a700 1 -- 172.21.15.19:0/2493036125 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f851803e580 msgr2=0x7f8518040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1829553843 2024-05-09T01:59:51.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.366+0000 7f853091a700 1 --2- 172.21.15.19:0/2493036125 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f851803e580 0x7f8518040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T01:59:51.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.484+0000 7f853337f700 1 -- 172.21.15.19:0/2493036125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f852c194ec0 con 0x7f852c1089f0 2024-05-09T01:59:51.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.486+0000 7f85227fc700 1 -- 172.21.15.19:0/2493036125 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f852c194ec0 con 0x7f852c1089f0 2024-05-09T01:59:51.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.491+0000 7f853337f700 1 -- 172.21.15.19:0/2493036125 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f851803e580 msgr2=0x7f8518040a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T01:59:51.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.491+0000 7f853337f700 1 --2- 172.21.15.19:0/2493036125 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f851803e580 0x7f8518040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:51.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.491+0000 7f853337f700 1 -- 172.21.15.19:0/2493036125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c1089f0 msgr2=0x7f852c1a00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:51.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.491+0000 7f853337f700 1 --2- 172.21.15.19:0/2493036125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c1089f0 0x7f852c1a00c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f85280090e0 tx=0x7f8528011940 comp rx=0 tx=0).stop 2024-05-09T01:59:51.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.492+0000 7f853337f700 1 -- 172.21.15.19:0/2493036125 shutdown_connections 2024-05-09T01:59:51.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.492+0000 7f853337f700 1 --2- 172.21.15.19:0/2493036125 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f851803e580 0x7f8518040a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:51.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.492+0000 7f853337f700 1 --2- 172.21.15.19:0/2493036125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c1089f0 0x7f852c1a00c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:51.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.492+0000 7f853337f700 1 -- 172.21.15.19:0/2493036125 >> 172.21.15.19:0/2493036125 conn(0x7f852c104090 msgr2=0x7f852c191f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:51.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.492+0000 7f853337f700 1 -- 172.21.15.19:0/2493036125 shutdown_connections 2024-05-09T01:59:51.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:50.492+0000 7f853337f700 1 -- 172.21.15.19:0/2493036125 wait complete. 2024-05-09T01:59:51.080 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-05-09T01:59:51.080 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 5... 2024-05-09T01:59:51.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:51 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2493036125' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T01:59:51.320 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:51 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:51.305+0000 7fe22e12a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T01:59:51.674 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:51 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:51.491+0000 7fe22e12a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T01:59:51.674 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:51 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:51.673+0000 7fe22e12a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T01:59:52.291 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:51 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:51.979+0000 7fe22e12a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T01:59:52.291 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:52 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:52.077+0000 7fe22e12a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T01:59:52.291 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:52 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:52.289+0000 7fe22e12a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T01:59:52.569 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:52 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:52.494+0000 7fe22e12a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T01:59:53.320 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:53 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:53.130+0000 7fe22e12a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T01:59:53.819 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:53 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:53.364+0000 7fe22e12a200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T01:59:54.069 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:53 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:53.817+0000 7fe22e12a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T01:59:54.070 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:53 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:53.920+0000 7fe22e12a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T01:59:54.564 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:54 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:54.307+0000 7fe22e12a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T01:59:54.820 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:54 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:54.687+0000 7fe22e12a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T01:59:54.820 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:54 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:54.785+0000 7fe22e12a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T01:59:55.320 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:55 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:55.045+0000 7fe22e12a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T01:59:55.320 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:55 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:55.189+0000 7fe22e12a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T01:59:55.321 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:55 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:55.291+0000 7fe22e12a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T01:59:56.070 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:55 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:55.634+0000 7fe22e12a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T01:59:56.070 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:55 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:55.732+0000 7fe22e12a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T01:59:56.659 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:56 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:56.369+0000 7fe22e12a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T01:59:56.659 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 01:59:56 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T01:59:56.602+0000 7fe22e12a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T01:59:57.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:56 smithi019 ceph-mon[30151]: Active manager daemon x restarted 2024-05-09T01:59:57.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:56 smithi019 ceph-mon[30151]: Activating manager daemon x 2024-05-09T01:59:57.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:56 smithi019 ceph-mon[30151]: osdmap e2: 0 total, 0 up, 0 in 2024-05-09T01:59:57.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:56 smithi019 ceph-mon[30151]: mgrmap e6: x(active, starting, since 0.00391141s) 2024-05-09T01:59:57.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:56 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T01:59:57.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:56 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T01:59:57.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:56 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T01:59:57.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:56 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T01:59:57.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:56 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T01:59:57.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:56 smithi019 ceph-mon[30151]: Manager daemon x is now available 2024-05-09T01:59:57.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:56 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' 2024-05-09T01:59:57.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:56 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' 2024-05-09T01:59:57.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:56 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:59:57.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:56 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:59:57.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:57 smithi019 ceph-mon[30151]: Found migration_current of "None". Setting to last migration. 2024-05-09T01:59:57.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:57 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T01:59:57.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:57 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T01:59:57.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:57 smithi019 ceph-mon[30151]: mgrmap e7: x(active, since 1.0062s) 2024-05-09T01:59:58.192 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-09T01:59:58.192 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-09T01:59:58.192 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T01:59:58.192 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-09T01:59:58.192 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.551+0000 7f3d33028700 1 Processor -- start 2024-05-09T01:59:58.192 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.551+0000 7f3d33028700 1 -- start start 2024-05-09T01:59:58.192 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.551+0000 7f3d33028700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c10a5e0 0x7f3d2c10ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:58.193 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.551+0000 7f3d33028700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c07d890 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.193 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.552+0000 7f3d30dc4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c10a5e0 0x7f3d2c10ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:58.193 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.552+0000 7f3d30dc4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c10a5e0 0x7f3d2c10ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37110/0 (socket says 172.21.15.19:37110) 2024-05-09T01:59:58.193 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.552+0000 7f3d30dc4700 1 -- 172.21.15.19:0/456710129 learned_addr learned my addr 172.21.15.19:0/456710129 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:58.193 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.552+0000 7f3d30dc4700 1 -- 172.21.15.19:0/456710129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2c10cf40 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.193 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.552+0000 7f3d30dc4700 1 --2- 172.21.15.19:0/456710129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c10a5e0 0x7f3d2c10ca00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3d200098a0 tx=0x7f3d20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1d2a7fb72fe3a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:58.193 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.552+0000 7f3d2b7fe700 1 -- 172.21.15.19:0/456710129 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d20004160 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.193 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.553+0000 7f3d2b7fe700 1 -- 172.21.15.19:0/456710129 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3d20004300 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.193 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.553+0000 7f3d2b7fe700 1 -- 172.21.15.19:0/456710129 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d20011af0 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.193 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.554+0000 7f3d33028700 1 -- 172.21.15.19:0/456710129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c10a5e0 msgr2=0x7f3d2c10ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:58.193 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.554+0000 7f3d33028700 1 --2- 172.21.15.19:0/456710129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c10a5e0 0x7f3d2c10ca00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3d200098a0 tx=0x7f3d20009bb0 comp rx=0 tx=0).stop 2024-05-09T01:59:58.193 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.554+0000 7f3d33028700 1 -- 172.21.15.19:0/456710129 shutdown_connections 2024-05-09T01:59:58.193 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.554+0000 7f3d33028700 1 --2- 172.21.15.19:0/456710129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c10a5e0 0x7f3d2c10ca00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:58.193 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.554+0000 7f3d33028700 1 -- 172.21.15.19:0/456710129 >> 172.21.15.19:0/456710129 conn(0x7f3d2c104040 msgr2=0x7f3d2c1064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:58.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.554+0000 7f3d33028700 1 -- 172.21.15.19:0/456710129 shutdown_connections 2024-05-09T01:59:58.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.554+0000 7f3d33028700 1 -- 172.21.15.19:0/456710129 wait complete. 2024-05-09T01:59:58.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.554+0000 7f3d33028700 1 Processor -- start 2024-05-09T01:59:58.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.555+0000 7f3d33028700 1 -- start start 2024-05-09T01:59:58.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.555+0000 7f3d33028700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c10a5e0 0x7f3d2c19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:58.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.555+0000 7f3d33028700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c10df00 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.555+0000 7f3d30dc4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c10a5e0 0x7f3d2c19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:58.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.555+0000 7f3d30dc4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c10a5e0 0x7f3d2c19de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37120/0 (socket says 172.21.15.19:37120) 2024-05-09T01:59:58.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.555+0000 7f3d30dc4700 1 -- 172.21.15.19:0/1276261734 learned_addr learned my addr 172.21.15.19:0/1276261734 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:58.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.555+0000 7f3d30dc4700 1 -- 172.21.15.19:0/1276261734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2c19e3c0 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.556+0000 7f3d30dc4700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c10a5e0 0x7f3d2c19de80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009fb0 tx=0x7f3d20011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:58.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.556+0000 7f3d29ffb700 1 -- 172.21.15.19:0/1276261734 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d20011bc0 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.556+0000 7f3d29ffb700 1 -- 172.21.15.19:0/1276261734 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3d20024440 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.556+0000 7f3d29ffb700 1 -- 172.21.15.19:0/1276261734 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d2001a5c0 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.556+0000 7f3d33028700 1 -- 172.21.15.19:0/1276261734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d2c19e650 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.556+0000 7f3d33028700 1 -- 172.21.15.19:0/1276261734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d2c1a1260 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.557+0000 7f3d29ffb700 1 -- 172.21.15.19:0/1276261734 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f3d2001a830 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.557+0000 7f3d29ffb700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f3d1c03e530 0x7f3d1c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:58.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.557+0000 7f3d33028700 1 -- 172.21.15.19:0/1276261734 --> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3d2c064e20 con 0x7f3d1c03e530 2024-05-09T01:59:58.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.557+0000 7f3d29ffb700 1 -- 172.21.15.19:0/1276261734 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3d20053a50 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.557+0000 7f3d2bfff700 1 -- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f3d1c03e530 msgr2=0x7f3d1c040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1829553843 2024-05-09T01:59:58.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.558+0000 7f3d2bfff700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f3d1c03e530 0x7f3d1c040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T01:59:58.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.759+0000 7f3d2bfff700 1 -- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f3d1c03e530 msgr2=0x7f3d1c040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1829553843 2024-05-09T01:59:58.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:51.759+0000 7f3d2bfff700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f3d1c03e530 0x7f3d1c040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T01:59:58.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:52.160+0000 7f3d2bfff700 1 -- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f3d1c03e530 msgr2=0x7f3d1c040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1829553843 2024-05-09T01:59:58.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:52.160+0000 7f3d2bfff700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f3d1c03e530 0x7f3d1c040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T01:59:58.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:52.961+0000 7f3d2bfff700 1 -- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f3d1c03e530 msgr2=0x7f3d1c040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1829553843 2024-05-09T01:59:58.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:52.961+0000 7f3d2bfff700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f3d1c03e530 0x7f3d1c040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T01:59:58.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:54.562+0000 7f3d2bfff700 1 -- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f3d1c03e530 msgr2=0x7f3d1c040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1829553843 2024-05-09T01:59:58.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:54.562+0000 7f3d2bfff700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f3d1c03e530 0x7f3d1c040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T01:59:58.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:56.606+0000 7f3d29ffb700 1 -- 172.21.15.19:0/1276261734 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f3d20021d00 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:56.606+0000 7f3d29ffb700 1 -- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f3d1c03e530 msgr2=0x7f3d1c040a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T01:59:58.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:56.606+0000 7f3d29ffb700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f3d1c03e530 0x7f3d1c040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:58.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.607+0000 7f3d29ffb700 1 -- 172.21.15.19:0/1276261734 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f3d200245f0 con 0x7f3d2c10a5e0 2024-05-09T01:59:58.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.607+0000 7f3d29ffb700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f3d1c041c80 0x7f3d1c0440a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:58.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.607+0000 7f3d29ffb700 1 -- 172.21.15.19:0/1276261734 --> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3d2003f920 con 0x7f3d1c041c80 2024-05-09T01:59:58.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.608+0000 7f3d2bfff700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f3d1c041c80 0x7f3d1c0440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:58.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.609+0000 7f3d2bfff700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f3d1c041c80 0x7f3d1c0440a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3d18003a10 tx=0x7f3d180092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:58.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.610+0000 7f3d29ffb700 1 -- 172.21.15.19:0/1276261734 <== mgr.14120 v2:172.21.15.19:6800/2101052000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f3d2003f920 con 0x7f3d1c041c80 2024-05-09T01:59:58.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.625+0000 7f3d33028700 1 -- 172.21.15.19:0/1276261734 --> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3d2c105020 con 0x7f3d1c041c80 2024-05-09T01:59:58.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.625+0000 7f3d29ffb700 1 -- 172.21.15.19:0/1276261734 <== mgr.14120 v2:172.21.15.19:6800/2101052000 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f3d2c105020 con 0x7f3d1c041c80 2024-05-09T01:59:58.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.625+0000 7f3d33028700 1 -- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f3d1c041c80 msgr2=0x7f3d1c0440a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:58.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.625+0000 7f3d33028700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f3d1c041c80 0x7f3d1c0440a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3d18003a10 tx=0x7f3d180092b0 comp rx=0 tx=0).stop 2024-05-09T01:59:58.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.625+0000 7f3d33028700 1 -- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c10a5e0 msgr2=0x7f3d2c19de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:58.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.625+0000 7f3d33028700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c10a5e0 0x7f3d2c19de80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009fb0 tx=0x7f3d20011890 comp rx=0 tx=0).stop 2024-05-09T01:59:58.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.625+0000 7f3d33028700 1 -- 172.21.15.19:0/1276261734 shutdown_connections 2024-05-09T01:59:58.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.625+0000 7f3d33028700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f3d1c041c80 0x7f3d1c0440a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:58.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.625+0000 7f3d33028700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c10a5e0 0x7f3d2c19de80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:58.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.625+0000 7f3d33028700 1 --2- 172.21.15.19:0/1276261734 >> [v2:172.21.15.19:6800/1829553843,v1:172.21.15.19:6801/1829553843] conn(0x7f3d1c03e530 0x7f3d1c040a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:58.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.625+0000 7f3d33028700 1 -- 172.21.15.19:0/1276261734 >> 172.21.15.19:0/1276261734 conn(0x7f3d2c104040 msgr2=0x7f3d2c104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:58.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.625+0000 7f3d33028700 1 -- 172.21.15.19:0/1276261734 shutdown_connections 2024-05-09T01:59:58.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:57.625+0000 7f3d33028700 1 -- 172.21.15.19:0/1276261734 wait complete. 2024-05-09T01:59:58.197 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 5 is available 2024-05-09T01:59:58.197 INFO:teuthology.orchestra.run.smithi019.stdout:Setting orchestrator backend to cephadm... 2024-05-09T01:59:58.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:58 smithi019 ceph-mon[30151]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T01:59:58.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:58 smithi019 ceph-mon[30151]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T01:59:58.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:58 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' 2024-05-09T01:59:58.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:58 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' 2024-05-09T01:59:59.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.607+0000 7f780fbaa700 1 Processor -- start 2024-05-09T01:59:59.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.607+0000 7f780fbaa700 1 -- start start 2024-05-09T01:59:59.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.608+0000 7f780fbaa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f780810a5b0 0x7f780810c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:59.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.608+0000 7f780fbaa700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f780807d890 con 0x7f780810a5b0 2024-05-09T01:59:59.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.608+0000 7f780d946700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f780810a5b0 0x7f780810c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:59.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.608+0000 7f780d946700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f780810a5b0 0x7f780810c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34460/0 (socket says 172.21.15.19:34460) 2024-05-09T01:59:59.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.608+0000 7f780d946700 1 -- 172.21.15.19:0/3409029042 learned_addr learned my addr 172.21.15.19:0/3409029042 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:59.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.608+0000 7f780d946700 1 -- 172.21.15.19:0/3409029042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780810cf10 con 0x7f780810a5b0 2024-05-09T01:59:59.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.608+0000 7f780d946700 1 --2- 172.21.15.19:0/3409029042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f780810a5b0 0x7f780810c9d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f78040098a0 tx=0x7f7804009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cfec41d6016c9f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:59.431 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.608+0000 7f780c944700 1 -- 172.21.15.19:0/3409029042 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7804004160 con 0x7f780810a5b0 2024-05-09T01:59:59.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.609+0000 7f780c944700 1 -- 172.21.15.19:0/3409029042 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7804004300 con 0x7f780810a5b0 2024-05-09T01:59:59.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.609+0000 7f780c944700 1 -- 172.21.15.19:0/3409029042 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7804011af0 con 0x7f780810a5b0 2024-05-09T01:59:59.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.610+0000 7f780fbaa700 1 -- 172.21.15.19:0/3409029042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f780810a5b0 msgr2=0x7f780810c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:59.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.610+0000 7f780fbaa700 1 --2- 172.21.15.19:0/3409029042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f780810a5b0 0x7f780810c9d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f78040098a0 tx=0x7f7804009bb0 comp rx=0 tx=0).stop 2024-05-09T01:59:59.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.610+0000 7f780fbaa700 1 -- 172.21.15.19:0/3409029042 shutdown_connections 2024-05-09T01:59:59.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.610+0000 7f780fbaa700 1 --2- 172.21.15.19:0/3409029042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f780810a5b0 0x7f780810c9d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:59.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.610+0000 7f780fbaa700 1 -- 172.21.15.19:0/3409029042 >> 172.21.15.19:0/3409029042 conn(0x7f7808104010 msgr2=0x7f78081064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:59.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.610+0000 7f780fbaa700 1 -- 172.21.15.19:0/3409029042 shutdown_connections 2024-05-09T01:59:59.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.610+0000 7f780fbaa700 1 -- 172.21.15.19:0/3409029042 wait complete. 2024-05-09T01:59:59.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.610+0000 7f780fbaa700 1 Processor -- start 2024-05-09T01:59:59.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.611+0000 7f780fbaa700 1 -- start start 2024-05-09T01:59:59.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.611+0000 7f780fbaa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f780810a5b0 0x7f780819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:59.432 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.611+0000 7f780fbaa700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f780810ded0 con 0x7f780810a5b0 2024-05-09T01:59:59.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.611+0000 7f780d946700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f780810a5b0 0x7f780819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:59.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.611+0000 7f780d946700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f780810a5b0 0x7f780819de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34476/0 (socket says 172.21.15.19:34476) 2024-05-09T01:59:59.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.611+0000 7f780d946700 1 -- 172.21.15.19:0/3452708929 learned_addr learned my addr 172.21.15.19:0/3452708929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T01:59:59.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.611+0000 7f780d946700 1 -- 172.21.15.19:0/3452708929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780819e390 con 0x7f780810a5b0 2024-05-09T01:59:59.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.611+0000 7f780d946700 1 --2- 172.21.15.19:0/3452708929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f780810a5b0 0x7f780819de50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7804009fb0 tx=0x7f7804011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:59.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.612+0000 7f77feffd700 1 -- 172.21.15.19:0/3452708929 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7804011bc0 con 0x7f780810a5b0 2024-05-09T01:59:59.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.612+0000 7f77feffd700 1 -- 172.21.15.19:0/3452708929 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7804024440 con 0x7f780810a5b0 2024-05-09T01:59:59.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.612+0000 7f780fbaa700 1 -- 172.21.15.19:0/3452708929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780819e620 con 0x7f780810a5b0 2024-05-09T01:59:59.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.612+0000 7f77feffd700 1 -- 172.21.15.19:0/3452708929 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f780401a5c0 con 0x7f780810a5b0 2024-05-09T01:59:59.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.612+0000 7f780fbaa700 1 -- 172.21.15.19:0/3452708929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78081a1230 con 0x7f780810a5b0 2024-05-09T01:59:59.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.613+0000 7f780fbaa700 1 -- 172.21.15.19:0/3452708929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7808064e20 con 0x7f780810a5b0 2024-05-09T01:59:59.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.613+0000 7f77feffd700 1 -- 172.21.15.19:0/3452708929 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f780401a880 con 0x7f780810a5b0 2024-05-09T01:59:59.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.613+0000 7f77feffd700 1 --2- 172.21.15.19:0/3452708929 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f77f403e3d0 0x7f77f40408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:59:59.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.613+0000 7f77feffd700 1 -- 172.21.15.19:0/3452708929 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f78040537c0 con 0x7f780810a5b0 2024-05-09T01:59:59.433 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.613+0000 7f780d145700 1 --2- 172.21.15.19:0/3452708929 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f77f403e3d0 0x7f77f40408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:59:59.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.614+0000 7f780d145700 1 --2- 172.21.15.19:0/3452708929 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f77f403e3d0 0x7f77f40408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77f8006fd0 tx=0x7f77f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:59:59.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.618+0000 7f77feffd700 1 -- 172.21.15.19:0/3452708929 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f780400b3e0 con 0x7f780810a5b0 2024-05-09T01:59:59.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.661+0000 7f77feffd700 1 -- 172.21.15.19:0/3452708929 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f780402d3f0 con 0x7f780810a5b0 2024-05-09T01:59:59.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.851+0000 7f780fbaa700 1 -- 172.21.15.19:0/3452708929 --> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f7808104ff0 con 0x7f77f403e3d0 2024-05-09T01:59:59.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.856+0000 7f77feffd700 1 -- 172.21.15.19:0/3452708929 <== mgr.14120 v2:172.21.15.19:6800/2101052000 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7808104ff0 con 0x7f77f403e3d0 2024-05-09T01:59:59.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.858+0000 7f780fbaa700 1 -- 172.21.15.19:0/3452708929 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f77f403e3d0 msgr2=0x7f77f40408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:59.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.858+0000 7f780fbaa700 1 --2- 172.21.15.19:0/3452708929 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f77f403e3d0 0x7f77f40408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77f8006fd0 tx=0x7f77f8006e40 comp rx=0 tx=0).stop 2024-05-09T01:59:59.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.858+0000 7f780fbaa700 1 -- 172.21.15.19:0/3452708929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f780810a5b0 msgr2=0x7f780819de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:59:59.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.858+0000 7f780fbaa700 1 --2- 172.21.15.19:0/3452708929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f780810a5b0 0x7f780819de50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7804009fb0 tx=0x7f7804011890 comp rx=0 tx=0).stop 2024-05-09T01:59:59.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.859+0000 7f780fbaa700 1 -- 172.21.15.19:0/3452708929 shutdown_connections 2024-05-09T01:59:59.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.859+0000 7f780fbaa700 1 --2- 172.21.15.19:0/3452708929 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f77f403e3d0 0x7f77f40408e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:59.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.859+0000 7f780fbaa700 1 --2- 172.21.15.19:0/3452708929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f780810a5b0 0x7f780819de50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:59:59.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.859+0000 7f780fbaa700 1 -- 172.21.15.19:0/3452708929 >> 172.21.15.19:0/3452708929 conn(0x7f7808104010 msgr2=0x7f78081048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:59:59.434 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.859+0000 7f780fbaa700 1 -- 172.21.15.19:0/3452708929 shutdown_connections 2024-05-09T01:59:59.435 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:58.859+0000 7f780fbaa700 1 -- 172.21.15.19:0/3452708929 wait complete. 2024-05-09T01:59:59.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:59 smithi019 ceph-mon[30151]: mgrmap e8: x(active, since 2s) 2024-05-09T01:59:59.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:59 smithi019 ceph-mon[30151]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T01:59:59.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:59 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' 2024-05-09T01:59:59.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 01:59:59 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:00:00.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:00 smithi019 ceph-mon[30151]: [09/May/2024:02:00:00] ENGINE Bus STARTING 2024-05-09T02:00:00.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:00 smithi019 ceph-mon[30151]: [09/May/2024:02:00:00] ENGINE Serving on http://172.21.15.19:8765 2024-05-09T02:00:00.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:00 smithi019 ceph-mon[30151]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:00:00.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:00 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:00:00.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-09T02:00:00.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.896+0000 7ff5286a1700 1 Processor -- start 2024-05-09T02:00:00.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.896+0000 7ff5286a1700 1 -- start start 2024-05-09T02:00:00.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.896+0000 7ff5286a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff520107de0 0x7ff5201081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:00.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.896+0000 7ff5286a1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201087c0 con 0x7ff520107de0 2024-05-09T02:00:00.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.896+0000 7ff52643d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff520107de0 0x7ff5201081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:00.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.896+0000 7ff52643d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff520107de0 0x7ff5201081f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34478/0 (socket says 172.21.15.19:34478) 2024-05-09T02:00:00.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.896+0000 7ff52643d700 1 -- 172.21.15.19:0/1269720167 learned_addr learned my addr 172.21.15.19:0/1269720167 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:00.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.897+0000 7ff52643d700 1 -- 172.21.15.19:0/1269720167 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52010ade0 con 0x7ff520107de0 2024-05-09T02:00:00.755 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.897+0000 7ff52643d700 1 --2- 172.21.15.19:0/1269720167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff520107de0 0x7ff5201081f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0098a0 tx=0x7ff51c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2823f7a202581be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:00.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.897+0000 7ff52543b700 1 -- 172.21.15.19:0/1269720167 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff51c004160 con 0x7ff520107de0 2024-05-09T02:00:00.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.897+0000 7ff52543b700 1 -- 172.21.15.19:0/1269720167 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff51c004300 con 0x7ff520107de0 2024-05-09T02:00:00.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.897+0000 7ff52543b700 1 -- 172.21.15.19:0/1269720167 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff51c011af0 con 0x7ff520107de0 2024-05-09T02:00:00.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.898+0000 7ff5286a1700 1 -- 172.21.15.19:0/1269720167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff520107de0 msgr2=0x7ff5201081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:00.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.898+0000 7ff5286a1700 1 --2- 172.21.15.19:0/1269720167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff520107de0 0x7ff5201081f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0098a0 tx=0x7ff51c009bb0 comp rx=0 tx=0).stop 2024-05-09T02:00:00.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.898+0000 7ff5286a1700 1 -- 172.21.15.19:0/1269720167 shutdown_connections 2024-05-09T02:00:00.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.898+0000 7ff5286a1700 1 --2- 172.21.15.19:0/1269720167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff520107de0 0x7ff5201081f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:00.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.898+0000 7ff5286a1700 1 -- 172.21.15.19:0/1269720167 >> 172.21.15.19:0/1269720167 conn(0x7ff520103810 msgr2=0x7ff520105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:00.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.898+0000 7ff5286a1700 1 -- 172.21.15.19:0/1269720167 shutdown_connections 2024-05-09T02:00:00.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.898+0000 7ff5286a1700 1 -- 172.21.15.19:0/1269720167 wait complete. 2024-05-09T02:00:00.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.899+0000 7ff5286a1700 1 Processor -- start 2024-05-09T02:00:00.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.899+0000 7ff5286a1700 1 -- start start 2024-05-09T02:00:00.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.899+0000 7ff5286a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff520107de0 0x7ff52010a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:00.756 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.899+0000 7ff5286a1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff52010e010 con 0x7ff520107de0 2024-05-09T02:00:00.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.899+0000 7ff52643d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff520107de0 0x7ff52010a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:00.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.899+0000 7ff52643d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff520107de0 0x7ff52010a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34492/0 (socket says 172.21.15.19:34492) 2024-05-09T02:00:00.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.899+0000 7ff52643d700 1 -- 172.21.15.19:0/2850812834 learned_addr learned my addr 172.21.15.19:0/2850812834 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:00.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.900+0000 7ff52643d700 1 -- 172.21.15.19:0/2850812834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52010aaa0 con 0x7ff520107de0 2024-05-09T02:00:00.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.900+0000 7ff52643d700 1 --2- 172.21.15.19:0/2850812834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff520107de0 0x7ff52010a560 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff51c011670 tx=0x7ff51c004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:00.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.900+0000 7ff5177fe700 1 -- 172.21.15.19:0/2850812834 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff51c0036a0 con 0x7ff520107de0 2024-05-09T02:00:00.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.900+0000 7ff5177fe700 1 -- 172.21.15.19:0/2850812834 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff51c01a430 con 0x7ff520107de0 2024-05-09T02:00:00.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.900+0000 7ff5286a1700 1 -- 172.21.15.19:0/2850812834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff520108cf0 con 0x7ff520107de0 2024-05-09T02:00:00.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.900+0000 7ff5177fe700 1 -- 172.21.15.19:0/2850812834 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff51c019550 con 0x7ff520107de0 2024-05-09T02:00:00.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.900+0000 7ff5286a1700 1 -- 172.21.15.19:0/2850812834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5201091d0 con 0x7ff520107de0 2024-05-09T02:00:00.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.901+0000 7ff5286a1700 1 -- 172.21.15.19:0/2850812834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff520064e20 con 0x7ff520107de0 2024-05-09T02:00:00.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.901+0000 7ff5177fe700 1 -- 172.21.15.19:0/2850812834 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7ff51c019810 con 0x7ff520107de0 2024-05-09T02:00:00.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.902+0000 7ff5177fe700 1 --2- 172.21.15.19:0/2850812834 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7ff50c03e4e0 0x7ff50c0409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:00.757 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.902+0000 7ff5177fe700 1 -- 172.21.15.19:0/2850812834 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff51c0537e0 con 0x7ff520107de0 2024-05-09T02:00:00.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.902+0000 7ff525c3c700 1 --2- 172.21.15.19:0/2850812834 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7ff50c03e4e0 0x7ff50c0409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:00.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.902+0000 7ff525c3c700 1 --2- 172.21.15.19:0/2850812834 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7ff50c03e4e0 0x7ff50c0409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff510006fd0 tx=0x7ff510006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:00.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T01:59:59.907+0000 7ff5177fe700 1 -- 172.21.15.19:0/2850812834 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff51c017e90 con 0x7ff520107de0 2024-05-09T02:00:00.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:00.126+0000 7ff5286a1700 1 -- 172.21.15.19:0/2850812834 --> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7ff520109860 con 0x7ff50c03e4e0 2024-05-09T02:00:00.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:00.126+0000 7ff5177fe700 1 -- 172.21.15.19:0/2850812834 <== mgr.14120 v2:172.21.15.19:6800/2101052000 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7ff520109860 con 0x7ff50c03e4e0 2024-05-09T02:00:00.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:00.128+0000 7ff5286a1700 1 -- 172.21.15.19:0/2850812834 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7ff50c03e4e0 msgr2=0x7ff50c0409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:00.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:00.128+0000 7ff5286a1700 1 --2- 172.21.15.19:0/2850812834 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7ff50c03e4e0 0x7ff50c0409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff510006fd0 tx=0x7ff510006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:00.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:00.129+0000 7ff5286a1700 1 -- 172.21.15.19:0/2850812834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff520107de0 msgr2=0x7ff52010a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:00.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:00.129+0000 7ff5286a1700 1 --2- 172.21.15.19:0/2850812834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff520107de0 0x7ff52010a560 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff51c011670 tx=0x7ff51c004280 comp rx=0 tx=0).stop 2024-05-09T02:00:00.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:00.129+0000 7ff5286a1700 1 -- 172.21.15.19:0/2850812834 shutdown_connections 2024-05-09T02:00:00.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:00.129+0000 7ff5286a1700 1 --2- 172.21.15.19:0/2850812834 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7ff50c03e4e0 0x7ff50c0409f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:00.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:00.129+0000 7ff5286a1700 1 --2- 172.21.15.19:0/2850812834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff520107de0 0x7ff52010a560 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:00.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:00.129+0000 7ff5286a1700 1 -- 172.21.15.19:0/2850812834 >> 172.21.15.19:0/2850812834 conn(0x7ff520103810 msgr2=0x7ff5201040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:00.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:00.129+0000 7ff5286a1700 1 -- 172.21.15.19:0/2850812834 shutdown_connections 2024-05-09T02:00:00.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:00.129+0000 7ff5286a1700 1 -- 172.21.15.19:0/2850812834 wait complete. 2024-05-09T02:00:00.759 INFO:teuthology.orchestra.run.smithi019.stdout:Generating ssh key... 2024-05-09T02:00:01.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:01 smithi019 ceph-mon[30151]: [09/May/2024:02:00:00] ENGINE Serving on https://172.21.15.19:7150 2024-05-09T02:00:01.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:01 smithi019 ceph-mon[30151]: [09/May/2024:02:00:00] ENGINE Bus STARTED 2024-05-09T02:00:01.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:01 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' 2024-05-09T02:00:01.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:01 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' 2024-05-09T02:00:01.820 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: Generating public/private rsa key pair. 2024-05-09T02:00:01.820 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: Your identification has been saved in /tmp/tmpu7bv9bd5/key. 2024-05-09T02:00:01.820 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: Your public key has been saved in /tmp/tmpu7bv9bd5/key.pub. 2024-05-09T02:00:01.821 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: The key fingerprint is: 2024-05-09T02:00:01.821 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: SHA256:swyEjKafLy2JutTbwkGzUJTRoH+rbprnDjeBzig6CSs ceph-a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:00:01.821 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: The key's randomart image is: 2024-05-09T02:00:01.821 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: +---[RSA 3072]----+ 2024-05-09T02:00:01.821 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: | .+= | 2024-05-09T02:00:01.821 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: | .= o | 2024-05-09T02:00:01.821 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: | .+ o . | 2024-05-09T02:00:01.821 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: | +oo . | 2024-05-09T02:00:01.821 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: |..ooo.. S | 2024-05-09T02:00:01.821 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: |=..+o .o o | 2024-05-09T02:00:01.821 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: |+O+*.. o | 2024-05-09T02:00:01.821 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: |E B** | 2024-05-09T02:00:01.821 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: |*=BOo. | 2024-05-09T02:00:01.821 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: +----[SHA256]-----+ 2024-05-09T02:00:02.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.196+0000 7f3da0c0a700 1 Processor -- start 2024-05-09T02:00:02.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.196+0000 7f3da0c0a700 1 -- start start 2024-05-09T02:00:02.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.196+0000 7f3da0c0a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108a00 0x7f3d9c108e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:02.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.196+0000 7f3da0c0a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c109400 con 0x7f3d9c108a00 2024-05-09T02:00:02.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.196+0000 7f3d9a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108a00 0x7f3d9c108e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:02.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.196+0000 7f3d9a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108a00 0x7f3d9c108e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34504/0 (socket says 172.21.15.19:34504) 2024-05-09T02:00:02.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.196+0000 7f3d9a59c700 1 -- 172.21.15.19:0/463406452 learned_addr learned my addr 172.21.15.19:0/463406452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:02.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.197+0000 7f3d9a59c700 1 -- 172.21.15.19:0/463406452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c109bd0 con 0x7f3d9c108a00 2024-05-09T02:00:02.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.197+0000 7f3d9a59c700 1 --2- 172.21.15.19:0/463406452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108a00 0x7f3d9c108e10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0098a0 tx=0x7f3d8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23e3da4bf8cb941 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:02.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.197+0000 7f3d9959a700 1 -- 172.21.15.19:0/463406452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d8c004160 con 0x7f3d9c108a00 2024-05-09T02:00:02.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.197+0000 7f3d9959a700 1 -- 172.21.15.19:0/463406452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3d8c004300 con 0x7f3d9c108a00 2024-05-09T02:00:02.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.197+0000 7f3d9959a700 1 -- 172.21.15.19:0/463406452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d8c011af0 con 0x7f3d9c108a00 2024-05-09T02:00:02.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.198+0000 7f3da0c0a700 1 -- 172.21.15.19:0/463406452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108a00 msgr2=0x7f3d9c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:02.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.198+0000 7f3da0c0a700 1 --2- 172.21.15.19:0/463406452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108a00 0x7f3d9c108e10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0098a0 tx=0x7f3d8c009bb0 comp rx=0 tx=0).stop 2024-05-09T02:00:02.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.198+0000 7f3da0c0a700 1 -- 172.21.15.19:0/463406452 shutdown_connections 2024-05-09T02:00:02.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.198+0000 7f3da0c0a700 1 --2- 172.21.15.19:0/463406452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108a00 0x7f3d9c108e10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:02.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.198+0000 7f3da0c0a700 1 -- 172.21.15.19:0/463406452 >> 172.21.15.19:0/463406452 conn(0x7f3d9c1040e0 msgr2=0x7f3d9c106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:02.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.198+0000 7f3da0c0a700 1 -- 172.21.15.19:0/463406452 shutdown_connections 2024-05-09T02:00:02.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.198+0000 7f3da0c0a700 1 -- 172.21.15.19:0/463406452 wait complete. 2024-05-09T02:00:02.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.199+0000 7f3da0c0a700 1 Processor -- start 2024-05-09T02:00:02.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.199+0000 7f3da0c0a700 1 -- start start 2024-05-09T02:00:02.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.199+0000 7f3da0c0a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108a00 0x7f3d9c07d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:02.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.199+0000 7f3da0c0a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c10cf80 con 0x7f3d9c108a00 2024-05-09T02:00:02.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.199+0000 7f3d9a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108a00 0x7f3d9c07d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:02.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.199+0000 7f3d9a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108a00 0x7f3d9c07d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34514/0 (socket says 172.21.15.19:34514) 2024-05-09T02:00:02.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.199+0000 7f3d9a59c700 1 -- 172.21.15.19:0/2440910162 learned_addr learned my addr 172.21.15.19:0/2440910162 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:02.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.200+0000 7f3d9a59c700 1 -- 172.21.15.19:0/2440910162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c19ea80 con 0x7f3d9c108a00 2024-05-09T02:00:02.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.200+0000 7f3d9a59c700 1 --2- 172.21.15.19:0/2440910162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108a00 0x7f3d9c07d1c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c011670 tx=0x7f3d8c004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:02.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.200+0000 7f3d8b7fe700 1 -- 172.21.15.19:0/2440910162 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d8c0036a0 con 0x7f3d9c108a00 2024-05-09T02:00:02.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.200+0000 7f3da0c0a700 1 -- 172.21.15.19:0/2440910162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9c07d700 con 0x7f3d9c108a00 2024-05-09T02:00:02.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.200+0000 7f3d8b7fe700 1 -- 172.21.15.19:0/2440910162 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3d8c003cd0 con 0x7f3d9c108a00 2024-05-09T02:00:02.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.200+0000 7f3d8b7fe700 1 -- 172.21.15.19:0/2440910162 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d8c0193c0 con 0x7f3d9c108a00 2024-05-09T02:00:02.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.200+0000 7f3da0c0a700 1 -- 172.21.15.19:0/2440910162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9c1a3340 con 0x7f3d9c108a00 2024-05-09T02:00:02.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.201+0000 7f3da0c0a700 1 -- 172.21.15.19:0/2440910162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9c064e20 con 0x7f3d9c108a00 2024-05-09T02:00:02.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.201+0000 7f3d8b7fe700 1 -- 172.21.15.19:0/2440910162 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3d8c019680 con 0x7f3d9c108a00 2024-05-09T02:00:02.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.201+0000 7f3d8b7fe700 1 --2- 172.21.15.19:0/2440910162 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f3d8403e4e0 0x7f3d840409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:02.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.202+0000 7f3d8b7fe700 1 -- 172.21.15.19:0/2440910162 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3d8c0536c0 con 0x7f3d9c108a00 2024-05-09T02:00:02.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.202+0000 7f3d99d9b700 1 --2- 172.21.15.19:0/2440910162 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f3d8403e4e0 0x7f3d840409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:02.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.202+0000 7f3d99d9b700 1 --2- 172.21.15.19:0/2440910162 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f3d8403e4e0 0x7f3d840409f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3d90006fd0 tx=0x7f3d90006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:02.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.207+0000 7f3d8b7fe700 1 -- 172.21.15.19:0/2440910162 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3d8c0243f0 con 0x7f3d9c108a00 2024-05-09T02:00:02.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.423+0000 7f3da0c0a700 1 -- 172.21.15.19:0/2440910162 --> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f3d9c1060a0 con 0x7f3d8403e4e0 2024-05-09T02:00:02.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.629+0000 7f3d8b7fe700 1 -- 172.21.15.19:0/2440910162 <== mgr.14120 v2:172.21.15.19:6800/2101052000 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3d9c1060a0 con 0x7f3d8403e4e0 2024-05-09T02:00:02.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.631+0000 7f3da0c0a700 1 -- 172.21.15.19:0/2440910162 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f3d8403e4e0 msgr2=0x7f3d840409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:02.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.631+0000 7f3da0c0a700 1 --2- 172.21.15.19:0/2440910162 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f3d8403e4e0 0x7f3d840409f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3d90006fd0 tx=0x7f3d90006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:02.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.631+0000 7f3da0c0a700 1 -- 172.21.15.19:0/2440910162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108a00 msgr2=0x7f3d9c07d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:02.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.631+0000 7f3da0c0a700 1 --2- 172.21.15.19:0/2440910162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108a00 0x7f3d9c07d1c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c011670 tx=0x7f3d8c004280 comp rx=0 tx=0).stop 2024-05-09T02:00:02.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.631+0000 7f3da0c0a700 1 -- 172.21.15.19:0/2440910162 shutdown_connections 2024-05-09T02:00:02.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.631+0000 7f3da0c0a700 1 --2- 172.21.15.19:0/2440910162 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f3d8403e4e0 0x7f3d840409f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:02.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.631+0000 7f3da0c0a700 1 --2- 172.21.15.19:0/2440910162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9c108a00 0x7f3d9c07d1c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:02.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.631+0000 7f3da0c0a700 1 -- 172.21.15.19:0/2440910162 >> 172.21.15.19:0/2440910162 conn(0x7f3d9c1040e0 msgr2=0x7f3d9c105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:02.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.631+0000 7f3da0c0a700 1 -- 172.21.15.19:0/2440910162 shutdown_connections 2024-05-09T02:00:02.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:01.631+0000 7f3da0c0a700 1 -- 172.21.15.19:0/2440910162 wait complete. 2024-05-09T02:00:02.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:02 smithi019 ceph-mon[30151]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:00:02.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:02 smithi019 ceph-mon[30151]: Generating ssh key... 2024-05-09T02:00:03.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbWclUMyiQoX9eX9b3TOTneWODYIhWw4CY1M38viaSXZwm1T5QRprH+EIXg7ceA5s+WTWdZ7LPTq/oxjmu6hU81vZKTVrLvIcZbkfJPRbSkNvs03I+JOBIP+dlcELLOFTxRu8HgnXUO6mDHmwA00pXQH6cKvWgbYo+O1JkG7FnVVZjxhvb5wOu8QpLaAGZaOcxBBcqaSRG+IEa0nnyBV7lPCDtc4S+f0K2A+Tqmp1X9UKW6mLUlXS7Hw3G5nlUFGIy39DIY+Kn9uSApLa5MyLrPTx+L7Erm5jYGHsqTzhczmFvmk0NEyUMtR3wGOQKDGkeqiIeUKZLvrRHNZQ2Nv7K78g9g9/PCd3G7BzqiuHO+zaNm3nATyB5A2c1oc9G0FneFmUYaVO+EoX2bs3Efbqn010SkH5NZ5QnerMGGAaZvAilcE/iBiRIVAi+a6BpM+RcgSmRo6e2phnKU0yrUnjDuVX/Y9rIXPex+GH240klJayvN4c7Yn2tRWjvcZ2l4TU= ceph-a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:00:03.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.630+0000 7f6582591700 1 Processor -- start 2024-05-09T02:00:03.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.630+0000 7f6582591700 1 -- start start 2024-05-09T02:00:03.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.630+0000 7f6582591700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657c10a610 0x7f657c10ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:03.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.630+0000 7f6582591700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f657c07d830 con 0x7f657c10a610 2024-05-09T02:00:03.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.631+0000 7f657bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657c10a610 0x7f657c10ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:03.452 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.631+0000 7f657bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657c10a610 0x7f657c10ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34520/0 (socket says 172.21.15.19:34520) 2024-05-09T02:00:03.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.631+0000 7f657bfff700 1 -- 172.21.15.19:0/234040887 learned_addr learned my addr 172.21.15.19:0/234040887 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:03.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.631+0000 7f657bfff700 1 -- 172.21.15.19:0/234040887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f657c10cf70 con 0x7f657c10a610 2024-05-09T02:00:03.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.631+0000 7f657bfff700 1 --2- 172.21.15.19:0/234040887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657c10a610 0x7f657c10ca30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f656c0098a0 tx=0x7f656c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4521a27931de2f7c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:03.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.631+0000 7f657affd700 1 -- 172.21.15.19:0/234040887 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f656c004160 con 0x7f657c10a610 2024-05-09T02:00:03.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.631+0000 7f657affd700 1 -- 172.21.15.19:0/234040887 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f656c004300 con 0x7f657c10a610 2024-05-09T02:00:03.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.631+0000 7f657affd700 1 -- 172.21.15.19:0/234040887 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f656c011af0 con 0x7f657c10a610 2024-05-09T02:00:03.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.632+0000 7f6582591700 1 -- 172.21.15.19:0/234040887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657c10a610 msgr2=0x7f657c10ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:03.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.632+0000 7f6582591700 1 --2- 172.21.15.19:0/234040887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657c10a610 0x7f657c10ca30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f656c0098a0 tx=0x7f656c009bb0 comp rx=0 tx=0).stop 2024-05-09T02:00:03.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.632+0000 7f6582591700 1 -- 172.21.15.19:0/234040887 shutdown_connections 2024-05-09T02:00:03.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.632+0000 7f6582591700 1 --2- 172.21.15.19:0/234040887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657c10a610 0x7f657c10ca30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:03.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.632+0000 7f6582591700 1 -- 172.21.15.19:0/234040887 >> 172.21.15.19:0/234040887 conn(0x7f657c104010 msgr2=0x7f657c1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:03.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.632+0000 7f6582591700 1 -- 172.21.15.19:0/234040887 shutdown_connections 2024-05-09T02:00:03.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.632+0000 7f6582591700 1 -- 172.21.15.19:0/234040887 wait complete. 2024-05-09T02:00:03.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.633+0000 7f6582591700 1 Processor -- start 2024-05-09T02:00:03.453 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.633+0000 7f6582591700 1 -- start start 2024-05-09T02:00:03.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.633+0000 7f6582591700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657c10a610 0x7f657c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:03.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.633+0000 7f6582591700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f657c10ded0 con 0x7f657c10a610 2024-05-09T02:00:03.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.633+0000 7f657bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657c10a610 0x7f657c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:03.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.633+0000 7f657bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657c10a610 0x7f657c19de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34536/0 (socket says 172.21.15.19:34536) 2024-05-09T02:00:03.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.633+0000 7f657bfff700 1 -- 172.21.15.19:0/156424947 learned_addr learned my addr 172.21.15.19:0/156424947 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:03.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.634+0000 7f657bfff700 1 -- 172.21.15.19:0/156424947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f657c19e360 con 0x7f657c10a610 2024-05-09T02:00:03.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.634+0000 7f657bfff700 1 --2- 172.21.15.19:0/156424947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657c10a610 0x7f657c19de20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f656c0117b0 tx=0x7f656c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:03.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.634+0000 7f65797fa700 1 -- 172.21.15.19:0/156424947 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f656c0039c0 con 0x7f657c10a610 2024-05-09T02:00:03.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.634+0000 7f65797fa700 1 -- 172.21.15.19:0/156424947 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f656c003b60 con 0x7f657c10a610 2024-05-09T02:00:03.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.634+0000 7f6582591700 1 -- 172.21.15.19:0/156424947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f657c19e5f0 con 0x7f657c10a610 2024-05-09T02:00:03.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.634+0000 7f65797fa700 1 -- 172.21.15.19:0/156424947 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f656c0196a0 con 0x7f657c10a610 2024-05-09T02:00:03.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.634+0000 7f6582591700 1 -- 172.21.15.19:0/156424947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f657c1a1200 con 0x7f657c10a610 2024-05-09T02:00:03.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.635+0000 7f6582591700 1 -- 172.21.15.19:0/156424947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f657c064e20 con 0x7f657c10a610 2024-05-09T02:00:03.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.635+0000 7f65797fa700 1 -- 172.21.15.19:0/156424947 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f656c019960 con 0x7f657c10a610 2024-05-09T02:00:03.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.635+0000 7f65797fa700 1 --2- 172.21.15.19:0/156424947 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f656803e530 0x7f6568040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:03.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.635+0000 7f65797fa700 1 -- 172.21.15.19:0/156424947 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f656c053930 con 0x7f657c10a610 2024-05-09T02:00:03.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.636+0000 7f657b7fe700 1 --2- 172.21.15.19:0/156424947 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f656803e530 0x7f6568040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:03.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.636+0000 7f657b7fe700 1 --2- 172.21.15.19:0/156424947 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f656803e530 0x7f6568040a40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6564006fd0 tx=0x7f6564006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:03.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.641+0000 7f65797fa700 1 -- 172.21.15.19:0/156424947 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f656c00b3e0 con 0x7f657c10a610 2024-05-09T02:00:03.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.857+0000 7f6582591700 1 -- 172.21.15.19:0/156424947 --> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f657c104fe0 con 0x7f656803e530 2024-05-09T02:00:03.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.858+0000 7f65797fa700 1 -- 172.21.15.19:0/156424947 <== mgr.14120 v2:172.21.15.19:6800/2101052000 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f657c104fe0 con 0x7f656803e530 2024-05-09T02:00:03.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.860+0000 7f6582591700 1 -- 172.21.15.19:0/156424947 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f656803e530 msgr2=0x7f6568040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:03.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.860+0000 7f6582591700 1 --2- 172.21.15.19:0/156424947 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f656803e530 0x7f6568040a40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6564006fd0 tx=0x7f6564006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:03.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.860+0000 7f6582591700 1 -- 172.21.15.19:0/156424947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657c10a610 msgr2=0x7f657c19de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:03.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.860+0000 7f6582591700 1 --2- 172.21.15.19:0/156424947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657c10a610 0x7f657c19de20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f656c0117b0 tx=0x7f656c011b70 comp rx=0 tx=0).stop 2024-05-09T02:00:03.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.860+0000 7f6582591700 1 -- 172.21.15.19:0/156424947 shutdown_connections 2024-05-09T02:00:03.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.860+0000 7f6582591700 1 --2- 172.21.15.19:0/156424947 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f656803e530 0x7f6568040a40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:03.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.860+0000 7f6582591700 1 --2- 172.21.15.19:0/156424947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657c10a610 0x7f657c19de20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:03.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.860+0000 7f6582591700 1 -- 172.21.15.19:0/156424947 >> 172.21.15.19:0/156424947 conn(0x7f657c104010 msgr2=0x7f657c1048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:03.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.860+0000 7f6582591700 1 -- 172.21.15.19:0/156424947 shutdown_connections 2024-05-09T02:00:03.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:02.860+0000 7f6582591700 1 -- 172.21.15.19:0/156424947 wait complete. 2024-05-09T02:00:03.456 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-09T02:00:03.456 INFO:teuthology.orchestra.run.smithi019.stdout:Adding key to root@localhost authorized_keys... 2024-05-09T02:00:03.456 INFO:teuthology.orchestra.run.smithi019.stdout:Adding host smithi019... 2024-05-09T02:00:03.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:03 smithi019 ceph-mon[30151]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:00:04.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:04 smithi019 ceph-mon[30151]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:00:06.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:05 smithi019 ceph-mon[30151]: Deploying cephadm binary to smithi019 2024-05-09T02:00:08.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Added host 'smithi019' with addr '172.21.15.19' 2024-05-09T02:00:08.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.886+0000 7f719470f700 1 Processor -- start 2024-05-09T02:00:08.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.886+0000 7f719470f700 1 -- start start 2024-05-09T02:00:08.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.886+0000 7f719470f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c109ae0 0x7f718c109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:08.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.886+0000 7f719470f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f718c10a4e0 con 0x7f718c109ae0 2024-05-09T02:00:08.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.886+0000 7f71924ab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c109ae0 0x7f718c109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:08.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.887+0000 7f71924ab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c109ae0 0x7f718c109ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34540/0 (socket says 172.21.15.19:34540) 2024-05-09T02:00:08.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.887+0000 7f71924ab700 1 -- 172.21.15.19:0/744686505 learned_addr learned my addr 172.21.15.19:0/744686505 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:08.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.887+0000 7f71924ab700 1 -- 172.21.15.19:0/744686505 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c10acb0 con 0x7f718c109ae0 2024-05-09T02:00:08.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.887+0000 7f71924ab700 1 --2- 172.21.15.19:0/744686505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c109ae0 0x7f718c109ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f71880098a0 tx=0x7f7188009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ccb11e0bcea9b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:08.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.887+0000 7f71914a9700 1 -- 172.21.15.19:0/744686505 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7188004160 con 0x7f718c109ae0 2024-05-09T02:00:08.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.887+0000 7f71914a9700 1 -- 172.21.15.19:0/744686505 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7188004300 con 0x7f718c109ae0 2024-05-09T02:00:08.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.887+0000 7f71914a9700 1 -- 172.21.15.19:0/744686505 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7188011af0 con 0x7f718c109ae0 2024-05-09T02:00:08.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.888+0000 7f719470f700 1 -- 172.21.15.19:0/744686505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c109ae0 msgr2=0x7f718c109ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:08.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.888+0000 7f719470f700 1 --2- 172.21.15.19:0/744686505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c109ae0 0x7f718c109ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f71880098a0 tx=0x7f7188009bb0 comp rx=0 tx=0).stop 2024-05-09T02:00:08.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.888+0000 7f719470f700 1 -- 172.21.15.19:0/744686505 shutdown_connections 2024-05-09T02:00:08.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.888+0000 7f719470f700 1 --2- 172.21.15.19:0/744686505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c109ae0 0x7f718c109ef0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:08.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.888+0000 7f719470f700 1 -- 172.21.15.19:0/744686505 >> 172.21.15.19:0/744686505 conn(0x7f718c105160 msgr2=0x7f718c1075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:08.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.888+0000 7f719470f700 1 -- 172.21.15.19:0/744686505 shutdown_connections 2024-05-09T02:00:08.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.888+0000 7f719470f700 1 -- 172.21.15.19:0/744686505 wait complete. 2024-05-09T02:00:08.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.889+0000 7f719470f700 1 Processor -- start 2024-05-09T02:00:08.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.889+0000 7f719470f700 1 -- start start 2024-05-09T02:00:08.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.889+0000 7f719470f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c109ae0 0x7f718c1a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:08.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.889+0000 7f719470f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f718c07d240 con 0x7f718c109ae0 2024-05-09T02:00:08.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.889+0000 7f71924ab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c109ae0 0x7f718c1a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:08.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.889+0000 7f71924ab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c109ae0 0x7f718c1a0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34550/0 (socket says 172.21.15.19:34550) 2024-05-09T02:00:08.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.889+0000 7f71924ab700 1 -- 172.21.15.19:0/72171730 learned_addr learned my addr 172.21.15.19:0/72171730 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:08.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.890+0000 7f71924ab700 1 -- 172.21.15.19:0/72171730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c1a05c0 con 0x7f718c109ae0 2024-05-09T02:00:08.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.890+0000 7f71924ab700 1 --2- 172.21.15.19:0/72171730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c109ae0 0x7f718c1a0080 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f71880090e0 tx=0x7f7188011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:08.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.890+0000 7f71837fe700 1 -- 172.21.15.19:0/72171730 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71880039e0 con 0x7f718c109ae0 2024-05-09T02:00:08.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.890+0000 7f719470f700 1 -- 172.21.15.19:0/72171730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718c1a0850 con 0x7f718c109ae0 2024-05-09T02:00:08.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.890+0000 7f71837fe700 1 -- 172.21.15.19:0/72171730 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7188003b80 con 0x7f718c109ae0 2024-05-09T02:00:08.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.890+0000 7f71837fe700 1 -- 172.21.15.19:0/72171730 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71880246e0 con 0x7f718c109ae0 2024-05-09T02:00:08.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.890+0000 7f719470f700 1 -- 172.21.15.19:0/72171730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f718c1a3460 con 0x7f718c109ae0 2024-05-09T02:00:08.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.891+0000 7f719470f700 1 -- 172.21.15.19:0/72171730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f718c064e20 con 0x7f718c109ae0 2024-05-09T02:00:08.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.891+0000 7f71837fe700 1 -- 172.21.15.19:0/72171730 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7188022020 con 0x7f718c109ae0 2024-05-09T02:00:08.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.892+0000 7f71837fe700 1 --2- 172.21.15.19:0/72171730 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f717803e530 0x7f7178040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:08.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.892+0000 7f71837fe700 1 -- 172.21.15.19:0/72171730 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f71880538a0 con 0x7f718c109ae0 2024-05-09T02:00:08.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.892+0000 7f7191caa700 1 --2- 172.21.15.19:0/72171730 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f717803e530 0x7f7178040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:08.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.892+0000 7f7191caa700 1 --2- 172.21.15.19:0/72171730 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f717803e530 0x7f7178040a40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f717c006fd0 tx=0x7f717c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:08.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:03.897+0000 7f71837fe700 1 -- 172.21.15.19:0/72171730 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f71880172b0 con 0x7f718c109ae0 2024-05-09T02:00:08.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:04.117+0000 7f719470f700 1 -- 172.21.15.19:0/72171730 --> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}) -- 0x7f718c063c00 con 0x7f717803e530 2024-05-09T02:00:08.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:08.390+0000 7f71837fe700 1 -- 172.21.15.19:0/72171730 <== mgr.14120 v2:172.21.15.19:6800/2101052000 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f718c063c00 con 0x7f717803e530 2024-05-09T02:00:08.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:08.393+0000 7f719470f700 1 -- 172.21.15.19:0/72171730 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f717803e530 msgr2=0x7f7178040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:08.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:08.393+0000 7f719470f700 1 --2- 172.21.15.19:0/72171730 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f717803e530 0x7f7178040a40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f717c006fd0 tx=0x7f717c006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:08.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:08.393+0000 7f719470f700 1 -- 172.21.15.19:0/72171730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c109ae0 msgr2=0x7f718c1a0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:08.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:08.393+0000 7f719470f700 1 --2- 172.21.15.19:0/72171730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c109ae0 0x7f718c1a0080 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f71880090e0 tx=0x7f7188011940 comp rx=0 tx=0).stop 2024-05-09T02:00:08.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:08.393+0000 7f719470f700 1 -- 172.21.15.19:0/72171730 shutdown_connections 2024-05-09T02:00:08.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:08.393+0000 7f719470f700 1 --2- 172.21.15.19:0/72171730 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f717803e530 0x7f7178040a40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:08.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:08.393+0000 7f719470f700 1 --2- 172.21.15.19:0/72171730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f718c109ae0 0x7f718c1a0080 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:08.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:08.393+0000 7f719470f700 1 -- 172.21.15.19:0/72171730 >> 172.21.15.19:0/72171730 conn(0x7f718c105160 msgr2=0x7f718c1075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:08.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:08.393+0000 7f719470f700 1 -- 172.21.15.19:0/72171730 shutdown_connections 2024-05-09T02:00:08.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:08.393+0000 7f719470f700 1 -- 172.21.15.19:0/72171730 wait complete. 2024-05-09T02:00:08.971 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mon service... 2024-05-09T02:00:09.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:09 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' 2024-05-09T02:00:09.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:09 smithi019 ceph-mon[30151]: Added host smithi019 2024-05-09T02:00:09.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:09 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:00:10.418 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-09T02:00:10.418 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.441+0000 7fcbccddb700 1 Processor -- start 2024-05-09T02:00:10.418 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.441+0000 7fcbccddb700 1 -- start start 2024-05-09T02:00:10.418 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.442+0000 7fcbccddb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8107de0 0x7fcbc81081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:10.418 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.442+0000 7fcbccddb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81087c0 con 0x7fcbc8107de0 2024-05-09T02:00:10.418 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.442+0000 7fcbc659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8107de0 0x7fcbc81081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:10.418 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.442+0000 7fcbc659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8107de0 0x7fcbc81081f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58456/0 (socket says 172.21.15.19:58456) 2024-05-09T02:00:10.418 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.442+0000 7fcbc659c700 1 -- 172.21.15.19:0/1176733087 learned_addr learned my addr 172.21.15.19:0/1176733087 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:10.418 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.442+0000 7fcbc659c700 1 -- 172.21.15.19:0/1176733087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc810ade0 con 0x7fcbc8107de0 2024-05-09T02:00:10.418 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.442+0000 7fcbc659c700 1 --2- 172.21.15.19:0/1176733087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8107de0 0x7fcbc81081f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80098a0 tx=0x7fcbb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f889e9506a4b9b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:10.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.443+0000 7fcbc559a700 1 -- 172.21.15.19:0/1176733087 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbb8004160 con 0x7fcbc8107de0 2024-05-09T02:00:10.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.443+0000 7fcbc559a700 1 -- 172.21.15.19:0/1176733087 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcbb8004300 con 0x7fcbc8107de0 2024-05-09T02:00:10.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.443+0000 7fcbc559a700 1 -- 172.21.15.19:0/1176733087 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbb8011af0 con 0x7fcbc8107de0 2024-05-09T02:00:10.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.443+0000 7fcbccddb700 1 -- 172.21.15.19:0/1176733087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8107de0 msgr2=0x7fcbc81081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:10.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.443+0000 7fcbccddb700 1 --2- 172.21.15.19:0/1176733087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8107de0 0x7fcbc81081f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80098a0 tx=0x7fcbb8009bb0 comp rx=0 tx=0).stop 2024-05-09T02:00:10.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.444+0000 7fcbccddb700 1 -- 172.21.15.19:0/1176733087 shutdown_connections 2024-05-09T02:00:10.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.444+0000 7fcbccddb700 1 --2- 172.21.15.19:0/1176733087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8107de0 0x7fcbc81081f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:10.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.444+0000 7fcbccddb700 1 -- 172.21.15.19:0/1176733087 >> 172.21.15.19:0/1176733087 conn(0x7fcbc81037b0 msgr2=0x7fcbc8105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:10.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.444+0000 7fcbccddb700 1 -- 172.21.15.19:0/1176733087 shutdown_connections 2024-05-09T02:00:10.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.444+0000 7fcbccddb700 1 -- 172.21.15.19:0/1176733087 wait complete. 2024-05-09T02:00:10.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.444+0000 7fcbccddb700 1 Processor -- start 2024-05-09T02:00:10.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.444+0000 7fcbccddb700 1 -- start start 2024-05-09T02:00:10.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.445+0000 7fcbccddb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8107de0 0x7fcbc819e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:10.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.445+0000 7fcbccddb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc810e010 con 0x7fcbc8107de0 2024-05-09T02:00:10.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.445+0000 7fcbc659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8107de0 0x7fcbc819e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:10.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.445+0000 7fcbc659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8107de0 0x7fcbc819e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58466/0 (socket says 172.21.15.19:58466) 2024-05-09T02:00:10.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.445+0000 7fcbc659c700 1 -- 172.21.15.19:0/1207238984 learned_addr learned my addr 172.21.15.19:0/1207238984 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:10.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.445+0000 7fcbc659c700 1 -- 172.21.15.19:0/1207238984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc819e680 con 0x7fcbc8107de0 2024-05-09T02:00:10.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.445+0000 7fcbc659c700 1 --2- 172.21.15.19:0/1207238984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8107de0 0x7fcbc819e140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80090e0 tx=0x7fcbb8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:10.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.445+0000 7fcbb77fe700 1 -- 172.21.15.19:0/1207238984 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbb80039e0 con 0x7fcbc8107de0 2024-05-09T02:00:10.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.445+0000 7fcbb77fe700 1 -- 172.21.15.19:0/1207238984 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcbb8003b80 con 0x7fcbc8107de0 2024-05-09T02:00:10.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.446+0000 7fcbb77fe700 1 -- 172.21.15.19:0/1207238984 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbb80246e0 con 0x7fcbc8107de0 2024-05-09T02:00:10.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.446+0000 7fcbccddb700 1 -- 172.21.15.19:0/1207238984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc819e910 con 0x7fcbc8107de0 2024-05-09T02:00:10.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.446+0000 7fcbccddb700 1 -- 172.21.15.19:0/1207238984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc81a1520 con 0x7fcbc8107de0 2024-05-09T02:00:10.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.447+0000 7fcbb77fe700 1 -- 172.21.15.19:0/1207238984 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fcbb8022020 con 0x7fcbc8107de0 2024-05-09T02:00:10.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.447+0000 7fcbccddb700 1 -- 172.21.15.19:0/1207238984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc8064e20 con 0x7fcbc8107de0 2024-05-09T02:00:10.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.447+0000 7fcbb77fe700 1 --2- 172.21.15.19:0/1207238984 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fcbb003e530 0x7fcbb0040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:10.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.447+0000 7fcbb77fe700 1 -- 172.21.15.19:0/1207238984 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcbb8053880 con 0x7fcbc8107de0 2024-05-09T02:00:10.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.447+0000 7fcbc5d9b700 1 --2- 172.21.15.19:0/1207238984 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fcbb003e530 0x7fcbb0040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:10.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.448+0000 7fcbc5d9b700 1 --2- 172.21.15.19:0/1207238984 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fcbb003e530 0x7fcbb0040a40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00ad30 tx=0x7fcbbc0093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:10.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.453+0000 7fcbb77fe700 1 -- 172.21.15.19:0/1207238984 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcbb801a8c0 con 0x7fcbc8107de0 2024-05-09T02:00:10.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.709+0000 7fcbccddb700 1 -- 172.21.15.19:0/1207238984 --> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcbc8104830 con 0x7fcbb003e530 2024-05-09T02:00:10.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.717+0000 7fcbb77fe700 1 -- 172.21.15.19:0/1207238984 <== mgr.14120 v2:172.21.15.19:6800/2101052000 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcbc8104830 con 0x7fcbb003e530 2024-05-09T02:00:10.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.719+0000 7fcbccddb700 1 -- 172.21.15.19:0/1207238984 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fcbb003e530 msgr2=0x7fcbb0040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:10.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.719+0000 7fcbccddb700 1 --2- 172.21.15.19:0/1207238984 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fcbb003e530 0x7fcbb0040a40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00ad30 tx=0x7fcbbc0093f0 comp rx=0 tx=0).stop 2024-05-09T02:00:10.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.719+0000 7fcbccddb700 1 -- 172.21.15.19:0/1207238984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8107de0 msgr2=0x7fcbc819e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:10.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.719+0000 7fcbccddb700 1 --2- 172.21.15.19:0/1207238984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8107de0 0x7fcbc819e140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80090e0 tx=0x7fcbb8011940 comp rx=0 tx=0).stop 2024-05-09T02:00:10.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.719+0000 7fcbccddb700 1 -- 172.21.15.19:0/1207238984 shutdown_connections 2024-05-09T02:00:10.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.719+0000 7fcbccddb700 1 --2- 172.21.15.19:0/1207238984 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fcbb003e530 0x7fcbb0040a40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:10.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.719+0000 7fcbccddb700 1 --2- 172.21.15.19:0/1207238984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbc8107de0 0x7fcbc819e140 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:10.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.719+0000 7fcbccddb700 1 -- 172.21.15.19:0/1207238984 >> 172.21.15.19:0/1207238984 conn(0x7fcbc81037b0 msgr2=0x7fcbc81040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:10.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.719+0000 7fcbccddb700 1 -- 172.21.15.19:0/1207238984 shutdown_connections 2024-05-09T02:00:10.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:09.719+0000 7fcbccddb700 1 -- 172.21.15.19:0/1207238984 wait complete. 2024-05-09T02:00:10.421 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mgr service... 2024-05-09T02:00:10.939 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:10 smithi019 ceph-mon[30151]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:00:10.939 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:10 smithi019 ceph-mon[30151]: Saving service mon spec with placement count:5 2024-05-09T02:00:10.939 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:10 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' 2024-05-09T02:00:12.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:12 smithi019 ceph-mon[30151]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:00:12.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:12 smithi019 ceph-mon[30151]: Saving service mgr spec with placement count:2 2024-05-09T02:00:12.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:12 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' 2024-05-09T02:00:12.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:12 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' 2024-05-09T02:00:12.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-09T02:00:12.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.927+0000 7fcc47816700 1 Processor -- start 2024-05-09T02:00:12.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.927+0000 7fcc47816700 1 -- start start 2024-05-09T02:00:12.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.927+0000 7fcc47816700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40107df0 0x7fcc40108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:12.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.927+0000 7fcc47816700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401087d0 con 0x7fcc40107df0 2024-05-09T02:00:12.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.927+0000 7fcc455b2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40107df0 0x7fcc40108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:12.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.927+0000 7fcc455b2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40107df0 0x7fcc40108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58472/0 (socket says 172.21.15.19:58472) 2024-05-09T02:00:12.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.927+0000 7fcc455b2700 1 -- 172.21.15.19:0/1632598427 learned_addr learned my addr 172.21.15.19:0/1632598427 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:12.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.928+0000 7fcc455b2700 1 -- 172.21.15.19:0/1632598427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4010adf0 con 0x7fcc40107df0 2024-05-09T02:00:12.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.928+0000 7fcc455b2700 1 --2- 172.21.15.19:0/1632598427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40107df0 0x7fcc40108200 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0098a0 tx=0x7fcc3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385ae6274e0d7b06 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:12.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.928+0000 7fcc37fff700 1 -- 172.21.15.19:0/1632598427 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc3c004160 con 0x7fcc40107df0 2024-05-09T02:00:12.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.928+0000 7fcc37fff700 1 -- 172.21.15.19:0/1632598427 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc3c004300 con 0x7fcc40107df0 2024-05-09T02:00:12.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.928+0000 7fcc37fff700 1 -- 172.21.15.19:0/1632598427 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc3c011af0 con 0x7fcc40107df0 2024-05-09T02:00:12.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.929+0000 7fcc47816700 1 -- 172.21.15.19:0/1632598427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40107df0 msgr2=0x7fcc40108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:12.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.929+0000 7fcc47816700 1 --2- 172.21.15.19:0/1632598427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40107df0 0x7fcc40108200 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0098a0 tx=0x7fcc3c009bb0 comp rx=0 tx=0).stop 2024-05-09T02:00:12.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.929+0000 7fcc47816700 1 -- 172.21.15.19:0/1632598427 shutdown_connections 2024-05-09T02:00:12.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.929+0000 7fcc47816700 1 --2- 172.21.15.19:0/1632598427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40107df0 0x7fcc40108200 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:12.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.929+0000 7fcc47816700 1 -- 172.21.15.19:0/1632598427 >> 172.21.15.19:0/1632598427 conn(0x7fcc40103860 msgr2=0x7fcc40105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:12.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.929+0000 7fcc47816700 1 -- 172.21.15.19:0/1632598427 shutdown_connections 2024-05-09T02:00:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.929+0000 7fcc47816700 1 -- 172.21.15.19:0/1632598427 wait complete. 2024-05-09T02:00:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.930+0000 7fcc47816700 1 Processor -- start 2024-05-09T02:00:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.930+0000 7fcc47816700 1 -- start start 2024-05-09T02:00:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.930+0000 7fcc47816700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40107df0 0x7fcc4019e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.930+0000 7fcc47816700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4010e020 con 0x7fcc40107df0 2024-05-09T02:00:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.930+0000 7fcc455b2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40107df0 0x7fcc4019e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.930+0000 7fcc455b2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40107df0 0x7fcc4019e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58488/0 (socket says 172.21.15.19:58488) 2024-05-09T02:00:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.930+0000 7fcc455b2700 1 -- 172.21.15.19:0/245585383 learned_addr learned my addr 172.21.15.19:0/245585383 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.930+0000 7fcc455b2700 1 -- 172.21.15.19:0/245585383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4019e630 con 0x7fcc40107df0 2024-05-09T02:00:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.930+0000 7fcc455b2700 1 --2- 172.21.15.19:0/245585383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40107df0 0x7fcc4019e0f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c011670 tx=0x7fcc3c004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.931+0000 7fcc367fc700 1 -- 172.21.15.19:0/245585383 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc3c0036a0 con 0x7fcc40107df0 2024-05-09T02:00:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.931+0000 7fcc367fc700 1 -- 172.21.15.19:0/245585383 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc3c01a430 con 0x7fcc40107df0 2024-05-09T02:00:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.931+0000 7fcc47816700 1 -- 172.21.15.19:0/245585383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4019e8c0 con 0x7fcc40107df0 2024-05-09T02:00:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.931+0000 7fcc367fc700 1 -- 172.21.15.19:0/245585383 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc3c019550 con 0x7fcc40107df0 2024-05-09T02:00:12.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.931+0000 7fcc47816700 1 -- 172.21.15.19:0/245585383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc401a14d0 con 0x7fcc40107df0 2024-05-09T02:00:12.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.932+0000 7fcc367fc700 1 -- 172.21.15.19:0/245585383 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fcc3c0197c0 con 0x7fcc40107df0 2024-05-09T02:00:12.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.932+0000 7fcc47816700 1 -- 172.21.15.19:0/245585383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc40064e20 con 0x7fcc40107df0 2024-05-09T02:00:12.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.932+0000 7fcc367fc700 1 --2- 172.21.15.19:0/245585383 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fcc2c03e530 0x7fcc2c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:12.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.932+0000 7fcc367fc700 1 -- 172.21.15.19:0/245585383 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcc3c053900 con 0x7fcc40107df0 2024-05-09T02:00:12.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.932+0000 7fcc44db1700 1 --2- 172.21.15.19:0/245585383 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fcc2c03e530 0x7fcc2c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:12.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.933+0000 7fcc44db1700 1 --2- 172.21.15.19:0/245585383 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fcc2c03e530 0x7fcc2c040a40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcc30006fd0 tx=0x7fcc30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:12.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:10.938+0000 7fcc367fc700 1 -- 172.21.15.19:0/245585383 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc3c02ce20 con 0x7fcc40107df0 2024-05-09T02:00:12.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:11.174+0000 7fcc47816700 1 -- 172.21.15.19:0/245585383 --> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcc401058d0 con 0x7fcc2c03e530 2024-05-09T02:00:12.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:11.182+0000 7fcc367fc700 1 -- 172.21.15.19:0/245585383 <== mgr.14120 v2:172.21.15.19:6800/2101052000 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcc401058d0 con 0x7fcc2c03e530 2024-05-09T02:00:12.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:11.184+0000 7fcc47816700 1 -- 172.21.15.19:0/245585383 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fcc2c03e530 msgr2=0x7fcc2c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:12.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:11.184+0000 7fcc47816700 1 --2- 172.21.15.19:0/245585383 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fcc2c03e530 0x7fcc2c040a40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcc30006fd0 tx=0x7fcc30006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:12.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:11.184+0000 7fcc47816700 1 -- 172.21.15.19:0/245585383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40107df0 msgr2=0x7fcc4019e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:12.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:11.184+0000 7fcc47816700 1 --2- 172.21.15.19:0/245585383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40107df0 0x7fcc4019e0f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c011670 tx=0x7fcc3c004280 comp rx=0 tx=0).stop 2024-05-09T02:00:12.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:11.184+0000 7fcc47816700 1 -- 172.21.15.19:0/245585383 shutdown_connections 2024-05-09T02:00:12.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:11.184+0000 7fcc47816700 1 --2- 172.21.15.19:0/245585383 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fcc2c03e530 0x7fcc2c040a40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:12.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:11.184+0000 7fcc47816700 1 --2- 172.21.15.19:0/245585383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc40107df0 0x7fcc4019e0f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:12.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:11.184+0000 7fcc47816700 1 -- 172.21.15.19:0/245585383 >> 172.21.15.19:0/245585383 conn(0x7fcc40103860 msgr2=0x7fcc401051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:12.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:11.185+0000 7fcc47816700 1 -- 172.21.15.19:0/245585383 shutdown_connections 2024-05-09T02:00:12.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:11.185+0000 7fcc47816700 1 -- 172.21.15.19:0/245585383 wait complete. 2024-05-09T02:00:13.647 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.856+0000 7fe6becdd700 1 Processor -- start 2024-05-09T02:00:13.647 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.856+0000 7fe6becdd700 1 -- start start 2024-05-09T02:00:13.647 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.856+0000 7fe6becdd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b8109af0 0x7fe6b8109f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:13.647 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.856+0000 7fe6becdd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b810a4f0 con 0x7fe6b8109af0 2024-05-09T02:00:13.647 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.856+0000 7fe6bca79700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b8109af0 0x7fe6b8109f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:13.647 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.856+0000 7fe6bca79700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b8109af0 0x7fe6b8109f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58496/0 (socket says 172.21.15.19:58496) 2024-05-09T02:00:13.647 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.856+0000 7fe6bca79700 1 -- 172.21.15.19:0/3420862098 learned_addr learned my addr 172.21.15.19:0/3420862098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:13.647 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.857+0000 7fe6bca79700 1 -- 172.21.15.19:0/3420862098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b810acc0 con 0x7fe6b8109af0 2024-05-09T02:00:13.647 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.857+0000 7fe6bca79700 1 --2- 172.21.15.19:0/3420862098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b8109af0 0x7fe6b8109f00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40098a0 tx=0x7fe6b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633d2e3dfe5ac5f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:13.647 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.857+0000 7fe6af7fe700 1 -- 172.21.15.19:0/3420862098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6b4004160 con 0x7fe6b8109af0 2024-05-09T02:00:13.647 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.857+0000 7fe6af7fe700 1 -- 172.21.15.19:0/3420862098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6b4004300 con 0x7fe6b8109af0 2024-05-09T02:00:13.647 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.857+0000 7fe6af7fe700 1 -- 172.21.15.19:0/3420862098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6b4011a20 con 0x7fe6b8109af0 2024-05-09T02:00:13.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.858+0000 7fe6becdd700 1 -- 172.21.15.19:0/3420862098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b8109af0 msgr2=0x7fe6b8109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:13.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.858+0000 7fe6becdd700 1 --2- 172.21.15.19:0/3420862098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b8109af0 0x7fe6b8109f00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40098a0 tx=0x7fe6b4009bb0 comp rx=0 tx=0).stop 2024-05-09T02:00:13.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.858+0000 7fe6becdd700 1 -- 172.21.15.19:0/3420862098 shutdown_connections 2024-05-09T02:00:13.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.858+0000 7fe6becdd700 1 --2- 172.21.15.19:0/3420862098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b8109af0 0x7fe6b8109f00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:13.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.858+0000 7fe6becdd700 1 -- 172.21.15.19:0/3420862098 >> 172.21.15.19:0/3420862098 conn(0x7fe6b8105170 msgr2=0x7fe6b81075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:13.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.858+0000 7fe6becdd700 1 -- 172.21.15.19:0/3420862098 shutdown_connections 2024-05-09T02:00:13.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.858+0000 7fe6becdd700 1 -- 172.21.15.19:0/3420862098 wait complete. 2024-05-09T02:00:13.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.859+0000 7fe6becdd700 1 Processor -- start 2024-05-09T02:00:13.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.859+0000 7fe6becdd700 1 -- start start 2024-05-09T02:00:13.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.859+0000 7fe6becdd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b8109af0 0x7fe6b819dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:13.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.859+0000 7fe6becdd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b810e070 con 0x7fe6b8109af0 2024-05-09T02:00:13.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.859+0000 7fe6bca79700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b8109af0 0x7fe6b819dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:13.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.859+0000 7fe6bca79700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b8109af0 0x7fe6b819dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58506/0 (socket says 172.21.15.19:58506) 2024-05-09T02:00:13.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.859+0000 7fe6bca79700 1 -- 172.21.15.19:0/3562713339 learned_addr learned my addr 172.21.15.19:0/3562713339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:13.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.859+0000 7fe6bca79700 1 -- 172.21.15.19:0/3562713339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b819e400 con 0x7fe6b8109af0 2024-05-09T02:00:13.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.859+0000 7fe6bca79700 1 --2- 172.21.15.19:0/3562713339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b8109af0 0x7fe6b819dec0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4011a20 tx=0x7fe6b4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:13.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.860+0000 7fe6adffb700 1 -- 172.21.15.19:0/3562713339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6b40039c0 con 0x7fe6b8109af0 2024-05-09T02:00:13.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.860+0000 7fe6adffb700 1 -- 172.21.15.19:0/3562713339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6b4003b60 con 0x7fe6b8109af0 2024-05-09T02:00:13.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.860+0000 7fe6becdd700 1 -- 172.21.15.19:0/3562713339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b819e690 con 0x7fe6b8109af0 2024-05-09T02:00:13.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.860+0000 7fe6adffb700 1 -- 172.21.15.19:0/3562713339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6b4019700 con 0x7fe6b8109af0 2024-05-09T02:00:13.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.860+0000 7fe6becdd700 1 -- 172.21.15.19:0/3562713339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b81a12a0 con 0x7fe6b8109af0 2024-05-09T02:00:13.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.861+0000 7fe6becdd700 1 -- 172.21.15.19:0/3562713339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b8064e20 con 0x7fe6b8109af0 2024-05-09T02:00:13.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.861+0000 7fe6adffb700 1 -- 172.21.15.19:0/3562713339 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fe6b40199b0 con 0x7fe6b8109af0 2024-05-09T02:00:13.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.861+0000 7fe6adffb700 1 --2- 172.21.15.19:0/3562713339 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fe6a003e530 0x7fe6a0040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:13.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.861+0000 7fe6adffb700 1 -- 172.21.15.19:0/3562713339 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe6b4053e40 con 0x7fe6b8109af0 2024-05-09T02:00:13.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.862+0000 7fe6affff700 1 --2- 172.21.15.19:0/3562713339 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fe6a003e530 0x7fe6a0040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:13.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.862+0000 7fe6affff700 1 --2- 172.21.15.19:0/3562713339 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fe6a003e530 0x7fe6a0040a40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8006fd0 tx=0x7fe6a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:13.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:12.867+0000 7fe6adffb700 1 -- 172.21.15.19:0/3562713339 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe6b402ce20 con 0x7fe6b8109af0 2024-05-09T02:00:13.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:13.080+0000 7fe6becdd700 1 -- 172.21.15.19:0/3562713339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fe6b81a1590 con 0x7fe6b8109af0 2024-05-09T02:00:13.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:13.085+0000 7fe6adffb700 1 -- 172.21.15.19:0/3562713339 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fe6b4023390 con 0x7fe6b8109af0 2024-05-09T02:00:13.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:13.088+0000 7fe6becdd700 1 -- 172.21.15.19:0/3562713339 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fe6a003e530 msgr2=0x7fe6a0040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:13.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:13.088+0000 7fe6becdd700 1 --2- 172.21.15.19:0/3562713339 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fe6a003e530 0x7fe6a0040a40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8006fd0 tx=0x7fe6a8006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:13.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:13.088+0000 7fe6becdd700 1 -- 172.21.15.19:0/3562713339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b8109af0 msgr2=0x7fe6b819dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:13.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:13.088+0000 7fe6becdd700 1 --2- 172.21.15.19:0/3562713339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b8109af0 0x7fe6b819dec0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4011a20 tx=0x7fe6b4011b00 comp rx=0 tx=0).stop 2024-05-09T02:00:13.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:13.088+0000 7fe6becdd700 1 -- 172.21.15.19:0/3562713339 shutdown_connections 2024-05-09T02:00:13.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:13.088+0000 7fe6becdd700 1 --2- 172.21.15.19:0/3562713339 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fe6a003e530 0x7fe6a0040a40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:13.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:13.088+0000 7fe6becdd700 1 --2- 172.21.15.19:0/3562713339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b8109af0 0x7fe6b819dec0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:13.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:13.088+0000 7fe6becdd700 1 -- 172.21.15.19:0/3562713339 >> 172.21.15.19:0/3562713339 conn(0x7fe6b8105170 msgr2=0x7fe6b81059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:13.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:13.089+0000 7fe6becdd700 1 -- 172.21.15.19:0/3562713339 shutdown_connections 2024-05-09T02:00:13.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:13.089+0000 7fe6becdd700 1 -- 172.21.15.19:0/3562713339 wait complete. 2024-05-09T02:00:13.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:13 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' 2024-05-09T02:00:13.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:13 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3562713339' entity='client.admin' 2024-05-09T02:00:14.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.032+0000 7f03669e0700 1 Processor -- start 2024-05-09T02:00:14.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.032+0000 7f03669e0700 1 -- start start 2024-05-09T02:00:14.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.032+0000 7f03669e0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036010a5b0 0x7f036010c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:14.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.032+0000 7f03669e0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f036007d890 con 0x7f036010a5b0 2024-05-09T02:00:14.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.033+0000 7f035ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036010a5b0 0x7f036010c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:14.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.033+0000 7f035ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036010a5b0 0x7f036010c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58508/0 (socket says 172.21.15.19:58508) 2024-05-09T02:00:14.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.033+0000 7f035ffff700 1 -- 172.21.15.19:0/1613025434 learned_addr learned my addr 172.21.15.19:0/1613025434 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:14.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.033+0000 7f035ffff700 1 -- 172.21.15.19:0/1613025434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036010cf10 con 0x7f036010a5b0 2024-05-09T02:00:14.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.033+0000 7f035ffff700 1 --2- 172.21.15.19:0/1613025434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036010a5b0 0x7f036010c9d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f03500098a0 tx=0x7f0350009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8780bb7e8b2a449a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:14.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.033+0000 7f035f7fe700 1 -- 172.21.15.19:0/1613025434 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0350004160 con 0x7f036010a5b0 2024-05-09T02:00:14.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.033+0000 7f035f7fe700 1 -- 172.21.15.19:0/1613025434 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0350004300 con 0x7f036010a5b0 2024-05-09T02:00:14.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.033+0000 7f035f7fe700 1 -- 172.21.15.19:0/1613025434 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0350011a20 con 0x7f036010a5b0 2024-05-09T02:00:14.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.034+0000 7f03669e0700 1 -- 172.21.15.19:0/1613025434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036010a5b0 msgr2=0x7f036010c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:14.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.034+0000 7f03669e0700 1 --2- 172.21.15.19:0/1613025434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036010a5b0 0x7f036010c9d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f03500098a0 tx=0x7f0350009bb0 comp rx=0 tx=0).stop 2024-05-09T02:00:14.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.034+0000 7f03669e0700 1 -- 172.21.15.19:0/1613025434 shutdown_connections 2024-05-09T02:00:14.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.034+0000 7f03669e0700 1 --2- 172.21.15.19:0/1613025434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036010a5b0 0x7f036010c9d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:14.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.034+0000 7f03669e0700 1 -- 172.21.15.19:0/1613025434 >> 172.21.15.19:0/1613025434 conn(0x7f0360104010 msgr2=0x7f03601064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:14.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.034+0000 7f03669e0700 1 -- 172.21.15.19:0/1613025434 shutdown_connections 2024-05-09T02:00:14.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.034+0000 7f03669e0700 1 -- 172.21.15.19:0/1613025434 wait complete. 2024-05-09T02:00:14.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.035+0000 7f03669e0700 1 Processor -- start 2024-05-09T02:00:14.898 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.035+0000 7f03669e0700 1 -- start start 2024-05-09T02:00:14.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.035+0000 7f03669e0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036010a5b0 0x7f0360109d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:14.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.035+0000 7f03669e0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f036010df10 con 0x7f036010a5b0 2024-05-09T02:00:14.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.035+0000 7f035ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036010a5b0 0x7f0360109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:14.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.035+0000 7f035ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036010a5b0 0x7f0360109d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58510/0 (socket says 172.21.15.19:58510) 2024-05-09T02:00:14.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.035+0000 7f035ffff700 1 -- 172.21.15.19:0/3172082571 learned_addr learned my addr 172.21.15.19:0/3172082571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:14.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.036+0000 7f035ffff700 1 -- 172.21.15.19:0/3172082571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036010a270 con 0x7f036010a5b0 2024-05-09T02:00:14.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.036+0000 7f035ffff700 1 --2- 172.21.15.19:0/3172082571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036010a5b0 0x7f0360109d30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0350009520 tx=0x7f0350011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:14.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.036+0000 7f035dffb700 1 -- 172.21.15.19:0/3172082571 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03500039e0 con 0x7f036010a5b0 2024-05-09T02:00:14.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.036+0000 7f035dffb700 1 -- 172.21.15.19:0/3172082571 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f035001b8f0 con 0x7f036010a5b0 2024-05-09T02:00:14.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.036+0000 7f03669e0700 1 -- 172.21.15.19:0/3172082571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03601084c0 con 0x7f036010a5b0 2024-05-09T02:00:14.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.036+0000 7f035dffb700 1 -- 172.21.15.19:0/3172082571 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03500197d0 con 0x7f036010a5b0 2024-05-09T02:00:14.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.036+0000 7f03669e0700 1 -- 172.21.15.19:0/3172082571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03601089a0 con 0x7f036010a5b0 2024-05-09T02:00:14.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.037+0000 7f035dffb700 1 -- 172.21.15.19:0/3172082571 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0350019a90 con 0x7f036010a5b0 2024-05-09T02:00:14.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.037+0000 7f03669e0700 1 -- 172.21.15.19:0/3172082571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0360064e20 con 0x7f036010a5b0 2024-05-09T02:00:14.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.038+0000 7f035dffb700 1 --2- 172.21.15.19:0/3172082571 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f034c03e580 0x7f034c040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:14.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.038+0000 7f035dffb700 1 -- 172.21.15.19:0/3172082571 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0350054080 con 0x7f036010a5b0 2024-05-09T02:00:14.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.038+0000 7f0357fff700 1 --2- 172.21.15.19:0/3172082571 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f034c03e580 0x7f034c040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:14.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.038+0000 7f0357fff700 1 --2- 172.21.15.19:0/3172082571 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f034c03e580 0x7f034c040a90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0348006fd0 tx=0x7f0348006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:14.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.043+0000 7f035dffb700 1 -- 172.21.15.19:0/3172082571 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f035001f030 con 0x7f036010a5b0 2024-05-09T02:00:14.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.261+0000 7f03669e0700 1 -- 172.21.15.19:0/3172082571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f03601090c0 con 0x7f036010a5b0 2024-05-09T02:00:14.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.264+0000 7f035dffb700 1 -- 172.21.15.19:0/3172082571 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f035000b3e0 con 0x7f036010a5b0 2024-05-09T02:00:14.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.268+0000 7f03669e0700 1 -- 172.21.15.19:0/3172082571 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f034c03e580 msgr2=0x7f034c040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:14.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.268+0000 7f03669e0700 1 --2- 172.21.15.19:0/3172082571 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f034c03e580 0x7f034c040a90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0348006fd0 tx=0x7f0348006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:14.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.268+0000 7f03669e0700 1 -- 172.21.15.19:0/3172082571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036010a5b0 msgr2=0x7f0360109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:14.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.268+0000 7f03669e0700 1 --2- 172.21.15.19:0/3172082571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036010a5b0 0x7f0360109d30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0350009520 tx=0x7f0350011aa0 comp rx=0 tx=0).stop 2024-05-09T02:00:14.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.268+0000 7f03669e0700 1 -- 172.21.15.19:0/3172082571 shutdown_connections 2024-05-09T02:00:14.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.268+0000 7f03669e0700 1 --2- 172.21.15.19:0/3172082571 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f034c03e580 0x7f034c040a90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:14.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.268+0000 7f03669e0700 1 --2- 172.21.15.19:0/3172082571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036010a5b0 0x7f0360109d30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:14.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.268+0000 7f03669e0700 1 -- 172.21.15.19:0/3172082571 >> 172.21.15.19:0/3172082571 conn(0x7f0360104010 msgr2=0x7f0360104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:14.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.268+0000 7f03669e0700 1 -- 172.21.15.19:0/3172082571 shutdown_connections 2024-05-09T02:00:14.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:14.268+0000 7f03669e0700 1 -- 172.21.15.19:0/3172082571 wait complete. 2024-05-09T02:00:14.901 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling the dashboard module... 2024-05-09T02:00:15.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:15 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3172082571' entity='client.admin' 2024-05-09T02:00:16.492 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:16 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1039893175' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-09T02:00:16.493 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:16 smithi019 ceph-mon[30151]: from='mgr.14120 172.21.15.19:0/4002402689' entity='mgr.x' 2024-05-09T02:00:16.493 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: ignoring --setuser ceph since I am not root 2024-05-09T02:00:16.493 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: ignoring --setgroup ceph since I am not root 2024-05-09T02:00:16.493 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:16.491+0000 7f4dbf02d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T02:00:16.800 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:16.689+0000 7f4dbf02d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T02:00:16.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.360+0000 7faa1f9fe700 1 Processor -- start 2024-05-09T02:00:16.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.360+0000 7faa1f9fe700 1 -- start start 2024-05-09T02:00:16.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.360+0000 7faa1f9fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa181089d0 0x7faa18108de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:16.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.360+0000 7faa1f9fe700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa181093d0 con 0x7faa181089d0 2024-05-09T02:00:16.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.361+0000 7faa1d79a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa181089d0 0x7faa18108de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:16.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.361+0000 7faa1d79a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa181089d0 0x7faa18108de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58520/0 (socket says 172.21.15.19:58520) 2024-05-09T02:00:16.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.361+0000 7faa1d79a700 1 -- 172.21.15.19:0/4151133692 learned_addr learned my addr 172.21.15.19:0/4151133692 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:16.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.361+0000 7faa1d79a700 1 -- 172.21.15.19:0/4151133692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa18109ba0 con 0x7faa181089d0 2024-05-09T02:00:16.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.361+0000 7faa1d79a700 1 --2- 172.21.15.19:0/4151133692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa181089d0 0x7faa18108de0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faa14009e80 tx=0x7faa1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c172d873aff902ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:16.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.361+0000 7faa0ffff700 1 -- 172.21.15.19:0/4151133692 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa14017070 con 0x7faa181089d0 2024-05-09T02:00:16.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.361+0000 7faa0ffff700 1 -- 172.21.15.19:0/4151133692 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa1400b8c0 con 0x7faa181089d0 2024-05-09T02:00:16.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.361+0000 7faa0ffff700 1 -- 172.21.15.19:0/4151133692 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa14004490 con 0x7faa181089d0 2024-05-09T02:00:16.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.362+0000 7faa1f9fe700 1 -- 172.21.15.19:0/4151133692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa181089d0 msgr2=0x7faa18108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:16.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.362+0000 7faa1f9fe700 1 --2- 172.21.15.19:0/4151133692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa181089d0 0x7faa18108de0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faa14009e80 tx=0x7faa1400b330 comp rx=0 tx=0).stop 2024-05-09T02:00:16.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.362+0000 7faa1f9fe700 1 -- 172.21.15.19:0/4151133692 shutdown_connections 2024-05-09T02:00:16.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.362+0000 7faa1f9fe700 1 --2- 172.21.15.19:0/4151133692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa181089d0 0x7faa18108de0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:16.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.362+0000 7faa1f9fe700 1 -- 172.21.15.19:0/4151133692 >> 172.21.15.19:0/4151133692 conn(0x7faa18104010 msgr2=0x7faa181064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:16.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.362+0000 7faa1f9fe700 1 -- 172.21.15.19:0/4151133692 shutdown_connections 2024-05-09T02:00:16.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.362+0000 7faa1f9fe700 1 -- 172.21.15.19:0/4151133692 wait complete. 2024-05-09T02:00:16.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.363+0000 7faa1f9fe700 1 Processor -- start 2024-05-09T02:00:16.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.363+0000 7faa1f9fe700 1 -- start start 2024-05-09T02:00:16.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.363+0000 7faa1f9fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa181089d0 0x7faa1819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:16.928 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.363+0000 7faa1f9fe700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa1807cb60 con 0x7faa181089d0 2024-05-09T02:00:16.928 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.363+0000 7faa1d79a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa181089d0 0x7faa1819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:16.928 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.363+0000 7faa1d79a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa181089d0 0x7faa1819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58530/0 (socket says 172.21.15.19:58530) 2024-05-09T02:00:16.928 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.363+0000 7faa1d79a700 1 -- 172.21.15.19:0/1039893175 learned_addr learned my addr 172.21.15.19:0/1039893175 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:16.928 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.364+0000 7faa1d79a700 1 -- 172.21.15.19:0/1039893175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1819c170 con 0x7faa181089d0 2024-05-09T02:00:16.928 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.364+0000 7faa1d79a700 1 --2- 172.21.15.19:0/1039893175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa181089d0 0x7faa1819bc30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa14009e50 tx=0x7faa1400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:16.928 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.364+0000 7faa0e7fc700 1 -- 172.21.15.19:0/1039893175 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa14020020 con 0x7faa181089d0 2024-05-09T02:00:16.928 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.364+0000 7faa0e7fc700 1 -- 172.21.15.19:0/1039893175 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa14012430 con 0x7faa181089d0 2024-05-09T02:00:16.928 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.364+0000 7faa1f9fe700 1 -- 172.21.15.19:0/1039893175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1819c400 con 0x7faa181089d0 2024-05-09T02:00:16.928 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.364+0000 7faa0e7fc700 1 -- 172.21.15.19:0/1039893175 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa14017040 con 0x7faa181089d0 2024-05-09T02:00:16.928 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.364+0000 7faa1f9fe700 1 -- 172.21.15.19:0/1039893175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa1819f010 con 0x7faa181089d0 2024-05-09T02:00:16.928 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.365+0000 7faa1f9fe700 1 -- 172.21.15.19:0/1039893175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa18064e20 con 0x7faa181089d0 2024-05-09T02:00:16.928 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.365+0000 7faa0e7fc700 1 -- 172.21.15.19:0/1039893175 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7faa14023020 con 0x7faa181089d0 2024-05-09T02:00:16.928 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.365+0000 7faa0e7fc700 1 --2- 172.21.15.19:0/1039893175 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7faa040406f0 0x7faa04042c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:16.928 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.365+0000 7faa0e7fc700 1 -- 172.21.15.19:0/1039893175 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faa1400f050 con 0x7faa181089d0 2024-05-09T02:00:16.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.365+0000 7faa1cf99700 1 --2- 172.21.15.19:0/1039893175 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7faa040406f0 0x7faa04042c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:16.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.366+0000 7faa1cf99700 1 --2- 172.21.15.19:0/1039893175 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7faa040406f0 0x7faa04042c00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faa08006fd0 tx=0x7faa08006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:16.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.371+0000 7faa0e7fc700 1 -- 172.21.15.19:0/1039893175 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faa1402a020 con 0x7faa181089d0 2024-05-09T02:00:16.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:15.649+0000 7faa1f9fe700 1 -- 172.21.15.19:0/1039893175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7faa18194e40 con 0x7faa181089d0 2024-05-09T02:00:16.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:16.268+0000 7faa0e7fc700 1 -- 172.21.15.19:0/1039893175 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7faa18194e40 con 0x7faa181089d0 2024-05-09T02:00:16.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:16.270+0000 7faa0e7fc700 1 -- 172.21.15.19:0/1039893175 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7faa140128a0 con 0x7faa181089d0 2024-05-09T02:00:16.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:16.271+0000 7faa1f9fe700 1 -- 172.21.15.19:0/1039893175 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7faa040406f0 msgr2=0x7faa04042c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:16.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:16.271+0000 7faa1f9fe700 1 --2- 172.21.15.19:0/1039893175 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7faa040406f0 0x7faa04042c00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faa08006fd0 tx=0x7faa08006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:16.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:16.271+0000 7faa1f9fe700 1 -- 172.21.15.19:0/1039893175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa181089d0 msgr2=0x7faa1819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:16.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:16.271+0000 7faa1f9fe700 1 --2- 172.21.15.19:0/1039893175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa181089d0 0x7faa1819bc30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa14009e50 tx=0x7faa1400bd50 comp rx=0 tx=0).stop 2024-05-09T02:00:16.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:16.271+0000 7faa1f9fe700 1 -- 172.21.15.19:0/1039893175 shutdown_connections 2024-05-09T02:00:16.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:16.271+0000 7faa1f9fe700 1 --2- 172.21.15.19:0/1039893175 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7faa040406f0 0x7faa04042c00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:16.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:16.271+0000 7faa1f9fe700 1 --2- 172.21.15.19:0/1039893175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa181089d0 0x7faa1819bc30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:16.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:16.271+0000 7faa1f9fe700 1 -- 172.21.15.19:0/1039893175 >> 172.21.15.19:0/1039893175 conn(0x7faa18104010 msgr2=0x7faa18191f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:16.929 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:16.272+0000 7faa1f9fe700 1 -- 172.21.15.19:0/1039893175 shutdown_connections 2024-05-09T02:00:16.930 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:16.272+0000 7faa1f9fe700 1 -- 172.21.15.19:0/1039893175 wait complete. 2024-05-09T02:00:17.054 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:16.799+0000 7f4dbf02d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T02:00:17.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:17 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1039893175' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-09T02:00:17.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:17 smithi019 ceph-mon[30151]: mgrmap e9: x(active, since 19s) 2024-05-09T02:00:17.320 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:17.144+0000 7f4dbf02d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T02:00:17.320 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:17.241+0000 7f4dbf02d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T02:00:18.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:18 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/520535637' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T02:00:18.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-09T02:00:18.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-09T02:00:18.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T02:00:18.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T02:00:18.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T02:00:18.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-09T02:00:18.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.407+0000 7f97dfca1700 1 Processor -- start 2024-05-09T02:00:18.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.408+0000 7f97dfca1700 1 -- start start 2024-05-09T02:00:18.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.408+0000 7f97dfca1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97d8109af0 0x7f97d8109f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:18.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.408+0000 7f97dfca1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97d810a4f0 con 0x7f97d8109af0 2024-05-09T02:00:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.408+0000 7f97dda3d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97d8109af0 0x7f97d8109f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.408+0000 7f97dda3d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97d8109af0 0x7f97d8109f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37304/0 (socket says 172.21.15.19:37304) 2024-05-09T02:00:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.408+0000 7f97dda3d700 1 -- 172.21.15.19:0/610618744 learned_addr learned my addr 172.21.15.19:0/610618744 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.409+0000 7f97dda3d700 1 -- 172.21.15.19:0/610618744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d810acc0 con 0x7f97d8109af0 2024-05-09T02:00:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.409+0000 7f97dda3d700 1 --2- 172.21.15.19:0/610618744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97d8109af0 0x7f97d8109f00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f97d40098a0 tx=0x7f97d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bcbac47c20b102a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.409+0000 7f97dca3b700 1 -- 172.21.15.19:0/610618744 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97d4004160 con 0x7f97d8109af0 2024-05-09T02:00:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.409+0000 7f97dca3b700 1 -- 172.21.15.19:0/610618744 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97d4004300 con 0x7f97d8109af0 2024-05-09T02:00:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.409+0000 7f97dca3b700 1 -- 172.21.15.19:0/610618744 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97d4011a20 con 0x7f97d8109af0 2024-05-09T02:00:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.410+0000 7f97dfca1700 1 -- 172.21.15.19:0/610618744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97d8109af0 msgr2=0x7f97d8109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.410+0000 7f97dfca1700 1 --2- 172.21.15.19:0/610618744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97d8109af0 0x7f97d8109f00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f97d40098a0 tx=0x7f97d4009bb0 comp rx=0 tx=0).stop 2024-05-09T02:00:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.410+0000 7f97dfca1700 1 -- 172.21.15.19:0/610618744 shutdown_connections 2024-05-09T02:00:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.410+0000 7f97dfca1700 1 --2- 172.21.15.19:0/610618744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97d8109af0 0x7f97d8109f00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.410+0000 7f97dfca1700 1 -- 172.21.15.19:0/610618744 >> 172.21.15.19:0/610618744 conn(0x7f97d8105170 msgr2=0x7f97d81075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.410+0000 7f97dfca1700 1 -- 172.21.15.19:0/610618744 shutdown_connections 2024-05-09T02:00:18.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.410+0000 7f97dfca1700 1 -- 172.21.15.19:0/610618744 wait complete. 2024-05-09T02:00:18.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.411+0000 7f97dfca1700 1 Processor -- start 2024-05-09T02:00:18.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.411+0000 7f97dfca1700 1 -- start start 2024-05-09T02:00:18.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.411+0000 7f97dfca1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97d8109af0 0x7f97d807f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:18.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.411+0000 7f97dfca1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97d810e070 con 0x7f97d8109af0 2024-05-09T02:00:18.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.411+0000 7f97dda3d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97d8109af0 0x7f97d807f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:18.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.411+0000 7f97dda3d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97d8109af0 0x7f97d807f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37310/0 (socket says 172.21.15.19:37310) 2024-05-09T02:00:18.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.411+0000 7f97dda3d700 1 -- 172.21.15.19:0/520535637 learned_addr learned my addr 172.21.15.19:0/520535637 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:18.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.412+0000 7f97dda3d700 1 -- 172.21.15.19:0/520535637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d807f9c0 con 0x7f97d8109af0 2024-05-09T02:00:18.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.412+0000 7f97dda3d700 1 --2- 172.21.15.19:0/520535637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97d8109af0 0x7f97d807f480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f97d4011a20 tx=0x7f97d4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:18.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.412+0000 7f97ceffd700 1 -- 172.21.15.19:0/520535637 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97d40039c0 con 0x7f97d8109af0 2024-05-09T02:00:18.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.412+0000 7f97dfca1700 1 -- 172.21.15.19:0/520535637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d807dc10 con 0x7f97d8109af0 2024-05-09T02:00:18.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.412+0000 7f97ceffd700 1 -- 172.21.15.19:0/520535637 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97d4003b60 con 0x7f97d8109af0 2024-05-09T02:00:18.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.412+0000 7f97ceffd700 1 -- 172.21.15.19:0/520535637 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97d4019700 con 0x7f97d8109af0 2024-05-09T02:00:18.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.412+0000 7f97dfca1700 1 -- 172.21.15.19:0/520535637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d807e0f0 con 0x7f97d8109af0 2024-05-09T02:00:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.413+0000 7f97ceffd700 1 -- 172.21.15.19:0/520535637 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f97d4019a00 con 0x7f97d8109af0 2024-05-09T02:00:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.413+0000 7f97dfca1700 1 -- 172.21.15.19:0/520535637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d8064e20 con 0x7f97d8109af0 2024-05-09T02:00:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.414+0000 7f97ceffd700 1 --2- 172.21.15.19:0/520535637 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f97c403e530 0x7f97c4040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.414+0000 7f97ceffd700 1 -- 172.21.15.19:0/520535637 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f97d4053a40 con 0x7f97d8109af0 2024-05-09T02:00:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.414+0000 7f97dd23c700 1 -- 172.21.15.19:0/520535637 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f97c403e530 msgr2=0x7f97c4040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2101052000 2024-05-09T02:00:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.414+0000 7f97dd23c700 1 --2- 172.21.15.19:0/520535637 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f97c403e530 0x7f97c4040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T02:00:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.419+0000 7f97ceffd700 1 -- 172.21.15.19:0/520535637 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f97d4020e20 con 0x7f97d8109af0 2024-05-09T02:00:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.615+0000 7f97dd23c700 1 -- 172.21.15.19:0/520535637 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f97c403e530 msgr2=0x7f97c4040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2101052000 2024-05-09T02:00:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.615+0000 7f97dd23c700 1 --2- 172.21.15.19:0/520535637 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f97c403e530 0x7f97c4040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T02:00:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.736+0000 7f97dfca1700 1 -- 172.21.15.19:0/520535637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f97d807e740 con 0x7f97d8109af0 2024-05-09T02:00:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.737+0000 7f97ceffd700 1 -- 172.21.15.19:0/520535637 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f97d400b3e0 con 0x7f97d8109af0 2024-05-09T02:00:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.739+0000 7f97dfca1700 1 -- 172.21.15.19:0/520535637 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f97c403e530 msgr2=0x7f97c4040a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:00:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.739+0000 7f97dfca1700 1 --2- 172.21.15.19:0/520535637 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f97c403e530 0x7f97c4040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.739+0000 7f97dfca1700 1 -- 172.21.15.19:0/520535637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97d8109af0 msgr2=0x7f97d807f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:18.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.739+0000 7f97dfca1700 1 --2- 172.21.15.19:0/520535637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97d8109af0 0x7f97d807f480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f97d4011a20 tx=0x7f97d4011b00 comp rx=0 tx=0).stop 2024-05-09T02:00:18.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.740+0000 7f97dfca1700 1 -- 172.21.15.19:0/520535637 shutdown_connections 2024-05-09T02:00:18.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.740+0000 7f97dfca1700 1 --2- 172.21.15.19:0/520535637 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7f97c403e530 0x7f97c4040a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:18.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.740+0000 7f97dfca1700 1 --2- 172.21.15.19:0/520535637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97d8109af0 0x7f97d807f480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:18.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.740+0000 7f97dfca1700 1 -- 172.21.15.19:0/520535637 >> 172.21.15.19:0/520535637 conn(0x7f97d8105170 msgr2=0x7f97d8105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:18.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.740+0000 7f97dfca1700 1 -- 172.21.15.19:0/520535637 shutdown_connections 2024-05-09T02:00:18.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:17.740+0000 7f97dfca1700 1 -- 172.21.15.19:0/520535637 wait complete. 2024-05-09T02:00:18.333 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-05-09T02:00:18.333 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 9... 2024-05-09T02:00:18.571 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:18 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:18.555+0000 7f4dbf02d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T02:00:18.928 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:18 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:18.745+0000 7f4dbf02d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T02:00:18.928 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:18 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:18.927+0000 7f4dbf02d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T02:00:19.552 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:19 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:19.236+0000 7f4dbf02d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T02:00:19.552 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:19 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:19.336+0000 7f4dbf02d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T02:00:19.552 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:19 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:19.551+0000 7f4dbf02d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T02:00:19.820 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:19 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:19.758+0000 7f4dbf02d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T02:00:20.570 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:20 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:20.400+0000 7f4dbf02d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T02:00:21.070 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:20 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:20.639+0000 7f4dbf02d200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T02:00:21.570 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:21 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:21.100+0000 7f4dbf02d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T02:00:21.570 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:21 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:21.204+0000 7f4dbf02d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T02:00:21.982 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:21 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:21.598+0000 7f4dbf02d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T02:00:21.983 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:21 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:21.981+0000 7f4dbf02d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T02:00:22.320 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:22 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:22.082+0000 7f4dbf02d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T02:00:22.820 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:22 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:22.345+0000 7f4dbf02d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T02:00:22.820 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:22 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:22.489+0000 7f4dbf02d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T02:00:22.820 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:22 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:22.593+0000 7f4dbf02d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T02:00:23.320 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:22 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:22.936+0000 7f4dbf02d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T02:00:23.320 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:23 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:23.037+0000 7f4dbf02d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T02:00:23.972 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:23 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:23.683+0000 7f4dbf02d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T02:00:23.973 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:00:23 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:00:23.917+0000 7f4dbf02d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T02:00:24.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:23 smithi019 ceph-mon[30151]: Active manager daemon x restarted 2024-05-09T02:00:24.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:23 smithi019 ceph-mon[30151]: Activating manager daemon x 2024-05-09T02:00:24.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:23 smithi019 ceph-mon[30151]: osdmap e3: 0 total, 0 up, 0 in 2024-05-09T02:00:24.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:23 smithi019 ceph-mon[30151]: mgrmap e10: x(active, starting, since 0.00471427s) 2024-05-09T02:00:24.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:00:24.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T02:00:24.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T02:00:24.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T02:00:24.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T02:00:24.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:23 smithi019 ceph-mon[30151]: Manager daemon x is now available 2024-05-09T02:00:24.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:00:25.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T02:00:25.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T02:00:25.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:24 smithi019 ceph-mon[30151]: mgrmap e11: x(active, since 1.00725s) 2024-05-09T02:00:25.540 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-09T02:00:25.540 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-09T02:00:25.540 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T02:00:25.540 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-09T02:00:25.540 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.779+0000 7fc6a1155700 1 Processor -- start 2024-05-09T02:00:25.540 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.780+0000 7fc6a1155700 1 -- start start 2024-05-09T02:00:25.541 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.780+0000 7fc6a1155700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c10a600 0x7fc69c10ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:25.541 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.780+0000 7fc6a1155700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c07ce20 con 0x7fc69c10a600 2024-05-09T02:00:25.541 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.780+0000 7fc69ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c10a600 0x7fc69c10ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:25.541 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.780+0000 7fc69ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c10a600 0x7fc69c10ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37326/0 (socket says 172.21.15.19:37326) 2024-05-09T02:00:25.541 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.780+0000 7fc69ad9d700 1 -- 172.21.15.19:0/2181569777 learned_addr learned my addr 172.21.15.19:0/2181569777 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:25.541 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.780+0000 7fc69ad9d700 1 -- 172.21.15.19:0/2181569777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c10cf60 con 0x7fc69c10a600 2024-05-09T02:00:25.542 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.781+0000 7fc69ad9d700 1 --2- 172.21.15.19:0/2181569777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c10a600 0x7fc69c10ca20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0098a0 tx=0x7fc68c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbd45d03373b62e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:25.542 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.781+0000 7fc699d9b700 1 -- 172.21.15.19:0/2181569777 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc68c004160 con 0x7fc69c10a600 2024-05-09T02:00:25.542 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.781+0000 7fc699d9b700 1 -- 172.21.15.19:0/2181569777 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc68c004300 con 0x7fc69c10a600 2024-05-09T02:00:25.542 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.781+0000 7fc699d9b700 1 -- 172.21.15.19:0/2181569777 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc68c011a20 con 0x7fc69c10a600 2024-05-09T02:00:25.542 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.782+0000 7fc6a1155700 1 -- 172.21.15.19:0/2181569777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c10a600 msgr2=0x7fc69c10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:25.542 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.782+0000 7fc6a1155700 1 --2- 172.21.15.19:0/2181569777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c10a600 0x7fc69c10ca20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0098a0 tx=0x7fc68c009bb0 comp rx=0 tx=0).stop 2024-05-09T02:00:25.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.782+0000 7fc6a1155700 1 -- 172.21.15.19:0/2181569777 shutdown_connections 2024-05-09T02:00:25.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.782+0000 7fc6a1155700 1 --2- 172.21.15.19:0/2181569777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c10a600 0x7fc69c10ca20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:25.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.782+0000 7fc6a1155700 1 -- 172.21.15.19:0/2181569777 >> 172.21.15.19:0/2181569777 conn(0x7fc69c104080 msgr2=0x7fc69c1064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:25.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.782+0000 7fc6a1155700 1 -- 172.21.15.19:0/2181569777 shutdown_connections 2024-05-09T02:00:25.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.782+0000 7fc6a1155700 1 -- 172.21.15.19:0/2181569777 wait complete. 2024-05-09T02:00:25.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.783+0000 7fc6a1155700 1 Processor -- start 2024-05-09T02:00:25.543 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.783+0000 7fc6a1155700 1 -- start start 2024-05-09T02:00:25.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.783+0000 7fc6a1155700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c10a600 0x7fc69c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:25.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.783+0000 7fc6a1155700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c10df00 con 0x7fc69c10a600 2024-05-09T02:00:25.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.783+0000 7fc69ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c10a600 0x7fc69c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:25.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.783+0000 7fc69ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c10a600 0x7fc69c19de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37328/0 (socket says 172.21.15.19:37328) 2024-05-09T02:00:25.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.783+0000 7fc69ad9d700 1 -- 172.21.15.19:0/1470980280 learned_addr learned my addr 172.21.15.19:0/1470980280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:25.544 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.784+0000 7fc69ad9d700 1 -- 172.21.15.19:0/1470980280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c19e360 con 0x7fc69c10a600 2024-05-09T02:00:25.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.784+0000 7fc69ad9d700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c10a600 0x7fc69c19de20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc68c004660 tx=0x7fc68c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:25.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.784+0000 7fc683fff700 1 -- 172.21.15.19:0/1470980280 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc68c003a60 con 0x7fc69c10a600 2024-05-09T02:00:25.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.784+0000 7fc683fff700 1 -- 172.21.15.19:0/1470980280 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc68c01b8f0 con 0x7fc69c10a600 2024-05-09T02:00:25.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.784+0000 7fc6a1155700 1 -- 172.21.15.19:0/1470980280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69c19e5f0 con 0x7fc69c10a600 2024-05-09T02:00:25.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.784+0000 7fc683fff700 1 -- 172.21.15.19:0/1470980280 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc68c023890 con 0x7fc69c10a600 2024-05-09T02:00:25.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.784+0000 7fc6a1155700 1 -- 172.21.15.19:0/1470980280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69c1a1200 con 0x7fc69c10a600 2024-05-09T02:00:25.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.785+0000 7fc683fff700 1 -- 172.21.15.19:0/1470980280 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fc68c02c3f0 con 0x7fc69c10a600 2024-05-09T02:00:25.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.786+0000 7fc683fff700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fc68403e590 0x7fc684040aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:25.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.786+0000 7fc683fff700 1 -- 172.21.15.19:0/1470980280 --> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc6840411b0 con 0x7fc68403e590 2024-05-09T02:00:25.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.786+0000 7fc683fff700 1 -- 172.21.15.19:0/1470980280 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc68c053f50 con 0x7fc69c10a600 2024-05-09T02:00:25.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.786+0000 7fc69a59c700 1 -- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fc68403e590 msgr2=0x7fc684040aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2101052000 2024-05-09T02:00:25.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.786+0000 7fc69a59c700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fc68403e590 0x7fc684040aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T02:00:25.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.987+0000 7fc69a59c700 1 -- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fc68403e590 msgr2=0x7fc684040aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2101052000 2024-05-09T02:00:25.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:18.987+0000 7fc69a59c700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fc68403e590 0x7fc684040aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T02:00:25.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:19.388+0000 7fc69a59c700 1 -- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fc68403e590 msgr2=0x7fc684040aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2101052000 2024-05-09T02:00:25.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:19.388+0000 7fc69a59c700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fc68403e590 0x7fc684040aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T02:00:25.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:20.189+0000 7fc69a59c700 1 -- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fc68403e590 msgr2=0x7fc684040aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2101052000 2024-05-09T02:00:25.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:20.189+0000 7fc69a59c700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fc68403e590 0x7fc684040aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T02:00:25.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:21.790+0000 7fc69a59c700 1 -- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fc68403e590 msgr2=0x7fc684040aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2101052000 2024-05-09T02:00:25.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:21.790+0000 7fc69a59c700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fc68403e590 0x7fc684040aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T02:00:25.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:23.922+0000 7fc683fff700 1 -- 172.21.15.19:0/1470980280 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7fc68c02c780 con 0x7fc69c10a600 2024-05-09T02:00:25.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:23.922+0000 7fc683fff700 1 -- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fc68403e590 msgr2=0x7fc684040aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:00:25.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:23.922+0000 7fc683fff700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fc68403e590 0x7fc684040aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:25.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.923+0000 7fc683fff700 1 -- 172.21.15.19:0/1470980280 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7fc68c055790 con 0x7fc69c10a600 2024-05-09T02:00:25.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.923+0000 7fc683fff700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc68c053ae0 0x7fc684043fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:25.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.924+0000 7fc683fff700 1 -- 172.21.15.19:0/1470980280 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc68c008e50 con 0x7fc68c053ae0 2024-05-09T02:00:25.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.924+0000 7fc69a59c700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc68c053ae0 0x7fc684043fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:25.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.925+0000 7fc69a59c700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc68c053ae0 0x7fc684043fd0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc690003a10 tx=0x7fc690009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:25.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.926+0000 7fc683fff700 1 -- 172.21.15.19:0/1470980280 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fc68c008e50 con 0x7fc68c053ae0 2024-05-09T02:00:25.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.934+0000 7fc6a1155700 1 -- 172.21.15.19:0/1470980280 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc69c064e20 con 0x7fc68c053ae0 2024-05-09T02:00:25.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.935+0000 7fc683fff700 1 -- 172.21.15.19:0/1470980280 <== mgr.14152 v2:172.21.15.19:6800/972543870 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fc69c064e20 con 0x7fc68c053ae0 2024-05-09T02:00:25.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.935+0000 7fc6a1155700 1 -- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc68c053ae0 msgr2=0x7fc684043fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:25.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.935+0000 7fc6a1155700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc68c053ae0 0x7fc684043fd0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc690003a10 tx=0x7fc690009270 comp rx=0 tx=0).stop 2024-05-09T02:00:25.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.935+0000 7fc6a1155700 1 -- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c10a600 msgr2=0x7fc69c19de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:25.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.935+0000 7fc6a1155700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c10a600 0x7fc69c19de20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc68c004660 tx=0x7fc68c011aa0 comp rx=0 tx=0).stop 2024-05-09T02:00:25.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.935+0000 7fc6a1155700 1 -- 172.21.15.19:0/1470980280 shutdown_connections 2024-05-09T02:00:25.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.935+0000 7fc6a1155700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc68c053ae0 0x7fc684043fd0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:25.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.935+0000 7fc6a1155700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69c10a600 0x7fc69c19de20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:25.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.935+0000 7fc6a1155700 1 --2- 172.21.15.19:0/1470980280 >> [v2:172.21.15.19:6800/2101052000,v1:172.21.15.19:6801/2101052000] conn(0x7fc68403e590 0x7fc684040aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:25.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.935+0000 7fc6a1155700 1 -- 172.21.15.19:0/1470980280 >> 172.21.15.19:0/1470980280 conn(0x7fc69c104080 msgr2=0x7fc69c104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:25.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.935+0000 7fc6a1155700 1 -- 172.21.15.19:0/1470980280 shutdown_connections 2024-05-09T02:00:25.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:24.935+0000 7fc6a1155700 1 -- 172.21.15.19:0/1470980280 wait complete. 2024-05-09T02:00:25.551 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 9 is available 2024-05-09T02:00:25.551 INFO:teuthology.orchestra.run.smithi019.stdout:Generating a dashboard self-signed certificate... 2024-05-09T02:00:26.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:25 smithi019 ceph-mon[30151]: [09/May/2024:02:00:25] ENGINE Bus STARTING 2024-05-09T02:00:26.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:27.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:26 smithi019 ceph-mon[30151]: [09/May/2024:02:00:25] ENGINE Serving on http://172.21.15.19:8765 2024-05-09T02:00:27.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:26 smithi019 ceph-mon[30151]: [09/May/2024:02:00:25] ENGINE Serving on https://172.21.15.19:7150 2024-05-09T02:00:27.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:26 smithi019 ceph-mon[30151]: [09/May/2024:02:00:25] ENGINE Bus STARTED 2024-05-09T02:00:27.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:27.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:27.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:26 smithi019 ceph-mon[30151]: mgrmap e12: x(active, since 2s) 2024-05-09T02:00:27.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-09T02:00:27.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.406+0000 7f8433d0f700 1 Processor -- start 2024-05-09T02:00:27.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.407+0000 7f8433d0f700 1 -- start start 2024-05-09T02:00:27.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.407+0000 7f8433d0f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f842c108a00 0x7f842c108e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:27.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.407+0000 7f8433d0f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f842c109400 con 0x7f842c108a00 2024-05-09T02:00:27.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.407+0000 7f8431aab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f842c108a00 0x7f842c108e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:27.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.407+0000 7f8431aab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f842c108a00 0x7f842c108e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37398/0 (socket says 172.21.15.19:37398) 2024-05-09T02:00:27.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.407+0000 7f8431aab700 1 -- 172.21.15.19:0/1326008877 learned_addr learned my addr 172.21.15.19:0/1326008877 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:27.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.407+0000 7f8431aab700 1 -- 172.21.15.19:0/1326008877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842c109bd0 con 0x7f842c108a00 2024-05-09T02:00:27.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.408+0000 7f8431aab700 1 --2- 172.21.15.19:0/1326008877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f842c108a00 0x7f842c108e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f84280098a0 tx=0x7f8428009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a44d50af0eae4d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:27.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.408+0000 7f8430aa9700 1 -- 172.21.15.19:0/1326008877 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8428004160 con 0x7f842c108a00 2024-05-09T02:00:27.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.408+0000 7f8430aa9700 1 -- 172.21.15.19:0/1326008877 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8428004300 con 0x7f842c108a00 2024-05-09T02:00:27.401 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.408+0000 7f8430aa9700 1 -- 172.21.15.19:0/1326008877 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8428011a20 con 0x7f842c108a00 2024-05-09T02:00:27.401 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.408+0000 7f8433d0f700 1 -- 172.21.15.19:0/1326008877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f842c108a00 msgr2=0x7f842c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:27.401 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.408+0000 7f8433d0f700 1 --2- 172.21.15.19:0/1326008877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f842c108a00 0x7f842c108e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f84280098a0 tx=0x7f8428009bb0 comp rx=0 tx=0).stop 2024-05-09T02:00:27.401 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.409+0000 7f8433d0f700 1 -- 172.21.15.19:0/1326008877 shutdown_connections 2024-05-09T02:00:27.401 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.409+0000 7f8433d0f700 1 --2- 172.21.15.19:0/1326008877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f842c108a00 0x7f842c108e10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:27.401 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.409+0000 7f8433d0f700 1 -- 172.21.15.19:0/1326008877 >> 172.21.15.19:0/1326008877 conn(0x7f842c1040e0 msgr2=0x7f842c106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:27.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.409+0000 7f8433d0f700 1 -- 172.21.15.19:0/1326008877 shutdown_connections 2024-05-09T02:00:27.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.409+0000 7f8433d0f700 1 -- 172.21.15.19:0/1326008877 wait complete. 2024-05-09T02:00:27.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.409+0000 7f8433d0f700 1 Processor -- start 2024-05-09T02:00:27.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.409+0000 7f8433d0f700 1 -- start start 2024-05-09T02:00:27.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.410+0000 7f8433d0f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f842c108a00 0x7f842c07d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:27.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.410+0000 7f8433d0f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f842c10cf80 con 0x7f842c108a00 2024-05-09T02:00:27.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.410+0000 7f8431aab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f842c108a00 0x7f842c07d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:27.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.410+0000 7f8431aab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f842c108a00 0x7f842c07d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37410/0 (socket says 172.21.15.19:37410) 2024-05-09T02:00:27.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.410+0000 7f8431aab700 1 -- 172.21.15.19:0/393002314 learned_addr learned my addr 172.21.15.19:0/393002314 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:27.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.410+0000 7f8431aab700 1 -- 172.21.15.19:0/393002314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842c07d760 con 0x7f842c108a00 2024-05-09T02:00:27.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.410+0000 7f8431aab700 1 --2- 172.21.15.19:0/393002314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f842c108a00 0x7f842c07d220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8428009130 tx=0x7f8428011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:27.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.410+0000 7f8422ffd700 1 -- 172.21.15.19:0/393002314 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8428003aa0 con 0x7f842c108a00 2024-05-09T02:00:27.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.410+0000 7f8433d0f700 1 -- 172.21.15.19:0/393002314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842c1a0be0 con 0x7f842c108a00 2024-05-09T02:00:27.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.410+0000 7f8422ffd700 1 -- 172.21.15.19:0/393002314 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842801a8e0 con 0x7f842c108a00 2024-05-09T02:00:27.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.410+0000 7f8422ffd700 1 -- 172.21.15.19:0/393002314 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84280248f0 con 0x7f842c108a00 2024-05-09T02:00:27.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.410+0000 7f8433d0f700 1 -- 172.21.15.19:0/393002314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842c1a10c0 con 0x7f842c108a00 2024-05-09T02:00:27.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.411+0000 7f8433d0f700 1 -- 172.21.15.19:0/393002314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f842c064e20 con 0x7f842c108a00 2024-05-09T02:00:27.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.411+0000 7f8422ffd700 1 -- 172.21.15.19:0/393002314 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f8428022020 con 0x7f842c108a00 2024-05-09T02:00:27.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.412+0000 7f8422ffd700 1 --2- 172.21.15.19:0/393002314 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f841803e460 0x7f8418040970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:27.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.412+0000 7f8422ffd700 1 -- 172.21.15.19:0/393002314 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f84280537d0 con 0x7f842c108a00 2024-05-09T02:00:27.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.412+0000 7f84312aa700 1 --2- 172.21.15.19:0/393002314 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f841803e460 0x7f8418040970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:27.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.412+0000 7f84312aa700 1 --2- 172.21.15.19:0/393002314 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f841803e460 0x7f8418040970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f841c006fd0 tx=0x7f841c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:27.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.417+0000 7f8422ffd700 1 -- 172.21.15.19:0/393002314 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8428029df0 con 0x7f842c108a00 2024-05-09T02:00:27.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.639+0000 7f8433d0f700 1 -- 172.21.15.19:0/393002314 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f842c106080 con 0x7f841803e460 2024-05-09T02:00:27.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.764+0000 7f8422ffd700 1 -- 172.21.15.19:0/393002314 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f842c106080 con 0x7f841803e460 2024-05-09T02:00:27.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.766+0000 7f8433d0f700 1 -- 172.21.15.19:0/393002314 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f841803e460 msgr2=0x7f8418040970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:27.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.766+0000 7f8433d0f700 1 --2- 172.21.15.19:0/393002314 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f841803e460 0x7f8418040970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f841c006fd0 tx=0x7f841c006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:27.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.766+0000 7f8433d0f700 1 -- 172.21.15.19:0/393002314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f842c108a00 msgr2=0x7f842c07d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:27.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.766+0000 7f8433d0f700 1 --2- 172.21.15.19:0/393002314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f842c108a00 0x7f842c07d220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8428009130 tx=0x7f8428011aa0 comp rx=0 tx=0).stop 2024-05-09T02:00:27.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.766+0000 7f8433d0f700 1 -- 172.21.15.19:0/393002314 shutdown_connections 2024-05-09T02:00:27.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.766+0000 7f8433d0f700 1 --2- 172.21.15.19:0/393002314 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f841803e460 0x7f8418040970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:27.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.766+0000 7f8433d0f700 1 --2- 172.21.15.19:0/393002314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f842c108a00 0x7f842c07d220 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:27.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.766+0000 7f8433d0f700 1 -- 172.21.15.19:0/393002314 >> 172.21.15.19:0/393002314 conn(0x7f842c1040e0 msgr2=0x7f842c105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:27.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.766+0000 7f8433d0f700 1 -- 172.21.15.19:0/393002314 shutdown_connections 2024-05-09T02:00:27.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:26.766+0000 7f8433d0f700 1 -- 172.21.15.19:0/393002314 wait complete. 2024-05-09T02:00:27.407 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial admin user... 2024-05-09T02:00:28.268 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:28 smithi019 ceph-mon[30151]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:00:28.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:28 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:29.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:29 smithi019 ceph-mon[30151]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:00:29.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:29 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:29.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$93SJ29FZ980NVO4dvB.k..aenTTE1ahcOqwQD6dT4dTE3XtS3AYHG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715220028, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-09T02:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.773+0000 7f1a38bf3700 1 Processor -- start 2024-05-09T02:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.773+0000 7f1a38bf3700 1 -- start start 2024-05-09T02:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.774+0000 7f1a38bf3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34109ae0 0x7f1a34109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.774+0000 7f1a38bf3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3410a4e0 con 0x7f1a34109ae0 2024-05-09T02:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.774+0000 7f1a3259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34109ae0 0x7f1a34109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.774+0000 7f1a3259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34109ae0 0x7f1a34109ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44046/0 (socket says 172.21.15.19:44046) 2024-05-09T02:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.774+0000 7f1a3259c700 1 -- 172.21.15.19:0/827727921 learned_addr learned my addr 172.21.15.19:0/827727921 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.774+0000 7f1a3259c700 1 -- 172.21.15.19:0/827727921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3410acb0 con 0x7f1a34109ae0 2024-05-09T02:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.774+0000 7f1a3259c700 1 --2- 172.21.15.19:0/827727921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34109ae0 0x7f1a34109ef0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a240098a0 tx=0x7f1a24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ccdbbb0babf3ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.775+0000 7f1a3159a700 1 -- 172.21.15.19:0/827727921 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a24004160 con 0x7f1a34109ae0 2024-05-09T02:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.775+0000 7f1a3159a700 1 -- 172.21.15.19:0/827727921 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a24004300 con 0x7f1a34109ae0 2024-05-09T02:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.775+0000 7f1a3159a700 1 -- 172.21.15.19:0/827727921 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a24011a20 con 0x7f1a34109ae0 2024-05-09T02:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.775+0000 7f1a38bf3700 1 -- 172.21.15.19:0/827727921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34109ae0 msgr2=0x7f1a34109ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.775+0000 7f1a38bf3700 1 --2- 172.21.15.19:0/827727921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34109ae0 0x7f1a34109ef0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a240098a0 tx=0x7f1a24009bb0 comp rx=0 tx=0).stop 2024-05-09T02:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.775+0000 7f1a38bf3700 1 -- 172.21.15.19:0/827727921 shutdown_connections 2024-05-09T02:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.775+0000 7f1a38bf3700 1 --2- 172.21.15.19:0/827727921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34109ae0 0x7f1a34109ef0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.775+0000 7f1a38bf3700 1 -- 172.21.15.19:0/827727921 >> 172.21.15.19:0/827727921 conn(0x7f1a34105160 msgr2=0x7f1a341075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.776+0000 7f1a38bf3700 1 -- 172.21.15.19:0/827727921 shutdown_connections 2024-05-09T02:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.776+0000 7f1a38bf3700 1 -- 172.21.15.19:0/827727921 wait complete. 2024-05-09T02:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.776+0000 7f1a38bf3700 1 Processor -- start 2024-05-09T02:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.777+0000 7f1a38bf3700 1 -- start start 2024-05-09T02:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.777+0000 7f1a38bf3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34109ae0 0x7f1a3419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:29.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.777+0000 7f1a38bf3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3410e060 con 0x7f1a34109ae0 2024-05-09T02:00:29.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.777+0000 7f1a3259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34109ae0 0x7f1a3419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:29.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.777+0000 7f1a3259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34109ae0 0x7f1a3419de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44062/0 (socket says 172.21.15.19:44062) 2024-05-09T02:00:29.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.777+0000 7f1a3259c700 1 -- 172.21.15.19:0/1843733905 learned_addr learned my addr 172.21.15.19:0/1843733905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:29.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.778+0000 7f1a3259c700 1 -- 172.21.15.19:0/1843733905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3419e3d0 con 0x7f1a34109ae0 2024-05-09T02:00:29.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.778+0000 7f1a3259c700 1 --2- 172.21.15.19:0/1843733905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34109ae0 0x7f1a3419de90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1a24011a20 tx=0x7f1a24011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:29.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.778+0000 7f1a237fe700 1 -- 172.21.15.19:0/1843733905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a240039c0 con 0x7f1a34109ae0 2024-05-09T02:00:29.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.778+0000 7f1a38bf3700 1 -- 172.21.15.19:0/1843733905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3419e660 con 0x7f1a34109ae0 2024-05-09T02:00:29.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.778+0000 7f1a38bf3700 1 -- 172.21.15.19:0/1843733905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a341a1270 con 0x7f1a34109ae0 2024-05-09T02:00:29.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.778+0000 7f1a237fe700 1 -- 172.21.15.19:0/1843733905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a24003b60 con 0x7f1a34109ae0 2024-05-09T02:00:29.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.778+0000 7f1a237fe700 1 -- 172.21.15.19:0/1843733905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a24019700 con 0x7f1a34109ae0 2024-05-09T02:00:29.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.779+0000 7f1a237fe700 1 -- 172.21.15.19:0/1843733905 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f1a240199f0 con 0x7f1a34109ae0 2024-05-09T02:00:29.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.779+0000 7f1a237fe700 1 --2- 172.21.15.19:0/1843733905 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1a1c03e590 0x7f1a1c040aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:29.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.779+0000 7f1a237fe700 1 -- 172.21.15.19:0/1843733905 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1a24054cb0 con 0x7f1a34109ae0 2024-05-09T02:00:29.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.779+0000 7f1a38bf3700 1 -- 172.21.15.19:0/1843733905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a14005360 con 0x7f1a34109ae0 2024-05-09T02:00:29.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.780+0000 7f1a31d9b700 1 --2- 172.21.15.19:0/1843733905 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1a1c03e590 0x7f1a1c040aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:29.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.780+0000 7f1a31d9b700 1 --2- 172.21.15.19:0/1843733905 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1a1c03e590 0x7f1a1c040aa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a28006fd0 tx=0x7f1a28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:29.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:27.785+0000 7f1a237fe700 1 -- 172.21.15.19:0/1843733905 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1a24017e90 con 0x7f1a34109ae0 2024-05-09T02:00:29.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:28.025+0000 7f1a38bf3700 1 -- 172.21.15.19:0/1843733905 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f1a14002430 con 0x7f1a1c03e590 2024-05-09T02:00:29.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:28.272+0000 7f1a237fe700 1 -- 172.21.15.19:0/1843733905 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f1a14002430 con 0x7f1a1c03e590 2024-05-09T02:00:29.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:28.274+0000 7f1a38bf3700 1 -- 172.21.15.19:0/1843733905 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1a1c03e590 msgr2=0x7f1a1c040aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:29.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:28.274+0000 7f1a38bf3700 1 --2- 172.21.15.19:0/1843733905 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1a1c03e590 0x7f1a1c040aa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a28006fd0 tx=0x7f1a28006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:29.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:28.274+0000 7f1a38bf3700 1 -- 172.21.15.19:0/1843733905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34109ae0 msgr2=0x7f1a3419de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:29.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:28.274+0000 7f1a38bf3700 1 --2- 172.21.15.19:0/1843733905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34109ae0 0x7f1a3419de90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1a24011a20 tx=0x7f1a24011b00 comp rx=0 tx=0).stop 2024-05-09T02:00:29.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:28.274+0000 7f1a38bf3700 1 -- 172.21.15.19:0/1843733905 shutdown_connections 2024-05-09T02:00:29.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:28.274+0000 7f1a38bf3700 1 --2- 172.21.15.19:0/1843733905 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1a1c03e590 0x7f1a1c040aa0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:29.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:28.274+0000 7f1a38bf3700 1 --2- 172.21.15.19:0/1843733905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34109ae0 0x7f1a3419de90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:29.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:28.275+0000 7f1a38bf3700 1 -- 172.21.15.19:0/1843733905 >> 172.21.15.19:0/1843733905 conn(0x7f1a34105160 msgr2=0x7f1a34105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:29.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:28.275+0000 7f1a38bf3700 1 -- 172.21.15.19:0/1843733905 shutdown_connections 2024-05-09T02:00:29.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:28.275+0000 7f1a38bf3700 1 -- 172.21.15.19:0/1843733905 wait complete. 2024-05-09T02:00:29.447 INFO:teuthology.orchestra.run.smithi019.stdout:Fetching dashboard port number... 2024-05-09T02:00:30.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:30 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/954449603' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-09T02:00:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 8443 2024-05-09T02:00:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.868+0000 7ff1e9cc7700 1 Processor -- start 2024-05-09T02:00:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.868+0000 7ff1e9cc7700 1 -- start start 2024-05-09T02:00:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.868+0000 7ff1e9cc7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e410a620 0x7ff1e410ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.868+0000 7ff1e9cc7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e407d6b0 con 0x7ff1e410a620 2024-05-09T02:00:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.869+0000 7ff1e37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e410a620 0x7ff1e410ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.869+0000 7ff1e37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e410a620 0x7ff1e410ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44070/0 (socket says 172.21.15.19:44070) 2024-05-09T02:00:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.869+0000 7ff1e37fe700 1 -- 172.21.15.19:0/3066108854 learned_addr learned my addr 172.21.15.19:0/3066108854 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.869+0000 7ff1e37fe700 1 -- 172.21.15.19:0/3066108854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e410cf80 con 0x7ff1e410a620 2024-05-09T02:00:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.869+0000 7ff1e37fe700 1 --2- 172.21.15.19:0/3066108854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e410a620 0x7ff1e410ca40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009e80 tx=0x7ff1d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c70bd21c99adeb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.869+0000 7ff1e2ffd700 1 -- 172.21.15.19:0/3066108854 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1d4017070 con 0x7ff1e410a620 2024-05-09T02:00:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.869+0000 7ff1e2ffd700 1 -- 172.21.15.19:0/3066108854 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1d400b8c0 con 0x7ff1e410a620 2024-05-09T02:00:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.869+0000 7ff1e2ffd700 1 -- 172.21.15.19:0/3066108854 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1d4004490 con 0x7ff1e410a620 2024-05-09T02:00:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.870+0000 7ff1e9cc7700 1 -- 172.21.15.19:0/3066108854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e410a620 msgr2=0x7ff1e410ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.870+0000 7ff1e9cc7700 1 --2- 172.21.15.19:0/3066108854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e410a620 0x7ff1e410ca40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009e80 tx=0x7ff1d400b330 comp rx=0 tx=0).stop 2024-05-09T02:00:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.870+0000 7ff1e9cc7700 1 -- 172.21.15.19:0/3066108854 shutdown_connections 2024-05-09T02:00:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.870+0000 7ff1e9cc7700 1 --2- 172.21.15.19:0/3066108854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e410a620 0x7ff1e410ca40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.870+0000 7ff1e9cc7700 1 -- 172.21.15.19:0/3066108854 >> 172.21.15.19:0/3066108854 conn(0x7ff1e41040e0 msgr2=0x7ff1e4106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.870+0000 7ff1e9cc7700 1 -- 172.21.15.19:0/3066108854 shutdown_connections 2024-05-09T02:00:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.870+0000 7ff1e9cc7700 1 -- 172.21.15.19:0/3066108854 wait complete. 2024-05-09T02:00:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.871+0000 7ff1e9cc7700 1 Processor -- start 2024-05-09T02:00:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.871+0000 7ff1e9cc7700 1 -- start start 2024-05-09T02:00:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.871+0000 7ff1e9cc7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e410a620 0x7ff1e419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.871+0000 7ff1e9cc7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e410df20 con 0x7ff1e410a620 2024-05-09T02:00:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.871+0000 7ff1e37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e410a620 0x7ff1e419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.871+0000 7ff1e37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e410a620 0x7ff1e419de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44086/0 (socket says 172.21.15.19:44086) 2024-05-09T02:00:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.871+0000 7ff1e37fe700 1 -- 172.21.15.19:0/954449603 learned_addr learned my addr 172.21.15.19:0/954449603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.872+0000 7ff1e37fe700 1 -- 172.21.15.19:0/954449603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e419e3a0 con 0x7ff1e410a620 2024-05-09T02:00:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.872+0000 7ff1e37fe700 1 --2- 172.21.15.19:0/954449603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e410a620 0x7ff1e419de60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40072a0 tx=0x7ff1d400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.872+0000 7ff1e17fa700 1 -- 172.21.15.19:0/954449603 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1d4020020 con 0x7ff1e410a620 2024-05-09T02:00:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.872+0000 7ff1e17fa700 1 -- 172.21.15.19:0/954449603 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1d4012430 con 0x7ff1e410a620 2024-05-09T02:00:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.872+0000 7ff1e9cc7700 1 -- 172.21.15.19:0/954449603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e419e630 con 0x7ff1e410a620 2024-05-09T02:00:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.872+0000 7ff1e9cc7700 1 -- 172.21.15.19:0/954449603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e41a1240 con 0x7ff1e410a620 2024-05-09T02:00:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.872+0000 7ff1e17fa700 1 -- 172.21.15.19:0/954449603 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1d4017040 con 0x7ff1e410a620 2024-05-09T02:00:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.873+0000 7ff1e17fa700 1 -- 172.21.15.19:0/954449603 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7ff1d4023020 con 0x7ff1e410a620 2024-05-09T02:00:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.873+0000 7ff1e9cc7700 1 -- 172.21.15.19:0/954449603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c8005360 con 0x7ff1e410a620 2024-05-09T02:00:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.873+0000 7ff1e17fa700 1 --2- 172.21.15.19:0/954449603 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff1c403e590 0x7ff1c4040aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.873+0000 7ff1e17fa700 1 -- 172.21.15.19:0/954449603 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff1d400f050 con 0x7ff1e410a620 2024-05-09T02:00:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.874+0000 7ff1d2ffd700 1 --2- 172.21.15.19:0/954449603 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff1c403e590 0x7ff1c4040aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.874+0000 7ff1d2ffd700 1 --2- 172.21.15.19:0/954449603 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff1c403e590 0x7ff1c4040aa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc006fd0 tx=0x7ff1cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:29.879+0000 7ff1e17fa700 1 -- 172.21.15.19:0/954449603 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff1d402a020 con 0x7ff1e410a620 2024-05-09T02:00:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:30.092+0000 7ff1e9cc7700 1 -- 172.21.15.19:0/954449603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7ff1c8005650 con 0x7ff1e410a620 2024-05-09T02:00:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:30.092+0000 7ff1e17fa700 1 -- 172.21.15.19:0/954449603 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7ff1d402a200 con 0x7ff1e410a620 2024-05-09T02:00:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:30.095+0000 7ff1e9cc7700 1 -- 172.21.15.19:0/954449603 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff1c403e590 msgr2=0x7ff1c4040aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:30.095+0000 7ff1e9cc7700 1 --2- 172.21.15.19:0/954449603 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff1c403e590 0x7ff1c4040aa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc006fd0 tx=0x7ff1cc006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:30.095+0000 7ff1e9cc7700 1 -- 172.21.15.19:0/954449603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e410a620 msgr2=0x7ff1e419de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:30.095+0000 7ff1e9cc7700 1 --2- 172.21.15.19:0/954449603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e410a620 0x7ff1e419de60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40072a0 tx=0x7ff1d400bd50 comp rx=0 tx=0).stop 2024-05-09T02:00:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:30.095+0000 7ff1e9cc7700 1 -- 172.21.15.19:0/954449603 shutdown_connections 2024-05-09T02:00:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:30.095+0000 7ff1e9cc7700 1 --2- 172.21.15.19:0/954449603 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff1c403e590 0x7ff1c4040aa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:30.095+0000 7ff1e9cc7700 1 --2- 172.21.15.19:0/954449603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1e410a620 0x7ff1e419de60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:30.095+0000 7ff1e9cc7700 1 -- 172.21.15.19:0/954449603 >> 172.21.15.19:0/954449603 conn(0x7ff1e41040e0 msgr2=0x7ff1e4105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:30.095+0000 7ff1e9cc7700 1 -- 172.21.15.19:0/954449603 shutdown_connections 2024-05-09T02:00:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:30.095+0000 7ff1e9cc7700 1 -- 172.21.15.19:0/954449603 wait complete. 2024-05-09T02:00:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T02:00:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-09T02:00:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T02:00:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-09T02:00:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-09T02:00:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-09T02:00:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph Dashboard is now available at: 2024-05-09T02:00:30.720 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:00:30.721 INFO:teuthology.orchestra.run.smithi019.stdout: URL: https://smithi019.front.sepia.ceph.com:8443/ 2024-05-09T02:00:30.721 INFO:teuthology.orchestra.run.smithi019.stdout: User: admin 2024-05-09T02:00:30.721 INFO:teuthology.orchestra.run.smithi019.stdout: Password: 9ru5bblur4 2024-05-09T02:00:30.721 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:00:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:Saving cluster configuration to /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config directory 2024-05-09T02:00:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling autotune for osd_memory_target 2024-05-09T02:00:31.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:31 smithi019 ceph-mon[30151]: mgrmap e13: x(active, since 6s) 2024-05-09T02:00:31.922 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.166+0000 7fee34872700 1 Processor -- start 2024-05-09T02:00:31.922 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.166+0000 7fee34872700 1 -- start start 2024-05-09T02:00:31.923 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.166+0000 7fee34872700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee30109b00 0x7fee30109f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:31.923 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.166+0000 7fee34872700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee3010a500 con 0x7fee30109b00 2024-05-09T02:00:31.923 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.166+0000 7fee2e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee30109b00 0x7fee30109f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:31.923 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.166+0000 7fee2e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee30109b00 0x7fee30109f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44102/0 (socket says 172.21.15.19:44102) 2024-05-09T02:00:31.923 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.166+0000 7fee2e59c700 1 -- 172.21.15.19:0/1310076372 learned_addr learned my addr 172.21.15.19:0/1310076372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:31.923 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.167+0000 7fee2e59c700 1 -- 172.21.15.19:0/1310076372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3010acd0 con 0x7fee30109b00 2024-05-09T02:00:31.923 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.167+0000 7fee2e59c700 1 --2- 172.21.15.19:0/1310076372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee30109b00 0x7fee30109f10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee200098a0 tx=0x7fee20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27d0128afbd6731 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:31.923 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.167+0000 7fee2d59a700 1 -- 172.21.15.19:0/1310076372 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee20004160 con 0x7fee30109b00 2024-05-09T02:00:31.923 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.167+0000 7fee2d59a700 1 -- 172.21.15.19:0/1310076372 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee20004300 con 0x7fee30109b00 2024-05-09T02:00:31.923 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.167+0000 7fee2d59a700 1 -- 172.21.15.19:0/1310076372 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee20011a20 con 0x7fee30109b00 2024-05-09T02:00:31.924 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.168+0000 7fee34872700 1 -- 172.21.15.19:0/1310076372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee30109b00 msgr2=0x7fee30109f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:31.924 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.168+0000 7fee34872700 1 --2- 172.21.15.19:0/1310076372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee30109b00 0x7fee30109f10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee200098a0 tx=0x7fee20009bb0 comp rx=0 tx=0).stop 2024-05-09T02:00:31.924 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.168+0000 7fee34872700 1 -- 172.21.15.19:0/1310076372 shutdown_connections 2024-05-09T02:00:31.924 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.168+0000 7fee34872700 1 --2- 172.21.15.19:0/1310076372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee30109b00 0x7fee30109f10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:31.924 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.168+0000 7fee34872700 1 -- 172.21.15.19:0/1310076372 >> 172.21.15.19:0/1310076372 conn(0x7fee301051e0 msgr2=0x7fee30107630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:31.924 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.168+0000 7fee34872700 1 -- 172.21.15.19:0/1310076372 shutdown_connections 2024-05-09T02:00:31.924 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.168+0000 7fee34872700 1 -- 172.21.15.19:0/1310076372 wait complete. 2024-05-09T02:00:31.924 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.169+0000 7fee34872700 1 Processor -- start 2024-05-09T02:00:31.925 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.169+0000 7fee34872700 1 -- start start 2024-05-09T02:00:31.925 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.169+0000 7fee34872700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee30109b00 0x7fee3007f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:31.925 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.169+0000 7fee34872700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee3010e080 con 0x7fee30109b00 2024-05-09T02:00:31.925 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.169+0000 7fee2e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee30109b00 0x7fee3007f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:31.925 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.169+0000 7fee2e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee30109b00 0x7fee3007f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44118/0 (socket says 172.21.15.19:44118) 2024-05-09T02:00:31.925 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.169+0000 7fee2e59c700 1 -- 172.21.15.19:0/1784731519 learned_addr learned my addr 172.21.15.19:0/1784731519 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:31.925 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.169+0000 7fee2e59c700 1 -- 172.21.15.19:0/1784731519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3007f820 con 0x7fee30109b00 2024-05-09T02:00:31.925 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.170+0000 7fee2e59c700 1 --2- 172.21.15.19:0/1784731519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee30109b00 0x7fee3007f2e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee20009130 tx=0x7fee20011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:31.925 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.170+0000 7fee1f7fe700 1 -- 172.21.15.19:0/1784731519 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee20003aa0 con 0x7fee30109b00 2024-05-09T02:00:31.925 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.170+0000 7fee1f7fe700 1 -- 172.21.15.19:0/1784731519 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee2001a8e0 con 0x7fee30109b00 2024-05-09T02:00:31.925 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.170+0000 7fee1f7fe700 1 -- 172.21.15.19:0/1784731519 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee200248f0 con 0x7fee30109b00 2024-05-09T02:00:31.925 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.170+0000 7fee34872700 1 -- 172.21.15.19:0/1784731519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3007da70 con 0x7fee30109b00 2024-05-09T02:00:31.925 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.170+0000 7fee34872700 1 -- 172.21.15.19:0/1784731519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3007df50 con 0x7fee30109b00 2024-05-09T02:00:31.925 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.171+0000 7fee1f7fe700 1 -- 172.21.15.19:0/1784731519 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fee20022020 con 0x7fee30109b00 2024-05-09T02:00:31.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.171+0000 7fee34872700 1 -- 172.21.15.19:0/1784731519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee30064e20 con 0x7fee30109b00 2024-05-09T02:00:31.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.171+0000 7fee1f7fe700 1 --2- 172.21.15.19:0/1784731519 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fee1803e620 0x7fee18040b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:31.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.171+0000 7fee1f7fe700 1 -- 172.21.15.19:0/1784731519 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fee20053a60 con 0x7fee30109b00 2024-05-09T02:00:31.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.171+0000 7fee2dd9b700 1 --2- 172.21.15.19:0/1784731519 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fee1803e620 0x7fee18040b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:31.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.172+0000 7fee2dd9b700 1 --2- 172.21.15.19:0/1784731519 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fee1803e620 0x7fee18040b30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee24006fd0 tx=0x7fee24006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:31.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.176+0000 7fee1f7fe700 1 -- 172.21.15.19:0/1784731519 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee2002c350 con 0x7fee30109b00 2024-05-09T02:00:31.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.390+0000 7fee34872700 1 -- 172.21.15.19:0/1784731519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fee3007e670 con 0x7fee30109b00 2024-05-09T02:00:31.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.390+0000 7fee1f7fe700 1 -- 172.21.15.19:0/1784731519 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fee20029df0 con 0x7fee30109b00 2024-05-09T02:00:31.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.393+0000 7fee34872700 1 -- 172.21.15.19:0/1784731519 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fee1803e620 msgr2=0x7fee18040b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:31.926 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.393+0000 7fee34872700 1 --2- 172.21.15.19:0/1784731519 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fee1803e620 0x7fee18040b30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee24006fd0 tx=0x7fee24006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:31.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.393+0000 7fee34872700 1 -- 172.21.15.19:0/1784731519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee30109b00 msgr2=0x7fee3007f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:31.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.393+0000 7fee34872700 1 --2- 172.21.15.19:0/1784731519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee30109b00 0x7fee3007f2e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee20009130 tx=0x7fee20011aa0 comp rx=0 tx=0).stop 2024-05-09T02:00:31.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.393+0000 7fee34872700 1 -- 172.21.15.19:0/1784731519 shutdown_connections 2024-05-09T02:00:31.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.393+0000 7fee34872700 1 --2- 172.21.15.19:0/1784731519 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fee1803e620 0x7fee18040b30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:31.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.393+0000 7fee34872700 1 --2- 172.21.15.19:0/1784731519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee30109b00 0x7fee3007f2e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:31.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.393+0000 7fee34872700 1 -- 172.21.15.19:0/1784731519 >> 172.21.15.19:0/1784731519 conn(0x7fee301051e0 msgr2=0x7fee30106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:31.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.393+0000 7fee34872700 1 -- 172.21.15.19:0/1784731519 shutdown_connections 2024-05-09T02:00:31.927 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:31.393+0000 7fee34872700 1 -- 172.21.15.19:0/1784731519 wait complete. 2024-05-09T02:00:33.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.361+0000 7f3d0b687700 1 Processor -- start 2024-05-09T02:00:33.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.361+0000 7f3d0b687700 1 -- start start 2024-05-09T02:00:33.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.361+0000 7f3d0b687700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d041089d0 0x7f3d04108de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:33.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.361+0000 7f3d0b687700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d041093d0 con 0x7f3d041089d0 2024-05-09T02:00:33.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.362+0000 7f3d09423700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d041089d0 0x7f3d04108de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:33.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.362+0000 7f3d09423700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d041089d0 0x7f3d04108de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44120/0 (socket says 172.21.15.19:44120) 2024-05-09T02:00:33.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.362+0000 7f3d09423700 1 -- 172.21.15.19:0/1638177933 learned_addr learned my addr 172.21.15.19:0/1638177933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:33.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.362+0000 7f3d09423700 1 -- 172.21.15.19:0/1638177933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d04109ba0 con 0x7f3d041089d0 2024-05-09T02:00:33.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.362+0000 7f3d09423700 1 --2- 172.21.15.19:0/1638177933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d041089d0 0x7f3d04108de0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d000098a0 tx=0x7f3d00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb4380278c4dcbd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:33.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.362+0000 7f3cfbfff700 1 -- 172.21.15.19:0/1638177933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d00004160 con 0x7f3d041089d0 2024-05-09T02:00:33.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.362+0000 7f3cfbfff700 1 -- 172.21.15.19:0/1638177933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d00004300 con 0x7f3d041089d0 2024-05-09T02:00:33.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.362+0000 7f3cfbfff700 1 -- 172.21.15.19:0/1638177933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d00011a20 con 0x7f3d041089d0 2024-05-09T02:00:33.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.363+0000 7f3d0b687700 1 -- 172.21.15.19:0/1638177933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d041089d0 msgr2=0x7f3d04108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:33.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.363+0000 7f3d0b687700 1 --2- 172.21.15.19:0/1638177933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d041089d0 0x7f3d04108de0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d000098a0 tx=0x7f3d00009bb0 comp rx=0 tx=0).stop 2024-05-09T02:00:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.363+0000 7f3d0b687700 1 -- 172.21.15.19:0/1638177933 shutdown_connections 2024-05-09T02:00:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.363+0000 7f3d0b687700 1 --2- 172.21.15.19:0/1638177933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d041089d0 0x7f3d04108de0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.363+0000 7f3d0b687700 1 -- 172.21.15.19:0/1638177933 >> 172.21.15.19:0/1638177933 conn(0x7f3d04104010 msgr2=0x7f3d041064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.363+0000 7f3d0b687700 1 -- 172.21.15.19:0/1638177933 shutdown_connections 2024-05-09T02:00:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.363+0000 7f3d0b687700 1 -- 172.21.15.19:0/1638177933 wait complete. 2024-05-09T02:00:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.364+0000 7f3d0b687700 1 Processor -- start 2024-05-09T02:00:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.364+0000 7f3d0b687700 1 -- start start 2024-05-09T02:00:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.364+0000 7f3d0b687700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d041089d0 0x7f3d04106070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.364+0000 7f3d0b687700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0410cf50 con 0x7f3d041089d0 2024-05-09T02:00:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.364+0000 7f3d09423700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d041089d0 0x7f3d04106070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.364+0000 7f3d09423700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d041089d0 0x7f3d04106070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44126/0 (socket says 172.21.15.19:44126) 2024-05-09T02:00:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.364+0000 7f3d09423700 1 -- 172.21.15.19:0/1214616776 learned_addr learned my addr 172.21.15.19:0/1214616776 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.364+0000 7f3d09423700 1 -- 172.21.15.19:0/1214616776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d041065b0 con 0x7f3d041089d0 2024-05-09T02:00:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.365+0000 7f3d09423700 1 --2- 172.21.15.19:0/1214616776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d041089d0 0x7f3d04106070 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009130 tx=0x7f3d00011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.365+0000 7f3cfa7fc700 1 -- 172.21.15.19:0/1214616776 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d00003aa0 con 0x7f3d041089d0 2024-05-09T02:00:33.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.365+0000 7f3cfa7fc700 1 -- 172.21.15.19:0/1214616776 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d0001a8e0 con 0x7f3d041089d0 2024-05-09T02:00:33.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.365+0000 7f3d0b687700 1 -- 172.21.15.19:0/1214616776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d04104800 con 0x7f3d041089d0 2024-05-09T02:00:33.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.365+0000 7f3cfa7fc700 1 -- 172.21.15.19:0/1214616776 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d000248f0 con 0x7f3d041089d0 2024-05-09T02:00:33.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.365+0000 7f3d0b687700 1 -- 172.21.15.19:0/1214616776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d04104ce0 con 0x7f3d041089d0 2024-05-09T02:00:33.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.366+0000 7f3d0b687700 1 -- 172.21.15.19:0/1214616776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d04064e20 con 0x7f3d041089d0 2024-05-09T02:00:33.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.366+0000 7f3cfa7fc700 1 -- 172.21.15.19:0/1214616776 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3d00022020 con 0x7f3d041089d0 2024-05-09T02:00:33.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.366+0000 7f3cfa7fc700 1 --2- 172.21.15.19:0/1214616776 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3cf003e680 0x7f3cf0040b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:33.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.366+0000 7f3cfa7fc700 1 -- 172.21.15.19:0/1214616776 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3d00053b50 con 0x7f3d041089d0 2024-05-09T02:00:33.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.366+0000 7f3d08c22700 1 --2- 172.21.15.19:0/1214616776 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3cf003e680 0x7f3cf0040b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:33.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.367+0000 7f3d08c22700 1 --2- 172.21.15.19:0/1214616776 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3cf003e680 0x7f3cf0040b90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4006fd0 tx=0x7f3cf4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:33.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.372+0000 7f3cfa7fc700 1 -- 172.21.15.19:0/1214616776 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3d0001aa90 con 0x7f3d041089d0 2024-05-09T02:00:33.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.703+0000 7f3d0b687700 1 -- 172.21.15.19:0/1214616776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f3d04196fd0 con 0x7f3d041089d0 2024-05-09T02:00:33.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.706+0000 7f3cfa7fc700 1 -- 172.21.15.19:0/1214616776 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f3d0001ad40 con 0x7f3d041089d0 2024-05-09T02:00:33.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.709+0000 7f3d0b687700 1 -- 172.21.15.19:0/1214616776 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3cf003e680 msgr2=0x7f3cf0040b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:33.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.709+0000 7f3d0b687700 1 --2- 172.21.15.19:0/1214616776 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3cf003e680 0x7f3cf0040b90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4006fd0 tx=0x7f3cf4006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:33.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.709+0000 7f3d0b687700 1 -- 172.21.15.19:0/1214616776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d041089d0 msgr2=0x7f3d04106070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:33.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.709+0000 7f3d0b687700 1 --2- 172.21.15.19:0/1214616776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d041089d0 0x7f3d04106070 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009130 tx=0x7f3d00011aa0 comp rx=0 tx=0).stop 2024-05-09T02:00:33.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.709+0000 7f3d0b687700 1 -- 172.21.15.19:0/1214616776 shutdown_connections 2024-05-09T02:00:33.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.709+0000 7f3d0b687700 1 --2- 172.21.15.19:0/1214616776 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3cf003e680 0x7f3cf0040b90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:33.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.709+0000 7f3d0b687700 1 --2- 172.21.15.19:0/1214616776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d041089d0 0x7f3d04106070 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:33.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.709+0000 7f3d0b687700 1 -- 172.21.15.19:0/1214616776 >> 172.21.15.19:0/1214616776 conn(0x7f3d04104010 msgr2=0x7f3d04194410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:33.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.709+0000 7f3d0b687700 1 -- 172.21.15.19:0/1214616776 shutdown_connections 2024-05-09T02:00:33.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-09T02:00:32.709+0000 7f3d0b687700 1 -- 172.21.15.19:0/1214616776 wait complete. 2024-05-09T02:00:33.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-09T02:00:33.290 INFO:teuthology.orchestra.run.smithi019.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-09T02:00:33.290 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:00:33.290 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-09T02:00:33.290 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:00:33.291 INFO:teuthology.orchestra.run.smithi019.stdout:Or, if you are only running a single cluster on this host: 2024-05-09T02:00:33.291 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:00:33.291 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-09T02:00:33.291 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:00:33.291 INFO:teuthology.orchestra.run.smithi019.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-09T02:00:33.291 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:00:33.291 INFO:teuthology.orchestra.run.smithi019.stdout: ceph telemetry on 2024-05-09T02:00:33.291 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:00:33.291 INFO:teuthology.orchestra.run.smithi019.stdout:For more information see: 2024-05-09T02:00:33.291 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:00:33.291 INFO:teuthology.orchestra.run.smithi019.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-09T02:00:33.291 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:00:33.291 INFO:teuthology.orchestra.run.smithi019.stdout:Bootstrap complete. 2024-05-09T02:00:33.332 INFO:tasks.cephadm:Fetching config... 2024-05-09T02:00:33.332 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T02:00:33.332 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-09T02:00:33.349 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-09T02:00:33.350 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T02:00:33.350 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-09T02:00:33.408 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-09T02:00:33.408 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T02:00:33.408 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-09T02:00:33.474 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-09T02:00:33.474 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T02:00:33.475 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-09T02:00:33.532 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-09T02:00:33.533 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbWclUMyiQoX9eX9b3TOTneWODYIhWw4CY1M38viaSXZwm1T5QRprH+EIXg7ceA5s+WTWdZ7LPTq/oxjmu6hU81vZKTVrLvIcZbkfJPRbSkNvs03I+JOBIP+dlcELLOFTxRu8HgnXUO6mDHmwA00pXQH6cKvWgbYo+O1JkG7FnVVZjxhvb5wOu8QpLaAGZaOcxBBcqaSRG+IEa0nnyBV7lPCDtc4S+f0K2A+Tqmp1X9UKW6mLUlXS7Hw3G5nlUFGIy39DIY+Kn9uSApLa5MyLrPTx+L7Erm5jYGHsqTzhczmFvmk0NEyUMtR3wGOQKDGkeqiIeUKZLvrRHNZQ2Nv7K78g9g9/PCd3G7BzqiuHO+zaNm3nATyB5A2c1oc9G0FneFmUYaVO+EoX2bs3Efbqn010SkH5NZ5QnerMGGAaZvAilcE/iBiRIVAi+a6BpM+RcgSmRo6e2phnKU0yrUnjDuVX/Y9rIXPex+GH240klJayvN4c7Yn2tRWjvcZ2l4TU= ceph-a0611f3e-0da7-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T02:00:33.611 INFO:teuthology.orchestra.run.smithi019.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbWclUMyiQoX9eX9b3TOTneWODYIhWw4CY1M38viaSXZwm1T5QRprH+EIXg7ceA5s+WTWdZ7LPTq/oxjmu6hU81vZKTVrLvIcZbkfJPRbSkNvs03I+JOBIP+dlcELLOFTxRu8HgnXUO6mDHmwA00pXQH6cKvWgbYo+O1JkG7FnVVZjxhvb5wOu8QpLaAGZaOcxBBcqaSRG+IEa0nnyBV7lPCDtc4S+f0K2A+Tqmp1X9UKW6mLUlXS7Hw3G5nlUFGIy39DIY+Kn9uSApLa5MyLrPTx+L7Erm5jYGHsqTzhczmFvmk0NEyUMtR3wGOQKDGkeqiIeUKZLvrRHNZQ2Nv7K78g9g9/PCd3G7BzqiuHO+zaNm3nATyB5A2c1oc9G0FneFmUYaVO+EoX2bs3Efbqn010SkH5NZ5QnerMGGAaZvAilcE/iBiRIVAi+a6BpM+RcgSmRo6e2phnKU0yrUnjDuVX/Y9rIXPex+GH240klJayvN4c7Yn2tRWjvcZ2l4TU= ceph-a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:00:33.623 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbWclUMyiQoX9eX9b3TOTneWODYIhWw4CY1M38viaSXZwm1T5QRprH+EIXg7ceA5s+WTWdZ7LPTq/oxjmu6hU81vZKTVrLvIcZbkfJPRbSkNvs03I+JOBIP+dlcELLOFTxRu8HgnXUO6mDHmwA00pXQH6cKvWgbYo+O1JkG7FnVVZjxhvb5wOu8QpLaAGZaOcxBBcqaSRG+IEa0nnyBV7lPCDtc4S+f0K2A+Tqmp1X9UKW6mLUlXS7Hw3G5nlUFGIy39DIY+Kn9uSApLa5MyLrPTx+L7Erm5jYGHsqTzhczmFvmk0NEyUMtR3wGOQKDGkeqiIeUKZLvrRHNZQ2Nv7K78g9g9/PCd3G7BzqiuHO+zaNm3nATyB5A2c1oc9G0FneFmUYaVO+EoX2bs3Efbqn010SkH5NZ5QnerMGGAaZvAilcE/iBiRIVAi+a6BpM+RcgSmRo6e2phnKU0yrUnjDuVX/Y9rIXPex+GH240klJayvN4c7Yn2tRWjvcZ2l4TU= ceph-a0611f3e-0da7-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T02:00:33.665 INFO:teuthology.orchestra.run.smithi059.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbWclUMyiQoX9eX9b3TOTneWODYIhWw4CY1M38viaSXZwm1T5QRprH+EIXg7ceA5s+WTWdZ7LPTq/oxjmu6hU81vZKTVrLvIcZbkfJPRbSkNvs03I+JOBIP+dlcELLOFTxRu8HgnXUO6mDHmwA00pXQH6cKvWgbYo+O1JkG7FnVVZjxhvb5wOu8QpLaAGZaOcxBBcqaSRG+IEa0nnyBV7lPCDtc4S+f0K2A+Tqmp1X9UKW6mLUlXS7Hw3G5nlUFGIy39DIY+Kn9uSApLa5MyLrPTx+L7Erm5jYGHsqTzhczmFvmk0NEyUMtR3wGOQKDGkeqiIeUKZLvrRHNZQ2Nv7K78g9g9/PCd3G7BzqiuHO+zaNm3nATyB5A2c1oc9G0FneFmUYaVO+EoX2bs3Efbqn010SkH5NZ5QnerMGGAaZvAilcE/iBiRIVAi+a6BpM+RcgSmRo6e2phnKU0yrUnjDuVX/Y9rIXPex+GH240klJayvN4c7Yn2tRWjvcZ2l4TU= ceph-a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:00:33.677 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbWclUMyiQoX9eX9b3TOTneWODYIhWw4CY1M38viaSXZwm1T5QRprH+EIXg7ceA5s+WTWdZ7LPTq/oxjmu6hU81vZKTVrLvIcZbkfJPRbSkNvs03I+JOBIP+dlcELLOFTxRu8HgnXUO6mDHmwA00pXQH6cKvWgbYo+O1JkG7FnVVZjxhvb5wOu8QpLaAGZaOcxBBcqaSRG+IEa0nnyBV7lPCDtc4S+f0K2A+Tqmp1X9UKW6mLUlXS7Hw3G5nlUFGIy39DIY+Kn9uSApLa5MyLrPTx+L7Erm5jYGHsqTzhczmFvmk0NEyUMtR3wGOQKDGkeqiIeUKZLvrRHNZQ2Nv7K78g9g9/PCd3G7BzqiuHO+zaNm3nATyB5A2c1oc9G0FneFmUYaVO+EoX2bs3Efbqn010SkH5NZ5QnerMGGAaZvAilcE/iBiRIVAi+a6BpM+RcgSmRo6e2phnKU0yrUnjDuVX/Y9rIXPex+GH240klJayvN4c7Yn2tRWjvcZ2l4TU= ceph-a0611f3e-0da7-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T02:00:33.719 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbWclUMyiQoX9eX9b3TOTneWODYIhWw4CY1M38viaSXZwm1T5QRprH+EIXg7ceA5s+WTWdZ7LPTq/oxjmu6hU81vZKTVrLvIcZbkfJPRbSkNvs03I+JOBIP+dlcELLOFTxRu8HgnXUO6mDHmwA00pXQH6cKvWgbYo+O1JkG7FnVVZjxhvb5wOu8QpLaAGZaOcxBBcqaSRG+IEa0nnyBV7lPCDtc4S+f0K2A+Tqmp1X9UKW6mLUlXS7Hw3G5nlUFGIy39DIY+Kn9uSApLa5MyLrPTx+L7Erm5jYGHsqTzhczmFvmk0NEyUMtR3wGOQKDGkeqiIeUKZLvrRHNZQ2Nv7K78g9g9/PCd3G7BzqiuHO+zaNm3nATyB5A2c1oc9G0FneFmUYaVO+EoX2bs3Efbqn010SkH5NZ5QnerMGGAaZvAilcE/iBiRIVAi+a6BpM+RcgSmRo6e2phnKU0yrUnjDuVX/Y9rIXPex+GH240klJayvN4c7Yn2tRWjvcZ2l4TU= ceph-a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:00:33.731 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-09T02:00:33.817 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:33 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1214616776' entity='client.admin' 2024-05-09T02:00:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:00:35.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.981+0000 7f582a79b700 1 -- 172.21.15.19:0/3299150956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5824102ae0 msgr2=0x7f5824102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:35.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.981+0000 7f582a79b700 1 --2- 172.21.15.19:0/3299150956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5824102ae0 0x7f5824102ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5814009910 tx=0x7f5814009c20 comp rx=0 tx=0).stop 2024-05-09T02:00:35.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.981+0000 7f582a79b700 1 -- 172.21.15.19:0/3299150956 shutdown_connections 2024-05-09T02:00:35.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.981+0000 7f582a79b700 1 --2- 172.21.15.19:0/3299150956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5824102ae0 0x7f5824102ef0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:35.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.981+0000 7f582a79b700 1 -- 172.21.15.19:0/3299150956 >> 172.21.15.19:0/3299150956 conn(0x7f58240fe140 msgr2=0x7f58241005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:35.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.981+0000 7f582a79b700 1 -- 172.21.15.19:0/3299150956 shutdown_connections 2024-05-09T02:00:35.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.981+0000 7f582a79b700 1 -- 172.21.15.19:0/3299150956 wait complete. 2024-05-09T02:00:35.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.981+0000 7f582a79b700 1 Processor -- start 2024-05-09T02:00:35.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.981+0000 7f582a79b700 1 -- start start 2024-05-09T02:00:35.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.982+0000 7f582a79b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5824102ae0 0x7f58240fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:35.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.982+0000 7f582a79b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5824108aa0 con 0x7f5824102ae0 2024-05-09T02:00:35.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.982+0000 7f5823fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5824102ae0 0x7f58240fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:35.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.982+0000 7f5823fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5824102ae0 0x7f58240fff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44152/0 (socket says 172.21.15.19:44152) 2024-05-09T02:00:35.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.982+0000 7f5823fff700 1 -- 172.21.15.19:0/3890415581 learned_addr learned my addr 172.21.15.19:0/3890415581 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:35.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.982+0000 7f5823fff700 1 -- 172.21.15.19:0/3890415581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5824100490 con 0x7f5824102ae0 2024-05-09T02:00:35.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.982+0000 7f5823fff700 1 --2- 172.21.15.19:0/3890415581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5824102ae0 0x7f58240fff50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58140057d0 tx=0x7f5814005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:35.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.983+0000 7f58217fa700 1 -- 172.21.15.19:0/3890415581 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5814027070 con 0x7f5824102ae0 2024-05-09T02:00:35.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.983+0000 7f58217fa700 1 -- 172.21.15.19:0/3890415581 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f581400be10 con 0x7f5824102ae0 2024-05-09T02:00:35.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.983+0000 7f58217fa700 1 -- 172.21.15.19:0/3890415581 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5814022dd0 con 0x7f5824102ae0 2024-05-09T02:00:35.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.983+0000 7f582a79b700 1 -- 172.21.15.19:0/3890415581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58240fe6e0 con 0x7f5824102ae0 2024-05-09T02:00:35.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.983+0000 7f582a79b700 1 -- 172.21.15.19:0/3890415581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58240febc0 con 0x7f5824102ae0 2024-05-09T02:00:35.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.984+0000 7f582a79b700 1 -- 172.21.15.19:0/3890415581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5824109220 con 0x7f5824102ae0 2024-05-09T02:00:35.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.984+0000 7f58217fa700 1 -- 172.21.15.19:0/3890415581 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f581402a070 con 0x7f5824102ae0 2024-05-09T02:00:35.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.984+0000 7f58217fa700 1 --2- 172.21.15.19:0/3890415581 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f580c03e4c0 0x7f580c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:35.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.984+0000 7f58217fa700 1 -- 172.21.15.19:0/3890415581 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f581401e030 con 0x7f5824102ae0 2024-05-09T02:00:35.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.984+0000 7f58237fe700 1 --2- 172.21.15.19:0/3890415581 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f580c03e4c0 0x7f580c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:35.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.985+0000 7f58237fe700 1 --2- 172.21.15.19:0/3890415581 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f580c03e4c0 0x7f580c0409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5818006fd0 tx=0x7f5818006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:35.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:35.990+0000 7f58217fa700 1 -- 172.21.15.19:0/3890415581 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5814013070 con 0x7f5824102ae0 2024-05-09T02:00:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:36.206+0000 7f582a79b700 1 -- 172.21.15.19:0/3890415581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f58241926e0 con 0x7f5824102ae0 2024-05-09T02:00:36.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:36.210+0000 7f58217fa700 1 -- 172.21.15.19:0/3890415581 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f581400f460 con 0x7f5824102ae0 2024-05-09T02:00:36.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:36.211+0000 7f582a79b700 1 -- 172.21.15.19:0/3890415581 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f580c03e4c0 msgr2=0x7f580c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:36.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:36.211+0000 7f582a79b700 1 --2- 172.21.15.19:0/3890415581 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f580c03e4c0 0x7f580c0409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5818006fd0 tx=0x7f5818006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:36.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:36.211+0000 7f582a79b700 1 -- 172.21.15.19:0/3890415581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5824102ae0 msgr2=0x7f58240fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:36.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:36.211+0000 7f582a79b700 1 --2- 172.21.15.19:0/3890415581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5824102ae0 0x7f58240fff50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58140057d0 tx=0x7f5814005800 comp rx=0 tx=0).stop 2024-05-09T02:00:36.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:36.211+0000 7f582a79b700 1 -- 172.21.15.19:0/3890415581 shutdown_connections 2024-05-09T02:00:36.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:36.211+0000 7f582a79b700 1 --2- 172.21.15.19:0/3890415581 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f580c03e4c0 0x7f580c0409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:36.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:36.211+0000 7f582a79b700 1 --2- 172.21.15.19:0/3890415581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5824102ae0 0x7f58240fff50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:36.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:36.211+0000 7f582a79b700 1 -- 172.21.15.19:0/3890415581 >> 172.21.15.19:0/3890415581 conn(0x7f58240fe140 msgr2=0x7f582406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:36.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:36.211+0000 7f582a79b700 1 -- 172.21.15.19:0/3890415581 shutdown_connections 2024-05-09T02:00:36.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:36.211+0000 7f582a79b700 1 -- 172.21.15.19:0/3890415581 wait complete. 2024-05-09T02:00:36.988 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-09T02:00:36.988 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-09T02:00:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:00:37.488 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:37 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3890415581' entity='client.admin' 2024-05-09T02:00:37.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:37.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:37.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:00:37.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:37.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:00:37.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.750+0000 7f85fb240700 1 -- 172.21.15.19:0/438276649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4104730 msgr2=0x7f85f4106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.750+0000 7f85fb240700 1 --2- 172.21.15.19:0/438276649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4104730 0x7f85f4106b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009910 tx=0x7f85e4009c20 comp rx=0 tx=0).stop 2024-05-09T02:00:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.750+0000 7f85fb240700 1 -- 172.21.15.19:0/438276649 shutdown_connections 2024-05-09T02:00:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.750+0000 7f85fb240700 1 --2- 172.21.15.19:0/438276649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4104730 0x7f85f4106b50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.750+0000 7f85fb240700 1 -- 172.21.15.19:0/438276649 >> 172.21.15.19:0/438276649 conn(0x7f85f40fe110 msgr2=0x7f85f41005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.751+0000 7f85fb240700 1 -- 172.21.15.19:0/438276649 shutdown_connections 2024-05-09T02:00:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.751+0000 7f85fb240700 1 -- 172.21.15.19:0/438276649 wait complete. 2024-05-09T02:00:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.751+0000 7f85fb240700 1 Processor -- start 2024-05-09T02:00:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.751+0000 7f85fb240700 1 -- start start 2024-05-09T02:00:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.751+0000 7f85fb240700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4104730 0x7f85f4103eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:38.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.752+0000 7f85fb240700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85f4109b90 con 0x7f85f4104730 2024-05-09T02:00:38.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.752+0000 7f85f8fdc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4104730 0x7f85f4103eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:38.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.752+0000 7f85f8fdc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4104730 0x7f85f4103eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57672/0 (socket says 172.21.15.19:57672) 2024-05-09T02:00:38.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.752+0000 7f85f8fdc700 1 -- 172.21.15.19:0/1766518269 learned_addr learned my addr 172.21.15.19:0/1766518269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.752+0000 7f85f8fdc700 1 -- 172.21.15.19:0/1766518269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f41043f0 con 0x7f85f4104730 2024-05-09T02:00:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.752+0000 7f85f8fdc700 1 --2- 172.21.15.19:0/1766518269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4104730 0x7f85f4103eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f85e4006010 tx=0x7f85e4005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.752+0000 7f85f1ffb700 1 -- 172.21.15.19:0/1766518269 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85e4026070 con 0x7f85f4104730 2024-05-09T02:00:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.752+0000 7f85fb240700 1 -- 172.21.15.19:0/1766518269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f4102640 con 0x7f85f4104730 2024-05-09T02:00:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.752+0000 7f85f1ffb700 1 -- 172.21.15.19:0/1766518269 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85e40044a0 con 0x7f85f4104730 2024-05-09T02:00:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.752+0000 7f85f1ffb700 1 -- 172.21.15.19:0/1766518269 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85e40039d0 con 0x7f85f4104730 2024-05-09T02:00:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.752+0000 7f85fb240700 1 -- 172.21.15.19:0/1766518269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f4102b20 con 0x7f85f4104730 2024-05-09T02:00:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.753+0000 7f85fb240700 1 -- 172.21.15.19:0/1766518269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f410a310 con 0x7f85f4104730 2024-05-09T02:00:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.753+0000 7f85f1ffb700 1 -- 172.21.15.19:0/1766518269 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f85e4029070 con 0x7f85f4104730 2024-05-09T02:00:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.754+0000 7f85f1ffb700 1 --2- 172.21.15.19:0/1766518269 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f85dc03e510 0x7f85dc040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.754+0000 7f85f1ffb700 1 -- 172.21.15.19:0/1766518269 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f85e401e070 con 0x7f85f4104730 2024-05-09T02:00:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.754+0000 7f85f3fff700 1 --2- 172.21.15.19:0/1766518269 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f85dc03e510 0x7f85dc040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.754+0000 7f85f3fff700 1 --2- 172.21.15.19:0/1766518269 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f85dc03e510 0x7f85dc040a20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f85e8006fd0 tx=0x7f85e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.759+0000 7f85f1ffb700 1 -- 172.21.15.19:0/1766518269 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f85e4013070 con 0x7f85f4104730 2024-05-09T02:00:38.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.978+0000 7f85fb240700 1 -- 172.21.15.19:0/1766518269 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f85f4192530 con 0x7f85dc03e510 2024-05-09T02:00:38.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.982+0000 7f85f1ffb700 1 -- 172.21.15.19:0/1766518269 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f85f4192530 con 0x7f85dc03e510 2024-05-09T02:00:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.984+0000 7f85fb240700 1 -- 172.21.15.19:0/1766518269 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f85dc03e510 msgr2=0x7f85dc040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.984+0000 7f85fb240700 1 --2- 172.21.15.19:0/1766518269 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f85dc03e510 0x7f85dc040a20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f85e8006fd0 tx=0x7f85e8006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.984+0000 7f85fb240700 1 -- 172.21.15.19:0/1766518269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4104730 msgr2=0x7f85f4103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.984+0000 7f85fb240700 1 --2- 172.21.15.19:0/1766518269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4104730 0x7f85f4103eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f85e4006010 tx=0x7f85e4005ad0 comp rx=0 tx=0).stop 2024-05-09T02:00:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.984+0000 7f85fb240700 1 -- 172.21.15.19:0/1766518269 shutdown_connections 2024-05-09T02:00:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.984+0000 7f85fb240700 1 --2- 172.21.15.19:0/1766518269 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f85dc03e510 0x7f85dc040a20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.984+0000 7f85fb240700 1 --2- 172.21.15.19:0/1766518269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85f4104730 0x7f85f4103eb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.984+0000 7f85fb240700 1 -- 172.21.15.19:0/1766518269 >> 172.21.15.19:0/1766518269 conn(0x7f85f40fe110 msgr2=0x7f85f40fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.984+0000 7f85fb240700 1 -- 172.21.15.19:0/1766518269 shutdown_connections 2024-05-09T02:00:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:38.984+0000 7f85fb240700 1 -- 172.21.15.19:0/1766518269 wait complete. 2024-05-09T02:00:39.738 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi059 2024-05-09T02:00:39.738 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T02:00:39.739 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/etc/ceph/ceph.conf 2024-05-09T02:00:39.757 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T02:00:39.757 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:00:39.815 INFO:tasks.cephadm:Adding host smithi059 to orchestrator... 2024-05-09T02:00:39.815 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch host add smithi059 2024-05-09T02:00:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:00:40.242 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:39 smithi019 ceph-mon[30151]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:00:40.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:40.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:00:40.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:00:40.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:00:40.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:39 smithi019 ceph-mon[30151]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-09T02:00:40.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:39 smithi019 ceph-mon[30151]: Updating smithi019:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:00:41.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:41 smithi019 ceph-mon[30151]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:00:41.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:41 smithi019 ceph-mon[30151]: Updating smithi019:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T02:00:41.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:41 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:41.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:41 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:41.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:41 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.587+0000 7f9524a83700 1 -- 172.21.15.19:0/3861518650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9520104cc0 msgr2=0x7f95201050d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.587+0000 7f9524a83700 1 --2- 172.21.15.19:0/3861518650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9520104cc0 0x7f95201050d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f951000b0a0 tx=0x7f951000b3b0 comp rx=0 tx=0).stop 2024-05-09T02:00:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.587+0000 7f9524a83700 1 -- 172.21.15.19:0/3861518650 shutdown_connections 2024-05-09T02:00:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.587+0000 7f9524a83700 1 --2- 172.21.15.19:0/3861518650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9520104cc0 0x7f95201050d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.587+0000 7f9524a83700 1 -- 172.21.15.19:0/3861518650 >> 172.21.15.19:0/3861518650 conn(0x7f95200fe570 msgr2=0x7f95201009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.587+0000 7f9524a83700 1 -- 172.21.15.19:0/3861518650 shutdown_connections 2024-05-09T02:00:41.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.587+0000 7f9524a83700 1 -- 172.21.15.19:0/3861518650 wait complete. 2024-05-09T02:00:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.588+0000 7f9524a83700 1 Processor -- start 2024-05-09T02:00:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.589+0000 7f9524a83700 1 -- start start 2024-05-09T02:00:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.589+0000 7f9524a83700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9520104cc0 0x7f952019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.589+0000 7f9524a83700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f952010e120 con 0x7f9520104cc0 2024-05-09T02:00:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.589+0000 7f951e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9520104cc0 0x7f952019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.589+0000 7f951e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9520104cc0 0x7f952019f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57682/0 (socket says 172.21.15.19:57682) 2024-05-09T02:00:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.589+0000 7f951e59c700 1 -- 172.21.15.19:0/824035005 learned_addr learned my addr 172.21.15.19:0/824035005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.589+0000 7f951e59c700 1 -- 172.21.15.19:0/824035005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952019fb00 con 0x7f9520104cc0 2024-05-09T02:00:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.589+0000 7f951e59c700 1 --2- 172.21.15.19:0/824035005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9520104cc0 0x7f952019f5c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f951000d010 tx=0x7f9510009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.590+0000 7f95177fe700 1 -- 172.21.15.19:0/824035005 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9510021020 con 0x7f9520104cc0 2024-05-09T02:00:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.590+0000 7f95177fe700 1 -- 172.21.15.19:0/824035005 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9510004000 con 0x7f9520104cc0 2024-05-09T02:00:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.590+0000 7f95177fe700 1 -- 172.21.15.19:0/824035005 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9510017040 con 0x7f9520104cc0 2024-05-09T02:00:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.590+0000 7f9524a83700 1 -- 172.21.15.19:0/824035005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952019fd90 con 0x7f9520104cc0 2024-05-09T02:00:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.590+0000 7f9524a83700 1 -- 172.21.15.19:0/824035005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9520199750 con 0x7f9520104cc0 2024-05-09T02:00:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.591+0000 7f9524a83700 1 -- 172.21.15.19:0/824035005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f952006c3c0 con 0x7f9520104cc0 2024-05-09T02:00:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.591+0000 7f95177fe700 1 -- 172.21.15.19:0/824035005 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f9510024020 con 0x7f9520104cc0 2024-05-09T02:00:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.591+0000 7f95177fe700 1 --2- 172.21.15.19:0/824035005 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f950c03e5b0 0x7f950c040ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.591+0000 7f95177fe700 1 -- 172.21.15.19:0/824035005 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f951000f050 con 0x7f9520104cc0 2024-05-09T02:00:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.591+0000 7f951dd9b700 1 --2- 172.21.15.19:0/824035005 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f950c03e5b0 0x7f950c040ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.591+0000 7f951dd9b700 1 --2- 172.21.15.19:0/824035005 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f950c03e5b0 0x7f950c040ac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9508006fd0 tx=0x7f9508006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.596+0000 7f95177fe700 1 -- 172.21.15.19:0/824035005 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f951002b020 con 0x7f9520104cc0 2024-05-09T02:00:41.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:41.814+0000 7f9524a83700 1 -- 172.21.15.19:0/824035005 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi059", "target": ["mon-mgr", ""]}) -- 0x7f952019ac60 con 0x7f950c03e5b0 2024-05-09T02:00:42.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:42 smithi019 ceph-mon[30151]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi059", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:00:43.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:43 smithi019 ceph-mon[30151]: Deploying cephadm binary to smithi059 2024-05-09T02:00:45.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:44 smithi019 ceph-mon[30151]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:00:46.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:46.122+0000 7f95177fe700 1 -- 172.21.15.19:0/824035005 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f952019ac60 con 0x7f950c03e5b0 2024-05-09T02:00:46.122 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi059' with addr '172.21.15.59' 2024-05-09T02:00:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:46.124+0000 7f9524a83700 1 -- 172.21.15.19:0/824035005 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f950c03e5b0 msgr2=0x7f950c040ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:46.124+0000 7f9524a83700 1 --2- 172.21.15.19:0/824035005 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f950c03e5b0 0x7f950c040ac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9508006fd0 tx=0x7f9508006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:46.124+0000 7f9524a83700 1 -- 172.21.15.19:0/824035005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9520104cc0 msgr2=0x7f952019f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:46.124+0000 7f9524a83700 1 --2- 172.21.15.19:0/824035005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9520104cc0 0x7f952019f5c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f951000d010 tx=0x7f9510009790 comp rx=0 tx=0).stop 2024-05-09T02:00:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:46.124+0000 7f9524a83700 1 -- 172.21.15.19:0/824035005 shutdown_connections 2024-05-09T02:00:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:46.124+0000 7f9524a83700 1 --2- 172.21.15.19:0/824035005 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f950c03e5b0 0x7f950c040ac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:46.124+0000 7f9524a83700 1 --2- 172.21.15.19:0/824035005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9520104cc0 0x7f952019f5c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:46.125+0000 7f9524a83700 1 -- 172.21.15.19:0/824035005 >> 172.21.15.19:0/824035005 conn(0x7f95200fe570 msgr2=0x7f9520105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:46.125+0000 7f9524a83700 1 -- 172.21.15.19:0/824035005 shutdown_connections 2024-05-09T02:00:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:46.125+0000 7f9524a83700 1 -- 172.21.15.19:0/824035005 wait complete. 2024-05-09T02:00:46.740 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T02:00:46.993 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:00:47.449 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:47 smithi019 ceph-mon[30151]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:00:47.449 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:47.449 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:47 smithi019 ceph-mon[30151]: Added host smithi059 2024-05-09T02:00:47.449 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:00:47.449 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:48.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:48 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:48.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.606+0000 7fad850fb700 1 -- 172.21.15.19:0/2703969815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad80108a60 msgr2=0x7fad80108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:48.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.606+0000 7fad850fb700 1 --2- 172.21.15.19:0/2703969815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad80108a60 0x7fad80108e70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fad68009910 tx=0x7fad68009c20 comp rx=0 tx=0).stop 2024-05-09T02:00:48.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.606+0000 7fad850fb700 1 -- 172.21.15.19:0/2703969815 shutdown_connections 2024-05-09T02:00:48.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.606+0000 7fad850fb700 1 --2- 172.21.15.19:0/2703969815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad80108a60 0x7fad80108e70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:48.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.606+0000 7fad850fb700 1 -- 172.21.15.19:0/2703969815 >> 172.21.15.19:0/2703969815 conn(0x7fad800fe5b0 msgr2=0x7fad80100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:48.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.606+0000 7fad850fb700 1 -- 172.21.15.19:0/2703969815 shutdown_connections 2024-05-09T02:00:48.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.606+0000 7fad850fb700 1 -- 172.21.15.19:0/2703969815 wait complete. 2024-05-09T02:00:48.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.607+0000 7fad850fb700 1 Processor -- start 2024-05-09T02:00:48.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.607+0000 7fad850fb700 1 -- start start 2024-05-09T02:00:48.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.607+0000 7fad850fb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad80108a60 0x7fad800ff530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:48.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.607+0000 7fad850fb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad8010cfb0 con 0x7fad80108a60 2024-05-09T02:00:48.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.608+0000 7fad7ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad80108a60 0x7fad800ff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.608+0000 7fad7ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad80108a60 0x7fad800ff530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49410/0 (socket says 172.21.15.19:49410) 2024-05-09T02:00:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.608+0000 7fad7ed9d700 1 -- 172.21.15.19:0/3629773251 learned_addr learned my addr 172.21.15.19:0/3629773251 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.608+0000 7fad7ed9d700 1 -- 172.21.15.19:0/3629773251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad80101380 con 0x7fad80108a60 2024-05-09T02:00:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.608+0000 7fad7ed9d700 1 --2- 172.21.15.19:0/3629773251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad80108a60 0x7fad800ff530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fad680056d0 tx=0x7fad680057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.608+0000 7fad77fff700 1 -- 172.21.15.19:0/3629773251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad68027070 con 0x7fad80108a60 2024-05-09T02:00:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.608+0000 7fad77fff700 1 -- 172.21.15.19:0/3629773251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad6800be10 con 0x7fad80108a60 2024-05-09T02:00:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.608+0000 7fad77fff700 1 -- 172.21.15.19:0/3629773251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad68022d20 con 0x7fad80108a60 2024-05-09T02:00:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.608+0000 7fad850fb700 1 -- 172.21.15.19:0/3629773251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad800ffa70 con 0x7fad80108a60 2024-05-09T02:00:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.608+0000 7fad850fb700 1 -- 172.21.15.19:0/3629773251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad800fff30 con 0x7fad80108a60 2024-05-09T02:00:48.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.609+0000 7fad77fff700 1 -- 172.21.15.19:0/3629773251 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fad6802a070 con 0x7fad80108a60 2024-05-09T02:00:48.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.609+0000 7fad850fb700 1 -- 172.21.15.19:0/3629773251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad80104170 con 0x7fad80108a60 2024-05-09T02:00:48.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.610+0000 7fad77fff700 1 --2- 172.21.15.19:0/3629773251 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fad6c03e570 0x7fad6c040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:48.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.610+0000 7fad77fff700 1 -- 172.21.15.19:0/3629773251 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fad6801e030 con 0x7fad80108a60 2024-05-09T02:00:48.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.610+0000 7fad7e59c700 1 --2- 172.21.15.19:0/3629773251 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fad6c03e570 0x7fad6c040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:48.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.610+0000 7fad7e59c700 1 --2- 172.21.15.19:0/3629773251 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fad6c03e570 0x7fad6c040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad70006fd0 tx=0x7fad70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:48.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.615+0000 7fad77fff700 1 -- 172.21.15.19:0/3629773251 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad68013070 con 0x7fad80108a60 2024-05-09T02:00:48.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.836+0000 7fad850fb700 1 -- 172.21.15.19:0/3629773251 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fad80100b00 con 0x7fad6c03e570 2024-05-09T02:00:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.837+0000 7fad77fff700 1 -- 172.21.15.19:0/3629773251 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fad80100b00 con 0x7fad6c03e570 2024-05-09T02:00:48.838 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:00:48.838 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.59", "hostname": "smithi059", "labels": [], "status": ""}] 2024-05-09T02:00:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.839+0000 7fad850fb700 1 -- 172.21.15.19:0/3629773251 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fad6c03e570 msgr2=0x7fad6c040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.839+0000 7fad850fb700 1 --2- 172.21.15.19:0/3629773251 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fad6c03e570 0x7fad6c040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad70006fd0 tx=0x7fad70006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.839+0000 7fad850fb700 1 -- 172.21.15.19:0/3629773251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad80108a60 msgr2=0x7fad800ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.839+0000 7fad850fb700 1 --2- 172.21.15.19:0/3629773251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad80108a60 0x7fad800ff530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fad680056d0 tx=0x7fad680057b0 comp rx=0 tx=0).stop 2024-05-09T02:00:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.839+0000 7fad850fb700 1 -- 172.21.15.19:0/3629773251 shutdown_connections 2024-05-09T02:00:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.839+0000 7fad850fb700 1 --2- 172.21.15.19:0/3629773251 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fad6c03e570 0x7fad6c040a80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.839+0000 7fad850fb700 1 --2- 172.21.15.19:0/3629773251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad80108a60 0x7fad800ff530 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.839+0000 7fad850fb700 1 -- 172.21.15.19:0/3629773251 >> 172.21.15.19:0/3629773251 conn(0x7fad800fe5b0 msgr2=0x7fad80106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.839+0000 7fad850fb700 1 -- 172.21.15.19:0/3629773251 shutdown_connections 2024-05-09T02:00:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:48.839+0000 7fad850fb700 1 -- 172.21.15.19:0/3629773251 wait complete. 2024-05-09T02:00:49.500 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:49 smithi019 ceph-mon[30151]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:00:49.500 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:49 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:49.533 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-05-09T02:00:49.533 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T02:00:49.533 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-05-09T02:00:49.551 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T02:00:49.551 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:00:49.609 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-05-09T02:00:49.609 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch host add smithi196 2024-05-09T02:00:49.863 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:00:50.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:50 smithi019 ceph-mon[30151]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:00:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.424+0000 7fdb93568700 1 -- 172.21.15.19:0/264251999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb8c108a60 msgr2=0x7fdb8c108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.424+0000 7fdb93568700 1 --2- 172.21.15.19:0/264251999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb8c108a60 0x7fdb8c108e70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009910 tx=0x7fdb88009c20 comp rx=0 tx=0).stop 2024-05-09T02:00:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.424+0000 7fdb93568700 1 -- 172.21.15.19:0/264251999 shutdown_connections 2024-05-09T02:00:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.424+0000 7fdb93568700 1 --2- 172.21.15.19:0/264251999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb8c108a60 0x7fdb8c108e70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.424+0000 7fdb93568700 1 -- 172.21.15.19:0/264251999 >> 172.21.15.19:0/264251999 conn(0x7fdb8c0fe570 msgr2=0x7fdb8c1009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.424+0000 7fdb93568700 1 -- 172.21.15.19:0/264251999 shutdown_connections 2024-05-09T02:00:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.424+0000 7fdb93568700 1 -- 172.21.15.19:0/264251999 wait complete. 2024-05-09T02:00:51.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:51 smithi019 ceph-mon[30151]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:00:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.425+0000 7fdb93568700 1 Processor -- start 2024-05-09T02:00:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.425+0000 7fdb93568700 1 -- start start 2024-05-09T02:00:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.425+0000 7fdb93568700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb8c108a60 0x7fdb8c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.425+0000 7fdb93568700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c10cfb0 con 0x7fdb8c108a60 2024-05-09T02:00:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.425+0000 7fdb91304700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb8c108a60 0x7fdb8c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.425+0000 7fdb91304700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb8c108a60 0x7fdb8c19f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49416/0 (socket says 172.21.15.19:49416) 2024-05-09T02:00:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.425+0000 7fdb91304700 1 -- 172.21.15.19:0/716971678 learned_addr learned my addr 172.21.15.19:0/716971678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.425+0000 7fdb91304700 1 -- 172.21.15.19:0/716971678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8c19fb30 con 0x7fdb8c108a60 2024-05-09T02:00:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.426+0000 7fdb91304700 1 --2- 172.21.15.19:0/716971678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb8c108a60 0x7fdb8c19f5f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb880056d0 tx=0x7fdb880057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.426+0000 7fdb827fc700 1 -- 172.21.15.19:0/716971678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb88027070 con 0x7fdb8c108a60 2024-05-09T02:00:51.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.426+0000 7fdb827fc700 1 -- 172.21.15.19:0/716971678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb8800be10 con 0x7fdb8c108a60 2024-05-09T02:00:51.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.426+0000 7fdb93568700 1 -- 172.21.15.19:0/716971678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8c19fe20 con 0x7fdb8c108a60 2024-05-09T02:00:51.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.426+0000 7fdb827fc700 1 -- 172.21.15.19:0/716971678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb88022d20 con 0x7fdb8c108a60 2024-05-09T02:00:51.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.426+0000 7fdb93568700 1 -- 172.21.15.19:0/716971678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8c199950 con 0x7fdb8c108a60 2024-05-09T02:00:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.427+0000 7fdb827fc700 1 -- 172.21.15.19:0/716971678 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fdb8802a070 con 0x7fdb8c108a60 2024-05-09T02:00:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.427+0000 7fdb93568700 1 -- 172.21.15.19:0/716971678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb8c104170 con 0x7fdb8c108a60 2024-05-09T02:00:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.427+0000 7fdb827fc700 1 --2- 172.21.15.19:0/716971678 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fdb7803e510 0x7fdb78040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.427+0000 7fdb827fc700 1 -- 172.21.15.19:0/716971678 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdb8801e030 con 0x7fdb8c108a60 2024-05-09T02:00:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.428+0000 7fdb90b03700 1 --2- 172.21.15.19:0/716971678 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fdb7803e510 0x7fdb78040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.428+0000 7fdb90b03700 1 --2- 172.21.15.19:0/716971678 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fdb7803e510 0x7fdb78040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c006fd0 tx=0x7fdb7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.433+0000 7fdb827fc700 1 -- 172.21.15.19:0/716971678 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdb88013070 con 0x7fdb8c108a60 2024-05-09T02:00:51.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:51.652+0000 7fdb93568700 1 -- 172.21.15.19:0/716971678 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}) -- 0x7fdb8c19ad00 con 0x7fdb7803e510 2024-05-09T02:00:53.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:52 smithi019 ceph-mon[30151]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:00:53.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:53.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:53.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:53.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:53.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi059", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:00:53.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:00:53.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:00:53.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:52 smithi019 ceph-mon[30151]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-09T02:00:53.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:52 smithi019 ceph-mon[30151]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:00:53.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:52 smithi019 ceph-mon[30151]: Updating smithi059:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:00:53.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:52 smithi019 ceph-mon[30151]: Deploying cephadm binary to smithi196 2024-05-09T02:00:54.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:53 smithi019 ceph-mon[30151]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:00:54.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:53 smithi019 ceph-mon[30151]: Updating smithi059:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T02:00:54.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:53 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:54.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:53 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:54.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:53 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:55.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:54 smithi019 ceph-mon[30151]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:00:55.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:55.945+0000 7fdb827fc700 1 -- 172.21.15.19:0/716971678 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fdb8c19ad00 con 0x7fdb7803e510 2024-05-09T02:00:55.947 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-05-09T02:00:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:55.948+0000 7fdb93568700 1 -- 172.21.15.19:0/716971678 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fdb7803e510 msgr2=0x7fdb78040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:55.948+0000 7fdb93568700 1 --2- 172.21.15.19:0/716971678 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fdb7803e510 0x7fdb78040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c006fd0 tx=0x7fdb7c006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:55.948+0000 7fdb93568700 1 -- 172.21.15.19:0/716971678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb8c108a60 msgr2=0x7fdb8c19f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:55.948+0000 7fdb93568700 1 --2- 172.21.15.19:0/716971678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb8c108a60 0x7fdb8c19f5f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb880056d0 tx=0x7fdb880057b0 comp rx=0 tx=0).stop 2024-05-09T02:00:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:55.948+0000 7fdb93568700 1 -- 172.21.15.19:0/716971678 shutdown_connections 2024-05-09T02:00:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:55.948+0000 7fdb93568700 1 --2- 172.21.15.19:0/716971678 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fdb7803e510 0x7fdb78040a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:55.948+0000 7fdb93568700 1 --2- 172.21.15.19:0/716971678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb8c108a60 0x7fdb8c19f5f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:55.948+0000 7fdb93568700 1 -- 172.21.15.19:0/716971678 >> 172.21.15.19:0/716971678 conn(0x7fdb8c0fe570 msgr2=0x7fdb8c106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:55.948+0000 7fdb93568700 1 -- 172.21.15.19:0/716971678 shutdown_connections 2024-05-09T02:00:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:55.948+0000 7fdb93568700 1 -- 172.21.15.19:0/716971678 wait complete. 2024-05-09T02:00:56.489 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T02:00:56.742 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:00:57.150 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:56 smithi019 ceph-mon[30151]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:00:57.150 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:56 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:57.150 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:56 smithi019 ceph-mon[30151]: Added host smithi196 2024-05-09T02:00:57.150 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:56 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:00:57.150 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:56 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:58.173 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:58 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:58.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.290+0000 7f8118216700 1 -- 172.21.15.19:0/1268229911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8110103df0 msgr2=0x7f8110104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:58.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.290+0000 7f8118216700 1 --2- 172.21.15.19:0/1268229911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8110103df0 0x7f8110104200 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f810c009910 tx=0x7f810c009c20 comp rx=0 tx=0).stop 2024-05-09T02:00:58.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.290+0000 7f8118216700 1 -- 172.21.15.19:0/1268229911 shutdown_connections 2024-05-09T02:00:58.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.290+0000 7f8118216700 1 --2- 172.21.15.19:0/1268229911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8110103df0 0x7f8110104200 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:58.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.291+0000 7f8118216700 1 -- 172.21.15.19:0/1268229911 >> 172.21.15.19:0/1268229911 conn(0x7f81100ff6a0 msgr2=0x7f8110101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:58.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.291+0000 7f8118216700 1 -- 172.21.15.19:0/1268229911 shutdown_connections 2024-05-09T02:00:58.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.291+0000 7f8118216700 1 -- 172.21.15.19:0/1268229911 wait complete. 2024-05-09T02:00:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.291+0000 7f8118216700 1 Processor -- start 2024-05-09T02:00:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.291+0000 7f8118216700 1 -- start start 2024-05-09T02:00:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.292+0000 7f8118216700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8110103df0 0x7f811019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.292+0000 7f8118216700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f811010e0e0 con 0x7f8110103df0 2024-05-09T02:00:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.292+0000 7f8115fb2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8110103df0 0x7f811019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.292+0000 7f8115fb2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8110103df0 0x7f811019f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38328/0 (socket says 172.21.15.19:38328) 2024-05-09T02:00:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.292+0000 7f8115fb2700 1 -- 172.21.15.19:0/396110842 learned_addr learned my addr 172.21.15.19:0/396110842 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:00:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.292+0000 7f8115fb2700 1 -- 172.21.15.19:0/396110842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811019fb30 con 0x7f8110103df0 2024-05-09T02:00:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.292+0000 7f8115fb2700 1 --2- 172.21.15.19:0/396110842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8110103df0 0x7f811019f5f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f810c006010 tx=0x7f810c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.292+0000 7f8106ffd700 1 -- 172.21.15.19:0/396110842 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f810c026070 con 0x7f8110103df0 2024-05-09T02:00:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.292+0000 7f8118216700 1 -- 172.21.15.19:0/396110842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f811019fe20 con 0x7f8110103df0 2024-05-09T02:00:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.292+0000 7f8106ffd700 1 -- 172.21.15.19:0/396110842 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f810c00bba0 con 0x7f8110103df0 2024-05-09T02:00:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.292+0000 7f8106ffd700 1 -- 172.21.15.19:0/396110842 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f810c003910 con 0x7f8110103df0 2024-05-09T02:00:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.292+0000 7f8118216700 1 -- 172.21.15.19:0/396110842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8110199950 con 0x7f8110103df0 2024-05-09T02:00:58.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.293+0000 7f8118216700 1 -- 172.21.15.19:0/396110842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81101052a0 con 0x7f8110103df0 2024-05-09T02:00:58.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.293+0000 7f8106ffd700 1 -- 172.21.15.19:0/396110842 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f810c029070 con 0x7f8110103df0 2024-05-09T02:00:58.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.294+0000 7f8106ffd700 1 --2- 172.21.15.19:0/396110842 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f80fc03e510 0x7f80fc040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:00:58.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.294+0000 7f8106ffd700 1 -- 172.21.15.19:0/396110842 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f810c01e070 con 0x7f8110103df0 2024-05-09T02:00:58.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.294+0000 7f81157b1700 1 --2- 172.21.15.19:0/396110842 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f80fc03e510 0x7f80fc040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:00:58.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.294+0000 7f81157b1700 1 --2- 172.21.15.19:0/396110842 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f80fc03e510 0x7f80fc040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8100006fd0 tx=0x7f8100006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:00:58.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.299+0000 7f8106ffd700 1 -- 172.21.15.19:0/396110842 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f810c013070 con 0x7f8110103df0 2024-05-09T02:00:58.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.519+0000 7f8118216700 1 -- 172.21.15.19:0/396110842 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f811019ac90 con 0x7f80fc03e510 2024-05-09T02:00:58.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.520+0000 7f8106ffd700 1 -- 172.21.15.19:0/396110842 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f811019ac90 con 0x7f80fc03e510 2024-05-09T02:00:58.521 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:00:58.521 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.59", "hostname": "smithi059", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-05-09T02:00:58.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.522+0000 7f8118216700 1 -- 172.21.15.19:0/396110842 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f80fc03e510 msgr2=0x7f80fc040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:58.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.522+0000 7f8118216700 1 --2- 172.21.15.19:0/396110842 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f80fc03e510 0x7f80fc040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8100006fd0 tx=0x7f8100006e40 comp rx=0 tx=0).stop 2024-05-09T02:00:58.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.522+0000 7f8118216700 1 -- 172.21.15.19:0/396110842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8110103df0 msgr2=0x7f811019f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:00:58.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.522+0000 7f8118216700 1 --2- 172.21.15.19:0/396110842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8110103df0 0x7f811019f5f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f810c006010 tx=0x7f810c005780 comp rx=0 tx=0).stop 2024-05-09T02:00:58.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.522+0000 7f8118216700 1 -- 172.21.15.19:0/396110842 shutdown_connections 2024-05-09T02:00:58.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.522+0000 7f8118216700 1 --2- 172.21.15.19:0/396110842 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f80fc03e510 0x7f80fc040a20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:58.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.522+0000 7f8118216700 1 --2- 172.21.15.19:0/396110842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8110103df0 0x7f811019f5f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:00:58.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.522+0000 7f8118216700 1 -- 172.21.15.19:0/396110842 >> 172.21.15.19:0/396110842 conn(0x7f81100ff6a0 msgr2=0x7f8110109920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:00:58.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.522+0000 7f8118216700 1 -- 172.21.15.19:0/396110842 shutdown_connections 2024-05-09T02:00:58.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:00:58.522+0000 7f8118216700 1 -- 172.21.15.19:0/396110842 wait complete. 2024-05-09T02:00:59.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:59 smithi019 ceph-mon[30151]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:00:59.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:00:59 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:00:59.147 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-09T02:00:59.147 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-09T02:00:59.392 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:01:00.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:00 smithi019 ceph-mon[30151]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:01:00.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.794+0000 7fe7abcbd700 1 -- 172.21.15.19:0/1221384095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4104d30 msgr2=0x7fe7a4105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:00.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.794+0000 7fe7abcbd700 1 --2- 172.21.15.19:0/1221384095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4104d30 0x7fe7a4105140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000b0a0 tx=0x7fe7a000b3b0 comp rx=0 tx=0).stop 2024-05-09T02:01:00.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.794+0000 7fe7abcbd700 1 -- 172.21.15.19:0/1221384095 shutdown_connections 2024-05-09T02:01:00.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.794+0000 7fe7abcbd700 1 --2- 172.21.15.19:0/1221384095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4104d30 0x7fe7a4105140 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:00.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.794+0000 7fe7abcbd700 1 -- 172.21.15.19:0/1221384095 >> 172.21.15.19:0/1221384095 conn(0x7fe7a40785b0 msgr2=0x7fe7a4078a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:00.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.794+0000 7fe7abcbd700 1 -- 172.21.15.19:0/1221384095 shutdown_connections 2024-05-09T02:01:00.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.795+0000 7fe7abcbd700 1 -- 172.21.15.19:0/1221384095 wait complete. 2024-05-09T02:01:00.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.795+0000 7fe7abcbd700 1 Processor -- start 2024-05-09T02:01:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.795+0000 7fe7abcbd700 1 -- start start 2024-05-09T02:01:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.795+0000 7fe7abcbd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4104d30 0x7fe7a4102a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.795+0000 7fe7abcbd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a410e130 con 0x7fe7a4104d30 2024-05-09T02:01:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.796+0000 7fe7a9a59700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4104d30 0x7fe7a4102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.796+0000 7fe7a9a59700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4104d30 0x7fe7a4102a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38344/0 (socket says 172.21.15.19:38344) 2024-05-09T02:01:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.796+0000 7fe7a9a59700 1 -- 172.21.15.19:0/2346624412 learned_addr learned my addr 172.21.15.19:0/2346624412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:01:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.796+0000 7fe7a9a59700 1 -- 172.21.15.19:0/2346624412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a4104870 con 0x7fe7a4104d30 2024-05-09T02:01:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.796+0000 7fe7a9a59700 1 --2- 172.21.15.19:0/2346624412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4104d30 0x7fe7a4102a20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0015040 tx=0x7fe7a0009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.796+0000 7fe79affd700 1 -- 172.21.15.19:0/2346624412 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7a0021020 con 0x7fe7a4104d30 2024-05-09T02:01:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.796+0000 7fe79affd700 1 -- 172.21.15.19:0/2346624412 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a0007970 con 0x7fe7a4104d30 2024-05-09T02:01:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.796+0000 7fe7abcbd700 1 -- 172.21.15.19:0/2346624412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a4102f60 con 0x7fe7a4104d30 2024-05-09T02:01:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.796+0000 7fe79affd700 1 -- 172.21.15.19:0/2346624412 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7a0017040 con 0x7fe7a4104d30 2024-05-09T02:01:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.796+0000 7fe7abcbd700 1 -- 172.21.15.19:0/2346624412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a41033c0 con 0x7fe7a4104d30 2024-05-09T02:01:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.797+0000 7fe79affd700 1 -- 172.21.15.19:0/2346624412 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe7a0024020 con 0x7fe7a4104d30 2024-05-09T02:01:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.797+0000 7fe7abcbd700 1 -- 172.21.15.19:0/2346624412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a406b630 con 0x7fe7a4104d30 2024-05-09T02:01:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.798+0000 7fe79affd700 1 --2- 172.21.15.19:0/2346624412 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe79003e570 0x7fe790040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.798+0000 7fe79affd700 1 -- 172.21.15.19:0/2346624412 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe7a000f050 con 0x7fe7a4104d30 2024-05-09T02:01:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.798+0000 7fe7a9258700 1 --2- 172.21.15.19:0/2346624412 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe79003e570 0x7fe790040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.798+0000 7fe7a9258700 1 --2- 172.21.15.19:0/2346624412 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe79003e570 0x7fe790040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe794006fd0 tx=0x7fe794006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:00.803+0000 7fe79affd700 1 -- 172.21.15.19:0/2346624412 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe7a002b020 con 0x7fe7a4104d30 2024-05-09T02:01:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:01.025+0000 7fe7abcbd700 1 -- 172.21.15.19:0/2346624412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fe7a4103f80 con 0x7fe7a4104d30 2024-05-09T02:01:01.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:01 smithi019 ceph-mon[30151]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:02.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:02.018+0000 7fe79affd700 1 -- 172.21.15.19:0/2346624412 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fe7a002e9e0 con 0x7fe7a4104d30 2024-05-09T02:01:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:02.022+0000 7fe7abcbd700 1 -- 172.21.15.19:0/2346624412 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe79003e570 msgr2=0x7fe790040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:02.022+0000 7fe7abcbd700 1 --2- 172.21.15.19:0/2346624412 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe79003e570 0x7fe790040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe794006fd0 tx=0x7fe794006e40 comp rx=0 tx=0).stop 2024-05-09T02:01:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:02.022+0000 7fe7abcbd700 1 -- 172.21.15.19:0/2346624412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4104d30 msgr2=0x7fe7a4102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:02.022+0000 7fe7abcbd700 1 --2- 172.21.15.19:0/2346624412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4104d30 0x7fe7a4102a20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0015040 tx=0x7fe7a0009790 comp rx=0 tx=0).stop 2024-05-09T02:01:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:02.022+0000 7fe7abcbd700 1 -- 172.21.15.19:0/2346624412 shutdown_connections 2024-05-09T02:01:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:02.022+0000 7fe7abcbd700 1 --2- 172.21.15.19:0/2346624412 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe79003e570 0x7fe790040a80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:02.022+0000 7fe7abcbd700 1 --2- 172.21.15.19:0/2346624412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7a4104d30 0x7fe7a4102a20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:02.023+0000 7fe7abcbd700 1 -- 172.21.15.19:0/2346624412 >> 172.21.15.19:0/2346624412 conn(0x7fe7a40785b0 msgr2=0x7fe7a4105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:02.023+0000 7fe7abcbd700 1 -- 172.21.15.19:0/2346624412 shutdown_connections 2024-05-09T02:01:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:02.023+0000 7fe7abcbd700 1 -- 172.21.15.19:0/2346624412 wait complete. 2024-05-09T02:01:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2024-05-09T02:01:02.221 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:02 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2346624412' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T02:01:02.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:02.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:02.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:02.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:02.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:01:02.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:02.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:02.724 INFO:tasks.cephadm:Adding mon.a on smithi019 2024-05-09T02:01:02.725 INFO:tasks.cephadm:Adding mon.b on smithi059 2024-05-09T02:01:02.725 INFO:tasks.cephadm:Adding mon.c on smithi196 2024-05-09T02:01:02.725 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c' 2024-05-09T02:01:02.949 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:01:03.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:03 smithi019 ceph-mon[30151]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-09T02:01:03.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:03 smithi019 ceph-mon[30151]: Updating smithi196:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:01:03.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:03 smithi019 ceph-mon[30151]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:03.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:03 smithi019 ceph-mon[30151]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:01:03.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:03 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2346624412' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T02:01:03.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:03 smithi019 ceph-mon[30151]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T02:01:03.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:03 smithi019 ceph-mon[30151]: Updating smithi196:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T02:01:03.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:03 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:03.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:03 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:03.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:03 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:04.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.353+0000 7f0b01bd7700 1 -- 172.21.15.196:0/3810919924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0afc104d50 msgr2=0x7f0afc105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:04.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.353+0000 7f0b01bd7700 1 --2- 172.21.15.196:0/3810919924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0afc104d50 0x7f0afc105160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009910 tx=0x7f0aec009c20 comp rx=0 tx=0).stop 2024-05-09T02:01:04.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.353+0000 7f0b01bd7700 1 -- 172.21.15.196:0/3810919924 shutdown_connections 2024-05-09T02:01:04.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.353+0000 7f0b01bd7700 1 --2- 172.21.15.196:0/3810919924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0afc104d50 0x7f0afc105160 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:04.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.353+0000 7f0b01bd7700 1 -- 172.21.15.196:0/3810919924 >> 172.21.15.196:0/3810919924 conn(0x7f0afc0785b0 msgr2=0x7f0afc078a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:04.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.353+0000 7f0b01bd7700 1 -- 172.21.15.196:0/3810919924 shutdown_connections 2024-05-09T02:01:04.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.353+0000 7f0b01bd7700 1 -- 172.21.15.196:0/3810919924 wait complete. 2024-05-09T02:01:04.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.353+0000 7f0b01bd7700 1 Processor -- start 2024-05-09T02:01:04.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.353+0000 7f0b01bd7700 1 -- start start 2024-05-09T02:01:04.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.354+0000 7f0b01bd7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0afc104d50 0x7f0afc19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:04.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.354+0000 7f0b01bd7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc10e140 con 0x7f0afc104d50 2024-05-09T02:01:04.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.354+0000 7f0afb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0afc104d50 0x7f0afc19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:04.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.354+0000 7f0afb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0afc104d50 0x7f0afc19f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.196:51414/0 (socket says 172.21.15.196:51414) 2024-05-09T02:01:04.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.354+0000 7f0afb7fe700 1 -- 172.21.15.196:0/3172774641 learned_addr learned my addr 172.21.15.196:0/3172774641 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T02:01:04.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.355+0000 7f0afb7fe700 1 -- 172.21.15.196:0/3172774641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0afc19fac0 con 0x7f0afc104d50 2024-05-09T02:01:04.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.355+0000 7f0afb7fe700 1 --2- 172.21.15.196:0/3172774641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0afc104d50 0x7f0afc19f580 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0056d0 tx=0x7f0aec0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:04.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.355+0000 7f0af97fa700 1 -- 172.21.15.196:0/3172774641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aec026070 con 0x7f0afc104d50 2024-05-09T02:01:04.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.355+0000 7f0af97fa700 1 -- 172.21.15.196:0/3172774641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aec004370 con 0x7f0afc104d50 2024-05-09T02:01:04.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.355+0000 7f0b01bd7700 1 -- 172.21.15.196:0/3172774641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0afc19fd50 con 0x7f0afc104d50 2024-05-09T02:01:04.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.355+0000 7f0af97fa700 1 -- 172.21.15.196:0/3172774641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aec0039b0 con 0x7f0afc104d50 2024-05-09T02:01:04.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.355+0000 7f0b01bd7700 1 -- 172.21.15.196:0/3172774641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0afc1997c0 con 0x7f0afc104d50 2024-05-09T02:01:04.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.356+0000 7f0b01bd7700 1 -- 172.21.15.196:0/3172774641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0afc06b6d0 con 0x7f0afc104d50 2024-05-09T02:01:04.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.356+0000 7f0af97fa700 1 -- 172.21.15.196:0/3172774641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0aec029070 con 0x7f0afc104d50 2024-05-09T02:01:04.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.356+0000 7f0af97fa700 1 --2- 172.21.15.196:0/3172774641 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0ae803e510 0x7f0ae8040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:04.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.356+0000 7f0af97fa700 1 -- 172.21.15.196:0/3172774641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0aec01e070 con 0x7f0afc104d50 2024-05-09T02:01:04.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.357+0000 7f0af2ffd700 1 --2- 172.21.15.196:0/3172774641 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0ae803e510 0x7f0ae8040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:04.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.357+0000 7f0af2ffd700 1 --2- 172.21.15.196:0/3172774641 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0ae803e510 0x7f0ae8040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4006fd0 tx=0x7f0ae4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:04.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.362+0000 7f0af97fa700 1 -- 172.21.15.196:0/3172774641 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0aec013070 con 0x7f0afc104d50 2024-05-09T02:01:04.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.577+0000 7f0b01bd7700 1 -- 172.21.15.196:0/3172774641 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}) -- 0x7f0afc19acd0 con 0x7f0ae803e510 2024-05-09T02:01:04.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.585+0000 7f0af97fa700 1 -- 172.21.15.196:0/3172774641 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0afc19acd0 con 0x7f0ae803e510 2024-05-09T02:01:04.586 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-05-09T02:01:04.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.587+0000 7f0b01bd7700 1 -- 172.21.15.196:0/3172774641 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0ae803e510 msgr2=0x7f0ae8040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:04.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.587+0000 7f0b01bd7700 1 --2- 172.21.15.196:0/3172774641 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0ae803e510 0x7f0ae8040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4006fd0 tx=0x7f0ae4006e40 comp rx=0 tx=0).stop 2024-05-09T02:01:04.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.587+0000 7f0b01bd7700 1 -- 172.21.15.196:0/3172774641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0afc104d50 msgr2=0x7f0afc19f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:04.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.587+0000 7f0b01bd7700 1 --2- 172.21.15.196:0/3172774641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0afc104d50 0x7f0afc19f580 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0056d0 tx=0x7f0aec0057b0 comp rx=0 tx=0).stop 2024-05-09T02:01:04.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.587+0000 7f0b01bd7700 1 -- 172.21.15.196:0/3172774641 shutdown_connections 2024-05-09T02:01:04.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.587+0000 7f0b01bd7700 1 --2- 172.21.15.196:0/3172774641 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0ae803e510 0x7f0ae8040a20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:04.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.587+0000 7f0b01bd7700 1 --2- 172.21.15.196:0/3172774641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0afc104d50 0x7f0afc19f580 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:04.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.587+0000 7f0b01bd7700 1 -- 172.21.15.196:0/3172774641 >> 172.21.15.196:0/3172774641 conn(0x7f0afc0785b0 msgr2=0x7f0afc105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:04.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.587+0000 7f0b01bd7700 1 -- 172.21.15.196:0/3172774641 shutdown_connections 2024-05-09T02:01:04.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:04.587+0000 7f0b01bd7700 1 -- 172.21.15.196:0/3172774641 wait complete. 2024-05-09T02:01:05.148 DEBUG:teuthology.orchestra.run.smithi059:mon.b> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.b.service 2024-05-09T02:01:05.151 DEBUG:teuthology.orchestra.run.smithi196:mon.c> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.c.service 2024-05-09T02:01:05.153 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T02:01:05.154 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T02:01:05.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:05 smithi019 ceph-mon[30151]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:05.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:05 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:05.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:05 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:05.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:05 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:05.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:05 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:05.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:05 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:05.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:05 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T02:01:05.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:05 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:05.847 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:01:06.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:06 smithi019 ceph-mon[30151]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:01:06.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:06 smithi019 ceph-mon[30151]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c;count:3 2024-05-09T02:01:06.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:06 smithi019 ceph-mon[30151]: Deploying daemon mon.c on smithi196 2024-05-09T02:01:07.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:07 smithi019 ceph-mon[30151]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:09.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.007+0000 7effff778700 1 -- 172.21.15.196:0/2287268834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff8107a50 msgr2=0x7efff8107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:09.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.007+0000 7effff778700 1 --2- 172.21.15.196:0/2287268834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff8107a50 0x7efff8107e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efff4009910 tx=0x7efff4009c20 comp rx=0 tx=0).stop 2024-05-09T02:01:09.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.007+0000 7effff778700 1 -- 172.21.15.196:0/2287268834 shutdown_connections 2024-05-09T02:01:09.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.007+0000 7effff778700 1 --2- 172.21.15.196:0/2287268834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff8107a50 0x7efff8107e60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:09.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.007+0000 7effff778700 1 -- 172.21.15.196:0/2287268834 >> 172.21.15.196:0/2287268834 conn(0x7efff80fde30 msgr2=0x7efff8100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:09.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.007+0000 7effff778700 1 -- 172.21.15.196:0/2287268834 shutdown_connections 2024-05-09T02:01:09.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.007+0000 7effff778700 1 -- 172.21.15.196:0/2287268834 wait complete. 2024-05-09T02:01:09.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.008+0000 7effff778700 1 Processor -- start 2024-05-09T02:01:09.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.008+0000 7effff778700 1 -- start start 2024-05-09T02:01:09.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.008+0000 7effff778700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff8107a50 0x7efff8197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:09.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.008+0000 7effff778700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efff810bfe0 con 0x7efff8107a50 2024-05-09T02:01:09.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.008+0000 7efffd514700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff8107a50 0x7efff8197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:09.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.008+0000 7efffd514700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff8107a50 0x7efff8197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.196:33076/0 (socket says 172.21.15.196:33076) 2024-05-09T02:01:09.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.008+0000 7efffd514700 1 -- 172.21.15.196:0/2087334171 learned_addr learned my addr 172.21.15.196:0/2087334171 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T02:01:09.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.009+0000 7efffd514700 1 -- 172.21.15.196:0/2087334171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff8197cb0 con 0x7efff8107a50 2024-05-09T02:01:09.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.009+0000 7efffd514700 1 --2- 172.21.15.196:0/2087334171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff8107a50 0x7efff8197770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efff40056d0 tx=0x7efff40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:09.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.009+0000 7effee7fc700 1 -- 172.21.15.196:0/2087334171 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efff4026070 con 0x7efff8107a50 2024-05-09T02:01:09.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.009+0000 7effff778700 1 -- 172.21.15.196:0/2087334171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff8197f40 con 0x7efff8107a50 2024-05-09T02:01:09.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.009+0000 7effff778700 1 -- 172.21.15.196:0/2087334171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff81983a0 con 0x7efff8107a50 2024-05-09T02:01:09.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.009+0000 7effee7fc700 1 -- 172.21.15.196:0/2087334171 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff400bc00 con 0x7efff8107a50 2024-05-09T02:01:09.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.009+0000 7effee7fc700 1 -- 172.21.15.196:0/2087334171 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efff4003920 con 0x7efff8107a50 2024-05-09T02:01:09.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.010+0000 7effff778700 1 -- 172.21.15.196:0/2087334171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff81041e0 con 0x7efff8107a50 2024-05-09T02:01:09.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.010+0000 7effee7fc700 1 -- 172.21.15.196:0/2087334171 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7efff4029070 con 0x7efff8107a50 2024-05-09T02:01:09.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.010+0000 7effee7fc700 1 --2- 172.21.15.196:0/2087334171 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7effe4040770 0x7effe4042c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:09.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.011+0000 7effee7fc700 1 -- 172.21.15.196:0/2087334171 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efff401e070 con 0x7efff8107a50 2024-05-09T02:01:09.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.011+0000 7efffcd13700 1 --2- 172.21.15.196:0/2087334171 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7effe4040770 0x7effe4042c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:09.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.011+0000 7efffcd13700 1 --2- 172.21.15.196:0/2087334171 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7effe4040770 0x7effe4042c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7effe8006fd0 tx=0x7effe8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:09.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.016+0000 7effee7fc700 1 -- 172.21.15.196:0/2087334171 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efff4013070 con 0x7efff8107a50 2024-05-09T02:01:09.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:09 smithi019 ceph-mon[30151]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:09.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.325+0000 7effff778700 1 -- 172.21.15.196:0/2087334171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7efff8198ad0 con 0x7efff8107a50 2024-05-09T02:01:09.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.326+0000 7effee7fc700 1 -- 172.21.15.196:0/2087334171 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7efff400bdb0 con 0x7efff8107a50 2024-05-09T02:01:09.327 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T02:01:09.328 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"a0611f3e-0da7-11ef-bc97-c7b262605968","modified":"2024-05-09T01:59:13.406931Z","created":"2024-05-09T01:59:13.406931Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T02:01:09.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.329+0000 7effff778700 1 -- 172.21.15.196:0/2087334171 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7effe4040770 msgr2=0x7effe4042c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:09.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.329+0000 7effff778700 1 --2- 172.21.15.196:0/2087334171 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7effe4040770 0x7effe4042c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7effe8006fd0 tx=0x7effe8006e40 comp rx=0 tx=0).stop 2024-05-09T02:01:09.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.329+0000 7effff778700 1 -- 172.21.15.196:0/2087334171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff8107a50 msgr2=0x7efff8197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:09.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.329+0000 7effff778700 1 --2- 172.21.15.196:0/2087334171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff8107a50 0x7efff8197770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efff40056d0 tx=0x7efff40057b0 comp rx=0 tx=0).stop 2024-05-09T02:01:09.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.329+0000 7effff778700 1 -- 172.21.15.196:0/2087334171 shutdown_connections 2024-05-09T02:01:09.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.329+0000 7effff778700 1 --2- 172.21.15.196:0/2087334171 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7effe4040770 0x7effe4042c80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:09.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.329+0000 7effff778700 1 --2- 172.21.15.196:0/2087334171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff8107a50 0x7efff8197770 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:09.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.329+0000 7effff778700 1 -- 172.21.15.196:0/2087334171 >> 172.21.15.196:0/2087334171 conn(0x7efff80fde30 msgr2=0x7efff8104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:09.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.329+0000 7effff778700 1 -- 172.21.15.196:0/2087334171 shutdown_connections 2024-05-09T02:01:09.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:09.329+0000 7effff778700 1 -- 172.21.15.196:0/2087334171 wait complete. 2024-05-09T02:01:09.331 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-05-09T02:01:09.988 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:09 smithi196 systemd[1]: Started Ceph mon.c for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:01:11.133 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T02:01:11.133 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T02:01:11.371 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:01:15.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:01:15.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: mon.a calling monitor election 2024-05-09T02:01:15.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:15.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:15.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:15.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:15.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: mon.c calling monitor election 2024-05-09T02:01:15.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:15.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:15.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:15.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:15.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:15.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T02:01:15.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: monmap epoch 2 2024-05-09T02:01:15.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: fsid a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:01:15.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: last_changed 2024-05-09T02:01:09.669702+0000 2024-05-09T02:01:15.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: created 2024-05-09T01:59:13.406931+0000 2024-05-09T02:01:15.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: min_mon_release 19 (squid) 2024-05-09T02:01:15.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: election_strategy: 1 2024-05-09T02:01:15.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T02:01:15.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-05-09T02:01:15.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: fsmap 2024-05-09T02:01:15.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T02:01:15.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: mgrmap e13: x(active, since 50s) 2024-05-09T02:01:15.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: overall HEALTH_OK 2024-05-09T02:01:15.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:15.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:15.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:15.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T02:01:15.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:15.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:01:15.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: mon.a calling monitor election 2024-05-09T02:01:15.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: mon.c calling monitor election 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: monmap epoch 2 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: fsid a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: last_changed 2024-05-09T02:01:09.669702+0000 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: created 2024-05-09T01:59:13.406931+0000 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: min_mon_release 19 (squid) 2024-05-09T02:01:15.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: election_strategy: 1 2024-05-09T02:01:15.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T02:01:15.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-05-09T02:01:15.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: fsmap 2024-05-09T02:01:15.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T02:01:15.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: mgrmap e13: x(active, since 50s) 2024-05-09T02:01:15.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: overall HEALTH_OK 2024-05-09T02:01:15.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:15.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:15.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:15.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T02:01:15.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:15.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.646+0000 7fd14cf69700 1 -- 172.21.15.196:0/2702702350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1340035c0 msgr2=0x7fd134005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:15.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.646+0000 7fd14cf69700 1 --2- 172.21.15.196:0/2702702350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1340035c0 0x7fd134005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd148103b80 tx=0x7fd140009e80 comp rx=0 tx=0).stop 2024-05-09T02:01:15.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.646+0000 7fd14cf69700 1 -- 172.21.15.196:0/2702702350 shutdown_connections 2024-05-09T02:01:15.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.646+0000 7fd14cf69700 1 --2- 172.21.15.196:0/2702702350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1340035c0 0x7fd134005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:15.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.646+0000 7fd14cf69700 1 -- 172.21.15.196:0/2702702350 >> 172.21.15.196:0/2702702350 conn(0x7fd1480ff790 msgr2=0x7fd148101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:15.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.646+0000 7fd14cf69700 1 -- 172.21.15.196:0/2702702350 shutdown_connections 2024-05-09T02:01:15.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.646+0000 7fd14cf69700 1 -- 172.21.15.196:0/2702702350 wait complete. 2024-05-09T02:01:15.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.647+0000 7fd14cf69700 1 Processor -- start 2024-05-09T02:01:15.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.647+0000 7fd14cf69700 1 -- start start 2024-05-09T02:01:15.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.648+0000 7fd14cf69700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1340035c0 0x7fd14819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:15.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.648+0000 7fd14cf69700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd14819fc70 0x7fd148199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:15.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.648+0000 7fd14cf69700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd14810bc00 con 0x7fd14819fc70 2024-05-09T02:01:15.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.648+0000 7fd14cf69700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd14810ba80 con 0x7fd1340035c0 2024-05-09T02:01:15.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.648+0000 7fd14659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1340035c0 0x7fd14819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:15.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.648+0000 7fd14659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1340035c0 0x7fd14819f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:42582/0 (socket says 172.21.15.196:42582) 2024-05-09T02:01:15.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.648+0000 7fd14659c700 1 -- 172.21.15.196:0/3511014721 learned_addr learned my addr 172.21.15.196:0/3511014721 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T02:01:15.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.648+0000 7fd14659c700 1 -- 172.21.15.196:0/3511014721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1340035c0 msgr2=0x7fd14819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-09T02:01:15.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.648+0000 7fd14659c700 1 -- 172.21.15.196:0/3511014721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1340035c0 msgr2=0x7fd14819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T02:01:15.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.648+0000 7fd14659c700 1 --2- 172.21.15.196:0/3511014721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1340035c0 0x7fd14819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T02:01:15.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.648+0000 7fd14659c700 1 --2- 172.21.15.196:0/3511014721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1340035c0 0x7fd14819f730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T02:01:15.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.648+0000 7fd145d9b700 1 --2- 172.21.15.196:0/3511014721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd14819fc70 0x7fd148199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:15.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.648+0000 7fd145d9b700 1 -- 172.21.15.196:0/3511014721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1340035c0 msgr2=0x7fd14819f730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:01:15.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.648+0000 7fd145d9b700 1 --2- 172.21.15.196:0/3511014721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1340035c0 0x7fd14819f730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:15.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.648+0000 7fd145d9b700 1 -- 172.21.15.196:0/3511014721 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1481a0240 con 0x7fd14819fc70 2024-05-09T02:01:15.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.649+0000 7fd145d9b700 1 --2- 172.21.15.196:0/3511014721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd14819fc70 0x7fd148199950 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd13000b4d0 tx=0x7fd13000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:15.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.649+0000 7fd13f7fe700 1 -- 172.21.15.196:0/3511014721 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd130011630 con 0x7fd14819fc70 2024-05-09T02:01:15.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.649+0000 7fd13f7fe700 1 -- 172.21.15.196:0/3511014721 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd130011c80 con 0x7fd14819fc70 2024-05-09T02:01:15.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.649+0000 7fd14cf69700 1 -- 172.21.15.196:0/3511014721 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd148199ef0 con 0x7fd14819fc70 2024-05-09T02:01:15.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.649+0000 7fd13f7fe700 1 -- 172.21.15.196:0/3511014721 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd130010da0 con 0x7fd14819fc70 2024-05-09T02:01:15.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.649+0000 7fd14cf69700 1 -- 172.21.15.196:0/3511014721 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd14819a480 con 0x7fd14819fc70 2024-05-09T02:01:15.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.650+0000 7fd13f7fe700 1 -- 172.21.15.196:0/3511014721 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd130018020 con 0x7fd14819fc70 2024-05-09T02:01:15.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.650+0000 7fd14cf69700 1 -- 172.21.15.196:0/3511014721 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd14819a790 con 0x7fd14819fc70 2024-05-09T02:01:15.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.650+0000 7fd13f7fe700 1 --2- 172.21.15.196:0/3511014721 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd12803e2d0 0x7fd1280407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:15.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.650+0000 7fd13f7fe700 1 -- 172.21.15.196:0/3511014721 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd130052180 con 0x7fd14819fc70 2024-05-09T02:01:15.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.651+0000 7fd14659c700 1 --2- 172.21.15.196:0/3511014721 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd12803e2d0 0x7fd1280407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:15.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.651+0000 7fd14659c700 1 --2- 172.21.15.196:0/3511014721 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd12803e2d0 0x7fd1280407e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd14001c040 tx=0x7fd1400091d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:15.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.656+0000 7fd13f7fe700 1 -- 172.21.15.196:0/3511014721 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd13001f020 con 0x7fd14819fc70 2024-05-09T02:01:15.903 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:15 smithi196 ceph-mon[34718]: Deploying daemon mon.b on smithi059 2024-05-09T02:01:15.903 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:15.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.950+0000 7fd14cf69700 1 -- 172.21.15.196:0/3511014721 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd14802e5f0 con 0x7fd14819fc70 2024-05-09T02:01:15.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.951+0000 7fd13f7fe700 1 -- 172.21.15.196:0/3511014721 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fd1300187b0 con 0x7fd14819fc70 2024-05-09T02:01:15.952 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T02:01:15.952 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"a0611f3e-0da7-11ef-bc97-c7b262605968","modified":"2024-05-09T02:01:09.669702Z","created":"2024-05-09T01:59:13.406931Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T02:01:15.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.953+0000 7fd14cf69700 1 -- 172.21.15.196:0/3511014721 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd12803e2d0 msgr2=0x7fd1280407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:15.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.953+0000 7fd14cf69700 1 --2- 172.21.15.196:0/3511014721 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd12803e2d0 0x7fd1280407e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd14001c040 tx=0x7fd1400091d0 comp rx=0 tx=0).stop 2024-05-09T02:01:15.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.953+0000 7fd14cf69700 1 -- 172.21.15.196:0/3511014721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd14819fc70 msgr2=0x7fd148199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:15.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.953+0000 7fd14cf69700 1 --2- 172.21.15.196:0/3511014721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd14819fc70 0x7fd148199950 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd13000b4d0 tx=0x7fd13000b890 comp rx=0 tx=0).stop 2024-05-09T02:01:15.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.953+0000 7fd14cf69700 1 -- 172.21.15.196:0/3511014721 shutdown_connections 2024-05-09T02:01:15.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.953+0000 7fd14cf69700 1 --2- 172.21.15.196:0/3511014721 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd12803e2d0 0x7fd1280407e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:15.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.953+0000 7fd14cf69700 1 --2- 172.21.15.196:0/3511014721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1340035c0 0x7fd14819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:15.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.953+0000 7fd14cf69700 1 --2- 172.21.15.196:0/3511014721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd14819fc70 0x7fd148199950 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:15.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.953+0000 7fd14cf69700 1 -- 172.21.15.196:0/3511014721 >> 172.21.15.196:0/3511014721 conn(0x7fd1480ff790 msgr2=0x7fd148107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:15.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.953+0000 7fd14cf69700 1 -- 172.21.15.196:0/3511014721 shutdown_connections 2024-05-09T02:01:15.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:15.953+0000 7fd14cf69700 1 -- 172.21.15.196:0/3511014721 wait complete. 2024-05-09T02:01:15.955 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-05-09T02:01:16.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:15 smithi019 ceph-mon[30151]: Deploying daemon mon.b on smithi059 2024-05-09T02:01:16.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:16.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:16 smithi196 ceph-mon[34718]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:16.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:16 smithi196 ceph-mon[34718]: from='client.? 172.21.15.196:0/3511014721' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T02:01:17.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:16 smithi019 ceph-mon[30151]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:17.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:16 smithi019 ceph-mon[30151]: from='client.? 172.21.15.196:0/3511014721' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T02:01:17.540 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T02:01:17.540 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T02:01:17.782 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:01:18.133 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 systemd[1]: Starting Ceph mon.b for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:01:18.414 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 podman[34762]: 2024-05-09 02:01:18.333514504 +0000 UTC m=+0.118160120 container create e28aece55a381b4e8ebc446b21c52484a127984ae57b1b5baf548d803f017e2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-05-09T02:01:18.414 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 podman[34762]: 2024-05-09 02:01:18.240153281 +0000 UTC m=+0.024798913 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:01:18.702 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 podman[34762]: 2024-05-09 02:01:18.510622232 +0000 UTC m=+0.295267849 container init e28aece55a381b4e8ebc446b21c52484a127984ae57b1b5baf548d803f017e2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T02:01:18.702 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 podman[34762]: 2024-05-09 02:01:18.513745837 +0000 UTC m=+0.298391453 container start e28aece55a381b4e8ebc446b21c52484a127984ae57b1b5baf548d803f017e2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-b, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T02:01:18.702 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T02:01:18.702 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev), process ceph-mon, pid 2 2024-05-09T02:01:18.703 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pidfile_write: ignore empty --pid-file 2024-05-09T02:01:18.703 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: load: jerasure load: lrc 2024-05-09T02:01:18.703 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: RocksDB version: 7.9.2 2024-05-09T02:01:18.703 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Git sha 0 2024-05-09T02:01:18.703 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Compile date 2024-05-08 19:33:14 2024-05-09T02:01:18.703 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: DB SUMMARY 2024-05-09T02:01:18.703 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: DB Session ID: HM7538BE8KZKBB2ORJOO 2024-05-09T02:01:18.703 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: CURRENT file: CURRENT 2024-05-09T02:01:18.703 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T02:01:18.703 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-09T02:01:18.703 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-09T02:01:18.703 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-09T02:01:18.704 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.error_if_exists: 0 2024-05-09T02:01:18.704 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.create_if_missing: 0 2024-05-09T02:01:18.704 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.paranoid_checks: 1 2024-05-09T02:01:18.704 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T02:01:18.704 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T02:01:18.704 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T02:01:18.704 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.env: 0x55b18190e500 2024-05-09T02:01:18.704 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T02:01:18.704 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.info_log: 0x55b183f4e020 2024-05-09T02:01:18.704 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T02:01:18.704 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.statistics: (nil) 2024-05-09T02:01:18.704 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.use_fsync: 0 2024-05-09T02:01:18.704 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_log_file_size: 0 2024-05-09T02:01:18.704 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T02:01:18.705 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T02:01:18.705 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T02:01:18.705 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T02:01:18.705 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.allow_fallocate: 1 2024-05-09T02:01:18.705 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T02:01:18.705 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T02:01:18.705 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.use_direct_reads: 0 2024-05-09T02:01:18.705 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T02:01:18.705 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T02:01:18.705 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.db_log_dir: 2024-05-09T02:01:18.705 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.wal_dir: 2024-05-09T02:01:18.705 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T02:01:18.705 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T02:01:18.705 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T02:01:18.706 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T02:01:18.706 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T02:01:18.706 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T02:01:18.706 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T02:01:18.706 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T02:01:18.706 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.write_buffer_manager: 0x55b183e188c0 2024-05-09T02:01:18.706 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T02:01:18.706 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T02:01:18.706 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T02:01:18.706 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T02:01:18.706 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T02:01:18.706 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T02:01:18.706 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T02:01:18.707 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T02:01:18.707 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.unordered_write: 0 2024-05-09T02:01:18.707 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T02:01:18.707 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T02:01:18.707 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T02:01:18.707 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T02:01:18.707 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.row_cache: None 2024-05-09T02:01:18.707 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.wal_filter: None 2024-05-09T02:01:18.709 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T02:01:18.709 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T02:01:18.709 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.two_write_queues: 0 2024-05-09T02:01:18.709 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T02:01:18.709 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.wal_compression: 0 2024-05-09T02:01:18.709 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.atomic_flush: 0 2024-05-09T02:01:18.709 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T02:01:18.709 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T02:01:18.709 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T02:01:18.709 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.log_readahead_size: 0 2024-05-09T02:01:18.709 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T02:01:18.710 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T02:01:18.710 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T02:01:18.710 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T02:01:18.710 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T02:01:18.710 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T02:01:18.710 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T02:01:18.710 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_background_jobs: 2 2024-05-09T02:01:18.710 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_background_compactions: -1 2024-05-09T02:01:18.710 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_subcompactions: 1 2024-05-09T02:01:18.710 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T02:01:18.710 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T02:01:18.710 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T02:01:18.710 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T02:01:18.710 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T02:01:18.711 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T02:01:18.711 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T02:01:18.711 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T02:01:18.711 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_open_files: -1 2024-05-09T02:01:18.711 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T02:01:18.711 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T02:01:18.711 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T02:01:18.711 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T02:01:18.711 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_background_flushes: -1 2024-05-09T02:01:18.711 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Compression algorithms supported: 2024-05-09T02:01:18.711 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T02:01:18.711 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: kZSTD supported: 0 2024-05-09T02:01:18.711 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: kXpressCompression supported: 0 2024-05-09T02:01:18.711 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T02:01:18.712 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: kZlibCompression supported: 1 2024-05-09T02:01:18.712 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: kSnappyCompression supported: 1 2024-05-09T02:01:18.712 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: kLZ4Compression supported: 1 2024-05-09T02:01:18.712 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: kBZip2Compression supported: 0 2024-05-09T02:01:18.712 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T02:01:18.712 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T02:01:18.712 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-09T02:01:18.712 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T02:01:18.712 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T02:01:18.712 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.merge_operator: 2024-05-09T02:01:18.712 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compaction_filter: None 2024-05-09T02:01:18.712 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T02:01:18.712 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T02:01:18.712 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T02:01:18.713 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T02:01:18.713 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b183f4e1e0) 2024-05-09T02:01:18.713 INFO:journalctl@ceph.mon.b.smithi059.stdout: cache_index_and_filter_blocks: 1 2024-05-09T02:01:18.713 INFO:journalctl@ceph.mon.b.smithi059.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T02:01:18.713 INFO:journalctl@ceph.mon.b.smithi059.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T02:01:18.713 INFO:journalctl@ceph.mon.b.smithi059.stdout: pin_top_level_index_and_filter: 1 2024-05-09T02:01:18.713 INFO:journalctl@ceph.mon.b.smithi059.stdout: index_type: 0 2024-05-09T02:01:18.713 INFO:journalctl@ceph.mon.b.smithi059.stdout: data_block_index_type: 0 2024-05-09T02:01:18.713 INFO:journalctl@ceph.mon.b.smithi059.stdout: index_shortening: 1 2024-05-09T02:01:18.713 INFO:journalctl@ceph.mon.b.smithi059.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T02:01:18.713 INFO:journalctl@ceph.mon.b.smithi059.stdout: checksum: 4 2024-05-09T02:01:18.713 INFO:journalctl@ceph.mon.b.smithi059.stdout: no_block_cache: 0 2024-05-09T02:01:18.713 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_cache: 0x55b183e4b1f0 2024-05-09T02:01:18.713 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_cache_name: BinnedLRUCache 2024-05-09T02:01:18.714 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_cache_options: 2024-05-09T02:01:18.714 INFO:journalctl@ceph.mon.b.smithi059.stdout: capacity : 536870912 2024-05-09T02:01:18.714 INFO:journalctl@ceph.mon.b.smithi059.stdout: num_shard_bits : 4 2024-05-09T02:01:18.714 INFO:journalctl@ceph.mon.b.smithi059.stdout: strict_capacity_limit : 0 2024-05-09T02:01:18.714 INFO:journalctl@ceph.mon.b.smithi059.stdout: high_pri_pool_ratio: 0.000 2024-05-09T02:01:18.714 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_cache_compressed: (nil) 2024-05-09T02:01:18.714 INFO:journalctl@ceph.mon.b.smithi059.stdout: persistent_cache: (nil) 2024-05-09T02:01:18.714 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_size: 4096 2024-05-09T02:01:18.714 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_size_deviation: 10 2024-05-09T02:01:18.714 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_restart_interval: 16 2024-05-09T02:01:18.714 INFO:journalctl@ceph.mon.b.smithi059.stdout: index_block_restart_interval: 1 2024-05-09T02:01:18.714 INFO:journalctl@ceph.mon.b.smithi059.stdout: metadata_block_size: 4096 2024-05-09T02:01:18.714 INFO:journalctl@ceph.mon.b.smithi059.stdout: partition_filters: 0 2024-05-09T02:01:18.714 INFO:journalctl@ceph.mon.b.smithi059.stdout: use_delta_encoding: 1 2024-05-09T02:01:18.714 INFO:journalctl@ceph.mon.b.smithi059.stdout: filter_policy: bloomfilter 2024-05-09T02:01:18.715 INFO:journalctl@ceph.mon.b.smithi059.stdout: whole_key_filtering: 1 2024-05-09T02:01:18.715 INFO:journalctl@ceph.mon.b.smithi059.stdout: verify_compression: 0 2024-05-09T02:01:18.715 INFO:journalctl@ceph.mon.b.smithi059.stdout: read_amp_bytes_per_bit: 0 2024-05-09T02:01:18.715 INFO:journalctl@ceph.mon.b.smithi059.stdout: format_version: 5 2024-05-09T02:01:18.715 INFO:journalctl@ceph.mon.b.smithi059.stdout: enable_index_compression: 1 2024-05-09T02:01:18.715 INFO:journalctl@ceph.mon.b.smithi059.stdout: block_align: 0 2024-05-09T02:01:18.715 INFO:journalctl@ceph.mon.b.smithi059.stdout: max_auto_readahead_size: 262144 2024-05-09T02:01:18.715 INFO:journalctl@ceph.mon.b.smithi059.stdout: prepopulate_block_cache: 0 2024-05-09T02:01:18.715 INFO:journalctl@ceph.mon.b.smithi059.stdout: initial_auto_readahead_size: 8192 2024-05-09T02:01:18.715 INFO:journalctl@ceph.mon.b.smithi059.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T02:01:18.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T02:01:18.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T02:01:18.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compression: NoCompression 2024-05-09T02:01:18.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T02:01:18.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T02:01:18.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T02:01:18.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.num_levels: 7 2024-05-09T02:01:18.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T02:01:18.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T02:01:18.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T02:01:18.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T02:01:18.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T02:01:18.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T02:01:18.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T02:01:18.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T02:01:18.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T02:01:18.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T02:01:18.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T02:01:18.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T02:01:18.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T02:01:18.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T02:01:18.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T02:01:18.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T02:01:18.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T02:01:18.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T02:01:18.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T02:01:18.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T02:01:18.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T02:01:18.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T02:01:18.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T02:01:18.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T02:01:18.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T02:01:18.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T02:01:18.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T02:01:18.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T02:01:18.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T02:01:18.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T02:01:18.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T02:01:18.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T02:01:18.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T02:01:18.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T02:01:18.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T02:01:18.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T02:01:18.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T02:01:18.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T02:01:18.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T02:01:18.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T02:01:18.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T02:01:18.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T02:01:18.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T02:01:18.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T02:01:18.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T02:01:18.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T02:01:18.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T02:01:18.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T02:01:18.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T02:01:18.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T02:01:18.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T02:01:18.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T02:01:18.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T02:01:18.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-09T02:01:18.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.inplace_update_support: 0 2024-05-09T02:01:18.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T02:01:18.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T02:01:18.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T02:01:18.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T02:01:18.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.bloom_locality: 0 2024-05-09T02:01:18.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.max_successive_merges: 0 2024-05-09T02:01:18.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T02:01:18.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T02:01:18.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T02:01:18.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T02:01:18.722 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.ttl: 2592000 2024-05-09T02:01:18.722 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T02:01:18.722 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T02:01:18.722 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T02:01:18.722 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.enable_blob_files: false 2024-05-09T02:01:18.722 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.min_blob_size: 0 2024-05-09T02:01:18.722 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T02:01:18.722 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T02:01:18.722 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T02:01:18.722 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T02:01:18.722 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T02:01:18.722 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T02:01:18.722 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T02:01:18.722 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T02:01:18.723 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-09T02:01:18.723 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-09T02:01:18.723 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 02619f1b-a23c-4612-936a-54a097183c99 2024-05-09T02:01:18.723 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715220078555078, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-09T02:01:18.723 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-09T02:01:18.723 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715220078555537, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715220078, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "02619f1b-a23c-4612-936a-54a097183c99", "db_session_id": "HM7538BE8KZKBB2ORJOO", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-09T02:01:18.723 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715220078555630, "job": 1, "event": "recovery_finished"} 2024-05-09T02:01:18.723 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-09T02:01:18.723 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T02:01:18.723 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b183e2fc00 2024-05-09T02:01:18.723 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: DB pointer 0x55b188c22000 2024-05-09T02:01:18.723 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T02:01:18.723 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T02:01:18.723 INFO:journalctl@ceph.mon.b.smithi059.stdout: ** DB Stats ** 2024-05-09T02:01:18.723 INFO:journalctl@ceph.mon.b.smithi059.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T02:01:18.724 INFO:journalctl@ceph.mon.b.smithi059.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T02:01:18.724 INFO:journalctl@ceph.mon.b.smithi059.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T02:01:18.724 INFO:journalctl@ceph.mon.b.smithi059.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T02:01:18.724 INFO:journalctl@ceph.mon.b.smithi059.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T02:01:18.724 INFO:journalctl@ceph.mon.b.smithi059.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T02:01:18.724 INFO:journalctl@ceph.mon.b.smithi059.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T02:01:18.724 INFO:journalctl@ceph.mon.b.smithi059.stdout: 2024-05-09T02:01:18.724 INFO:journalctl@ceph.mon.b.smithi059.stdout: ** Compaction Stats [default] ** 2024-05-09T02:01:18.724 INFO:journalctl@ceph.mon.b.smithi059.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T02:01:18.724 INFO:journalctl@ceph.mon.b.smithi059.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T02:01:18.724 INFO:journalctl@ceph.mon.b.smithi059.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T02:01:18.724 INFO:journalctl@ceph.mon.b.smithi059.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T02:01:18.724 INFO:journalctl@ceph.mon.b.smithi059.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T02:01:18.724 INFO:journalctl@ceph.mon.b.smithi059.stdout: 2024-05-09T02:01:18.725 INFO:journalctl@ceph.mon.b.smithi059.stdout: ** Compaction Stats [default] ** 2024-05-09T02:01:18.725 INFO:journalctl@ceph.mon.b.smithi059.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T02:01:18.725 INFO:journalctl@ceph.mon.b.smithi059.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T02:01:18.725 INFO:journalctl@ceph.mon.b.smithi059.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T02:01:18.725 INFO:journalctl@ceph.mon.b.smithi059.stdout: 2024-05-09T02:01:18.725 INFO:journalctl@ceph.mon.b.smithi059.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T02:01:18.725 INFO:journalctl@ceph.mon.b.smithi059.stdout: 2024-05-09T02:01:18.725 INFO:journalctl@ceph.mon.b.smithi059.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T02:01:18.725 INFO:journalctl@ceph.mon.b.smithi059.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T02:01:18.725 INFO:journalctl@ceph.mon.b.smithi059.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T02:01:18.725 INFO:journalctl@ceph.mon.b.smithi059.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T02:01:18.725 INFO:journalctl@ceph.mon.b.smithi059.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T02:01:18.725 INFO:journalctl@ceph.mon.b.smithi059.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T02:01:18.725 INFO:journalctl@ceph.mon.b.smithi059.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T02:01:18.725 INFO:journalctl@ceph.mon.b.smithi059.stdout: Interval compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T02:01:18.726 INFO:journalctl@ceph.mon.b.smithi059.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T02:01:18.726 INFO:journalctl@ceph.mon.b.smithi059.stdout: Block cache BinnedLRUCache@0x55b183e4b1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.4e-05 secs_since: 0 2024-05-09T02:01:18.726 INFO:journalctl@ceph.mon.b.smithi059.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-09T02:01:18.726 INFO:journalctl@ceph.mon.b.smithi059.stdout: 2024-05-09T02:01:18.726 INFO:journalctl@ceph.mon.b.smithi059.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T02:01:18.726 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-09T02:01:18.726 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: using public_addr v2:172.21.15.59:0/0 -> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] 2024-05-09T02:01:18.726 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: starting mon.b rank -1 at public addrs [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] at bind addrs [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:01:18.726 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.b@-1(???) e0 preinit fsid a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:01:18.726 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.b@-1(synchronizing).mds e1 new map 2024-05-09T02:01:18.726 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-09T02:01:18.726 INFO:journalctl@ceph.mon.b.smithi059.stdout: e1 2024-05-09T02:01:18.726 INFO:journalctl@ceph.mon.b.smithi059.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T02:01:18.727 INFO:journalctl@ceph.mon.b.smithi059.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T02:01:18.727 INFO:journalctl@ceph.mon.b.smithi059.stdout: legacy client fscid: -1 2024-05-09T02:01:18.727 INFO:journalctl@ceph.mon.b.smithi059.stdout: 2024-05-09T02:01:18.727 INFO:journalctl@ceph.mon.b.smithi059.stdout: No filesystems configured 2024-05-09T02:01:18.727 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-09T02:01:18.727 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-09T02:01:18.727 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-09T02:01:18.727 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-09T02:01:18.727 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-09T02:01:18.727 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-09T02:01:18.727 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T02:01:18.727 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T02:01:18.727 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T02:01:18.727 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T02:01:18.727 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi059", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: Updating smithi059:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:01:18.728 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: Deploying cephadm binary to smithi196 2024-05-09T02:01:18.729 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:01:18.729 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: Updating smithi059:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T02:01:18.729 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.729 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.729 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.729 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:18.729 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:18.729 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.729 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: Added host smithi196 2024-05-09T02:01:18.729 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:18.729 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.729 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.729 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:18.729 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.730 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:01:18.730 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:18.730 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2346624412' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T02:01:18.730 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.730 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.730 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.730 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.730 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:01:18.730 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:18.730 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:18.730 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-09T02:01:18.730 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: Updating smithi196:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:01:18.730 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:18.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:01:18.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2346624412' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T02:01:18.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T02:01:18.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: Updating smithi196:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T02:01:18.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:18.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:18.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:18.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:18.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T02:01:18.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:18.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:01:18.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c;count:3 2024-05-09T02:01:18.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: Deploying daemon mon.c on smithi196 2024-05-09T02:01:18.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:18.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:18.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:01:18.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.a calling monitor election 2024-05-09T02:01:18.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:18.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:18.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:18.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:18.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.c calling monitor election 2024-05-09T02:01:18.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:18.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:18.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:18.733 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:18.733 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:18.733 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T02:01:18.733 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: monmap epoch 2 2024-05-09T02:01:18.733 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: fsid a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:01:18.733 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: last_changed 2024-05-09T02:01:09.669702+0000 2024-05-09T02:01:18.733 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: created 2024-05-09T01:59:13.406931+0000 2024-05-09T02:01:18.733 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: min_mon_release 19 (squid) 2024-05-09T02:01:18.733 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: election_strategy: 1 2024-05-09T02:01:18.733 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T02:01:18.733 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-05-09T02:01:18.733 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: fsmap 2024-05-09T02:01:18.733 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T02:01:18.733 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mgrmap e13: x(active, since 50s) 2024-05-09T02:01:18.733 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: overall HEALTH_OK 2024-05-09T02:01:18.734 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.734 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.734 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:18.734 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T02:01:18.734 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:18.734 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: Deploying daemon mon.b on smithi059 2024-05-09T02:01:18.734 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:18.734 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:18.734 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: from='client.? 172.21.15.196:0/3511014721' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T02:01:18.734 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 ceph-mon[34781]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T02:01:18.734 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 bash[34762]: e28aece55a381b4e8ebc446b21c52484a127984ae57b1b5baf548d803f017e2c 2024-05-09T02:01:18.734 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:18 smithi059 systemd[1]: Started Ceph mon.b for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:01:23.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.835+0000 7f7b9602f700 1 -- 172.21.15.196:0/4000313999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b780035c0 msgr2=0x7f7b78005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:23.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.835+0000 7f7b9602f700 1 --2- 172.21.15.196:0/4000313999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b780035c0 0x7f7b78005aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7b90102a70 tx=0x7f7b8000fe60 comp rx=0 tx=0).stop 2024-05-09T02:01:23.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.835+0000 7f7b9602f700 1 -- 172.21.15.196:0/4000313999 shutdown_connections 2024-05-09T02:01:23.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.835+0000 7f7b9602f700 1 --2- 172.21.15.196:0/4000313999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b780035c0 0x7f7b78005aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:23.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.835+0000 7f7b9602f700 1 -- 172.21.15.196:0/4000313999 >> 172.21.15.196:0/4000313999 conn(0x7f7b900797e0 msgr2=0x7f7b90079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:23.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.835+0000 7f7b9602f700 1 -- 172.21.15.196:0/4000313999 shutdown_connections 2024-05-09T02:01:23.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.835+0000 7f7b9602f700 1 -- 172.21.15.196:0/4000313999 wait complete. 2024-05-09T02:01:23.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.836+0000 7f7b9602f700 1 Processor -- start 2024-05-09T02:01:23.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b9602f700 1 -- start start 2024-05-09T02:01:23.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b9602f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b780035c0 0x7f7b90105640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:23.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b9602f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b90105b80 0x7f7b90105f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:23.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b9602f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b901a5e00 0x7f7b901a8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:23.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b9602f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9010aaa0 con 0x7f7b90105b80 2024-05-09T02:01:23.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b9602f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9010a920 con 0x7f7b901a5e00 2024-05-09T02:01:23.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b9602f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9010ac20 con 0x7f7b780035c0 2024-05-09T02:01:23.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b8effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b90105b80 0x7f7b90105f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:23.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b8f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b780035c0 0x7f7b90105640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:23.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b8effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b90105b80 0x7f7b90105f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.196:55030/0 (socket says 172.21.15.196:55030) 2024-05-09T02:01:23.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b8effd700 1 -- 172.21.15.196:0/61059323 learned_addr learned my addr 172.21.15.196:0/61059323 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T02:01:23.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b8ffff700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b901a5e00 0x7f7b901a8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:23.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b8f7fe700 1 -- 172.21.15.196:0/61059323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b780035c0 msgr2=0x7f7b90105640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-09T02:01:23.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b8f7fe700 1 -- 172.21.15.196:0/61059323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b780035c0 msgr2=0x7f7b90105640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T02:01:23.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b8f7fe700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b780035c0 0x7f7b90105640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T02:01:23.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b8f7fe700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b780035c0 0x7f7b90105640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T02:01:23.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b8ffff700 1 -- 172.21.15.196:0/61059323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b901a5e00 msgr2=0x7f7b901a8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-09T02:01:23.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b8ffff700 1 -- 172.21.15.196:0/61059323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b901a5e00 msgr2=0x7f7b901a8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T02:01:23.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.837+0000 7f7b8ffff700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b901a5e00 0x7f7b901a8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T02:01:23.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.838+0000 7f7b8ffff700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b901a5e00 0x7f7b901a8220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T02:01:23.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.838+0000 7f7b8effd700 1 -- 172.21.15.196:0/61059323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b780035c0 msgr2=0x7f7b90105640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:01:23.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.838+0000 7f7b8effd700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b780035c0 0x7f7b90105640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:23.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.838+0000 7f7b8effd700 1 -- 172.21.15.196:0/61059323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b901a5e00 msgr2=0x7f7b901a8220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:01:23.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.838+0000 7f7b8effd700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b901a5e00 0x7f7b901a8220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:23.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.838+0000 7f7b8effd700 1 -- 172.21.15.196:0/61059323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b901a88a0 con 0x7f7b90105b80 2024-05-09T02:01:23.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.838+0000 7f7b8effd700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b90105b80 0x7f7b90105f90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400b780 tx=0x7f7b8400bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:23.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.838+0000 7f7b8cff9700 1 -- 172.21.15.196:0/61059323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84004230 con 0x7f7b90105b80 2024-05-09T02:01:23.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.838+0000 7f7b8cff9700 1 -- 172.21.15.196:0/61059323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b840043d0 con 0x7f7b90105b80 2024-05-09T02:01:23.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.838+0000 7f7b9602f700 1 -- 172.21.15.196:0/61059323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b901a8b90 con 0x7f7b90105b80 2024-05-09T02:01:23.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.838+0000 7f7b8cff9700 1 -- 172.21.15.196:0/61059323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84011bd0 con 0x7f7b90105b80 2024-05-09T02:01:23.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.838+0000 7f7b9602f700 1 -- 172.21.15.196:0/61059323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b901a9120 con 0x7f7b90105b80 2024-05-09T02:01:23.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.839+0000 7f7b8cff9700 1 -- 172.21.15.196:0/61059323 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7b8401a400 con 0x7f7b90105b80 2024-05-09T02:01:23.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.839+0000 7f7b9602f700 1 -- 172.21.15.196:0/61059323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b90064e20 con 0x7f7b90105b80 2024-05-09T02:01:23.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.839+0000 7f7b8cff9700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7b7c03e9a0 0x7f7b7c040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:23.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.840+0000 7f7b8cff9700 1 -- 172.21.15.196:0/61059323 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7b840533f0 con 0x7f7b90105b80 2024-05-09T02:01:23.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.840+0000 7f7b8f7fe700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7b7c03e9a0 0x7f7b7c040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:23.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.840+0000 7f7b8f7fe700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7b7c03e9a0 0x7f7b7c040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000fa10 tx=0x7f7b80005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:23.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:23.845+0000 7f7b8cff9700 1 -- 172.21.15.196:0/61059323 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7b84020740 con 0x7f7b90105b80 2024-05-09T02:01:23.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:01:23.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: mon.a calling monitor election 2024-05-09T02:01:23.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:23.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: mon.c calling monitor election 2024-05-09T02:01:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: mon.b calling monitor election 2024-05-09T02:01:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T02:01:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: monmap epoch 3 2024-05-09T02:01:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: fsid a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:01:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: last_changed 2024-05-09T02:01:18.585898+0000 2024-05-09T02:01:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: created 2024-05-09T01:59:13.406931+0000 2024-05-09T02:01:23.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: min_mon_release 19 (squid) 2024-05-09T02:01:23.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: election_strategy: 1 2024-05-09T02:01:23.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T02:01:23.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-05-09T02:01:23.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: 2: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.b 2024-05-09T02:01:23.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: fsmap 2024-05-09T02:01:23.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T02:01:23.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: mgrmap e13: x(active, since 59s) 2024-05-09T02:01:23.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: overall HEALTH_OK 2024-05-09T02:01:23.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:23.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:23.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:23.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:24.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:01:24.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: mon.a calling monitor election 2024-05-09T02:01:24.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:24.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:24.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: mon.c calling monitor election 2024-05-09T02:01:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: mon.b calling monitor election 2024-05-09T02:01:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T02:01:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: monmap epoch 3 2024-05-09T02:01:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: fsid a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:01:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: last_changed 2024-05-09T02:01:18.585898+0000 2024-05-09T02:01:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: created 2024-05-09T01:59:13.406931+0000 2024-05-09T02:01:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: min_mon_release 19 (squid) 2024-05-09T02:01:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: election_strategy: 1 2024-05-09T02:01:24.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T02:01:24.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-05-09T02:01:24.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: 2: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.b 2024-05-09T02:01:24.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: fsmap 2024-05-09T02:01:24.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T02:01:24.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: mgrmap e13: x(active, since 59s) 2024-05-09T02:01:24.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: overall HEALTH_OK 2024-05-09T02:01:24.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:24.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:24.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:24.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:24.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:01:24.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: mon.a calling monitor election 2024-05-09T02:01:24.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:24.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:01:24.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: mon.c calling monitor election 2024-05-09T02:01:24.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:24.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:24.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:24.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: mon.b calling monitor election 2024-05-09T02:01:24.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:24.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:24.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:24.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:24.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T02:01:24.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: monmap epoch 3 2024-05-09T02:01:24.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: fsid a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:01:24.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: last_changed 2024-05-09T02:01:18.585898+0000 2024-05-09T02:01:24.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: created 2024-05-09T01:59:13.406931+0000 2024-05-09T02:01:24.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: min_mon_release 19 (squid) 2024-05-09T02:01:24.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: election_strategy: 1 2024-05-09T02:01:24.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-09T02:01:24.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-05-09T02:01:24.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: 2: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.b 2024-05-09T02:01:24.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: fsmap 2024-05-09T02:01:24.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T02:01:24.074 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: mgrmap e13: x(active, since 59s) 2024-05-09T02:01:24.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: overall HEALTH_OK 2024-05-09T02:01:24.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:24.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:24.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:24.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:24.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:24.154+0000 7f7b9602f700 1 -- 172.21.15.196:0/61059323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7b901a93f0 con 0x7f7b90105b80 2024-05-09T02:01:24.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:24.155+0000 7f7b8cff9700 1 -- 172.21.15.196:0/61059323 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f7b84015df0 con 0x7f7b90105b80 2024-05-09T02:01:24.156 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-09T02:01:24.156 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"a0611f3e-0da7-11ef-bc97-c7b262605968","modified":"2024-05-09T02:01:18.585898Z","created":"2024-05-09T01:59:13.406931Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-09T02:01:24.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:24.157+0000 7f7b9602f700 1 -- 172.21.15.196:0/61059323 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7b7c03e9a0 msgr2=0x7f7b7c040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:24.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:24.157+0000 7f7b9602f700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7b7c03e9a0 0x7f7b7c040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000fa10 tx=0x7f7b80005de0 comp rx=0 tx=0).stop 2024-05-09T02:01:24.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:24.157+0000 7f7b9602f700 1 -- 172.21.15.196:0/61059323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b90105b80 msgr2=0x7f7b90105f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:24.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:24.157+0000 7f7b9602f700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b90105b80 0x7f7b90105f90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400b780 tx=0x7f7b8400bb40 comp rx=0 tx=0).stop 2024-05-09T02:01:24.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:24.157+0000 7f7b9602f700 1 -- 172.21.15.196:0/61059323 shutdown_connections 2024-05-09T02:01:24.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:24.157+0000 7f7b9602f700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7b7c03e9a0 0x7f7b7c040eb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:24.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:24.157+0000 7f7b9602f700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b901a5e00 0x7f7b901a8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:24.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:24.157+0000 7f7b9602f700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b780035c0 0x7f7b90105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:24.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:24.157+0000 7f7b9602f700 1 --2- 172.21.15.196:0/61059323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b90105b80 0x7f7b90105f90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:24.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:24.157+0000 7f7b9602f700 1 -- 172.21.15.196:0/61059323 >> 172.21.15.196:0/61059323 conn(0x7f7b900797e0 msgr2=0x7f7b90101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:24.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:24.157+0000 7f7b9602f700 1 -- 172.21.15.196:0/61059323 shutdown_connections 2024-05-09T02:01:24.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:24.157+0000 7f7b9602f700 1 -- 172.21.15.196:0/61059323 wait complete. 2024-05-09T02:01:24.159 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-05-09T02:01:24.717 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-09T02:01:24.717 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-09T02:01:24.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:24.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:24 smithi059 ceph-mon[34781]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:24.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:24.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:24 smithi059 ceph-mon[34781]: from='client.? 172.21.15.196:0/61059323' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T02:01:24.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:01:25.043 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:25.043 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:24 smithi196 ceph-mon[34718]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:25.043 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:25.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:24 smithi196 ceph-mon[34718]: from='client.? 172.21.15.196:0/61059323' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T02:01:25.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:25.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:25.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:24 smithi019 ceph-mon[30151]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:25.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:25.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:24 smithi019 ceph-mon[30151]: from='client.? 172.21.15.196:0/61059323' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T02:01:25.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:01:26.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:26.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:26.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:26.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:26.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:26.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:26.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:26.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.577+0000 7fd3eaea3700 1 -- 172.21.15.19:0/2190924922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e4108ac0 msgr2=0x7fd3e4108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:26.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.577+0000 7fd3eaea3700 1 --2- 172.21.15.19:0/2190924922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e4108ac0 0x7fd3e4108ed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009910 tx=0x7fd3d8009c20 comp rx=0 tx=0).stop 2024-05-09T02:01:26.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.577+0000 7fd3eaea3700 1 -- 172.21.15.19:0/2190924922 shutdown_connections 2024-05-09T02:01:26.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.577+0000 7fd3eaea3700 1 --2- 172.21.15.19:0/2190924922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e4108ac0 0x7fd3e4108ed0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:26.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.577+0000 7fd3eaea3700 1 -- 172.21.15.19:0/2190924922 >> 172.21.15.19:0/2190924922 conn(0x7fd3e40fe5b0 msgr2=0x7fd3e4100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:26.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.577+0000 7fd3eaea3700 1 -- 172.21.15.19:0/2190924922 shutdown_connections 2024-05-09T02:01:26.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.577+0000 7fd3eaea3700 1 -- 172.21.15.19:0/2190924922 wait complete. 2024-05-09T02:01:26.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.578+0000 7fd3eaea3700 1 Processor -- start 2024-05-09T02:01:26.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.578+0000 7fd3eaea3700 1 -- start start 2024-05-09T02:01:26.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3eaea3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4108ac0 0x7fd3e419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:26.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3eaea3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e419b070 0x7fd3e419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:26.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3eaea3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e41a3ba0 0x7fd3e41a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:26.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3eaea3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e410d010 con 0x7fd3e419b070 2024-05-09T02:01:26.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3eaea3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e410ce90 con 0x7fd3e41a3ba0 2024-05-09T02:01:26.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3eaea3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e410d190 con 0x7fd3e4108ac0 2024-05-09T02:01:26.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3e3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e419b070 0x7fd3e419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:26.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3e3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e419b070 0x7fd3e419b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41030/0 (socket says 172.21.15.19:41030) 2024-05-09T02:01:26.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3e3fff700 1 -- 172.21.15.19:0/12622611 learned_addr learned my addr 172.21.15.19:0/12622611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:01:26.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3e9440700 1 --2- 172.21.15.19:0/12622611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e41a3ba0 0x7fd3e41a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:26.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3e8c3f700 1 --2- 172.21.15.19:0/12622611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4108ac0 0x7fd3e419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:26.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3e3fff700 1 -- 172.21.15.19:0/12622611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4108ac0 msgr2=0x7fd3e419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:26.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3e3fff700 1 --2- 172.21.15.19:0/12622611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4108ac0 0x7fd3e419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:26.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3e3fff700 1 -- 172.21.15.19:0/12622611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e41a3ba0 msgr2=0x7fd3e41a5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:26.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3e3fff700 1 --2- 172.21.15.19:0/12622611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e41a3ba0 0x7fd3e41a5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:26.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3e3fff700 1 -- 172.21.15.19:0/12622611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e41a6750 con 0x7fd3e419b070 2024-05-09T02:01:26.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.579+0000 7fd3e3fff700 1 --2- 172.21.15.19:0/12622611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e419b070 0x7fd3e419b480 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000b780 tx=0x7fd3d000bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:26.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.580+0000 7fd3e1ffb700 1 -- 172.21.15.19:0/12622611 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0004230 con 0x7fd3e419b070 2024-05-09T02:01:26.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.580+0000 7fd3eaea3700 1 -- 172.21.15.19:0/12622611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e41a6a40 con 0x7fd3e419b070 2024-05-09T02:01:26.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.580+0000 7fd3e1ffb700 1 -- 172.21.15.19:0/12622611 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d00043d0 con 0x7fd3e419b070 2024-05-09T02:01:26.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.580+0000 7fd3e1ffb700 1 -- 172.21.15.19:0/12622611 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0011bd0 con 0x7fd3e419b070 2024-05-09T02:01:26.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.580+0000 7fd3eaea3700 1 -- 172.21.15.19:0/12622611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e41a6f80 con 0x7fd3e419b070 2024-05-09T02:01:26.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.581+0000 7fd3e1ffb700 1 -- 172.21.15.19:0/12622611 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd3d001a400 con 0x7fd3e419b070 2024-05-09T02:01:26.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.581+0000 7fd3eaea3700 1 -- 172.21.15.19:0/12622611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e4064e20 con 0x7fd3e419b070 2024-05-09T02:01:26.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.581+0000 7fd3e1ffb700 1 --2- 172.21.15.19:0/12622611 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd3d403e9a0 0x7fd3d4040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:26.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.581+0000 7fd3e1ffb700 1 -- 172.21.15.19:0/12622611 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd3d00523b0 con 0x7fd3e419b070 2024-05-09T02:01:26.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.582+0000 7fd3e8c3f700 1 --2- 172.21.15.19:0/12622611 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd3d403e9a0 0x7fd3d4040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:26.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.582+0000 7fd3e8c3f700 1 --2- 172.21.15.19:0/12622611 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd3d403e9a0 0x7fd3d4040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009750 tx=0x7fd3d8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.589+0000 7fd3e1ffb700 1 -- 172.21.15.19:0/12622611 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd3d0020740 con 0x7fd3e419b070 2024-05-09T02:01:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.806+0000 7fd3eaea3700 1 -- 172.21.15.19:0/12622611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fd3e41a7210 con 0x7fd3e419b070 2024-05-09T02:01:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.806+0000 7fd3e1ffb700 1 -- 172.21.15.19:0/12622611 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fd3d001a840 con 0x7fd3e419b070 2024-05-09T02:01:26.808 INFO:teuthology.orchestra.run.smithi019.stdout:# minimal ceph.conf for a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:01:26.808 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-05-09T02:01:26.808 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:01:26.808 INFO:teuthology.orchestra.run.smithi019.stdout: mon_host = [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-05-09T02:01:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.809+0000 7fd3eaea3700 1 -- 172.21.15.19:0/12622611 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd3d403e9a0 msgr2=0x7fd3d4040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.809+0000 7fd3eaea3700 1 --2- 172.21.15.19:0/12622611 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd3d403e9a0 0x7fd3d4040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009750 tx=0x7fd3d8005fb0 comp rx=0 tx=0).stop 2024-05-09T02:01:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.809+0000 7fd3eaea3700 1 -- 172.21.15.19:0/12622611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e419b070 msgr2=0x7fd3e419b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.809+0000 7fd3eaea3700 1 --2- 172.21.15.19:0/12622611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e419b070 0x7fd3e419b480 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000b780 tx=0x7fd3d000bb40 comp rx=0 tx=0).stop 2024-05-09T02:01:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.809+0000 7fd3eaea3700 1 -- 172.21.15.19:0/12622611 shutdown_connections 2024-05-09T02:01:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.809+0000 7fd3eaea3700 1 --2- 172.21.15.19:0/12622611 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd3d403e9a0 0x7fd3d4040eb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.809+0000 7fd3eaea3700 1 --2- 172.21.15.19:0/12622611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e41a3ba0 0x7fd3e41a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.809+0000 7fd3eaea3700 1 --2- 172.21.15.19:0/12622611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4108ac0 0x7fd3e419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.809+0000 7fd3eaea3700 1 --2- 172.21.15.19:0/12622611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e419b070 0x7fd3e419b480 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.809+0000 7fd3eaea3700 1 -- 172.21.15.19:0/12622611 >> 172.21.15.19:0/12622611 conn(0x7fd3e40fe5b0 msgr2=0x7fd3e40ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.809+0000 7fd3eaea3700 1 -- 172.21.15.19:0/12622611 shutdown_connections 2024-05-09T02:01:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:26.810+0000 7fd3eaea3700 1 -- 172.21.15.19:0/12622611 wait complete. 2024-05-09T02:01:27.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-09T02:01:27.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-09T02:01:27.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-09T02:01:27.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: Updating smithi019:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:01:27.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: Updating smithi196:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:01:27.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: Updating smithi059:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:01:27.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:27.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T02:01:27.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T02:01:27.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T02:01:27.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:27.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: Reconfiguring daemon mon.a on smithi019 2024-05-09T02:01:27.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:26 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/12622611' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:27.215 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-09T02:01:27.215 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-09T02:01:27.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-09T02:01:27.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: Updating smithi019:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:01:27.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: Updating smithi196:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:01:27.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: Updating smithi059:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:01:27.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:27.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T02:01:27.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T02:01:27.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T02:01:27.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:27.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: Reconfiguring daemon mon.a on smithi019 2024-05-09T02:01:27.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:26 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/12622611' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:27.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-09T02:01:27.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-09T02:01:27.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-09T02:01:27.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: Updating smithi019:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:01:27.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: Updating smithi196:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:01:27.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: Updating smithi059:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:01:27.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:27.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:27.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T02:01:27.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T02:01:27.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T02:01:27.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:27.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: Reconfiguring daemon mon.a on smithi019 2024-05-09T02:01:27.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:26 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/12622611' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:28.132 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-09T02:01:28.132 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T02:01:28.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T02:01:28.160 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T02:01:28.160 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:01:28.226 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T02:01:28.226 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T02:01:28.254 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T02:01:28.254 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:01:28.319 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T02:01:28.319 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T02:01:28.350 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T02:01:28.350 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:01:28.419 INFO:tasks.cephadm:Adding mgr.x on smithi019 2024-05-09T02:01:28.419 INFO:tasks.cephadm:Adding mgr.y on smithi059 2024-05-09T02:01:28.419 INFO:tasks.cephadm:Adding mgr.z on smithi196 2024-05-09T02:01:28.420 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi019=x;smithi059=y;smithi196=z' 2024-05-09T02:01:28.695 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:01:28.775 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:28 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:28.775 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:28 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:28.775 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:28 smithi059 ceph-mon[34781]: Reconfiguring mon.b (monmap changed)... 2024-05-09T02:01:28.775 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:28 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T02:01:28.775 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:28 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T02:01:28.775 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:28 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:28.776 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:28 smithi059 ceph-mon[34781]: Reconfiguring daemon mon.b on smithi059 2024-05-09T02:01:28.776 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:28 smithi059 ceph-mon[34781]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:28.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:28 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:28 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:28 smithi196 ceph-mon[34718]: Reconfiguring mon.b (monmap changed)... 2024-05-09T02:01:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:28 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T02:01:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:28 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T02:01:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:28 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:28 smithi196 ceph-mon[34718]: Reconfiguring daemon mon.b on smithi059 2024-05-09T02:01:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:28 smithi196 ceph-mon[34718]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:28.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:28 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:28.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:28 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:28.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:28 smithi019 ceph-mon[30151]: Reconfiguring mon.b (monmap changed)... 2024-05-09T02:01:28.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:28 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T02:01:28.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:28 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T02:01:28.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:28 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:28.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:28 smithi019 ceph-mon[30151]: Reconfiguring daemon mon.b on smithi059 2024-05-09T02:01:28.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:28 smithi019 ceph-mon[30151]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:30.255 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:30 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:30.255 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:30 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:30.255 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:30 smithi196 ceph-mon[34718]: Reconfiguring mon.c (monmap changed)... 2024-05-09T02:01:30.255 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:30 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T02:01:30.255 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:30 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T02:01:30.255 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:30 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:30.255 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:30 smithi196 ceph-mon[34718]: Reconfiguring daemon mon.c on smithi196 2024-05-09T02:01:30.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.268+0000 7fe3c221d700 1 -- 172.21.15.196:0/2448394829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3bc102bf0 msgr2=0x7fe3bc103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:30.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.268+0000 7fe3c221d700 1 --2- 172.21.15.196:0/2448394829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3bc102bf0 0x7fe3bc103000 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009910 tx=0x7fe3ac009c20 comp rx=0 tx=0).stop 2024-05-09T02:01:30.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.268+0000 7fe3c221d700 1 -- 172.21.15.196:0/2448394829 shutdown_connections 2024-05-09T02:01:30.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.268+0000 7fe3c221d700 1 --2- 172.21.15.196:0/2448394829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3bc102bf0 0x7fe3bc103000 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:30.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.268+0000 7fe3c221d700 1 -- 172.21.15.196:0/2448394829 >> 172.21.15.196:0/2448394829 conn(0x7fe3bc0fe250 msgr2=0x7fe3bc1006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:30.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.268+0000 7fe3c221d700 1 -- 172.21.15.196:0/2448394829 shutdown_connections 2024-05-09T02:01:30.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.268+0000 7fe3c221d700 1 -- 172.21.15.196:0/2448394829 wait complete. 2024-05-09T02:01:30.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.269+0000 7fe3c221d700 1 Processor -- start 2024-05-09T02:01:30.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.269+0000 7fe3c221d700 1 -- start start 2024-05-09T02:01:30.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.269+0000 7fe3c221d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3bc102bf0 0x7fe3bc06c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:30.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.269+0000 7fe3c221d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3bc19c720 0x7fe3bc19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:30.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.269+0000 7fe3c221d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3bc19f080 0x7fe3bc1a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:30.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.269+0000 7fe3c221d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc108bb0 con 0x7fe3bc19f080 2024-05-09T02:01:30.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.269+0000 7fe3c221d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc108a30 con 0x7fe3bc102bf0 2024-05-09T02:01:30.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.269+0000 7fe3c221d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc108d30 con 0x7fe3bc19c720 2024-05-09T02:01:30.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.269+0000 7fe3baffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3bc19c720 0x7fe3bc19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:30.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.269+0000 7fe3baffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3bc19c720 0x7fe3bc19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52268/0 (socket says 172.21.15.196:52268) 2024-05-09T02:01:30.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.269+0000 7fe3baffd700 1 -- 172.21.15.196:0/152782868 learned_addr learned my addr 172.21.15.196:0/152782868 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T02:01:30.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.269+0000 7fe3bb7fe700 1 --2- 172.21.15.196:0/152782868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3bc102bf0 0x7fe3bc06c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:30.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.269+0000 7fe3bbfff700 1 --2- 172.21.15.196:0/152782868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3bc19f080 0x7fe3bc1a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:30.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.269+0000 7fe3baffd700 1 -- 172.21.15.196:0/152782868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3bc102bf0 msgr2=0x7fe3bc06c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:30.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.269+0000 7fe3baffd700 1 --2- 172.21.15.196:0/152782868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3bc102bf0 0x7fe3bc06c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:30.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.270+0000 7fe3baffd700 1 -- 172.21.15.196:0/152782868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3bc19f080 msgr2=0x7fe3bc1a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:30.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.270+0000 7fe3baffd700 1 --2- 172.21.15.196:0/152782868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3bc19f080 0x7fe3bc1a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:30.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.270+0000 7fe3baffd700 1 -- 172.21.15.196:0/152782868 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3bc1a1d10 con 0x7fe3bc19c720 2024-05-09T02:01:30.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.270+0000 7fe3bbfff700 1 --2- 172.21.15.196:0/152782868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3bc19f080 0x7fe3bc1a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:01:30.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.270+0000 7fe3baffd700 1 --2- 172.21.15.196:0/152782868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3bc19c720 0x7fe3bc19eb40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000b780 tx=0x7fe3b000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:30.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.270+0000 7fe3b8ff9700 1 -- 172.21.15.196:0/152782868 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0004230 con 0x7fe3bc19c720 2024-05-09T02:01:30.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.270+0000 7fe3b8ff9700 1 -- 172.21.15.196:0/152782868 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b00043d0 con 0x7fe3bc19c720 2024-05-09T02:01:30.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.270+0000 7fe3c221d700 1 -- 172.21.15.196:0/152782868 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3bc1a1fa0 con 0x7fe3bc19c720 2024-05-09T02:01:30.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.270+0000 7fe3b8ff9700 1 -- 172.21.15.196:0/152782868 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0011b20 con 0x7fe3bc19c720 2024-05-09T02:01:30.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.270+0000 7fe3c221d700 1 -- 172.21.15.196:0/152782868 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3bc1a2530 con 0x7fe3bc19c720 2024-05-09T02:01:30.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.271+0000 7fe3c221d700 1 -- 172.21.15.196:0/152782868 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3bc064e20 con 0x7fe3bc19c720 2024-05-09T02:01:30.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.271+0000 7fe3b8ff9700 1 -- 172.21.15.196:0/152782868 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe3b0011cc0 con 0x7fe3bc19c720 2024-05-09T02:01:30.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.271+0000 7fe3b8ff9700 1 --2- 172.21.15.196:0/152782868 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe3a403e9a0 0x7fe3a4040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:30.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.271+0000 7fe3b8ff9700 1 -- 172.21.15.196:0/152782868 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe3b0052300 con 0x7fe3bc19c720 2024-05-09T02:01:30.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.272+0000 7fe3bb7fe700 1 --2- 172.21.15.196:0/152782868 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe3a403e9a0 0x7fe3a4040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:30.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.272+0000 7fe3bb7fe700 1 --2- 172.21.15.196:0/152782868 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe3a403e9a0 0x7fe3a4040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac006010 tx=0x7fe3ac01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:30.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.277+0000 7fe3b8ff9700 1 -- 172.21.15.196:0/152782868 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe3b00151c0 con 0x7fe3bc19c720 2024-05-09T02:01:30.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:30 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:30.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:30 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:30.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:30 smithi059 ceph-mon[34781]: Reconfiguring mon.c (monmap changed)... 2024-05-09T02:01:30.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:30 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T02:01:30.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:30 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T02:01:30.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:30 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:30.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:30 smithi059 ceph-mon[34781]: Reconfiguring daemon mon.c on smithi196 2024-05-09T02:01:30.501 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.500+0000 7fe3c221d700 1 -- 172.21.15.196:0/152782868 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi059=y;smithi196=z", "target": ["mon-mgr", ""]}) -- 0x7fe3bc0ffff0 con 0x7fe3a403e9a0 2024-05-09T02:01:30.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.506+0000 7fe3b8ff9700 1 -- 172.21.15.196:0/152782868 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe3bc0ffff0 con 0x7fe3a403e9a0 2024-05-09T02:01:30.508 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-05-09T02:01:30.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.509+0000 7fe3c221d700 1 -- 172.21.15.196:0/152782868 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe3a403e9a0 msgr2=0x7fe3a4040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:30.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.509+0000 7fe3c221d700 1 --2- 172.21.15.196:0/152782868 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe3a403e9a0 0x7fe3a4040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac006010 tx=0x7fe3ac01a040 comp rx=0 tx=0).stop 2024-05-09T02:01:30.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.509+0000 7fe3c221d700 1 -- 172.21.15.196:0/152782868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3bc19c720 msgr2=0x7fe3bc19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:30.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.509+0000 7fe3c221d700 1 --2- 172.21.15.196:0/152782868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3bc19c720 0x7fe3bc19eb40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000b780 tx=0x7fe3b000ba90 comp rx=0 tx=0).stop 2024-05-09T02:01:30.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.509+0000 7fe3c221d700 1 -- 172.21.15.196:0/152782868 shutdown_connections 2024-05-09T02:01:30.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.510+0000 7fe3c221d700 1 --2- 172.21.15.196:0/152782868 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe3a403e9a0 0x7fe3a4040eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:30.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.510+0000 7fe3c221d700 1 --2- 172.21.15.196:0/152782868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3bc19f080 0x7fe3bc1a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:30.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.510+0000 7fe3c221d700 1 --2- 172.21.15.196:0/152782868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3bc102bf0 0x7fe3bc06c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:30.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.510+0000 7fe3c221d700 1 --2- 172.21.15.196:0/152782868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3bc19c720 0x7fe3bc19eb40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:30.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.510+0000 7fe3c221d700 1 -- 172.21.15.196:0/152782868 >> 172.21.15.196:0/152782868 conn(0x7fe3bc0fe250 msgr2=0x7fe3bc0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:30.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.510+0000 7fe3c221d700 1 -- 172.21.15.196:0/152782868 shutdown_connections 2024-05-09T02:01:30.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:01:30.510+0000 7fe3c221d700 1 -- 172.21.15.196:0/152782868 wait complete. 2024-05-09T02:01:30.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:30 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:30.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:30 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:30.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:30 smithi019 ceph-mon[30151]: Reconfiguring mon.c (monmap changed)... 2024-05-09T02:01:30.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:30 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T02:01:30.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:30 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T02:01:30.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:30 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:30.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:30 smithi019 ceph-mon[30151]: Reconfiguring daemon mon.c on smithi196 2024-05-09T02:01:31.213 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:31 smithi196 ceph-mon[34718]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:31.213 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:31 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:31.214 DEBUG:teuthology.orchestra.run.smithi059:mgr.y> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mgr.y.service 2024-05-09T02:01:31.217 DEBUG:teuthology.orchestra.run.smithi196:mgr.z> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mgr.z.service 2024-05-09T02:01:31.219 INFO:tasks.cephadm:Deploying OSDs... 2024-05-09T02:01:31.219 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T02:01:31.219 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T02:01:31.240 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T02:01:31.240 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-05-09T02:01:31.300 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T02:01:31.300 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T02:01:31.300 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T02:01:31.300 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T02:01:31.300 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T02:01:31.300 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-09 02:00:34.299134558 +0000 2024-05-09T02:01:31.300 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-09 01:55:45.516499425 +0000 2024-05-09T02:01:31.300 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-09 01:55:45.516499425 +0000 2024-05-09T02:01:31.300 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-09 01:55:45.516499425 +0000 2024-05-09T02:01:31.300 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T02:01:31.369 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-09T02:01:31.370 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-09T02:01:31.370 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000266178 s, 1.9 MB/s 2024-05-09T02:01:31.371 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T02:01:31.432 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-05-09T02:01:31.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:31 smithi059 ceph-mon[34781]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:31.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:31 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:31.491 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T02:01:31.492 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T02:01:31.492 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-09T02:01:31.492 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T02:01:31.492 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T02:01:31.492 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-09 02:00:34.334134216 +0000 2024-05-09T02:01:31.492 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-09 01:55:45.870496102 +0000 2024-05-09T02:01:31.492 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-09 01:55:45.870496102 +0000 2024-05-09T02:01:31.492 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-09 01:55:45.870496102 +0000 2024-05-09T02:01:31.492 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T02:01:31.558 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:31 smithi019 ceph-mon[30151]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:31.558 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:31 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:31.561 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-09T02:01:31.561 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-09T02:01:31.561 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.00021133 s, 2.4 MB/s 2024-05-09T02:01:31.563 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T02:01:31.623 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-05-09T02:01:31.682 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T02:01:31.682 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T02:01:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-09T02:01:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T02:01:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T02:01:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-09 02:00:34.378133786 +0000 2024-05-09T02:01:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-09 01:55:46.223492788 +0000 2024-05-09T02:01:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-09 01:55:46.223492788 +0000 2024-05-09T02:01:31.683 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-09 01:55:46.223492788 +0000 2024-05-09T02:01:31.683 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T02:01:31.748 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-09T02:01:31.748 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-09T02:01:31.748 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000209209 s, 2.4 MB/s 2024-05-09T02:01:31.749 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T02:01:31.807 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-05-09T02:01:31.863 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T02:01:31.863 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T02:01:31.863 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-09T02:01:31.863 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T02:01:31.864 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T02:01:31.864 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-09 02:00:34.415133425 +0000 2024-05-09T02:01:31.864 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-09 01:55:46.586489381 +0000 2024-05-09T02:01:31.864 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-09 01:55:46.586489381 +0000 2024-05-09T02:01:31.864 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-09 01:55:46.586489381 +0000 2024-05-09T02:01:31.864 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T02:01:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-09T02:01:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-09T02:01:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000207959 s, 2.5 MB/s 2024-05-09T02:01:31.929 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T02:01:31.986 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-09T02:01:31.986 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T02:01:32.005 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T02:01:32.005 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_1 2024-05-09T02:01:32.064 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T02:01:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T02:01:32.065 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T02:01:32.065 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T02:01:32.065 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T02:01:32.065 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-09 02:01:18.000732917 +0000 2024-05-09T02:01:32.065 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-09 01:56:00.275193205 +0000 2024-05-09T02:01:32.065 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-09 01:56:00.275193205 +0000 2024-05-09T02:01:32.065 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-09 01:56:00.275193205 +0000 2024-05-09T02:01:32.065 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T02:01:32.133 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-09T02:01:32.133 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-09T02:01:32.133 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.0003127 s, 1.6 MB/s 2024-05-09T02:01:32.135 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T02:01:32.194 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_2 2024-05-09T02:01:32.252 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T02:01:32.252 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T02:01:32.252 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-09T02:01:32.253 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T02:01:32.253 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T02:01:32.253 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-09 02:01:18.001732905 +0000 2024-05-09T02:01:32.253 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-09 01:56:00.634188593 +0000 2024-05-09T02:01:32.253 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-09 01:56:00.634188593 +0000 2024-05-09T02:01:32.253 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-09 01:56:00.634188593 +0000 2024-05-09T02:01:32.253 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T02:01:32.319 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-09T02:01:32.319 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-09T02:01:32.319 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000186025 s, 2.8 MB/s 2024-05-09T02:01:32.320 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T02:01:32.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:32 smithi019 ceph-mon[30151]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi059=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:01:32.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:32 smithi019 ceph-mon[30151]: Saving service mgr spec with placement smithi019=x;smithi059=y;smithi196=z;count:3 2024-05-09T02:01:32.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:32 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:32.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:32 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:32.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:32 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:32.379 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_3 2024-05-09T02:01:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T02:01:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T02:01:32.438 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T02:01:32.438 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T02:01:32.439 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T02:01:32.439 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-09 02:01:18.001732905 +0000 2024-05-09T02:01:32.439 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-09 01:56:00.997183931 +0000 2024-05-09T02:01:32.439 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-09 01:56:00.997183931 +0000 2024-05-09T02:01:32.439 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-09 01:56:00.997183931 +0000 2024-05-09T02:01:32.439 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T02:01:32.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:32 smithi059 ceph-mon[34781]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi059=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:01:32.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:32 smithi059 ceph-mon[34781]: Saving service mgr spec with placement smithi019=x;smithi059=y;smithi196=z;count:3 2024-05-09T02:01:32.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:32 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:32.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:32 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:32.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:32 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:32.492 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-09T02:01:32.492 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-09T02:01:32.492 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000205428 s, 2.5 MB/s 2024-05-09T02:01:32.493 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T02:01:32.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:32 smithi196 ceph-mon[34718]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi059=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:01:32.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:32 smithi196 ceph-mon[34718]: Saving service mgr spec with placement smithi019=x;smithi059=y;smithi196=z;count:3 2024-05-09T02:01:32.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:32 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:32.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:32 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:32.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:32 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:32.551 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_4 2024-05-09T02:01:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T02:01:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T02:01:32.609 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-09T02:01:32.609 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T02:01:32.609 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T02:01:32.609 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-09 02:01:18.002732892 +0000 2024-05-09T02:01:32.609 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-09 01:56:01.349179409 +0000 2024-05-09T02:01:32.609 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-09 01:56:01.349179409 +0000 2024-05-09T02:01:32.609 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-09 01:56:01.349179409 +0000 2024-05-09T02:01:32.610 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T02:01:32.674 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-09T02:01:32.675 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-09T02:01:32.675 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000220861 s, 2.3 MB/s 2024-05-09T02:01:32.676 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T02:01:32.733 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-09T02:01:32.733 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T02:01:32.749 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T02:01:32.749 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-05-09T02:01:32.804 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T02:01:32.804 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T02:01:32.804 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-05-09T02:01:32.804 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T02:01:32.804 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T02:01:32.804 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-05-09 02:01:08.922413367 +0000 2024-05-09T02:01:32.804 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-05-09 01:56:07.640638838 +0000 2024-05-09T02:01:32.804 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-05-09 01:56:07.640638838 +0000 2024-05-09T02:01:32.804 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-05-09 01:56:07.640638838 +0000 2024-05-09T02:01:32.805 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T02:01:32.867 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-05-09T02:01:32.867 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-05-09T02:01:32.867 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000266471 s, 1.9 MB/s 2024-05-09T02:01:32.868 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T02:01:32.924 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-05-09T02:01:32.979 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T02:01:32.979 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T02:01:32.979 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-05-09T02:01:32.979 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T02:01:32.979 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T02:01:32.979 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-05-09 02:01:08.923413347 +0000 2024-05-09T02:01:32.979 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-05-09 01:56:07.997631277 +0000 2024-05-09T02:01:32.979 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-05-09 01:56:07.997631277 +0000 2024-05-09T02:01:32.980 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-05-09 01:56:07.997631277 +0000 2024-05-09T02:01:32.980 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T02:01:33.041 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-05-09T02:01:33.041 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-05-09T02:01:33.041 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000226763 s, 2.3 MB/s 2024-05-09T02:01:33.046 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T02:01:33.098 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-05-09T02:01:33.153 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T02:01:33.153 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T02:01:33.154 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T02:01:33.154 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T02:01:33.154 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T02:01:33.154 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-05-09 02:01:08.923413347 +0000 2024-05-09T02:01:33.154 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-05-09 01:56:08.322624395 +0000 2024-05-09T02:01:33.154 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-05-09 01:56:08.322624395 +0000 2024-05-09T02:01:33.154 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-05-09 01:56:08.322624395 +0000 2024-05-09T02:01:33.154 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T02:01:33.217 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-05-09T02:01:33.217 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-05-09T02:01:33.217 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000178207 s, 2.9 MB/s 2024-05-09T02:01:33.218 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T02:01:33.274 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-05-09T02:01:33.328 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T02:01:33.329 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T02:01:33.329 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-09T02:01:33.329 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T02:01:33.329 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T02:01:33.329 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-05-09 02:01:08.924413326 +0000 2024-05-09T02:01:33.329 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-05-09 01:56:08.659617258 +0000 2024-05-09T02:01:33.329 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-05-09 01:56:08.659617258 +0000 2024-05-09T02:01:33.329 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-05-09 01:56:08.659617258 +0000 2024-05-09T02:01:33.329 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T02:01:33.390 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-05-09T02:01:33.391 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-05-09T02:01:33.391 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000192947 s, 2.7 MB/s 2024-05-09T02:01:33.392 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T02:01:33.447 INFO:tasks.cephadm:Deploying osd.0 on smithi019 with /dev/vg_nvme/lv_4... 2024-05-09T02:01:33.447 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T02:01:33.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:33 smithi059 ceph-mon[34781]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:33.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:33 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:33.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:33 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:33.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:33 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:33.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:33 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T02:01:33.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:33 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T02:01:33.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:33 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T02:01:33.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:33 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:33.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:33 smithi059 ceph-mon[34781]: Deploying daemon mgr.z on smithi196 2024-05-09T02:01:33.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:33 smithi019 ceph-mon[30151]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:33.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:33 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:33.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:33 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:33.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:33 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:33.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:33 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T02:01:33.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:33 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T02:01:33.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:33 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T02:01:33.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:33 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:33.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:33 smithi019 ceph-mon[30151]: Deploying daemon mgr.z on smithi196 2024-05-09T02:01:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:33 smithi196 ceph-mon[34718]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:33 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:33 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:33 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:33 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T02:01:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:33 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T02:01:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:33 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T02:01:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:33 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:33 smithi196 ceph-mon[34718]: Deploying daemon mgr.z on smithi196 2024-05-09T02:01:33.701 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:01:34.447 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:34 smithi196 systemd[1]: Starting Ceph mgr.z for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:01:34.758 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:34 smithi196 podman[36555]: 2024-05-09 02:01:34.446048155 +0000 UTC m=+0.101807332 container create 8fab2740a03d5be50c56a03b68ef5ab470fe8701c1d9bcec28d4c306ceecb81e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-09T02:01:34.758 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:34 smithi196 podman[36555]: 2024-05-09 02:01:34.361391446 +0000 UTC m=+0.017150593 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:01:34.758 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:34 smithi196 podman[36555]: 2024-05-09 02:01:34.657000617 +0000 UTC m=+0.312759751 container init 8fab2740a03d5be50c56a03b68ef5ab470fe8701c1d9bcec28d4c306ceecb81e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-09T02:01:34.758 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:34 smithi196 podman[36555]: 2024-05-09 02:01:34.659791778 +0000 UTC m=+0.315550926 container start 8fab2740a03d5be50c56a03b68ef5ab470fe8701c1d9bcec28d4c306ceecb81e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-09T02:01:34.758 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:34 smithi196 ceph-mgr[36574]: -- 172.21.15.196:0/3364524265 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5573257f45a0 con 0x55732a57e900 2024-05-09T02:01:34.758 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:34 smithi196 bash[36555]: 8fab2740a03d5be50c56a03b68ef5ab470fe8701c1d9bcec28d4c306ceecb81e 2024-05-09T02:01:35.009 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:34 smithi196 systemd[1]: Started Ceph mgr.z for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:01:35.009 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:34 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:34.821+0000 7fb573f51200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T02:01:35.009 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:35 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:35.006+0000 7fb573f51200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T02:01:35.293 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:35 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:35.108+0000 7fb573f51200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T02:01:35.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:35 smithi196 ceph-mon[34718]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:35.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:35.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:35.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:35.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T02:01:35.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T02:01:35.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T02:01:35.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:35.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:35 smithi019 ceph-mon[30151]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:35.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:35.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:35.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:35.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T02:01:35.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T02:01:35.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T02:01:35.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:35.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:35 smithi059 ceph-mon[34781]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:35.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:35.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:35.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:35.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T02:01:35.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T02:01:35.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T02:01:35.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:35.565 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T02:01:35.566 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T02:01:35.566 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-09T02:01:35.566 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-09T02:01:35.566 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122686 s, 855 MB/s 2024-05-09T02:01:35.566 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-09T02:01:35.793 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:35 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:35.414+0000 7fb573f51200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T02:01:35.793 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:35 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:35.499+0000 7fb573f51200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T02:01:36.203 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_4 2024-05-09T02:01:36.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:36 smithi059 ceph-mon[34781]: Deploying daemon mgr.y on smithi059 2024-05-09T02:01:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:01:36.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:36 smithi196 ceph-mon[34718]: Deploying daemon mgr.y on smithi059 2024-05-09T02:01:36.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:36 smithi019 ceph-mon[30151]: Deploying daemon mgr.y on smithi059 2024-05-09T02:01:36.948 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:36 smithi059 systemd[1]: Starting Ceph mgr.y for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:01:37.016 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:36 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:36.674+0000 7fb573f51200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T02:01:37.017 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:36 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:36.841+0000 7fb573f51200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T02:01:37.017 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:37 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:37.014+0000 7fb573f51200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T02:01:37.201 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:37 smithi059 ceph-mon[34781]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:37.201 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:36 smithi059 podman[36155]: 2024-05-09 02:01:36.850791269 +0000 UTC m=+0.018491114 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:01:37.201 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:36 smithi059 podman[36155]: 2024-05-09 02:01:36.960896299 +0000 UTC m=+0.128596175 container create f0c89ea92ac19a9194b88ffd7ebf2ba59460317c77b0d07ba111b3ca48ed50fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-09T02:01:37.201 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:37 smithi059 podman[36155]: 2024-05-09 02:01:37.163159515 +0000 UTC m=+0.330859351 container init f0c89ea92ac19a9194b88ffd7ebf2ba59460317c77b0d07ba111b3ca48ed50fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-09T02:01:37.201 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:37 smithi059 podman[36155]: 2024-05-09 02:01:37.165770076 +0000 UTC m=+0.333469912 container start f0c89ea92ac19a9194b88ffd7ebf2ba59460317c77b0d07ba111b3ca48ed50fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-09T02:01:37.201 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:37 smithi059 ceph-mgr[36174]: -- 172.21.15.59:0/3732263899 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5594673cc5a0 con 0x55946c156480 2024-05-09T02:01:37.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:37 smithi196 ceph-mon[34718]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:37.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:37 smithi019 ceph-mon[30151]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:37.465 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:37 smithi059 bash[36155]: f0c89ea92ac19a9194b88ffd7ebf2ba59460317c77b0d07ba111b3ca48ed50fb 2024-05-09T02:01:37.465 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:37 smithi059 systemd[1]: Started Ceph mgr.y for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:01:37.465 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:37 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:37.337+0000 7fc48c5e4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T02:01:37.621 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:37 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:37.313+0000 7fb573f51200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T02:01:37.621 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:37 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:37.408+0000 7fb573f51200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T02:01:37.621 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:37 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:37.619+0000 7fb573f51200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T02:01:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.911+0000 7f849f1da700 1 -- 172.21.15.19:0/3893255920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8498107480 msgr2=0x7f8498109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.911+0000 7f849f1da700 1 --2- 172.21.15.19:0/3893255920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8498107480 0x7f8498109960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f848c009990 tx=0x7f848c009ca0 comp rx=0 tx=0).stop 2024-05-09T02:01:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.911+0000 7f849f1da700 1 -- 172.21.15.19:0/3893255920 shutdown_connections 2024-05-09T02:01:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.911+0000 7f849f1da700 1 --2- 172.21.15.19:0/3893255920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8498109ff0 0x7f849810c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.911+0000 7f849f1da700 1 --2- 172.21.15.19:0/3893255920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f849806c2d0 0x7f8498106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.911+0000 7f849f1da700 1 --2- 172.21.15.19:0/3893255920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8498107480 0x7f8498109960 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.911+0000 7f849f1da700 1 -- 172.21.15.19:0/3893255920 >> 172.21.15.19:0/3893255920 conn(0x7f84980fe880 msgr2=0x7f8498100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.911+0000 7f849f1da700 1 -- 172.21.15.19:0/3893255920 shutdown_connections 2024-05-09T02:01:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.911+0000 7f849f1da700 1 -- 172.21.15.19:0/3893255920 wait complete. 2024-05-09T02:01:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f849f1da700 1 Processor -- start 2024-05-09T02:01:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f849f1da700 1 -- start start 2024-05-09T02:01:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f849f1da700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f849806c2d0 0x7f8498104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f849f1da700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8498107480 0x7f8498102db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f849f1da700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8498109ff0 0x7f84981032f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f849f1da700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f849810fd80 con 0x7f8498107480 2024-05-09T02:01:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f849f1da700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f849810fc00 con 0x7f8498109ff0 2024-05-09T02:01:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f849f1da700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f849810ff00 con 0x7f849806c2d0 2024-05-09T02:01:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f8497fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8498107480 0x7f8498102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f8497fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8498107480 0x7f8498102db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38960/0 (socket says 172.21.15.19:38960) 2024-05-09T02:01:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f8497fff700 1 -- 172.21.15.19:0/1186494465 learned_addr learned my addr 172.21.15.19:0/1186494465 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:01:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f849cf76700 1 --2- 172.21.15.19:0/1186494465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f849806c2d0 0x7f8498104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f849d777700 1 --2- 172.21.15.19:0/1186494465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8498109ff0 0x7f84981032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f8497fff700 1 -- 172.21.15.19:0/1186494465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f849806c2d0 msgr2=0x7f8498104700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f8497fff700 1 --2- 172.21.15.19:0/1186494465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f849806c2d0 0x7f8498104700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f8497fff700 1 -- 172.21.15.19:0/1186494465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8498109ff0 msgr2=0x7f84981032f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f8497fff700 1 --2- 172.21.15.19:0/1186494465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8498109ff0 0x7f84981032f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.912+0000 7f8497fff700 1 -- 172.21.15.19:0/1186494465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8498103a50 con 0x7f8498107480 2024-05-09T02:01:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.913+0000 7f8497fff700 1 --2- 172.21.15.19:0/1186494465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8498107480 0x7f8498102db0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f848c00b760 tx=0x7f848c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.913+0000 7f8495ffb700 1 -- 172.21.15.19:0/1186494465 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c026070 con 0x7f8498107480 2024-05-09T02:01:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.913+0000 7f849f1da700 1 -- 172.21.15.19:0/1186494465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84981a49c0 con 0x7f8498107480 2024-05-09T02:01:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.913+0000 7f8495ffb700 1 -- 172.21.15.19:0/1186494465 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f848c021410 con 0x7f8498107480 2024-05-09T02:01:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.913+0000 7f8495ffb700 1 -- 172.21.15.19:0/1186494465 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c017650 con 0x7f8498107480 2024-05-09T02:01:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.913+0000 7f849f1da700 1 -- 172.21.15.19:0/1186494465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84981a4db0 con 0x7f8498107480 2024-05-09T02:01:37.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.914+0000 7f8495ffb700 1 -- 172.21.15.19:0/1186494465 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f848c029070 con 0x7f8498107480 2024-05-09T02:01:37.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.914+0000 7f849f1da700 1 -- 172.21.15.19:0/1186494465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84980512c0 con 0x7f8498107480 2024-05-09T02:01:37.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.914+0000 7f8495ffb700 1 --2- 172.21.15.19:0/1186494465 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f848003e750 0x7f8480040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:01:37.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.914+0000 7f8495ffb700 1 -- 172.21.15.19:0/1186494465 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f848c053790 con 0x7f8498107480 2024-05-09T02:01:37.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.914+0000 7f849cf76700 1 --2- 172.21.15.19:0/1186494465 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f848003e750 0x7f8480040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:01:37.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.915+0000 7f849cf76700 1 --2- 172.21.15.19:0/1186494465 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f848003e750 0x7f8480040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8488009c60 tx=0x7f8488009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:01:37.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:37.920+0000 7f8495ffb700 1 -- 172.21.15.19:0/1186494465 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f848c013070 con 0x7f8498107480 2024-05-09T02:01:37.965 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:37 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:37.532+0000 7fc48c5e4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T02:01:37.965 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:37 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:37.640+0000 7fc48c5e4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T02:01:38.043 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:37 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:37.820+0000 7fb573f51200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T02:01:38.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:38.143+0000 7f849f1da700 1 -- 172.21.15.19:0/1186494465 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f849802ea00 con 0x7f848003e750 2024-05-09T02:01:38.337 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:37 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:37.972+0000 7fc48c5e4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T02:01:38.337 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:38 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:38.072+0000 7fc48c5e4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T02:01:38.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:38 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:38.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:38 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:38.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:38 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:38.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:38 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:38.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:38 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:38.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:38 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:01:38.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:38 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:01:38.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:38 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:38.692 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:38 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:38.466+0000 7fb573f51200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T02:01:38.692 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:38 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:38.689+0000 7fb573f51200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T02:01:38.692 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:38 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:38.692 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:38 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:38.692 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:38 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:38.692 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:38 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:38.692 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:38 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:38.692 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:38 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:01:38.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:38 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:01:38.693 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:38 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:38.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:38 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:38.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:38 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:38.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:38 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:38.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:38 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:38.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:38 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:38.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:38 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:01:38.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:38 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:01:38.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:38 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:39.543 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:39 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:39.135+0000 7fb573f51200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T02:01:39.544 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:39 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:39.231+0000 7fb573f51200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T02:01:39.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:39 smithi196 ceph-mon[34718]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:39.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:39 smithi196 ceph-mon[34718]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:01:39.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:39.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:39.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T02:01:39.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T02:01:39.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:39.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:39 smithi019 ceph-mon[30151]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:39.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:39 smithi019 ceph-mon[30151]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:01:39.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:39.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:39.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T02:01:39.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T02:01:39.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:39.715 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:39 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:39.365+0000 7fc48c5e4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T02:01:39.715 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:39 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:39.541+0000 7fc48c5e4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T02:01:39.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:39 smithi059 ceph-mon[34781]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:39.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:39 smithi059 ceph-mon[34781]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:01:39.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:39.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:39.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:39.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T02:01:39.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T02:01:39.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:39.956 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:39 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:39.602+0000 7fb573f51200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T02:01:39.956 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:39 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:39.953+0000 7fb573f51200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T02:01:40.017 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:39 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:39.718+0000 7fc48c5e4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T02:01:40.288 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:40 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:40.045+0000 7fb573f51200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T02:01:40.322 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:40 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:40.015+0000 7fc48c5e4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T02:01:40.323 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:40 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:40.111+0000 7fc48c5e4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T02:01:40.323 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:40 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:40.321+0000 7fc48c5e4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T02:01:40.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:40 smithi196 ceph-mon[34718]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T02:01:40.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:40 smithi196 ceph-mon[34718]: Reconfiguring daemon mgr.x on smithi019 2024-05-09T02:01:40.544 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:40 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:40.286+0000 7fb573f51200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T02:01:40.544 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:40 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:40.421+0000 7fb573f51200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T02:01:40.544 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:40 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:40.516+0000 7fb573f51200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T02:01:40.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:40 smithi059 ceph-mon[34781]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T02:01:40.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:40 smithi059 ceph-mon[34781]: Reconfiguring daemon mgr.x on smithi019 2024-05-09T02:01:40.716 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:40 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:40.520+0000 7fc48c5e4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T02:01:40.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:40 smithi019 ceph-mon[30151]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T02:01:40.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:40 smithi019 ceph-mon[30151]: Reconfiguring daemon mgr.x on smithi019 2024-05-09T02:01:41.293 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:40 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:40.845+0000 7fb573f51200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T02:01:41.294 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:40 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:40.940+0000 7fb573f51200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T02:01:41.465 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:41 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:41.150+0000 7fc48c5e4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T02:01:41.465 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:41 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:41.379+0000 7fc48c5e4200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T02:01:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:41 smithi059 ceph-mon[34781]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:41 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3803112163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "996c6a3d-3e3a-451c-8789-36006e0b3dce"}]: dispatch 2024-05-09T02:01:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:41 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3803112163' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "996c6a3d-3e3a-451c-8789-36006e0b3dce"}]': finished 2024-05-09T02:01:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:41 smithi059 ceph-mon[34781]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T02:01:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:41 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:01:41.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:41 smithi019 ceph-mon[30151]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:41.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:41 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3803112163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "996c6a3d-3e3a-451c-8789-36006e0b3dce"}]: dispatch 2024-05-09T02:01:41.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:41 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3803112163' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "996c6a3d-3e3a-451c-8789-36006e0b3dce"}]': finished 2024-05-09T02:01:41.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:41 smithi019 ceph-mon[30151]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T02:01:41.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:41 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:01:41.758 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:41 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:41.539+0000 7fb573f51200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T02:01:41.758 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:01:41 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:01:41.755+0000 7fb573f51200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T02:01:41.758 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:41 smithi196 ceph-mon[34718]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:41.758 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:41 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3803112163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "996c6a3d-3e3a-451c-8789-36006e0b3dce"}]: dispatch 2024-05-09T02:01:41.758 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:41 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3803112163' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "996c6a3d-3e3a-451c-8789-36006e0b3dce"}]': finished 2024-05-09T02:01:41.758 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:41 smithi196 ceph-mon[34718]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T02:01:41.758 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:41 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:01:42.215 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:41 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:41.815+0000 7fc48c5e4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T02:01:42.215 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:41 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:41.914+0000 7fc48c5e4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T02:01:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:42.379+0000 7f8495ffb700 1 -- 172.21.15.19:0/1186494465 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f848c009d10 con 0x7f8498107480 2024-05-09T02:01:42.671 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:42 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:42.297+0000 7fc48c5e4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T02:01:42.671 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:42 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:42.669+0000 7fc48c5e4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T02:01:42.671 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:42 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3322498868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T02:01:42.671 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:42 smithi059 ceph-mon[34781]: Standby manager daemon z started 2024-05-09T02:01:42.671 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:42 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.196:0/4196725349' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T02:01:42.671 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:42 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.196:0/4196725349' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T02:01:42.672 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:42 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.196:0/4196725349' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T02:01:42.672 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:42 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.196:0/4196725349' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T02:01:42.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:42 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3322498868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T02:01:42.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:42 smithi196 ceph-mon[34718]: Standby manager daemon z started 2024-05-09T02:01:42.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:42 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.196:0/4196725349' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T02:01:42.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:42 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.196:0/4196725349' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T02:01:42.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:42 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.196:0/4196725349' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T02:01:42.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:42 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.196:0/4196725349' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T02:01:42.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:42 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3322498868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T02:01:42.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:42 smithi019 ceph-mon[30151]: Standby manager daemon z started 2024-05-09T02:01:42.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:42 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.196:0/4196725349' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T02:01:42.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:42 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.196:0/4196725349' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T02:01:42.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:42 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.196:0/4196725349' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T02:01:42.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:42 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.196:0/4196725349' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T02:01:42.965 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:42 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:42.765+0000 7fc48c5e4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T02:01:43.377 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:43 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:43.017+0000 7fc48c5e4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T02:01:43.377 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:43 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:43.155+0000 7fc48c5e4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T02:01:43.377 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:43 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:43.254+0000 7fc48c5e4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T02:01:43.690 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:43 smithi059 ceph-mon[34781]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:43.690 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:43 smithi059 ceph-mon[34781]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-09T02:01:43.690 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T02:01:43.691 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:43 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:43.594+0000 7fc48c5e4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T02:01:43.691 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:43 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:43.689+0000 7fc48c5e4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T02:01:43.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:43 smithi196 ceph-mon[34718]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:43.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:43 smithi196 ceph-mon[34718]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-09T02:01:43.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T02:01:43.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:43 smithi019 ceph-mon[30151]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:43.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:43 smithi019 ceph-mon[30151]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-09T02:01:43.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T02:01:44.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:44 smithi059 ceph-mon[34781]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:44.720 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:44 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:44.317+0000 7fc48c5e4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T02:01:44.720 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:01:44 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:01:44.541+0000 7fc48c5e4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T02:01:44.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:44 smithi196 ceph-mon[34718]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:44.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:44 smithi019 ceph-mon[30151]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:45.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:45.440+0000 7f8495ffb700 1 -- 172.21.15.19:0/1186494465 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f848c02ace0 con 0x7f8498107480 2024-05-09T02:01:45.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:45 smithi019 ceph-mon[30151]: Standby manager daemon y started 2024-05-09T02:01:45.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:45 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.59:0/1832924726' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T02:01:45.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:45 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.59:0/1832924726' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T02:01:45.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:45 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.59:0/1832924726' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T02:01:45.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:45 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.59:0/1832924726' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T02:01:45.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:45 smithi059 ceph-mon[34781]: Standby manager daemon y started 2024-05-09T02:01:45.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:45 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.59:0/1832924726' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T02:01:45.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:45 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.59:0/1832924726' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T02:01:45.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:45 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.59:0/1832924726' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T02:01:45.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:45 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.59:0/1832924726' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T02:01:45.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:45 smithi196 ceph-mon[34718]: Standby manager daemon y started 2024-05-09T02:01:45.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:45 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.59:0/1832924726' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T02:01:45.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:45 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.59:0/1832924726' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T02:01:45.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:45 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.59:0/1832924726' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T02:01:45.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:45 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.59:0/1832924726' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T02:01:46.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:46 smithi019 ceph-mon[30151]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-09T02:01:46.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:46 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T02:01:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:46 smithi019 ceph-mon[30151]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:46.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:46 smithi059 ceph-mon[34781]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-09T02:01:46.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:46 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T02:01:46.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:46 smithi059 ceph-mon[34781]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:46.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:46 smithi196 ceph-mon[34718]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-09T02:01:46.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:46 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T02:01:46.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:46 smithi196 ceph-mon[34718]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:47.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:47.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:47.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:47.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:47.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:47.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:47.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:47 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:47.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:47 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:47.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:47 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:47.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:47 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:47.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:47 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:47.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:47 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:47.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:47 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:47.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:47 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:47.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:47 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:47.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:47 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:47.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:47 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:47.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:47 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:48.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:48 smithi019 ceph-mon[30151]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:48.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:48 smithi059 ceph-mon[34781]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:49.043 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:48 smithi196 ceph-mon[34718]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:49.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:49 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T02:01:49.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:49 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:49.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:49 smithi019 ceph-mon[30151]: Deploying daemon osd.0 on smithi019 2024-05-09T02:01:49.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:49 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T02:01:49.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:49 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:49.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:49 smithi059 ceph-mon[34781]: Deploying daemon osd.0 on smithi019 2024-05-09T02:01:50.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:49 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T02:01:50.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:49 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:50.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:49 smithi196 ceph-mon[34718]: Deploying daemon osd.0 on smithi019 2024-05-09T02:01:50.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:50 smithi019 ceph-mon[30151]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:50.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:50 smithi059 ceph-mon[34781]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:51.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:50 smithi196 ceph-mon[34718]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:53.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:52 smithi196 ceph-mon[34718]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:53.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:52 smithi019 ceph-mon[30151]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:53.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:52 smithi059 ceph-mon[34781]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:55.266 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:54 smithi019 ceph-mon[30151]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:55.267 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:54 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:55.267 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:54 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:55.267 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:54 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:55.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:54 smithi196 ceph-mon[34718]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:55.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:54 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:55.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:54 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:55.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:54 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:55.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:54 smithi059 ceph-mon[34781]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:55.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:54 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:55.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:54 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:55.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:54 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:57.281 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:56 smithi019 ceph-mon[30151]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:57.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:56 smithi196 ceph-mon[34718]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:57.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:56 smithi059 ceph-mon[34781]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:57.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:57.954+0000 7f8495ffb700 1 -- 172.21.15.19:0/1186494465 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f849802ea00 con 0x7f848003e750 2024-05-09T02:01:57.957 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 0 on host 'smithi019' 2024-05-09T02:01:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:57.957+0000 7f849f1da700 1 -- 172.21.15.19:0/1186494465 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f848003e750 msgr2=0x7f8480040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:57.957+0000 7f849f1da700 1 --2- 172.21.15.19:0/1186494465 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f848003e750 0x7f8480040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8488009c60 tx=0x7f8488009260 comp rx=0 tx=0).stop 2024-05-09T02:01:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:57.957+0000 7f849f1da700 1 -- 172.21.15.19:0/1186494465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8498107480 msgr2=0x7f8498102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:01:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:57.957+0000 7f849f1da700 1 --2- 172.21.15.19:0/1186494465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8498107480 0x7f8498102db0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f848c00b760 tx=0x7f848c009580 comp rx=0 tx=0).stop 2024-05-09T02:01:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:57.957+0000 7f849f1da700 1 -- 172.21.15.19:0/1186494465 shutdown_connections 2024-05-09T02:01:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:57.957+0000 7f849f1da700 1 --2- 172.21.15.19:0/1186494465 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f848003e750 0x7f8480040c60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:57.957+0000 7f849f1da700 1 --2- 172.21.15.19:0/1186494465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8498109ff0 0x7f84981032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:57.957+0000 7f849f1da700 1 --2- 172.21.15.19:0/1186494465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f849806c2d0 0x7f8498104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:57.957+0000 7f849f1da700 1 --2- 172.21.15.19:0/1186494465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8498107480 0x7f8498102db0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:01:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:57.957+0000 7f849f1da700 1 -- 172.21.15.19:0/1186494465 >> 172.21.15.19:0/1186494465 conn(0x7f84980fe880 msgr2=0x7f8498105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:01:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:57.957+0000 7f849f1da700 1 -- 172.21.15.19:0/1186494465 shutdown_connections 2024-05-09T02:01:57.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:01:57.957+0000 7f849f1da700 1 -- 172.21.15.19:0/1186494465 wait complete. 2024-05-09T02:01:58.530 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:58 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:58.530 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:58 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:58.530 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:58 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:58.531 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:58 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:58.531 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:58 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:58.531 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:58 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:58.531 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:58 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:58.531 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:58 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:58.558 DEBUG:teuthology.orchestra.run.smithi019:osd.0> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.0.service 2024-05-09T02:01:58.560 INFO:tasks.cephadm:Deploying osd.1 on smithi059 with /dev/vg_nvme/lv_4... 2024-05-09T02:01:58.560 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T02:01:58.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:58 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:58.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:58 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:58.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:58 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:58.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:58 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:58.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:58 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:58.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:58 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:58 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:58 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:58.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:58 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:58.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:58 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:58.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:58 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:01:58.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:58 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:01:58.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:58 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:58.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:58 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:58.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:58 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:01:58.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:58 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:01:58.815 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T02:01:59.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:59 smithi059 ceph-mon[34781]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:59.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:01:59 smithi059 ceph-mon[34781]: from='osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T02:01:59.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:59 smithi019 ceph-mon[30151]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:59.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:01:59 smithi019 ceph-mon[30151]: from='osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T02:01:59.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:59 smithi196 ceph-mon[34718]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:01:59.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:01:59 smithi196 ceph-mon[34718]: from='osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T02:02:00.635 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:00 smithi059 ceph-mon[34781]: from='osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T02:02:00.635 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:00 smithi059 ceph-mon[34781]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T02:02:00.635 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:00 smithi059 ceph-mon[34781]: from='osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T02:02:00.635 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:00 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:02:00.635 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:00 smithi059 ceph-mon[34781]: from='osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T02:02:00.635 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T02:02:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T02:02:00.636 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-05-09T02:02:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-05-09T02:02:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126347 s, 830 MB/s 2024-05-09T02:02:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-05-09T02:02:00.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:00 smithi196 ceph-mon[34718]: from='osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T02:02:00.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:00 smithi196 ceph-mon[34718]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T02:02:00.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:00 smithi196 ceph-mon[34718]: from='osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T02:02:00.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:00 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:02:00.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:00 smithi196 ceph-mon[34718]: from='osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T02:02:00.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:00 smithi019 ceph-mon[30151]: from='osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T02:02:00.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:00 smithi019 ceph-mon[30151]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T02:02:00.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:00 smithi019 ceph-mon[30151]: from='osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T02:02:00.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:00 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:02:00.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:00 smithi019 ceph-mon[30151]: from='osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T02:02:01.200 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_4 2024-05-09T02:02:01.321 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:02:00 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-0[42587]: 2024-05-09T02:02:00.920+0000 7fc59311d700 -1 osd.0 0 waiting for initial osdmap 2024-05-09T02:02:01.321 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:02:00 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-0[42587]: 2024-05-09T02:02:00.923+0000 7fc58a6c9700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T02:02:01.454 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T02:02:01.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:01 smithi059 ceph-mon[34781]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:02:01.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:01 smithi059 ceph-mon[34781]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T02:02:01.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:01 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:02:01.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:01 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:02:01.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:01 smithi059 ceph-mon[34781]: from='osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576]' entity='osd.0' 2024-05-09T02:02:01.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:01 smithi019 ceph-mon[30151]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:02:01.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:01 smithi019 ceph-mon[30151]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T02:02:01.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:01 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:02:01.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:01 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:02:01.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:01 smithi019 ceph-mon[30151]: from='osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576]' entity='osd.0' 2024-05-09T02:02:01.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:01 smithi196 ceph-mon[34718]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:02:01.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:01 smithi196 ceph-mon[34718]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T02:02:01.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:01 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:02:01.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:01 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:02:01.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:01 smithi196 ceph-mon[34718]: from='osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576]' entity='osd.0' 2024-05-09T02:02:02.485 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:02 smithi059 ceph-mon[34781]: purged_snaps scrub starts 2024-05-09T02:02:02.485 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:02 smithi059 ceph-mon[34781]: purged_snaps scrub ok 2024-05-09T02:02:02.485 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:02:02.485 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:02.485 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:02.485 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:02:02.485 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:02.485 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:02.485 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:02:02.485 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:02.485 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:02 smithi059 ceph-mon[34781]: osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576] boot 2024-05-09T02:02:02.485 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:02 smithi059 ceph-mon[34781]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T02:02:02.485 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:02:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:02 smithi196 ceph-mon[34718]: purged_snaps scrub starts 2024-05-09T02:02:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:02 smithi196 ceph-mon[34718]: purged_snaps scrub ok 2024-05-09T02:02:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:02:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:02:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:02:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:02 smithi196 ceph-mon[34718]: osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576] boot 2024-05-09T02:02:02.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:02 smithi196 ceph-mon[34718]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T02:02:02.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:02:02.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:02 smithi019 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T02:02:02.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:02 smithi019 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T02:02:02.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:02:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:02:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:02:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:02 smithi019 ceph-mon[30151]: osd.0 [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576] boot 2024-05-09T02:02:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:02 smithi019 ceph-mon[30151]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T02:02:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:02:03.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.010+0000 7f9f298bc700 1 -- 172.21.15.59:0/500147320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f24107480 msgr2=0x7f9f24109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:02:03.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.010+0000 7f9f298bc700 1 --2- 172.21.15.59:0/500147320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f24107480 0x7f9f24109960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009990 tx=0x7f9f18009ca0 comp rx=0 tx=0).stop 2024-05-09T02:02:03.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.010+0000 7f9f298bc700 1 -- 172.21.15.59:0/500147320 shutdown_connections 2024-05-09T02:02:03.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.010+0000 7f9f298bc700 1 --2- 172.21.15.59:0/500147320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f24109ff0 0x7f9f2410c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:03.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.010+0000 7f9f298bc700 1 --2- 172.21.15.59:0/500147320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f2406c2d0 0x7f9f24106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:03.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.010+0000 7f9f298bc700 1 --2- 172.21.15.59:0/500147320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f24107480 0x7f9f24109960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:03.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.010+0000 7f9f298bc700 1 -- 172.21.15.59:0/500147320 >> 172.21.15.59:0/500147320 conn(0x7f9f240fe880 msgr2=0x7f9f24100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:02:03.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.010+0000 7f9f298bc700 1 -- 172.21.15.59:0/500147320 shutdown_connections 2024-05-09T02:02:03.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.010+0000 7f9f298bc700 1 -- 172.21.15.59:0/500147320 wait complete. 2024-05-09T02:02:03.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.011+0000 7f9f298bc700 1 Processor -- start 2024-05-09T02:02:03.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.011+0000 7f9f298bc700 1 -- start start 2024-05-09T02:02:03.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.011+0000 7f9f298bc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f2406c2d0 0x7f9f2419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:02:03.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.011+0000 7f9f298bc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f24107480 0x7f9f2419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:02:03.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.011+0000 7f9f298bc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f24109ff0 0x7f9f241a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:02:03.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.011+0000 7f9f298bc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2410fd80 con 0x7f9f24109ff0 2024-05-09T02:02:03.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.011+0000 7f9f298bc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2410fc00 con 0x7f9f24107480 2024-05-09T02:02:03.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.011+0000 7f9f298bc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2410ff00 con 0x7f9f2406c2d0 2024-05-09T02:02:03.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f227fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f24107480 0x7f9f2419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:02:03.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f227fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f24107480 0x7f9f2419ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40510/0 (socket says 172.21.15.59:40510) 2024-05-09T02:02:03.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f227fc700 1 -- 172.21.15.59:0/2256389412 learned_addr learned my addr 172.21.15.59:0/2256389412 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-09T02:02:03.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f237fe700 1 --2- 172.21.15.59:0/2256389412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f24109ff0 0x7f9f241a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:02:03.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f22ffd700 1 --2- 172.21.15.59:0/2256389412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f2406c2d0 0x7f9f2419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:02:03.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f227fc700 1 -- 172.21.15.59:0/2256389412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f2406c2d0 msgr2=0x7f9f2419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:02:03.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f227fc700 1 --2- 172.21.15.59:0/2256389412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f2406c2d0 0x7f9f2419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:03.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f227fc700 1 -- 172.21.15.59:0/2256389412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f24109ff0 msgr2=0x7f9f241a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:02:03.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f227fc700 1 --2- 172.21.15.59:0/2256389412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f24109ff0 0x7f9f241a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:03.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f227fc700 1 -- 172.21.15.59:0/2256389412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f241a2400 con 0x7f9f24107480 2024-05-09T02:02:03.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f227fc700 1 --2- 172.21.15.59:0/2256389412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f24107480 0x7f9f2419ade0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f18006010 tx=0x7f9f180047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:02:03.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f288ba700 1 -- 172.21.15.59:0/2256389412 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18027070 con 0x7f9f24107480 2024-05-09T02:02:03.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f288ba700 1 -- 172.21.15.59:0/2256389412 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f180040c0 con 0x7f9f24107480 2024-05-09T02:02:03.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f298bc700 1 -- 172.21.15.59:0/2256389412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f241a2690 con 0x7f9f24107480 2024-05-09T02:02:03.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f288ba700 1 -- 172.21.15.59:0/2256389412 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18022d90 con 0x7f9f24107480 2024-05-09T02:02:03.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.012+0000 7f9f298bc700 1 -- 172.21.15.59:0/2256389412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f241a2ae0 con 0x7f9f24107480 2024-05-09T02:02:03.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.013+0000 7f9f298bc700 1 -- 172.21.15.59:0/2256389412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f240512c0 con 0x7f9f24107480 2024-05-09T02:02:03.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.016+0000 7f9f288ba700 1 -- 172.21.15.59:0/2256389412 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f9f1802a070 con 0x7f9f24107480 2024-05-09T02:02:03.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.016+0000 7f9f288ba700 1 --2- 172.21.15.59:0/2256389412 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f9f0c0b27f0 0x7f9f0c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:02:03.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.017+0000 7f9f288ba700 1 -- 172.21.15.59:0/2256389412 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f9f180e1900 con 0x7f9f24107480 2024-05-09T02:02:03.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.017+0000 7f9f22ffd700 1 --2- 172.21.15.59:0/2256389412 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f9f0c0b27f0 0x7f9f0c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:02:03.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.017+0000 7f9f22ffd700 1 --2- 172.21.15.59:0/2256389412 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f9f0c0b27f0 0x7f9f0c0b4d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9f14005950 tx=0x7f9f140058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:02:03.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.021+0000 7f9f288ba700 1 -- 172.21.15.59:0/2256389412 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9f18013070 con 0x7f9f24107480 2024-05-09T02:02:03.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:03.244+0000 7f9f298bc700 1 -- 172.21.15.59:0/2256389412 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9f24063cc0 con 0x7f9f0c0b27f0 2024-05-09T02:02:03.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:03 smithi059 ceph-mon[34781]: Detected new or changed devices on smithi019 2024-05-09T02:02:03.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:03 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi019 to 17093M 2024-05-09T02:02:03.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:03 smithi059 ceph-mon[34781]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:02:03.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:03 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:02:03.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:03 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:02:03.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:03 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:03.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:03 smithi196 ceph-mon[34718]: Detected new or changed devices on smithi019 2024-05-09T02:02:03.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:03 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi019 to 17093M 2024-05-09T02:02:03.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:03 smithi196 ceph-mon[34718]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:02:03.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:03 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:02:03.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:03 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:02:03.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:03 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:03.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:03 smithi019 ceph-mon[30151]: Detected new or changed devices on smithi019 2024-05-09T02:02:03.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:03 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi019 to 17093M 2024-05-09T02:02:03.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:03 smithi019 ceph-mon[30151]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T02:02:03.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:03 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:02:03.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:03 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:02:03.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:03 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:04.400 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:04 smithi059 ceph-mon[34781]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:02:04.400 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:04 smithi059 ceph-mon[34781]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T02:02:04.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:04 smithi196 ceph-mon[34718]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:02:04.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:04 smithi196 ceph-mon[34718]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T02:02:04.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:04 smithi019 ceph-mon[30151]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:02:04.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:04 smithi019 ceph-mon[30151]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T02:02:05.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:05 smithi059 ceph-mon[34781]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:05.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:05 smithi196 ceph-mon[34718]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:05.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:05 smithi019 ceph-mon[30151]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:06.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:06 smithi059 ceph-mon[34781]: from='client.? 172.21.15.59:0/295306353' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac68d09-1198-4e81-86c7-e37f7c41aa5f"}]: dispatch 2024-05-09T02:02:06.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:06 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac68d09-1198-4e81-86c7-e37f7c41aa5f"}]: dispatch 2024-05-09T02:02:06.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:06 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ac68d09-1198-4e81-86c7-e37f7c41aa5f"}]': finished 2024-05-09T02:02:06.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:06 smithi059 ceph-mon[34781]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T02:02:06.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:06 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:06.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:06 smithi059 ceph-mon[34781]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:06.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:06 smithi196 ceph-mon[34718]: from='client.? 172.21.15.59:0/295306353' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac68d09-1198-4e81-86c7-e37f7c41aa5f"}]: dispatch 2024-05-09T02:02:06.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:06 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac68d09-1198-4e81-86c7-e37f7c41aa5f"}]: dispatch 2024-05-09T02:02:06.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:06 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ac68d09-1198-4e81-86c7-e37f7c41aa5f"}]': finished 2024-05-09T02:02:06.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:06 smithi196 ceph-mon[34718]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T02:02:06.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:06 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:06.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:06 smithi196 ceph-mon[34718]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:06.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:06 smithi019 ceph-mon[30151]: from='client.? 172.21.15.59:0/295306353' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac68d09-1198-4e81-86c7-e37f7c41aa5f"}]: dispatch 2024-05-09T02:02:06.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:06 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac68d09-1198-4e81-86c7-e37f7c41aa5f"}]: dispatch 2024-05-09T02:02:06.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:06 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ac68d09-1198-4e81-86c7-e37f7c41aa5f"}]': finished 2024-05-09T02:02:06.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:06 smithi019 ceph-mon[30151]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T02:02:06.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:06 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:06.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:06 smithi019 ceph-mon[30151]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:07.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:07 smithi059 ceph-mon[34781]: from='client.? 172.21.15.59:0/129981033' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T02:02:07.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:07 smithi196 ceph-mon[34718]: from='client.? 172.21.15.59:0/129981033' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T02:02:07.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:07 smithi019 ceph-mon[30151]: from='client.? 172.21.15.59:0/129981033' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T02:02:08.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:08 smithi059 ceph-mon[34781]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:08.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:08 smithi196 ceph-mon[34718]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:08.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:08 smithi019 ceph-mon[30151]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:11.127 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:10 smithi059 ceph-mon[34781]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:11.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:10 smithi196 ceph-mon[34718]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:11.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:10 smithi019 ceph-mon[30151]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:13.215 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:12 smithi059 ceph-mon[34781]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:13.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:12 smithi196 ceph-mon[34718]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:13.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:12 smithi019 ceph-mon[30151]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:14.047 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:14 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T02:02:14.047 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:14 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:14.047 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:14 smithi059 ceph-mon[34781]: Deploying daemon osd.1 on smithi059 2024-05-09T02:02:14.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T02:02:14.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:14.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:14 smithi196 ceph-mon[34718]: Deploying daemon osd.1 on smithi059 2024-05-09T02:02:14.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T02:02:14.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:14.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:14 smithi019 ceph-mon[30151]: Deploying daemon osd.1 on smithi059 2024-05-09T02:02:15.127 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:15 smithi059 ceph-mon[34781]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:15.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:15 smithi196 ceph-mon[34718]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:15.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:15 smithi019 ceph-mon[30151]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:17.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:17 smithi059 ceph-mon[34781]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:17.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:17 smithi196 ceph-mon[34718]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:17.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:17 smithi019 ceph-mon[30151]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:19.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:19 smithi196 ceph-mon[34718]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:19.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:19 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:19.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:19 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:19.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:19 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:02:19.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:19 smithi019 ceph-mon[30151]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:19.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:19 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:19.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:19 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:19.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:19 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:02:19.392 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:19 smithi059 ceph-mon[34781]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:19.392 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:19 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:19.392 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:19 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:19.392 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:19 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:02:21.215 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:21 smithi059 ceph-mon[34781]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:21.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:21 smithi196 ceph-mon[34718]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:21.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:21 smithi019 ceph-mon[30151]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:22.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:22.416+0000 7f9f288ba700 1 -- 172.21.15.59:0/2256389412 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9f24063cc0 con 0x7f9f0c0b27f0 2024-05-09T02:02:22.418 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 1 on host 'smithi059' 2024-05-09T02:02:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:22.418+0000 7f9f298bc700 1 -- 172.21.15.59:0/2256389412 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f9f0c0b27f0 msgr2=0x7f9f0c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:02:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:22.418+0000 7f9f298bc700 1 --2- 172.21.15.59:0/2256389412 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f9f0c0b27f0 0x7f9f0c0b4d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9f14005950 tx=0x7f9f140058e0 comp rx=0 tx=0).stop 2024-05-09T02:02:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:22.419+0000 7f9f298bc700 1 -- 172.21.15.59:0/2256389412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f24107480 msgr2=0x7f9f2419ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:02:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:22.419+0000 7f9f298bc700 1 --2- 172.21.15.59:0/2256389412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f24107480 0x7f9f2419ade0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f18006010 tx=0x7f9f180047b0 comp rx=0 tx=0).stop 2024-05-09T02:02:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:22.419+0000 7f9f298bc700 1 -- 172.21.15.59:0/2256389412 shutdown_connections 2024-05-09T02:02:22.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:22.419+0000 7f9f298bc700 1 --2- 172.21.15.59:0/2256389412 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f9f0c0b27f0 0x7f9f0c0b4d00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:22.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:22.419+0000 7f9f298bc700 1 --2- 172.21.15.59:0/2256389412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f24109ff0 0x7f9f241a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:22.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:22.419+0000 7f9f298bc700 1 --2- 172.21.15.59:0/2256389412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f2406c2d0 0x7f9f2419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:22.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:22.419+0000 7f9f298bc700 1 --2- 172.21.15.59:0/2256389412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f24107480 0x7f9f2419ade0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:22.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:22.419+0000 7f9f298bc700 1 -- 172.21.15.59:0/2256389412 >> 172.21.15.59:0/2256389412 conn(0x7f9f240fe880 msgr2=0x7f9f24105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:02:22.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:22.419+0000 7f9f298bc700 1 -- 172.21.15.59:0/2256389412 shutdown_connections 2024-05-09T02:02:22.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:02:22.419+0000 7f9f298bc700 1 -- 172.21.15.59:0/2256389412 wait complete. 2024-05-09T02:02:22.645 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:22 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:22.645 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:22 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:22.645 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:22 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:22.645 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:22 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:02:22.645 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:22 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:22.645 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:22 smithi059 ceph-mon[34781]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:22.645 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:22 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:22.645 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:22 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:22.645 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:22 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:02:22.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:22 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:22.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:22 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:22.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:22 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:22.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:22 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:02:22.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:22 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:22.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:22 smithi196 ceph-mon[34718]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:22.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:22 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:22.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:22 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:22.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:22 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:02:22.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:22 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:22.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:22 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:22.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:22 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:22.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:22 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:02:22.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:22 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:22.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:22 smithi019 ceph-mon[30151]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:22.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:22 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:22.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:22 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:22.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:22 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:02:23.120 DEBUG:teuthology.orchestra.run.smithi059:osd.1> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.1.service 2024-05-09T02:02:23.122 INFO:tasks.cephadm:Deploying osd.2 on smithi196 with /dev/vg_nvme/lv_4... 2024-05-09T02:02:23.123 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T02:02:23.367 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:02:23.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:23 smithi059 ceph-mon[34781]: from='osd.1 [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T02:02:23.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:23 smithi059 ceph-mon[34781]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T02:02:23.775 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:23 smithi196 ceph-mon[34718]: from='osd.1 [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T02:02:23.775 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:23 smithi196 ceph-mon[34718]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T02:02:23.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:23 smithi019 ceph-mon[30151]: from='osd.1 [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T02:02:23.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:23 smithi019 ceph-mon[30151]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T02:02:24.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:24 smithi196 ceph-mon[34718]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T02:02:24.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:24 smithi196 ceph-mon[34718]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T02:02:24.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:24.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:24 smithi196 ceph-mon[34718]: from='osd.1 [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:02:24.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:24 smithi196 ceph-mon[34718]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:02:24.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:24 smithi196 ceph-mon[34718]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:24.687 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:24 smithi059 ceph-mon[34781]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T02:02:24.688 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:24 smithi059 ceph-mon[34781]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T02:02:24.688 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:24.688 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:24 smithi059 ceph-mon[34781]: from='osd.1 [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:02:24.688 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:24 smithi059 ceph-mon[34781]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:02:24.688 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:24 smithi059 ceph-mon[34781]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:24.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:24 smithi019 ceph-mon[30151]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T02:02:24.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:24 smithi019 ceph-mon[30151]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T02:02:24.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:24.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:24 smithi019 ceph-mon[30151]: from='osd.1 [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:02:24.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:24 smithi019 ceph-mon[30151]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:02:24.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:24 smithi019 ceph-mon[30151]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:24.938 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:02:24 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-1[39386]: 2024-05-09T02:02:24.932+0000 7fd173fce700 -1 osd.1 0 waiting for initial osdmap 2024-05-09T02:02:24.938 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:02:24 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-1[39386]: 2024-05-09T02:02:24.936+0000 7fd16bd7b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T02:02:25.108 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T02:02:25.109 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T02:02:25.109 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-05-09T02:02:25.109 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-05-09T02:02:25.109 INFO:teuthology.orchestra.run.smithi196.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0151756 s, 691 MB/s 2024-05-09T02:02:25.109 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-05-09T02:02:25.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:25 smithi196 ceph-mon[34718]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T02:02:25.636 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:25 smithi196 ceph-mon[34718]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T02:02:25.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:25.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:25.637 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:25 smithi196 ceph-mon[34718]: from='osd.1 ' entity='osd.1' 2024-05-09T02:02:25.664 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_4 2024-05-09T02:02:25.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:25 smithi059 ceph-mon[34781]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T02:02:25.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:25 smithi059 ceph-mon[34781]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T02:02:25.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:25.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:25.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:25 smithi059 ceph-mon[34781]: from='osd.1 ' entity='osd.1' 2024-05-09T02:02:25.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:25 smithi019 ceph-mon[30151]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T02:02:25.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:25 smithi019 ceph-mon[30151]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T02:02:25.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:25.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:25.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:25 smithi019 ceph-mon[30151]: from='osd.1 ' entity='osd.1' 2024-05-09T02:02:25.902 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:02:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: purged_snaps scrub starts 2024-05-09T02:02:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: purged_snaps scrub ok 2024-05-09T02:02:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: osd.1 [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353] boot 2024-05-09T02:02:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T02:02:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: Detected new or changed devices on smithi059 2024-05-09T02:02:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:02:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi059 to 17093M 2024-05-09T02:02:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:26.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:26.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:02:26.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:26.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T02:02:26.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T02:02:26.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:26.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: osd.1 [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353] boot 2024-05-09T02:02:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T02:02:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: Detected new or changed devices on smithi059 2024-05-09T02:02:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:02:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi059 to 17093M 2024-05-09T02:02:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:02:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:26.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: purged_snaps scrub starts 2024-05-09T02:02:26.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: purged_snaps scrub ok 2024-05-09T02:02:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T02:02:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: osd.1 [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353] boot 2024-05-09T02:02:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T02:02:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:02:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: Detected new or changed devices on smithi059 2024-05-09T02:02:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:02:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi059 to 17093M 2024-05-09T02:02:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:02:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:27.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.439+0000 7f7313d6a700 1 -- 172.21.15.196:0/2337760785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f730c103260 msgr2=0x7f730c103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:02:27.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.439+0000 7f7313d6a700 1 --2- 172.21.15.196:0/2337760785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f730c103260 0x7f730c103670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7308009990 tx=0x7f7308009ca0 comp rx=0 tx=0).stop 2024-05-09T02:02:27.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.439+0000 7f7313d6a700 1 -- 172.21.15.196:0/2337760785 shutdown_connections 2024-05-09T02:02:27.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.439+0000 7f7313d6a700 1 --2- 172.21.15.196:0/2337760785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f730c104d30 0x7f730c10b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:27.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.439+0000 7f7313d6a700 1 --2- 172.21.15.196:0/2337760785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f730c103260 0x7f730c103670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:27.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.439+0000 7f7313d6a700 1 --2- 172.21.15.196:0/2337760785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f730c104390 0x7f730c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:27.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.439+0000 7f7313d6a700 1 -- 172.21.15.196:0/2337760785 >> 172.21.15.196:0/2337760785 conn(0x7f730c0fe8c0 msgr2=0x7f730c100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:02:27.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.439+0000 7f7313d6a700 1 -- 172.21.15.196:0/2337760785 shutdown_connections 2024-05-09T02:02:27.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.439+0000 7f7313d6a700 1 -- 172.21.15.196:0/2337760785 wait complete. 2024-05-09T02:02:27.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.440+0000 7f7313d6a700 1 Processor -- start 2024-05-09T02:02:27.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.440+0000 7f7313d6a700 1 -- start start 2024-05-09T02:02:27.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.440+0000 7f7313d6a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f730c103260 0x7f730c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:02:27.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.440+0000 7f7313d6a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f730c104390 0x7f730c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:02:27.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.440+0000 7f7313d6a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f730c104d30 0x7f730c1a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:02:27.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.440+0000 7f7313d6a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f730c10ed20 con 0x7f730c104d30 2024-05-09T02:02:27.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.440+0000 7f7313d6a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f730c10eba0 con 0x7f730c103260 2024-05-09T02:02:27.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.440+0000 7f7313d6a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f730c10eea0 con 0x7f730c104390 2024-05-09T02:02:27.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.440+0000 7f7311305700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f730c104390 0x7f730c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:02:27.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.440+0000 7f7311305700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f730c104390 0x7f730c199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58126/0 (socket says 172.21.15.196:58126) 2024-05-09T02:02:27.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.440+0000 7f7311305700 1 -- 172.21.15.196:0/586211900 learned_addr learned my addr 172.21.15.196:0/586211900 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T02:02:27.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.441+0000 7f7312307700 1 --2- 172.21.15.196:0/586211900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f730c104d30 0x7f730c1a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:02:27.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.441+0000 7f7311b06700 1 --2- 172.21.15.196:0/586211900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f730c103260 0x7f730c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:02:27.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.441+0000 7f7311305700 1 -- 172.21.15.196:0/586211900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f730c103260 msgr2=0x7f730c199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:02:27.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.441+0000 7f7311305700 1 --2- 172.21.15.196:0/586211900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f730c103260 0x7f730c199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:27.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.441+0000 7f7311305700 1 -- 172.21.15.196:0/586211900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f730c104d30 msgr2=0x7f730c1a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:02:27.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.441+0000 7f7311305700 1 --2- 172.21.15.196:0/586211900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f730c104d30 0x7f730c1a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:27.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.441+0000 7f7311305700 1 -- 172.21.15.196:0/586211900 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730c1a13f0 con 0x7f730c104390 2024-05-09T02:02:27.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.441+0000 7f7311b06700 1 --2- 172.21.15.196:0/586211900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f730c103260 0x7f730c199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:02:27.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.441+0000 7f7311305700 1 --2- 172.21.15.196:0/586211900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f730c104390 0x7f730c199dd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00e9a0 tx=0x7f72fc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:02:27.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.441+0000 7f7302ffd700 1 -- 172.21.15.196:0/586211900 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc00cbd0 con 0x7f730c104390 2024-05-09T02:02:27.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.441+0000 7f7302ffd700 1 -- 172.21.15.196:0/586211900 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72fc00cd70 con 0x7f730c104390 2024-05-09T02:02:27.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.441+0000 7f7302ffd700 1 -- 172.21.15.196:0/586211900 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc018680 con 0x7f730c104390 2024-05-09T02:02:27.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.441+0000 7f7313d6a700 1 -- 172.21.15.196:0/586211900 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730c1a16e0 con 0x7f730c104390 2024-05-09T02:02:27.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.441+0000 7f7313d6a700 1 -- 172.21.15.196:0/586211900 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f730c1a1c70 con 0x7f730c104390 2024-05-09T02:02:27.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.442+0000 7f7313d6a700 1 -- 172.21.15.196:0/586211900 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f730c0512c0 con 0x7f730c104390 2024-05-09T02:02:27.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.444+0000 7f7302ffd700 1 -- 172.21.15.196:0/586211900 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f72fc010430 con 0x7f730c104390 2024-05-09T02:02:27.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.444+0000 7f7302ffd700 1 --2- 172.21.15.196:0/586211900 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f72f80b2870 0x7f72f80b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:02:27.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.444+0000 7f7302ffd700 1 -- 172.21.15.196:0/586211900 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f72fc0e1030 con 0x7f730c104390 2024-05-09T02:02:27.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.444+0000 7f7311b06700 1 --2- 172.21.15.196:0/586211900 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f72f80b2870 0x7f72f80b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:02:27.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.445+0000 7f7311b06700 1 --2- 172.21.15.196:0/586211900 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f72f80b2870 0x7f72f80b4d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f730800b2e0 tx=0x7f730800b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:02:27.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.448+0000 7f7302ffd700 1 -- 172.21.15.196:0/586211900 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f72fc0aa5a0 con 0x7f730c104390 2024-05-09T02:02:27.663 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:27.661+0000 7f7313d6a700 1 -- 172.21.15.196:0/586211900 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f730c063c00 con 0x7f72f80b2870 2024-05-09T02:02:28.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:28 smithi196 ceph-mon[34718]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T02:02:28.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:28 smithi196 ceph-mon[34718]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:02:28.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:28 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:02:28.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:28 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:02:28.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:28 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:28.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:28 smithi196 ceph-mon[34718]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:28.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:28 smithi019 ceph-mon[30151]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T02:02:28.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:28 smithi019 ceph-mon[30151]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:02:28.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:28 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:02:28.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:28 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:02:28.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:28 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:28.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:28 smithi019 ceph-mon[30151]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:28.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:28 smithi059 ceph-mon[34781]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T02:02:28.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:28 smithi059 ceph-mon[34781]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:02:28.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:28 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:02:28.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:28 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:02:28.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:28 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:28.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:28 smithi059 ceph-mon[34781]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:31.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:31 smithi196 ceph-mon[34718]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:31.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:31 smithi196 ceph-mon[34718]: from='client.? 172.21.15.196:0/4226023774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "818a0de1-04e6-471d-83df-a68833011b4a"}]: dispatch 2024-05-09T02:02:31.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:31 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "818a0de1-04e6-471d-83df-a68833011b4a"}]: dispatch 2024-05-09T02:02:31.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:31 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "818a0de1-04e6-471d-83df-a68833011b4a"}]': finished 2024-05-09T02:02:31.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:31 smithi196 ceph-mon[34718]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T02:02:31.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:31 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:31.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:31 smithi196 ceph-mon[34718]: from='client.? 172.21.15.196:0/550480649' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T02:02:31.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:31 smithi019 ceph-mon[30151]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:31.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:31 smithi019 ceph-mon[30151]: from='client.? 172.21.15.196:0/4226023774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "818a0de1-04e6-471d-83df-a68833011b4a"}]: dispatch 2024-05-09T02:02:31.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:31 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "818a0de1-04e6-471d-83df-a68833011b4a"}]: dispatch 2024-05-09T02:02:31.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:31 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "818a0de1-04e6-471d-83df-a68833011b4a"}]': finished 2024-05-09T02:02:31.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:31 smithi019 ceph-mon[30151]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T02:02:31.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:31 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:31.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:31 smithi019 ceph-mon[30151]: from='client.? 172.21.15.196:0/550480649' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T02:02:31.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:31 smithi059 ceph-mon[34781]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:31.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:31 smithi059 ceph-mon[34781]: from='client.? 172.21.15.196:0/4226023774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "818a0de1-04e6-471d-83df-a68833011b4a"}]: dispatch 2024-05-09T02:02:31.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:31 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "818a0de1-04e6-471d-83df-a68833011b4a"}]: dispatch 2024-05-09T02:02:31.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:31 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "818a0de1-04e6-471d-83df-a68833011b4a"}]': finished 2024-05-09T02:02:31.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:31 smithi059 ceph-mon[34781]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T02:02:31.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:31 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:31.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:31 smithi059 ceph-mon[34781]: from='client.? 172.21.15.196:0/550480649' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T02:02:33.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:33 smithi196 ceph-mon[34718]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:33.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:33 smithi019 ceph-mon[30151]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:33.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:33 smithi059 ceph-mon[34781]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:35.268 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:35 smithi059 ceph-mon[34781]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:35.280 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:35 smithi196 ceph-mon[34718]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:35.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:35 smithi019 ceph-mon[30151]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:37.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:37 smithi196 ceph-mon[34718]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:37.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:37 smithi019 ceph-mon[30151]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:37.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:37 smithi059 ceph-mon[34781]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:38.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:38 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T02:02:38.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:38 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:38.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:38 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T02:02:38.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:38 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:38.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:38 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T02:02:38.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:38 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:39.204 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:39 smithi196 ceph-mon[34718]: Deploying daemon osd.2 on smithi196 2024-05-09T02:02:39.204 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:39 smithi196 ceph-mon[34718]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:39.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:39 smithi019 ceph-mon[30151]: Deploying daemon osd.2 on smithi196 2024-05-09T02:02:39.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:39 smithi019 ceph-mon[30151]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:39.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:39 smithi059 ceph-mon[34781]: Deploying daemon osd.2 on smithi196 2024-05-09T02:02:39.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:39 smithi059 ceph-mon[34781]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:41.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:41 smithi196 ceph-mon[34718]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:41.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:41 smithi019 ceph-mon[30151]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:41.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:41 smithi059 ceph-mon[34781]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:43.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:43 smithi196 ceph-mon[34718]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:43.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:02:43.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:43.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:43.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:43 smithi019 ceph-mon[30151]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:43.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:02:43.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:43.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:43.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:43 smithi059 ceph-mon[34781]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:02:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:45.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:45 smithi196 ceph-mon[34718]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:45.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:45 smithi019 ceph-mon[30151]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:45.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:45 smithi059 ceph-mon[34781]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:46.082 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:46.080+0000 7f7302ffd700 1 -- 172.21.15.196:0/586211900 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f730c063c00 con 0x7f72f80b2870 2024-05-09T02:02:46.082 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 2 on host 'smithi196' 2024-05-09T02:02:46.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:46.083+0000 7f7313d6a700 1 -- 172.21.15.196:0/586211900 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f72f80b2870 msgr2=0x7f72f80b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:02:46.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:46.083+0000 7f7313d6a700 1 --2- 172.21.15.196:0/586211900 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f72f80b2870 0x7f72f80b4d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f730800b2e0 tx=0x7f730800b720 comp rx=0 tx=0).stop 2024-05-09T02:02:46.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:46.083+0000 7f7313d6a700 1 -- 172.21.15.196:0/586211900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f730c104390 msgr2=0x7f730c199dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:02:46.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:46.083+0000 7f7313d6a700 1 --2- 172.21.15.196:0/586211900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f730c104390 0x7f730c199dd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00e9a0 tx=0x7f72fc00ecb0 comp rx=0 tx=0).stop 2024-05-09T02:02:46.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:46.083+0000 7f7313d6a700 1 -- 172.21.15.196:0/586211900 shutdown_connections 2024-05-09T02:02:46.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:46.083+0000 7f7313d6a700 1 --2- 172.21.15.196:0/586211900 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f72f80b2870 0x7f72f80b4d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:46.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:46.083+0000 7f7313d6a700 1 --2- 172.21.15.196:0/586211900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f730c104d30 0x7f730c1a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:46.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:46.083+0000 7f7313d6a700 1 --2- 172.21.15.196:0/586211900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f730c103260 0x7f730c199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:46.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:46.083+0000 7f7313d6a700 1 --2- 172.21.15.196:0/586211900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f730c104390 0x7f730c199dd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:46.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:46.083+0000 7f7313d6a700 1 -- 172.21.15.196:0/586211900 >> 172.21.15.196:0/586211900 conn(0x7f730c0fe8c0 msgr2=0x7f730c1075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:02:46.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:46.083+0000 7f7313d6a700 1 -- 172.21.15.196:0/586211900 shutdown_connections 2024-05-09T02:02:46.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:02:46.083+0000 7f7313d6a700 1 -- 172.21.15.196:0/586211900 wait complete. 2024-05-09T02:02:46.769 DEBUG:teuthology.orchestra.run.smithi196:osd.2> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.2.service 2024-05-09T02:02:46.771 INFO:tasks.cephadm:Deploying osd.3 on smithi019 with /dev/vg_nvme/lv_3... 2024-05-09T02:02:46.771 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T02:02:47.029 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:02:47.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:47 smithi019 ceph-mon[30151]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:47.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:47.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:47.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:47.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:47.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:47 smithi019 ceph-mon[30151]: from='osd.2 [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T02:02:47.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:47 smithi019 ceph-mon[30151]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T02:02:47.256 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:47 smithi196 ceph-mon[34718]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:47.256 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:47 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:47.256 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:47 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:47.256 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:47 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:47.256 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:47 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:47.256 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:47 smithi196 ceph-mon[34718]: from='osd.2 [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T02:02:47.256 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:47 smithi196 ceph-mon[34718]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T02:02:47.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:47 smithi059 ceph-mon[34781]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:47 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:47 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:47 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:47 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:47 smithi059 ceph-mon[34781]: from='osd.2 [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T02:02:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:47 smithi059 ceph-mon[34781]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T02:02:48.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:48 smithi196 ceph-mon[34718]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T02:02:48.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:48 smithi196 ceph-mon[34718]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T02:02:48.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:48 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:48.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:48 smithi196 ceph-mon[34718]: from='osd.2 [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:02:48.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:48 smithi196 ceph-mon[34718]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:02:48.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:48 smithi019 ceph-mon[30151]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T02:02:48.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:48 smithi019 ceph-mon[30151]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T02:02:48.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:48 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:48.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:48 smithi019 ceph-mon[30151]: from='osd.2 [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:02:48.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:48 smithi019 ceph-mon[30151]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:02:48.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:48 smithi059 ceph-mon[34781]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T02:02:48.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:48 smithi059 ceph-mon[34781]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T02:02:48.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:48 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:48.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:48 smithi059 ceph-mon[34781]: from='osd.2 [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:02:48.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:48 smithi059 ceph-mon[34781]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:02:48.882 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:02:48 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-2[39882]: 2024-05-09T02:02:48.627+0000 7f1d37067700 -1 osd.2 0 waiting for initial osdmap 2024-05-09T02:02:48.882 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:02:48 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-2[39882]: 2024-05-09T02:02:48.631+0000 7f1d2ee14700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T02:02:48.973 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T02:02:48.973 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T02:02:48.973 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-09T02:02:48.973 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-09T02:02:48.973 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123743 s, 847 MB/s 2024-05-09T02:02:48.973 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-09T02:02:49.249 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:49 smithi196 ceph-mon[34718]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:49.249 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:49 smithi196 ceph-mon[34718]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T02:02:49.249 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:49 smithi196 ceph-mon[34718]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T02:02:49.249 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:49 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:49.249 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:49 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:49.249 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:49 smithi196 ceph-mon[34718]: from='osd.2 ' entity='osd.2' 2024-05-09T02:02:49.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:49 smithi059 ceph-mon[34781]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:49.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:49 smithi059 ceph-mon[34781]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T02:02:49.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:49 smithi059 ceph-mon[34781]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T02:02:49.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:49 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:49.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:49 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:49.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:49 smithi059 ceph-mon[34781]: from='osd.2 ' entity='osd.2' 2024-05-09T02:02:49.470 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:49 smithi019 ceph-mon[30151]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:49.470 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:49 smithi019 ceph-mon[30151]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T02:02:49.470 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:49 smithi019 ceph-mon[30151]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T02:02:49.470 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:49 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:49.470 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:49 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:49.470 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:49 smithi019 ceph-mon[30151]: from='osd.2 ' entity='osd.2' 2024-05-09T02:02:49.648 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_3 2024-05-09T02:02:49.902 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:02:50.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T02:02:50.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T02:02:50.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:50.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:50.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:50.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:02:50.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:50.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:50.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:02:50.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:50.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: osd.2 [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639] boot 2024-05-09T02:02:50.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T02:02:50.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:50.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:02:50.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:50.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:02:50.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:50.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T02:02:50.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: purged_snaps scrub starts 2024-05-09T02:02:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: purged_snaps scrub ok 2024-05-09T02:02:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:02:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:02:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: osd.2 [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639] boot 2024-05-09T02:02:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T02:02:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:02:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:02:50.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:50.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: purged_snaps scrub starts 2024-05-09T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: purged_snaps scrub ok 2024-05-09T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: osd.2 [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639] boot 2024-05-09T02:02:50.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T02:02:50.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:02:50.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:02:50.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:50.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:02:50.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:50.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T02:02:51.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:51 smithi019 ceph-mon[30151]: Detected new or changed devices on smithi196 2024-05-09T02:02:51.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:51 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi196 to 17088M 2024-05-09T02:02:51.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:51 smithi019 ceph-mon[30151]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:51.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:51 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T02:02:51.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:51 smithi019 ceph-mon[30151]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T02:02:51.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:51 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T02:02:51.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:51 smithi059 ceph-mon[34781]: Detected new or changed devices on smithi196 2024-05-09T02:02:51.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:51 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi196 to 17088M 2024-05-09T02:02:51.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:51 smithi059 ceph-mon[34781]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:51.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:51 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T02:02:51.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:51 smithi059 ceph-mon[34781]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T02:02:51.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:51 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T02:02:51.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.486+0000 7ff97e03e700 1 -- 172.21.15.19:0/1536666884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff978103230 msgr2=0x7ff978103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:02:51.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.486+0000 7ff97e03e700 1 --2- 172.21.15.19:0/1536666884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff978103230 0x7ff978103640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff968009990 tx=0x7ff968009ca0 comp rx=0 tx=0).stop 2024-05-09T02:02:51.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.486+0000 7ff97e03e700 1 -- 172.21.15.19:0/1536666884 shutdown_connections 2024-05-09T02:02:51.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.486+0000 7ff97e03e700 1 --2- 172.21.15.19:0/1536666884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff978104d00 0x7ff97810b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:51.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.486+0000 7ff97e03e700 1 --2- 172.21.15.19:0/1536666884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff978103230 0x7ff978103640 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.486+0000 7ff97e03e700 1 --2- 172.21.15.19:0/1536666884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff978104360 0x7ff978104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.486+0000 7ff97e03e700 1 -- 172.21.15.19:0/1536666884 >> 172.21.15.19:0/1536666884 conn(0x7ff9780fe870 msgr2=0x7ff978100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:02:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.486+0000 7ff97e03e700 1 -- 172.21.15.19:0/1536666884 shutdown_connections 2024-05-09T02:02:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.486+0000 7ff97e03e700 1 -- 172.21.15.19:0/1536666884 wait complete. 2024-05-09T02:02:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff97e03e700 1 Processor -- start 2024-05-09T02:02:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff97e03e700 1 -- start start 2024-05-09T02:02:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff97e03e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff978103230 0x7ff97819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:02:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff97e03e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff978104360 0x7ff97819b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:02:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff97e03e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff978104d00 0x7ff9781a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:02:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff97e03e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff97810edc0 con 0x7ff978104360 2024-05-09T02:02:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff97e03e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff97810ec40 con 0x7ff978103230 2024-05-09T02:02:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff97e03e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff97810ef40 con 0x7ff978104d00 2024-05-09T02:02:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff976ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff978104360 0x7ff97819b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:02:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff976ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff978104360 0x7ff97819b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58392/0 (socket says 172.21.15.19:58392) 2024-05-09T02:02:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff976ffd700 1 -- 172.21.15.19:0/609399870 learned_addr learned my addr 172.21.15.19:0/609399870 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:02:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff9777fe700 1 --2- 172.21.15.19:0/609399870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff978103230 0x7ff97819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:02:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff977fff700 1 --2- 172.21.15.19:0/609399870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff978104d00 0x7ff9781a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:02:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff976ffd700 1 -- 172.21.15.19:0/609399870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff978104d00 msgr2=0x7ff9781a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:02:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff976ffd700 1 --2- 172.21.15.19:0/609399870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff978104d00 0x7ff9781a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff976ffd700 1 -- 172.21.15.19:0/609399870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff978103230 msgr2=0x7ff97819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:02:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff976ffd700 1 --2- 172.21.15.19:0/609399870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff978103230 0x7ff97819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:02:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.487+0000 7ff976ffd700 1 -- 172.21.15.19:0/609399870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9781a2680 con 0x7ff978104360 2024-05-09T02:02:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.488+0000 7ff976ffd700 1 --2- 172.21.15.19:0/609399870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff978104360 0x7ff97819b060 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00b530 tx=0x7ff96c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:02:51.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.488+0000 7ff974ff9700 1 -- 172.21.15.19:0/609399870 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c015020 con 0x7ff978104360 2024-05-09T02:02:51.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.488+0000 7ff97e03e700 1 -- 172.21.15.19:0/609399870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9781a2970 con 0x7ff978104360 2024-05-09T02:02:51.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.488+0000 7ff974ff9700 1 -- 172.21.15.19:0/609399870 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff96c011ce0 con 0x7ff978104360 2024-05-09T02:02:51.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.488+0000 7ff974ff9700 1 -- 172.21.15.19:0/609399870 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c010e40 con 0x7ff978104360 2024-05-09T02:02:51.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.488+0000 7ff97e03e700 1 -- 172.21.15.19:0/609399870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9781a2f30 con 0x7ff978104360 2024-05-09T02:02:51.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.489+0000 7ff97e03e700 1 -- 172.21.15.19:0/609399870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9780512c0 con 0x7ff978104360 2024-05-09T02:02:51.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.492+0000 7ff974ff9700 1 -- 172.21.15.19:0/609399870 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7ff96c018020 con 0x7ff978104360 2024-05-09T02:02:51.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.493+0000 7ff974ff9700 1 --2- 172.21.15.19:0/609399870 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff9600b27f0 0x7ff9600b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:02:51.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.493+0000 7ff9777fe700 1 --2- 172.21.15.19:0/609399870 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff9600b27f0 0x7ff9600b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:02:51.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.493+0000 7ff974ff9700 1 -- 172.21.15.19:0/609399870 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7ff96c0e0b80 con 0x7ff978104360 2024-05-09T02:02:51.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.493+0000 7ff9777fe700 1 --2- 172.21.15.19:0/609399870 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff9600b27f0 0x7ff9600b4d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff9680038a0 tx=0x7ff96801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:02:51.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.497+0000 7ff974ff9700 1 -- 172.21.15.19:0/609399870 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff96c01f020 con 0x7ff978104360 2024-05-09T02:02:51.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:51 smithi196 ceph-mon[34718]: Detected new or changed devices on smithi196 2024-05-09T02:02:51.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:51 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi196 to 17088M 2024-05-09T02:02:51.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:51 smithi196 ceph-mon[34718]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T02:02:51.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:51 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T02:02:51.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:51 smithi196 ceph-mon[34718]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T02:02:51.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:51 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T02:02:51.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:51.731+0000 7ff97e03e700 1 -- 172.21.15.19:0/609399870 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff9781a31c0 con 0x7ff9600b27f0 2024-05-09T02:02:51.741 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47270]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T02:02:51.741 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47270]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T02:02:51.741 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47270]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T02:02:51.741 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47270]: pam_unix(sudo:session): session closed for user root 2024-05-09T02:02:51.741 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47273]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T02:02:51.741 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47273]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T02:02:51.741 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47273]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T02:02:51.741 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47273]: pam_unix(sudo:session): session closed for user root 2024-05-09T02:02:52.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47343]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T02:02:52.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47343]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T02:02:52.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47343]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T02:02:52.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47343]: pam_unix(sudo:session): session closed for user root 2024-05-09T02:02:52.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47346]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T02:02:52.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47346]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T02:02:52.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47346]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T02:02:52.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:51 smithi019 sudo[47346]: pam_unix(sudo:session): session closed for user root 2024-05-09T02:02:52.215 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 sudo[43088]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T02:02:52.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 sudo[43088]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T02:02:52.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 sudo[43088]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T02:02:52.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 sudo[43088]: pam_unix(sudo:session): session closed for user root 2024-05-09T02:02:52.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 sudo[43091]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T02:02:52.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 sudo[43091]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T02:02:52.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 sudo[43091]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T02:02:52.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 sudo[43091]: pam_unix(sudo:session): session closed for user root 2024-05-09T02:02:52.216 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:02:51 smithi059 sudo[43082]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T02:02:52.216 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:02:51 smithi059 sudo[43082]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T02:02:52.216 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:02:51 smithi059 sudo[43082]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T02:02:52.216 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:02:51 smithi059 sudo[43082]: pam_unix(sudo:session): session closed for user root 2024-05-09T02:02:52.217 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:02:51 smithi059 sudo[43085]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T02:02:52.217 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:02:51 smithi059 sudo[43085]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T02:02:52.217 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:02:51 smithi059 sudo[43085]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T02:02:52.217 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:02:51 smithi059 sudo[43085]: pam_unix(sudo:session): session closed for user root 2024-05-09T02:02:52.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43528]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T02:02:52.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43528]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T02:02:52.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43528]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T02:02:52.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43528]: pam_unix(sudo:session): session closed for user root 2024-05-09T02:02:52.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43531]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T02:02:52.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43531]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T02:02:52.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43531]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T02:02:52.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43531]: pam_unix(sudo:session): session closed for user root 2024-05-09T02:02:52.294 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43522]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T02:02:52.294 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43522]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T02:02:52.294 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43522]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T02:02:52.295 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43522]: pam_unix(sudo:session): session closed for user root 2024-05-09T02:02:52.295 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43525]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T02:02:52.295 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43525]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T02:02:52.295 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43525]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T02:02:52.295 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:02:51 smithi196 sudo[43525]: pam_unix(sudo:session): session closed for user root 2024-05-09T02:02:52.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T02:02:52.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T02:02:52.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:02:52.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:02:52.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:02:52.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:52.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:52.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:52.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:52.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T02:02:52.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:02:52.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T02:02:52.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:02:52.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:02:52.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:02:52.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T02:02:52.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:02:52.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:02:52.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:02:52.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T02:02:52.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T02:02:52.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:02:52.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:02:52.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:02:52.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:52 smithi019 ceph-mon[30151]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T02:02:52.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:02:52.927+0000 7ff974ff9700 1 -- 172.21.15.19:0/609399870 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff96c0a5850 con 0x7ff978104360 2024-05-09T02:02:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T02:02:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T02:02:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:02:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:02:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:02:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T02:02:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:02:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T02:02:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:02:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:02:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:02:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T02:02:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:02:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:02:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:02:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T02:02:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T02:02:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:02:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:02:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:02:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:52 smithi059 ceph-mon[34781]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T02:02:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T02:02:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T02:02:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:02:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:02:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:02:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:02:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:02:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:02:53.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:52 smithi196 ceph-mon[34718]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T02:02:53.758 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:53 smithi019 ceph-mon[30151]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T02:02:53.758 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:53 smithi019 ceph-mon[30151]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T02:02:53.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:53 smithi059 ceph-mon[34781]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T02:02:53.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:53 smithi059 ceph-mon[34781]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T02:02:54.043 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:53 smithi196 ceph-mon[34718]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T02:02:54.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:53 smithi196 ceph-mon[34718]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T02:02:54.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:54 smithi059 ceph-mon[34781]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:02:55.043 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:54 smithi196 ceph-mon[34718]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:02:55.061 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:54 smithi019 ceph-mon[30151]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:02:55.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:55 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1131154319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b204854-86b8-454d-aee0-8bb5775ce828"}]: dispatch 2024-05-09T02:02:55.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:55 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1131154319' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b204854-86b8-454d-aee0-8bb5775ce828"}]': finished 2024-05-09T02:02:55.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:55 smithi059 ceph-mon[34781]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T02:02:55.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:02:55.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:55 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/146546029' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T02:02:56.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:55 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1131154319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b204854-86b8-454d-aee0-8bb5775ce828"}]: dispatch 2024-05-09T02:02:56.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:55 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1131154319' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b204854-86b8-454d-aee0-8bb5775ce828"}]': finished 2024-05-09T02:02:56.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:55 smithi196 ceph-mon[34718]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T02:02:56.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:02:56.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:55 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/146546029' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T02:02:56.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:55 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1131154319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b204854-86b8-454d-aee0-8bb5775ce828"}]: dispatch 2024-05-09T02:02:56.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:55 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1131154319' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b204854-86b8-454d-aee0-8bb5775ce828"}]': finished 2024-05-09T02:02:56.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:55 smithi019 ceph-mon[30151]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T02:02:56.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:02:56.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:55 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/146546029' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T02:02:56.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:56 smithi059 ceph-mon[34781]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:02:57.043 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:56 smithi196 ceph-mon[34718]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:02:57.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:56 smithi019 ceph-mon[30151]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:02:59.022 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:02:59 smithi019 ceph-mon[30151]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:02:59.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:02:59 smithi196 ceph-mon[34718]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:02:59.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:02:59 smithi059 ceph-mon[34781]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:01.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:01 smithi196 ceph-mon[34718]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:01.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:01 smithi019 ceph-mon[30151]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:01.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:01 smithi059 ceph-mon[34781]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:03.139 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:03 smithi019 ceph-mon[30151]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:03.139 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:03 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T02:03:03.139 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:03 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:03.139 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:03 smithi019 ceph-mon[30151]: Deploying daemon osd.3 on smithi019 2024-05-09T02:03:03.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:03 smithi196 ceph-mon[34718]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:03.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:03 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T02:03:03.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:03 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:03.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:03 smithi196 ceph-mon[34718]: Deploying daemon osd.3 on smithi019 2024-05-09T02:03:03.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:03 smithi059 ceph-mon[34781]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:03.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:03 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T02:03:03.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:03 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:03.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:03 smithi059 ceph-mon[34781]: Deploying daemon osd.3 on smithi019 2024-05-09T02:03:05.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:05 smithi019 ceph-mon[30151]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:05.268 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:05 smithi059 ceph-mon[34781]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:05.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:05 smithi196 ceph-mon[34718]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:07.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:07 smithi196 ceph-mon[34718]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:07.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:07 smithi019 ceph-mon[30151]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:07.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:07 smithi059 ceph-mon[34781]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:09.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:09 smithi019 ceph-mon[30151]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:09.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:09 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:09.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:09 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:03:09.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:09 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:09.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:09 smithi059 ceph-mon[34781]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:09.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:09 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:09.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:09 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:03:09.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:09 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:09.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:09 smithi196 ceph-mon[34718]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:09.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:09 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:09.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:09 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:03:09.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:09 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:11.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:11 smithi059 ceph-mon[34781]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:11.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:11 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:11.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:11 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:11.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:11 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:11.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:11 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:03:11.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:11 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:11.529 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:11 smithi019 ceph-mon[30151]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:11.529 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:11 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:11.530 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:11 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:11.530 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:11 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:11.530 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:11 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:03:11.530 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:11 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:11.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:11 smithi196 ceph-mon[34718]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:11.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:11 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:11.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:11 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:11.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:11 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:11.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:11 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:03:11.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:11 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:11.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:03:11.941+0000 7ff974ff9700 1 -- 172.21.15.19:0/609399870 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff9781a31c0 con 0x7ff9600b27f0 2024-05-09T02:03:11.943 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 3 on host 'smithi019' 2024-05-09T02:03:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:03:11.944+0000 7ff97e03e700 1 -- 172.21.15.19:0/609399870 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff9600b27f0 msgr2=0x7ff9600b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:03:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:03:11.944+0000 7ff97e03e700 1 --2- 172.21.15.19:0/609399870 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff9600b27f0 0x7ff9600b4d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff9680038a0 tx=0x7ff96801a040 comp rx=0 tx=0).stop 2024-05-09T02:03:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:03:11.944+0000 7ff97e03e700 1 -- 172.21.15.19:0/609399870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff978104360 msgr2=0x7ff97819b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:03:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:03:11.944+0000 7ff97e03e700 1 --2- 172.21.15.19:0/609399870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff978104360 0x7ff97819b060 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00b530 tx=0x7ff96c00b840 comp rx=0 tx=0).stop 2024-05-09T02:03:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:03:11.944+0000 7ff97e03e700 1 -- 172.21.15.19:0/609399870 shutdown_connections 2024-05-09T02:03:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:03:11.944+0000 7ff97e03e700 1 --2- 172.21.15.19:0/609399870 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff9600b27f0 0x7ff9600b4d00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:03:11.944+0000 7ff97e03e700 1 --2- 172.21.15.19:0/609399870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff978104d00 0x7ff9781a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:11.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:03:11.944+0000 7ff97e03e700 1 --2- 172.21.15.19:0/609399870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff978103230 0x7ff97819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:11.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:03:11.944+0000 7ff97e03e700 1 --2- 172.21.15.19:0/609399870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff978104360 0x7ff97819b060 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:11.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:03:11.944+0000 7ff97e03e700 1 -- 172.21.15.19:0/609399870 >> 172.21.15.19:0/609399870 conn(0x7ff9780fe870 msgr2=0x7ff978100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:03:11.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:03:11.944+0000 7ff97e03e700 1 -- 172.21.15.19:0/609399870 shutdown_connections 2024-05-09T02:03:11.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:03:11.944+0000 7ff97e03e700 1 -- 172.21.15.19:0/609399870 wait complete. 2024-05-09T02:03:12.661 DEBUG:teuthology.orchestra.run.smithi019:osd.3> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.3.service 2024-05-09T02:03:12.664 INFO:tasks.cephadm:Deploying osd.4 on smithi059 with /dev/vg_nvme/lv_3... 2024-05-09T02:03:12.664 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T02:03:12.920 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T02:03:12.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:12 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:12.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:12 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:12.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:12 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:03:12.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:12 smithi059 ceph-mon[34781]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:12.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:12 smithi059 ceph-mon[34781]: from='osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T02:03:13.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:12 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:13.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:12 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:13.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:12 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:03:13.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:12 smithi019 ceph-mon[30151]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:13.172 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:12 smithi019 ceph-mon[30151]: from='osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T02:03:13.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:12 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:13.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:12 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:13.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:12 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:03:13.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:12 smithi196 ceph-mon[34718]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:13.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:12 smithi196 ceph-mon[34718]: from='osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T02:03:14.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:13 smithi059 ceph-mon[34781]: from='osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T02:03:14.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:13 smithi059 ceph-mon[34781]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T02:03:14.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:13 smithi059 ceph-mon[34781]: from='osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T02:03:14.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:13 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:14.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:13 smithi019 ceph-mon[30151]: from='osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T02:03:14.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:13 smithi019 ceph-mon[30151]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T02:03:14.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:13 smithi019 ceph-mon[30151]: from='osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T02:03:14.220 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:13 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:14.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:13 smithi196 ceph-mon[34718]: from='osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T02:03:14.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:13 smithi196 ceph-mon[34718]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T02:03:14.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:13 smithi196 ceph-mon[34718]: from='osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T02:03:14.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:13 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:14.571 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:03:14 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-3[49562]: 2024-05-09T02:03:14.414+0000 7f85f11f3700 -1 osd.3 0 waiting for initial osdmap 2024-05-09T02:03:14.571 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:03:14 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-3[49562]: 2024-05-09T02:03:14.420+0000 7f85e879f700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T02:03:14.979 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T02:03:14.979 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T02:03:14.979 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-05-09T02:03:14.979 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-05-09T02:03:14.979 INFO:teuthology.orchestra.run.smithi059.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0136563 s, 768 MB/s 2024-05-09T02:03:14.979 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-05-09T02:03:15.215 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:14 smithi059 ceph-mon[34781]: from='osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T02:03:15.215 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:14 smithi059 ceph-mon[34781]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T02:03:15.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:14 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:15.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:14 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:15.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:14 smithi059 ceph-mon[34781]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:15.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:14 smithi059 ceph-mon[34781]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T02:03:15.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:14 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:15.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:14 smithi059 ceph-mon[34781]: from='osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599]' entity='osd.3' 2024-05-09T02:03:15.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:14 smithi196 ceph-mon[34718]: from='osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T02:03:15.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:14 smithi196 ceph-mon[34718]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T02:03:15.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:15.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:15.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:14 smithi196 ceph-mon[34718]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:15.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:14 smithi196 ceph-mon[34718]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T02:03:15.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:14 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:15.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:14 smithi196 ceph-mon[34718]: from='osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599]' entity='osd.3' 2024-05-09T02:03:15.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:14 smithi019 ceph-mon[30151]: from='osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T02:03:15.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:14 smithi019 ceph-mon[30151]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T02:03:15.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:15.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:15.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:14 smithi019 ceph-mon[30151]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T02:03:15.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:14 smithi019 ceph-mon[30151]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T02:03:15.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:14 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:15.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:14 smithi019 ceph-mon[30151]: from='osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599]' entity='osd.3' 2024-05-09T02:03:15.572 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_3 2024-05-09T02:03:15.825 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T02:03:16.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:15 smithi019 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T02:03:16.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:15 smithi019 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T02:03:16.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:16.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:15 smithi019 ceph-mon[30151]: osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599] boot 2024-05-09T02:03:16.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:15 smithi019 ceph-mon[30151]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T02:03:16.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:16.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:16.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:16.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:03:16.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:03:16.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:16.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:16.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:03:16.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:16.215 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:15 smithi059 ceph-mon[34781]: purged_snaps scrub starts 2024-05-09T02:03:16.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:15 smithi059 ceph-mon[34781]: purged_snaps scrub ok 2024-05-09T02:03:16.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:16.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:15 smithi059 ceph-mon[34781]: osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599] boot 2024-05-09T02:03:16.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:15 smithi059 ceph-mon[34781]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T02:03:16.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:16.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:16.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:16.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:03:16.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:03:16.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:16.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:16.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:03:16.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:16.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:15 smithi196 ceph-mon[34718]: purged_snaps scrub starts 2024-05-09T02:03:16.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:15 smithi196 ceph-mon[34718]: purged_snaps scrub ok 2024-05-09T02:03:16.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:16.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:15 smithi196 ceph-mon[34718]: osd.3 [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599] boot 2024-05-09T02:03:16.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:15 smithi196 ceph-mon[34718]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T02:03:16.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:03:16.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:16.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:16.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:03:16.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:03:16.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:16.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:16.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:03:16.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:17.407 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:17 smithi059 ceph-mon[34781]: Detected new or changed devices on smithi019 2024-05-09T02:03:17.408 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:17 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi019 to 8546M 2024-05-09T02:03:17.408 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:17 smithi059 ceph-mon[34781]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:17.408 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:17 smithi059 ceph-mon[34781]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T02:03:17.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:17 smithi196 ceph-mon[34718]: Detected new or changed devices on smithi019 2024-05-09T02:03:17.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:17 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi019 to 8546M 2024-05-09T02:03:17.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:17 smithi196 ceph-mon[34718]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:17.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:17 smithi196 ceph-mon[34718]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T02:03:17.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:17 smithi019 ceph-mon[30151]: Detected new or changed devices on smithi019 2024-05-09T02:03:17.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:17 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi019 to 8546M 2024-05-09T02:03:17.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:17 smithi019 ceph-mon[30151]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:17.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:17 smithi019 ceph-mon[30151]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T02:03:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.660+0000 7f1c746c8700 1 -- 172.21.15.59:0/2160889195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c104370 msgr2=0x7f1c6c104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:03:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.660+0000 7f1c746c8700 1 --2- 172.21.15.59:0/2160889195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c104370 0x7f1c6c104780 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009990 tx=0x7f1c5c009ca0 comp rx=0 tx=0).stop 2024-05-09T02:03:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.660+0000 7f1c746c8700 1 -- 172.21.15.59:0/2160889195 shutdown_connections 2024-05-09T02:03:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.660+0000 7f1c746c8700 1 --2- 172.21.15.59:0/2160889195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c104d10 0x7f1c6c10b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.660+0000 7f1c746c8700 1 --2- 172.21.15.59:0/2160889195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c6c103240 0x7f1c6c103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.660+0000 7f1c746c8700 1 --2- 172.21.15.59:0/2160889195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c104370 0x7f1c6c104780 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.660+0000 7f1c746c8700 1 -- 172.21.15.59:0/2160889195 >> 172.21.15.59:0/2160889195 conn(0x7f1c6c0fe880 msgr2=0x7f1c6c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:03:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.661+0000 7f1c746c8700 1 -- 172.21.15.59:0/2160889195 shutdown_connections 2024-05-09T02:03:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.661+0000 7f1c746c8700 1 -- 172.21.15.59:0/2160889195 wait complete. 2024-05-09T02:03:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.662+0000 7f1c746c8700 1 Processor -- start 2024-05-09T02:03:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.662+0000 7f1c746c8700 1 -- start start 2024-05-09T02:03:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.662+0000 7f1c746c8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c6c103240 0x7f1c6c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:03:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.662+0000 7f1c746c8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c104370 0x7f1c6c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:03:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.662+0000 7f1c746c8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c104d10 0x7f1c6c1a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:03:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.662+0000 7f1c746c8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c10eca0 con 0x7f1c6c104d10 2024-05-09T02:03:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.662+0000 7f1c746c8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c10eb20 con 0x7f1c6c104370 2024-05-09T02:03:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.662+0000 7f1c746c8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c10ee20 con 0x7f1c6c103240 2024-05-09T02:03:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.662+0000 7f1c71c63700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c104370 0x7f1c6c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:03:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.662+0000 7f1c71c63700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c104370 0x7f1c6c19f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51632/0 (socket says 172.21.15.59:51632) 2024-05-09T02:03:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.662+0000 7f1c71c63700 1 -- 172.21.15.59:0/271664696 learned_addr learned my addr 172.21.15.59:0/271664696 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-09T02:03:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.662+0000 7f1c72464700 1 --2- 172.21.15.59:0/271664696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c6c103240 0x7f1c6c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:03:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.662+0000 7f1c72c65700 1 --2- 172.21.15.59:0/271664696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c104d10 0x7f1c6c1a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:03:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.663+0000 7f1c71c63700 1 -- 172.21.15.59:0/271664696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c6c103240 msgr2=0x7f1c6c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:03:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.663+0000 7f1c71c63700 1 --2- 172.21.15.59:0/271664696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c6c103240 0x7f1c6c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.663+0000 7f1c71c63700 1 -- 172.21.15.59:0/271664696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c104d10 msgr2=0x7f1c6c1a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:03:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.663+0000 7f1c71c63700 1 --2- 172.21.15.59:0/271664696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c104d10 0x7f1c6c1a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.663+0000 7f1c71c63700 1 -- 172.21.15.59:0/271664696 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c1a2390 con 0x7f1c6c104370 2024-05-09T02:03:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.663+0000 7f1c72464700 1 --2- 172.21.15.59:0/271664696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c6c103240 0x7f1c6c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:03:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.663+0000 7f1c71c63700 1 --2- 172.21.15.59:0/271664696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c104370 0x7f1c6c19f660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c006010 tx=0x7f1c5c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:03:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.663+0000 7f1c637fe700 1 -- 172.21.15.59:0/271664696 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c027070 con 0x7f1c6c104370 2024-05-09T02:03:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.663+0000 7f1c637fe700 1 -- 172.21.15.59:0/271664696 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c5c0040c0 con 0x7f1c6c104370 2024-05-09T02:03:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.663+0000 7f1c746c8700 1 -- 172.21.15.59:0/271664696 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6c1a2620 con 0x7f1c6c104370 2024-05-09T02:03:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.663+0000 7f1c637fe700 1 -- 172.21.15.59:0/271664696 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c022d90 con 0x7f1c6c104370 2024-05-09T02:03:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.663+0000 7f1c746c8700 1 -- 172.21.15.59:0/271664696 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6c1a2ad0 con 0x7f1c6c104370 2024-05-09T02:03:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.665+0000 7f1c746c8700 1 -- 172.21.15.59:0/271664696 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c6c0512c0 con 0x7f1c6c104370 2024-05-09T02:03:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.666+0000 7f1c637fe700 1 -- 172.21.15.59:0/271664696 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1c5c02a070 con 0x7f1c6c104370 2024-05-09T02:03:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.666+0000 7f1c637fe700 1 --2- 172.21.15.59:0/271664696 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1c580b2900 0x7f1c580b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:03:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.667+0000 7f1c637fe700 1 -- 172.21.15.59:0/271664696 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f1c5c0e1d10 con 0x7f1c6c104370 2024-05-09T02:03:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.667+0000 7f1c72464700 1 --2- 172.21.15.59:0/271664696 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1c580b2900 0x7f1c580b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:03:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.667+0000 7f1c72464700 1 --2- 172.21.15.59:0/271664696 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1c580b2900 0x7f1c580b4e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c104270 tx=0x7f1c6800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:03:17.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.672+0000 7f1c637fe700 1 -- 172.21.15.59:0/271664696 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1c5c013070 con 0x7f1c6c104370 2024-05-09T02:03:17.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:17.895+0000 7f1c746c8700 1 -- 172.21.15.59:0/271664696 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1c6c063c00 con 0x7f1c580b2900 2024-05-09T02:03:18.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:18 smithi059 ceph-mon[34781]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T02:03:18.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:03:18.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:03:18.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:18.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:18 smithi196 ceph-mon[34718]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T02:03:18.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:18 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:03:18.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:18 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:03:18.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:18 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:18.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:18 smithi019 ceph-mon[30151]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T02:03:18.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:18 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:03:18.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:18 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:03:18.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:18 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:19.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:19 smithi059 ceph-mon[34781]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:03:19.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:19 smithi059 ceph-mon[34781]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:19.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:19 smithi196 ceph-mon[34718]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:03:19.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:19 smithi196 ceph-mon[34718]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:19.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:19 smithi019 ceph-mon[30151]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:03:19.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:19 smithi019 ceph-mon[30151]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:21.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:21 smithi196 ceph-mon[34718]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:21.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:21 smithi196 ceph-mon[34718]: from='client.? 172.21.15.59:0/1411676446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "091667f1-e887-4a94-8723-6f85b676fef8"}]: dispatch 2024-05-09T02:03:21.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:21 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "091667f1-e887-4a94-8723-6f85b676fef8"}]: dispatch 2024-05-09T02:03:21.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:21 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "091667f1-e887-4a94-8723-6f85b676fef8"}]': finished 2024-05-09T02:03:21.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:21 smithi196 ceph-mon[34718]: osdmap e29: 5 total, 4 up, 5 in 2024-05-09T02:03:21.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:21.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:21 smithi019 ceph-mon[30151]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:21.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:21 smithi019 ceph-mon[30151]: from='client.? 172.21.15.59:0/1411676446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "091667f1-e887-4a94-8723-6f85b676fef8"}]: dispatch 2024-05-09T02:03:21.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:21 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "091667f1-e887-4a94-8723-6f85b676fef8"}]: dispatch 2024-05-09T02:03:21.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:21 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "091667f1-e887-4a94-8723-6f85b676fef8"}]': finished 2024-05-09T02:03:21.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:21 smithi019 ceph-mon[30151]: osdmap e29: 5 total, 4 up, 5 in 2024-05-09T02:03:21.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:21.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:21 smithi059 ceph-mon[34781]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:21.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:21 smithi059 ceph-mon[34781]: from='client.? 172.21.15.59:0/1411676446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "091667f1-e887-4a94-8723-6f85b676fef8"}]: dispatch 2024-05-09T02:03:21.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:21 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "091667f1-e887-4a94-8723-6f85b676fef8"}]: dispatch 2024-05-09T02:03:21.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:21 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "091667f1-e887-4a94-8723-6f85b676fef8"}]': finished 2024-05-09T02:03:21.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:21 smithi059 ceph-mon[34781]: osdmap e29: 5 total, 4 up, 5 in 2024-05-09T02:03:21.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:22.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:22 smithi196 ceph-mon[34718]: from='client.? 172.21.15.59:0/2178830421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T02:03:22.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:22 smithi019 ceph-mon[30151]: from='client.? 172.21.15.59:0/2178830421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T02:03:22.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:22 smithi059 ceph-mon[34781]: from='client.? 172.21.15.59:0/2178830421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T02:03:23.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:23 smithi196 ceph-mon[34718]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T02:03:23.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:23 smithi019 ceph-mon[30151]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T02:03:23.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:23 smithi059 ceph-mon[34781]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T02:03:25.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:25 smithi196 ceph-mon[34718]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-09T02:03:25.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:25 smithi019 ceph-mon[30151]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-09T02:03:25.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:25 smithi059 ceph-mon[34781]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-09T02:03:27.453 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:27 smithi059 ceph-mon[34781]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-09T02:03:27.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:27 smithi196 ceph-mon[34718]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-09T02:03:27.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:27 smithi019 ceph-mon[30151]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-09T02:03:29.435 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:29 smithi059 ceph-mon[34781]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:03:29.435 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:29 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T02:03:29.435 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:29 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:29.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:29 smithi196 ceph-mon[34718]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:03:29.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:29 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T02:03:29.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:29 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:29.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:29 smithi019 ceph-mon[30151]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:03:29.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:29 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T02:03:29.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:29 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:30.403 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:30 smithi059 ceph-mon[34781]: Deploying daemon osd.4 on smithi059 2024-05-09T02:03:30.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:30 smithi196 ceph-mon[34718]: Deploying daemon osd.4 on smithi059 2024-05-09T02:03:30.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:30 smithi019 ceph-mon[30151]: Deploying daemon osd.4 on smithi059 2024-05-09T02:03:31.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:31 smithi196 ceph-mon[34718]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:03:31.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:31 smithi019 ceph-mon[30151]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:03:31.593 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:31 smithi059 ceph-mon[34781]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:03:33.302 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:33 smithi059 ceph-mon[34781]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-09T02:03:33.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:33 smithi196 ceph-mon[34718]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-09T02:03:33.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:33 smithi019 ceph-mon[30151]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-09T02:03:35.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:35 smithi059 ceph-mon[34781]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:35.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:35.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:35.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:03:35.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:35 smithi196 ceph-mon[34718]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:35.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:35.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:35.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:03:35.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:35 smithi019 ceph-mon[30151]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:35.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:35.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:35.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:03:37.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:37 smithi059 ceph-mon[34781]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:37.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:37 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:37.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:37 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:37.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:37 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:37.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:37 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:03:37.380 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:37 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:37.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:37 smithi196 ceph-mon[34718]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:37.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:37 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:37.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:37 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:37.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:37 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:37.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:37 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:03:37.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:37 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:37.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:37 smithi019 ceph-mon[30151]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:37.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:37.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:37.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:37.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:03:37.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:38.171 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 4 on host 'smithi059' 2024-05-09T02:03:38.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:38.169+0000 7f1c637fe700 1 -- 172.21.15.59:0/271664696 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1c6c063c00 con 0x7f1c580b2900 2024-05-09T02:03:38.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:38.171+0000 7f1c746c8700 1 -- 172.21.15.59:0/271664696 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1c580b2900 msgr2=0x7f1c580b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:03:38.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:38.171+0000 7f1c746c8700 1 --2- 172.21.15.59:0/271664696 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1c580b2900 0x7f1c580b4e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c104270 tx=0x7f1c6800b410 comp rx=0 tx=0).stop 2024-05-09T02:03:38.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:38.172+0000 7f1c746c8700 1 -- 172.21.15.59:0/271664696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c104370 msgr2=0x7f1c6c19f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:03:38.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:38.172+0000 7f1c746c8700 1 --2- 172.21.15.59:0/271664696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c104370 0x7f1c6c19f660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c006010 tx=0x7f1c5c0047b0 comp rx=0 tx=0).stop 2024-05-09T02:03:38.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:38.172+0000 7f1c746c8700 1 -- 172.21.15.59:0/271664696 shutdown_connections 2024-05-09T02:03:38.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:38.172+0000 7f1c746c8700 1 --2- 172.21.15.59:0/271664696 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1c580b2900 0x7f1c580b4e10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:38.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:38.172+0000 7f1c746c8700 1 --2- 172.21.15.59:0/271664696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c104d10 0x7f1c6c1a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:38.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:38.172+0000 7f1c746c8700 1 --2- 172.21.15.59:0/271664696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c6c103240 0x7f1c6c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:38.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:38.172+0000 7f1c746c8700 1 --2- 172.21.15.59:0/271664696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c104370 0x7f1c6c19f660 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:38.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:38.172+0000 7f1c746c8700 1 -- 172.21.15.59:0/271664696 >> 172.21.15.59:0/271664696 conn(0x7f1c6c0fe880 msgr2=0x7f1c6c1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:03:38.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:38.172+0000 7f1c746c8700 1 -- 172.21.15.59:0/271664696 shutdown_connections 2024-05-09T02:03:38.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:03:38.172+0000 7f1c746c8700 1 -- 172.21.15.59:0/271664696 wait complete. 2024-05-09T02:03:38.370 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:38 smithi059 ceph-mon[34781]: from='osd.4 [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T02:03:38.371 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:38 smithi059 ceph-mon[34781]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T02:03:38.371 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:38 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:38.371 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:38 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:38.371 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:38 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:03:38.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:38 smithi196 ceph-mon[34718]: from='osd.4 [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T02:03:38.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:38 smithi196 ceph-mon[34718]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T02:03:38.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:38 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:38.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:38 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:38.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:38 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:03:38.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:38 smithi019 ceph-mon[30151]: from='osd.4 [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T02:03:38.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:38 smithi019 ceph-mon[30151]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T02:03:38.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:38 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:38.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:38 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:38.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:38 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:03:38.821 DEBUG:teuthology.orchestra.run.smithi059:osd.4> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.4.service 2024-05-09T02:03:38.823 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/vg_nvme/lv_3... 2024-05-09T02:03:38.823 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T02:03:39.065 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:03:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:39 smithi059 ceph-mon[34781]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:39 smithi059 ceph-mon[34781]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T02:03:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:39 smithi059 ceph-mon[34781]: from='osd.4 [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:03:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:39 smithi059 ceph-mon[34781]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T02:03:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:39 smithi059 ceph-mon[34781]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:03:39.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:39 smithi196 ceph-mon[34718]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:39.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:39 smithi196 ceph-mon[34718]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T02:03:39.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:39 smithi196 ceph-mon[34718]: from='osd.4 [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:03:39.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:39 smithi196 ceph-mon[34718]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T02:03:39.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:39.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:39 smithi196 ceph-mon[34718]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:03:39.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:39 smithi019 ceph-mon[30151]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:39.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:39 smithi019 ceph-mon[30151]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T02:03:39.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:39 smithi019 ceph-mon[30151]: from='osd.4 [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:03:39.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:39 smithi019 ceph-mon[30151]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T02:03:39.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:39.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:39 smithi019 ceph-mon[30151]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:03:39.966 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:03:39 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4[45904]: 2024-05-09T02:03:39.664+0000 7fad35fdd700 -1 osd.4 0 waiting for initial osdmap 2024-05-09T02:03:39.966 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:03:39 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4[45904]: 2024-05-09T02:03:39.672+0000 7fad2dd8a700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T02:03:40.268 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:40 smithi059 ceph-mon[34781]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T02:03:40.269 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:40 smithi059 ceph-mon[34781]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T02:03:40.269 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:40 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:40.269 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:40 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:40.269 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:40 smithi059 ceph-mon[34781]: from='osd.4 ' entity='osd.4' 2024-05-09T02:03:40.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:40 smithi196 ceph-mon[34718]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T02:03:40.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:40 smithi196 ceph-mon[34718]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T02:03:40.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:40 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:40.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:40 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:40.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:40 smithi196 ceph-mon[34718]: from='osd.4 ' entity='osd.4' 2024-05-09T02:03:40.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:40 smithi019 ceph-mon[30151]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T02:03:40.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:40 smithi019 ceph-mon[30151]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T02:03:40.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:40 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:40.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:40 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:40.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:40 smithi019 ceph-mon[30151]: from='osd.4 ' entity='osd.4' 2024-05-09T02:03:40.986 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T02:03:40.987 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T02:03:40.987 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-05-09T02:03:40.987 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-05-09T02:03:40.987 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123542 s, 849 MB/s 2024-05-09T02:03:40.987 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-05-09T02:03:41.303 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:41 smithi196 ceph-mon[34718]: purged_snaps scrub starts 2024-05-09T02:03:41.303 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:41 smithi196 ceph-mon[34718]: purged_snaps scrub ok 2024-05-09T02:03:41.303 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:41 smithi196 ceph-mon[34718]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:41.304 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:41 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:41.304 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:41 smithi196 ceph-mon[34718]: osd.4 [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330] boot 2024-05-09T02:03:41.304 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:41 smithi196 ceph-mon[34718]: osdmap e32: 5 total, 5 up, 5 in 2024-05-09T02:03:41.304 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:41 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:41 smithi059 ceph-mon[34781]: purged_snaps scrub starts 2024-05-09T02:03:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:41 smithi059 ceph-mon[34781]: purged_snaps scrub ok 2024-05-09T02:03:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:41 smithi059 ceph-mon[34781]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:41 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:41 smithi059 ceph-mon[34781]: osd.4 [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330] boot 2024-05-09T02:03:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:41 smithi059 ceph-mon[34781]: osdmap e32: 5 total, 5 up, 5 in 2024-05-09T02:03:41.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:41 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:41.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:41 smithi019 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T02:03:41.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:41 smithi019 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T02:03:41.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:41 smithi019 ceph-mon[30151]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T02:03:41.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:41 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:41.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:41 smithi019 ceph-mon[30151]: osd.4 [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330] boot 2024-05-09T02:03:41.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:41 smithi019 ceph-mon[30151]: osdmap e32: 5 total, 5 up, 5 in 2024-05-09T02:03:41.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:41 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:03:41.602 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_3 2024-05-09T02:03:41.848 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:03:42.956 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:42 smithi196 ceph-mon[34718]: osdmap e33: 5 total, 5 up, 5 in 2024-05-09T02:03:42.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:42 smithi196 ceph-mon[34718]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:42.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:42 smithi196 ceph-mon[34718]: Detected new or changed devices on smithi059 2024-05-09T02:03:42.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:42 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:42.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:42 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:42.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:42 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:03:42.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:42 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:03:42.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:42 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi059 to 8546M 2024-05-09T02:03:42.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:42 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:42.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:42 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:42.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:42 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:03:42.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:42 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:42.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:42 smithi059 ceph-mon[34781]: osdmap e33: 5 total, 5 up, 5 in 2024-05-09T02:03:42.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:42 smithi059 ceph-mon[34781]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:42.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:42 smithi059 ceph-mon[34781]: Detected new or changed devices on smithi059 2024-05-09T02:03:42.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:42 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:42.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:42 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:42.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:42 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:03:42.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:42 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:03:42.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:42 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi059 to 8546M 2024-05-09T02:03:42.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:42 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:42.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:42 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:42.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:42 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:03:42.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:42 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:43.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:42 smithi019 ceph-mon[30151]: osdmap e33: 5 total, 5 up, 5 in 2024-05-09T02:03:43.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:42 smithi019 ceph-mon[30151]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:43.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:42 smithi019 ceph-mon[30151]: Detected new or changed devices on smithi059 2024-05-09T02:03:43.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:42 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:43.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:42 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:43.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:42 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:03:43.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:42 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:03:43.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:42 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi059 to 8546M 2024-05-09T02:03:43.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:42 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:43.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:42 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:43.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:42 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:03:43.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:42 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:03:43.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.488+0000 7fb9c6a68700 1 -- 172.21.15.196:0/3376248264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c0102350 msgr2=0x7fb9c0102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:03:43.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.488+0000 7fb9c6a68700 1 --2- 172.21.15.196:0/3376248264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c0102350 0x7fb9c0102760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009990 tx=0x7fb9ac009ca0 comp rx=0 tx=0).stop 2024-05-09T02:03:43.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.488+0000 7fb9c6a68700 1 -- 172.21.15.196:0/3376248264 shutdown_connections 2024-05-09T02:03:43.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.488+0000 7fb9c6a68700 1 --2- 172.21.15.196:0/3376248264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9c0102ca0 0x7fb9c010c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:43.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.488+0000 7fb9c6a68700 1 --2- 172.21.15.196:0/3376248264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9c0105900 0x7fb9c0105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:43.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.488+0000 7fb9c6a68700 1 --2- 172.21.15.196:0/3376248264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c0102350 0x7fb9c0102760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:43.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.488+0000 7fb9c6a68700 1 -- 172.21.15.196:0/3376248264 >> 172.21.15.196:0/3376248264 conn(0x7fb9c00fe020 msgr2=0x7fb9c01004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:03:43.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.488+0000 7fb9c6a68700 1 -- 172.21.15.196:0/3376248264 shutdown_connections 2024-05-09T02:03:43.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.488+0000 7fb9c6a68700 1 -- 172.21.15.196:0/3376248264 wait complete. 2024-05-09T02:03:43.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c6a68700 1 Processor -- start 2024-05-09T02:03:43.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c6a68700 1 -- start start 2024-05-09T02:03:43.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c6a68700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9c0102350 0x7fb9c019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:03:43.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c6a68700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9c0102ca0 0x7fb9c019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:03:43.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c6a68700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c0105900 0x7fb9c01a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:03:43.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c6a68700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c010fdc0 con 0x7fb9c0102350 2024-05-09T02:03:43.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c6a68700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c010fc40 con 0x7fb9c0102ca0 2024-05-09T02:03:43.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c6a68700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c010ff40 con 0x7fb9c0105900 2024-05-09T02:03:43.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c5005700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c0105900 0x7fb9c01a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:03:43.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c5005700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c0105900 0x7fb9c01a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:33488/0 (socket says 172.21.15.196:33488) 2024-05-09T02:03:43.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c5005700 1 -- 172.21.15.196:0/2808308000 learned_addr learned my addr 172.21.15.196:0/2808308000 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T02:03:43.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9bffff700 1 --2- 172.21.15.196:0/2808308000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9c0102ca0 0x7fb9c019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:03:43.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c4804700 1 --2- 172.21.15.196:0/2808308000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9c0102350 0x7fb9c019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:03:43.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c5005700 1 -- 172.21.15.196:0/2808308000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9c0102ca0 msgr2=0x7fb9c019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:03:43.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c5005700 1 --2- 172.21.15.196:0/2808308000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9c0102ca0 0x7fb9c019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:43.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c5005700 1 -- 172.21.15.196:0/2808308000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9c0102350 msgr2=0x7fb9c019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:03:43.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c5005700 1 --2- 172.21.15.196:0/2808308000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9c0102350 0x7fb9c019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:03:43.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c5005700 1 -- 172.21.15.196:0/2808308000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c01a2440 con 0x7fb9c0105900 2024-05-09T02:03:43.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.489+0000 7fb9c5005700 1 --2- 172.21.15.196:0/2808308000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c0105900 0x7fb9c01a1dc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8007ee0 tx=0x7fb9b800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:03:43.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.490+0000 7fb9bdffb700 1 -- 172.21.15.196:0/2808308000 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8004bb0 con 0x7fb9c0105900 2024-05-09T02:03:43.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.490+0000 7fb9bdffb700 1 -- 172.21.15.196:0/2808308000 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9b8004d50 con 0x7fb9c0105900 2024-05-09T02:03:43.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.490+0000 7fb9c6a68700 1 -- 172.21.15.196:0/2808308000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c01a2730 con 0x7fb9c0105900 2024-05-09T02:03:43.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.490+0000 7fb9bdffb700 1 -- 172.21.15.196:0/2808308000 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8005920 con 0x7fb9c0105900 2024-05-09T02:03:43.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.490+0000 7fb9c6a68700 1 -- 172.21.15.196:0/2808308000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c01a2c70 con 0x7fb9c0105900 2024-05-09T02:03:43.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.491+0000 7fb9c6a68700 1 -- 172.21.15.196:0/2808308000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c00512c0 con 0x7fb9c0105900 2024-05-09T02:03:43.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.492+0000 7fb9bdffb700 1 -- 172.21.15.196:0/2808308000 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb9b801a020 con 0x7fb9c0105900 2024-05-09T02:03:43.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.493+0000 7fb9bdffb700 1 --2- 172.21.15.196:0/2808308000 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb9b00b2880 0x7fb9b00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:03:43.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.493+0000 7fb9bdffb700 1 -- 172.21.15.196:0/2808308000 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7fb9b80e07f0 con 0x7fb9c0105900 2024-05-09T02:03:43.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.493+0000 7fb9c4804700 1 --2- 172.21.15.196:0/2808308000 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb9b00b2880 0x7fb9b00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:03:43.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.494+0000 7fb9c4804700 1 --2- 172.21.15.196:0/2808308000 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb9b00b2880 0x7fb9b00b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb9c006c080 tx=0x7fb9b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:03:43.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.498+0000 7fb9bdffb700 1 -- 172.21.15.196:0/2808308000 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb9b8010070 con 0x7fb9c0105900 2024-05-09T02:03:43.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:03:43.712+0000 7fb9c6a68700 1 -- 172.21.15.196:0/2808308000 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb9c0063c00 con 0x7fb9b00b2880 2024-05-09T02:03:44.043 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:03:44.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:03:44.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:44.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:03:44.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:03:44.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:44.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:03:44.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:03:44.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:45.043 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:44 smithi196 ceph-mon[34718]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:03:45.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:44 smithi196 ceph-mon[34718]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:45.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:44 smithi019 ceph-mon[30151]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:03:45.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:44 smithi019 ceph-mon[30151]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:45.215 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:44 smithi059 ceph-mon[34781]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:03:45.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:44 smithi059 ceph-mon[34781]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:47.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:47 smithi196 ceph-mon[34718]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:47.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:47 smithi196 ceph-mon[34718]: from='client.? 172.21.15.196:0/2980442555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec"}]: dispatch 2024-05-09T02:03:47.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:47 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec"}]: dispatch 2024-05-09T02:03:47.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:47 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec"}]': finished 2024-05-09T02:03:47.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:47 smithi196 ceph-mon[34718]: osdmap e34: 6 total, 5 up, 6 in 2024-05-09T02:03:47.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:47 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:03:47.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:47 smithi019 ceph-mon[30151]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:47.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:47 smithi019 ceph-mon[30151]: from='client.? 172.21.15.196:0/2980442555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec"}]: dispatch 2024-05-09T02:03:47.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:47 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec"}]: dispatch 2024-05-09T02:03:47.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:47 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec"}]': finished 2024-05-09T02:03:47.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:47 smithi019 ceph-mon[30151]: osdmap e34: 6 total, 5 up, 6 in 2024-05-09T02:03:47.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:03:47.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:47 smithi059 ceph-mon[34781]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:47.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:47 smithi059 ceph-mon[34781]: from='client.? 172.21.15.196:0/2980442555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec"}]: dispatch 2024-05-09T02:03:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:47 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec"}]: dispatch 2024-05-09T02:03:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:47 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec"}]': finished 2024-05-09T02:03:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:47 smithi059 ceph-mon[34781]: osdmap e34: 6 total, 5 up, 6 in 2024-05-09T02:03:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:47 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:03:48.215 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:48 smithi059 ceph-mon[34781]: from='client.? 172.21.15.196:0/843781582' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T02:03:48.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:48 smithi196 ceph-mon[34718]: from='client.? 172.21.15.196:0/843781582' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T02:03:48.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:48 smithi019 ceph-mon[30151]: from='client.? 172.21.15.196:0/843781582' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T02:03:49.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:49 smithi196 ceph-mon[34718]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:49.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:49 smithi019 ceph-mon[30151]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:49.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:49 smithi059 ceph-mon[34781]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:51.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:51 smithi196 ceph-mon[34718]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:51.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:51 smithi019 ceph-mon[30151]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:51.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:51 smithi059 ceph-mon[34781]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:53.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:53 smithi196 ceph-mon[34718]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:53.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:53 smithi019 ceph-mon[30151]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:53.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:53 smithi059 ceph-mon[34781]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:54.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:54 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T02:03:54.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:54 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:54.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:54 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T02:03:54.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:54 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:54.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:54 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T02:03:54.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:54 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:03:55.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:55 smithi196 ceph-mon[34718]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:55.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:55 smithi196 ceph-mon[34718]: Deploying daemon osd.5 on smithi196 2024-05-09T02:03:55.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:55 smithi019 ceph-mon[30151]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:55.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:55 smithi019 ceph-mon[30151]: Deploying daemon osd.5 on smithi196 2024-05-09T02:03:55.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:55 smithi059 ceph-mon[34781]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:55.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:55 smithi059 ceph-mon[34781]: Deploying daemon osd.5 on smithi196 2024-05-09T02:03:57.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:57 smithi196 ceph-mon[34718]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:57.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:57 smithi019 ceph-mon[30151]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:57.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:57 smithi059 ceph-mon[34781]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:59.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:03:59 smithi196 ceph-mon[34718]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:59.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:03:59 smithi019 ceph-mon[30151]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:03:59.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:03:59 smithi059 ceph-mon[34781]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:04:00.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:00 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:00.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:00 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:00.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:00 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:04:00.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:00 smithi059 ceph-mon[34781]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:04:01.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:00 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:01.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:00 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:01.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:00 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:04:01.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:00 smithi196 ceph-mon[34718]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:04:01.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:00 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:01.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:00 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:01.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:00 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:04:01.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:00 smithi019 ceph-mon[30151]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:04:03.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:03 smithi196 ceph-mon[34718]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:04:03.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:03 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:03.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:03 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:03.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:03 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:03.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:03 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:04:03.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:03 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:03.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:03 smithi019 ceph-mon[30151]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:04:03.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:03 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:03.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:03 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:03.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:03 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:03.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:03 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:04:03.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:03 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:03.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:03 smithi059 ceph-mon[34781]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:04:03.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:03 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:03.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:03 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:03.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:03 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:03.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:03 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:04:03.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:03 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:03.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:04:03.535+0000 7fb9bdffb700 1 -- 172.21.15.196:0/2808308000 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb9c0063c00 con 0x7fb9b00b2880 2024-05-09T02:04:03.538 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-05-09T02:04:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:04:03.538+0000 7fb9c6a68700 1 -- 172.21.15.196:0/2808308000 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb9b00b2880 msgr2=0x7fb9b00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:04:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:04:03.538+0000 7fb9c6a68700 1 --2- 172.21.15.196:0/2808308000 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb9b00b2880 0x7fb9b00b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb9c006c080 tx=0x7fb9b4009260 comp rx=0 tx=0).stop 2024-05-09T02:04:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:04:03.538+0000 7fb9c6a68700 1 -- 172.21.15.196:0/2808308000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c0105900 msgr2=0x7fb9c01a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:04:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:04:03.538+0000 7fb9c6a68700 1 --2- 172.21.15.196:0/2808308000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c0105900 0x7fb9c01a1dc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8007ee0 tx=0x7fb9b800d3b0 comp rx=0 tx=0).stop 2024-05-09T02:04:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:04:03.538+0000 7fb9c6a68700 1 -- 172.21.15.196:0/2808308000 shutdown_connections 2024-05-09T02:04:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:04:03.538+0000 7fb9c6a68700 1 --2- 172.21.15.196:0/2808308000 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb9b00b2880 0x7fb9b00b4d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:04:03.538+0000 7fb9c6a68700 1 --2- 172.21.15.196:0/2808308000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c0105900 0x7fb9c01a1dc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:04:03.538+0000 7fb9c6a68700 1 --2- 172.21.15.196:0/2808308000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9c0102350 0x7fb9c019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:04:03.538+0000 7fb9c6a68700 1 --2- 172.21.15.196:0/2808308000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9c0102ca0 0x7fb9c019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:04:03.538+0000 7fb9c6a68700 1 -- 172.21.15.196:0/2808308000 >> 172.21.15.196:0/2808308000 conn(0x7fb9c00fe020 msgr2=0x7fb9c01088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:04:03.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:04:03.538+0000 7fb9c6a68700 1 -- 172.21.15.196:0/2808308000 shutdown_connections 2024-05-09T02:04:03.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:04:03.538+0000 7fb9c6a68700 1 -- 172.21.15.196:0/2808308000 wait complete. 2024-05-09T02:04:04.147 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:04 smithi196 ceph-mon[34718]: from='osd.5 [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T02:04:04.148 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:04 smithi196 ceph-mon[34718]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T02:04:04.148 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:04 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:04.148 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:04 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:04.148 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:04 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:04:04.173 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.5.service 2024-05-09T02:04:04.175 INFO:tasks.cephadm:Deploying osd.6 on smithi019 with /dev/vg_nvme/lv_2... 2024-05-09T02:04:04.176 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T02:04:04.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:04 smithi019 ceph-mon[30151]: from='osd.5 [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T02:04:04.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:04 smithi019 ceph-mon[30151]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T02:04:04.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:04 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:04.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:04 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:04.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:04 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:04:04.431 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:04:04.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:04 smithi059 ceph-mon[34781]: from='osd.5 [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T02:04:04.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:04 smithi059 ceph-mon[34781]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T02:04:04.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:04 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:04.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:04 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:04.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:04 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:04:05.287 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:05 smithi196 ceph-mon[34718]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:04:05.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:05 smithi196 ceph-mon[34718]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T02:04:05.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:05 smithi196 ceph-mon[34718]: from='osd.5 [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:04:05.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:05 smithi196 ceph-mon[34718]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T02:04:05.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:05 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:04:05.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:05 smithi196 ceph-mon[34718]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:04:05.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:05 smithi019 ceph-mon[30151]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:04:05.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:05 smithi019 ceph-mon[30151]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T02:04:05.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:05 smithi019 ceph-mon[30151]: from='osd.5 [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:04:05.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:05 smithi019 ceph-mon[30151]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T02:04:05.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:05 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:04:05.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:05 smithi019 ceph-mon[30151]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:04:05.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:05 smithi059 ceph-mon[34781]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:04:05.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:05 smithi059 ceph-mon[34781]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T02:04:05.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:05 smithi059 ceph-mon[34781]: from='osd.5 [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:04:05.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:05 smithi059 ceph-mon[34781]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T02:04:05.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:05 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:04:05.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:05 smithi059 ceph-mon[34781]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:04:05.544 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:04:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:04:05.472+0000 7fcfd39bc700 -1 osd.5 0 waiting for initial osdmap 2024-05-09T02:04:05.544 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:04:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:04:05.479+0000 7fcfcb769700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T02:04:06.179 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:06 smithi019 ceph-mon[30151]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T02:04:06.179 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:06 smithi019 ceph-mon[30151]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T02:04:06.179 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:06 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:04:06.179 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:06 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:04:06.179 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:06 smithi019 ceph-mon[30151]: from='osd.5 ' entity='osd.5' 2024-05-09T02:04:06.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:06 smithi196 ceph-mon[34718]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T02:04:06.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:06 smithi196 ceph-mon[34718]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T02:04:06.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:06 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:04:06.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:06 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:04:06.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:06 smithi196 ceph-mon[34718]: from='osd.5 ' entity='osd.5' 2024-05-09T02:04:06.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:06 smithi059 ceph-mon[34781]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T02:04:06.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:06 smithi059 ceph-mon[34781]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T02:04:06.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:06 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:04:06.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:06 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:04:06.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:06 smithi059 ceph-mon[34781]: from='osd.5 ' entity='osd.5' 2024-05-09T02:04:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T02:04:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T02:04:06.782 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-09T02:04:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-09T02:04:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116103 s, 903 MB/s 2024-05-09T02:04:06.782 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 2024-05-09T02:04:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-09T02:04:07.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:07 smithi019 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T02:04:07.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:07 smithi019 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T02:04:07.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:07 smithi019 ceph-mon[30151]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:04:07.163 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:07 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:04:07.163 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:07 smithi019 ceph-mon[30151]: osd.5 [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898] boot 2024-05-09T02:04:07.163 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:07 smithi019 ceph-mon[30151]: osdmap e37: 6 total, 6 up, 6 in 2024-05-09T02:04:07.163 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:07 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:04:07.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:07 smithi196 ceph-mon[34718]: purged_snaps scrub starts 2024-05-09T02:04:07.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:07 smithi196 ceph-mon[34718]: purged_snaps scrub ok 2024-05-09T02:04:07.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:07 smithi196 ceph-mon[34718]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:04:07.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:07 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:04:07.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:07 smithi196 ceph-mon[34718]: osd.5 [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898] boot 2024-05-09T02:04:07.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:07 smithi196 ceph-mon[34718]: osdmap e37: 6 total, 6 up, 6 in 2024-05-09T02:04:07.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:07 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:04:07.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:07 smithi059 ceph-mon[34781]: purged_snaps scrub starts 2024-05-09T02:04:07.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:07 smithi059 ceph-mon[34781]: purged_snaps scrub ok 2024-05-09T02:04:07.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:07 smithi059 ceph-mon[34781]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-05-09T02:04:07.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:07 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:04:07.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:07 smithi059 ceph-mon[34781]: osd.5 [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898] boot 2024-05-09T02:04:07.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:07 smithi059 ceph-mon[34781]: osdmap e37: 6 total, 6 up, 6 in 2024-05-09T02:04:07.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:07 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:04:07.472 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_2 2024-05-09T02:04:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:04:08.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:08 smithi019 ceph-mon[30151]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T02:04:08.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:08 smithi019 ceph-mon[30151]: Detected new or changed devices on smithi196 2024-05-09T02:04:08.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:08 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:08.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:08 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:08.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:08 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:04:08.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:08 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:04:08.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:08 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi196 to 8544M 2024-05-09T02:04:08.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:08 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:08.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:08 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:08.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:08 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:04:08.184 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:08 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:08.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:08 smithi059 ceph-mon[34781]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T02:04:08.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:08 smithi059 ceph-mon[34781]: Detected new or changed devices on smithi196 2024-05-09T02:04:08.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:08 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:08.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:08 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:08.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:08 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:04:08.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:08 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:04:08.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:08 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi196 to 8544M 2024-05-09T02:04:08.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:08 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:08.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:08 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:08.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:08 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:04:08.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:08 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:08.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:08 smithi196 ceph-mon[34718]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T02:04:08.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:08 smithi196 ceph-mon[34718]: Detected new or changed devices on smithi196 2024-05-09T02:04:08.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:08 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:08.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:08 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:08.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:08 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:04:08.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:08 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:04:08.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:08 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi196 to 8544M 2024-05-09T02:04:08.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:08 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:08.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:08 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:08.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:08 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:04:08.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:08 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:09.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:09 smithi019 ceph-mon[30151]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:09.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:09 smithi019 ceph-mon[30151]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T02:04:09.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:09 smithi059 ceph-mon[34781]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:09.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:09 smithi059 ceph-mon[34781]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T02:04:09.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:09 smithi196 ceph-mon[34718]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:09.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:09 smithi196 ceph-mon[34718]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T02:04:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.551+0000 7f3c30cc4700 1 -- 172.21.15.19:0/1382670838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c2c103f60 msgr2=0x7f3c2c104370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:04:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.551+0000 7f3c30cc4700 1 --2- 172.21.15.19:0/1382670838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c2c103f60 0x7f3c2c104370 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009990 tx=0x7f3c14009ca0 comp rx=0 tx=0).stop 2024-05-09T02:04:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.551+0000 7f3c30cc4700 1 -- 172.21.15.19:0/1382670838 shutdown_connections 2024-05-09T02:04:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.551+0000 7f3c30cc4700 1 --2- 172.21.15.19:0/1382670838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c2c1049e0 0x7f3c2c10c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.551+0000 7f3c30cc4700 1 --2- 172.21.15.19:0/1382670838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c2c1059b0 0x7f3c2c105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.551+0000 7f3c30cc4700 1 --2- 172.21.15.19:0/1382670838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c2c103f60 0x7f3c2c104370 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.551+0000 7f3c30cc4700 1 -- 172.21.15.19:0/1382670838 >> 172.21.15.19:0/1382670838 conn(0x7f3c2c0ff9d0 msgr2=0x7f3c2c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:04:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.551+0000 7f3c30cc4700 1 -- 172.21.15.19:0/1382670838 shutdown_connections 2024-05-09T02:04:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.551+0000 7f3c30cc4700 1 -- 172.21.15.19:0/1382670838 wait complete. 2024-05-09T02:04:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.552+0000 7f3c30cc4700 1 Processor -- start 2024-05-09T02:04:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.552+0000 7f3c30cc4700 1 -- start start 2024-05-09T02:04:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c30cc4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c2c103f60 0x7f3c2c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:04:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c30cc4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c2c1049e0 0x7f3c2c19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:04:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c30cc4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c2c1059b0 0x7f3c2c1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:04:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c30cc4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c10fef0 con 0x7f3c2c1049e0 2024-05-09T02:04:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c30cc4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c10fd70 con 0x7f3c2c103f60 2024-05-09T02:04:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c30cc4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c110070 con 0x7f3c2c1059b0 2024-05-09T02:04:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c29d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c2c1049e0 0x7f3c2c19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:04:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c29d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c2c1049e0 0x7f3c2c19af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40210/0 (socket says 172.21.15.19:40210) 2024-05-09T02:04:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c29d9b700 1 -- 172.21.15.19:0/3656277229 learned_addr learned my addr 172.21.15.19:0/3656277229 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:04:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c2a59c700 1 --2- 172.21.15.19:0/3656277229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c2c103f60 0x7f3c2c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:04:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c2ad9d700 1 --2- 172.21.15.19:0/3656277229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c2c1059b0 0x7f3c2c1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:04:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c29d9b700 1 -- 172.21.15.19:0/3656277229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c2c1059b0 msgr2=0x7f3c2c1a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:04:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c29d9b700 1 --2- 172.21.15.19:0/3656277229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c2c1059b0 0x7f3c2c1a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c29d9b700 1 -- 172.21.15.19:0/3656277229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c2c103f60 msgr2=0x7f3c2c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:04:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c29d9b700 1 --2- 172.21.15.19:0/3656277229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c2c103f60 0x7f3c2c19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c29d9b700 1 -- 172.21.15.19:0/3656277229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c1a2590 con 0x7f3c2c1049e0 2024-05-09T02:04:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c2ad9d700 1 --2- 172.21.15.19:0/3656277229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c2c1059b0 0x7f3c2c1a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:04:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.553+0000 7f3c29d9b700 1 --2- 172.21.15.19:0/3656277229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c2c1049e0 0x7f3c2c19af70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3c14006010 tx=0x7f3c140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:04:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.554+0000 7f3c237fe700 1 -- 172.21.15.19:0/3656277229 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14027070 con 0x7f3c2c1049e0 2024-05-09T02:04:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.554+0000 7f3c30cc4700 1 -- 172.21.15.19:0/3656277229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2c1a2820 con 0x7f3c2c1049e0 2024-05-09T02:04:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.554+0000 7f3c237fe700 1 -- 172.21.15.19:0/3656277229 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c14004140 con 0x7f3c2c1049e0 2024-05-09T02:04:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.554+0000 7f3c237fe700 1 -- 172.21.15.19:0/3656277229 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1400f460 con 0x7f3c2c1049e0 2024-05-09T02:04:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.554+0000 7f3c30cc4700 1 -- 172.21.15.19:0/3656277229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2c1a2c70 con 0x7f3c2c1049e0 2024-05-09T02:04:09.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.555+0000 7f3c30cc4700 1 -- 172.21.15.19:0/3656277229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c2c0512c0 con 0x7f3c2c1049e0 2024-05-09T02:04:09.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.556+0000 7f3c237fe700 1 -- 172.21.15.19:0/3656277229 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3c1402a070 con 0x7f3c2c1049e0 2024-05-09T02:04:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.557+0000 7f3c237fe700 1 --2- 172.21.15.19:0/3656277229 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3c180b2880 0x7f3c180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:04:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.557+0000 7f3c2a59c700 1 --2- 172.21.15.19:0/3656277229 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3c180b2880 0x7f3c180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:04:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.557+0000 7f3c237fe700 1 -- 172.21.15.19:0/3656277229 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4533+0+0 (secure 0 0 0) 0x7f3c140e2640 con 0x7f3c2c1049e0 2024-05-09T02:04:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.558+0000 7f3c2a59c700 1 --2- 172.21.15.19:0/3656277229 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3c180b2880 0x7f3c180b4d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c1053a0 tx=0x7f3c1c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:04:09.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.561+0000 7f3c237fe700 1 -- 172.21.15.19:0/3656277229 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3c14013070 con 0x7f3c2c1049e0 2024-05-09T02:04:09.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:09.784+0000 7f3c30cc4700 1 -- 172.21.15.19:0/3656277229 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3c2c063c00 con 0x7f3c180b2880 2024-05-09T02:04:10.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:10 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:04:10.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:10 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:04:10.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:10 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:10.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:10 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:04:10.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:10 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:04:10.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:10 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:10.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:10 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:04:10.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:10 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:04:10.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:10 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:11.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:11 smithi019 ceph-mon[30151]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:04:11.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:11 smithi019 ceph-mon[30151]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:11.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:11 smithi059 ceph-mon[34781]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:04:11.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:11 smithi059 ceph-mon[34781]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:11.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:11 smithi196 ceph-mon[34718]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:04:11.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:11 smithi196 ceph-mon[34718]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:13.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:13 smithi019 ceph-mon[30151]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T02:04:13.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:13 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3201907412' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e352851-1279-475f-ba8b-eaf9ca73df74"}]: dispatch 2024-05-09T02:04:13.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:13 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3201907412' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e352851-1279-475f-ba8b-eaf9ca73df74"}]': finished 2024-05-09T02:04:13.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:13 smithi019 ceph-mon[30151]: osdmap e40: 7 total, 6 up, 7 in 2024-05-09T02:04:13.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:13 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:13.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:13 smithi059 ceph-mon[34781]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T02:04:13.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:13 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3201907412' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e352851-1279-475f-ba8b-eaf9ca73df74"}]: dispatch 2024-05-09T02:04:13.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:13 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3201907412' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e352851-1279-475f-ba8b-eaf9ca73df74"}]': finished 2024-05-09T02:04:13.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:13 smithi059 ceph-mon[34781]: osdmap e40: 7 total, 6 up, 7 in 2024-05-09T02:04:13.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:13 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:13.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:13 smithi196 ceph-mon[34718]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T02:04:13.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:13 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3201907412' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e352851-1279-475f-ba8b-eaf9ca73df74"}]: dispatch 2024-05-09T02:04:13.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:13 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3201907412' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e352851-1279-475f-ba8b-eaf9ca73df74"}]': finished 2024-05-09T02:04:13.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:13 smithi196 ceph-mon[34718]: osdmap e40: 7 total, 6 up, 7 in 2024-05-09T02:04:13.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:13 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:14.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:14 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3550910133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T02:04:14.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:14 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3550910133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T02:04:14.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:14 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3550910133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T02:04:15.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:15 smithi059 ceph-mon[34781]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-09T02:04:15.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:15 smithi196 ceph-mon[34718]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-09T02:04:15.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:15 smithi019 ceph-mon[30151]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-09T02:04:17.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:17 smithi019 ceph-mon[30151]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T02:04:17.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:17 smithi059 ceph-mon[34781]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T02:04:17.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:17 smithi196 ceph-mon[34718]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T02:04:19.313 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:19 smithi019 ceph-mon[30151]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:04:19.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:19 smithi059 ceph-mon[34781]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:04:19.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:19 smithi196 ceph-mon[34718]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:04:21.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:21 smithi019 ceph-mon[30151]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:04:21.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T02:04:21.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:21.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:21 smithi059 ceph-mon[34781]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:04:21.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T02:04:21.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:21.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:21 smithi196 ceph-mon[34718]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:04:21.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T02:04:21.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:22.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:22 smithi019 ceph-mon[30151]: Deploying daemon osd.6 on smithi019 2024-05-09T02:04:22.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:22 smithi059 ceph-mon[34781]: Deploying daemon osd.6 on smithi019 2024-05-09T02:04:22.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:22 smithi196 ceph-mon[34718]: Deploying daemon osd.6 on smithi019 2024-05-09T02:04:23.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:23 smithi019 ceph-mon[30151]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:23.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:23 smithi059 ceph-mon[34781]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:23.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:23 smithi196 ceph-mon[34718]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:25.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:25 smithi059 ceph-mon[34781]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:25.509 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:25 smithi019 ceph-mon[30151]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:25.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:25 smithi196 ceph-mon[34718]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:27 smithi059 ceph-mon[34781]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:27 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:27 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:04:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:27 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:27.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:27 smithi196 ceph-mon[34718]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:27.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:27 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:27.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:27 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:04:27.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:27 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:27.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:27 smithi019 ceph-mon[30151]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:27.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:27 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:27.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:27 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:04:27.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:27 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:29.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:29 smithi059 ceph-mon[34781]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:29.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:29 smithi019 ceph-mon[30151]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:29.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:29 smithi196 ceph-mon[34718]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:30.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:30.644+0000 7f3c237fe700 1 -- 172.21.15.19:0/3656277229 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3c2c063c00 con 0x7f3c180b2880 2024-05-09T02:04:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 6 on host 'smithi019' 2024-05-09T02:04:30.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:30.646+0000 7f3c30cc4700 1 -- 172.21.15.19:0/3656277229 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3c180b2880 msgr2=0x7f3c180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:04:30.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:30.646+0000 7f3c30cc4700 1 --2- 172.21.15.19:0/3656277229 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3c180b2880 0x7f3c180b4d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c1053a0 tx=0x7f3c1c009040 comp rx=0 tx=0).stop 2024-05-09T02:04:30.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:30.646+0000 7f3c30cc4700 1 -- 172.21.15.19:0/3656277229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c2c1049e0 msgr2=0x7f3c2c19af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:04:30.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:30.646+0000 7f3c30cc4700 1 --2- 172.21.15.19:0/3656277229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c2c1049e0 0x7f3c2c19af70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3c14006010 tx=0x7f3c140047b0 comp rx=0 tx=0).stop 2024-05-09T02:04:30.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:30.646+0000 7f3c30cc4700 1 -- 172.21.15.19:0/3656277229 shutdown_connections 2024-05-09T02:04:30.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:30.646+0000 7f3c30cc4700 1 --2- 172.21.15.19:0/3656277229 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3c180b2880 0x7f3c180b4d90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:30.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:30.646+0000 7f3c30cc4700 1 --2- 172.21.15.19:0/3656277229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c2c1059b0 0x7f3c2c1a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:30.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:30.646+0000 7f3c30cc4700 1 --2- 172.21.15.19:0/3656277229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c2c103f60 0x7f3c2c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:30.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:30.646+0000 7f3c30cc4700 1 --2- 172.21.15.19:0/3656277229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c2c1049e0 0x7f3c2c19af70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:30.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:30.646+0000 7f3c30cc4700 1 -- 172.21.15.19:0/3656277229 >> 172.21.15.19:0/3656277229 conn(0x7f3c2c0ff9d0 msgr2=0x7f3c2c1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:04:30.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:30.647+0000 7f3c30cc4700 1 -- 172.21.15.19:0/3656277229 shutdown_connections 2024-05-09T02:04:30.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:04:30.647+0000 7f3c30cc4700 1 -- 172.21.15.19:0/3656277229 wait complete. 2024-05-09T02:04:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:30 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:30 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:30 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:30 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:04:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:30 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:30 smithi196 ceph-mon[34718]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:30 smithi196 ceph-mon[34718]: from='osd.6 [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T02:04:30.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:30 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:30.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:30 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:30.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:30 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:30 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:04:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:30 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:30 smithi019 ceph-mon[30151]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:30 smithi019 ceph-mon[30151]: from='osd.6 [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T02:04:30.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:30 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:30.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:30 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:30.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:30 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:30.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:30 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:04:30.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:30 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:30.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:30 smithi059 ceph-mon[34781]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:30.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:30 smithi059 ceph-mon[34781]: from='osd.6 [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T02:04:31.411 DEBUG:teuthology.orchestra.run.smithi019:osd.6> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.6.service 2024-05-09T02:04:31.413 INFO:tasks.cephadm:Deploying osd.7 on smithi059 with /dev/vg_nvme/lv_2... 2024-05-09T02:04:31.414 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T02:04:31.674 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T02:04:31.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:31 smithi059 ceph-mon[34781]: from='osd.6 [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T02:04:31.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:31 smithi059 ceph-mon[34781]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T02:04:31.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:31 smithi059 ceph-mon[34781]: from='osd.6 [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T02:04:31.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:31 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:31.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:31 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:31.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:31 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:04:31.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:31 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:31.722 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:31 smithi019 ceph-mon[30151]: from='osd.6 [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T02:04:31.722 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:31 smithi019 ceph-mon[30151]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T02:04:31.722 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:31 smithi019 ceph-mon[30151]: from='osd.6 [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T02:04:31.722 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:31 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:31.722 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:31 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:31.722 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:31 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:04:31.722 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:31 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:31.722 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:04:31 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:04:31.500+0000 7f7f31b3d700 -1 osd.6 0 waiting for initial osdmap 2024-05-09T02:04:31.723 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:04:31 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:04:31.510+0000 7f7f2d8f2700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T02:04:31.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:31 smithi196 ceph-mon[34718]: from='osd.6 [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T02:04:31.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:31 smithi196 ceph-mon[34718]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T02:04:31.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:31 smithi196 ceph-mon[34718]: from='osd.6 [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T02:04:31.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:31 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:31.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:31 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:31.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:31 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:04:31.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:31 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:32.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:32 smithi196 ceph-mon[34718]: from='osd.6 [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T02:04:32.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:32 smithi196 ceph-mon[34718]: osdmap e42: 7 total, 6 up, 7 in 2024-05-09T02:04:32.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:32 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:32.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:32 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:32.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:32 smithi196 ceph-mon[34718]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:32.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:32 smithi019 ceph-mon[30151]: from='osd.6 [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T02:04:32.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:32 smithi019 ceph-mon[30151]: osdmap e42: 7 total, 6 up, 7 in 2024-05-09T02:04:32.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:32 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:32.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:32 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:32.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:32 smithi019 ceph-mon[30151]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:32.855 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:32 smithi059 ceph-mon[34781]: from='osd.6 [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T02:04:32.855 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:32 smithi059 ceph-mon[34781]: osdmap e42: 7 total, 6 up, 7 in 2024-05-09T02:04:32.855 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:32 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:32.855 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:32 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:32.855 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:32 smithi059 ceph-mon[34781]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-09T02:04:33.735 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T02:04:33.735 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T02:04:33.735 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-05-09T02:04:33.735 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-05-09T02:04:33.735 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111035 s, 944 MB/s 2024-05-09T02:04:33.735 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-05-09T02:04:33.736 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:33 smithi059 ceph-mon[34781]: purged_snaps scrub starts 2024-05-09T02:04:33.736 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:33 smithi059 ceph-mon[34781]: purged_snaps scrub ok 2024-05-09T02:04:33.736 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:33 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:33.736 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:33 smithi059 ceph-mon[34781]: osd.6 [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637] boot 2024-05-09T02:04:33.736 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:33 smithi059 ceph-mon[34781]: osdmap e43: 7 total, 7 up, 7 in 2024-05-09T02:04:33.736 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:33 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:33.787 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:33 smithi019 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T02:04:33.787 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:33 smithi019 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T02:04:33.787 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:33 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:33.787 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:33 smithi019 ceph-mon[30151]: osd.6 [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637] boot 2024-05-09T02:04:33.787 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:33 smithi019 ceph-mon[30151]: osdmap e43: 7 total, 7 up, 7 in 2024-05-09T02:04:33.788 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:33 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:33.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:33 smithi196 ceph-mon[34718]: purged_snaps scrub starts 2024-05-09T02:04:33.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:33 smithi196 ceph-mon[34718]: purged_snaps scrub ok 2024-05-09T02:04:33.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:33 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:33.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:33 smithi196 ceph-mon[34718]: osd.6 [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637] boot 2024-05-09T02:04:33.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:33 smithi196 ceph-mon[34718]: osdmap e43: 7 total, 7 up, 7 in 2024-05-09T02:04:33.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:33 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:04:34.337 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_2 2024-05-09T02:04:34.598 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T02:04:34.630 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:34 smithi059 ceph-mon[34781]: osdmap e44: 7 total, 7 up, 7 in 2024-05-09T02:04:34.630 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:34 smithi059 ceph-mon[34781]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:34.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:34 smithi196 ceph-mon[34718]: osdmap e44: 7 total, 7 up, 7 in 2024-05-09T02:04:34.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:34 smithi196 ceph-mon[34718]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:34.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:34 smithi019 ceph-mon[30151]: osdmap e44: 7 total, 7 up, 7 in 2024-05-09T02:04:34.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:34 smithi019 ceph-mon[30151]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:35.884 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:35 smithi059 ceph-mon[34781]: Detected new or changed devices on smithi019 2024-05-09T02:04:35.884 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:35.884 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:35.884 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:04:35.884 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:04:35.885 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:04:35.885 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:35 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi019 to 5697M 2024-05-09T02:04:35.885 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:35.885 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:35.885 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:04:35.885 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:35 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:35.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:35 smithi196 ceph-mon[34718]: Detected new or changed devices on smithi019 2024-05-09T02:04:35.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:35.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:35.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:04:35.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:04:35.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:04:35.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:35 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi019 to 5697M 2024-05-09T02:04:35.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:35.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:35.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:04:35.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:35 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:36.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:35 smithi019 ceph-mon[30151]: Detected new or changed devices on smithi019 2024-05-09T02:04:36.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:36.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:36.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:04:36.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:04:36.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:04:36.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:35 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi019 to 5697M 2024-05-09T02:04:36.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:36.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:36.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:04:36.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:35 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:36.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.272+0000 7f07c4591700 1 -- 172.21.15.59:0/990951390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07bc103250 msgr2=0x7f07bc103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:04:36.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.272+0000 7f07c4591700 1 --2- 172.21.15.59:0/990951390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07bc103250 0x7f07bc103660 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009990 tx=0x7f07b8009ca0 comp rx=0 tx=0).stop 2024-05-09T02:04:36.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.273+0000 7f07c4591700 1 -- 172.21.15.59:0/990951390 shutdown_connections 2024-05-09T02:04:36.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.273+0000 7f07c4591700 1 --2- 172.21.15.59:0/990951390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07bc104d20 0x7f07bc10b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:36.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.273+0000 7f07c4591700 1 --2- 172.21.15.59:0/990951390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07bc103250 0x7f07bc103660 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:36.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.273+0000 7f07c4591700 1 --2- 172.21.15.59:0/990951390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07bc104380 0x7f07bc104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:36.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.273+0000 7f07c4591700 1 -- 172.21.15.59:0/990951390 >> 172.21.15.59:0/990951390 conn(0x7f07bc0fe890 msgr2=0x7f07bc100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:04:36.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.273+0000 7f07c4591700 1 -- 172.21.15.59:0/990951390 shutdown_connections 2024-05-09T02:04:36.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.273+0000 7f07c4591700 1 -- 172.21.15.59:0/990951390 wait complete. 2024-05-09T02:04:36.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.273+0000 7f07c4591700 1 Processor -- start 2024-05-09T02:04:36.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.273+0000 7f07c4591700 1 -- start start 2024-05-09T02:04:36.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c4591700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07bc103250 0x7f07bc076470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:04:36.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c4591700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07bc104380 0x7f07bc19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:04:36.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c4591700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07bc104d20 0x7f07bc1a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:04:36.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c4591700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc10ec90 con 0x7f07bc104d20 2024-05-09T02:04:36.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c4591700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc10eb10 con 0x7f07bc103250 2024-05-09T02:04:36.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c4591700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc10ee10 con 0x7f07bc104380 2024-05-09T02:04:36.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c232d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07bc103250 0x7f07bc076470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:04:36.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c232d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07bc103250 0x7f07bc076470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53870/0 (socket says 172.21.15.59:53870) 2024-05-09T02:04:36.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c232d700 1 -- 172.21.15.59:0/1641778759 learned_addr learned my addr 172.21.15.59:0/1641778759 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-09T02:04:36.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c1b2c700 1 --2- 172.21.15.59:0/1641778759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07bc104380 0x7f07bc19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:04:36.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c2b2e700 1 --2- 172.21.15.59:0/1641778759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07bc104d20 0x7f07bc1a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:04:36.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c232d700 1 -- 172.21.15.59:0/1641778759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07bc104380 msgr2=0x7f07bc19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:04:36.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c232d700 1 --2- 172.21.15.59:0/1641778759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07bc104380 0x7f07bc19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:36.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c232d700 1 -- 172.21.15.59:0/1641778759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07bc104d20 msgr2=0x7f07bc1a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:04:36.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c232d700 1 --2- 172.21.15.59:0/1641778759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07bc104d20 0x7f07bc1a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:36.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c232d700 1 -- 172.21.15.59:0/1641778759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc1a2460 con 0x7f07bc103250 2024-05-09T02:04:36.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c232d700 1 --2- 172.21.15.59:0/1641778759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07bc103250 0x7f07bc076470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f07b8000c00 tx=0x7f07b80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:04:36.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07b37fe700 1 -- 172.21.15.59:0/1641778759 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8004830 con 0x7f07bc103250 2024-05-09T02:04:36.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.274+0000 7f07c4591700 1 -- 172.21.15.59:0/1641778759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07bc1a26f0 con 0x7f07bc103250 2024-05-09T02:04:36.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.275+0000 7f07b37fe700 1 -- 172.21.15.59:0/1641778759 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b8005020 con 0x7f07bc103250 2024-05-09T02:04:36.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.275+0000 7f07b37fe700 1 -- 172.21.15.59:0/1641778759 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8003d90 con 0x7f07bc103250 2024-05-09T02:04:36.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.275+0000 7f07c4591700 1 -- 172.21.15.59:0/1641778759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07bc1a2b40 con 0x7f07bc103250 2024-05-09T02:04:36.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.276+0000 7f07c4591700 1 -- 172.21.15.59:0/1641778759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07bc0512c0 con 0x7f07bc103250 2024-05-09T02:04:36.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.277+0000 7f07b37fe700 1 -- 172.21.15.59:0/1641778759 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f07b80051d0 con 0x7f07bc103250 2024-05-09T02:04:36.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.278+0000 7f07b37fe700 1 --2- 172.21.15.59:0/1641778759 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f07a80b2880 0x7f07a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:04:36.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.278+0000 7f07b37fe700 1 -- 172.21.15.59:0/1641778759 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7f07b80e23e0 con 0x7f07bc103250 2024-05-09T02:04:36.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.278+0000 7f07c1b2c700 1 --2- 172.21.15.59:0/1641778759 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f07a80b2880 0x7f07a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:04:36.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.278+0000 7f07c1b2c700 1 --2- 172.21.15.59:0/1641778759 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f07a80b2880 0x7f07a80b4d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f07bc075e20 tx=0x7f07ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:04:36.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.283+0000 7f07b37fe700 1 -- 172.21.15.59:0/1641778759 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f07b8013070 con 0x7f07bc103250 2024-05-09T02:04:36.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:36.507+0000 7f07c4591700 1 -- 172.21.15.59:0/1641778759 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f07bc063cc0 con 0x7f07a80b2880 2024-05-09T02:04:36.888 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:36 smithi196 ceph-mon[34718]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:36.888 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:36 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:04:36.888 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:36 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:04:36.888 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:36 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:36.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:36 smithi059 ceph-mon[34781]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:36.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:36 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:04:36.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:36 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:04:36.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:36 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:37.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:36 smithi019 ceph-mon[30151]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:37.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:36 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:04:37.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:36 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:04:37.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:36 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:37.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:37 smithi059 ceph-mon[34781]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:04:38.043 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:37 smithi196 ceph-mon[34718]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:04:38.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:37 smithi019 ceph-mon[30151]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:04:38.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:38 smithi059 ceph-mon[34781]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:39.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:38 smithi196 ceph-mon[34718]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:39.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:38 smithi019 ceph-mon[30151]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:39.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:39 smithi059 ceph-mon[34781]: from='client.? 172.21.15.59:0/1234643159' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83f53f59-9a60-4d87-9a8a-67e5c6f57004"}]: dispatch 2024-05-09T02:04:39.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:39 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83f53f59-9a60-4d87-9a8a-67e5c6f57004"}]: dispatch 2024-05-09T02:04:39.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:39 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83f53f59-9a60-4d87-9a8a-67e5c6f57004"}]': finished 2024-05-09T02:04:39.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:39 smithi059 ceph-mon[34781]: osdmap e45: 8 total, 7 up, 8 in 2024-05-09T02:04:39.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:04:40.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:39 smithi196 ceph-mon[34718]: from='client.? 172.21.15.59:0/1234643159' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83f53f59-9a60-4d87-9a8a-67e5c6f57004"}]: dispatch 2024-05-09T02:04:40.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:39 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83f53f59-9a60-4d87-9a8a-67e5c6f57004"}]: dispatch 2024-05-09T02:04:40.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:39 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83f53f59-9a60-4d87-9a8a-67e5c6f57004"}]': finished 2024-05-09T02:04:40.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:39 smithi196 ceph-mon[34718]: osdmap e45: 8 total, 7 up, 8 in 2024-05-09T02:04:40.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:04:40.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:39 smithi019 ceph-mon[30151]: from='client.? 172.21.15.59:0/1234643159' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83f53f59-9a60-4d87-9a8a-67e5c6f57004"}]: dispatch 2024-05-09T02:04:40.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:39 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83f53f59-9a60-4d87-9a8a-67e5c6f57004"}]: dispatch 2024-05-09T02:04:40.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:39 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83f53f59-9a60-4d87-9a8a-67e5c6f57004"}]': finished 2024-05-09T02:04:40.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:39 smithi019 ceph-mon[30151]: osdmap e45: 8 total, 7 up, 8 in 2024-05-09T02:04:40.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:04:40.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:40 smithi059 ceph-mon[34781]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:40.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:40 smithi059 ceph-mon[34781]: from='client.? 172.21.15.59:0/3436214445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T02:04:41.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:40 smithi196 ceph-mon[34718]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:41.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:40 smithi196 ceph-mon[34718]: from='client.? 172.21.15.59:0/3436214445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T02:04:41.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:40 smithi019 ceph-mon[30151]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:41.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:40 smithi019 ceph-mon[30151]: from='client.? 172.21.15.59:0/3436214445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T02:04:43.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:43 smithi196 ceph-mon[34718]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:43.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:43 smithi019 ceph-mon[30151]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:43 smithi059 ceph-mon[34781]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:45.215 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:45 smithi059 ceph-mon[34781]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:45.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:45 smithi196 ceph-mon[34718]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:45.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:45 smithi019 ceph-mon[30151]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:47.215 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:47 smithi059 ceph-mon[34781]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:47.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:47 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T02:04:47.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:47 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:47.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:47 smithi196 ceph-mon[34718]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:47.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:47 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T02:04:47.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:47 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:47.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:47 smithi019 ceph-mon[30151]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:47.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T02:04:47.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:47 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:04:48.215 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:48 smithi059 ceph-mon[34781]: Deploying daemon osd.7 on smithi059 2024-05-09T02:04:48.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:48 smithi196 ceph-mon[34718]: Deploying daemon osd.7 on smithi059 2024-05-09T02:04:48.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:48 smithi019 ceph-mon[30151]: Deploying daemon osd.7 on smithi059 2024-05-09T02:04:49.197 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:49 smithi059 ceph-mon[34781]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:49.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:49 smithi196 ceph-mon[34718]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:49.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:49 smithi019 ceph-mon[30151]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:51.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:51 smithi019 ceph-mon[30151]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:51.414 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:51 smithi059 ceph-mon[34781]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:51.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:51 smithi196 ceph-mon[34718]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:53.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:53 smithi019 ceph-mon[30151]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:53.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:53 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:53.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:53 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:53.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:53 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:04:53.340 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:53 smithi059 ceph-mon[34781]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:53.340 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:53 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:53.340 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:53 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:53.340 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:53 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:04:53.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:53 smithi196 ceph-mon[34718]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:53.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:53 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:53.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:53 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:53.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:53 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:04:55.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:55 smithi059 ceph-mon[34781]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:55.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:55 smithi019 ceph-mon[30151]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:55.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:55 smithi196 ceph-mon[34718]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:56.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:56.328+0000 7f07b37fe700 1 -- 172.21.15.59:0/1641778759 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f07bc063cc0 con 0x7f07a80b2880 2024-05-09T02:04:56.330 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 7 on host 'smithi059' 2024-05-09T02:04:56.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:56.330+0000 7f07c4591700 1 -- 172.21.15.59:0/1641778759 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f07a80b2880 msgr2=0x7f07a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:04:56.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:56.330+0000 7f07c4591700 1 --2- 172.21.15.59:0/1641778759 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f07a80b2880 0x7f07a80b4d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f07bc075e20 tx=0x7f07ac00b410 comp rx=0 tx=0).stop 2024-05-09T02:04:56.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:56.330+0000 7f07c4591700 1 -- 172.21.15.59:0/1641778759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07bc103250 msgr2=0x7f07bc076470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:04:56.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:56.330+0000 7f07c4591700 1 --2- 172.21.15.59:0/1641778759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07bc103250 0x7f07bc076470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f07b8000c00 tx=0x7f07b80047b0 comp rx=0 tx=0).stop 2024-05-09T02:04:56.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:56.330+0000 7f07c4591700 1 -- 172.21.15.59:0/1641778759 shutdown_connections 2024-05-09T02:04:56.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:56.331+0000 7f07c4591700 1 --2- 172.21.15.59:0/1641778759 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f07a80b2880 0x7f07a80b4d90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:56.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:56.331+0000 7f07c4591700 1 --2- 172.21.15.59:0/1641778759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07bc104d20 0x7f07bc1a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:56.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:56.331+0000 7f07c4591700 1 --2- 172.21.15.59:0/1641778759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07bc103250 0x7f07bc076470 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:56.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:56.331+0000 7f07c4591700 1 --2- 172.21.15.59:0/1641778759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07bc104380 0x7f07bc19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:04:56.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:56.331+0000 7f07c4591700 1 -- 172.21.15.59:0/1641778759 >> 172.21.15.59:0/1641778759 conn(0x7f07bc0fe890 msgr2=0x7f07bc1075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:04:56.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:56.331+0000 7f07c4591700 1 -- 172.21.15.59:0/1641778759 shutdown_connections 2024-05-09T02:04:56.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:04:56.331+0000 7f07c4591700 1 -- 172.21.15.59:0/1641778759 wait complete. 2024-05-09T02:04:56.961 DEBUG:teuthology.orchestra.run.smithi059:osd.7> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.7.service 2024-05-09T02:04:56.963 INFO:tasks.cephadm:Deploying osd.8 on smithi196 with /dev/vg_nvme/lv_2... 2024-05-09T02:04:56.963 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T02:04:57.210 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:04:57.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:57 smithi019 ceph-mon[30151]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:57.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:57 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:57.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:57 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:57.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:57 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:57.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:57 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:57.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:57 smithi019 ceph-mon[30151]: from='osd.7 [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T02:04:57.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:57 smithi019 ceph-mon[30151]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T02:04:57.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:57 smithi196 ceph-mon[34718]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:57.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:57 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:57.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:57 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:57.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:57 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:57.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:57 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:57.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:57 smithi196 ceph-mon[34718]: from='osd.7 [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T02:04:57.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:57 smithi196 ceph-mon[34718]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T02:04:57.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:57 smithi059 ceph-mon[34781]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:57.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:57 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:57.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:57 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:57.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:57 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:57.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:57 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:04:57.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:57 smithi059 ceph-mon[34781]: from='osd.7 [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T02:04:57.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:57 smithi059 ceph-mon[34781]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T02:04:58.633 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:58 smithi196 ceph-mon[34718]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T02:04:58.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:58 smithi196 ceph-mon[34718]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T02:04:58.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:58 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:04:58.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:58 smithi196 ceph-mon[34718]: from='osd.7 [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:04:58.634 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:58 smithi196 ceph-mon[34718]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:04:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:58 smithi059 ceph-mon[34781]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T02:04:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:58 smithi059 ceph-mon[34781]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T02:04:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:58 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:04:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:58 smithi059 ceph-mon[34781]: from='osd.7 [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:04:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:58 smithi059 ceph-mon[34781]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:04:58.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:58 smithi019 ceph-mon[30151]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T02:04:58.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:58 smithi019 ceph-mon[30151]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T02:04:58.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:58 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:04:58.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:58 smithi019 ceph-mon[30151]: from='osd.7 [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:04:58.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:58 smithi019 ceph-mon[30151]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:04:59.196 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T02:04:59.196 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T02:04:59.196 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-05-09T02:04:59.196 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-05-09T02:04:59.196 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130393 s, 804 MB/s 2024-05-09T02:04:59.196 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-05-09T02:04:59.215 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:04:58 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:04:58.855+0000 7fba4908b700 -1 osd.7 0 waiting for initial osdmap 2024-05-09T02:04:59.215 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:04:58 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:04:58.867+0000 7fba40e38700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T02:04:59.597 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:59 smithi196 ceph-mon[34718]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:59.597 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:59 smithi196 ceph-mon[34718]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T02:04:59.597 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:59 smithi196 ceph-mon[34718]: osdmap e47: 8 total, 7 up, 8 in 2024-05-09T02:04:59.597 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:59 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:04:59.597 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:59 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:04:59.597 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:59 smithi196 ceph-mon[34718]: from='osd.7 ' entity='osd.7' 2024-05-09T02:04:59.597 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:59 smithi196 ceph-mon[34718]: osd.7 [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053] boot 2024-05-09T02:04:59.598 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:59 smithi196 ceph-mon[34718]: osdmap e48: 8 total, 8 up, 8 in 2024-05-09T02:04:59.598 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:04:59 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:04:59.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:59 smithi059 ceph-mon[34781]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:59.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:59 smithi059 ceph-mon[34781]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T02:04:59.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:59 smithi059 ceph-mon[34781]: osdmap e47: 8 total, 7 up, 8 in 2024-05-09T02:04:59.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:59 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:04:59.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:59 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:04:59.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:59 smithi059 ceph-mon[34781]: from='osd.7 ' entity='osd.7' 2024-05-09T02:04:59.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:59 smithi059 ceph-mon[34781]: osd.7 [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053] boot 2024-05-09T02:04:59.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:59 smithi059 ceph-mon[34781]: osdmap e48: 8 total, 8 up, 8 in 2024-05-09T02:04:59.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:04:59 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:04:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:59 smithi019 ceph-mon[30151]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-09T02:04:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:59 smithi019 ceph-mon[30151]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T02:04:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:59 smithi019 ceph-mon[30151]: osdmap e47: 8 total, 7 up, 8 in 2024-05-09T02:04:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:59 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:04:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:59 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:04:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:59 smithi019 ceph-mon[30151]: from='osd.7 ' entity='osd.7' 2024-05-09T02:04:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:59 smithi019 ceph-mon[30151]: osd.7 [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053] boot 2024-05-09T02:04:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:59 smithi019 ceph-mon[30151]: osdmap e48: 8 total, 8 up, 8 in 2024-05-09T02:04:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:04:59 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:04:59.924 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_2 2024-05-09T02:05:00.183 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:05:00.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:00 smithi196 ceph-mon[34718]: purged_snaps scrub starts 2024-05-09T02:05:00.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:00 smithi196 ceph-mon[34718]: purged_snaps scrub ok 2024-05-09T02:05:00.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:00 smithi196 ceph-mon[34718]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T02:05:00.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:00 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:00.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:00 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:00.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:00 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:00.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:00 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:00.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:00 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:00.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:00 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:00.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:00 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:00.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:00 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:00.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:00 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:00 smithi059 ceph-mon[34781]: purged_snaps scrub starts 2024-05-09T02:05:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:00 smithi059 ceph-mon[34781]: purged_snaps scrub ok 2024-05-09T02:05:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:00 smithi059 ceph-mon[34781]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T02:05:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:00 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:00 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:00 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:00 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:00 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:00 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:00 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:00 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:00 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:00.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:00 smithi019 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T02:05:00.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:00 smithi019 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T02:05:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:00 smithi019 ceph-mon[30151]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T02:05:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:00 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:00 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:00 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:00 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:00 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:00 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:00 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:00 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:00 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:01.624 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:01 smithi196 ceph-mon[34718]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:01.624 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:01 smithi196 ceph-mon[34718]: Detected new or changed devices on smithi059 2024-05-09T02:05:01.624 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:01 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi059 to 5697M 2024-05-09T02:05:01.624 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:01 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:05:01.624 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:01 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:01.624 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:01 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:01.624 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:01 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:01.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:01 smithi059 ceph-mon[34781]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:01.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:01 smithi059 ceph-mon[34781]: Detected new or changed devices on smithi059 2024-05-09T02:05:01.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:01 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi059 to 5697M 2024-05-09T02:05:01.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:01 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:05:01.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:01 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:01.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:01 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:01.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:01 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:01.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:01 smithi019 ceph-mon[30151]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:01 smithi019 ceph-mon[30151]: Detected new or changed devices on smithi059 2024-05-09T02:05:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:01 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi059 to 5697M 2024-05-09T02:05:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:01 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:05:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:01 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:01 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:01 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:01.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.854+0000 7fae8378a700 1 -- 172.21.15.196:0/3343587312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae7c1049e0 msgr2=0x7fae7c10c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:05:01.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.854+0000 7fae8378a700 1 --2- 172.21.15.196:0/3343587312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae7c1049e0 0x7fae7c10c6f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fae7400b0a0 tx=0x7fae7400b3b0 comp rx=0 tx=0).stop 2024-05-09T02:05:01.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.854+0000 7fae8378a700 1 -- 172.21.15.196:0/3343587312 shutdown_connections 2024-05-09T02:05:01.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.854+0000 7fae8378a700 1 --2- 172.21.15.196:0/3343587312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae7c1049e0 0x7fae7c10c6f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:01.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.854+0000 7fae8378a700 1 --2- 172.21.15.196:0/3343587312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae7c1059b0 0x7fae7c105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:01.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.854+0000 7fae8378a700 1 --2- 172.21.15.196:0/3343587312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae7c103f60 0x7fae7c104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:01.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.854+0000 7fae8378a700 1 -- 172.21.15.196:0/3343587312 >> 172.21.15.196:0/3343587312 conn(0x7fae7c0ff9d0 msgr2=0x7fae7c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:05:01.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.854+0000 7fae8378a700 1 -- 172.21.15.196:0/3343587312 shutdown_connections 2024-05-09T02:05:01.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.854+0000 7fae8378a700 1 -- 172.21.15.196:0/3343587312 wait complete. 2024-05-09T02:05:01.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae8378a700 1 Processor -- start 2024-05-09T02:05:01.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae8378a700 1 -- start start 2024-05-09T02:05:01.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae8378a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae7c103f60 0x7fae7c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:05:01.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae8378a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae7c1049e0 0x7fae7c19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:05:01.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae8378a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae7c1059b0 0x7fae7c1a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:05:01.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae8378a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c10ff70 con 0x7fae7c1059b0 2024-05-09T02:05:01.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae8378a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c10fdf0 con 0x7fae7c103f60 2024-05-09T02:05:01.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae8378a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c1100f0 con 0x7fae7c1049e0 2024-05-09T02:05:01.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae80d25700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae7c1049e0 0x7fae7c19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:05:01.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae80d25700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae7c1049e0 0x7fae7c19af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:33508/0 (socket says 172.21.15.196:33508) 2024-05-09T02:05:01.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae80d25700 1 -- 172.21.15.196:0/1941688404 learned_addr learned my addr 172.21.15.196:0/1941688404 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T02:05:01.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae81526700 1 --2- 172.21.15.196:0/1941688404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae7c103f60 0x7fae7c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:05:01.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae81d27700 1 --2- 172.21.15.196:0/1941688404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae7c1059b0 0x7fae7c1a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:05:01.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae80d25700 1 -- 172.21.15.196:0/1941688404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae7c103f60 msgr2=0x7fae7c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:05:01.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae80d25700 1 --2- 172.21.15.196:0/1941688404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae7c103f60 0x7fae7c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:01.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae80d25700 1 -- 172.21.15.196:0/1941688404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae7c1059b0 msgr2=0x7fae7c1a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:05:01.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae80d25700 1 --2- 172.21.15.196:0/1941688404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae7c1059b0 0x7fae7c1a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:01.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae80d25700 1 -- 172.21.15.196:0/1941688404 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7c1a2560 con 0x7fae7c1049e0 2024-05-09T02:05:01.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.855+0000 7fae80d25700 1 --2- 172.21.15.196:0/1941688404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae7c1049e0 0x7fae7c19af40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00c920 tx=0x7fae6c00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:05:01.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.856+0000 7fae727fc700 1 -- 172.21.15.196:0/1941688404 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c007b00 con 0x7fae7c1049e0 2024-05-09T02:05:01.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.856+0000 7fae8378a700 1 -- 172.21.15.196:0/1941688404 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7c1a2850 con 0x7fae7c1049e0 2024-05-09T02:05:01.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.856+0000 7fae727fc700 1 -- 172.21.15.196:0/1941688404 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae6c004ce0 con 0x7fae7c1049e0 2024-05-09T02:05:01.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.856+0000 7fae727fc700 1 -- 172.21.15.196:0/1941688404 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c005500 con 0x7fae7c1049e0 2024-05-09T02:05:01.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.856+0000 7fae8378a700 1 -- 172.21.15.196:0/1941688404 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae7c1a2e10 con 0x7fae7c1049e0 2024-05-09T02:05:01.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.857+0000 7fae8378a700 1 -- 172.21.15.196:0/1941688404 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7c0512c0 con 0x7fae7c1049e0 2024-05-09T02:05:01.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.858+0000 7fae727fc700 1 -- 172.21.15.196:0/1941688404 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fae6c007ca0 con 0x7fae7c1049e0 2024-05-09T02:05:01.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.859+0000 7fae727fc700 1 --2- 172.21.15.196:0/1941688404 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fae680b2880 0x7fae680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:05:01.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.859+0000 7fae727fc700 1 -- 172.21.15.196:0/1941688404 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7fae6c0e1640 con 0x7fae7c1049e0 2024-05-09T02:05:01.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.859+0000 7fae81526700 1 --2- 172.21.15.196:0/1941688404 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fae680b2880 0x7fae680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:05:01.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.859+0000 7fae81526700 1 --2- 172.21.15.196:0/1941688404 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fae680b2880 0x7fae680b4d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fae7c1053a0 tx=0x7fae78009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:05:01.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:01.864+0000 7fae727fc700 1 -- 172.21.15.196:0/1941688404 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae6c014030 con 0x7fae7c1049e0 2024-05-09T02:05:02.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:02.073+0000 7fae8378a700 1 -- 172.21.15.196:0/1941688404 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fae7c02e9e0 con 0x7fae680b2880 2024-05-09T02:05:02.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:02 smithi196 ceph-mon[34718]: osdmap e50: 8 total, 8 up, 8 in 2024-05-09T02:05:02.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:05:02.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:05:02.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:02.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:02 smithi059 ceph-mon[34781]: osdmap e50: 8 total, 8 up, 8 in 2024-05-09T02:05:02.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:05:02.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:05:02.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:02 smithi019 ceph-mon[30151]: osdmap e50: 8 total, 8 up, 8 in 2024-05-09T02:05:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:05:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:05:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:03.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:03 smithi196 ceph-mon[34718]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:03.695 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:03 smithi196 ceph-mon[34718]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:05:03.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:03 smithi059 ceph-mon[34781]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:03.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:03 smithi059 ceph-mon[34781]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:05:03.722 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:03 smithi019 ceph-mon[30151]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:03.722 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:03 smithi019 ceph-mon[30151]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:05:05.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:05 smithi196 ceph-mon[34718]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:05.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:05 smithi196 ceph-mon[34718]: from='client.? 172.21.15.196:0/1007082439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8f1bfb5-1cc8-4a65-8396-239bd781e1e8"}]: dispatch 2024-05-09T02:05:05.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:05 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8f1bfb5-1cc8-4a65-8396-239bd781e1e8"}]: dispatch 2024-05-09T02:05:05.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:05 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8f1bfb5-1cc8-4a65-8396-239bd781e1e8"}]': finished 2024-05-09T02:05:05.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:05 smithi196 ceph-mon[34718]: osdmap e51: 9 total, 8 up, 9 in 2024-05-09T02:05:05.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:05 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:05:05.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:05 smithi059 ceph-mon[34781]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:05.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:05 smithi059 ceph-mon[34781]: from='client.? 172.21.15.196:0/1007082439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8f1bfb5-1cc8-4a65-8396-239bd781e1e8"}]: dispatch 2024-05-09T02:05:05.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:05 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8f1bfb5-1cc8-4a65-8396-239bd781e1e8"}]: dispatch 2024-05-09T02:05:05.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:05 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8f1bfb5-1cc8-4a65-8396-239bd781e1e8"}]': finished 2024-05-09T02:05:05.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:05 smithi059 ceph-mon[34781]: osdmap e51: 9 total, 8 up, 9 in 2024-05-09T02:05:05.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:05 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:05:05.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:05 smithi019 ceph-mon[30151]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:05.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:05 smithi019 ceph-mon[30151]: from='client.? 172.21.15.196:0/1007082439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8f1bfb5-1cc8-4a65-8396-239bd781e1e8"}]: dispatch 2024-05-09T02:05:05.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:05 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8f1bfb5-1cc8-4a65-8396-239bd781e1e8"}]: dispatch 2024-05-09T02:05:05.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:05 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8f1bfb5-1cc8-4a65-8396-239bd781e1e8"}]': finished 2024-05-09T02:05:05.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:05 smithi019 ceph-mon[30151]: osdmap e51: 9 total, 8 up, 9 in 2024-05-09T02:05:05.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:05 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:05:06.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:06 smithi059 ceph-mon[34781]: from='client.? 172.21.15.196:0/607461026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T02:05:06.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:06 smithi196 ceph-mon[34718]: from='client.? 172.21.15.196:0/607461026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T02:05:06.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:06 smithi019 ceph-mon[30151]: from='client.? 172.21.15.196:0/607461026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T02:05:07.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:07 smithi059 ceph-mon[34781]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T02:05:07.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:07 smithi196 ceph-mon[34718]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T02:05:07.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:07 smithi019 ceph-mon[30151]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T02:05:08.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:08 smithi196 ceph-mon[34718]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-09T02:05:08.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:08 smithi019 ceph-mon[30151]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-09T02:05:08.965 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:08 smithi059 ceph-mon[34781]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-09T02:05:11.153 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:11 smithi196 ceph-mon[34718]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-09T02:05:11.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:11 smithi019 ceph-mon[30151]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-09T02:05:11.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:11 smithi059 ceph-mon[34781]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-09T02:05:13.192 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:13 smithi196 ceph-mon[34718]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:05:13.193 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:13 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T02:05:13.193 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:13 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:13.193 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:13 smithi196 ceph-mon[34718]: Deploying daemon osd.8 on smithi196 2024-05-09T02:05:13.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:13 smithi019 ceph-mon[30151]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:05:13.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:13 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T02:05:13.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:13 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:13.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:13 smithi019 ceph-mon[30151]: Deploying daemon osd.8 on smithi196 2024-05-09T02:05:13.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:13 smithi059 ceph-mon[34781]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:05:13.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:13 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T02:05:13.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:13 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:13.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:13 smithi059 ceph-mon[34781]: Deploying daemon osd.8 on smithi196 2024-05-09T02:05:15.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:15 smithi196 ceph-mon[34718]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:05:15.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:15 smithi019 ceph-mon[30151]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:05:15.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:15 smithi059 ceph-mon[34781]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:05:17.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:17 smithi059 ceph-mon[34781]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:17.228 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:17 smithi196 ceph-mon[34718]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:17.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:17 smithi019 ceph-mon[30151]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:19.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:19 smithi059 ceph-mon[34781]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:19.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:19 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:19.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:19 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:19.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:19 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:05:19.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:19 smithi196 ceph-mon[34718]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:19.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:19 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:19.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:19 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:19.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:19 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:05:19.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:19 smithi019 ceph-mon[30151]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:19.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:19 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:19.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:19 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:19.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:19 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:05:21.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:21 smithi059 ceph-mon[34781]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:21 smithi196 ceph-mon[34718]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:21.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:21 smithi019 ceph-mon[30151]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:22.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:22.194+0000 7fae727fc700 1 -- 172.21.15.196:0/1941688404 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fae7c02e9e0 con 0x7fae680b2880 2024-05-09T02:05:22.196 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 8 on host 'smithi196' 2024-05-09T02:05:22.198 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:22.197+0000 7fae8378a700 1 -- 172.21.15.196:0/1941688404 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fae680b2880 msgr2=0x7fae680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:05:22.198 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:22.197+0000 7fae8378a700 1 --2- 172.21.15.196:0/1941688404 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fae680b2880 0x7fae680b4d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fae7c1053a0 tx=0x7fae78009190 comp rx=0 tx=0).stop 2024-05-09T02:05:22.199 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:22.197+0000 7fae8378a700 1 -- 172.21.15.196:0/1941688404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae7c1049e0 msgr2=0x7fae7c19af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:05:22.199 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:22.197+0000 7fae8378a700 1 --2- 172.21.15.196:0/1941688404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae7c1049e0 0x7fae7c19af40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00c920 tx=0x7fae6c00cc30 comp rx=0 tx=0).stop 2024-05-09T02:05:22.199 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:22.197+0000 7fae8378a700 1 -- 172.21.15.196:0/1941688404 shutdown_connections 2024-05-09T02:05:22.199 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:22.197+0000 7fae8378a700 1 --2- 172.21.15.196:0/1941688404 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fae680b2880 0x7fae680b4d90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:22.199 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:22.197+0000 7fae8378a700 1 --2- 172.21.15.196:0/1941688404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae7c1059b0 0x7fae7c1a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:22.199 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:22.197+0000 7fae8378a700 1 --2- 172.21.15.196:0/1941688404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae7c103f60 0x7fae7c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:22.199 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:22.197+0000 7fae8378a700 1 --2- 172.21.15.196:0/1941688404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae7c1049e0 0x7fae7c19af40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:22.199 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:22.197+0000 7fae8378a700 1 -- 172.21.15.196:0/1941688404 >> 172.21.15.196:0/1941688404 conn(0x7fae7c0ff9d0 msgr2=0x7fae7c1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:05:22.199 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:22.197+0000 7fae8378a700 1 -- 172.21.15.196:0/1941688404 shutdown_connections 2024-05-09T02:05:22.199 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:05:22.197+0000 7fae8378a700 1 -- 172.21.15.196:0/1941688404 wait complete. 2024-05-09T02:05:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:22 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:22 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:22 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:22 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:22 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:22 smithi059 ceph-mon[34781]: from='osd.8 [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T02:05:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:22 smithi059 ceph-mon[34781]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T02:05:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:22 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:22 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:22.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:22 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:22.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:22 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:22.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:22 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:22.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:22 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:22.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:22 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:22.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:22 smithi196 ceph-mon[34718]: from='osd.8 [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T02:05:22.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:22 smithi196 ceph-mon[34718]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T02:05:22.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:22 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:22.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:22 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:22.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:22 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:22.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:22 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:22.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:22 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:22.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:22 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:22.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:22 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:22.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:22 smithi019 ceph-mon[30151]: from='osd.8 [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T02:05:22.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:22 smithi019 ceph-mon[30151]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T02:05:22.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:22 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:22.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:22 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:23.050 DEBUG:teuthology.orchestra.run.smithi196:osd.8> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.8.service 2024-05-09T02:05:23.052 INFO:tasks.cephadm:Deploying osd.9 on smithi019 with /dev/vg_nvme/lv_1... 2024-05-09T02:05:23.053 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T02:05:23.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:23 smithi196 ceph-mon[34718]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:23.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:23 smithi196 ceph-mon[34718]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T02:05:23.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:23 smithi196 ceph-mon[34718]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T02:05:23.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:05:23.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:23 smithi196 ceph-mon[34718]: from='osd.8 [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:05:23.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:23 smithi196 ceph-mon[34718]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:05:23.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:05:23.291 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:05:23 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:05:23.224+0000 7f406afe2700 -1 osd.8 0 waiting for initial osdmap 2024-05-09T02:05:23.292 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:05:23 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:05:23.234+0000 7f4066d97700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T02:05:23.323 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:05:23.446 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:23 smithi019 ceph-mon[30151]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:23.446 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:23 smithi019 ceph-mon[30151]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T02:05:23.446 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:23 smithi019 ceph-mon[30151]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T02:05:23.446 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:05:23.446 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:23 smithi019 ceph-mon[30151]: from='osd.8 [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:05:23.447 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:23 smithi019 ceph-mon[30151]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:05:23.447 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:05:23.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:23 smithi059 ceph-mon[34781]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:23.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:23 smithi059 ceph-mon[34781]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T02:05:23.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:23 smithi059 ceph-mon[34781]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T02:05:23.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:05:23.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:23 smithi059 ceph-mon[34781]: from='osd.8 [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:05:23.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:23 smithi059 ceph-mon[34781]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:05:23.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:05:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:24 smithi196 ceph-mon[34718]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T02:05:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:24 smithi196 ceph-mon[34718]: osdmap e53: 9 total, 8 up, 9 in 2024-05-09T02:05:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:05:24.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:05:24.488 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:24 smithi019 ceph-mon[30151]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T02:05:24.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:24 smithi019 ceph-mon[30151]: osdmap e53: 9 total, 8 up, 9 in 2024-05-09T02:05:24.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:05:24.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:05:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:24 smithi059 ceph-mon[34781]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T02:05:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:24 smithi059 ceph-mon[34781]: osdmap e53: 9 total, 8 up, 9 in 2024-05-09T02:05:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:05:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:05:25.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:25 smithi019 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T02:05:25.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:25 smithi019 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T02:05:25.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:25 smithi019 ceph-mon[30151]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:25.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:25 smithi019 ceph-mon[30151]: osd.8 [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259] boot 2024-05-09T02:05:25.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:25 smithi019 ceph-mon[30151]: osdmap e54: 9 total, 9 up, 9 in 2024-05-09T02:05:25.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:05:25.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:25 smithi196 ceph-mon[34718]: purged_snaps scrub starts 2024-05-09T02:05:25.482 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:25 smithi196 ceph-mon[34718]: purged_snaps scrub ok 2024-05-09T02:05:25.482 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:25 smithi196 ceph-mon[34718]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:25.482 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:25 smithi196 ceph-mon[34718]: osd.8 [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259] boot 2024-05-09T02:05:25.482 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:25 smithi196 ceph-mon[34718]: osdmap e54: 9 total, 9 up, 9 in 2024-05-09T02:05:25.482 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:05:25.588 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T02:05:25.588 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T02:05:25.588 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-09T02:05:25.588 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-09T02:05:25.588 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114103 s, 919 MB/s 2024-05-09T02:05:25.588 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-09T02:05:25.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:25 smithi059 ceph-mon[34781]: purged_snaps scrub starts 2024-05-09T02:05:25.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:25 smithi059 ceph-mon[34781]: purged_snaps scrub ok 2024-05-09T02:05:25.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:25 smithi059 ceph-mon[34781]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-09T02:05:25.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:25 smithi059 ceph-mon[34781]: osd.8 [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259] boot 2024-05-09T02:05:25.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:25 smithi059 ceph-mon[34781]: osdmap e54: 9 total, 9 up, 9 in 2024-05-09T02:05:25.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:05:26.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:26 smithi019 ceph-mon[30151]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T02:05:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:26 smithi196 ceph-mon[34718]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T02:05:26.558 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_1 2024-05-09T02:05:26.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:26 smithi059 ceph-mon[34781]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T02:05:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:05:27.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:27 smithi196 ceph-mon[34718]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:27.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:27 smithi196 ceph-mon[34718]: Detected new or changed devices on smithi196 2024-05-09T02:05:27.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:27 smithi196 ceph-mon[34718]: osdmap e56: 9 total, 9 up, 9 in 2024-05-09T02:05:27.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:27 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:27.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:27 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:27.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:27 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:27.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:27 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:27.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:27 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:27.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:27 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi196 to 5696M 2024-05-09T02:05:27.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:27 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:27.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:27 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:27.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:27 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:27.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:27 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:27 smithi019 ceph-mon[30151]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:27 smithi019 ceph-mon[30151]: Detected new or changed devices on smithi196 2024-05-09T02:05:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:27 smithi019 ceph-mon[30151]: osdmap e56: 9 total, 9 up, 9 in 2024-05-09T02:05:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:27 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:27 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:27 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:27 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:27 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:27 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi196 to 5696M 2024-05-09T02:05:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:27 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:27.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:27 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:27.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:27 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:27.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:27 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:27 smithi059 ceph-mon[34781]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:27 smithi059 ceph-mon[34781]: Detected new or changed devices on smithi196 2024-05-09T02:05:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:27 smithi059 ceph-mon[34781]: osdmap e56: 9 total, 9 up, 9 in 2024-05-09T02:05:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:27 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:27 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:27 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:27 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:27 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:27 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi196 to 5696M 2024-05-09T02:05:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:27 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:27 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:27 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:27.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:27 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.389+0000 7ff4b9fdb700 1 -- 172.21.15.19:0/2115015046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4b4104d00 msgr2=0x7ff4b410b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:05:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.389+0000 7ff4b9fdb700 1 --2- 172.21.15.19:0/2115015046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4b4104d00 0x7ff4b410b5b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00b0a0 tx=0x7ff4ac00b3b0 comp rx=0 tx=0).stop 2024-05-09T02:05:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.389+0000 7ff4b9fdb700 1 -- 172.21.15.19:0/2115015046 shutdown_connections 2024-05-09T02:05:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.389+0000 7ff4b9fdb700 1 --2- 172.21.15.19:0/2115015046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4b4104d00 0x7ff4b410b5b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.389+0000 7ff4b9fdb700 1 --2- 172.21.15.19:0/2115015046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b4103230 0x7ff4b4103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.389+0000 7ff4b9fdb700 1 --2- 172.21.15.19:0/2115015046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b4104360 0x7ff4b4104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.389+0000 7ff4b9fdb700 1 -- 172.21.15.19:0/2115015046 >> 172.21.15.19:0/2115015046 conn(0x7ff4b40fe870 msgr2=0x7ff4b4100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:05:28.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.389+0000 7ff4b9fdb700 1 -- 172.21.15.19:0/2115015046 shutdown_connections 2024-05-09T02:05:28.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.389+0000 7ff4b9fdb700 1 -- 172.21.15.19:0/2115015046 wait complete. 2024-05-09T02:05:28.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.389+0000 7ff4b9fdb700 1 Processor -- start 2024-05-09T02:05:28.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4b9fdb700 1 -- start start 2024-05-09T02:05:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4b9fdb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b4103230 0x7ff4b4106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:05:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4b9fdb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4b4104360 0x7ff4b41056a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:05:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4b9fdb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b4104d00 0x7ff4b4105be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:05:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4b9fdb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b410ecb0 con 0x7ff4b4104360 2024-05-09T02:05:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4b9fdb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b410eb30 con 0x7ff4b4103230 2024-05-09T02:05:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4b9fdb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b410ee30 con 0x7ff4b4104d00 2024-05-09T02:05:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4aaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4b4104360 0x7ff4b41056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:05:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4aaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4b4104360 0x7ff4b41056a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45440/0 (socket says 172.21.15.19:45440) 2024-05-09T02:05:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4aaffd700 1 -- 172.21.15.19:0/4227352654 learned_addr learned my addr 172.21.15.19:0/4227352654 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:05:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4b3fff700 1 --2- 172.21.15.19:0/4227352654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b4104d00 0x7ff4b4105be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:05:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4b37fe700 1 --2- 172.21.15.19:0/4227352654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b4103230 0x7ff4b4106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:05:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4aaffd700 1 -- 172.21.15.19:0/4227352654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b4104d00 msgr2=0x7ff4b4105be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:05:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4aaffd700 1 --2- 172.21.15.19:0/4227352654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b4104d00 0x7ff4b4105be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:28.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4aaffd700 1 -- 172.21.15.19:0/4227352654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b4103230 msgr2=0x7ff4b4106ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:05:28.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4aaffd700 1 --2- 172.21.15.19:0/4227352654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b4103230 0x7ff4b4106ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:28.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.390+0000 7ff4aaffd700 1 -- 172.21.15.19:0/4227352654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b4106340 con 0x7ff4b4104360 2024-05-09T02:05:28.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.391+0000 7ff4aaffd700 1 --2- 172.21.15.19:0/4227352654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4b4104360 0x7ff4b41056a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00d950 tx=0x7ff49c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:05:28.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.391+0000 7ff4b17fa700 1 -- 172.21.15.19:0/4227352654 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c016070 con 0x7ff4b4104360 2024-05-09T02:05:28.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.391+0000 7ff4b9fdb700 1 -- 172.21.15.19:0/4227352654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b41a49c0 con 0x7ff4b4104360 2024-05-09T02:05:28.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.391+0000 7ff4b17fa700 1 -- 172.21.15.19:0/4227352654 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff49c00fd40 con 0x7ff4b4104360 2024-05-09T02:05:28.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.391+0000 7ff4b17fa700 1 -- 172.21.15.19:0/4227352654 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00e3b0 con 0x7ff4b4104360 2024-05-09T02:05:28.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.391+0000 7ff4b9fdb700 1 -- 172.21.15.19:0/4227352654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b41a4ef0 con 0x7ff4b4104360 2024-05-09T02:05:28.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.392+0000 7ff4b9fdb700 1 -- 172.21.15.19:0/4227352654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b41a50c0 con 0x7ff4b4104360 2024-05-09T02:05:28.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.394+0000 7ff4b17fa700 1 -- 172.21.15.19:0/4227352654 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff49c019070 con 0x7ff4b4104360 2024-05-09T02:05:28.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.394+0000 7ff4b17fa700 1 --2- 172.21.15.19:0/4227352654 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff4a00b2880 0x7ff4a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:05:28.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.395+0000 7ff4b37fe700 1 --2- 172.21.15.19:0/4227352654 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff4a00b2880 0x7ff4a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:05:28.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.395+0000 7ff4b17fa700 1 -- 172.21.15.19:0/4227352654 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7ff49c0e12b0 con 0x7ff4b4104360 2024-05-09T02:05:28.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.395+0000 7ff4b37fe700 1 --2- 172.21.15.19:0/4227352654 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff4a00b2880 0x7ff4a00b4d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4104260 tx=0x7ff4a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:05:28.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.398+0000 7ff4b17fa700 1 -- 172.21.15.19:0/4227352654 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff49c0a9980 con 0x7ff4b4104360 2024-05-09T02:05:28.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:28.625+0000 7ff4b9fdb700 1 -- 172.21.15.19:0/4227352654 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff4b41008c0 con 0x7ff4a00b2880 2024-05-09T02:05:29.531 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:29 smithi019 ceph-mon[30151]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:29.531 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:29 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:05:29.531 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:29 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:05:29.531 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:29 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:29.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:29 smithi196 ceph-mon[34718]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:29.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:29 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:05:29.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:29 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:05:29.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:29 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:29.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:29 smithi059 ceph-mon[34781]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:29.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:29 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:05:29.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:29 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:05:29.721 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:29 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:30.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:30 smithi196 ceph-mon[34718]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:05:30.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:30 smithi019 ceph-mon[30151]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:05:30.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:30 smithi059 ceph-mon[34781]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:05:31.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:31 smithi196 ceph-mon[34718]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T02:05:31.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:31 smithi019 ceph-mon[30151]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T02:05:31.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:31 smithi059 ceph-mon[34781]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T02:05:32.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:32 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/720154379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73c58dc3-fcb0-4440-92b3-bf41a2e58fbd"}]: dispatch 2024-05-09T02:05:32.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:32 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/720154379' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73c58dc3-fcb0-4440-92b3-bf41a2e58fbd"}]': finished 2024-05-09T02:05:32.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:32 smithi019 ceph-mon[30151]: osdmap e57: 10 total, 9 up, 10 in 2024-05-09T02:05:32.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:32 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:32.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:32 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/720154379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73c58dc3-fcb0-4440-92b3-bf41a2e58fbd"}]: dispatch 2024-05-09T02:05:32.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:32 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/720154379' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73c58dc3-fcb0-4440-92b3-bf41a2e58fbd"}]': finished 2024-05-09T02:05:32.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:32 smithi196 ceph-mon[34718]: osdmap e57: 10 total, 9 up, 10 in 2024-05-09T02:05:32.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:32 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:32 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/720154379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73c58dc3-fcb0-4440-92b3-bf41a2e58fbd"}]: dispatch 2024-05-09T02:05:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:32 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/720154379' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73c58dc3-fcb0-4440-92b3-bf41a2e58fbd"}]': finished 2024-05-09T02:05:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:32 smithi059 ceph-mon[34781]: osdmap e57: 10 total, 9 up, 10 in 2024-05-09T02:05:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:32 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:33 smithi196 ceph-mon[34718]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-09T02:05:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:33 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3852095475' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T02:05:33.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:33 smithi019 ceph-mon[30151]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-09T02:05:33.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:33 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3852095475' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T02:05:33.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:33 smithi059 ceph-mon[34781]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-09T02:05:33.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:33 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3852095475' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T02:05:35.518 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:35 smithi059 ceph-mon[34781]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T02:05:35.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:35 smithi196 ceph-mon[34718]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T02:05:35.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:35 smithi019 ceph-mon[30151]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T02:05:37.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:37 smithi196 ceph-mon[34718]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-09T02:05:37.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:37 smithi019 ceph-mon[30151]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-09T02:05:37.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:37 smithi059 ceph-mon[34781]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-09T02:05:39.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:39 smithi196 ceph-mon[34718]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:05:39.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:39 smithi019 ceph-mon[30151]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:05:39.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:39 smithi059 ceph-mon[34781]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:05:40.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:40 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T02:05:40.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:40 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:40.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:40 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T02:05:40.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:40 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:40.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:40 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T02:05:40.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:40 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:41.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:41 smithi196 ceph-mon[34718]: Deploying daemon osd.9 on smithi019 2024-05-09T02:05:41.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:41 smithi196 ceph-mon[34718]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:41.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:41 smithi019 ceph-mon[30151]: Deploying daemon osd.9 on smithi019 2024-05-09T02:05:41.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:41 smithi019 ceph-mon[30151]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:41.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:41 smithi059 ceph-mon[34781]: Deploying daemon osd.9 on smithi019 2024-05-09T02:05:41.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:41 smithi059 ceph-mon[34781]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:43.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:43 smithi196 ceph-mon[34718]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:43.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:43 smithi019 ceph-mon[30151]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:43.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:43 smithi059 ceph-mon[34781]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:45.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:45 smithi019 ceph-mon[30151]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:45.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:45 smithi196 ceph-mon[34718]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:45.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:45 smithi059 ceph-mon[34781]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:47.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:46 smithi019 ceph-mon[30151]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:47.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:46 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:47.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:46 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:47.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:46 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:05:47.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:46 smithi196 ceph-mon[34718]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:47.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:46 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:47.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:46 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:47.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:46 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:05:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:46 smithi059 ceph-mon[34781]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:46 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:46 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:46 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:05:49.263 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:48 smithi019 ceph-mon[30151]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:49.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:48 smithi196 ceph-mon[34718]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:49.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:48 smithi059 ceph-mon[34781]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:49.743 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 9 on host 'smithi019' 2024-05-09T02:05:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:49.742+0000 7ff4b17fa700 1 -- 172.21.15.19:0/4227352654 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff4b41008c0 con 0x7ff4a00b2880 2024-05-09T02:05:49.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:49.744+0000 7ff4b9fdb700 1 -- 172.21.15.19:0/4227352654 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff4a00b2880 msgr2=0x7ff4a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:05:49.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:49.744+0000 7ff4b9fdb700 1 --2- 172.21.15.19:0/4227352654 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff4a00b2880 0x7ff4a00b4d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4104260 tx=0x7ff4a400b410 comp rx=0 tx=0).stop 2024-05-09T02:05:49.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:49.744+0000 7ff4b9fdb700 1 -- 172.21.15.19:0/4227352654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4b4104360 msgr2=0x7ff4b41056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:05:49.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:49.744+0000 7ff4b9fdb700 1 --2- 172.21.15.19:0/4227352654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4b4104360 0x7ff4b41056a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00d950 tx=0x7ff49c00dc60 comp rx=0 tx=0).stop 2024-05-09T02:05:49.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:49.745+0000 7ff4b9fdb700 1 -- 172.21.15.19:0/4227352654 shutdown_connections 2024-05-09T02:05:49.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:49.745+0000 7ff4b9fdb700 1 --2- 172.21.15.19:0/4227352654 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff4a00b2880 0x7ff4a00b4d90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:49.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:49.745+0000 7ff4b9fdb700 1 --2- 172.21.15.19:0/4227352654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b4104d00 0x7ff4b4105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:49.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:49.745+0000 7ff4b9fdb700 1 --2- 172.21.15.19:0/4227352654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b4103230 0x7ff4b4106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:49.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:49.745+0000 7ff4b9fdb700 1 --2- 172.21.15.19:0/4227352654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4b4104360 0x7ff4b41056a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:49.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:49.745+0000 7ff4b9fdb700 1 -- 172.21.15.19:0/4227352654 >> 172.21.15.19:0/4227352654 conn(0x7ff4b40fe870 msgr2=0x7ff4b41001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:05:49.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:49.745+0000 7ff4b9fdb700 1 -- 172.21.15.19:0/4227352654 shutdown_connections 2024-05-09T02:05:49.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:05:49.745+0000 7ff4b9fdb700 1 -- 172.21.15.19:0/4227352654 wait complete. 2024-05-09T02:05:50.506 DEBUG:teuthology.orchestra.run.smithi019:osd.9> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.9.service 2024-05-09T02:05:50.508 INFO:tasks.cephadm:Deploying osd.10 on smithi059 with /dev/vg_nvme/lv_1... 2024-05-09T02:05:50.509 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T02:05:50.769 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T02:05:50.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:50.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:50.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:50 smithi019 ceph-mon[30151]: from='osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T02:05:50.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:50 smithi019 ceph-mon[30151]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:50.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:50.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:50.892 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:50.892 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:50.892 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:50 smithi059 ceph-mon[34781]: from='osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T02:05:50.892 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:50 smithi059 ceph-mon[34781]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:50.892 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:50.892 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:51.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:51.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:51.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:50 smithi196 ceph-mon[34718]: from='osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T02:05:51.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:50 smithi196 ceph-mon[34718]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:51.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:51.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:51.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:51 smithi019 ceph-mon[30151]: from='osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T02:05:51.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:51 smithi019 ceph-mon[30151]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T02:05:51.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:51 smithi019 ceph-mon[30151]: from='osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T02:05:51.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:51 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:51 smithi059 ceph-mon[34781]: from='osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:51 smithi059 ceph-mon[34781]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:51 smithi059 ceph-mon[34781]: from='osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T02:05:51.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:51 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:52.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:51 smithi196 ceph-mon[34718]: from='osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T02:05:52.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:51 smithi196 ceph-mon[34718]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T02:05:52.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:51 smithi196 ceph-mon[34718]: from='osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-09T02:05:52.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:51 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:52.437 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:05:52 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:05:52.158+0000 7f99777be700 -1 osd.9 0 waiting for initial osdmap 2024-05-09T02:05:52.437 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:05:52 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:05:52.170+0000 7f996f56b700 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T02:05:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:52 smithi059 ceph-mon[34781]: from='osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T02:05:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:52 smithi059 ceph-mon[34781]: osdmap e59: 10 total, 9 up, 10 in 2024-05-09T02:05:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:52 smithi059 ceph-mon[34781]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:52 smithi059 ceph-mon[34781]: from='osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886]' entity='osd.9' 2024-05-09T02:05:53.038 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T02:05:53.038 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T02:05:53.038 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-05-09T02:05:53.038 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-05-09T02:05:53.038 INFO:teuthology.orchestra.run.smithi059.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.012285 s, 854 MB/s 2024-05-09T02:05:53.038 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-05-09T02:05:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:52 smithi196 ceph-mon[34718]: from='osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T02:05:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:52 smithi196 ceph-mon[34718]: osdmap e59: 10 total, 9 up, 10 in 2024-05-09T02:05:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:52 smithi196 ceph-mon[34718]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:52 smithi196 ceph-mon[34718]: from='osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886]' entity='osd.9' 2024-05-09T02:05:53.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:52 smithi019 ceph-mon[30151]: from='osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-09T02:05:53.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:52 smithi019 ceph-mon[30151]: osdmap e59: 10 total, 9 up, 10 in 2024-05-09T02:05:53.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:53.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:53.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:52 smithi019 ceph-mon[30151]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T02:05:53.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:52 smithi019 ceph-mon[30151]: from='osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886]' entity='osd.9' 2024-05-09T02:05:53.687 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_1 2024-05-09T02:05:53.937 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T02:05:53.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:53 smithi059 ceph-mon[34781]: purged_snaps scrub starts 2024-05-09T02:05:53.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:53 smithi059 ceph-mon[34781]: purged_snaps scrub ok 2024-05-09T02:05:53.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:53 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:53.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:53 smithi059 ceph-mon[34781]: osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886] boot 2024-05-09T02:05:53.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:53 smithi059 ceph-mon[34781]: osdmap e60: 10 total, 10 up, 10 in 2024-05-09T02:05:53.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:53 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:54.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:53 smithi196 ceph-mon[34718]: purged_snaps scrub starts 2024-05-09T02:05:54.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:53 smithi196 ceph-mon[34718]: purged_snaps scrub ok 2024-05-09T02:05:54.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:53 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:54.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:53 smithi196 ceph-mon[34718]: osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886] boot 2024-05-09T02:05:54.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:53 smithi196 ceph-mon[34718]: osdmap e60: 10 total, 10 up, 10 in 2024-05-09T02:05:54.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:53 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:54.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:53 smithi019 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T02:05:54.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:53 smithi019 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T02:05:54.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:53 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:54.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:53 smithi019 ceph-mon[30151]: osd.9 [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886] boot 2024-05-09T02:05:54.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:53 smithi019 ceph-mon[30151]: osdmap e60: 10 total, 10 up, 10 in 2024-05-09T02:05:54.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:53 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:05:55.489 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:05:55.489 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: Detected new or changed devices on smithi019 2024-05-09T02:05:55.489 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:55.489 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:55.490 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:55.490 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:55.490 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:55.490 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:55.490 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi019 to 4273M 2024-05-09T02:05:55.490 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:55.490 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:55.490 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:55.490 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:55.490 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:05:55.490 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: osdmap e61: 10 total, 10 up, 10 in 2024-05-09T02:05:55.490 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:55.490 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:55.490 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:55.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:05:55.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: Detected new or changed devices on smithi019 2024-05-09T02:05:55.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:55.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:55.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:55.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:55.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:55.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:55.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi019 to 4273M 2024-05-09T02:05:55.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:55.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:55.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:55.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:55.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:05:55.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: osdmap e61: 10 total, 10 up, 10 in 2024-05-09T02:05:55.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:55.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:55.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:55.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:05:55.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: Detected new or changed devices on smithi019 2024-05-09T02:05:55.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:55.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:55.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:55.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:55.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:55.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:05:55.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi019 to 4273M 2024-05-09T02:05:55.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:55.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:55.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:55.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:55.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:05:55.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: osdmap e61: 10 total, 10 up, 10 in 2024-05-09T02:05:55.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:55.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:05:55.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:05:55.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.717+0000 7f30e6a37700 1 -- 172.21.15.59:0/1768753888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30e0104d20 msgr2=0x7f30e010b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:05:55.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.717+0000 7f30e6a37700 1 --2- 172.21.15.59:0/1768753888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30e0104d20 0x7f30e010b5d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f30d800b0a0 tx=0x7f30d800b3b0 comp rx=0 tx=0).stop 2024-05-09T02:05:55.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.717+0000 7f30e6a37700 1 -- 172.21.15.59:0/1768753888 shutdown_connections 2024-05-09T02:05:55.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.717+0000 7f30e6a37700 1 --2- 172.21.15.59:0/1768753888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30e0104d20 0x7f30e010b5d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:55.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.717+0000 7f30e6a37700 1 --2- 172.21.15.59:0/1768753888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e0103250 0x7f30e0103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:55.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.717+0000 7f30e6a37700 1 --2- 172.21.15.59:0/1768753888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30e0104380 0x7f30e0104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:55.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.717+0000 7f30e6a37700 1 -- 172.21.15.59:0/1768753888 >> 172.21.15.59:0/1768753888 conn(0x7f30e00fe890 msgr2=0x7f30e0100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:05:55.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.717+0000 7f30e6a37700 1 -- 172.21.15.59:0/1768753888 shutdown_connections 2024-05-09T02:05:55.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.717+0000 7f30e6a37700 1 -- 172.21.15.59:0/1768753888 wait complete. 2024-05-09T02:05:55.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.717+0000 7f30e6a37700 1 Processor -- start 2024-05-09T02:05:55.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.717+0000 7f30e6a37700 1 -- start start 2024-05-09T02:05:55.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30e6a37700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30e0103250 0x7f30e019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:05:55.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30e6a37700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30e0104380 0x7f30e019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:05:55.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30e6a37700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e0104d20 0x7f30e01a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:05:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30e6a37700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30e010ecc0 con 0x7f30e0104d20 2024-05-09T02:05:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30e6a37700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f30e010eb40 con 0x7f30e0103250 2024-05-09T02:05:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30e6a37700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30e010ee40 con 0x7f30e0104380 2024-05-09T02:05:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30dffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30e0103250 0x7f30e019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:05:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30dffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30e0103250 0x7f30e019a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52402/0 (socket says 172.21.15.59:52402) 2024-05-09T02:05:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30dffff700 1 -- 172.21.15.59:0/1916255037 learned_addr learned my addr 172.21.15.59:0/1916255037 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-09T02:05:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30df7fe700 1 --2- 172.21.15.59:0/1916255037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30e0104380 0x7f30e019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:05:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30e4fd4700 1 --2- 172.21.15.59:0/1916255037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e0104d20 0x7f30e01a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:05:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30dffff700 1 -- 172.21.15.59:0/1916255037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30e0104380 msgr2=0x7f30e019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:05:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30dffff700 1 --2- 172.21.15.59:0/1916255037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30e0104380 0x7f30e019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30dffff700 1 -- 172.21.15.59:0/1916255037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e0104d20 msgr2=0x7f30e01a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:05:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30dffff700 1 --2- 172.21.15.59:0/1916255037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e0104d20 0x7f30e01a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:05:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30dffff700 1 -- 172.21.15.59:0/1916255037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e01a23f0 con 0x7f30e0103250 2024-05-09T02:05:55.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.718+0000 7f30dffff700 1 --2- 172.21.15.59:0/1916255037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30e0103250 0x7f30e019a890 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f30d000e940 tx=0x7f30d000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:05:55.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.719+0000 7f30dd7fa700 1 -- 172.21.15.59:0/1916255037 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d000cbd0 con 0x7f30e0103250 2024-05-09T02:05:55.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.719+0000 7f30dd7fa700 1 -- 172.21.15.59:0/1916255037 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30d000ee90 con 0x7f30e0103250 2024-05-09T02:05:55.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.719+0000 7f30e6a37700 1 -- 172.21.15.59:0/1916255037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e01a26e0 con 0x7f30e0103250 2024-05-09T02:05:55.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.719+0000 7f30dd7fa700 1 -- 172.21.15.59:0/1916255037 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0003dc0 con 0x7f30e0103250 2024-05-09T02:05:55.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.719+0000 7f30e6a37700 1 -- 172.21.15.59:0/1916255037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e01a2c20 con 0x7f30e0103250 2024-05-09T02:05:55.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.720+0000 7f30e6a37700 1 -- 172.21.15.59:0/1916255037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e00512c0 con 0x7f30e0103250 2024-05-09T02:05:55.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.721+0000 7f30dd7fa700 1 -- 172.21.15.59:0/1916255037 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f30d0003f60 con 0x7f30e0103250 2024-05-09T02:05:55.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.722+0000 7f30dd7fa700 1 --2- 172.21.15.59:0/1916255037 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f30c80b2880 0x7f30c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:05:55.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.722+0000 7f30dd7fa700 1 -- 172.21.15.59:0/1916255037 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7f30d00e1c70 con 0x7f30e0103250 2024-05-09T02:05:55.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.722+0000 7f30df7fe700 1 --2- 172.21.15.59:0/1916255037 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f30c80b2880 0x7f30c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:05:55.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.723+0000 7f30df7fe700 1 --2- 172.21.15.59:0/1916255037 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f30c80b2880 0x7f30c80b4d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f30e019b940 tx=0x7f30d400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:05:55.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.727+0000 7f30dd7fa700 1 -- 172.21.15.59:0/1916255037 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f30d00aa190 con 0x7f30e0103250 2024-05-09T02:05:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:05:55.952+0000 7f30e6a37700 1 -- 172.21.15.59:0/1916255037 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f30e01a2ee0 con 0x7f30c80b2880 2024-05-09T02:05:56.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:56 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:05:56.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:56 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:05:56.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:56 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:56.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:56 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:05:56.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:56 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:05:56.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:56 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:56.594 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:56 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:05:56.594 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:56 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:05:56.594 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:56 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:05:57.510 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:57 smithi059 ceph-mon[34781]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:05:57.510 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:57 smithi059 ceph-mon[34781]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:05:57.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:57 smithi196 ceph-mon[34718]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:05:57.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:57 smithi196 ceph-mon[34718]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:05:57.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:57 smithi019 ceph-mon[30151]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:05:57.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:57 smithi019 ceph-mon[30151]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:05:59.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:59 smithi059 ceph-mon[34781]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:05:59.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:59 smithi059 ceph-mon[34781]: from='client.? 172.21.15.59:0/2450179157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8325282f-7325-4880-ab7d-b972717be9dd"}]: dispatch 2024-05-09T02:05:59.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:59 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8325282f-7325-4880-ab7d-b972717be9dd"}]: dispatch 2024-05-09T02:05:59.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:59 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8325282f-7325-4880-ab7d-b972717be9dd"}]': finished 2024-05-09T02:05:59.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:59 smithi059 ceph-mon[34781]: osdmap e62: 11 total, 10 up, 11 in 2024-05-09T02:05:59.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:05:59 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:05:59.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:59 smithi196 ceph-mon[34718]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:05:59.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:59 smithi196 ceph-mon[34718]: from='client.? 172.21.15.59:0/2450179157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8325282f-7325-4880-ab7d-b972717be9dd"}]: dispatch 2024-05-09T02:05:59.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:59 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8325282f-7325-4880-ab7d-b972717be9dd"}]: dispatch 2024-05-09T02:05:59.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:59 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8325282f-7325-4880-ab7d-b972717be9dd"}]': finished 2024-05-09T02:05:59.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:59 smithi196 ceph-mon[34718]: osdmap e62: 11 total, 10 up, 11 in 2024-05-09T02:05:59.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:05:59 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:05:59.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:59 smithi019 ceph-mon[30151]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:05:59.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:59 smithi019 ceph-mon[30151]: from='client.? 172.21.15.59:0/2450179157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8325282f-7325-4880-ab7d-b972717be9dd"}]: dispatch 2024-05-09T02:05:59.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:59 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8325282f-7325-4880-ab7d-b972717be9dd"}]: dispatch 2024-05-09T02:05:59.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:59 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8325282f-7325-4880-ab7d-b972717be9dd"}]': finished 2024-05-09T02:05:59.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:59 smithi019 ceph-mon[30151]: osdmap e62: 11 total, 10 up, 11 in 2024-05-09T02:05:59.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:05:59 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:06:00.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:00 smithi196 ceph-mon[34718]: from='client.? 172.21.15.59:0/3707901377' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T02:06:00.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:00 smithi019 ceph-mon[30151]: from='client.? 172.21.15.59:0/3707901377' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T02:06:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:00 smithi059 ceph-mon[34781]: from='client.? 172.21.15.59:0/3707901377' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T02:06:01.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:01 smithi196 ceph-mon[34718]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:01.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:01 smithi019 ceph-mon[30151]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:01.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:01 smithi059 ceph-mon[34781]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:03.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:03 smithi059 ceph-mon[34781]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:03.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:03 smithi196 ceph-mon[34718]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:03.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:03 smithi019 ceph-mon[30151]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:05.309 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:05 smithi059 ceph-mon[34781]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:05.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:05 smithi196 ceph-mon[34718]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:05.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:05 smithi019 ceph-mon[30151]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:06.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:06 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T02:06:06.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:06 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:06.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:06 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T02:06:06.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:06 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:06.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:06 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T02:06:06.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:06 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:07.382 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:07 smithi059 ceph-mon[34781]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:07.383 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:07 smithi059 ceph-mon[34781]: Deploying daemon osd.10 on smithi059 2024-05-09T02:06:07.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:07 smithi196 ceph-mon[34718]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:07.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:07 smithi196 ceph-mon[34718]: Deploying daemon osd.10 on smithi059 2024-05-09T02:06:07.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:07 smithi019 ceph-mon[30151]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:07.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:07 smithi019 ceph-mon[30151]: Deploying daemon osd.10 on smithi059 2024-05-09T02:06:09.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:09 smithi059 ceph-mon[34781]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:09.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:09 smithi196 ceph-mon[34718]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:09.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:09 smithi019 ceph-mon[30151]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:11.534 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:11 smithi059 ceph-mon[34781]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:11.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:11 smithi196 ceph-mon[34718]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:11.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:11 smithi019 ceph-mon[30151]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:13.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:12 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:13.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:12 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:06:13.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:12 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:13.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:12 smithi059 ceph-mon[34781]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:13.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:12 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:13.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:12 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:06:13.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:12 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:13.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:12 smithi196 ceph-mon[34718]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:13.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:12 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:13.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:12 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:06:13.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:12 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:13.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:12 smithi019 ceph-mon[30151]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:15.132 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:15 smithi059 ceph-mon[34781]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:15.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:15 smithi019 ceph-mon[30151]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:15.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:15 smithi196 ceph-mon[34718]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:15.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:06:15.620+0000 7f30dd7fa700 1 -- 172.21.15.59:0/1916255037 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f30e01a2ee0 con 0x7f30c80b2880 2024-05-09T02:06:15.622 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 10 on host 'smithi059' 2024-05-09T02:06:15.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:06:15.623+0000 7f30e6a37700 1 -- 172.21.15.59:0/1916255037 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f30c80b2880 msgr2=0x7f30c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:15.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:06:15.623+0000 7f30e6a37700 1 --2- 172.21.15.59:0/1916255037 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f30c80b2880 0x7f30c80b4d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f30e019b940 tx=0x7f30d400b410 comp rx=0 tx=0).stop 2024-05-09T02:06:15.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:06:15.623+0000 7f30e6a37700 1 -- 172.21.15.59:0/1916255037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30e0103250 msgr2=0x7f30e019a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:15.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:06:15.623+0000 7f30e6a37700 1 --2- 172.21.15.59:0/1916255037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30e0103250 0x7f30e019a890 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f30d000e940 tx=0x7f30d000ec50 comp rx=0 tx=0).stop 2024-05-09T02:06:15.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:06:15.623+0000 7f30e6a37700 1 -- 172.21.15.59:0/1916255037 shutdown_connections 2024-05-09T02:06:15.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:06:15.623+0000 7f30e6a37700 1 --2- 172.21.15.59:0/1916255037 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f30c80b2880 0x7f30c80b4d90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:15.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:06:15.623+0000 7f30e6a37700 1 --2- 172.21.15.59:0/1916255037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e0104d20 0x7f30e01a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:15.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:06:15.623+0000 7f30e6a37700 1 --2- 172.21.15.59:0/1916255037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30e0103250 0x7f30e019a890 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:15.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:06:15.623+0000 7f30e6a37700 1 --2- 172.21.15.59:0/1916255037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30e0104380 0x7f30e019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:15.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:06:15.623+0000 7f30e6a37700 1 -- 172.21.15.59:0/1916255037 >> 172.21.15.59:0/1916255037 conn(0x7f30e00fe890 msgr2=0x7f30e0100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:06:15.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:06:15.623+0000 7f30e6a37700 1 -- 172.21.15.59:0/1916255037 shutdown_connections 2024-05-09T02:06:15.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-09T02:06:15.623+0000 7f30e6a37700 1 -- 172.21.15.59:0/1916255037 wait complete. 2024-05-09T02:06:16.566 DEBUG:teuthology.orchestra.run.smithi059:osd.10> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.10.service 2024-05-09T02:06:16.568 INFO:tasks.cephadm:Deploying osd.11 on smithi196 with /dev/vg_nvme/lv_1... 2024-05-09T02:06:16.568 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T02:06:16.817 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:16 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:16.817 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:16 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:16.817 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:16 smithi196 ceph-mon[34718]: from='osd.10 [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T02:06:16.817 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:16 smithi196 ceph-mon[34718]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T02:06:16.817 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:16 smithi196 ceph-mon[34718]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:16.818 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:06:16.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:16 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:16.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:16 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:16.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:16 smithi019 ceph-mon[30151]: from='osd.10 [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T02:06:16.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:16 smithi019 ceph-mon[30151]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T02:06:16.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:16 smithi019 ceph-mon[30151]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:16.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:16 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:16.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:16 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:16.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:16 smithi059 ceph-mon[34781]: from='osd.10 [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T02:06:16.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:16 smithi059 ceph-mon[34781]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T02:06:16.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:16 smithi059 ceph-mon[34781]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:17 smithi059 ceph-mon[34781]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T02:06:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:17 smithi059 ceph-mon[34781]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T02:06:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:17 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:06:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:17 smithi059 ceph-mon[34781]: from='osd.10 [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:06:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:17 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:17 smithi059 ceph-mon[34781]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:06:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:17 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:17.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:06:17 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:06:17.593+0000 7fc22d82f700 -1 osd.10 0 waiting for initial osdmap 2024-05-09T02:06:17.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:06:17 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:06:17.607+0000 7fc2295e4700 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T02:06:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:17 smithi196 ceph-mon[34718]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T02:06:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:17 smithi196 ceph-mon[34718]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T02:06:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:17 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:06:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:17 smithi196 ceph-mon[34718]: from='osd.10 [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:06:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:17 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:17 smithi196 ceph-mon[34718]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:06:17.971 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:17 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:18.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:17 smithi019 ceph-mon[30151]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T02:06:18.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:17 smithi019 ceph-mon[30151]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T02:06:18.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:17 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:06:18.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:17 smithi019 ceph-mon[30151]: from='osd.10 [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:06:18.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:17 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:18.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:17 smithi019 ceph-mon[30151]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-09T02:06:18.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:17 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:18.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:18 smithi059 ceph-mon[34781]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T02:06:18.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:18 smithi059 ceph-mon[34781]: osdmap e64: 11 total, 10 up, 11 in 2024-05-09T02:06:18.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:06:18.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:06:18.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:18 smithi059 ceph-mon[34781]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:18.999 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:18 smithi196 ceph-mon[34718]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T02:06:18.999 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:18 smithi196 ceph-mon[34718]: osdmap e64: 11 total, 10 up, 11 in 2024-05-09T02:06:18.999 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:18 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:06:18.999 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:18 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:06:18.999 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:18 smithi196 ceph-mon[34718]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:18.999 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T02:06:18.999 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T02:06:18.999 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-05-09T02:06:18.999 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-05-09T02:06:18.999 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122236 s, 858 MB/s 2024-05-09T02:06:19.000 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-05-09T02:06:19.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:18 smithi019 ceph-mon[30151]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-09T02:06:19.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:18 smithi019 ceph-mon[30151]: osdmap e64: 11 total, 10 up, 11 in 2024-05-09T02:06:19.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:18 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:06:19.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:18 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:06:19.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:18 smithi019 ceph-mon[30151]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-09T02:06:19.850 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:19 smithi196 ceph-mon[34718]: purged_snaps scrub starts 2024-05-09T02:06:19.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:19 smithi196 ceph-mon[34718]: purged_snaps scrub ok 2024-05-09T02:06:19.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:19 smithi196 ceph-mon[34718]: osd.10 [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995] boot 2024-05-09T02:06:19.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:19 smithi196 ceph-mon[34718]: osdmap e65: 11 total, 11 up, 11 in 2024-05-09T02:06:19.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:19 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:06:19.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:19 smithi196 ceph-mon[34718]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T02:06:19.852 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_1 2024-05-09T02:06:19.962 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:19 smithi059 ceph-mon[34781]: purged_snaps scrub starts 2024-05-09T02:06:19.962 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:19 smithi059 ceph-mon[34781]: purged_snaps scrub ok 2024-05-09T02:06:19.962 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:19 smithi059 ceph-mon[34781]: osd.10 [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995] boot 2024-05-09T02:06:19.963 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:19 smithi059 ceph-mon[34781]: osdmap e65: 11 total, 11 up, 11 in 2024-05-09T02:06:19.963 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:19 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:06:19.963 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:19 smithi059 ceph-mon[34781]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T02:06:20.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:19 smithi019 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T02:06:20.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:19 smithi019 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T02:06:20.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:19 smithi019 ceph-mon[30151]: osd.10 [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995] boot 2024-05-09T02:06:20.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:19 smithi019 ceph-mon[30151]: osdmap e65: 11 total, 11 up, 11 in 2024-05-09T02:06:20.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:19 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:06:20.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:19 smithi019 ceph-mon[30151]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T02:06:20.099 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:06:21.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:21.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: Detected new or changed devices on smithi059 2024-05-09T02:06:21.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:21.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi059 to 4273M 2024-05-09T02:06:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:06:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:06:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:06:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:21 smithi196 ceph-mon[34718]: osdmap e67: 11 total, 11 up, 11 in 2024-05-09T02:06:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: Detected new or changed devices on smithi059 2024-05-09T02:06:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi059 to 4273M 2024-05-09T02:06:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:06:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:06:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:06:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:21 smithi019 ceph-mon[30151]: osdmap e67: 11 total, 11 up, 11 in 2024-05-09T02:06:21.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:21.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: Detected new or changed devices on smithi059 2024-05-09T02:06:21.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:21.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:21.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:21.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:21.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi059 to 4273M 2024-05-09T02:06:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:06:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:06:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:06:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:21 smithi059 ceph-mon[34781]: osdmap e67: 11 total, 11 up, 11 in 2024-05-09T02:06:21.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.933+0000 7f6ae0568700 1 -- 172.21.15.196:0/71144780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad8104370 msgr2=0x7f6ad8104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:21.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.933+0000 7f6ae0568700 1 --2- 172.21.15.196:0/71144780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad8104370 0x7f6ad8104780 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009990 tx=0x7f6ac8009ca0 comp rx=0 tx=0).stop 2024-05-09T02:06:21.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.934+0000 7f6ae0568700 1 -- 172.21.15.196:0/71144780 shutdown_connections 2024-05-09T02:06:21.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.934+0000 7f6ae0568700 1 --2- 172.21.15.196:0/71144780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad8105e40 0x7f6ad810c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:21.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.934+0000 7f6ae0568700 1 --2- 172.21.15.196:0/71144780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad8104370 0x7f6ad8104780 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:21.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.934+0000 7f6ae0568700 1 --2- 172.21.15.196:0/71144780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ad81054a0 0x7f6ad81058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:21.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.934+0000 7f6ae0568700 1 -- 172.21.15.196:0/71144780 >> 172.21.15.196:0/71144780 conn(0x7f6ad80ff9d0 msgr2=0x7f6ad8101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:06:21.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.934+0000 7f6ae0568700 1 -- 172.21.15.196:0/71144780 shutdown_connections 2024-05-09T02:06:21.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.934+0000 7f6ae0568700 1 -- 172.21.15.196:0/71144780 wait complete. 2024-05-09T02:06:21.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.934+0000 7f6ae0568700 1 Processor -- start 2024-05-09T02:06:21.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.934+0000 7f6ae0568700 1 -- start start 2024-05-09T02:06:21.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6ae0568700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad8104370 0x7f6ad819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:21.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6ae0568700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad81054a0 0x7f6ad819af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:21.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6ae0568700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ad8105e40 0x7f6ad81a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:21.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6ae0568700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad810feb0 con 0x7f6ad81054a0 2024-05-09T02:06:21.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6ae0568700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad810fd30 con 0x7f6ad8105e40 2024-05-09T02:06:21.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6ae0568700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad8110030 con 0x7f6ad8104370 2024-05-09T02:06:21.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6ade304700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad8104370 0x7f6ad819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:21.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6ade304700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad8104370 0x7f6ad819aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59668/0 (socket says 172.21.15.196:59668) 2024-05-09T02:06:21.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6ade304700 1 -- 172.21.15.196:0/3514073848 learned_addr learned my addr 172.21.15.196:0/3514073848 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T02:06:21.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6adeb05700 1 --2- 172.21.15.196:0/3514073848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ad8105e40 0x7f6ad81a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:21.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6addb03700 1 --2- 172.21.15.196:0/3514073848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad81054a0 0x7f6ad819af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:21.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6ade304700 1 -- 172.21.15.196:0/3514073848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ad8105e40 msgr2=0x7f6ad81a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:21.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6ade304700 1 --2- 172.21.15.196:0/3514073848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ad8105e40 0x7f6ad81a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:21.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6ade304700 1 -- 172.21.15.196:0/3514073848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad81054a0 msgr2=0x7f6ad819af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:21.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6ade304700 1 --2- 172.21.15.196:0/3514073848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad81054a0 0x7f6ad819af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:21.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6ade304700 1 -- 172.21.15.196:0/3514073848 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad81a25b0 con 0x7f6ad8104370 2024-05-09T02:06:21.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6adeb05700 1 --2- 172.21.15.196:0/3514073848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ad8105e40 0x7f6ad81a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:06:21.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6ade304700 1 --2- 172.21.15.196:0/3514073848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad8104370 0x7f6ad819aa50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8000c00 tx=0x7f6ac80047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:06:21.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.935+0000 7f6acf7fe700 1 -- 172.21.15.196:0/3514073848 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8027070 con 0x7f6ad8104370 2024-05-09T02:06:21.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.936+0000 7f6acf7fe700 1 -- 172.21.15.196:0/3514073848 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac8022470 con 0x7f6ad8104370 2024-05-09T02:06:21.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.936+0000 7f6ae0568700 1 -- 172.21.15.196:0/3514073848 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad81a2840 con 0x7f6ad8104370 2024-05-09T02:06:21.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.936+0000 7f6acf7fe700 1 -- 172.21.15.196:0/3514073848 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac800f6c0 con 0x7f6ad8104370 2024-05-09T02:06:21.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.936+0000 7f6ae0568700 1 -- 172.21.15.196:0/3514073848 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad81a2d70 con 0x7f6ad8104370 2024-05-09T02:06:21.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.937+0000 7f6ae0568700 1 -- 172.21.15.196:0/3514073848 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad80512c0 con 0x7f6ad8104370 2024-05-09T02:06:21.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.938+0000 7f6acf7fe700 1 -- 172.21.15.196:0/3514073848 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6ac802a070 con 0x7f6ad8104370 2024-05-09T02:06:21.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.939+0000 7f6acf7fe700 1 --2- 172.21.15.196:0/3514073848 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6ac40b2900 0x7f6ac40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:21.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.939+0000 7f6acf7fe700 1 -- 172.21.15.196:0/3514073848 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6694+0+0 (secure 0 0 0) 0x7f6ac80e3160 con 0x7f6ad8104370 2024-05-09T02:06:21.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.939+0000 7f6addb03700 1 --2- 172.21.15.196:0/3514073848 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6ac40b2900 0x7f6ac40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:21.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.940+0000 7f6addb03700 1 --2- 172.21.15.196:0/3514073848 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6ac40b2900 0x7f6ac40b4e10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6ad819bb00 tx=0x7f6ad4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:06:21.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:21.943+0000 7f6acf7fe700 1 -- 172.21.15.196:0/3514073848 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ac8031070 con 0x7f6ad8104370 2024-05-09T02:06:22.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:22.154+0000 7f6ae0568700 1 -- 172.21.15.196:0/3514073848 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6ad8063c00 con 0x7f6ac40b2900 2024-05-09T02:06:23.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:23 smithi196 ceph-mon[34718]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:23.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:23 smithi196 ceph-mon[34718]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:06:23.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:06:23.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:06:23.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:23.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:23 smithi019 ceph-mon[30151]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:23.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:23 smithi019 ceph-mon[30151]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:06:23.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:06:23.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:06:23.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:23 smithi059 ceph-mon[34781]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:23 smithi059 ceph-mon[34781]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:06:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T02:06:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T02:06:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:25.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:25 smithi196 ceph-mon[34718]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:25.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:25 smithi196 ceph-mon[34718]: from='client.? 172.21.15.196:0/1269509622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b19228c-b286-43c8-a408-169782f13075"}]: dispatch 2024-05-09T02:06:25.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:25 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b19228c-b286-43c8-a408-169782f13075"}]: dispatch 2024-05-09T02:06:25.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:25 smithi196 ceph-mon[34718]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b19228c-b286-43c8-a408-169782f13075"}]': finished 2024-05-09T02:06:25.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:25 smithi196 ceph-mon[34718]: osdmap e68: 12 total, 11 up, 12 in 2024-05-09T02:06:25.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:25.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:25 smithi019 ceph-mon[30151]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:25.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:25 smithi019 ceph-mon[30151]: from='client.? 172.21.15.196:0/1269509622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b19228c-b286-43c8-a408-169782f13075"}]: dispatch 2024-05-09T02:06:25.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:25 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b19228c-b286-43c8-a408-169782f13075"}]: dispatch 2024-05-09T02:06:25.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:25 smithi019 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b19228c-b286-43c8-a408-169782f13075"}]': finished 2024-05-09T02:06:25.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:25 smithi019 ceph-mon[30151]: osdmap e68: 12 total, 11 up, 12 in 2024-05-09T02:06:25.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:25.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:25 smithi059 ceph-mon[34781]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:25.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:25 smithi059 ceph-mon[34781]: from='client.? 172.21.15.196:0/1269509622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b19228c-b286-43c8-a408-169782f13075"}]: dispatch 2024-05-09T02:06:25.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:25 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b19228c-b286-43c8-a408-169782f13075"}]: dispatch 2024-05-09T02:06:25.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:25 smithi059 ceph-mon[34781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b19228c-b286-43c8-a408-169782f13075"}]': finished 2024-05-09T02:06:25.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:25 smithi059 ceph-mon[34781]: osdmap e68: 12 total, 11 up, 12 in 2024-05-09T02:06:25.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:26.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:26 smithi196 ceph-mon[34718]: from='client.? 172.21.15.196:0/736858813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T02:06:26.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:26 smithi019 ceph-mon[30151]: from='client.? 172.21.15.196:0/736858813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T02:06:26.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:26 smithi059 ceph-mon[34781]: from='client.? 172.21.15.196:0/736858813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T02:06:27.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:27 smithi019 ceph-mon[30151]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-09T02:06:27.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:27 smithi059 ceph-mon[34781]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-09T02:06:27.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:27 smithi196 ceph-mon[34718]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-09T02:06:29.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:29 smithi196 ceph-mon[34718]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T02:06:29.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:29 smithi019 ceph-mon[30151]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T02:06:29.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:29 smithi059 ceph-mon[34781]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T02:06:31.227 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:31 smithi196 ceph-mon[34718]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-09T02:06:31.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:31 smithi019 ceph-mon[30151]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-09T02:06:31.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:31 smithi059 ceph-mon[34781]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-09T02:06:33.319 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:33 smithi196 ceph-mon[34718]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:06:33.319 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:33 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T02:06:33.319 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:33 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:33.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:33 smithi019 ceph-mon[30151]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:06:33.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:33 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T02:06:33.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:33 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:33.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:33 smithi059 ceph-mon[34781]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:06:33.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:33 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T02:06:33.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:33 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:34.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:34 smithi019 ceph-mon[30151]: Deploying daemon osd.11 on smithi196 2024-05-09T02:06:34.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:34 smithi059 ceph-mon[34781]: Deploying daemon osd.11 on smithi196 2024-05-09T02:06:34.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:34 smithi196 ceph-mon[34718]: Deploying daemon osd.11 on smithi196 2024-05-09T02:06:35.275 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:35 smithi196 ceph-mon[34718]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:06:35.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:35 smithi019 ceph-mon[30151]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:06:35.367 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:35 smithi059 ceph-mon[34781]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T02:06:37.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:37 smithi019 ceph-mon[30151]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:37.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:37 smithi059 ceph-mon[34781]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:37.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:37 smithi196 ceph-mon[34718]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:39.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:39 smithi019 ceph-mon[30151]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:39.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:39.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:39.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:06:39.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:39 smithi196 ceph-mon[34718]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:39.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:39.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:39.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:06:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:39 smithi059 ceph-mon[34781]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:06:41.290 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:41 smithi196 ceph-mon[34718]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:41 smithi059 ceph-mon[34781]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:41.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:41 smithi019 ceph-mon[30151]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:43.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:43.025+0000 7f6acf7fe700 1 -- 172.21.15.196:0/3514073848 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6ad8063c00 con 0x7f6ac40b2900 2024-05-09T02:06:43.027 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 11 on host 'smithi196' 2024-05-09T02:06:43.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:43.027+0000 7f6ae0568700 1 -- 172.21.15.196:0/3514073848 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6ac40b2900 msgr2=0x7f6ac40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:43.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:43.027+0000 7f6ae0568700 1 --2- 172.21.15.196:0/3514073848 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6ac40b2900 0x7f6ac40b4e10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6ad819bb00 tx=0x7f6ad4008040 comp rx=0 tx=0).stop 2024-05-09T02:06:43.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:43.027+0000 7f6ae0568700 1 -- 172.21.15.196:0/3514073848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad8104370 msgr2=0x7f6ad819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:43.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:43.027+0000 7f6ae0568700 1 --2- 172.21.15.196:0/3514073848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad8104370 0x7f6ad819aa50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8000c00 tx=0x7f6ac80047b0 comp rx=0 tx=0).stop 2024-05-09T02:06:43.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:43.027+0000 7f6ae0568700 1 -- 172.21.15.196:0/3514073848 shutdown_connections 2024-05-09T02:06:43.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:43.027+0000 7f6ae0568700 1 --2- 172.21.15.196:0/3514073848 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6ac40b2900 0x7f6ac40b4e10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:43.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:43.027+0000 7f6ae0568700 1 --2- 172.21.15.196:0/3514073848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ad8105e40 0x7f6ad81a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:43.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:43.027+0000 7f6ae0568700 1 --2- 172.21.15.196:0/3514073848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad8104370 0x7f6ad819aa50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:43.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:43.027+0000 7f6ae0568700 1 --2- 172.21.15.196:0/3514073848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad81054a0 0x7f6ad819af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:43.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:43.028+0000 7f6ae0568700 1 -- 172.21.15.196:0/3514073848 >> 172.21.15.196:0/3514073848 conn(0x7f6ad80ff9d0 msgr2=0x7f6ad81086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:06:43.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:43.028+0000 7f6ae0568700 1 -- 172.21.15.196:0/3514073848 shutdown_connections 2024-05-09T02:06:43.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:43.028+0000 7f6ae0568700 1 -- 172.21.15.196:0/3514073848 wait complete. 2024-05-09T02:06:43.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:43 smithi196 ceph-mon[34718]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:43.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:43.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:43.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:43.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:06:43.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:43.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:43 smithi196 ceph-mon[34718]: from='osd.11 [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T02:06:43.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:43 smithi196 ceph-mon[34718]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T02:06:43.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:43.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:43.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:06:43.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:43 smithi019 ceph-mon[30151]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:43.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:43.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:43.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:43.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:06:43.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:43.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:43 smithi019 ceph-mon[30151]: from='osd.11 [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T02:06:43.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:43 smithi019 ceph-mon[30151]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T02:06:43.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:43.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:43.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:06:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:43 smithi059 ceph-mon[34781]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:06:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:43 smithi059 ceph-mon[34781]: from='osd.11 [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T02:06:43.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:43 smithi059 ceph-mon[34781]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T02:06:43.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:43.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:43.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:06:43.751 DEBUG:teuthology.orchestra.run.smithi196:osd.11> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.11.service 2024-05-09T02:06:43.754 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-09T02:06:43.754 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-09T02:06:44.018 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:06:44.391 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:44 smithi019 ceph-mon[30151]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T02:06:44.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:44 smithi019 ceph-mon[30151]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T02:06:44.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:44 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:44.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:44 smithi019 ceph-mon[30151]: from='osd.11 [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:06:44.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:44 smithi019 ceph-mon[30151]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:06:44.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:44 smithi059 ceph-mon[34781]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T02:06:44.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:44 smithi059 ceph-mon[34781]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T02:06:44.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:44 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:44.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:44 smithi059 ceph-mon[34781]: from='osd.11 [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:06:44.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:44 smithi059 ceph-mon[34781]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:06:44.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:44 smithi196 ceph-mon[34718]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T02:06:44.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:44 smithi196 ceph-mon[34718]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T02:06:44.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:44 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:44.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:44 smithi196 ceph-mon[34718]: from='osd.11 [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:06:44.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:44 smithi196 ceph-mon[34718]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-09T02:06:45.044 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:06:44 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:06:44.560+0000 7fea62d65700 -1 osd.11 0 waiting for initial osdmap 2024-05-09T02:06:45.044 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:06:44 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:06:44.575+0000 7fea5ab12700 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T02:06:45.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:45 smithi196 ceph-mon[34718]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:45.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:45 smithi196 ceph-mon[34718]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T02:06:45.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:45 smithi196 ceph-mon[34718]: osdmap e70: 12 total, 11 up, 12 in 2024-05-09T02:06:45.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:45 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:45.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:45 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:45.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:45 smithi196 ceph-mon[34718]: from='osd.11 ' entity='osd.11' 2024-05-09T02:06:45.465 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:45 smithi019 ceph-mon[30151]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:45.465 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:45 smithi019 ceph-mon[30151]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T02:06:45.465 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:45 smithi019 ceph-mon[30151]: osdmap e70: 12 total, 11 up, 12 in 2024-05-09T02:06:45.465 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:45 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:45.465 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:45 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:45.465 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:45 smithi019 ceph-mon[30151]: from='osd.11 ' entity='osd.11' 2024-05-09T02:06:45.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:45 smithi059 ceph-mon[34781]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:45.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:45 smithi059 ceph-mon[34781]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-09T02:06:45.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:45 smithi059 ceph-mon[34781]: osdmap e70: 12 total, 11 up, 12 in 2024-05-09T02:06:45.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:45 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:45.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:45 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:45.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:45 smithi059 ceph-mon[34781]: from='osd.11 ' entity='osd.11' 2024-05-09T02:06:45.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.818+0000 7f3a58fb2700 1 -- 172.21.15.19:0/4077429036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a54103240 msgr2=0x7f3a54103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:45.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.818+0000 7f3a58fb2700 1 --2- 172.21.15.19:0/4077429036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a54103240 0x7f3a54103650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009990 tx=0x7f3a44009ca0 comp rx=0 tx=0).stop 2024-05-09T02:06:45.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.819+0000 7f3a58fb2700 1 -- 172.21.15.19:0/4077429036 shutdown_connections 2024-05-09T02:06:45.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.819+0000 7f3a58fb2700 1 --2- 172.21.15.19:0/4077429036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a54104d10 0x7f3a5410b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:45.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.819+0000 7f3a58fb2700 1 --2- 172.21.15.19:0/4077429036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a54103240 0x7f3a54103650 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:45.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.819+0000 7f3a58fb2700 1 --2- 172.21.15.19:0/4077429036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a54104370 0x7f3a54104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:45.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.819+0000 7f3a58fb2700 1 -- 172.21.15.19:0/4077429036 >> 172.21.15.19:0/4077429036 conn(0x7f3a540fe880 msgr2=0x7f3a54100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:06:45.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.819+0000 7f3a58fb2700 1 -- 172.21.15.19:0/4077429036 shutdown_connections 2024-05-09T02:06:45.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.819+0000 7f3a58fb2700 1 -- 172.21.15.19:0/4077429036 wait complete. 2024-05-09T02:06:45.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.819+0000 7f3a58fb2700 1 Processor -- start 2024-05-09T02:06:45.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.819+0000 7f3a58fb2700 1 -- start start 2024-05-09T02:06:45.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a58fb2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a54103240 0x7f3a5419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:45.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a58fb2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a54104370 0x7f3a5419d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a58fb2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a54104d10 0x7f3a541a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a58fb2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5410ec80 con 0x7f3a54104d10 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a58fb2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5410eb00 con 0x7f3a54104370 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a58fb2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5410ee00 con 0x7f3a54103240 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a52d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a54104d10 0x7f3a541a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a52d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a54104d10 0x7f3a541a40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59946/0 (socket says 172.21.15.19:59946) 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a52d9d700 1 -- 172.21.15.19:0/2592759593 learned_addr learned my addr 172.21.15.19:0/2592759593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a5259c700 1 --2- 172.21.15.19:0/2592759593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a54103240 0x7f3a5419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a51d9b700 1 --2- 172.21.15.19:0/2592759593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a54104370 0x7f3a5419d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a52d9d700 1 -- 172.21.15.19:0/2592759593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a54103240 msgr2=0x7f3a5419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a52d9d700 1 --2- 172.21.15.19:0/2592759593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a54103240 0x7f3a5419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a52d9d700 1 -- 172.21.15.19:0/2592759593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a54104370 msgr2=0x7f3a5419d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a52d9d700 1 --2- 172.21.15.19:0/2592759593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a54104370 0x7f3a5419d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a52d9d700 1 -- 172.21.15.19:0/2592759593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a541a4750 con 0x7f3a54104d10 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a51d9b700 1 --2- 172.21.15.19:0/2592759593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a54104370 0x7f3a5419d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:06:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.820+0000 7f3a52d9d700 1 --2- 172.21.15.19:0/2592759593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a54104d10 0x7f3a541a40d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00ada0 tx=0x7f3a4c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:06:45.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.821+0000 7f3a4b7fe700 1 -- 172.21.15.19:0/2592759593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c017020 con 0x7f3a54104d10 2024-05-09T02:06:45.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.821+0000 7f3a4b7fe700 1 -- 172.21.15.19:0/2592759593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a4c00ce30 con 0x7f3a54104d10 2024-05-09T02:06:45.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.821+0000 7f3a4b7fe700 1 -- 172.21.15.19:0/2592759593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c005350 con 0x7f3a54104d10 2024-05-09T02:06:45.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.821+0000 7f3a58fb2700 1 -- 172.21.15.19:0/2592759593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a541a4a40 con 0x7f3a54104d10 2024-05-09T02:06:45.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.821+0000 7f3a58fb2700 1 -- 172.21.15.19:0/2592759593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a541a4f30 con 0x7f3a54104d10 2024-05-09T02:06:45.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.822+0000 7f3a58fb2700 1 -- 172.21.15.19:0/2592759593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a540512c0 con 0x7f3a54104d10 2024-05-09T02:06:45.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.823+0000 7f3a4b7fe700 1 -- 172.21.15.19:0/2592759593 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3a4c00c980 con 0x7f3a54104d10 2024-05-09T02:06:45.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.824+0000 7f3a4b7fe700 1 --2- 172.21.15.19:0/2592759593 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3a400b2900 0x7f3a400b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:45.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.824+0000 7f3a5259c700 1 --2- 172.21.15.19:0/2592759593 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3a400b2900 0x7f3a400b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:45.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.824+0000 7f3a4b7fe700 1 -- 172.21.15.19:0/2592759593 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f3a4c0e1940 con 0x7f3a54104d10 2024-05-09T02:06:45.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.825+0000 7f3a5259c700 1 --2- 172.21.15.19:0/2592759593 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3a400b2900 0x7f3a400b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400b6c0 tx=0x7f3a4400b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:06:45.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:45.828+0000 7f3a4b7fe700 1 -- 172.21.15.19:0/2592759593 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3a4c021020 con 0x7f3a54104d10 2024-05-09T02:06:46.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:46.056+0000 7f3a58fb2700 1 -- 172.21.15.19:0/2592759593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3a541a52c0 con 0x7f3a54104d10 2024-05-09T02:06:46.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:46.057+0000 7f3a4b7fe700 1 -- 172.21.15.19:0/2592759593 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f3a4c0a9b90 con 0x7f3a54104d10 2024-05-09T02:06:46.059 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:06:46.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:46.060+0000 7f3a58fb2700 1 -- 172.21.15.19:0/2592759593 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3a400b2900 msgr2=0x7f3a400b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:46.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:46.060+0000 7f3a58fb2700 1 --2- 172.21.15.19:0/2592759593 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3a400b2900 0x7f3a400b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400b6c0 tx=0x7f3a4400b720 comp rx=0 tx=0).stop 2024-05-09T02:06:46.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:46.060+0000 7f3a58fb2700 1 -- 172.21.15.19:0/2592759593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a54104d10 msgr2=0x7f3a541a40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:46.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:46.060+0000 7f3a58fb2700 1 --2- 172.21.15.19:0/2592759593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a54104d10 0x7f3a541a40d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00ada0 tx=0x7f3a4c00c330 comp rx=0 tx=0).stop 2024-05-09T02:06:46.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:46.060+0000 7f3a58fb2700 1 -- 172.21.15.19:0/2592759593 shutdown_connections 2024-05-09T02:06:46.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:46.060+0000 7f3a58fb2700 1 --2- 172.21.15.19:0/2592759593 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3a400b2900 0x7f3a400b4e10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:46.060+0000 7f3a58fb2700 1 --2- 172.21.15.19:0/2592759593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a54104d10 0x7f3a541a40d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:46.060+0000 7f3a58fb2700 1 --2- 172.21.15.19:0/2592759593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a54103240 0x7f3a5419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:46.060+0000 7f3a58fb2700 1 --2- 172.21.15.19:0/2592759593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a54104370 0x7f3a5419d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:46.060+0000 7f3a58fb2700 1 -- 172.21.15.19:0/2592759593 >> 172.21.15.19:0/2592759593 conn(0x7f3a540fe880 msgr2=0x7f3a5406b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:06:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:46.060+0000 7f3a58fb2700 1 -- 172.21.15.19:0/2592759593 shutdown_connections 2024-05-09T02:06:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:46.060+0000 7f3a58fb2700 1 -- 172.21.15.19:0/2592759593 wait complete. 2024-05-09T02:06:46.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:46 smithi019 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T02:06:46.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:46 smithi019 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T02:06:46.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:46 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:46.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:46 smithi019 ceph-mon[30151]: osd.11 [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001] boot 2024-05-09T02:06:46.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:46 smithi019 ceph-mon[30151]: osdmap e71: 12 total, 12 up, 12 in 2024-05-09T02:06:46.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:46 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:46.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:46 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2592759593' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T02:06:46.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:46 smithi059 ceph-mon[34781]: purged_snaps scrub starts 2024-05-09T02:06:46.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:46 smithi059 ceph-mon[34781]: purged_snaps scrub ok 2024-05-09T02:06:46.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:46 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:46.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:46 smithi059 ceph-mon[34781]: osd.11 [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001] boot 2024-05-09T02:06:46.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:46 smithi059 ceph-mon[34781]: osdmap e71: 12 total, 12 up, 12 in 2024-05-09T02:06:46.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:46 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:46.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:46 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2592759593' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T02:06:46.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:46 smithi196 ceph-mon[34718]: purged_snaps scrub starts 2024-05-09T02:06:46.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:46 smithi196 ceph-mon[34718]: purged_snaps scrub ok 2024-05-09T02:06:46.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:46 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:46.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:46 smithi196 ceph-mon[34718]: osd.11 [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001] boot 2024-05-09T02:06:46.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:46 smithi196 ceph-mon[34718]: osdmap e71: 12 total, 12 up, 12 in 2024-05-09T02:06:46.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:46 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:06:46.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:46 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2592759593' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T02:06:46.902 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":71,"num_osds":12,"num_up_osds":12,"osd_up_since":1715220405,"num_in_osds":12,"osd_in_since":1715220384,"num_remapped_pgs":0} 2024-05-09T02:06:46.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T02:06:47.159 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:06:47.283 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:47 smithi019 ceph-mon[30151]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:47.283 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:47 smithi019 ceph-mon[30151]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T02:06:47.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:47 smithi196 ceph-mon[34718]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:47.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:47 smithi196 ceph-mon[34718]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T02:06:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:47 smithi059 ceph-mon[34781]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-09T02:06:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:47 smithi059 ceph-mon[34781]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T02:06:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:48 smithi059 ceph-mon[34781]: Detected new or changed devices on smithi196 2024-05-09T02:06:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:48 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:48 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:48 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:48 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:48 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:48.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:48 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:48.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:48 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi196 to 4272M 2024-05-09T02:06:48.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:48 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:48.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:48 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:48.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:48 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:06:48.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:48 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:48.719 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:48 smithi019 ceph-mon[30151]: Detected new or changed devices on smithi196 2024-05-09T02:06:48.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:48 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:48.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:48 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:48.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:48 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:48.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:48 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:48.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:48 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:48.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:48 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:48.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:48 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi196 to 4272M 2024-05-09T02:06:48.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:48 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:48.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:48 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:48.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:48 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:06:48.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:48 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:48.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:48 smithi196 ceph-mon[34718]: Detected new or changed devices on smithi196 2024-05-09T02:06:48.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:48 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:48.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:48 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:48.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:48 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:48.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:48 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:48.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:48 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:48.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:48 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:06:48.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:48 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi196 to 4272M 2024-05-09T02:06:48.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:48 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:48.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:48 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:48.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:48 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:06:48.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:48 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.984+0000 7fc7e4f92700 1 -- 172.21.15.19:0/2877854503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e010e8f0 msgr2=0x7fc7e0110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.984+0000 7fc7e4f92700 1 --2- 172.21.15.19:0/2877854503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e010e8f0 0x7fc7e0110dd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800b0a0 tx=0x7fc7d800b3b0 comp rx=0 tx=0).stop 2024-05-09T02:06:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.984+0000 7fc7e4f92700 1 -- 172.21.15.19:0/2877854503 shutdown_connections 2024-05-09T02:06:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.984+0000 7fc7e4f92700 1 --2- 172.21.15.19:0/2877854503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e010e8f0 0x7fc7e0110dd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.984+0000 7fc7e4f92700 1 --2- 172.21.15.19:0/2877854503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7e01054c0 0x7fc7e01058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.984+0000 7fc7e4f92700 1 --2- 172.21.15.19:0/2877854503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e0105e10 0x7fc7e010e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.984+0000 7fc7e4f92700 1 -- 172.21.15.19:0/2877854503 >> 172.21.15.19:0/2877854503 conn(0x7fc7e00fedf0 msgr2=0x7fc7e0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:06:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.984+0000 7fc7e4f92700 1 -- 172.21.15.19:0/2877854503 shutdown_connections 2024-05-09T02:06:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.984+0000 7fc7e4f92700 1 -- 172.21.15.19:0/2877854503 wait complete. 2024-05-09T02:06:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.985+0000 7fc7e4f92700 1 Processor -- start 2024-05-09T02:06:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.985+0000 7fc7e4f92700 1 -- start start 2024-05-09T02:06:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7e4f92700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e01054c0 0x7fc7e01031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7e4f92700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7e0105e10 0x7fc7e01036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7e4f92700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e010e8f0 0x7fc7e0103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7e4f92700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0114450 con 0x7fc7e010e8f0 2024-05-09T02:06:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7e4f92700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01142d0 con 0x7fc7e01054c0 2024-05-09T02:06:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7e4f92700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01145d0 con 0x7fc7e0105e10 2024-05-09T02:06:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7ded9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e010e8f0 0x7fc7e0103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7ded9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e010e8f0 0x7fc7e0103c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42762/0 (socket says 172.21.15.19:42762) 2024-05-09T02:06:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7ded9d700 1 -- 172.21.15.19:0/840241012 learned_addr learned my addr 172.21.15.19:0/840241012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:06:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7ddd9b700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7e0105e10 0x7fc7e01036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7de59c700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e01054c0 0x7fc7e01031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7ded9d700 1 -- 172.21.15.19:0/840241012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7e0105e10 msgr2=0x7fc7e01036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7ded9d700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7e0105e10 0x7fc7e01036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7ded9d700 1 -- 172.21.15.19:0/840241012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e01054c0 msgr2=0x7fc7e01031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7ded9d700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e01054c0 0x7fc7e01031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7ded9d700 1 -- 172.21.15.19:0/840241012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e01a90d0 con 0x7fc7e010e8f0 2024-05-09T02:06:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7ddd9b700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7e0105e10 0x7fc7e01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:06:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.986+0000 7fc7de59c700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e01054c0 0x7fc7e01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:06:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.987+0000 7fc7ded9d700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e010e8f0 0x7fc7e0103c30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80062a0 tx=0x7fc7d8004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:06:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.987+0000 7fc7cf7fe700 1 -- 172.21.15.19:0/840241012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8021070 con 0x7fc7e010e8f0 2024-05-09T02:06:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.987+0000 7fc7cf7fe700 1 -- 172.21.15.19:0/840241012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7d8007d00 con 0x7fc7e010e8f0 2024-05-09T02:06:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.987+0000 7fc7e4f92700 1 -- 172.21.15.19:0/840241012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e01a9360 con 0x7fc7e010e8f0 2024-05-09T02:06:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.987+0000 7fc7cf7fe700 1 -- 172.21.15.19:0/840241012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d801d450 con 0x7fc7e010e8f0 2024-05-09T02:06:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.987+0000 7fc7e4f92700 1 -- 172.21.15.19:0/840241012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e01a9810 con 0x7fc7e010e8f0 2024-05-09T02:06:48.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.988+0000 7fc7e4f92700 1 -- 172.21.15.19:0/840241012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e00512c0 con 0x7fc7e010e8f0 2024-05-09T02:06:48.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.989+0000 7fc7cf7fe700 1 -- 172.21.15.19:0/840241012 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc7d8004020 con 0x7fc7e010e8f0 2024-05-09T02:06:48.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.990+0000 7fc7cf7fe700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc7c80b2880 0x7fc7c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:48.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.991+0000 7fc7de59c700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc7c80b2880 0x7fc7c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:48.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.991+0000 7fc7cf7fe700 1 -- 172.21.15.19:0/840241012 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fc7d800f020 con 0x7fc7e010e8f0 2024-05-09T02:06:48.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.991+0000 7fc7de59c700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc7c80b2880 0x7fc7c80b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0005fd0 tx=0x7fc7d0005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:06:48.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:48.996+0000 7fc7cf7fe700 1 -- 172.21.15.19:0/840241012 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc7d802a020 con 0x7fc7e010e8f0 2024-05-09T02:06:49.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:49.210+0000 7fc7e4f92700 1 -- 172.21.15.19:0/840241012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7e01a9b50 con 0x7fc7e010e8f0 2024-05-09T02:06:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:49.211+0000 7fc7cf7fe700 1 -- 172.21.15.19:0/840241012 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18409 (secure 0 0 0) 0x7fc7d80ab6d0 con 0x7fc7e010e8f0 2024-05-09T02:06:49.214 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:06:49.214 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"fsid":"a0611f3e-0da7-11ef-bc97-c7b262605968","created":"2024-05-09T01:59:27.591343+0000","modified":"2024-05-09T02:06:46.567193+0000","last_up_change":"2024-05-09T02:06:45.565065+0000","last_in_change":"2024-05-09T02:06:24.967186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T02:02:50.018959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"996c6a3d-3e3a-451c-8789-36006e0b3dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6803","nonce":3002373576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6805","nonce":3002373576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6809","nonce":3002373576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6807","nonce":3002373576}]},"public_addr":"172.21.15.19:6803/3002373576","cluster_addr":"172.21.15.19:6805/3002373576","heartbeat_back_addr":"172.21.15.19:6809/3002373576","heartbeat_front_addr":"172.21.15.19:6807/3002373576","state":["exists","up"]},{"osd":1,"uuid":"1ac68d09-1198-4e81-86c7-e37f7c41aa5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6801","nonce":827458353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6803","nonce":827458353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6807","nonce":827458353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6805","nonce":827458353}]},"public_addr":"172.21.15.59:6801/827458353","cluster_addr":"172.21.15.59:6803/827458353","heartbeat_back_addr":"172.21.15.59:6807/827458353","heartbeat_front_addr":"172.21.15.59:6805/827458353","state":["exists","up"]},{"osd":2,"uuid":"818a0de1-04e6-471d-83df-a68833011b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6801","nonce":1659504639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6803","nonce":1659504639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6807","nonce":1659504639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6805","nonce":1659504639}]},"public_addr":"172.21.15.196:6801/1659504639","cluster_addr":"172.21.15.196:6803/1659504639","heartbeat_back_addr":"172.21.15.196:6807/1659504639","heartbeat_front_addr":"172.21.15.196:6805/1659504639","state":["exists","up"]},{"osd":3,"uuid":"7b204854-86b8-454d-aee0-8bb5775ce828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6811","nonce":2707671599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6813","nonce":2707671599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6817","nonce":2707671599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6815","nonce":2707671599}]},"public_addr":"172.21.15.19:6811/2707671599","cluster_addr":"172.21.15.19:6813/2707671599","heartbeat_back_addr":"172.21.15.19:6817/2707671599","heartbeat_front_addr":"172.21.15.19:6815/2707671599","state":["exists","up"]},{"osd":4,"uuid":"091667f1-e887-4a94-8723-6f85b676fef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6809","nonce":223037330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6811","nonce":223037330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6815","nonce":223037330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6813","nonce":223037330}]},"public_addr":"172.21.15.59:6809/223037330","cluster_addr":"172.21.15.59:6811/223037330","heartbeat_back_addr":"172.21.15.59:6815/223037330","heartbeat_front_addr":"172.21.15.59:6813/223037330","state":["exists","up"]},{"osd":5,"uuid":"2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6809","nonce":2516161898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6811","nonce":2516161898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6815","nonce":2516161898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6813","nonce":2516161898}]},"public_addr":"172.21.15.196:6809/2516161898","cluster_addr":"172.21.15.196:6811/2516161898","heartbeat_back_addr":"172.21.15.196:6815/2516161898","heartbeat_front_addr":"172.21.15.196:6813/2516161898","state":["exists","up"]},{"osd":6,"uuid":"4e352851-1279-475f-ba8b-eaf9ca73df74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6819","nonce":2150451637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6821","nonce":2150451637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6825","nonce":2150451637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6823","nonce":2150451637}]},"public_addr":"172.21.15.19:6819/2150451637","cluster_addr":"172.21.15.19:6821/2150451637","heartbeat_back_addr":"172.21.15.19:6825/2150451637","heartbeat_front_addr":"172.21.15.19:6823/2150451637","state":["exists","up"]},{"osd":7,"uuid":"83f53f59-9a60-4d87-9a8a-67e5c6f57004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6817","nonce":967902053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6819","nonce":967902053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6823","nonce":967902053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6821","nonce":967902053}]},"public_addr":"172.21.15.59:6817/967902053","cluster_addr":"172.21.15.59:6819/967902053","heartbeat_back_addr":"172.21.15.59:6823/967902053","heartbeat_front_addr":"172.21.15.59:6821/967902053","state":["exists","up"]},{"osd":8,"uuid":"a8f1bfb5-1cc8-4a65-8396-239bd781e1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6817","nonce":2946064259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6819","nonce":2946064259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6823","nonce":2946064259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6821","nonce":2946064259}]},"public_addr":"172.21.15.196:6817/2946064259","cluster_addr":"172.21.15.196:6819/2946064259","heartbeat_back_addr":"172.21.15.196:6823/2946064259","heartbeat_front_addr":"172.21.15.196:6821/2946064259","state":["exists","up"]},{"osd":9,"uuid":"73c58dc3-fcb0-4440-92b3-bf41a2e58fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6827","nonce":1735017886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6829","nonce":1735017886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6833","nonce":1735017886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6831","nonce":1735017886}]},"public_addr":"172.21.15.19:6827/1735017886","cluster_addr":"172.21.15.19:6829/1735017886","heartbeat_back_addr":"172.21.15.19:6833/1735017886","heartbeat_front_addr":"172.21.15.19:6831/1735017886","state":["exists","up"]},{"osd":10,"uuid":"8325282f-7325-4880-ab7d-b972717be9dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6825","nonce":2932899995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6827","nonce":2932899995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6831","nonce":2932899995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6829","nonce":2932899995}]},"public_addr":"172.21.15.59:6825/2932899995","cluster_addr":"172.21.15.59:6827/2932899995","heartbeat_back_addr":"172.21.15.59:6831/2932899995","heartbeat_front_addr":"172.21.15.59:6829/2932899995","state":["exists","up"]},{"osd":11,"uuid":"8b19228c-b286-43c8-a408-169782f13075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6825","nonce":2608023001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6827","nonce":2608023001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6831","nonce":2608023001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6829","nonce":2608023001}]},"public_addr":"172.21.15.196:6825/2608023001","cluster_addr":"172.21.15.196:6827/2608023001","heartbeat_back_addr":"172.21.15.196:6831/2608023001","heartbeat_front_addr":"172.21.15.196:6829/2608023001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:01:59.457748+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:23.640666+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:47.768243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:13.016746+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:39.060898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:04.508313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:31.366217+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:57.579758+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:23.177396+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:50.966399+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:16.838666+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:43.959387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3291838961":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1024133854":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1176455101":"2024-05-10T02:00:23.920230+0000","172.21.15.19:6801/2101052000":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/4018987404":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/3079777324":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6801/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/179703528":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/2101052000":"2024-05-10T02:00:23.920230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T02:06:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:49.214+0000 7fc7e4f92700 1 -- 172.21.15.19:0/840241012 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc7c80b2880 msgr2=0x7fc7c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:49.214+0000 7fc7e4f92700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc7c80b2880 0x7fc7c80b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0005fd0 tx=0x7fc7d0005dc0 comp rx=0 tx=0).stop 2024-05-09T02:06:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:49.214+0000 7fc7e4f92700 1 -- 172.21.15.19:0/840241012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e010e8f0 msgr2=0x7fc7e0103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:49.214+0000 7fc7e4f92700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e010e8f0 0x7fc7e0103c30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80062a0 tx=0x7fc7d8004ed0 comp rx=0 tx=0).stop 2024-05-09T02:06:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:49.214+0000 7fc7e4f92700 1 -- 172.21.15.19:0/840241012 shutdown_connections 2024-05-09T02:06:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:49.214+0000 7fc7e4f92700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc7c80b2880 0x7fc7c80b4d90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:49.214+0000 7fc7e4f92700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e010e8f0 0x7fc7e0103c30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:49.214+0000 7fc7e4f92700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e01054c0 0x7fc7e01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:49.214+0000 7fc7e4f92700 1 --2- 172.21.15.19:0/840241012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7e0105e10 0x7fc7e01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:49.214+0000 7fc7e4f92700 1 -- 172.21.15.19:0/840241012 >> 172.21.15.19:0/840241012 conn(0x7fc7e00fedf0 msgr2=0x7fc7e0108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:06:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:49.214+0000 7fc7e4f92700 1 -- 172.21.15.19:0/840241012 shutdown_connections 2024-05-09T02:06:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:49.214+0000 7fc7e4f92700 1 -- 172.21.15.19:0/840241012 wait complete. 2024-05-09T02:06:49.603 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:49 smithi019 ceph-mon[30151]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:49.603 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:49 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/840241012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:06:49.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:49 smithi059 ceph-mon[34781]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:49.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:49 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/840241012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:06:49.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:49 smithi196 ceph-mon[34718]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:49.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:49 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/840241012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:06:50.045 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T02:02:50.018959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T02:06:50.045 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-09T02:06:50.304 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:06:51.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:51 smithi019 ceph-mon[30151]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:51.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:51 smithi059 ceph-mon[34781]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:51.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:51 smithi196 ceph-mon[34718]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:52.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.140+0000 7f87958f4700 1 -- 172.21.15.19:0/2147944589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8790109850 msgr2=0x7f87901111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.140+0000 7f87958f4700 1 --2- 172.21.15.19:0/2147944589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8790109850 0x7f87901111b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f878800b0d0 tx=0x7f878800b3e0 comp rx=0 tx=0).stop 2024-05-09T02:06:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.140+0000 7f87958f4700 1 -- 172.21.15.19:0/2147944589 shutdown_connections 2024-05-09T02:06:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.140+0000 7f87958f4700 1 --2- 172.21.15.19:0/2147944589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8790109850 0x7f87901111b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.140+0000 7f87958f4700 1 --2- 172.21.15.19:0/2147944589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87901045d0 0x7f87901049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.140+0000 7f87958f4700 1 --2- 172.21.15.19:0/2147944589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8790108f00 0x7f8790109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.140+0000 7f87958f4700 1 -- 172.21.15.19:0/2147944589 >> 172.21.15.19:0/2147944589 conn(0x7f87900ffea0 msgr2=0x7f87901022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:06:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.140+0000 7f87958f4700 1 -- 172.21.15.19:0/2147944589 shutdown_connections 2024-05-09T02:06:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.140+0000 7f87958f4700 1 -- 172.21.15.19:0/2147944589 wait complete. 2024-05-09T02:06:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.140+0000 7f87958f4700 1 Processor -- start 2024-05-09T02:06:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f87958f4700 1 -- start start 2024-05-09T02:06:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f87958f4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87901045d0 0x7f879019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f87958f4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8790108f00 0x7f879019d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f87958f4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8790109850 0x7f87901a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f87958f4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87901144d0 con 0x7f8790108f00 2024-05-09T02:06:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f87958f4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8790114350 con 0x7f8790109850 2024-05-09T02:06:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f87958f4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8790114650 con 0x7f87901045d0 2024-05-09T02:06:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f8787fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8790108f00 0x7f879019d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f8787fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8790108f00 0x7f879019d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42786/0 (socket says 172.21.15.19:42786) 2024-05-09T02:06:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f8787fff700 1 -- 172.21.15.19:0/3591387244 learned_addr learned my addr 172.21.15.19:0/3591387244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:06:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f878effd700 1 --2- 172.21.15.19:0/3591387244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87901045d0 0x7f879019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f878f7fe700 1 --2- 172.21.15.19:0/3591387244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8790109850 0x7f87901a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f8787fff700 1 -- 172.21.15.19:0/3591387244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87901045d0 msgr2=0x7f879019cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f8787fff700 1 --2- 172.21.15.19:0/3591387244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87901045d0 0x7f879019cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f8787fff700 1 -- 172.21.15.19:0/3591387244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8790109850 msgr2=0x7f87901a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f8787fff700 1 --2- 172.21.15.19:0/3591387244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8790109850 0x7f87901a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.141+0000 7f8787fff700 1 -- 172.21.15.19:0/3591387244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87901a1e90 con 0x7f8790108f00 2024-05-09T02:06:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.142+0000 7f8787fff700 1 --2- 172.21.15.19:0/3591387244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8790108f00 0x7f879019d560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f877800d980 tx=0x7f877800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:06:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.142+0000 7f87948f2700 1 -- 172.21.15.19:0/3591387244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778017070 con 0x7f8790108f00 2024-05-09T02:06:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.142+0000 7f87958f4700 1 -- 172.21.15.19:0/3591387244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87901a2180 con 0x7f8790108f00 2024-05-09T02:06:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.142+0000 7f87948f2700 1 -- 172.21.15.19:0/3591387244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8778009de0 con 0x7f8790108f00 2024-05-09T02:06:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.142+0000 7f87948f2700 1 -- 172.21.15.19:0/3591387244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877800f3c0 con 0x7f8790108f00 2024-05-09T02:06:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.142+0000 7f87958f4700 1 -- 172.21.15.19:0/3591387244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87901a9aa0 con 0x7f8790108f00 2024-05-09T02:06:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.143+0000 7f87958f4700 1 -- 172.21.15.19:0/3591387244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87900512c0 con 0x7f8790108f00 2024-05-09T02:06:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.145+0000 7f87948f2700 1 -- 172.21.15.19:0/3591387244 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8778009930 con 0x7f8790108f00 2024-05-09T02:06:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.145+0000 7f87948f2700 1 --2- 172.21.15.19:0/3591387244 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f877c0b2880 0x7f877c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.145+0000 7f878effd700 1 --2- 172.21.15.19:0/3591387244 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f877c0b2880 0x7f877c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:52.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.145+0000 7f87948f2700 1 -- 172.21.15.19:0/3591387244 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f87780e17b0 con 0x7f8790108f00 2024-05-09T02:06:52.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.146+0000 7f878effd700 1 --2- 172.21.15.19:0/3591387244 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f877c0b2880 0x7f877c0b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8780005950 tx=0x7f878000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:06:52.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.149+0000 7f87948f2700 1 -- 172.21.15.19:0/3591387244 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87780a9970 con 0x7f8790108f00 2024-05-09T02:06:52.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.360+0000 7f87958f4700 1 -- 172.21.15.19:0/3591387244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87901a9c70 con 0x7f8790108f00 2024-05-09T02:06:52.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.361+0000 7f87948f2700 1 -- 172.21.15.19:0/3591387244 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f87780a9270 con 0x7f8790108f00 2024-05-09T02:06:52.364 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-05-09T02:06:52.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.363+0000 7f87958f4700 1 -- 172.21.15.19:0/3591387244 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f877c0b2880 msgr2=0x7f877c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:52.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.363+0000 7f87958f4700 1 --2- 172.21.15.19:0/3591387244 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f877c0b2880 0x7f877c0b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8780005950 tx=0x7f878000b410 comp rx=0 tx=0).stop 2024-05-09T02:06:52.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.364+0000 7f87958f4700 1 -- 172.21.15.19:0/3591387244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8790108f00 msgr2=0x7f879019d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:52.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.364+0000 7f87958f4700 1 --2- 172.21.15.19:0/3591387244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8790108f00 0x7f879019d560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f877800d980 tx=0x7f877800dc90 comp rx=0 tx=0).stop 2024-05-09T02:06:52.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.364+0000 7f87958f4700 1 -- 172.21.15.19:0/3591387244 shutdown_connections 2024-05-09T02:06:52.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.364+0000 7f87958f4700 1 --2- 172.21.15.19:0/3591387244 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f877c0b2880 0x7f877c0b4d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:52.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.364+0000 7f87958f4700 1 --2- 172.21.15.19:0/3591387244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8790109850 0x7f87901a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:52.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.364+0000 7f87958f4700 1 --2- 172.21.15.19:0/3591387244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87901045d0 0x7f879019cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:52.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.364+0000 7f87958f4700 1 --2- 172.21.15.19:0/3591387244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8790108f00 0x7f879019d560 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:52.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.364+0000 7f87958f4700 1 -- 172.21.15.19:0/3591387244 >> 172.21.15.19:0/3591387244 conn(0x7f87900ffea0 msgr2=0x7f8790110e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:06:52.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.364+0000 7f87958f4700 1 -- 172.21.15.19:0/3591387244 shutdown_connections 2024-05-09T02:06:52.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:52.364+0000 7f87958f4700 1 -- 172.21.15.19:0/3591387244 wait complete. 2024-05-09T02:06:52.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:52 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3591387244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T02:06:52.789 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:52 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3591387244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T02:06:52.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:52 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3591387244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T02:06:53.086 INFO:tasks.cephadm:Adding mds.a on smithi019 2024-05-09T02:06:53.087 INFO:tasks.cephadm:Adding mds.d on smithi019 2024-05-09T02:06:53.087 INFO:tasks.cephadm:Adding mds.g on smithi019 2024-05-09T02:06:53.087 INFO:tasks.cephadm:Adding mds.j on smithi019 2024-05-09T02:06:53.087 INFO:tasks.cephadm:Adding mds.b on smithi059 2024-05-09T02:06:53.087 INFO:tasks.cephadm:Adding mds.e on smithi059 2024-05-09T02:06:53.087 INFO:tasks.cephadm:Adding mds.h on smithi059 2024-05-09T02:06:53.087 INFO:tasks.cephadm:Adding mds.k on smithi059 2024-05-09T02:06:53.087 INFO:tasks.cephadm:Adding mds.c on smithi196 2024-05-09T02:06:53.087 INFO:tasks.cephadm:Adding mds.f on smithi196 2024-05-09T02:06:53.087 INFO:tasks.cephadm:Adding mds.i on smithi196 2024-05-09T02:06:53.087 INFO:tasks.cephadm:Adding mds.l on smithi196 2024-05-09T02:06:53.087 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l' 2024-05-09T02:06:53.338 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:06:53.713 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:53 smithi196 ceph-mon[34718]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:53.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:53 smithi059 ceph-mon[34781]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:53.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:53 smithi019 ceph-mon[30151]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:55.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.033+0000 7f16a7a1f700 1 -- 172.21.15.196:0/3086413880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16a0104370 msgr2=0x7f16a0104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:55.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.033+0000 7f16a7a1f700 1 --2- 172.21.15.196:0/3086413880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16a0104370 0x7f16a0104780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1690009990 tx=0x7f1690009ca0 comp rx=0 tx=0).stop 2024-05-09T02:06:55.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.033+0000 7f16a7a1f700 1 -- 172.21.15.196:0/3086413880 shutdown_connections 2024-05-09T02:06:55.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.033+0000 7f16a7a1f700 1 --2- 172.21.15.196:0/3086413880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a0105e40 0x7f16a010c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:55.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.033+0000 7f16a7a1f700 1 --2- 172.21.15.196:0/3086413880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16a0104370 0x7f16a0104780 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:55.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.033+0000 7f16a7a1f700 1 --2- 172.21.15.196:0/3086413880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16a01054a0 0x7f16a01058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:55.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.033+0000 7f16a7a1f700 1 -- 172.21.15.196:0/3086413880 >> 172.21.15.196:0/3086413880 conn(0x7f16a00ff9d0 msgr2=0x7f16a0101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:06:55.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.034+0000 7f16a7a1f700 1 -- 172.21.15.196:0/3086413880 shutdown_connections 2024-05-09T02:06:55.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.034+0000 7f16a7a1f700 1 -- 172.21.15.196:0/3086413880 wait complete. 2024-05-09T02:06:55.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.034+0000 7f16a7a1f700 1 Processor -- start 2024-05-09T02:06:55.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.034+0000 7f16a7a1f700 1 -- start start 2024-05-09T02:06:55.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.034+0000 7f16a7a1f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16a0104370 0x7f16a019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:55.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.034+0000 7f16a7a1f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a01054a0 0x7f16a019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:55.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.034+0000 7f16a7a1f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16a0105e40 0x7f16a01a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:55.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.034+0000 7f16a7a1f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16a010feb0 con 0x7f16a0104370 2024-05-09T02:06:55.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.034+0000 7f16a7a1f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f16a010fd30 con 0x7f16a01054a0 2024-05-09T02:06:55.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.034+0000 7f16a7a1f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16a0110030 con 0x7f16a0105e40 2024-05-09T02:06:55.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.035+0000 7f16a5fbc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16a0105e40 0x7f16a01a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:55.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.035+0000 7f16a5fbc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16a0105e40 0x7f16a01a1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35282/0 (socket says 172.21.15.196:35282) 2024-05-09T02:06:55.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.035+0000 7f16a5fbc700 1 -- 172.21.15.196:0/1387592504 learned_addr learned my addr 172.21.15.196:0/1387592504 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-09T02:06:55.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.035+0000 7f16a57bb700 1 --2- 172.21.15.196:0/1387592504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16a0104370 0x7f16a019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:55.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.035+0000 7f16a4fba700 1 --2- 172.21.15.196:0/1387592504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a01054a0 0x7f16a019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:55.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.035+0000 7f16a5fbc700 1 -- 172.21.15.196:0/1387592504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a01054a0 msgr2=0x7f16a019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:55.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.035+0000 7f16a5fbc700 1 --2- 172.21.15.196:0/1387592504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a01054a0 0x7f16a019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:55.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.035+0000 7f16a5fbc700 1 -- 172.21.15.196:0/1387592504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16a0104370 msgr2=0x7f16a019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:55.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.035+0000 7f16a5fbc700 1 --2- 172.21.15.196:0/1387592504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16a0104370 0x7f16a019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:55.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.035+0000 7f16a5fbc700 1 -- 172.21.15.196:0/1387592504 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a01a25e0 con 0x7f16a0105e40 2024-05-09T02:06:55.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.035+0000 7f16a5fbc700 1 --2- 172.21.15.196:0/1387592504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16a0105e40 0x7f16a01a1f60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f169800b970 tx=0x7f169800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:06:55.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.035+0000 7f16967fc700 1 -- 172.21.15.196:0/1387592504 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1698019070 con 0x7f16a0105e40 2024-05-09T02:06:55.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.035+0000 7f16967fc700 1 -- 172.21.15.196:0/1387592504 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f169800cde0 con 0x7f16a0105e40 2024-05-09T02:06:55.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.035+0000 7f16a7a1f700 1 -- 172.21.15.196:0/1387592504 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a01a28d0 con 0x7f16a0105e40 2024-05-09T02:06:55.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.035+0000 7f16a7a1f700 1 -- 172.21.15.196:0/1387592504 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a01a2e40 con 0x7f16a0105e40 2024-05-09T02:06:55.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.036+0000 7f16967fc700 1 -- 172.21.15.196:0/1387592504 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1698007370 con 0x7f16a0105e40 2024-05-09T02:06:55.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.037+0000 7f16a7a1f700 1 -- 172.21.15.196:0/1387592504 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1684005360 con 0x7f16a0105e40 2024-05-09T02:06:55.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.038+0000 7f16967fc700 1 -- 172.21.15.196:0/1387592504 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f169801c070 con 0x7f16a0105e40 2024-05-09T02:06:55.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.038+0000 7f16967fc700 1 --2- 172.21.15.196:0/1387592504 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f168c0b2880 0x7f168c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:55.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.039+0000 7f16967fc700 1 -- 172.21.15.196:0/1387592504 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f16980e12b0 con 0x7f16a0105e40 2024-05-09T02:06:55.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.039+0000 7f16a57bb700 1 --2- 172.21.15.196:0/1387592504 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f168c0b2880 0x7f168c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:55.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.040+0000 7f16a57bb700 1 --2- 172.21.15.196:0/1387592504 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f168c0b2880 0x7f168c0b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f16900038a0 tx=0x7f169001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:06:55.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.042+0000 7f16967fc700 1 -- 172.21.15.196:0/1387592504 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f16980a9470 con 0x7f16a0105e40 2024-05-09T02:06:55.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.254+0000 7f16a7a1f700 1 -- 172.21.15.196:0/1387592504 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}) -- 0x7f1684000cf0 con 0x7f168c0b2880 2024-05-09T02:06:55.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.264+0000 7f16967fc700 1 -- 172.21.15.196:0/1387592504 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f1684000cf0 con 0x7f168c0b2880 2024-05-09T02:06:55.266 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mds.all update... 2024-05-09T02:06:55.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.266+0000 7f16a7a1f700 1 -- 172.21.15.196:0/1387592504 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f168c0b2880 msgr2=0x7f168c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:55.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.266+0000 7f16a7a1f700 1 --2- 172.21.15.196:0/1387592504 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f168c0b2880 0x7f168c0b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f16900038a0 tx=0x7f169001a040 comp rx=0 tx=0).stop 2024-05-09T02:06:55.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.266+0000 7f16a7a1f700 1 -- 172.21.15.196:0/1387592504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16a0105e40 msgr2=0x7f16a01a1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:55.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.266+0000 7f16a7a1f700 1 --2- 172.21.15.196:0/1387592504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16a0105e40 0x7f16a01a1f60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f169800b970 tx=0x7f169800bc80 comp rx=0 tx=0).stop 2024-05-09T02:06:55.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.266+0000 7f16a7a1f700 1 -- 172.21.15.196:0/1387592504 shutdown_connections 2024-05-09T02:06:55.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.266+0000 7f16a7a1f700 1 --2- 172.21.15.196:0/1387592504 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f168c0b2880 0x7f168c0b4d90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:55.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.266+0000 7f16a7a1f700 1 --2- 172.21.15.196:0/1387592504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16a0105e40 0x7f16a01a1f60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:55.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.266+0000 7f16a7a1f700 1 --2- 172.21.15.196:0/1387592504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16a0104370 0x7f16a019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:55.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.266+0000 7f16a7a1f700 1 --2- 172.21.15.196:0/1387592504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a01054a0 0x7f16a019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:55.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.266+0000 7f16a7a1f700 1 -- 172.21.15.196:0/1387592504 >> 172.21.15.196:0/1387592504 conn(0x7f16a00ff9d0 msgr2=0x7f16a01086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:06:55.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.266+0000 7f16a7a1f700 1 -- 172.21.15.196:0/1387592504 shutdown_connections 2024-05-09T02:06:55.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-09T02:06:55.266+0000 7f16a7a1f700 1 -- 172.21.15.196:0/1387592504 wait complete. 2024-05-09T02:06:55.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:55 smithi196 ceph-mon[34718]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:55.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:55.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:06:55.715 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:55 smithi059 ceph-mon[34781]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:55.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:55.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:06:55.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:55 smithi019 ceph-mon[30151]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:55.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:55.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:06:55.936 DEBUG:teuthology.orchestra.run.smithi019:mds.a> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.a.service 2024-05-09T02:06:55.939 DEBUG:teuthology.orchestra.run.smithi019:mds.d> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.d.service 2024-05-09T02:06:55.982 DEBUG:teuthology.orchestra.run.smithi019:mds.g> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.g.service 2024-05-09T02:06:56.026 DEBUG:teuthology.orchestra.run.smithi019:mds.j> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.j.service 2024-05-09T02:06:56.069 DEBUG:teuthology.orchestra.run.smithi059:mds.b> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.b.service 2024-05-09T02:06:56.071 DEBUG:teuthology.orchestra.run.smithi059:mds.e> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.e.service 2024-05-09T02:06:56.114 DEBUG:teuthology.orchestra.run.smithi059:mds.h> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.h.service 2024-05-09T02:06:56.157 DEBUG:teuthology.orchestra.run.smithi059:mds.k> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.k.service 2024-05-09T02:06:56.200 DEBUG:teuthology.orchestra.run.smithi196:mds.c> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.c.service 2024-05-09T02:06:56.203 DEBUG:teuthology.orchestra.run.smithi196:mds.f> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.f.service 2024-05-09T02:06:56.245 DEBUG:teuthology.orchestra.run.smithi196:mds.i> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.i.service 2024-05-09T02:06:56.288 DEBUG:teuthology.orchestra.run.smithi196:mds.l> sudo journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.l.service 2024-05-09T02:06:56.331 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-09T02:06:56.331 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:06:56.521 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:56 smithi196 ceph-mon[34718]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:06:56.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:56 smithi196 ceph-mon[34718]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-05-09T02:06:56.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:56 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:56.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:56 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:06:56.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:56 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:56.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:56 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:06:56.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:56 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:06:56.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:56 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:56.585 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:06:56.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:56 smithi019 ceph-mon[30151]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:06:56.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:56 smithi019 ceph-mon[30151]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-05-09T02:06:56.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:56 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:56.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:56 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:06:56.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:56 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:56.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:56 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:06:56.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:56 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:06:56.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:56 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:56 smithi059 ceph-mon[34781]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:06:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:56 smithi059 ceph-mon[34781]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-05-09T02:06:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:56 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:56 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:06:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:56 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:56 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:06:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:56 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:06:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:56 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:57.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:57 smithi059 ceph-mon[34781]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:57.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:57 smithi059 ceph-mon[34781]: Deploying daemon mds.l on smithi196 2024-05-09T02:06:57.739 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:57 smithi019 ceph-mon[30151]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:57.739 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:57 smithi019 ceph-mon[30151]: Deploying daemon mds.l on smithi196 2024-05-09T02:06:57.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:57 smithi196 ceph-mon[34718]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:57.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:57 smithi196 ceph-mon[34718]: Deploying daemon mds.l on smithi196 2024-05-09T02:06:58.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.396+0000 7f57583c9700 1 -- 172.21.15.19:0/1658277073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5750108740 msgr2=0x7f57501100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:58.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.396+0000 7f57583c9700 1 --2- 172.21.15.19:0/1658277073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5750108740 0x7f57501100a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f574800b0d0 tx=0x7f574800b3e0 comp rx=0 tx=0).stop 2024-05-09T02:06:58.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.396+0000 7f57583c9700 1 -- 172.21.15.19:0/1658277073 shutdown_connections 2024-05-09T02:06:58.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.396+0000 7f57583c9700 1 --2- 172.21.15.19:0/1658277073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5750108740 0x7f57501100a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:58.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.396+0000 7f57583c9700 1 --2- 172.21.15.19:0/1658277073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57501034c0 0x7f57501038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:58.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.396+0000 7f57583c9700 1 --2- 172.21.15.19:0/1658277073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5750107df0 0x7f5750108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:58.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.396+0000 7f57583c9700 1 -- 172.21.15.19:0/1658277073 >> 172.21.15.19:0/1658277073 conn(0x7f57500fedf0 msgr2=0x7f5750101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:06:58.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.396+0000 7f57583c9700 1 -- 172.21.15.19:0/1658277073 shutdown_connections 2024-05-09T02:06:58.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.396+0000 7f57583c9700 1 -- 172.21.15.19:0/1658277073 wait complete. 2024-05-09T02:06:58.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.397+0000 7f57583c9700 1 Processor -- start 2024-05-09T02:06:58.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.397+0000 7f57583c9700 1 -- start start 2024-05-09T02:06:58.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f57583c9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57501034c0 0x7f575019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:58.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f57583c9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5750107df0 0x7f575019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:58.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f57583c9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5750108740 0x7f575019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f57583c9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f57501144c0 con 0x7f5750108740 2024-05-09T02:06:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f57583c9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5750114340 con 0x7f5750107df0 2024-05-09T02:06:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f57583c9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5750114640 con 0x7f57501034c0 2024-05-09T02:06:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f5756966700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5750108740 0x7f575019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f5756966700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5750108740 0x7f575019f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40856/0 (socket says 172.21.15.19:40856) 2024-05-09T02:06:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f5756966700 1 -- 172.21.15.19:0/975660080 learned_addr learned my addr 172.21.15.19:0/975660080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:06:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f5756165700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57501034c0 0x7f575019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f5755964700 1 --2- 172.21.15.19:0/975660080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5750107df0 0x7f575019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f5756966700 1 -- 172.21.15.19:0/975660080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57501034c0 msgr2=0x7f575019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f5756966700 1 --2- 172.21.15.19:0/975660080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57501034c0 0x7f575019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f5756966700 1 -- 172.21.15.19:0/975660080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5750107df0 msgr2=0x7f575019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f5756966700 1 --2- 172.21.15.19:0/975660080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5750107df0 0x7f575019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f5756966700 1 -- 172.21.15.19:0/975660080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575019fc80 con 0x7f5750108740 2024-05-09T02:06:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f5756165700 1 --2- 172.21.15.19:0/975660080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57501034c0 0x7f575019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:06:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f5756966700 1 --2- 172.21.15.19:0/975660080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5750108740 0x7f575019f5e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5748009850 tx=0x7f5748007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:06:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.398+0000 7f57477fe700 1 -- 172.21.15.19:0/975660080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574801f070 con 0x7f5750108740 2024-05-09T02:06:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.399+0000 7f57583c9700 1 -- 172.21.15.19:0/975660080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575019ff10 con 0x7f5750108740 2024-05-09T02:06:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.399+0000 7f57477fe700 1 -- 172.21.15.19:0/975660080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5748004c50 con 0x7f5750108740 2024-05-09T02:06:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.399+0000 7f57477fe700 1 -- 172.21.15.19:0/975660080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574801b7b0 con 0x7f5750108740 2024-05-09T02:06:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.399+0000 7f57583c9700 1 -- 172.21.15.19:0/975660080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57501a03f0 con 0x7f5750108740 2024-05-09T02:06:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.399+0000 7f57583c9700 1 -- 172.21.15.19:0/975660080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57500512c0 con 0x7f5750108740 2024-05-09T02:06:58.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.402+0000 7f57477fe700 1 -- 172.21.15.19:0/975660080 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5748022070 con 0x7f5750108740 2024-05-09T02:06:58.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.402+0000 7f57477fe700 1 --2- 172.21.15.19:0/975660080 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f573c0b2880 0x7f573c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:06:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.403+0000 7f5756165700 1 --2- 172.21.15.19:0/975660080 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f573c0b2880 0x7f573c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:06:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.403+0000 7f57477fe700 1 -- 172.21.15.19:0/975660080 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f57480e2df0 con 0x7f5750108740 2024-05-09T02:06:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.403+0000 7f5756165700 1 --2- 172.21.15.19:0/975660080 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f573c0b2880 0x7f573c0b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f574c011490 tx=0x7f574c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:06:58.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.406+0000 7f57477fe700 1 -- 172.21.15.19:0/975660080 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f57480ab040 con 0x7f5750108740 2024-05-09T02:06:58.544 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:06:58 smithi196 systemd[1]: Starting Ceph mds.l for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:06:58.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.667+0000 7f57583c9700 1 -- 172.21.15.19:0/975660080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f575002e5f0 con 0x7f5750108740 2024-05-09T02:06:58.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.667+0000 7f57477fe700 1 -- 172.21.15.19:0/975660080 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f5748009cb0 con 0x7f5750108740 2024-05-09T02:06:58.670 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:06:58.670 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-09T02:06:58.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.669+0000 7f57583c9700 1 -- 172.21.15.19:0/975660080 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f573c0b2880 msgr2=0x7f573c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:58.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.669+0000 7f57583c9700 1 --2- 172.21.15.19:0/975660080 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f573c0b2880 0x7f573c0b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f574c011490 tx=0x7f574c009260 comp rx=0 tx=0).stop 2024-05-09T02:06:58.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.669+0000 7f57583c9700 1 -- 172.21.15.19:0/975660080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5750108740 msgr2=0x7f575019f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:06:58.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.669+0000 7f57583c9700 1 --2- 172.21.15.19:0/975660080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5750108740 0x7f575019f5e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5748009850 tx=0x7f5748007a20 comp rx=0 tx=0).stop 2024-05-09T02:06:58.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.669+0000 7f57583c9700 1 -- 172.21.15.19:0/975660080 shutdown_connections 2024-05-09T02:06:58.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.669+0000 7f57583c9700 1 --2- 172.21.15.19:0/975660080 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f573c0b2880 0x7f573c0b4d90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:58.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.669+0000 7f57583c9700 1 --2- 172.21.15.19:0/975660080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5750108740 0x7f575019f5e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:58.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.669+0000 7f57583c9700 1 --2- 172.21.15.19:0/975660080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57501034c0 0x7f575019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:58.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.669+0000 7f57583c9700 1 --2- 172.21.15.19:0/975660080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5750107df0 0x7f575019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:06:58.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.669+0000 7f57583c9700 1 -- 172.21.15.19:0/975660080 >> 172.21.15.19:0/975660080 conn(0x7f57500fedf0 msgr2=0x7f57501096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:06:58.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.670+0000 7f57583c9700 1 -- 172.21.15.19:0/975660080 shutdown_connections 2024-05-09T02:06:58.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:06:58.670+0000 7f57583c9700 1 -- 172.21.15.19:0/975660080 wait complete. 2024-05-09T02:06:58.673 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 1 2024-05-09T02:06:58.860 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:06:58 smithi196 podman[64664]: 2024-05-09 02:06:58.471716299 +0000 UTC m=+0.018313313 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:06:58.860 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:06:58 smithi196 podman[64664]: 2024-05-09 02:06:58.615148588 +0000 UTC m=+0.161745636 container create c0f206047bfa50a15c733dbdc90c585bfc07fcb6894e07d58a19871eed5164bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-09T02:06:58.860 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:06:58 smithi196 podman[64664]: 2024-05-09 02:06:58.859365615 +0000 UTC m=+0.405962618 container init c0f206047bfa50a15c733dbdc90c585bfc07fcb6894e07d58a19871eed5164bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-09T02:06:59.294 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:06:58 smithi196 podman[64664]: 2024-05-09 02:06:58.861941376 +0000 UTC m=+0.408538390 container start c0f206047bfa50a15c733dbdc90c585bfc07fcb6894e07d58a19871eed5164bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-l, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T02:06:59.294 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:06:58 smithi196 ceph-mds[64683]: -- 172.21.15.196:0/3239510931 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561053b17a40 con 0x561053b5fa80 2024-05-09T02:06:59.294 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:06:58 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-l[64679]: starting mds.l at 2024-05-09T02:06:59.294 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:06:58 smithi196 bash[64664]: c0f206047bfa50a15c733dbdc90c585bfc07fcb6894e07d58a19871eed5164bf 2024-05-09T02:06:59.295 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:06:58 smithi196 systemd[1]: Started Ceph mds.l for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:06:59.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:59 smithi059 ceph-mon[34781]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:59.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:59 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/975660080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:06:59.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:59 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.196:6832/416853517,v1:172.21.15.196:6833/416853517] up:boot 2024-05-09T02:06:59.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:59 smithi059 ceph-mon[34781]: fsmap 1 up:standby 2024-05-09T02:06:59.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:59 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T02:06:59.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:59 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:59.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:59 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:59.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:59 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:59.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:59 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:06:59.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:59 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:06:59.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:06:59 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:59.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:59 smithi196 ceph-mon[34718]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:59.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:59 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/975660080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:06:59.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:59 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.196:6832/416853517,v1:172.21.15.196:6833/416853517] up:boot 2024-05-09T02:06:59.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:59 smithi196 ceph-mon[34718]: fsmap 1 up:standby 2024-05-09T02:06:59.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:59 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T02:06:59.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:59 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:59.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:59 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:59.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:59 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:59.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:59 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:06:59.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:59 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:06:59.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:06:59 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:06:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:59 smithi019 ceph-mon[30151]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:06:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:59 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/975660080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:06:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:59 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.196:6832/416853517,v1:172.21.15.196:6833/416853517] up:boot 2024-05-09T02:06:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:59 smithi019 ceph-mon[30151]: fsmap 1 up:standby 2024-05-09T02:06:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:59 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T02:06:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:59 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:59 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:59 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:06:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:59 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:06:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:59 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:06:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:06:59 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:00 smithi059 ceph-mon[34781]: Deploying daemon mds.k on smithi059 2024-05-09T02:07:00.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:00 smithi196 ceph-mon[34718]: Deploying daemon mds.k on smithi059 2024-05-09T02:07:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:00 smithi019 ceph-mon[30151]: Deploying daemon mds.k on smithi059 2024-05-09T02:07:01.436 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:07:01 smithi059 systemd[1]: Starting Ceph mds.k for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:07:01.477 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:07:01.705 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:01 smithi059 ceph-mon[34781]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:01.705 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:07:01 smithi059 podman[63815]: 2024-05-09 02:07:01.492258389 +0000 UTC m=+0.018675681 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:07:01.705 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:07:01 smithi059 podman[63815]: 2024-05-09 02:07:01.595955199 +0000 UTC m=+0.122372458 container create 23ac5e3c1872106d001bfadb096566002393d4c9d961fffeb01435e74a40b4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-k, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T02:07:01.741 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:07:01.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:01 smithi196 ceph-mon[34718]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:01.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:01 smithi019 ceph-mon[30151]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:01.966 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:07:01 smithi059 podman[63815]: 2024-05-09 02:07:01.82378325 +0000 UTC m=+0.350200517 container init 23ac5e3c1872106d001bfadb096566002393d4c9d961fffeb01435e74a40b4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-k, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-09T02:07:01.966 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:07:01 smithi059 podman[63815]: 2024-05-09 02:07:01.826589777 +0000 UTC m=+0.353007036 container start 23ac5e3c1872106d001bfadb096566002393d4c9d961fffeb01435e74a40b4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-k, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T02:07:01.966 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:07:01 smithi059 ceph-mds[63835]: -- 172.21.15.59:0/4152967564 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bc7ea29a40 con 0x55bc7ea71a80 2024-05-09T02:07:01.966 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:07:01 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-k[63830]: starting mds.k at 2024-05-09T02:07:01.966 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:07:01 smithi059 bash[63815]: 23ac5e3c1872106d001bfadb096566002393d4c9d961fffeb01435e74a40b4f7 2024-05-09T02:07:01.966 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:07:01 smithi059 systemd[1]: Started Ceph mds.k for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:07:03.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:03.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:03.192 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:02 smithi019 ceph-mon[30151]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:03.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:03.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:03.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:03.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:03.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:02 smithi019 ceph-mon[30151]: Deploying daemon mds.j on smithi019 2024-05-09T02:07:03.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:02 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.59:6832/2040289281,v1:172.21.15.59:6833/2040289281] up:boot 2024-05-09T02:07:03.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:02 smithi019 ceph-mon[30151]: fsmap 2 up:standby 2024-05-09T02:07:03.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:02 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T02:07:03.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:03.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:03.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:02 smithi196 ceph-mon[34718]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:03.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:03.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:03.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:03.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:03.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:02 smithi196 ceph-mon[34718]: Deploying daemon mds.j on smithi019 2024-05-09T02:07:03.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:02 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.59:6832/2040289281,v1:172.21.15.59:6833/2040289281] up:boot 2024-05-09T02:07:03.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:02 smithi196 ceph-mon[34718]: fsmap 2 up:standby 2024-05-09T02:07:03.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:02 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T02:07:03.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:03.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:03.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:02 smithi059 ceph-mon[34781]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:03.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:03.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:03.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:03.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:03.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:02 smithi059 ceph-mon[34781]: Deploying daemon mds.j on smithi019 2024-05-09T02:07:03.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:02 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.59:6832/2040289281,v1:172.21.15.59:6833/2040289281] up:boot 2024-05-09T02:07:03.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:02 smithi059 ceph-mon[34781]: fsmap 2 up:standby 2024-05-09T02:07:03.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:02 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T02:07:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.759+0000 7f7bde09d700 1 -- 172.21.15.19:0/809120034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bd81034c0 msgr2=0x7f7bd81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.759+0000 7f7bde09d700 1 --2- 172.21.15.19:0/809120034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bd81034c0 0x7f7bd81038d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80099c0 tx=0x7f7bc8009cd0 comp rx=0 tx=0).stop 2024-05-09T02:07:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.759+0000 7f7bde09d700 1 -- 172.21.15.19:0/809120034 shutdown_connections 2024-05-09T02:07:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.759+0000 7f7bde09d700 1 --2- 172.21.15.19:0/809120034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd8108740 0x7f7bd81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.759+0000 7f7bde09d700 1 --2- 172.21.15.19:0/809120034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bd81034c0 0x7f7bd81038d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.759+0000 7f7bde09d700 1 --2- 172.21.15.19:0/809120034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bd8107df0 0x7f7bd8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.759+0000 7f7bde09d700 1 -- 172.21.15.19:0/809120034 >> 172.21.15.19:0/809120034 conn(0x7f7bd80fedf0 msgr2=0x7f7bd8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.759+0000 7f7bde09d700 1 -- 172.21.15.19:0/809120034 shutdown_connections 2024-05-09T02:07:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.759+0000 7f7bde09d700 1 -- 172.21.15.19:0/809120034 wait complete. 2024-05-09T02:07:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.760+0000 7f7bde09d700 1 Processor -- start 2024-05-09T02:07:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.760+0000 7f7bde09d700 1 -- start start 2024-05-09T02:07:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.760+0000 7f7bde09d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bd81034c0 0x7f7bd819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.760+0000 7f7bde09d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd8107df0 0x7f7bd819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.760+0000 7f7bde09d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bd8108740 0x7f7bd819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.760+0000 7f7bde09d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd81133e0 con 0x7f7bd8108740 2024-05-09T02:07:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.760+0000 7f7bde09d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd8113260 con 0x7f7bd81034c0 2024-05-09T02:07:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.760+0000 7f7bde09d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd8113560 con 0x7f7bd8107df0 2024-05-09T02:07:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.760+0000 7f7bd7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bd8108740 0x7f7bd819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.760+0000 7f7bd7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bd8108740 0x7f7bd819c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40874/0 (socket says 172.21.15.19:40874) 2024-05-09T02:07:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.760+0000 7f7bd7fff700 1 -- 172.21.15.19:0/1494195478 learned_addr learned my addr 172.21.15.19:0/1494195478 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:07:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.761+0000 7f7bd77fe700 1 --2- 172.21.15.19:0/1494195478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bd81034c0 0x7f7bd819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.761+0000 7f7bd6ffd700 1 --2- 172.21.15.19:0/1494195478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd8107df0 0x7f7bd819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.761+0000 7f7bd7fff700 1 -- 172.21.15.19:0/1494195478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd8107df0 msgr2=0x7f7bd819c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.761+0000 7f7bd7fff700 1 --2- 172.21.15.19:0/1494195478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd8107df0 0x7f7bd819c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.761+0000 7f7bd7fff700 1 -- 172.21.15.19:0/1494195478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bd81034c0 msgr2=0x7f7bd819bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.761+0000 7f7bd7fff700 1 --2- 172.21.15.19:0/1494195478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bd81034c0 0x7f7bd819bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.761+0000 7f7bd7fff700 1 -- 172.21.15.19:0/1494195478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd81a7000 con 0x7f7bd8108740 2024-05-09T02:07:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.761+0000 7f7bd6ffd700 1 --2- 172.21.15.19:0/1494195478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd8107df0 0x7f7bd819c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:07:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.761+0000 7f7bd7fff700 1 --2- 172.21.15.19:0/1494195478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bd8108740 0x7f7bd819c7e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0007d70 tx=0x7f7bd000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.761+0000 7f7bd4ff9700 1 -- 172.21.15.19:0/1494195478 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0016020 con 0x7f7bd8108740 2024-05-09T02:07:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.761+0000 7f7bde09d700 1 -- 172.21.15.19:0/1494195478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd81a72f0 con 0x7f7bd8108740 2024-05-09T02:07:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.761+0000 7f7bd4ff9700 1 -- 172.21.15.19:0/1494195478 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bd000ad40 con 0x7f7bd8108740 2024-05-09T02:07:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.761+0000 7f7bd4ff9700 1 -- 172.21.15.19:0/1494195478 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd00115e0 con 0x7f7bd8108740 2024-05-09T02:07:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.761+0000 7f7bde09d700 1 -- 172.21.15.19:0/1494195478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd81a7830 con 0x7f7bd8108740 2024-05-09T02:07:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.762+0000 7f7bde09d700 1 -- 172.21.15.19:0/1494195478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd80512c0 con 0x7f7bd8108740 2024-05-09T02:07:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.764+0000 7f7bd4ff9700 1 -- 172.21.15.19:0/1494195478 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7bd0019020 con 0x7f7bd8108740 2024-05-09T02:07:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.765+0000 7f7bd4ff9700 1 --2- 172.21.15.19:0/1494195478 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7bc00b2880 0x7f7bc00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.765+0000 7f7bd77fe700 1 --2- 172.21.15.19:0/1494195478 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7bc00b2880 0x7f7bc00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.765+0000 7f7bd4ff9700 1 -- 172.21.15.19:0/1494195478 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f7bd00e2260 con 0x7f7bd8108740 2024-05-09T02:07:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.765+0000 7f7bd77fe700 1 --2- 172.21.15.19:0/1494195478 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7bc00b2880 0x7f7bc00b4d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009990 tx=0x7f7bc8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:03.768+0000 7f7bd4ff9700 1 -- 172.21.15.19:0/1494195478 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7bd0020020 con 0x7f7bd8108740 2024-05-09T02:07:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:04.035+0000 7f7bde09d700 1 -- 172.21.15.19:0/1494195478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bd802e5f0 con 0x7f7bd8108740 2024-05-09T02:07:04.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:04.035+0000 7f7bd4ff9700 1 -- 172.21.15.19:0/1494195478 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f7bd00aa420 con 0x7f7bd8108740 2024-05-09T02:07:04.037 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:07:04.038 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T02:07:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:04.037+0000 7f7bde09d700 1 -- 172.21.15.19:0/1494195478 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7bc00b2880 msgr2=0x7f7bc00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:04.037+0000 7f7bde09d700 1 --2- 172.21.15.19:0/1494195478 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7bc00b2880 0x7f7bc00b4d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009990 tx=0x7f7bc8000bc0 comp rx=0 tx=0).stop 2024-05-09T02:07:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:04.037+0000 7f7bde09d700 1 -- 172.21.15.19:0/1494195478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bd8108740 msgr2=0x7f7bd819c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:04.037+0000 7f7bde09d700 1 --2- 172.21.15.19:0/1494195478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bd8108740 0x7f7bd819c7e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0007d70 tx=0x7f7bd000a340 comp rx=0 tx=0).stop 2024-05-09T02:07:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:04.038+0000 7f7bde09d700 1 -- 172.21.15.19:0/1494195478 shutdown_connections 2024-05-09T02:07:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:04.038+0000 7f7bde09d700 1 --2- 172.21.15.19:0/1494195478 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7bc00b2880 0x7f7bc00b4d90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:04.038+0000 7f7bde09d700 1 --2- 172.21.15.19:0/1494195478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bd8108740 0x7f7bd819c7e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:04.038+0000 7f7bde09d700 1 --2- 172.21.15.19:0/1494195478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bd81034c0 0x7f7bd819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:04.038+0000 7f7bde09d700 1 --2- 172.21.15.19:0/1494195478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd8107df0 0x7f7bd819c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:04.038+0000 7f7bde09d700 1 -- 172.21.15.19:0/1494195478 >> 172.21.15.19:0/1494195478 conn(0x7f7bd80fedf0 msgr2=0x7f7bd81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:04.038+0000 7f7bde09d700 1 -- 172.21.15.19:0/1494195478 shutdown_connections 2024-05-09T02:07:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:04.038+0000 7f7bde09d700 1 -- 172.21.15.19:0/1494195478 wait complete. 2024-05-09T02:07:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 3 2024-05-09T02:07:04.267 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:04 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1494195478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:04.346 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:04 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1494195478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:04.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:04 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1494195478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:05.347 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:05 smithi019 ceph-mon[30151]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:05.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:05 smithi196 ceph-mon[34718]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:05.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:05 smithi059 ceph-mon[34781]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:05.991 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 02:07:05 smithi019 podman[68959]: 2024-05-09 02:07:05.989940381 +0000 UTC m=+0.144670772 container create 49737d7ffb42c9daeb5a0b2c7c05c5aeeb3285ea536a5032e0a877105bf49c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2) 2024-05-09T02:07:06.571 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 02:07:06 smithi019 podman[68959]: 2024-05-09 02:07:06.242858217 +0000 UTC m=+0.397588613 container init 49737d7ffb42c9daeb5a0b2c7c05c5aeeb3285ea536a5032e0a877105bf49c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-j, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-09T02:07:06.571 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 02:07:06 smithi019 podman[68959]: 2024-05-09 02:07:06.245751939 +0000 UTC m=+0.400482340 container start 49737d7ffb42c9daeb5a0b2c7c05c5aeeb3285ea536a5032e0a877105bf49c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-j, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T02:07:06.571 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 02:07:06 smithi019 ceph-mds[68978]: -- 172.21.15.19:0/3271013351 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565360931a40 con 0x565360979600 2024-05-09T02:07:06.571 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 02:07:06 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-j[68974]: starting mds.j at 2024-05-09T02:07:06.571 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 02:07:06 smithi019 bash[68959]: 49737d7ffb42c9daeb5a0b2c7c05c5aeeb3285ea536a5032e0a877105bf49c18 2024-05-09T02:07:06.571 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 02:07:06 smithi019 systemd[1]: Started Ceph mds.j for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:07:06.806 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:07:07.666 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:07:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:07 smithi059 ceph-mon[34781]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:07 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:07 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:07 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:07 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:07 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:07 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:07 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.19:6834/4121765116,v1:172.21.15.19:6835/4121765116] up:boot 2024-05-09T02:07:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:07 smithi059 ceph-mon[34781]: fsmap 3 up:standby 2024-05-09T02:07:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:07 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T02:07:07.791 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:07 smithi019 ceph-mon[30151]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:07.791 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:07 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:07.791 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:07 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:07.791 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:07 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:07.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:07 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:07.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:07 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:07.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:07 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:07.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:07 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.19:6834/4121765116,v1:172.21.15.19:6835/4121765116] up:boot 2024-05-09T02:07:07.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:07 smithi019 ceph-mon[30151]: fsmap 3 up:standby 2024-05-09T02:07:07.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:07 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T02:07:07.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:07 smithi196 ceph-mon[34718]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:07.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:07 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:07.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:07 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:07.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:07 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:07.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:07 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:07.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:07 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:07.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:07 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:07.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:07 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.19:6834/4121765116,v1:172.21.15.19:6835/4121765116] up:boot 2024-05-09T02:07:07.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:07 smithi196 ceph-mon[34718]: fsmap 3 up:standby 2024-05-09T02:07:07.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:07 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T02:07:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:08 smithi059 ceph-mon[34781]: Deploying daemon mds.d on smithi019 2024-05-09T02:07:08.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:08 smithi019 ceph-mon[30151]: Deploying daemon mds.d on smithi019 2024-05-09T02:07:08.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:08 smithi196 ceph-mon[34718]: Deploying daemon mds.d on smithi019 2024-05-09T02:07:09.347 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 02:07:09 smithi019 systemd[1]: Starting Ceph mds.d for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:07:09.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:09 smithi059 ceph-mon[34781]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:09.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:09 smithi019 ceph-mon[30151]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:09.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:09 smithi196 ceph-mon[34718]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.050+0000 7fe529e69700 1 -- 172.21.15.19:0/1307771996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe524107df0 msgr2=0x7fe524108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.050+0000 7fe529e69700 1 --2- 172.21.15.19:0/1307771996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe524107df0 0x7fe524108200 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe5180099c0 tx=0x7fe518009cd0 comp rx=0 tx=0).stop 2024-05-09T02:07:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.050+0000 7fe529e69700 1 -- 172.21.15.19:0/1307771996 shutdown_connections 2024-05-09T02:07:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.050+0000 7fe529e69700 1 --2- 172.21.15.19:0/1307771996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe524108740 0x7fe5241100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.050+0000 7fe529e69700 1 --2- 172.21.15.19:0/1307771996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5241034c0 0x7fe5241038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.050+0000 7fe529e69700 1 --2- 172.21.15.19:0/1307771996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe524107df0 0x7fe524108200 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.050+0000 7fe529e69700 1 -- 172.21.15.19:0/1307771996 >> 172.21.15.19:0/1307771996 conn(0x7fe5240fedf0 msgr2=0x7fe524101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.051+0000 7fe529e69700 1 -- 172.21.15.19:0/1307771996 shutdown_connections 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.051+0000 7fe529e69700 1 -- 172.21.15.19:0/1307771996 wait complete. 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.051+0000 7fe529e69700 1 Processor -- start 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.051+0000 7fe529e69700 1 -- start start 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe529e69700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5241034c0 0x7fe52419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe529e69700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe524107df0 0x7fe52419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe529e69700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe524108740 0x7fe52419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe529e69700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe524113350 con 0x7fe5241034c0 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe529e69700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241131d0 con 0x7fe524108740 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe529e69700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241134d0 con 0x7fe524107df0 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe5237fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5241034c0 0x7fe52419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe5237fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5241034c0 0x7fe52419ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48990/0 (socket says 172.21.15.19:48990) 2024-05-09T02:07:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe5237fe700 1 -- 172.21.15.19:0/1229117683 learned_addr learned my addr 172.21.15.19:0/1229117683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe523fff700 1 --2- 172.21.15.19:0/1229117683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe524108740 0x7fe52419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe522ffd700 1 --2- 172.21.15.19:0/1229117683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe524107df0 0x7fe52419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe5237fe700 1 -- 172.21.15.19:0/1229117683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe524107df0 msgr2=0x7fe52419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe5237fe700 1 --2- 172.21.15.19:0/1229117683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe524107df0 0x7fe52419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe5237fe700 1 -- 172.21.15.19:0/1229117683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe524108740 msgr2=0x7fe52419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe5237fe700 1 --2- 172.21.15.19:0/1229117683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe524108740 0x7fe52419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe5237fe700 1 -- 172.21.15.19:0/1229117683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52419fd00 con 0x7fe5241034c0 2024-05-09T02:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.052+0000 7fe522ffd700 1 --2- 172.21.15.19:0/1229117683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe524107df0 0x7fe52419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.053+0000 7fe5237fe700 1 --2- 172.21.15.19:0/1229117683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5241034c0 0x7fe52419ae30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe51400ada0 tx=0x7fe51400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.053+0000 7fe520ff9700 1 -- 172.21.15.19:0/1229117683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514017020 con 0x7fe5241034c0 2024-05-09T02:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.053+0000 7fe520ff9700 1 -- 172.21.15.19:0/1229117683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe51400ce30 con 0x7fe5241034c0 2024-05-09T02:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.053+0000 7fe520ff9700 1 -- 172.21.15.19:0/1229117683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514005480 con 0x7fe5241034c0 2024-05-09T02:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.053+0000 7fe529e69700 1 -- 172.21.15.19:0/1229117683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52419fff0 con 0x7fe5241034c0 2024-05-09T02:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.053+0000 7fe529e69700 1 -- 172.21.15.19:0/1229117683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5241a7910 con 0x7fe5241034c0 2024-05-09T02:07:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.054+0000 7fe529e69700 1 -- 172.21.15.19:0/1229117683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5240512c0 con 0x7fe5241034c0 2024-05-09T02:07:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.055+0000 7fe520ff9700 1 -- 172.21.15.19:0/1229117683 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe514004ad0 con 0x7fe5241034c0 2024-05-09T02:07:10.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.056+0000 7fe520ff9700 1 --2- 172.21.15.19:0/1229117683 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe50c0b2880 0x7fe50c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:10.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.057+0000 7fe522ffd700 1 --2- 172.21.15.19:0/1229117683 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe50c0b2880 0x7fe50c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:10.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.057+0000 7fe520ff9700 1 -- 172.21.15.19:0/1229117683 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fe5140e1be0 con 0x7fe5241034c0 2024-05-09T02:07:10.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.057+0000 7fe522ffd700 1 --2- 172.21.15.19:0/1229117683 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe50c0b2880 0x7fe50c0b4d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe518000c00 tx=0x7fe51801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:10.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.060+0000 7fe520ff9700 1 -- 172.21.15.19:0/1229117683 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe514021020 con 0x7fe5241034c0 2024-05-09T02:07:10.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.336+0000 7fe529e69700 1 -- 172.21.15.19:0/1229117683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe52402e5f0 con 0x7fe5241034c0 2024-05-09T02:07:10.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.336+0000 7fe520ff9700 1 -- 172.21.15.19:0/1229117683 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7fe5140a9d90 con 0x7fe5241034c0 2024-05-09T02:07:10.339 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:07:10.339 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T02:07:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.339+0000 7fe529e69700 1 -- 172.21.15.19:0/1229117683 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe50c0b2880 msgr2=0x7fe50c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.339+0000 7fe529e69700 1 --2- 172.21.15.19:0/1229117683 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe50c0b2880 0x7fe50c0b4d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe518000c00 tx=0x7fe51801a040 comp rx=0 tx=0).stop 2024-05-09T02:07:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.339+0000 7fe529e69700 1 -- 172.21.15.19:0/1229117683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5241034c0 msgr2=0x7fe52419ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.339+0000 7fe529e69700 1 --2- 172.21.15.19:0/1229117683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5241034c0 0x7fe52419ae30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe51400ada0 tx=0x7fe51400c330 comp rx=0 tx=0).stop 2024-05-09T02:07:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.339+0000 7fe529e69700 1 -- 172.21.15.19:0/1229117683 shutdown_connections 2024-05-09T02:07:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.339+0000 7fe529e69700 1 --2- 172.21.15.19:0/1229117683 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe50c0b2880 0x7fe50c0b4d90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.339+0000 7fe529e69700 1 --2- 172.21.15.19:0/1229117683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe524108740 0x7fe52419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.339+0000 7fe529e69700 1 --2- 172.21.15.19:0/1229117683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5241034c0 0x7fe52419ae30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:10.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.339+0000 7fe529e69700 1 --2- 172.21.15.19:0/1229117683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe524107df0 0x7fe52419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:10.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.339+0000 7fe529e69700 1 -- 172.21.15.19:0/1229117683 >> 172.21.15.19:0/1229117683 conn(0x7fe5240fedf0 msgr2=0x7fe524075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:10.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.339+0000 7fe529e69700 1 -- 172.21.15.19:0/1229117683 shutdown_connections 2024-05-09T02:07:10.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:10.339+0000 7fe529e69700 1 -- 172.21.15.19:0/1229117683 wait complete. 2024-05-09T02:07:10.342 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 4 2024-05-09T02:07:10.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:10 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1229117683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:10.571 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 02:07:10 smithi019 podman[69411]: 2024-05-09 02:07:10.208257416 +0000 UTC m=+0.019301756 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:07:10.571 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 02:07:10 smithi019 podman[69411]: 2024-05-09 02:07:10.328727093 +0000 UTC m=+0.139771421 container create 40e01ca93a14effc5b8c6abb05e611581eab598d776d02a95be6a9ba14404449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-09T02:07:10.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:10 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1229117683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:10.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:10 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1229117683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:11.104 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 02:07:10 smithi019 podman[69411]: 2024-05-09 02:07:10.825007743 +0000 UTC m=+0.636052064 container init 40e01ca93a14effc5b8c6abb05e611581eab598d776d02a95be6a9ba14404449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-09T02:07:11.104 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 02:07:10 smithi019 podman[69411]: 2024-05-09 02:07:10.828403703 +0000 UTC m=+0.639448033 container start 40e01ca93a14effc5b8c6abb05e611581eab598d776d02a95be6a9ba14404449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T02:07:11.104 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 02:07:10 smithi019 ceph-mds[69444]: -- 172.21.15.19:0/1526125498 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5564f5365a40 con 0x5564f53ada80 2024-05-09T02:07:11.104 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 02:07:10 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-d[69440]: starting mds.d at 2024-05-09T02:07:11.105 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 02:07:10 smithi019 bash[69411]: 40e01ca93a14effc5b8c6abb05e611581eab598d776d02a95be6a9ba14404449 2024-05-09T02:07:11.105 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 02:07:10 smithi019 systemd[1]: Started Ceph mds.d for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:07:11.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:11 smithi059 ceph-mon[34781]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:11.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:11 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:11.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:11 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:11.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:11 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:11.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:11 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:11.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:11 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:11.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:11 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:11.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:11 smithi196 ceph-mon[34718]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:11.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:11 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:11.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:11 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:11.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:11 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:11.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:11 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:11.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:11 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:11.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:11 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:11.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:11 smithi019 ceph-mon[30151]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:11.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:11 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:11.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:11 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:11.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:11 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:11.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:11 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:11.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:11 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:11.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:11 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:12.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:12 smithi059 ceph-mon[34781]: Deploying daemon mds.e on smithi059 2024-05-09T02:07:12.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:12 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.19:6836/2298435247,v1:172.21.15.19:6837/2298435247] up:boot 2024-05-09T02:07:12.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:12 smithi059 ceph-mon[34781]: fsmap 4 up:standby 2024-05-09T02:07:12.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:12 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T02:07:12.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:12 smithi196 ceph-mon[34718]: Deploying daemon mds.e on smithi059 2024-05-09T02:07:12.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:12 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.19:6836/2298435247,v1:172.21.15.19:6837/2298435247] up:boot 2024-05-09T02:07:12.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:12 smithi196 ceph-mon[34718]: fsmap 4 up:standby 2024-05-09T02:07:12.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:12 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T02:07:12.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:12 smithi019 ceph-mon[30151]: Deploying daemon mds.e on smithi059 2024-05-09T02:07:12.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:12 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.19:6836/2298435247,v1:172.21.15.19:6837/2298435247] up:boot 2024-05-09T02:07:12.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:12 smithi019 ceph-mon[30151]: fsmap 4 up:standby 2024-05-09T02:07:12.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:12 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T02:07:13.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:07:13.679 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:13 smithi059 ceph-mon[34781]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:13.679 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:07:13 smithi059 systemd[1]: Starting Ceph mds.e for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:07:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:07:13.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:13 smithi196 ceph-mon[34718]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:13.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:13 smithi019 ceph-mon[30151]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:13.937 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:07:13 smithi059 podman[64134]: 2024-05-09 02:07:13.580813367 +0000 UTC m=+0.018685854 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:07:13.937 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:07:13 smithi059 podman[64134]: 2024-05-09 02:07:13.691760041 +0000 UTC m=+0.129632502 container create f355fbdce2f62126f5f7495fb28b168c9967a5f3a3cf867e798bf6fded3b7175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-e, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-05-09T02:07:13.937 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:07:13 smithi059 podman[64134]: 2024-05-09 02:07:13.936215629 +0000 UTC m=+0.374088091 container init f355fbdce2f62126f5f7495fb28b168c9967a5f3a3cf867e798bf6fded3b7175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T02:07:14.216 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:07:13 smithi059 podman[64134]: 2024-05-09 02:07:13.939284374 +0000 UTC m=+0.377156842 container start f355fbdce2f62126f5f7495fb28b168c9967a5f3a3cf867e798bf6fded3b7175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-e, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-09T02:07:14.216 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:07:13 smithi059 ceph-mds[64153]: -- 172.21.15.59:0/486134452 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5592f23c3a40 con 0x5592f240ba80 2024-05-09T02:07:14.216 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:07:13 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-e[64149]: starting mds.e at 2024-05-09T02:07:14.216 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:07:14 smithi059 bash[64134]: f355fbdce2f62126f5f7495fb28b168c9967a5f3a3cf867e798bf6fded3b7175 2024-05-09T02:07:14.216 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:07:14 smithi059 systemd[1]: Started Ceph mds.e for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:07:15.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:15 smithi196 ceph-mon[34718]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:15.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:15.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:15.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:15.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:15.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:15.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:15.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:15 smithi196 ceph-mon[34718]: Deploying daemon mds.f on smithi196 2024-05-09T02:07:15.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:15 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.59:6834/978263217,v1:172.21.15.59:6835/978263217] up:boot 2024-05-09T02:07:15.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:15 smithi196 ceph-mon[34718]: fsmap 5 up:standby 2024-05-09T02:07:15.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:15 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T02:07:15.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:15 smithi019 ceph-mon[30151]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:15.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:15.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:15.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:15.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:15.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:15.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:15.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:15 smithi019 ceph-mon[30151]: Deploying daemon mds.f on smithi196 2024-05-09T02:07:15.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:15 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.59:6834/978263217,v1:172.21.15.59:6835/978263217] up:boot 2024-05-09T02:07:15.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:15 smithi019 ceph-mon[30151]: fsmap 5 up:standby 2024-05-09T02:07:15.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:15 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T02:07:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.462+0000 7f9429393700 1 -- 172.21.15.19:0/513813359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9424109850 msgr2=0x7f94241111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.462+0000 7f9429393700 1 --2- 172.21.15.19:0/513813359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9424109850 0x7f94241111b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f941c00b0d0 tx=0x7f941c00b3e0 comp rx=0 tx=0).stop 2024-05-09T02:07:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.462+0000 7f9429393700 1 -- 172.21.15.19:0/513813359 shutdown_connections 2024-05-09T02:07:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.462+0000 7f9429393700 1 --2- 172.21.15.19:0/513813359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9424109850 0x7f94241111b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.462+0000 7f9429393700 1 --2- 172.21.15.19:0/513813359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94241045d0 0x7f94241049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.462+0000 7f9429393700 1 --2- 172.21.15.19:0/513813359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9424108f00 0x7f9424109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.462+0000 7f9429393700 1 -- 172.21.15.19:0/513813359 >> 172.21.15.19:0/513813359 conn(0x7f94240ffea0 msgr2=0x7f94241022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:15.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.462+0000 7f9429393700 1 -- 172.21.15.19:0/513813359 shutdown_connections 2024-05-09T02:07:15.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.462+0000 7f9429393700 1 -- 172.21.15.19:0/513813359 wait complete. 2024-05-09T02:07:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.463+0000 7f9429393700 1 Processor -- start 2024-05-09T02:07:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.463+0000 7f9429393700 1 -- start start 2024-05-09T02:07:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.463+0000 7f9429393700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94241045d0 0x7f942419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.463+0000 7f9429393700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9424108f00 0x7f942419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.463+0000 7f9429393700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9424109850 0x7f942419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.463+0000 7f9429393700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9424114480 con 0x7f9424108f00 2024-05-09T02:07:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.463+0000 7f9429393700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9424114300 con 0x7f94241045d0 2024-05-09T02:07:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.463+0000 7f9429393700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9424114600 con 0x7f9424109850 2024-05-09T02:07:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.463+0000 7f94227fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9424108f00 0x7f942419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.463+0000 7f94227fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9424108f00 0x7f942419b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49016/0 (socket says 172.21.15.19:49016) 2024-05-09T02:07:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.463+0000 7f94227fc700 1 -- 172.21.15.19:0/207946785 learned_addr learned my addr 172.21.15.19:0/207946785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:07:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.464+0000 7f94237fe700 1 --2- 172.21.15.19:0/207946785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9424109850 0x7f942419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.464+0000 7f9422ffd700 1 --2- 172.21.15.19:0/207946785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94241045d0 0x7f942419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.464+0000 7f94227fc700 1 -- 172.21.15.19:0/207946785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9424109850 msgr2=0x7f942419f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.464+0000 7f94227fc700 1 --2- 172.21.15.19:0/207946785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9424109850 0x7f942419f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.464+0000 7f94227fc700 1 -- 172.21.15.19:0/207946785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94241045d0 msgr2=0x7f942419ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.464+0000 7f94227fc700 1 --2- 172.21.15.19:0/207946785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94241045d0 0x7f942419ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.464+0000 7f94227fc700 1 -- 172.21.15.19:0/207946785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942419fc60 con 0x7f9424108f00 2024-05-09T02:07:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.464+0000 7f9422ffd700 1 --2- 172.21.15.19:0/207946785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94241045d0 0x7f942419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:07:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.464+0000 7f94227fc700 1 --2- 172.21.15.19:0/207946785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9424108f00 0x7f942419b330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f941800d980 tx=0x7f941800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.464+0000 7f940bfff700 1 -- 172.21.15.19:0/207946785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418017070 con 0x7f9424108f00 2024-05-09T02:07:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.464+0000 7f940bfff700 1 -- 172.21.15.19:0/207946785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9418009de0 con 0x7f9424108f00 2024-05-09T02:07:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.464+0000 7f9429393700 1 -- 172.21.15.19:0/207946785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942419ff50 con 0x7f9424108f00 2024-05-09T02:07:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.464+0000 7f940bfff700 1 -- 172.21.15.19:0/207946785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941800f3c0 con 0x7f9424108f00 2024-05-09T02:07:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.464+0000 7f9429393700 1 -- 172.21.15.19:0/207946785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94241a7870 con 0x7f9424108f00 2024-05-09T02:07:15.468 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:15 smithi059 ceph-mon[34781]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:15.468 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:15.468 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:15.469 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:15.469 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:15.469 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:15.469 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:15.469 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:15 smithi059 ceph-mon[34781]: Deploying daemon mds.f on smithi196 2024-05-09T02:07:15.469 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:15 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.59:6834/978263217,v1:172.21.15.59:6835/978263217] up:boot 2024-05-09T02:07:15.469 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:15 smithi059 ceph-mon[34781]: fsmap 5 up:standby 2024-05-09T02:07:15.469 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:15 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T02:07:15.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.465+0000 7f9429393700 1 -- 172.21.15.19:0/207946785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94240512c0 con 0x7f9424108f00 2024-05-09T02:07:15.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.467+0000 7f940bfff700 1 -- 172.21.15.19:0/207946785 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9418009930 con 0x7f9424108f00 2024-05-09T02:07:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.468+0000 7f940bfff700 1 --2- 172.21.15.19:0/207946785 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f940c0b2900 0x7f940c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.468+0000 7f9422ffd700 1 --2- 172.21.15.19:0/207946785 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f940c0b2900 0x7f940c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.468+0000 7f940bfff700 1 -- 172.21.15.19:0/207946785 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f94180e1740 con 0x7f9424108f00 2024-05-09T02:07:15.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.469+0000 7f9422ffd700 1 --2- 172.21.15.19:0/207946785 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f940c0b2900 0x7f940c0b4e10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9414005950 tx=0x7f94140058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.472+0000 7f940bfff700 1 -- 172.21.15.19:0/207946785 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f94180a9980 con 0x7f9424108f00 2024-05-09T02:07:15.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.734+0000 7f9429393700 1 -- 172.21.15.19:0/207946785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f942419c0f0 con 0x7f9424108f00 2024-05-09T02:07:15.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.735+0000 7f940bfff700 1 -- 172.21.15.19:0/207946785 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7f94180a9280 con 0x7f9424108f00 2024-05-09T02:07:15.737 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:07:15.737 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-09T02:07:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.737+0000 7f9429393700 1 -- 172.21.15.19:0/207946785 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f940c0b2900 msgr2=0x7f940c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.737+0000 7f9429393700 1 --2- 172.21.15.19:0/207946785 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f940c0b2900 0x7f940c0b4e10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9414005950 tx=0x7f94140058e0 comp rx=0 tx=0).stop 2024-05-09T02:07:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.737+0000 7f9429393700 1 -- 172.21.15.19:0/207946785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9424108f00 msgr2=0x7f942419b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.737+0000 7f9429393700 1 --2- 172.21.15.19:0/207946785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9424108f00 0x7f942419b330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f941800d980 tx=0x7f941800dc90 comp rx=0 tx=0).stop 2024-05-09T02:07:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.737+0000 7f9429393700 1 -- 172.21.15.19:0/207946785 shutdown_connections 2024-05-09T02:07:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.737+0000 7f9429393700 1 --2- 172.21.15.19:0/207946785 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f940c0b2900 0x7f940c0b4e10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.737+0000 7f9429393700 1 --2- 172.21.15.19:0/207946785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9424109850 0x7f942419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.737+0000 7f9429393700 1 --2- 172.21.15.19:0/207946785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94241045d0 0x7f942419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.737+0000 7f9429393700 1 --2- 172.21.15.19:0/207946785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9424108f00 0x7f942419b330 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.737+0000 7f9429393700 1 -- 172.21.15.19:0/207946785 >> 172.21.15.19:0/207946785 conn(0x7f94240ffea0 msgr2=0x7f9424108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.737+0000 7f9429393700 1 -- 172.21.15.19:0/207946785 shutdown_connections 2024-05-09T02:07:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:15.737+0000 7f9429393700 1 -- 172.21.15.19:0/207946785 wait complete. 2024-05-09T02:07:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 6 2024-05-09T02:07:16.439 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:16 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/207946785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:16.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:16 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/207946785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:16.491 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:16 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/207946785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:17.044 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 02:07:16 smithi196 systemd[1]: Started Ceph mds.f for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:07:17.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:17 smithi059 ceph-mon[34781]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:17.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:17 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:17.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:17 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:17.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:17 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:17.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:17 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:17.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:17 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:17.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:17 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:17.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:17 smithi196 ceph-mon[34718]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:17.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:17 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:17.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:17 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:17.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:17 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:17.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:17 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:17.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:17 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:17.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:17 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:17.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:17 smithi019 ceph-mon[30151]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:17.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:17 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:17.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:17 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:17.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:17 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:17.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:17 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:17.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:17 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:17.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:17 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:18.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:18 smithi196 ceph-mon[34718]: Deploying daemon mds.c on smithi196 2024-05-09T02:07:18.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:18 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] up:boot 2024-05-09T02:07:18.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:18 smithi196 ceph-mon[34718]: fsmap 6 up:standby 2024-05-09T02:07:18.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:18 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T02:07:18.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:18 smithi059 ceph-mon[34781]: Deploying daemon mds.c on smithi196 2024-05-09T02:07:18.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:18 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] up:boot 2024-05-09T02:07:18.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:18 smithi059 ceph-mon[34781]: fsmap 6 up:standby 2024-05-09T02:07:18.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:18 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T02:07:18.470 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:07:18.500 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:18 smithi019 ceph-mon[30151]: Deploying daemon mds.c on smithi196 2024-05-09T02:07:18.500 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:18 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] up:boot 2024-05-09T02:07:18.500 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:18 smithi019 ceph-mon[30151]: fsmap 6 up:standby 2024-05-09T02:07:18.500 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:18 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T02:07:18.737 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:07:19.202 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:19 smithi019 ceph-mon[30151]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:19.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:19 smithi196 ceph-mon[34718]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:19.295 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:07:19 smithi196 systemd[1]: Starting Ceph mds.c for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:07:19.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:19 smithi059 ceph-mon[34781]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:19.745 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:07:19 smithi196 podman[65275]: 2024-05-09 02:07:19.236058357 +0000 UTC m=+0.018735181 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:07:19.745 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:07:19 smithi196 podman[65275]: 2024-05-09 02:07:19.365982937 +0000 UTC m=+0.148659743 container create 1b60ffd11ad0658943e02e8d038327b8d19b4ad7b89915d6e6a0ccae7f962f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-c, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-09T02:07:19.745 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:07:19 smithi196 podman[65275]: 2024-05-09 02:07:19.7437273 +0000 UTC m=+0.526404106 container init 1b60ffd11ad0658943e02e8d038327b8d19b4ad7b89915d6e6a0ccae7f962f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-c, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-09T02:07:20.044 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:07:19 smithi196 podman[65275]: 2024-05-09 02:07:19.746327578 +0000 UTC m=+0.529004387 container start 1b60ffd11ad0658943e02e8d038327b8d19b4ad7b89915d6e6a0ccae7f962f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-c, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-09T02:07:20.044 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:07:19 smithi196 ceph-mds[65295]: -- 172.21.15.196:0/1016627454 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d425439a40 con 0x55d425481600 2024-05-09T02:07:20.044 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:07:19 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-c[65291]: starting mds.c at 2024-05-09T02:07:20.044 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:07:19 smithi196 bash[65275]: 1b60ffd11ad0658943e02e8d038327b8d19b4ad7b89915d6e6a0ccae7f962f3b 2024-05-09T02:07:20.044 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:07:19 smithi196 systemd[1]: Started Ceph mds.c for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:07:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.415+0000 7f9ae2bcf700 1 -- 172.21.15.19:0/3888076511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc105480 msgr2=0x7f9adc105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.415+0000 7f9ae2bcf700 1 --2- 172.21.15.19:0/3888076511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc105480 0x7f9adc105890 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009990 tx=0x7f9ad0009ca0 comp rx=0 tx=0).stop 2024-05-09T02:07:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.415+0000 7f9ae2bcf700 1 -- 172.21.15.19:0/3888076511 shutdown_connections 2024-05-09T02:07:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.415+0000 7f9ae2bcf700 1 --2- 172.21.15.19:0/3888076511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9adc10e8f0 0x7f9adc110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.415+0000 7f9ae2bcf700 1 --2- 172.21.15.19:0/3888076511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc105480 0x7f9adc105890 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.415+0000 7f9ae2bcf700 1 --2- 172.21.15.19:0/3888076511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9adc105dd0 0x7f9adc10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.415+0000 7f9ae2bcf700 1 -- 172.21.15.19:0/3888076511 >> 172.21.15.19:0/3888076511 conn(0x7f9adc0fef30 msgr2=0x7f9adc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.415+0000 7f9ae2bcf700 1 -- 172.21.15.19:0/3888076511 shutdown_connections 2024-05-09T02:07:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.415+0000 7f9ae2bcf700 1 -- 172.21.15.19:0/3888076511 wait complete. 2024-05-09T02:07:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.416+0000 7f9ae2bcf700 1 Processor -- start 2024-05-09T02:07:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.416+0000 7f9ae2bcf700 1 -- start start 2024-05-09T02:07:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.416+0000 7f9ae2bcf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc105480 0x7f9adc19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.416+0000 7f9ae2bcf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9adc105dd0 0x7f9adc19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.416+0000 7f9ae2bcf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9adc10e8f0 0x7f9adc19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.416+0000 7f9ae2bcf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc114440 con 0x7f9adc105480 2024-05-09T02:07:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.416+0000 7f9ae2bcf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc1142c0 con 0x7f9adc10e8f0 2024-05-09T02:07:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.416+0000 7f9ae2bcf700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc1145c0 con 0x7f9adc105dd0 2024-05-09T02:07:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.417+0000 7f9ae096b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc105480 0x7f9adc19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.417+0000 7f9ae096b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc105480 0x7f9adc19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58968/0 (socket says 172.21.15.19:58968) 2024-05-09T02:07:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.417+0000 7f9ae096b700 1 -- 172.21.15.19:0/1649810918 learned_addr learned my addr 172.21.15.19:0/1649810918 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:07:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.417+0000 7f9adbfff700 1 --2- 172.21.15.19:0/1649810918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9adc105dd0 0x7f9adc19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.417+0000 7f9ae116c700 1 --2- 172.21.15.19:0/1649810918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9adc10e8f0 0x7f9adc19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.417+0000 7f9ae096b700 1 -- 172.21.15.19:0/1649810918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9adc105dd0 msgr2=0x7f9adc19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.417+0000 7f9ae096b700 1 --2- 172.21.15.19:0/1649810918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9adc105dd0 0x7f9adc19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.417+0000 7f9ae096b700 1 -- 172.21.15.19:0/1649810918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9adc10e8f0 msgr2=0x7f9adc19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.417+0000 7f9ae096b700 1 --2- 172.21.15.19:0/1649810918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9adc10e8f0 0x7f9adc19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.417+0000 7f9ae096b700 1 -- 172.21.15.19:0/1649810918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9adc19fcc0 con 0x7f9adc105480 2024-05-09T02:07:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.417+0000 7f9ae096b700 1 --2- 172.21.15.19:0/1649810918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc105480 0x7f9adc19adf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0000c00 tx=0x7f9ad0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.417+0000 7f9ad9ffb700 1 -- 172.21.15.19:0/1649810918 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0027070 con 0x7f9adc105480 2024-05-09T02:07:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.417+0000 7f9ad9ffb700 1 -- 172.21.15.19:0/1649810918 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ad0022470 con 0x7f9adc105480 2024-05-09T02:07:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.417+0000 7f9ae2bcf700 1 -- 172.21.15.19:0/1649810918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9adc19ff50 con 0x7f9adc105480 2024-05-09T02:07:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.417+0000 7f9ad9ffb700 1 -- 172.21.15.19:0/1649810918 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0017650 con 0x7f9adc105480 2024-05-09T02:07:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.418+0000 7f9ae2bcf700 1 -- 172.21.15.19:0/1649810918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9adc1a78d0 con 0x7f9adc105480 2024-05-09T02:07:20.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.419+0000 7f9ae2bcf700 1 -- 172.21.15.19:0/1649810918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9adc0512c0 con 0x7f9adc105480 2024-05-09T02:07:20.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.420+0000 7f9ad9ffb700 1 -- 172.21.15.19:0/1649810918 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9ad002a070 con 0x7f9adc105480 2024-05-09T02:07:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.421+0000 7f9ad9ffb700 1 --2- 172.21.15.19:0/1649810918 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f9acc0b2900 0x7f9acc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.421+0000 7f9adbfff700 1 --2- 172.21.15.19:0/1649810918 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f9acc0b2900 0x7f9acc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.421+0000 7f9ad9ffb700 1 -- 172.21.15.19:0/1649810918 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f9ad00e2ad0 con 0x7f9adc105480 2024-05-09T02:07:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.421+0000 7f9adbfff700 1 --2- 172.21.15.19:0/1649810918 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f9acc0b2900 0x7f9acc0b4e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9adc1023b0 tx=0x7f9ac8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.426+0000 7f9ad9ffb700 1 -- 172.21.15.19:0/1649810918 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9ad0013070 con 0x7f9adc105480 2024-05-09T02:07:20.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.688+0000 7f9ae2bcf700 1 -- 172.21.15.19:0/1649810918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9adc19c150 con 0x7f9adc105480 2024-05-09T02:07:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.688+0000 7f9ad9ffb700 1 -- 172.21.15.19:0/1649810918 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6343 (secure 0 0 0) 0x7f9ad00aac80 con 0x7f9adc105480 2024-05-09T02:07:20.691 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:07:20.691 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-09T02:07:20.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.691+0000 7f9ae2bcf700 1 -- 172.21.15.19:0/1649810918 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f9acc0b2900 msgr2=0x7f9acc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:20.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.691+0000 7f9ae2bcf700 1 --2- 172.21.15.19:0/1649810918 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f9acc0b2900 0x7f9acc0b4e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9adc1023b0 tx=0x7f9ac8009260 comp rx=0 tx=0).stop 2024-05-09T02:07:20.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.691+0000 7f9ae2bcf700 1 -- 172.21.15.19:0/1649810918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc105480 msgr2=0x7f9adc19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:20.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.691+0000 7f9ae2bcf700 1 --2- 172.21.15.19:0/1649810918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc105480 0x7f9adc19adf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0000c00 tx=0x7f9ad0009580 comp rx=0 tx=0).stop 2024-05-09T02:07:20.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.691+0000 7f9ae2bcf700 1 -- 172.21.15.19:0/1649810918 shutdown_connections 2024-05-09T02:07:20.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.691+0000 7f9ae2bcf700 1 --2- 172.21.15.19:0/1649810918 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f9acc0b2900 0x7f9acc0b4e10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:20.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.691+0000 7f9ae2bcf700 1 --2- 172.21.15.19:0/1649810918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9adc10e8f0 0x7f9adc19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:20.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.691+0000 7f9ae2bcf700 1 --2- 172.21.15.19:0/1649810918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc105480 0x7f9adc19adf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:20.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.691+0000 7f9ae2bcf700 1 --2- 172.21.15.19:0/1649810918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9adc105dd0 0x7f9adc19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:20.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.691+0000 7f9ae2bcf700 1 -- 172.21.15.19:0/1649810918 >> 172.21.15.19:0/1649810918 conn(0x7f9adc0fef30 msgr2=0x7f9adc0ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:20.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.691+0000 7f9ae2bcf700 1 -- 172.21.15.19:0/1649810918 shutdown_connections 2024-05-09T02:07:20.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:20.691+0000 7f9ae2bcf700 1 -- 172.21.15.19:0/1649810918 wait complete. 2024-05-09T02:07:20.694 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-05-09T02:07:21.065 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:20 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:21.065 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:20 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:21.065 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:20 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:21.065 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:20 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:21.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:20 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:21.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:20 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:21.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:20 smithi019 ceph-mon[30151]: Deploying daemon mds.h on smithi059 2024-05-09T02:07:21.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:20 smithi019 ceph-mon[30151]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:21.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:20 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] up:boot 2024-05-09T02:07:21.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:20 smithi019 ceph-mon[30151]: fsmap 7 up:standby 2024-05-09T02:07:21.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:20 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T02:07:21.066 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:20 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1649810918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:21.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:20 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:21.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:20 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:21.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:20 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:21.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:20 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:21.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:20 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:21.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:20 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:21.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:20 smithi059 ceph-mon[34781]: Deploying daemon mds.h on smithi059 2024-05-09T02:07:21.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:20 smithi059 ceph-mon[34781]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:21.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:20 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] up:boot 2024-05-09T02:07:21.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:20 smithi059 ceph-mon[34781]: fsmap 7 up:standby 2024-05-09T02:07:21.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:20 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T02:07:21.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:20 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1649810918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:21.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:20 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:21.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:20 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:21.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:20 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:21.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:20 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:21.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:20 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:21.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:20 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:21.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:20 smithi196 ceph-mon[34718]: Deploying daemon mds.h on smithi059 2024-05-09T02:07:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:20 smithi196 ceph-mon[34718]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:20 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] up:boot 2024-05-09T02:07:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:20 smithi196 ceph-mon[34718]: fsmap 7 up:standby 2024-05-09T02:07:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:20 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T02:07:21.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:20 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1649810918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:22.407 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:07:22 smithi059 systemd[1]: Starting Ceph mds.h for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:07:22.681 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:07:22 smithi059 podman[64437]: 2024-05-09 02:07:22.308585633 +0000 UTC m=+0.018920945 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:07:22.681 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:07:22 smithi059 podman[64437]: 2024-05-09 02:07:22.553188812 +0000 UTC m=+0.263524159 container create 3c406c8ba9fd450955bf3ac46406466a9b922b359fe64526a6c349a9f9df7a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-09T02:07:22.966 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:07:22 smithi059 podman[64437]: 2024-05-09 02:07:22.814489174 +0000 UTC m=+0.524824466 container init 3c406c8ba9fd450955bf3ac46406466a9b922b359fe64526a6c349a9f9df7a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-09T02:07:22.966 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:07:22 smithi059 podman[64437]: 2024-05-09 02:07:22.817521393 +0000 UTC m=+0.527856681 container start 3c406c8ba9fd450955bf3ac46406466a9b922b359fe64526a6c349a9f9df7a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T02:07:22.966 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:07:22 smithi059 ceph-mds[64456]: -- 172.21.15.59:0/430017000 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e49deda40 con 0x559e49e35600 2024-05-09T02:07:22.966 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:07:22 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-h[64452]: starting mds.h at 2024-05-09T02:07:22.967 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:07:22 smithi059 bash[64437]: 3c406c8ba9fd450955bf3ac46406466a9b922b359fe64526a6c349a9f9df7a32 2024-05-09T02:07:22.967 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:07:22 smithi059 systemd[1]: Started Ceph mds.h for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:07:23.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:23 smithi019 ceph-mon[30151]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:23.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:23.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:23.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:23.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:23.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:23.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:23 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:23.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:07:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:23 smithi059 ceph-mon[34781]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:23 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:23.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:23 smithi196 ceph-mon[34718]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:23.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:23.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:23.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:23.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:23.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:23.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:23 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:23.709 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:07:24.102 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:24 smithi019 ceph-mon[30151]: Deploying daemon mds.i on smithi196 2024-05-09T02:07:24.102 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:24 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] up:boot 2024-05-09T02:07:24.102 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:24 smithi019 ceph-mon[30151]: fsmap 8 up:standby 2024-05-09T02:07:24.103 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T02:07:24.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:24 smithi196 ceph-mon[34718]: Deploying daemon mds.i on smithi196 2024-05-09T02:07:24.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:24 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] up:boot 2024-05-09T02:07:24.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:24 smithi196 ceph-mon[34718]: fsmap 8 up:standby 2024-05-09T02:07:24.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T02:07:24.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:24 smithi059 ceph-mon[34781]: Deploying daemon mds.i on smithi196 2024-05-09T02:07:24.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:24 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] up:boot 2024-05-09T02:07:24.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:24 smithi059 ceph-mon[34781]: fsmap 8 up:standby 2024-05-09T02:07:24.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T02:07:25.204 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:25 smithi019 ceph-mon[30151]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:25.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:25 smithi196 ceph-mon[34718]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:25.295 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:07:25 smithi196 systemd[1]: Starting Ceph mds.i for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:07:25.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:25 smithi059 ceph-mon[34781]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:25.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.468+0000 7f733487d700 1 -- 172.21.15.19:0/106904486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73301034e0 msgr2=0x7f73301038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:25.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.468+0000 7f733487d700 1 --2- 172.21.15.19:0/106904486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73301034e0 0x7f73301038f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f73200099c0 tx=0x7f7320009cd0 comp rx=0 tx=0).stop 2024-05-09T02:07:25.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.468+0000 7f733487d700 1 -- 172.21.15.19:0/106904486 shutdown_connections 2024-05-09T02:07:25.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.468+0000 7f733487d700 1 --2- 172.21.15.19:0/106904486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7330108760 0x7f73301100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:25.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.468+0000 7f733487d700 1 --2- 172.21.15.19:0/106904486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73301034e0 0x7f73301038f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:25.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.468+0000 7f733487d700 1 --2- 172.21.15.19:0/106904486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7330107e10 0x7f7330108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:25.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.468+0000 7f733487d700 1 -- 172.21.15.19:0/106904486 >> 172.21.15.19:0/106904486 conn(0x7f73300fef30 msgr2=0x7f7330101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:25.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.468+0000 7f733487d700 1 -- 172.21.15.19:0/106904486 shutdown_connections 2024-05-09T02:07:25.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.468+0000 7f733487d700 1 -- 172.21.15.19:0/106904486 wait complete. 2024-05-09T02:07:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.469+0000 7f733487d700 1 Processor -- start 2024-05-09T02:07:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.469+0000 7f733487d700 1 -- start start 2024-05-09T02:07:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f733487d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73301034e0 0x7f7330199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f733487d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7330107e10 0x7f733019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f733487d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7330108760 0x7f733019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f733487d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73301133e0 con 0x7f7330107e10 2024-05-09T02:07:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f733487d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7330113260 con 0x7f73301034e0 2024-05-09T02:07:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f733487d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7330113560 con 0x7f7330108760 2024-05-09T02:07:25.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f732dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7330107e10 0x7f733019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:25.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f732dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7330107e10 0x7f733019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58978/0 (socket says 172.21.15.19:58978) 2024-05-09T02:07:25.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f732dd9b700 1 -- 172.21.15.19:0/3887155450 learned_addr learned my addr 172.21.15.19:0/3887155450 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:07:25.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f732ed9d700 1 --2- 172.21.15.19:0/3887155450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7330108760 0x7f733019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:25.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f732e59c700 1 --2- 172.21.15.19:0/3887155450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73301034e0 0x7f7330199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:25.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f732dd9b700 1 -- 172.21.15.19:0/3887155450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7330108760 msgr2=0x7f733019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:25.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f732dd9b700 1 --2- 172.21.15.19:0/3887155450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7330108760 0x7f733019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:25.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f732dd9b700 1 -- 172.21.15.19:0/3887155450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73301034e0 msgr2=0x7f7330199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:25.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f732dd9b700 1 --2- 172.21.15.19:0/3887155450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73301034e0 0x7f7330199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:25.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f732dd9b700 1 -- 172.21.15.19:0/3887155450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733019ec80 con 0x7f7330107e10 2024-05-09T02:07:25.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.470+0000 7f732dd9b700 1 --2- 172.21.15.19:0/3887155450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7330107e10 0x7f733019a350 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7318007b50 tx=0x7f7318007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:25.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.471+0000 7f73277fe700 1 -- 172.21.15.19:0/3887155450 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731800e500 con 0x7f7330107e10 2024-05-09T02:07:25.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.471+0000 7f733487d700 1 -- 172.21.15.19:0/3887155450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733019ef70 con 0x7f7330107e10 2024-05-09T02:07:25.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.471+0000 7f73277fe700 1 -- 172.21.15.19:0/3887155450 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f731800eb50 con 0x7f7330107e10 2024-05-09T02:07:25.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.471+0000 7f73277fe700 1 -- 172.21.15.19:0/3887155450 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731800aa80 con 0x7f7330107e10 2024-05-09T02:07:25.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.471+0000 7f733487d700 1 -- 172.21.15.19:0/3887155450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f733006c040 con 0x7f7330107e10 2024-05-09T02:07:25.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.472+0000 7f733487d700 1 -- 172.21.15.19:0/3887155450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73300512c0 con 0x7f7330107e10 2024-05-09T02:07:25.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.473+0000 7f73277fe700 1 -- 172.21.15.19:0/3887155450 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f731800ac80 con 0x7f7330107e10 2024-05-09T02:07:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.474+0000 7f73277fe700 1 --2- 172.21.15.19:0/3887155450 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f731c0b2880 0x7f731c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.474+0000 7f732e59c700 1 --2- 172.21.15.19:0/3887155450 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f731c0b2880 0x7f731c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.474+0000 7f73277fe700 1 -- 172.21.15.19:0/3887155450 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f73180e31d0 con 0x7f7330107e10 2024-05-09T02:07:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.475+0000 7f732e59c700 1 --2- 172.21.15.19:0/3887155450 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f731c0b2880 0x7f731c0b4d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7320003800 tx=0x7f732001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:25.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.479+0000 7f73277fe700 1 -- 172.21.15.19:0/3887155450 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7318023020 con 0x7f7330107e10 2024-05-09T02:07:25.676 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:07:25 smithi196 podman[65584]: 2024-05-09 02:07:25.306695131 +0000 UTC m=+0.018383319 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:07:25.676 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:07:25 smithi196 podman[65584]: 2024-05-09 02:07:25.413433071 +0000 UTC m=+0.125121300 container create 6e8623f5a409b3f80c0fc0edee7d872ad4789c304ca3c3e8a528f1357e7609c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-09T02:07:25.676 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:07:25 smithi196 podman[65584]: 2024-05-09 02:07:25.674681903 +0000 UTC m=+0.386370119 container init 6e8623f5a409b3f80c0fc0edee7d872ad4789c304ca3c3e8a528f1357e7609c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T02:07:25.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.743+0000 7f733487d700 1 -- 172.21.15.19:0/3887155450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f733006ca70 con 0x7f7330107e10 2024-05-09T02:07:25.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.743+0000 7f73277fe700 1 -- 172.21.15.19:0/3887155450 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7168 (secure 0 0 0) 0x7f73180ab390 con 0x7f7330107e10 2024-05-09T02:07:25.745 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:07:25.746 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-09T02:07:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.746+0000 7f733487d700 1 -- 172.21.15.19:0/3887155450 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f731c0b2880 msgr2=0x7f731c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.746+0000 7f733487d700 1 --2- 172.21.15.19:0/3887155450 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f731c0b2880 0x7f731c0b4d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7320003800 tx=0x7f732001a040 comp rx=0 tx=0).stop 2024-05-09T02:07:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.746+0000 7f733487d700 1 -- 172.21.15.19:0/3887155450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7330107e10 msgr2=0x7f733019a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.746+0000 7f733487d700 1 --2- 172.21.15.19:0/3887155450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7330107e10 0x7f733019a350 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7318007b50 tx=0x7f7318007e60 comp rx=0 tx=0).stop 2024-05-09T02:07:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.746+0000 7f733487d700 1 -- 172.21.15.19:0/3887155450 shutdown_connections 2024-05-09T02:07:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.746+0000 7f733487d700 1 --2- 172.21.15.19:0/3887155450 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f731c0b2880 0x7f731c0b4d90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.746+0000 7f733487d700 1 --2- 172.21.15.19:0/3887155450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7330108760 0x7f733019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.746+0000 7f733487d700 1 --2- 172.21.15.19:0/3887155450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73301034e0 0x7f7330199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.746+0000 7f733487d700 1 --2- 172.21.15.19:0/3887155450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7330107e10 0x7f733019a350 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.746+0000 7f733487d700 1 -- 172.21.15.19:0/3887155450 >> 172.21.15.19:0/3887155450 conn(0x7f73300fef30 msgr2=0x7f73301096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.746+0000 7f733487d700 1 -- 172.21.15.19:0/3887155450 shutdown_connections 2024-05-09T02:07:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:25.746+0000 7f733487d700 1 -- 172.21.15.19:0/3887155450 wait complete. 2024-05-09T02:07:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-05-09T02:07:26.044 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:07:25 smithi196 podman[65584]: 2024-05-09 02:07:25.677679442 +0000 UTC m=+0.389367640 container start 6e8623f5a409b3f80c0fc0edee7d872ad4789c304ca3c3e8a528f1357e7609c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T02:07:26.044 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:07:25 smithi196 ceph-mds[65603]: -- 172.21.15.196:0/1163127176 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5ce441a40 con 0x55f5ce489600 2024-05-09T02:07:26.044 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:07:25 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i[65599]: starting mds.i at 2024-05-09T02:07:26.045 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:07:25 smithi196 bash[65584]: 6e8623f5a409b3f80c0fc0edee7d872ad4789c304ca3c3e8a528f1357e7609c9 2024-05-09T02:07:26.045 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:07:25 smithi196 systemd[1]: Started Ceph mds.i for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:07:26.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:26 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3887155450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:26.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:26.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:26.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:26.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:26.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:26.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:26 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:26.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:26 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3887155450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:26.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:26.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:26.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:26.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:26.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:26.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:26 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:26 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3887155450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:26 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:27.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:27 smithi059 ceph-mon[34781]: Deploying daemon mds.g on smithi019 2024-05-09T02:07:27.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:27 smithi059 ceph-mon[34781]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:27.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:27 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] up:boot 2024-05-09T02:07:27.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:27 smithi059 ceph-mon[34781]: fsmap 9 up:standby 2024-05-09T02:07:27.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:27 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T02:07:27.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:27 smithi196 ceph-mon[34718]: Deploying daemon mds.g on smithi019 2024-05-09T02:07:27.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:27 smithi196 ceph-mon[34718]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:27.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:27 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] up:boot 2024-05-09T02:07:27.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:27 smithi196 ceph-mon[34718]: fsmap 9 up:standby 2024-05-09T02:07:27.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:27 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T02:07:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:27 smithi019 ceph-mon[30151]: Deploying daemon mds.g on smithi019 2024-05-09T02:07:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:27 smithi019 ceph-mon[30151]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:27 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] up:boot 2024-05-09T02:07:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:27 smithi019 ceph-mon[30151]: fsmap 9 up:standby 2024-05-09T02:07:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:27 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T02:07:28.321 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:07:28 smithi019 systemd[1]: Starting Ceph mds.g for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:07:28.413 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:07:28.611 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:07:28 smithi019 podman[70218]: 2024-05-09 02:07:28.23570589 +0000 UTC m=+0.018692928 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:07:28.611 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:07:28 smithi019 podman[70218]: 2024-05-09 02:07:28.350971817 +0000 UTC m=+0.133958842 container create 39eab8a6e5710a2b455524e879dce33337911ce1d1a185238b2c6b0e0d05e822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-g, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T02:07:28.611 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:07:28 smithi019 podman[70218]: 2024-05-09 02:07:28.570760125 +0000 UTC m=+0.353747151 container init 39eab8a6e5710a2b455524e879dce33337911ce1d1a185238b2c6b0e0d05e822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-g, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T02:07:28.611 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:07:28 smithi019 podman[70218]: 2024-05-09 02:07:28.573859216 +0000 UTC m=+0.356846244 container start 39eab8a6e5710a2b455524e879dce33337911ce1d1a185238b2c6b0e0d05e822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-g, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T02:07:28.761 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:07:28.885 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:07:28 smithi019 ceph-mds[70255]: -- 172.21.15.19:0/4212625588 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb7bbd7a40 con 0x55cb7bc1f600 2024-05-09T02:07:28.886 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:07:28 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-g[70241]: starting mds.g at 2024-05-09T02:07:28.886 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:07:28 smithi019 bash[70218]: 39eab8a6e5710a2b455524e879dce33337911ce1d1a185238b2c6b0e0d05e822 2024-05-09T02:07:28.886 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:07:28 smithi019 systemd[1]: Started Ceph mds.g for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:07:29.187 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:29 smithi019 ceph-mon[30151]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:29.187 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:29 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:29.187 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:29 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:29.187 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:29 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:29.187 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:29 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:29.187 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:29 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:29.187 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:29 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:29.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:29 smithi196 ceph-mon[34718]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:29.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:29 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:29.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:29 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:29.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:29 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:29.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:29 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:29.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:29 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:29.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:29 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:29.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:29 smithi059 ceph-mon[34781]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:29.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:29 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:29.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:29 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:29.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:29 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:29.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:29 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:29.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:29 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:29.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:29 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:30.264 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:30 smithi019 ceph-mon[30151]: Deploying daemon mds.a on smithi019 2024-05-09T02:07:30.264 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:30 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.19:6838/810391856,v1:172.21.15.19:6839/810391856] up:boot 2024-05-09T02:07:30.265 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:30 smithi019 ceph-mon[30151]: fsmap 10 up:standby 2024-05-09T02:07:30.265 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:30 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T02:07:30.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:30 smithi059 ceph-mon[34781]: Deploying daemon mds.a on smithi019 2024-05-09T02:07:30.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:30 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.19:6838/810391856,v1:172.21.15.19:6839/810391856] up:boot 2024-05-09T02:07:30.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:30 smithi059 ceph-mon[34781]: fsmap 10 up:standby 2024-05-09T02:07:30.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:30 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T02:07:30.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.523+0000 7fd9c909e700 1 -- 172.21.15.19:0/2452685695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c41092d0 msgr2=0x7fd9c41096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:30.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.523+0000 7fd9c909e700 1 --2- 172.21.15.19:0/2452685695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c41092d0 0x7fd9c41096e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009990 tx=0x7fd9b4009ca0 comp rx=0 tx=0).stop 2024-05-09T02:07:30.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.523+0000 7fd9c909e700 1 -- 172.21.15.19:0/2452685695 shutdown_connections 2024-05-09T02:07:30.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.523+0000 7fd9c909e700 1 --2- 172.21.15.19:0/2452685695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9c4104330 0x7fd9c4110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:30.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.523+0000 7fd9c909e700 1 --2- 172.21.15.19:0/2452685695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c41092d0 0x7fd9c41096e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:30.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.523+0000 7fd9c909e700 1 --2- 172.21.15.19:0/2452685695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c41039e0 0x7fd9c4103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:30.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.523+0000 7fd9c909e700 1 -- 172.21.15.19:0/2452685695 >> 172.21.15.19:0/2452685695 conn(0x7fd9c40fe540 msgr2=0x7fd9c4100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:30.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.523+0000 7fd9c909e700 1 -- 172.21.15.19:0/2452685695 shutdown_connections 2024-05-09T02:07:30.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.523+0000 7fd9c909e700 1 -- 172.21.15.19:0/2452685695 wait complete. 2024-05-09T02:07:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.524+0000 7fd9c909e700 1 Processor -- start 2024-05-09T02:07:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.524+0000 7fd9c909e700 1 -- start start 2024-05-09T02:07:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c909e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9c41039e0 0x7fd9c4104eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c909e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c4104330 0x7fd9c41053f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c909e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c41092d0 0x7fd9c4105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c909e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c4113370 con 0x7fd9c41092d0 2024-05-09T02:07:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c909e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41131f0 con 0x7fd9c41039e0 2024-05-09T02:07:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c909e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41134f0 con 0x7fd9c4104330 2024-05-09T02:07:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c41092d0 0x7fd9c4105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c41092d0 0x7fd9c4105930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44448/0 (socket says 172.21.15.19:44448) 2024-05-09T02:07:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c359e700 1 -- 172.21.15.19:0/649427737 learned_addr learned my addr 172.21.15.19:0/649427737 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:07:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c2d9d700 1 --2- 172.21.15.19:0/649427737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9c41039e0 0x7fd9c4104eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c259c700 1 --2- 172.21.15.19:0/649427737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c4104330 0x7fd9c41053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c359e700 1 -- 172.21.15.19:0/649427737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c4104330 msgr2=0x7fd9c41053f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c359e700 1 --2- 172.21.15.19:0/649427737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c4104330 0x7fd9c41053f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c359e700 1 -- 172.21.15.19:0/649427737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9c41039e0 msgr2=0x7fd9c4104eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c359e700 1 --2- 172.21.15.19:0/649427737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9c41039e0 0x7fd9c4104eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c359e700 1 -- 172.21.15.19:0/649427737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c41a91d0 con 0x7fd9c41092d0 2024-05-09T02:07:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.525+0000 7fd9c2d9d700 1 --2- 172.21.15.19:0/649427737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9c41039e0 0x7fd9c4104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:07:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.526+0000 7fd9c359e700 1 --2- 172.21.15.19:0/649427737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c41092d0 0x7fd9c4105930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00ada0 tx=0x7fd9bc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.526+0000 7fd9abfff700 1 -- 172.21.15.19:0/649427737 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc017020 con 0x7fd9c41092d0 2024-05-09T02:07:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.526+0000 7fd9c909e700 1 -- 172.21.15.19:0/649427737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c41a94c0 con 0x7fd9c41092d0 2024-05-09T02:07:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.526+0000 7fd9abfff700 1 -- 172.21.15.19:0/649427737 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9bc00ce30 con 0x7fd9c41092d0 2024-05-09T02:07:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.526+0000 7fd9abfff700 1 -- 172.21.15.19:0/649427737 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc005350 con 0x7fd9c41092d0 2024-05-09T02:07:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.526+0000 7fd9c909e700 1 -- 172.21.15.19:0/649427737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c41a9a00 con 0x7fd9c41092d0 2024-05-09T02:07:30.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.527+0000 7fd9c909e700 1 -- 172.21.15.19:0/649427737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c40512c0 con 0x7fd9c41092d0 2024-05-09T02:07:30.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.529+0000 7fd9abfff700 1 -- 172.21.15.19:0/649427737 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd9bc00c980 con 0x7fd9c41092d0 2024-05-09T02:07:30.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.530+0000 7fd9abfff700 1 --2- 172.21.15.19:0/649427737 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd9ac0b2880 0x7fd9ac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:30.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.530+0000 7fd9c2d9d700 1 --2- 172.21.15.19:0/649427737 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd9ac0b2880 0x7fd9ac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:30.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.530+0000 7fd9abfff700 1 -- 172.21.15.19:0/649427737 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fd9bc0e1a80 con 0x7fd9c41092d0 2024-05-09T02:07:30.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.530+0000 7fd9c2d9d700 1 --2- 172.21.15.19:0/649427737 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd9ac0b2880 0x7fd9ac0b4d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4006010 tx=0x7fd9b401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:30.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.533+0000 7fd9abfff700 1 -- 172.21.15.19:0/649427737 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd9bc021020 con 0x7fd9c41092d0 2024-05-09T02:07:30.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:30 smithi196 ceph-mon[34718]: Deploying daemon mds.a on smithi019 2024-05-09T02:07:30.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:30 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.19:6838/810391856,v1:172.21.15.19:6839/810391856] up:boot 2024-05-09T02:07:30.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:30 smithi196 ceph-mon[34718]: fsmap 10 up:standby 2024-05-09T02:07:30.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:30 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T02:07:30.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.803+0000 7fd9c909e700 1 -- 172.21.15.19:0/649427737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9c41a9c60 con 0x7fd9c41092d0 2024-05-09T02:07:30.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.803+0000 7fd9abfff700 1 -- 172.21.15.19:0/649427737 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8815 (secure 0 0 0) 0x7fd9bc0a9c30 con 0x7fd9c41092d0 2024-05-09T02:07:30.806 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:07:30.806 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T02:07:30.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.806+0000 7fd9c909e700 1 -- 172.21.15.19:0/649427737 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd9ac0b2880 msgr2=0x7fd9ac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:30.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.806+0000 7fd9c909e700 1 --2- 172.21.15.19:0/649427737 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd9ac0b2880 0x7fd9ac0b4d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4006010 tx=0x7fd9b401a040 comp rx=0 tx=0).stop 2024-05-09T02:07:30.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.806+0000 7fd9c909e700 1 -- 172.21.15.19:0/649427737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c41092d0 msgr2=0x7fd9c4105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.806+0000 7fd9c909e700 1 --2- 172.21.15.19:0/649427737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c41092d0 0x7fd9c4105930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00ada0 tx=0x7fd9bc00c330 comp rx=0 tx=0).stop 2024-05-09T02:07:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.806+0000 7fd9c909e700 1 -- 172.21.15.19:0/649427737 shutdown_connections 2024-05-09T02:07:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.806+0000 7fd9c909e700 1 --2- 172.21.15.19:0/649427737 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd9ac0b2880 0x7fd9ac0b4d90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.806+0000 7fd9c909e700 1 --2- 172.21.15.19:0/649427737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9c41092d0 0x7fd9c4105930 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.806+0000 7fd9c909e700 1 --2- 172.21.15.19:0/649427737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9c41039e0 0x7fd9c4104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.806+0000 7fd9c909e700 1 --2- 172.21.15.19:0/649427737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c4104330 0x7fd9c41053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.806+0000 7fd9c909e700 1 -- 172.21.15.19:0/649427737 >> 172.21.15.19:0/649427737 conn(0x7fd9c40fe540 msgr2=0x7fd9c4107150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.806+0000 7fd9c909e700 1 -- 172.21.15.19:0/649427737 shutdown_connections 2024-05-09T02:07:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:30.806+0000 7fd9c909e700 1 -- 172.21.15.19:0/649427737 wait complete. 2024-05-09T02:07:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 11 2024-05-09T02:07:31.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:31 smithi059 ceph-mon[34781]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:31.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:31 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/649427737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:31.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:31 smithi196 ceph-mon[34718]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:31.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:31 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/649427737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:31.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:31 smithi019 ceph-mon[30151]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:31.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:31 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/649427737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:32.685 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:07:32 smithi019 podman[70684]: 2024-05-09 02:07:32.549146194 +0000 UTC m=+0.018696140 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:07:32.685 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:07:32 smithi019 podman[70684]: 2024-05-09 02:07:32.683298023 +0000 UTC m=+0.152847951 container create 3dc436e93cf4dec1ddd220b0cd739038c6d5d2a25dfed94fc48c1c4f5f515cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-09T02:07:33.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:33 smithi019 ceph-mon[30151]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:33.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:33 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:33.321 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:07:32 smithi019 podman[70684]: 2024-05-09 02:07:32.944614202 +0000 UTC m=+0.414164128 container init 3dc436e93cf4dec1ddd220b0cd739038c6d5d2a25dfed94fc48c1c4f5f515cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-09T02:07:33.321 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:07:32 smithi019 podman[70684]: 2024-05-09 02:07:32.94767496 +0000 UTC m=+0.417224892 container start 3dc436e93cf4dec1ddd220b0cd739038c6d5d2a25dfed94fc48c1c4f5f515cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-a, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2) 2024-05-09T02:07:33.321 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:07:32 smithi019 ceph-mds[70703]: -- 172.21.15.19:0/2821076463 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be35679a40 con 0x55be356c1a80 2024-05-09T02:07:33.321 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:07:32 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-a[70699]: starting mds.a at 2024-05-09T02:07:33.322 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:07:33 smithi019 bash[70684]: 3dc436e93cf4dec1ddd220b0cd739038c6d5d2a25dfed94fc48c1c4f5f515cbe 2024-05-09T02:07:33.322 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:07:33 smithi019 systemd[1]: Started Ceph mds.a for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:07:33.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:33 smithi059 ceph-mon[34781]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:33.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:33 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:33 smithi196 ceph-mon[34718]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:33 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:34.378 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:07:34.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:34 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:34.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:34 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.19:6840/3162604494,v1:172.21.15.19:6841/3162604494] up:boot 2024-05-09T02:07:34.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:34 smithi019 ceph-mon[30151]: fsmap 11 up:standby 2024-05-09T02:07:34.406 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:34 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T02:07:34.406 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:34 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:34.406 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:34 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:34.406 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:34 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:34.406 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:34 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:34.406 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:34 smithi019 ceph-mon[30151]: Deploying daemon mds.b on smithi059 2024-05-09T02:07:34.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:34 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:34.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:34 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.19:6840/3162604494,v1:172.21.15.19:6841/3162604494] up:boot 2024-05-09T02:07:34.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:34 smithi059 ceph-mon[34781]: fsmap 11 up:standby 2024-05-09T02:07:34.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:34 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T02:07:34.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:34 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:34.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:34 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:34.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:34 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:34.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:34 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:34.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:34 smithi059 ceph-mon[34781]: Deploying daemon mds.b on smithi059 2024-05-09T02:07:34.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:34 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:34.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:34 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.19:6840/3162604494,v1:172.21.15.19:6841/3162604494] up:boot 2024-05-09T02:07:34.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:34 smithi196 ceph-mon[34718]: fsmap 11 up:standby 2024-05-09T02:07:34.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:34 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T02:07:34.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:34 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:34.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:34 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T02:07:34.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:34 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T02:07:34.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:34 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:34.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:34 smithi196 ceph-mon[34718]: Deploying daemon mds.b on smithi059 2024-05-09T02:07:34.651 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:07:35.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:35 smithi059 ceph-mon[34781]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:35.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:35 smithi019 ceph-mon[30151]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:35.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:35 smithi196 ceph-mon[34718]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:35.760 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:07:35 smithi059 podman[64744]: 2024-05-09 02:07:35.589729531 +0000 UTC m=+0.018703171 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:07:35.760 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:07:35 smithi059 podman[64744]: 2024-05-09 02:07:35.758736842 +0000 UTC m=+0.187710478 container create de12d771cb4367d534f6990c4294529011a99daa99ea3e84c05666456b75de69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-b, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-09T02:07:36.302 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:07:36 smithi059 podman[64744]: 2024-05-09 02:07:36.011603153 +0000 UTC m=+0.440576784 container init de12d771cb4367d534f6990c4294529011a99daa99ea3e84c05666456b75de69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-09T02:07:36.303 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:07:36 smithi059 podman[64744]: 2024-05-09 02:07:36.014660294 +0000 UTC m=+0.443633927 container start de12d771cb4367d534f6990c4294529011a99daa99ea3e84c05666456b75de69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=) 2024-05-09T02:07:36.303 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:07:36 smithi059 ceph-mds[64763]: -- 172.21.15.59:0/3488274045 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5654bad49a40 con 0x5654bad91180 2024-05-09T02:07:36.303 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:07:36 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-b[64759]: starting mds.b at 2024-05-09T02:07:36.303 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:07:36 smithi059 bash[64744]: de12d771cb4367d534f6990c4294529011a99daa99ea3e84c05666456b75de69 2024-05-09T02:07:36.303 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:07:36 smithi059 systemd[1]: Started Ceph mds.b for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:07:36.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.365+0000 7f81947e8700 1 -- 172.21.15.19:0/4027152534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f818c1030e0 msgr2=0x7f818c111170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:36.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.365+0000 7f81947e8700 1 --2- 172.21.15.19:0/4027152534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f818c1030e0 0x7f818c111170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f818400b0a0 tx=0x7f818400b3b0 comp rx=0 tx=0).stop 2024-05-09T02:07:36.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.365+0000 7f81947e8700 1 -- 172.21.15.19:0/4027152534 shutdown_connections 2024-05-09T02:07:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.365+0000 7f81947e8700 1 --2- 172.21.15.19:0/4027152534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f818c1030e0 0x7f818c111170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.365+0000 7f81947e8700 1 --2- 172.21.15.19:0/4027152534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f818c109300 0x7f818c109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.365+0000 7f81947e8700 1 --2- 172.21.15.19:0/4027152534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f818c102790 0x7f818c102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.365+0000 7f81947e8700 1 -- 172.21.15.19:0/4027152534 >> 172.21.15.19:0/4027152534 conn(0x7f818c0fe520 msgr2=0x7f818c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.365+0000 7f81947e8700 1 -- 172.21.15.19:0/4027152534 shutdown_connections 2024-05-09T02:07:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.365+0000 7f81947e8700 1 -- 172.21.15.19:0/4027152534 wait complete. 2024-05-09T02:07:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.366+0000 7f81947e8700 1 Processor -- start 2024-05-09T02:07:36.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.366+0000 7f81947e8700 1 -- start start 2024-05-09T02:07:36.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.366+0000 7f81947e8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f818c102790 0x7f818c19d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:36.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.366+0000 7f81947e8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f818c1030e0 0x7f818c19d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:36.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.366+0000 7f81947e8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f818c109300 0x7f818c1a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:36.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.366+0000 7f81947e8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1145a0 con 0x7f818c109300 2024-05-09T02:07:36.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.366+0000 7f81947e8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f818c114420 con 0x7f818c102790 2024-05-09T02:07:36.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.366+0000 7f81947e8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f818c114720 con 0x7f818c1030e0 2024-05-09T02:07:36.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.366+0000 7f8192d85700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f818c109300 0x7f818c1a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:36.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.366+0000 7f8192584700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f818c102790 0x7f818c19d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:36.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.366+0000 7f8192d85700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f818c109300 0x7f818c1a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44484/0 (socket says 172.21.15.19:44484) 2024-05-09T02:07:36.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.366+0000 7f8192d85700 1 -- 172.21.15.19:0/1963785712 learned_addr learned my addr 172.21.15.19:0/1963785712 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:07:36.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.366+0000 7f8191d83700 1 --2- 172.21.15.19:0/1963785712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f818c1030e0 0x7f818c19d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:36.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.367+0000 7f8192d85700 1 -- 172.21.15.19:0/1963785712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f818c1030e0 msgr2=0x7f818c19d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:36.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.367+0000 7f8192d85700 1 --2- 172.21.15.19:0/1963785712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f818c1030e0 0x7f818c19d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.367+0000 7f8192d85700 1 -- 172.21.15.19:0/1963785712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f818c102790 msgr2=0x7f818c19d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.367+0000 7f8192d85700 1 --2- 172.21.15.19:0/1963785712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f818c102790 0x7f818c19d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.367+0000 7f8192d85700 1 -- 172.21.15.19:0/1963785712 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818c1a1f20 con 0x7f818c109300 2024-05-09T02:07:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.367+0000 7f8191d83700 1 --2- 172.21.15.19:0/1963785712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f818c1030e0 0x7f818c19d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:07:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.367+0000 7f8192d85700 1 --2- 172.21.15.19:0/1963785712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f818c109300 0x7f818c1a1880 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f818400b830 tx=0x7f8184004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.367+0000 7f81837fe700 1 -- 172.21.15.19:0/1963785712 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184021070 con 0x7f818c109300 2024-05-09T02:07:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.367+0000 7f81837fe700 1 -- 172.21.15.19:0/1963785712 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8184007e30 con 0x7f818c109300 2024-05-09T02:07:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.367+0000 7f81947e8700 1 -- 172.21.15.19:0/1963785712 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818c1a21b0 con 0x7f818c109300 2024-05-09T02:07:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.367+0000 7f81837fe700 1 -- 172.21.15.19:0/1963785712 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81840047d0 con 0x7f818c109300 2024-05-09T02:07:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.367+0000 7f81947e8700 1 -- 172.21.15.19:0/1963785712 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f818c1a9b30 con 0x7f818c109300 2024-05-09T02:07:36.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.368+0000 7f81947e8700 1 -- 172.21.15.19:0/1963785712 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f818c105ab0 con 0x7f818c109300 2024-05-09T02:07:36.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.370+0000 7f81837fe700 1 -- 172.21.15.19:0/1963785712 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8184007980 con 0x7f818c109300 2024-05-09T02:07:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.370+0000 7f81837fe700 1 --2- 172.21.15.19:0/1963785712 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f81780b2880 0x7f81780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.371+0000 7f8192584700 1 --2- 172.21.15.19:0/1963785712 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f81780b2880 0x7f81780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.371+0000 7f81837fe700 1 -- 172.21.15.19:0/1963785712 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f818400f020 con 0x7f818c109300 2024-05-09T02:07:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.371+0000 7f8192584700 1 --2- 172.21.15.19:0/1963785712 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f81780b2880 0x7f81780b4d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f818c06ca70 tx=0x7f818800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.375+0000 7f81837fe700 1 -- 172.21.15.19:0/1963785712 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f818402a020 con 0x7f818c109300 2024-05-09T02:07:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.643+0000 7f81947e8700 1 -- 172.21.15.19:0/1963785712 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f818c19e3b0 con 0x7f818c109300 2024-05-09T02:07:36.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.643+0000 7f81837fe700 1 -- 172.21.15.19:0/1963785712 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10467 (secure 0 0 0) 0x7f81840ab980 con 0x7f818c109300 2024-05-09T02:07:36.645 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:07:36.645 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T02:07:36.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.646+0000 7f81947e8700 1 -- 172.21.15.19:0/1963785712 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f81780b2880 msgr2=0x7f81780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:36.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.646+0000 7f81947e8700 1 --2- 172.21.15.19:0/1963785712 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f81780b2880 0x7f81780b4d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f818c06ca70 tx=0x7f818800b410 comp rx=0 tx=0).stop 2024-05-09T02:07:36.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.646+0000 7f81947e8700 1 -- 172.21.15.19:0/1963785712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f818c109300 msgr2=0x7f818c1a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:36.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.646+0000 7f81947e8700 1 --2- 172.21.15.19:0/1963785712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f818c109300 0x7f818c1a1880 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f818400b830 tx=0x7f8184004ce0 comp rx=0 tx=0).stop 2024-05-09T02:07:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.646+0000 7f81947e8700 1 -- 172.21.15.19:0/1963785712 shutdown_connections 2024-05-09T02:07:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.646+0000 7f81947e8700 1 --2- 172.21.15.19:0/1963785712 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f81780b2880 0x7f81780b4d90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.646+0000 7f81947e8700 1 --2- 172.21.15.19:0/1963785712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f818c109300 0x7f818c1a1880 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.646+0000 7f81947e8700 1 --2- 172.21.15.19:0/1963785712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f818c102790 0x7f818c19d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.646+0000 7f81947e8700 1 --2- 172.21.15.19:0/1963785712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f818c1030e0 0x7f818c19d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.646+0000 7f81947e8700 1 -- 172.21.15.19:0/1963785712 >> 172.21.15.19:0/1963785712 conn(0x7f818c0fe520 msgr2=0x7f818c10faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.646+0000 7f81947e8700 1 -- 172.21.15.19:0/1963785712 shutdown_connections 2024-05-09T02:07:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:36.646+0000 7f81947e8700 1 -- 172.21.15.19:0/1963785712 wait complete. 2024-05-09T02:07:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 13 2024-05-09T02:07:37.289 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:37 smithi019 ceph-mon[30151]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:37.289 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:37 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] up:boot 2024-05-09T02:07:37.289 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:37 smithi019 ceph-mon[30151]: fsmap 12 up:standby 2024-05-09T02:07:37.289 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T02:07:37.289 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:37.289 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:37.289 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:37.290 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:37.290 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:37.290 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:37 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:07:37.290 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:37 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1963785712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:37.319 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-09T02:07:37.418 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:37 smithi059 ceph-mon[34781]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:37.418 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:37 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] up:boot 2024-05-09T02:07:37.418 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:37 smithi059 ceph-mon[34781]: fsmap 12 up:standby 2024-05-09T02:07:37.418 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:37 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T02:07:37.419 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:37 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:37.419 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:37 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:37.419 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:37 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:37.419 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:37 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:37.419 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:37 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:37.419 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:37 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:07:37.419 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:37 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1963785712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:37.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:37 smithi196 ceph-mon[34718]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:37.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:37 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] up:boot 2024-05-09T02:07:37.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:37 smithi196 ceph-mon[34718]: fsmap 12 up:standby 2024-05-09T02:07:37.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:37 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T02:07:37.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:37 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:37.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:37 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:37.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:37 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:37.487 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:37 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:37.488 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:37 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:37.488 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:37 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:07:37.488 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:37 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1963785712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:07:38.272 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:07:39.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:39 smithi019 ceph-mon[30151]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:39.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:39.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:39.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:39.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:39.432 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:39 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:39 smithi059 ceph-mon[34781]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:39 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:39.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:39 smithi196 ceph-mon[34718]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:39.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:39.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:39.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:39.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:39.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:39 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:40.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.023+0000 7f669b733700 1 -- 172.21.15.19:0/3040398705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6694108740 msgr2=0x7f66941100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:40.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.023+0000 7f669b733700 1 --2- 172.21.15.19:0/3040398705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6694108740 0x7f66941100a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f668c00b0d0 tx=0x7f668c00b3e0 comp rx=0 tx=0).stop 2024-05-09T02:07:40.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.023+0000 7f669b733700 1 -- 172.21.15.19:0/3040398705 shutdown_connections 2024-05-09T02:07:40.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.023+0000 7f669b733700 1 --2- 172.21.15.19:0/3040398705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6694108740 0x7f66941100a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:40.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.023+0000 7f669b733700 1 --2- 172.21.15.19:0/3040398705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66941034c0 0x7f66941038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:40.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.023+0000 7f669b733700 1 --2- 172.21.15.19:0/3040398705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6694107df0 0x7f6694108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:40.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.023+0000 7f669b733700 1 -- 172.21.15.19:0/3040398705 >> 172.21.15.19:0/3040398705 conn(0x7f66940fedc0 msgr2=0x7f66941011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:40.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.023+0000 7f669b733700 1 -- 172.21.15.19:0/3040398705 shutdown_connections 2024-05-09T02:07:40.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.023+0000 7f669b733700 1 -- 172.21.15.19:0/3040398705 wait complete. 2024-05-09T02:07:40.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.024+0000 7f669b733700 1 Processor -- start 2024-05-09T02:07:40.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.024+0000 7f669b733700 1 -- start start 2024-05-09T02:07:40.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.024+0000 7f669b733700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66941034c0 0x7f669419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:40.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.024+0000 7f669b733700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6694107df0 0x7f669419c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:40.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.024+0000 7f669b733700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6694108740 0x7f669419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:40.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.024+0000 7f669b733700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6694113370 con 0x7f6694107df0 2024-05-09T02:07:40.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.024+0000 7f669b733700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f66941131f0 con 0x7f6694108740 2024-05-09T02:07:40.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.024+0000 7f669b733700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66941134f0 con 0x7f66941034c0 2024-05-09T02:07:40.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.024+0000 7f6698cce700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6694107df0 0x7f669419c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:40.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.024+0000 7f6698cce700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6694107df0 0x7f669419c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55308/0 (socket says 172.21.15.19:55308) 2024-05-09T02:07:40.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.024+0000 7f6698cce700 1 -- 172.21.15.19:0/1677674874 learned_addr learned my addr 172.21.15.19:0/1677674874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:07:40.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.024+0000 7f66994cf700 1 --2- 172.21.15.19:0/1677674874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66941034c0 0x7f669419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:40.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.025+0000 7f6699cd0700 1 --2- 172.21.15.19:0/1677674874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6694108740 0x7f669419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:40.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.025+0000 7f6698cce700 1 -- 172.21.15.19:0/1677674874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66941034c0 msgr2=0x7f669419bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:40.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.025+0000 7f6698cce700 1 --2- 172.21.15.19:0/1677674874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66941034c0 0x7f669419bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:40.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.025+0000 7f6698cce700 1 -- 172.21.15.19:0/1677674874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6694108740 msgr2=0x7f669419c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:40.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.025+0000 7f6698cce700 1 --2- 172.21.15.19:0/1677674874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6694108740 0x7f669419c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:40.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.025+0000 7f6698cce700 1 -- 172.21.15.19:0/1677674874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66941a6f00 con 0x7f6694107df0 2024-05-09T02:07:40.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.025+0000 7f6698cce700 1 --2- 172.21.15.19:0/1677674874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6694107df0 0x7f669419c0d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f668400e970 tx=0x7f668400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:40.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.025+0000 7f668a7fc700 1 -- 172.21.15.19:0/1677674874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684004a30 con 0x7f6694107df0 2024-05-09T02:07:40.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.025+0000 7f669b733700 1 -- 172.21.15.19:0/1677674874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66941a71f0 con 0x7f6694107df0 2024-05-09T02:07:40.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.025+0000 7f668a7fc700 1 -- 172.21.15.19:0/1677674874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f668400cc90 con 0x7f6694107df0 2024-05-09T02:07:40.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.025+0000 7f668a7fc700 1 -- 172.21.15.19:0/1677674874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684005610 con 0x7f6694107df0 2024-05-09T02:07:40.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.025+0000 7f669b733700 1 -- 172.21.15.19:0/1677674874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66941a7780 con 0x7f6694107df0 2024-05-09T02:07:40.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.026+0000 7f669b733700 1 -- 172.21.15.19:0/1677674874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66940512c0 con 0x7f6694107df0 2024-05-09T02:07:40.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.028+0000 7f668a7fc700 1 -- 172.21.15.19:0/1677674874 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f668400c7e0 con 0x7f6694107df0 2024-05-09T02:07:40.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.029+0000 7f668a7fc700 1 --2- 172.21.15.19:0/1677674874 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f66800b2880 0x7f66800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:40.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.029+0000 7f66994cf700 1 --2- 172.21.15.19:0/1677674874 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f66800b2880 0x7f66800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:40.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.029+0000 7f668a7fc700 1 -- 172.21.15.19:0/1677674874 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f66840e2670 con 0x7f6694107df0 2024-05-09T02:07:40.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.030+0000 7f66994cf700 1 --2- 172.21.15.19:0/1677674874 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f66800b2880 0x7f66800b4d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6690009c60 tx=0x7f6690009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:40.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.033+0000 7f668a7fc700 1 -- 172.21.15.19:0/1677674874 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f66840aa830 con 0x7f6694107df0 2024-05-09T02:07:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.258+0000 7f669b733700 1 -- 172.21.15.19:0/1677674874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f669402e5f0 con 0x7f6694107df0 2024-05-09T02:07:40.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.259+0000 7f668a7fc700 1 -- 172.21.15.19:0/1677674874 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f66840aa130 con 0x7f6694107df0 2024-05-09T02:07:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:07:40.260 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-05-09T02:07:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:07:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-05-09T02:07:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-05-09T02:07:40.260 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:07:40.261 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-05-09T02:07:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.261+0000 7f669b733700 1 -- 172.21.15.19:0/1677674874 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f66800b2880 msgr2=0x7f66800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.261+0000 7f669b733700 1 --2- 172.21.15.19:0/1677674874 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f66800b2880 0x7f66800b4d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6690009c60 tx=0x7f6690009260 comp rx=0 tx=0).stop 2024-05-09T02:07:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.261+0000 7f669b733700 1 -- 172.21.15.19:0/1677674874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6694107df0 msgr2=0x7f669419c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.261+0000 7f669b733700 1 --2- 172.21.15.19:0/1677674874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6694107df0 0x7f669419c0d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f668400e970 tx=0x7f668400ec80 comp rx=0 tx=0).stop 2024-05-09T02:07:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.261+0000 7f669b733700 1 -- 172.21.15.19:0/1677674874 shutdown_connections 2024-05-09T02:07:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.261+0000 7f669b733700 1 --2- 172.21.15.19:0/1677674874 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f66800b2880 0x7f66800b4d90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.261+0000 7f669b733700 1 --2- 172.21.15.19:0/1677674874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6694108740 0x7f669419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.261+0000 7f669b733700 1 --2- 172.21.15.19:0/1677674874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66941034c0 0x7f669419bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.261+0000 7f669b733700 1 --2- 172.21.15.19:0/1677674874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6694107df0 0x7f669419c0d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.261+0000 7f669b733700 1 -- 172.21.15.19:0/1677674874 >> 172.21.15.19:0/1677674874 conn(0x7f66940fedc0 msgr2=0x7f6694107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.261+0000 7f669b733700 1 -- 172.21.15.19:0/1677674874 shutdown_connections 2024-05-09T02:07:40.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:40.261+0000 7f669b733700 1 -- 172.21.15.19:0/1677674874 wait complete. 2024-05-09T02:07:41.092 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T02:07:41.092 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T02:07:41.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:41 smithi019 ceph-mon[30151]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:41.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:41 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:41.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:41 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:41.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:41 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1677674874' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T02:07:41.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:41 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:41.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:41 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:07:41.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:41 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:41.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:41 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:07:41.352 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:07:41.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:41 smithi059 ceph-mon[34781]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:41 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:41 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:41 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1677674874' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T02:07:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:41 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:41 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:07:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:41 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:41 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:07:41.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:41 smithi196 ceph-mon[34718]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:41.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:41 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:41.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:41 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:41.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:41 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1677674874' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T02:07:41.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:41 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:41.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:41 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:07:41.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:41 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:41.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:41 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:07:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.046+0000 7fc3d86dc700 1 -- 172.21.15.19:0/2643902046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d0107df0 msgr2=0x7fc3d0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.046+0000 7fc3d86dc700 1 --2- 172.21.15.19:0/2643902046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d0107df0 0x7fc3d0108200 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00099c0 tx=0x7fc3c0009cd0 comp rx=0 tx=0).stop 2024-05-09T02:07:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.046+0000 7fc3d86dc700 1 -- 172.21.15.19:0/2643902046 shutdown_connections 2024-05-09T02:07:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.046+0000 7fc3d86dc700 1 --2- 172.21.15.19:0/2643902046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3d0108740 0x7fc3d01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.046+0000 7fc3d86dc700 1 --2- 172.21.15.19:0/2643902046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3d01034c0 0x7fc3d01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.046+0000 7fc3d86dc700 1 --2- 172.21.15.19:0/2643902046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d0107df0 0x7fc3d0108200 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.046+0000 7fc3d86dc700 1 -- 172.21.15.19:0/2643902046 >> 172.21.15.19:0/2643902046 conn(0x7fc3d00fedc0 msgr2=0x7fc3d01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.046+0000 7fc3d86dc700 1 -- 172.21.15.19:0/2643902046 shutdown_connections 2024-05-09T02:07:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.046+0000 7fc3d86dc700 1 -- 172.21.15.19:0/2643902046 wait complete. 2024-05-09T02:07:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.047+0000 7fc3d86dc700 1 Processor -- start 2024-05-09T02:07:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.047+0000 7fc3d86dc700 1 -- start start 2024-05-09T02:07:43.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.047+0000 7fc3d86dc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3d01034c0 0x7fc3d019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:43.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.047+0000 7fc3d86dc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d0107df0 0x7fc3d019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:43.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.047+0000 7fc3d86dc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3d0108740 0x7fc3d019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.047+0000 7fc3d86dc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d0113350 con 0x7fc3d0107df0 2024-05-09T02:07:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.047+0000 7fc3d86dc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d01131d0 con 0x7fc3d0108740 2024-05-09T02:07:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.047+0000 7fc3d86dc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d01134d0 con 0x7fc3d01034c0 2024-05-09T02:07:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.047+0000 7fc3d5c77700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d0107df0 0x7fc3d019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.048+0000 7fc3d5c77700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d0107df0 0x7fc3d019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55326/0 (socket says 172.21.15.19:55326) 2024-05-09T02:07:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.048+0000 7fc3d5c77700 1 -- 172.21.15.19:0/2117113363 learned_addr learned my addr 172.21.15.19:0/2117113363 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:07:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.048+0000 7fc3d6c79700 1 --2- 172.21.15.19:0/2117113363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3d0108740 0x7fc3d019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.048+0000 7fc3d6478700 1 --2- 172.21.15.19:0/2117113363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3d01034c0 0x7fc3d019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.048+0000 7fc3d5c77700 1 -- 172.21.15.19:0/2117113363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3d01034c0 msgr2=0x7fc3d019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.048+0000 7fc3d5c77700 1 --2- 172.21.15.19:0/2117113363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3d01034c0 0x7fc3d019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.048+0000 7fc3d5c77700 1 -- 172.21.15.19:0/2117113363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3d0108740 msgr2=0x7fc3d019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.048+0000 7fc3d5c77700 1 --2- 172.21.15.19:0/2117113363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3d0108740 0x7fc3d019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.048+0000 7fc3d5c77700 1 -- 172.21.15.19:0/2117113363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d019fce0 con 0x7fc3d0107df0 2024-05-09T02:07:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.048+0000 7fc3d5c77700 1 --2- 172.21.15.19:0/2117113363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d0107df0 0x7fc3d019b3b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0006010 tx=0x7fc3c00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:43.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.048+0000 7fc3c77fe700 1 -- 172.21.15.19:0/2117113363 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0027070 con 0x7fc3d0107df0 2024-05-09T02:07:43.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.048+0000 7fc3c77fe700 1 -- 172.21.15.19:0/2117113363 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3c0003f20 con 0x7fc3d0107df0 2024-05-09T02:07:43.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.048+0000 7fc3d86dc700 1 -- 172.21.15.19:0/2117113363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d019ff70 con 0x7fc3d0107df0 2024-05-09T02:07:43.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.048+0000 7fc3c77fe700 1 -- 172.21.15.19:0/2117113363 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0022c60 con 0x7fc3d0107df0 2024-05-09T02:07:43.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.049+0000 7fc3d86dc700 1 -- 172.21.15.19:0/2117113363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d01a78f0 con 0x7fc3d0107df0 2024-05-09T02:07:43.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.050+0000 7fc3d86dc700 1 -- 172.21.15.19:0/2117113363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d00512c0 con 0x7fc3d0107df0 2024-05-09T02:07:43.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.051+0000 7fc3c77fe700 1 -- 172.21.15.19:0/2117113363 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc3c002a070 con 0x7fc3d0107df0 2024-05-09T02:07:43.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.052+0000 7fc3c77fe700 1 --2- 172.21.15.19:0/2117113363 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc3bc0b2900 0x7fc3bc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:43.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.052+0000 7fc3c77fe700 1 -- 172.21.15.19:0/2117113363 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fc3c0022e00 con 0x7fc3d0107df0 2024-05-09T02:07:43.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.052+0000 7fc3d6478700 1 --2- 172.21.15.19:0/2117113363 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc3bc0b2900 0x7fc3bc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:43.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.053+0000 7fc3d6478700 1 --2- 172.21.15.19:0/2117113363 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc3bc0b2900 0x7fc3bc0b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc005950 tx=0x7fc3cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:43.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.056+0000 7fc3c77fe700 1 -- 172.21.15.19:0/2117113363 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc3c0031080 con 0x7fc3d0107df0 2024-05-09T02:07:43.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:43.279+0000 7fc3d86dc700 1 -- 172.21.15.19:0/2117113363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc3d002e5f0 con 0x7fc3d0107df0 2024-05-09T02:07:43.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:43 smithi019 ceph-mon[30151]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:43.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:43.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:43.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:43.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:43 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:43 smithi059 ceph-mon[34781]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:43.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:43 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:43.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:43 smithi196 ceph-mon[34718]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:43.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:43.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:43.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:43.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:43 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:44.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:44.137+0000 7fc3c77fe700 1 -- 172.21.15.19:0/2117113363 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7fc3c0013070 con 0x7fc3d0107df0 2024-05-09T02:07:44.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:44.140+0000 7fc3d86dc700 1 -- 172.21.15.19:0/2117113363 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc3bc0b2900 msgr2=0x7fc3bc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:44.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:44.140+0000 7fc3d86dc700 1 --2- 172.21.15.19:0/2117113363 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc3bc0b2900 0x7fc3bc0b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc005950 tx=0x7fc3cc00b410 comp rx=0 tx=0).stop 2024-05-09T02:07:44.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:44.140+0000 7fc3d86dc700 1 -- 172.21.15.19:0/2117113363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d0107df0 msgr2=0x7fc3d019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:44.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:44.140+0000 7fc3d86dc700 1 --2- 172.21.15.19:0/2117113363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d0107df0 0x7fc3d019b3b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0006010 tx=0x7fc3c00047b0 comp rx=0 tx=0).stop 2024-05-09T02:07:44.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:44.140+0000 7fc3d86dc700 1 -- 172.21.15.19:0/2117113363 shutdown_connections 2024-05-09T02:07:44.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:44.140+0000 7fc3d86dc700 1 --2- 172.21.15.19:0/2117113363 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc3bc0b2900 0x7fc3bc0b4e10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:44.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:44.140+0000 7fc3d86dc700 1 --2- 172.21.15.19:0/2117113363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3d0108740 0x7fc3d019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:44.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:44.140+0000 7fc3d86dc700 1 --2- 172.21.15.19:0/2117113363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3d01034c0 0x7fc3d019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:44.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:44.140+0000 7fc3d86dc700 1 --2- 172.21.15.19:0/2117113363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d0107df0 0x7fc3d019b3b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:44.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:44.140+0000 7fc3d86dc700 1 -- 172.21.15.19:0/2117113363 >> 172.21.15.19:0/2117113363 conn(0x7fc3d00fedc0 msgr2=0x7fc3d006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:44.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:44.140+0000 7fc3d86dc700 1 -- 172.21.15.19:0/2117113363 shutdown_connections 2024-05-09T02:07:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:44.141+0000 7fc3d86dc700 1 -- 172.21.15.19:0/2117113363 wait complete. 2024-05-09T02:07:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-05-09T02:07:44.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:44 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2117113363' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T02:07:44.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:44 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2117113363' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T02:07:44.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:44 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2117113363' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T02:07:44.947 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T02:07:45.214 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:07:45.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:45 smithi019 ceph-mon[30151]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:45.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:45 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2117113363' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T02:07:45.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:45 smithi019 ceph-mon[30151]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T02:07:45.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:45 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:45.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:45 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:45.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:45 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:45.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:45 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:07:45.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:45 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:45.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:45 smithi059 ceph-mon[34781]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:45.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:45 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2117113363' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T02:07:45.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:45 smithi059 ceph-mon[34781]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T02:07:45.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:45 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:45.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:45 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:45.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:45 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:45.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:45 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:07:45.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:45 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:45.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:45 smithi196 ceph-mon[34718]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:45.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:45 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2117113363' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T02:07:45.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:45 smithi196 ceph-mon[34718]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T02:07:45.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:45 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:45.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:45 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:45.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:45 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:07:45.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:45 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:07:45.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:45 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:07:46.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:46 smithi019 ceph-mon[30151]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T02:07:46.465 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:46 smithi059 ceph-mon[34781]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T02:07:46.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:46 smithi196 ceph-mon[34718]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T02:07:46.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.878+0000 7f8d49ecf700 1 -- 172.21.15.19:0/2828580830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d44107e00 msgr2=0x7f8d44108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:46.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.878+0000 7f8d49ecf700 1 --2- 172.21.15.19:0/2828580830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d44107e00 0x7f8d44108210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8d380099c0 tx=0x7f8d38009cd0 comp rx=0 tx=0).stop 2024-05-09T02:07:46.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.878+0000 7f8d49ecf700 1 -- 172.21.15.19:0/2828580830 shutdown_connections 2024-05-09T02:07:46.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.878+0000 7f8d49ecf700 1 --2- 172.21.15.19:0/2828580830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d44108750 0x7f8d441100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:46.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.878+0000 7f8d49ecf700 1 --2- 172.21.15.19:0/2828580830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d44103520 0x7f8d44103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:46.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.878+0000 7f8d49ecf700 1 --2- 172.21.15.19:0/2828580830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d44107e00 0x7f8d44108210 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:46.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.878+0000 7f8d49ecf700 1 -- 172.21.15.19:0/2828580830 >> 172.21.15.19:0/2828580830 conn(0x7f8d440fef90 msgr2=0x7f8d441013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:46.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.878+0000 7f8d49ecf700 1 -- 172.21.15.19:0/2828580830 shutdown_connections 2024-05-09T02:07:46.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.878+0000 7f8d49ecf700 1 -- 172.21.15.19:0/2828580830 wait complete. 2024-05-09T02:07:46.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.879+0000 7f8d49ecf700 1 Processor -- start 2024-05-09T02:07:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.879+0000 7f8d49ecf700 1 -- start start 2024-05-09T02:07:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.879+0000 7f8d49ecf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d44103520 0x7f8d4419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.879+0000 7f8d49ecf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d44107e00 0x7f8d4419c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.879+0000 7f8d49ecf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d44108750 0x7f8d4419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.879+0000 7f8d49ecf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441133d0 con 0x7f8d44108750 2024-05-09T02:07:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.879+0000 7f8d49ecf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8d44113250 con 0x7f8d44107e00 2024-05-09T02:07:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.879+0000 7f8d49ecf700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8d44113550 con 0x7f8d44103520 2024-05-09T02:07:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.879+0000 7f8d43fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d44108750 0x7f8d4419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.879+0000 7f8d43fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d44108750 0x7f8d4419c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56198/0 (socket says 172.21.15.19:56198) 2024-05-09T02:07:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.879+0000 7f8d43fff700 1 -- 172.21.15.19:0/1923359078 learned_addr learned my addr 172.21.15.19:0/1923359078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:07:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.879+0000 7f8d42ffd700 1 --2- 172.21.15.19:0/1923359078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d44107e00 0x7f8d4419c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.880+0000 7f8d437fe700 1 --2- 172.21.15.19:0/1923359078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d44103520 0x7f8d4419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.880+0000 7f8d43fff700 1 -- 172.21.15.19:0/1923359078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d44103520 msgr2=0x7f8d4419bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.880+0000 7f8d43fff700 1 --2- 172.21.15.19:0/1923359078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d44103520 0x7f8d4419bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.880+0000 7f8d43fff700 1 -- 172.21.15.19:0/1923359078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d44107e00 msgr2=0x7f8d4419c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.880+0000 7f8d43fff700 1 --2- 172.21.15.19:0/1923359078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d44107e00 0x7f8d4419c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.880+0000 7f8d43fff700 1 -- 172.21.15.19:0/1923359078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d441a7080 con 0x7f8d44108750 2024-05-09T02:07:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.880+0000 7f8d43fff700 1 --2- 172.21.15.19:0/1923359078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d44108750 0x7f8d4419c7d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00b970 tx=0x7f8d3c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.880+0000 7f8d40ff9700 1 -- 172.21.15.19:0/1923359078 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c00d860 con 0x7f8d44108750 2024-05-09T02:07:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.880+0000 7f8d49ecf700 1 -- 172.21.15.19:0/1923359078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d441a7370 con 0x7f8d44108750 2024-05-09T02:07:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.880+0000 7f8d40ff9700 1 -- 172.21.15.19:0/1923359078 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d3c00e430 con 0x7f8d44108750 2024-05-09T02:07:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.880+0000 7f8d40ff9700 1 -- 172.21.15.19:0/1923359078 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c017660 con 0x7f8d44108750 2024-05-09T02:07:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.880+0000 7f8d49ecf700 1 -- 172.21.15.19:0/1923359078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d441a7880 con 0x7f8d44108750 2024-05-09T02:07:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.881+0000 7f8d49ecf700 1 -- 172.21.15.19:0/1923359078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d440512c0 con 0x7f8d44108750 2024-05-09T02:07:46.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.883+0000 7f8d40ff9700 1 -- 172.21.15.19:0/1923359078 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8d3c00da00 con 0x7f8d44108750 2024-05-09T02:07:46.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.884+0000 7f8d40ff9700 1 --2- 172.21.15.19:0/1923359078 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f8d2c0b2880 0x7f8d2c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:46.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.884+0000 7f8d437fe700 1 --2- 172.21.15.19:0/1923359078 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f8d2c0b2880 0x7f8d2c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:46.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.884+0000 7f8d40ff9700 1 -- 172.21.15.19:0/1923359078 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7497+0+0 (secure 0 0 0) 0x7f8d3c0e1ca0 con 0x7f8d44108750 2024-05-09T02:07:46.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.884+0000 7f8d437fe700 1 --2- 172.21.15.19:0/1923359078 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f8d2c0b2880 0x7f8d2c0b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8d34007be0 tx=0x7f8d340073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:46.888+0000 7f8d40ff9700 1 -- 172.21.15.19:0/1923359078 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8d3c053df0 con 0x7f8d44108750 2024-05-09T02:07:47.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:47.105+0000 7f8d49ecf700 1 -- 172.21.15.19:0/1923359078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8d4406c630 con 0x7f8d44108750 2024-05-09T02:07:47.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:47.160+0000 7f8d40ff9700 1 -- 172.21.15.19:0/1923359078 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) ==== 170+0+0 (secure 0 0 0) 0x7f8d3c013030 con 0x7f8d44108750 2024-05-09T02:07:47.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:47.164+0000 7f8d49ecf700 1 -- 172.21.15.19:0/1923359078 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f8d2c0b2880 msgr2=0x7f8d2c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:47.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:47.164+0000 7f8d49ecf700 1 --2- 172.21.15.19:0/1923359078 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f8d2c0b2880 0x7f8d2c0b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8d34007be0 tx=0x7f8d340073d0 comp rx=0 tx=0).stop 2024-05-09T02:07:47.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:47.164+0000 7f8d49ecf700 1 -- 172.21.15.19:0/1923359078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d44108750 msgr2=0x7f8d4419c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:47.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:47.164+0000 7f8d49ecf700 1 --2- 172.21.15.19:0/1923359078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d44108750 0x7f8d4419c7d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00b970 tx=0x7f8d3c00bc80 comp rx=0 tx=0).stop 2024-05-09T02:07:47.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:47.164+0000 7f8d49ecf700 1 -- 172.21.15.19:0/1923359078 shutdown_connections 2024-05-09T02:07:47.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:47.164+0000 7f8d49ecf700 1 --2- 172.21.15.19:0/1923359078 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f8d2c0b2880 0x7f8d2c0b4d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:47.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:47.164+0000 7f8d49ecf700 1 --2- 172.21.15.19:0/1923359078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d44108750 0x7f8d4419c7d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:47.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:47.164+0000 7f8d49ecf700 1 --2- 172.21.15.19:0/1923359078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d44103520 0x7f8d4419bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:47.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:47.164+0000 7f8d49ecf700 1 --2- 172.21.15.19:0/1923359078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d44107e00 0x7f8d4419c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:47.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:47.164+0000 7f8d49ecf700 1 -- 172.21.15.19:0/1923359078 >> 172.21.15.19:0/1923359078 conn(0x7f8d440fef90 msgr2=0x7f8d44107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:47.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:47.164+0000 7f8d49ecf700 1 -- 172.21.15.19:0/1923359078 shutdown_connections 2024-05-09T02:07:47.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:47.165+0000 7f8d49ecf700 1 -- 172.21.15.19:0/1923359078 wait complete. 2024-05-09T02:07:47.168 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-05-09T02:07:47.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:47 smithi019 ceph-mon[30151]: pgmap v285: 33 pgs: 2 creating+activating, 6 creating+peering, 15 unknown, 10 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:47.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:47 smithi019 ceph-mon[30151]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T02:07:47.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:47 smithi019 ceph-mon[30151]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T02:07:47.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:47 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1923359078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T02:07:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:47 smithi059 ceph-mon[34781]: pgmap v285: 33 pgs: 2 creating+activating, 6 creating+peering, 15 unknown, 10 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:47 smithi059 ceph-mon[34781]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T02:07:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:47 smithi059 ceph-mon[34781]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T02:07:47.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:47 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1923359078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T02:07:47.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:47 smithi196 ceph-mon[34718]: pgmap v285: 33 pgs: 2 creating+activating, 6 creating+peering, 15 unknown, 10 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:47.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:47 smithi196 ceph-mon[34718]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T02:07:47.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:47 smithi196 ceph-mon[34718]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T02:07:47.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:47 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1923359078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T02:07:47.973 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T02:07:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:07:48.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:48 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1923359078' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T02:07:48.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:48 smithi019 ceph-mon[30151]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T02:07:48.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:48 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1923359078' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T02:07:48.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:48 smithi059 ceph-mon[34781]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T02:07:48.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:48 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1923359078' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T02:07:48.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:48 smithi196 ceph-mon[34718]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T02:07:49.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:49 smithi059 ceph-mon[34781]: pgmap v288: 97 pgs: 2 creating+activating, 14 creating+peering, 62 unknown, 19 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:49.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:49 smithi059 ceph-mon[34781]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T02:07:49.496 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:49 smithi019 ceph-mon[30151]: pgmap v288: 97 pgs: 2 creating+activating, 14 creating+peering, 62 unknown, 19 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:49.497 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:49 smithi019 ceph-mon[30151]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T02:07:49.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:49 smithi196 ceph-mon[34718]: pgmap v288: 97 pgs: 2 creating+activating, 14 creating+peering, 62 unknown, 19 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:49.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:49 smithi196 ceph-mon[34718]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T02:07:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.044+0000 7fef5fc71700 1 -- 172.21.15.19:0/4283317936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef58108740 msgr2=0x7fef581100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.044+0000 7fef5fc71700 1 --2- 172.21.15.19:0/4283317936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef58108740 0x7fef581100a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fef5000b0d0 tx=0x7fef5000b3e0 comp rx=0 tx=0).stop 2024-05-09T02:07:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.044+0000 7fef5fc71700 1 -- 172.21.15.19:0/4283317936 shutdown_connections 2024-05-09T02:07:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.044+0000 7fef5fc71700 1 --2- 172.21.15.19:0/4283317936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef58108740 0x7fef581100a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.044+0000 7fef5fc71700 1 --2- 172.21.15.19:0/4283317936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef581034c0 0x7fef581038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.044+0000 7fef5fc71700 1 --2- 172.21.15.19:0/4283317936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef58107df0 0x7fef58108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.044+0000 7fef5fc71700 1 -- 172.21.15.19:0/4283317936 >> 172.21.15.19:0/4283317936 conn(0x7fef580fedf0 msgr2=0x7fef58101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.044+0000 7fef5fc71700 1 -- 172.21.15.19:0/4283317936 shutdown_connections 2024-05-09T02:07:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.044+0000 7fef5fc71700 1 -- 172.21.15.19:0/4283317936 wait complete. 2024-05-09T02:07:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.045+0000 7fef5fc71700 1 Processor -- start 2024-05-09T02:07:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.045+0000 7fef5fc71700 1 -- start start 2024-05-09T02:07:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.045+0000 7fef5fc71700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef581034c0 0x7fef5819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.045+0000 7fef5fc71700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef58107df0 0x7fef5819c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.045+0000 7fef5fc71700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef58108740 0x7fef5819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.045+0000 7fef5fc71700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fef58113370 con 0x7fef58108740 2024-05-09T02:07:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.045+0000 7fef5fc71700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fef581131f0 con 0x7fef581034c0 2024-05-09T02:07:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.045+0000 7fef5fc71700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef581134f0 con 0x7fef58107df0 2024-05-09T02:07:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.045+0000 7fef5e20e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef58108740 0x7fef5819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.045+0000 7fef5e20e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef58108740 0x7fef5819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56218/0 (socket says 172.21.15.19:56218) 2024-05-09T02:07:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.045+0000 7fef5e20e700 1 -- 172.21.15.19:0/11509562 learned_addr learned my addr 172.21.15.19:0/11509562 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:07:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.045+0000 7fef5d20c700 1 --2- 172.21.15.19:0/11509562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef58107df0 0x7fef5819c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.045+0000 7fef5da0d700 1 --2- 172.21.15.19:0/11509562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef581034c0 0x7fef5819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.046+0000 7fef5e20e700 1 -- 172.21.15.19:0/11509562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef58107df0 msgr2=0x7fef5819c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.046+0000 7fef5e20e700 1 --2- 172.21.15.19:0/11509562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef58107df0 0x7fef5819c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.046+0000 7fef5e20e700 1 -- 172.21.15.19:0/11509562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef581034c0 msgr2=0x7fef5819bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.046+0000 7fef5e20e700 1 --2- 172.21.15.19:0/11509562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef581034c0 0x7fef5819bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.046+0000 7fef5e20e700 1 -- 172.21.15.19:0/11509562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef581a6fa0 con 0x7fef58108740 2024-05-09T02:07:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.046+0000 7fef5e20e700 1 --2- 172.21.15.19:0/11509562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef58108740 0x7fef5819c6f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fef500095c0 tx=0x7fef50007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.046+0000 7fef56ffd700 1 -- 172.21.15.19:0/11509562 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5001f070 con 0x7fef58108740 2024-05-09T02:07:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.046+0000 7fef5fc71700 1 -- 172.21.15.19:0/11509562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef581a7230 con 0x7fef58108740 2024-05-09T02:07:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.046+0000 7fef56ffd700 1 -- 172.21.15.19:0/11509562 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef50007e20 con 0x7fef58108740 2024-05-09T02:07:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.046+0000 7fef56ffd700 1 -- 172.21.15.19:0/11509562 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5001b4a0 con 0x7fef58108740 2024-05-09T02:07:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.046+0000 7fef5fc71700 1 -- 172.21.15.19:0/11509562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef581a76e0 con 0x7fef58108740 2024-05-09T02:07:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.047+0000 7fef5fc71700 1 -- 172.21.15.19:0/11509562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef580512c0 con 0x7fef58108740 2024-05-09T02:07:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.049+0000 7fef56ffd700 1 -- 172.21.15.19:0/11509562 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fef50022070 con 0x7fef58108740 2024-05-09T02:07:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.050+0000 7fef56ffd700 1 --2- 172.21.15.19:0/11509562 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fef400b2880 0x7fef400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.050+0000 7fef5da0d700 1 --2- 172.21.15.19:0/11509562 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fef400b2880 0x7fef400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.050+0000 7fef56ffd700 1 -- 172.21.15.19:0/11509562 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7fef5000f020 con 0x7fef58108740 2024-05-09T02:07:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.050+0000 7fef5da0d700 1 --2- 172.21.15.19:0/11509562 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fef400b2880 0x7fef400b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fef48005950 tx=0x7fef4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:50.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.053+0000 7fef56ffd700 1 -- 172.21.15.19:0/11509562 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fef50029020 con 0x7fef58108740 2024-05-09T02:07:50.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:50.314+0000 7fef5fc71700 1 -- 172.21.15.19:0/11509562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fef5802e5f0 con 0x7fef58108740 2024-05-09T02:07:50.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:50 smithi059 ceph-mon[34781]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T02:07:50.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:50 smithi196 ceph-mon[34718]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T02:07:50.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:50 smithi019 ceph-mon[30151]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T02:07:51.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:51.191+0000 7fef56ffd700 1 -- 172.21.15.19:0/11509562 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fef500ab560 con 0x7fef58108740 2024-05-09T02:07:51.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:51.194+0000 7fef5fc71700 1 -- 172.21.15.19:0/11509562 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fef400b2880 msgr2=0x7fef400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:51.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:51.194+0000 7fef5fc71700 1 --2- 172.21.15.19:0/11509562 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fef400b2880 0x7fef400b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fef48005950 tx=0x7fef4800b410 comp rx=0 tx=0).stop 2024-05-09T02:07:51.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:51.194+0000 7fef5fc71700 1 -- 172.21.15.19:0/11509562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef58108740 msgr2=0x7fef5819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:51.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:51.194+0000 7fef5fc71700 1 --2- 172.21.15.19:0/11509562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef58108740 0x7fef5819c6f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fef500095c0 tx=0x7fef50007a20 comp rx=0 tx=0).stop 2024-05-09T02:07:51.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:51.194+0000 7fef5fc71700 1 -- 172.21.15.19:0/11509562 shutdown_connections 2024-05-09T02:07:51.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:51.194+0000 7fef5fc71700 1 --2- 172.21.15.19:0/11509562 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fef400b2880 0x7fef400b4d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:51.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:51.194+0000 7fef5fc71700 1 --2- 172.21.15.19:0/11509562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef58108740 0x7fef5819c6f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:51.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:51.194+0000 7fef5fc71700 1 --2- 172.21.15.19:0/11509562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef581034c0 0x7fef5819bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:51.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:51.194+0000 7fef5fc71700 1 --2- 172.21.15.19:0/11509562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef58107df0 0x7fef5819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:51.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:51.194+0000 7fef5fc71700 1 -- 172.21.15.19:0/11509562 >> 172.21.15.19:0/11509562 conn(0x7fef580fedf0 msgr2=0x7fef58107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:51.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:51.194+0000 7fef5fc71700 1 -- 172.21.15.19:0/11509562 shutdown_connections 2024-05-09T02:07:51.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:51.194+0000 7fef5fc71700 1 -- 172.21.15.19:0/11509562 wait complete. 2024-05-09T02:07:51.198 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T02:07:51.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:51 smithi059 ceph-mon[34781]: pgmap v291: 97 pgs: 2 creating+activating, 8 creating+peering, 36 unknown, 51 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:51.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:51 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/11509562' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T02:07:51.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:51 smithi059 ceph-mon[34781]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-09T02:07:51.512 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:51 smithi019 ceph-mon[30151]: pgmap v291: 97 pgs: 2 creating+activating, 8 creating+peering, 36 unknown, 51 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:51.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:51 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/11509562' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T02:07:51.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:51 smithi019 ceph-mon[30151]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-09T02:07:51.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:51 smithi196 ceph-mon[34718]: pgmap v291: 97 pgs: 2 creating+activating, 8 creating+peering, 36 unknown, 51 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:07:51.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:51 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/11509562' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T02:07:51.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:51 smithi196 ceph-mon[34718]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-09T02:07:51.925 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T02:07:52.188 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:07:52.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:52 smithi019 ceph-mon[30151]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T02:07:52.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:52 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/11509562' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T02:07:52.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:52 smithi019 ceph-mon[30151]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-09T02:07:52.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:52 smithi019 ceph-mon[30151]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-09T02:07:52.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:52 smithi059 ceph-mon[34781]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T02:07:52.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:52 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/11509562' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T02:07:52.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:52 smithi059 ceph-mon[34781]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-09T02:07:52.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:52 smithi059 ceph-mon[34781]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-09T02:07:52.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:52 smithi196 ceph-mon[34718]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T02:07:52.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:52 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/11509562' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T02:07:52.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:52 smithi196 ceph-mon[34718]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-09T02:07:52.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:52 smithi196 ceph-mon[34718]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-09T02:07:53.464 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:53 smithi019 ceph-mon[30151]: pgmap v293: 97 pgs: 8 creating+peering, 5 unknown, 84 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-09T02:07:53.464 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:53 smithi019 ceph-mon[30151]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T02:07:53.464 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:53 smithi019 ceph-mon[30151]: Cluster is now healthy 2024-05-09T02:07:53.465 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:53 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] up:active 2024-05-09T02:07:53.465 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:53 smithi019 ceph-mon[30151]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T02:07:53.465 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:53 smithi019 ceph-mon[30151]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T02:07:53.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:53 smithi059 ceph-mon[34781]: pgmap v293: 97 pgs: 8 creating+peering, 5 unknown, 84 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-09T02:07:53.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:53 smithi059 ceph-mon[34781]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T02:07:53.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:53 smithi059 ceph-mon[34781]: Cluster is now healthy 2024-05-09T02:07:53.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:53 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] up:active 2024-05-09T02:07:53.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:53 smithi059 ceph-mon[34781]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T02:07:53.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:53 smithi059 ceph-mon[34781]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T02:07:53.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:53 smithi196 ceph-mon[34718]: pgmap v293: 97 pgs: 8 creating+peering, 5 unknown, 84 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-09T02:07:53.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:53 smithi196 ceph-mon[34718]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T02:07:53.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:53 smithi196 ceph-mon[34718]: Cluster is now healthy 2024-05-09T02:07:53.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:53 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] up:active 2024-05-09T02:07:53.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:53 smithi196 ceph-mon[34718]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T02:07:53.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:53 smithi196 ceph-mon[34718]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T02:07:53.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.992+0000 7f1fee747700 1 -- 172.21.15.19:0/212479397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe8102790 msgr2=0x7f1fe8102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:53.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.992+0000 7f1fee747700 1 --2- 172.21.15.19:0/212479397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe8102790 0x7f1fe8102ba0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009990 tx=0x7f1fd0009ca0 comp rx=0 tx=0).stop 2024-05-09T02:07:53.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.992+0000 7f1fee747700 1 -- 172.21.15.19:0/212479397 shutdown_connections 2024-05-09T02:07:53.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.992+0000 7f1fee747700 1 --2- 172.21.15.19:0/212479397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe81030e0 0x7f1fe8111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:53.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.992+0000 7f1fee747700 1 --2- 172.21.15.19:0/212479397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe8109300 0x7f1fe8109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:53.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.992+0000 7f1fee747700 1 --2- 172.21.15.19:0/212479397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe8102790 0x7f1fe8102ba0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:53.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.992+0000 7f1fee747700 1 -- 172.21.15.19:0/212479397 >> 172.21.15.19:0/212479397 conn(0x7f1fe80fe520 msgr2=0x7f1fe8100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:53.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.992+0000 7f1fee747700 1 -- 172.21.15.19:0/212479397 shutdown_connections 2024-05-09T02:07:53.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.992+0000 7f1fee747700 1 -- 172.21.15.19:0/212479397 wait complete. 2024-05-09T02:07:53.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.993+0000 7f1fee747700 1 Processor -- start 2024-05-09T02:07:53.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.993+0000 7f1fee747700 1 -- start start 2024-05-09T02:07:53.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.993+0000 7f1fee747700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe8102790 0x7f1fe819af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:53.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.993+0000 7f1fee747700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe81030e0 0x7f1fe819b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:53.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.993+0000 7f1fee747700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe8109300 0x7f1fe819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:53.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.993+0000 7f1fee747700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81145a0 con 0x7f1fe8109300 2024-05-09T02:07:53.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.993+0000 7f1fee747700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8114420 con 0x7f1fe8102790 2024-05-09T02:07:53.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.993+0000 7f1fee747700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8114720 con 0x7f1fe81030e0 2024-05-09T02:07:53.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.993+0000 7f1fecce4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe8109300 0x7f1fe819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:53.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.993+0000 7f1fecce4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe8109300 0x7f1fe819f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56228/0 (socket says 172.21.15.19:56228) 2024-05-09T02:07:53.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.993+0000 7f1fecce4700 1 -- 172.21.15.19:0/970586957 learned_addr learned my addr 172.21.15.19:0/970586957 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:07:53.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.993+0000 7f1fdffff700 1 --2- 172.21.15.19:0/970586957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe81030e0 0x7f1fe819b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:53.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.993+0000 7f1fe7fff700 1 --2- 172.21.15.19:0/970586957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe8102790 0x7f1fe819af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:53.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.994+0000 7f1fecce4700 1 -- 172.21.15.19:0/970586957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe81030e0 msgr2=0x7f1fe819b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:53.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.994+0000 7f1fecce4700 1 --2- 172.21.15.19:0/970586957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe81030e0 0x7f1fe819b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:53.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.994+0000 7f1fecce4700 1 -- 172.21.15.19:0/970586957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe8102790 msgr2=0x7f1fe819af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:53.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.994+0000 7f1fecce4700 1 --2- 172.21.15.19:0/970586957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe8102790 0x7f1fe819af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:53.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.994+0000 7f1fecce4700 1 -- 172.21.15.19:0/970586957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe819fe00 con 0x7f1fe8109300 2024-05-09T02:07:53.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.994+0000 7f1fe7fff700 1 --2- 172.21.15.19:0/970586957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe8102790 0x7f1fe819af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:07:53.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.994+0000 7f1fecce4700 1 --2- 172.21.15.19:0/970586957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe8109300 0x7f1fe819f760 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0007ee0 tx=0x7f1fe000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:53.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.994+0000 7f1fe5ffb700 1 -- 172.21.15.19:0/970586957 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe000ddd0 con 0x7f1fe8109300 2024-05-09T02:07:53.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.994+0000 7f1fee747700 1 -- 172.21.15.19:0/970586957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe81a00f0 con 0x7f1fe8109300 2024-05-09T02:07:53.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.994+0000 7f1fe5ffb700 1 -- 172.21.15.19:0/970586957 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fe0004020 con 0x7f1fe8109300 2024-05-09T02:07:53.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.994+0000 7f1fe5ffb700 1 -- 172.21.15.19:0/970586957 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0015660 con 0x7f1fe8109300 2024-05-09T02:07:53.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.994+0000 7f1fee747700 1 -- 172.21.15.19:0/970586957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe81a7a10 con 0x7f1fe8109300 2024-05-09T02:07:53.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.995+0000 7f1fee747700 1 -- 172.21.15.19:0/970586957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe8105ab0 con 0x7f1fe8109300 2024-05-09T02:07:53.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.997+0000 7f1fe5ffb700 1 -- 172.21.15.19:0/970586957 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1fe00041d0 con 0x7f1fe8109300 2024-05-09T02:07:54.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.998+0000 7f1fe5ffb700 1 --2- 172.21.15.19:0/970586957 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1fd40b2880 0x7f1fd40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:54.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.998+0000 7f1fe7fff700 1 --2- 172.21.15.19:0/970586957 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1fd40b2880 0x7f1fd40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:54.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.998+0000 7f1fe5ffb700 1 -- 172.21.15.19:0/970586957 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f1fe00e23b0 con 0x7f1fe8109300 2024-05-09T02:07:54.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:53.998+0000 7f1fe7fff700 1 --2- 172.21.15.19:0/970586957 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1fd40b2880 0x7f1fd40b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1fe806ca70 tx=0x7f1fd8005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:54.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:54.002+0000 7f1fe5ffb700 1 -- 172.21.15.19:0/970586957 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1fe0010070 con 0x7f1fe8109300 2024-05-09T02:07:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:54.217+0000 7f1fee747700 1 -- 172.21.15.19:0/970586957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fe819c290 con 0x7f1fe8109300 2024-05-09T02:07:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:54.220+0000 7f1fe5ffb700 1 -- 172.21.15.19:0/970586957 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22172 (secure 0 0 0) 0x7f1fe00aa220 con 0x7f1fe8109300 2024-05-09T02:07:54.223 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:07:54.223 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"fsid":"a0611f3e-0da7-11ef-bc97-c7b262605968","created":"2024-05-09T01:59:27.591343+0000","modified":"2024-05-09T02:07:50.316346+0000","last_up_change":"2024-05-09T02:06:45.565065+0000","last_in_change":"2024-05-09T02:06:24.967186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T02:02:50.018959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T02:07:43.283868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T02:07:47.110614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"996c6a3d-3e3a-451c-8789-36006e0b3dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6803","nonce":3002373576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6805","nonce":3002373576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6809","nonce":3002373576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6807","nonce":3002373576}]},"public_addr":"172.21.15.19:6803/3002373576","cluster_addr":"172.21.15.19:6805/3002373576","heartbeat_back_addr":"172.21.15.19:6809/3002373576","heartbeat_front_addr":"172.21.15.19:6807/3002373576","state":["exists","up"]},{"osd":1,"uuid":"1ac68d09-1198-4e81-86c7-e37f7c41aa5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6801","nonce":827458353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6803","nonce":827458353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6807","nonce":827458353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6805","nonce":827458353}]},"public_addr":"172.21.15.59:6801/827458353","cluster_addr":"172.21.15.59:6803/827458353","heartbeat_back_addr":"172.21.15.59:6807/827458353","heartbeat_front_addr":"172.21.15.59:6805/827458353","state":["exists","up"]},{"osd":2,"uuid":"818a0de1-04e6-471d-83df-a68833011b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6801","nonce":1659504639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6803","nonce":1659504639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6807","nonce":1659504639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6805","nonce":1659504639}]},"public_addr":"172.21.15.196:6801/1659504639","cluster_addr":"172.21.15.196:6803/1659504639","heartbeat_back_addr":"172.21.15.196:6807/1659504639","heartbeat_front_addr":"172.21.15.196:6805/1659504639","state":["exists","up"]},{"osd":3,"uuid":"7b204854-86b8-454d-aee0-8bb5775ce828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6811","nonce":2707671599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6813","nonce":2707671599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6817","nonce":2707671599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6815","nonce":2707671599}]},"public_addr":"172.21.15.19:6811/2707671599","cluster_addr":"172.21.15.19:6813/2707671599","heartbeat_back_addr":"172.21.15.19:6817/2707671599","heartbeat_front_addr":"172.21.15.19:6815/2707671599","state":["exists","up"]},{"osd":4,"uuid":"091667f1-e887-4a94-8723-6f85b676fef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6809","nonce":223037330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6811","nonce":223037330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6815","nonce":223037330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6813","nonce":223037330}]},"public_addr":"172.21.15.59:6809/223037330","cluster_addr":"172.21.15.59:6811/223037330","heartbeat_back_addr":"172.21.15.59:6815/223037330","heartbeat_front_addr":"172.21.15.59:6813/223037330","state":["exists","up"]},{"osd":5,"uuid":"2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6809","nonce":2516161898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6811","nonce":2516161898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6815","nonce":2516161898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6813","nonce":2516161898}]},"public_addr":"172.21.15.196:6809/2516161898","cluster_addr":"172.21.15.196:6811/2516161898","heartbeat_back_addr":"172.21.15.196:6815/2516161898","heartbeat_front_addr":"172.21.15.196:6813/2516161898","state":["exists","up"]},{"osd":6,"uuid":"4e352851-1279-475f-ba8b-eaf9ca73df74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6819","nonce":2150451637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6821","nonce":2150451637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6825","nonce":2150451637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6823","nonce":2150451637}]},"public_addr":"172.21.15.19:6819/2150451637","cluster_addr":"172.21.15.19:6821/2150451637","heartbeat_back_addr":"172.21.15.19:6825/2150451637","heartbeat_front_addr":"172.21.15.19:6823/2150451637","state":["exists","up"]},{"osd":7,"uuid":"83f53f59-9a60-4d87-9a8a-67e5c6f57004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6817","nonce":967902053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6819","nonce":967902053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6823","nonce":967902053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6821","nonce":967902053}]},"public_addr":"172.21.15.59:6817/967902053","cluster_addr":"172.21.15.59:6819/967902053","heartbeat_back_addr":"172.21.15.59:6823/967902053","heartbeat_front_addr":"172.21.15.59:6821/967902053","state":["exists","up"]},{"osd":8,"uuid":"a8f1bfb5-1cc8-4a65-8396-239bd781e1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6817","nonce":2946064259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6819","nonce":2946064259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6823","nonce":2946064259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6821","nonce":2946064259}]},"public_addr":"172.21.15.196:6817/2946064259","cluster_addr":"172.21.15.196:6819/2946064259","heartbeat_back_addr":"172.21.15.196:6823/2946064259","heartbeat_front_addr":"172.21.15.196:6821/2946064259","state":["exists","up"]},{"osd":9,"uuid":"73c58dc3-fcb0-4440-92b3-bf41a2e58fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6827","nonce":1735017886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6829","nonce":1735017886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6833","nonce":1735017886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6831","nonce":1735017886}]},"public_addr":"172.21.15.19:6827/1735017886","cluster_addr":"172.21.15.19:6829/1735017886","heartbeat_back_addr":"172.21.15.19:6833/1735017886","heartbeat_front_addr":"172.21.15.19:6831/1735017886","state":["exists","up"]},{"osd":10,"uuid":"8325282f-7325-4880-ab7d-b972717be9dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6825","nonce":2932899995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6827","nonce":2932899995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6831","nonce":2932899995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6829","nonce":2932899995}]},"public_addr":"172.21.15.59:6825/2932899995","cluster_addr":"172.21.15.59:6827/2932899995","heartbeat_back_addr":"172.21.15.59:6831/2932899995","heartbeat_front_addr":"172.21.15.59:6829/2932899995","state":["exists","up"]},{"osd":11,"uuid":"8b19228c-b286-43c8-a408-169782f13075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6825","nonce":2608023001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6827","nonce":2608023001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6831","nonce":2608023001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6829","nonce":2608023001}]},"public_addr":"172.21.15.196:6825/2608023001","cluster_addr":"172.21.15.196:6827/2608023001","heartbeat_back_addr":"172.21.15.196:6831/2608023001","heartbeat_front_addr":"172.21.15.196:6829/2608023001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:01:59.457748+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:23.640666+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:47.768243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:13.016746+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:39.060898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:04.508313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:31.366217+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:57.579758+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:23.177396+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:50.966399+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:16.838666+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:43.959387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3291838961":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1024133854":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1176455101":"2024-05-10T02:00:23.920230+0000","172.21.15.19:6801/2101052000":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/4018987404":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/3079777324":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6801/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/179703528":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/2101052000":"2024-05-10T02:00:23.920230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T02:07:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:54.223+0000 7f1fee747700 1 -- 172.21.15.19:0/970586957 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1fd40b2880 msgr2=0x7f1fd40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:54.223+0000 7f1fee747700 1 --2- 172.21.15.19:0/970586957 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1fd40b2880 0x7f1fd40b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1fe806ca70 tx=0x7f1fd8005f70 comp rx=0 tx=0).stop 2024-05-09T02:07:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:54.223+0000 7f1fee747700 1 -- 172.21.15.19:0/970586957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe8109300 msgr2=0x7f1fe819f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:54.223+0000 7f1fee747700 1 --2- 172.21.15.19:0/970586957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe8109300 0x7f1fe819f760 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0007ee0 tx=0x7f1fe000d3b0 comp rx=0 tx=0).stop 2024-05-09T02:07:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:54.223+0000 7f1fee747700 1 -- 172.21.15.19:0/970586957 shutdown_connections 2024-05-09T02:07:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:54.223+0000 7f1fee747700 1 --2- 172.21.15.19:0/970586957 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1fd40b2880 0x7f1fd40b4d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:54.223+0000 7f1fee747700 1 --2- 172.21.15.19:0/970586957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe8109300 0x7f1fe819f760 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:54.223+0000 7f1fee747700 1 --2- 172.21.15.19:0/970586957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe8102790 0x7f1fe819af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:54.223+0000 7f1fee747700 1 --2- 172.21.15.19:0/970586957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe81030e0 0x7f1fe819b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:54.223+0000 7f1fee747700 1 -- 172.21.15.19:0/970586957 >> 172.21.15.19:0/970586957 conn(0x7f1fe80fe520 msgr2=0x7f1fe8107240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:54.223+0000 7f1fee747700 1 -- 172.21.15.19:0/970586957 shutdown_connections 2024-05-09T02:07:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:54.223+0000 7f1fee747700 1 -- 172.21.15.19:0/970586957 wait complete. 2024-05-09T02:07:54.841 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T02:07:55.095 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:07:55.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:55 smithi059 ceph-mon[34781]: pgmap v294: 97 pgs: 97 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2024-05-09T02:07:55.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:55 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/970586957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:07:55.478 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:55 smithi019 ceph-mon[30151]: pgmap v294: 97 pgs: 97 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2024-05-09T02:07:55.478 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:55 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/970586957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:07:55.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:55 smithi196 ceph-mon[34718]: pgmap v294: 97 pgs: 97 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2024-05-09T02:07:55.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:55 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/970586957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:07:56.544 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:07:56 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i[65599]: 2024-05-09T02:07:56.088+0000 7f4635e67700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T02:07:56.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.768+0000 7f0cfa4ba700 1 -- 172.21.15.19:0/2007366994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf4104650 msgr2=0x7f0cf4104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:56.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.768+0000 7f0cfa4ba700 1 --2- 172.21.15.19:0/2007366994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf4104650 0x7f0cf4104a60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40099c0 tx=0x7f0ce4009cd0 comp rx=0 tx=0).stop 2024-05-09T02:07:56.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.768+0000 7f0cfa4ba700 1 -- 172.21.15.19:0/2007366994 shutdown_connections 2024-05-09T02:07:56.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.768+0000 7f0cfa4ba700 1 --2- 172.21.15.19:0/2007366994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cf4109880 0x7f0cf41111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:56.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.768+0000 7f0cfa4ba700 1 --2- 172.21.15.19:0/2007366994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf4104650 0x7f0cf4104a60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.768+0000 7f0cfa4ba700 1 --2- 172.21.15.19:0/2007366994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cf4108f30 0x7f0cf4109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.768+0000 7f0cfa4ba700 1 -- 172.21.15.19:0/2007366994 >> 172.21.15.19:0/2007366994 conn(0x7f0cf4079f00 msgr2=0x7f0cf407a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.768+0000 7f0cfa4ba700 1 -- 172.21.15.19:0/2007366994 shutdown_connections 2024-05-09T02:07:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.768+0000 7f0cfa4ba700 1 -- 172.21.15.19:0/2007366994 wait complete. 2024-05-09T02:07:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.768+0000 7f0cfa4ba700 1 Processor -- start 2024-05-09T02:07:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.768+0000 7f0cfa4ba700 1 -- start start 2024-05-09T02:07:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cfa4ba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cf4104650 0x7f0cf4075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cfa4ba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf4108f30 0x7f0cf4076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cfa4ba700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cf4109880 0x7f0cf4076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cfa4ba700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41145a0 con 0x7f0cf4108f30 2024-05-09T02:07:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cfa4ba700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf4114420 con 0x7f0cf4104650 2024-05-09T02:07:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cfa4ba700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf4114720 con 0x7f0cf4109880 2024-05-09T02:07:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cebfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf4108f30 0x7f0cf4076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cebfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf4108f30 0x7f0cf4076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44766/0 (socket says 172.21.15.19:44766) 2024-05-09T02:07:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cebfff700 1 -- 172.21.15.19:0/4194866196 learned_addr learned my addr 172.21.15.19:0/4194866196 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:07:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cf3fff700 1 --2- 172.21.15.19:0/4194866196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cf4104650 0x7f0cf4075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cf8a57700 1 --2- 172.21.15.19:0/4194866196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cf4109880 0x7f0cf4076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cebfff700 1 -- 172.21.15.19:0/4194866196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cf4109880 msgr2=0x7f0cf4076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cebfff700 1 --2- 172.21.15.19:0/4194866196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cf4109880 0x7f0cf4076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cebfff700 1 -- 172.21.15.19:0/4194866196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cf4104650 msgr2=0x7f0cf4075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cebfff700 1 --2- 172.21.15.19:0/4194866196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cf4104650 0x7f0cf4075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.769+0000 7f0cebfff700 1 -- 172.21.15.19:0/4194866196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf41a9300 con 0x7f0cf4108f30 2024-05-09T02:07:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.770+0000 7f0cebfff700 1 --2- 172.21.15.19:0/4194866196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf4108f30 0x7f0cf4076000 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00b500 tx=0x7f0cdc00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.770+0000 7f0cf1ffb700 1 -- 172.21.15.19:0/4194866196 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc015020 con 0x7f0cf4108f30 2024-05-09T02:07:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.770+0000 7f0cf1ffb700 1 -- 172.21.15.19:0/4194866196 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cdc011ce0 con 0x7f0cf4108f30 2024-05-09T02:07:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.770+0000 7f0cf1ffb700 1 -- 172.21.15.19:0/4194866196 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc010cb0 con 0x7f0cf4108f30 2024-05-09T02:07:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.770+0000 7f0cfa4ba700 1 -- 172.21.15.19:0/4194866196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf41a95f0 con 0x7f0cf4108f30 2024-05-09T02:07:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.770+0000 7f0cfa4ba700 1 -- 172.21.15.19:0/4194866196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf41a9b30 con 0x7f0cf4108f30 2024-05-09T02:07:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.771+0000 7f0cfa4ba700 1 -- 172.21.15.19:0/4194866196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf40512c0 con 0x7f0cf4108f30 2024-05-09T02:07:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.772+0000 7f0cf1ffb700 1 -- 172.21.15.19:0/4194866196 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0cdc018020 con 0x7f0cf4108f30 2024-05-09T02:07:56.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.773+0000 7f0cf1ffb700 1 --2- 172.21.15.19:0/4194866196 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0ce00b2900 0x7f0ce00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:56.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.773+0000 7f0cf3fff700 1 --2- 172.21.15.19:0/4194866196 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0ce00b2900 0x7f0ce00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:56.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.773+0000 7f0cf1ffb700 1 -- 172.21.15.19:0/4194866196 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f0cdc0e1720 con 0x7f0cf4108f30 2024-05-09T02:07:56.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.774+0000 7f0cf3fff700 1 --2- 172.21.15.19:0/4194866196 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0ce00b2900 0x7f0ce00b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009990 tx=0x7f0ce4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.778+0000 7f0cf1ffb700 1 -- 172.21.15.19:0/4194866196 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0cdc01f020 con 0x7f0cf4108f30 2024-05-09T02:07:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.992+0000 7f0cfa4ba700 1 -- 172.21.15.19:0/4194866196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cf4076fa0 con 0x7f0cf4108f30 2024-05-09T02:07:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.993+0000 7f0cf1ffb700 1 -- 172.21.15.19:0/4194866196 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22172 (secure 0 0 0) 0x7f0cdc0a9590 con 0x7f0cf4108f30 2024-05-09T02:07:56.996 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:07:56.996 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"fsid":"a0611f3e-0da7-11ef-bc97-c7b262605968","created":"2024-05-09T01:59:27.591343+0000","modified":"2024-05-09T02:07:50.316346+0000","last_up_change":"2024-05-09T02:06:45.565065+0000","last_in_change":"2024-05-09T02:06:24.967186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T02:02:50.018959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T02:07:43.283868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T02:07:47.110614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"996c6a3d-3e3a-451c-8789-36006e0b3dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6803","nonce":3002373576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6805","nonce":3002373576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6809","nonce":3002373576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6807","nonce":3002373576}]},"public_addr":"172.21.15.19:6803/3002373576","cluster_addr":"172.21.15.19:6805/3002373576","heartbeat_back_addr":"172.21.15.19:6809/3002373576","heartbeat_front_addr":"172.21.15.19:6807/3002373576","state":["exists","up"]},{"osd":1,"uuid":"1ac68d09-1198-4e81-86c7-e37f7c41aa5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6801","nonce":827458353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6803","nonce":827458353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6807","nonce":827458353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6805","nonce":827458353}]},"public_addr":"172.21.15.59:6801/827458353","cluster_addr":"172.21.15.59:6803/827458353","heartbeat_back_addr":"172.21.15.59:6807/827458353","heartbeat_front_addr":"172.21.15.59:6805/827458353","state":["exists","up"]},{"osd":2,"uuid":"818a0de1-04e6-471d-83df-a68833011b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6801","nonce":1659504639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6803","nonce":1659504639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6807","nonce":1659504639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6805","nonce":1659504639}]},"public_addr":"172.21.15.196:6801/1659504639","cluster_addr":"172.21.15.196:6803/1659504639","heartbeat_back_addr":"172.21.15.196:6807/1659504639","heartbeat_front_addr":"172.21.15.196:6805/1659504639","state":["exists","up"]},{"osd":3,"uuid":"7b204854-86b8-454d-aee0-8bb5775ce828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6811","nonce":2707671599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6813","nonce":2707671599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6817","nonce":2707671599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6815","nonce":2707671599}]},"public_addr":"172.21.15.19:6811/2707671599","cluster_addr":"172.21.15.19:6813/2707671599","heartbeat_back_addr":"172.21.15.19:6817/2707671599","heartbeat_front_addr":"172.21.15.19:6815/2707671599","state":["exists","up"]},{"osd":4,"uuid":"091667f1-e887-4a94-8723-6f85b676fef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6809","nonce":223037330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6811","nonce":223037330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6815","nonce":223037330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6813","nonce":223037330}]},"public_addr":"172.21.15.59:6809/223037330","cluster_addr":"172.21.15.59:6811/223037330","heartbeat_back_addr":"172.21.15.59:6815/223037330","heartbeat_front_addr":"172.21.15.59:6813/223037330","state":["exists","up"]},{"osd":5,"uuid":"2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6809","nonce":2516161898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6811","nonce":2516161898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6815","nonce":2516161898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6813","nonce":2516161898}]},"public_addr":"172.21.15.196:6809/2516161898","cluster_addr":"172.21.15.196:6811/2516161898","heartbeat_back_addr":"172.21.15.196:6815/2516161898","heartbeat_front_addr":"172.21.15.196:6813/2516161898","state":["exists","up"]},{"osd":6,"uuid":"4e352851-1279-475f-ba8b-eaf9ca73df74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6819","nonce":2150451637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6821","nonce":2150451637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6825","nonce":2150451637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6823","nonce":2150451637}]},"public_addr":"172.21.15.19:6819/2150451637","cluster_addr":"172.21.15.19:6821/2150451637","heartbeat_back_addr":"172.21.15.19:6825/2150451637","heartbeat_front_addr":"172.21.15.19:6823/2150451637","state":["exists","up"]},{"osd":7,"uuid":"83f53f59-9a60-4d87-9a8a-67e5c6f57004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6817","nonce":967902053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6819","nonce":967902053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6823","nonce":967902053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6821","nonce":967902053}]},"public_addr":"172.21.15.59:6817/967902053","cluster_addr":"172.21.15.59:6819/967902053","heartbeat_back_addr":"172.21.15.59:6823/967902053","heartbeat_front_addr":"172.21.15.59:6821/967902053","state":["exists","up"]},{"osd":8,"uuid":"a8f1bfb5-1cc8-4a65-8396-239bd781e1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6817","nonce":2946064259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6819","nonce":2946064259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6823","nonce":2946064259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6821","nonce":2946064259}]},"public_addr":"172.21.15.196:6817/2946064259","cluster_addr":"172.21.15.196:6819/2946064259","heartbeat_back_addr":"172.21.15.196:6823/2946064259","heartbeat_front_addr":"172.21.15.196:6821/2946064259","state":["exists","up"]},{"osd":9,"uuid":"73c58dc3-fcb0-4440-92b3-bf41a2e58fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6827","nonce":1735017886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6829","nonce":1735017886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6833","nonce":1735017886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6831","nonce":1735017886}]},"public_addr":"172.21.15.19:6827/1735017886","cluster_addr":"172.21.15.19:6829/1735017886","heartbeat_back_addr":"172.21.15.19:6833/1735017886","heartbeat_front_addr":"172.21.15.19:6831/1735017886","state":["exists","up"]},{"osd":10,"uuid":"8325282f-7325-4880-ab7d-b972717be9dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6825","nonce":2932899995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6827","nonce":2932899995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6831","nonce":2932899995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6829","nonce":2932899995}]},"public_addr":"172.21.15.59:6825/2932899995","cluster_addr":"172.21.15.59:6827/2932899995","heartbeat_back_addr":"172.21.15.59:6831/2932899995","heartbeat_front_addr":"172.21.15.59:6829/2932899995","state":["exists","up"]},{"osd":11,"uuid":"8b19228c-b286-43c8-a408-169782f13075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6825","nonce":2608023001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6827","nonce":2608023001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6831","nonce":2608023001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6829","nonce":2608023001}]},"public_addr":"172.21.15.196:6825/2608023001","cluster_addr":"172.21.15.196:6827/2608023001","heartbeat_back_addr":"172.21.15.196:6831/2608023001","heartbeat_front_addr":"172.21.15.196:6829/2608023001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:01:59.457748+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:23.640666+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:47.768243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:13.016746+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:39.060898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:04.508313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:31.366217+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:57.579758+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:23.177396+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:50.966399+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:16.838666+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:43.959387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3291838961":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1024133854":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1176455101":"2024-05-10T02:00:23.920230+0000","172.21.15.19:6801/2101052000":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/4018987404":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/3079777324":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6801/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/179703528":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/2101052000":"2024-05-10T02:00:23.920230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T02:07:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.996+0000 7f0cfa4ba700 1 -- 172.21.15.19:0/4194866196 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0ce00b2900 msgr2=0x7f0ce00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.996+0000 7f0cfa4ba700 1 --2- 172.21.15.19:0/4194866196 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0ce00b2900 0x7f0ce00b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009990 tx=0x7f0ce4000bc0 comp rx=0 tx=0).stop 2024-05-09T02:07:56.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.996+0000 7f0cfa4ba700 1 -- 172.21.15.19:0/4194866196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf4108f30 msgr2=0x7f0cf4076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:56.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.996+0000 7f0cfa4ba700 1 --2- 172.21.15.19:0/4194866196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf4108f30 0x7f0cf4076000 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00b500 tx=0x7f0cdc00b810 comp rx=0 tx=0).stop 2024-05-09T02:07:56.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.996+0000 7f0cfa4ba700 1 -- 172.21.15.19:0/4194866196 shutdown_connections 2024-05-09T02:07:56.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.996+0000 7f0cfa4ba700 1 --2- 172.21.15.19:0/4194866196 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0ce00b2900 0x7f0ce00b4e10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:56.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.996+0000 7f0cfa4ba700 1 --2- 172.21.15.19:0/4194866196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cf4109880 0x7f0cf4076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:56.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.996+0000 7f0cfa4ba700 1 --2- 172.21.15.19:0/4194866196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cf4104650 0x7f0cf4075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:56.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.996+0000 7f0cfa4ba700 1 --2- 172.21.15.19:0/4194866196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf4108f30 0x7f0cf4076000 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:56.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.996+0000 7f0cfa4ba700 1 -- 172.21.15.19:0/4194866196 >> 172.21.15.19:0/4194866196 conn(0x7f0cf4079f00 msgr2=0x7f0cf4102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:56.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.996+0000 7f0cfa4ba700 1 -- 172.21.15.19:0/4194866196 shutdown_connections 2024-05-09T02:07:56.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:56.996+0000 7f0cfa4ba700 1 -- 172.21.15.19:0/4194866196 wait complete. 2024-05-09T02:07:57.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:57 smithi019 ceph-mon[30151]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-09T02:07:57.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:57 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/4194866196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:07:57.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:57 smithi059 ceph-mon[34781]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-09T02:07:57.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:57 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/4194866196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:07:57.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:57 smithi196 ceph-mon[34718]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-09T02:07:57.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:57 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/4194866196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:07:57.706 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-09T02:07:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:07:59.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:07:59 smithi019 ceph-mon[30151]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-09T02:07:59.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:07:59 smithi059 ceph-mon[34781]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-09T02:07:59.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:07:59 smithi196 ceph-mon[34718]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-09T02:07:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.839+0000 7f89762a3700 1 -- 172.21.15.19:0/2891536432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89701031c0 msgr2=0x7f8970111160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.839+0000 7f89762a3700 1 --2- 172.21.15.19:0/2891536432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89701031c0 0x7f8970111160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f896800b0a0 tx=0x7f896800b3b0 comp rx=0 tx=0).stop 2024-05-09T02:07:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.839+0000 7f89762a3700 1 -- 172.21.15.19:0/2891536432 shutdown_connections 2024-05-09T02:07:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.839+0000 7f89762a3700 1 --2- 172.21.15.19:0/2891536432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89701031c0 0x7f8970111160 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.839+0000 7f89762a3700 1 --2- 172.21.15.19:0/2891536432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89701092d0 0x7f89701096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.839+0000 7f89762a3700 1 --2- 172.21.15.19:0/2891536432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8970102870 0x7f8970102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.839+0000 7f89762a3700 1 -- 172.21.15.19:0/2891536432 >> 172.21.15.19:0/2891536432 conn(0x7f89700fe540 msgr2=0x7f8970100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:07:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.839+0000 7f89762a3700 1 -- 172.21.15.19:0/2891536432 shutdown_connections 2024-05-09T02:07:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.839+0000 7f89762a3700 1 -- 172.21.15.19:0/2891536432 wait complete. 2024-05-09T02:07:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.840+0000 7f89762a3700 1 Processor -- start 2024-05-09T02:07:59.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.840+0000 7f89762a3700 1 -- start start 2024-05-09T02:07:59.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.840+0000 7f89762a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8970102870 0x7f8970103a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:59.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.840+0000 7f89762a3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89701031c0 0x7f8970103f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:59.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.840+0000 7f89762a3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89701092d0 0x7f89701044d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.840+0000 7f89762a3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f89701145e0 con 0x7f89701031c0 2024-05-09T02:07:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.840+0000 7f89762a3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8970114460 con 0x7f8970102870 2024-05-09T02:07:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.840+0000 7f89762a3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8970114760 con 0x7f89701092d0 2024-05-09T02:07:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.840+0000 7f896f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89701031c0 0x7f8970103f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.840+0000 7f896f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89701031c0 0x7f8970103f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44792/0 (socket says 172.21.15.19:44792) 2024-05-09T02:07:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.840+0000 7f896f7fe700 1 -- 172.21.15.19:0/2256574823 learned_addr learned my addr 172.21.15.19:0/2256574823 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:07:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.840+0000 7f896ffff700 1 --2- 172.21.15.19:0/2256574823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8970102870 0x7f8970103a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.840+0000 7f8974840700 1 --2- 172.21.15.19:0/2256574823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89701092d0 0x7f89701044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.841+0000 7f896f7fe700 1 -- 172.21.15.19:0/2256574823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89701092d0 msgr2=0x7f89701044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.841+0000 7f896f7fe700 1 --2- 172.21.15.19:0/2256574823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89701092d0 0x7f89701044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.841+0000 7f896f7fe700 1 -- 172.21.15.19:0/2256574823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8970102870 msgr2=0x7f8970103a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:07:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.841+0000 7f896f7fe700 1 --2- 172.21.15.19:0/2256574823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8970102870 0x7f8970103a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:07:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.841+0000 7f896f7fe700 1 -- 172.21.15.19:0/2256574823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89701a91a0 con 0x7f89701031c0 2024-05-09T02:07:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.841+0000 7f8974840700 1 --2- 172.21.15.19:0/2256574823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89701092d0 0x7f89701044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:07:59.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.841+0000 7f896f7fe700 1 --2- 172.21.15.19:0/2256574823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89701031c0 0x7f8970103f90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f896400c9e0 tx=0x7f896400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:59.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.841+0000 7f896d7fa700 1 -- 172.21.15.19:0/2256574823 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964007aa0 con 0x7f89701031c0 2024-05-09T02:07:59.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.841+0000 7f89762a3700 1 -- 172.21.15.19:0/2256574823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89701a9490 con 0x7f89701031c0 2024-05-09T02:07:59.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.841+0000 7f896d7fa700 1 -- 172.21.15.19:0/2256574823 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8964007c40 con 0x7f89701031c0 2024-05-09T02:07:59.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.841+0000 7f896d7fa700 1 -- 172.21.15.19:0/2256574823 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964018660 con 0x7f89701031c0 2024-05-09T02:07:59.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.841+0000 7f89762a3700 1 -- 172.21.15.19:0/2256574823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89701a99d0 con 0x7f89701031c0 2024-05-09T02:07:59.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.842+0000 7f89762a3700 1 -- 172.21.15.19:0/2256574823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89700512c0 con 0x7f89701031c0 2024-05-09T02:07:59.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.844+0000 7f896d7fa700 1 -- 172.21.15.19:0/2256574823 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8964018800 con 0x7f89701031c0 2024-05-09T02:07:59.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.845+0000 7f896d7fa700 1 --2- 172.21.15.19:0/2256574823 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f89580b2880 0x7f89580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:07:59.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.845+0000 7f896ffff700 1 --2- 172.21.15.19:0/2256574823 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f89580b2880 0x7f89580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:07:59.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.845+0000 7f896d7fa700 1 -- 172.21.15.19:0/2256574823 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f89640e23d0 con 0x7f89701031c0 2024-05-09T02:07:59.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.845+0000 7f896ffff700 1 --2- 172.21.15.19:0/2256574823 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f89580b2880 0x7f89580b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8960005950 tx=0x7f896000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:07:59.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:07:59.848+0000 7f896d7fa700 1 -- 172.21.15.19:0/2256574823 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8964014030 con 0x7f89701031c0 2024-05-09T02:08:00.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:00.113+0000 7f89762a3700 1 -- 172.21.15.19:0/2256574823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f897002e5f0 con 0x7f89701031c0 2024-05-09T02:08:00.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:00.207+0000 7f896d7fa700 1 -- 172.21.15.19:0/2256574823 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f89640aa230 con 0x7f89701031c0 2024-05-09T02:08:00.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:00.210+0000 7f89762a3700 1 -- 172.21.15.19:0/2256574823 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f89580b2880 msgr2=0x7f89580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:00.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:00.210+0000 7f89762a3700 1 --2- 172.21.15.19:0/2256574823 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f89580b2880 0x7f89580b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8960005950 tx=0x7f896000b410 comp rx=0 tx=0).stop 2024-05-09T02:08:00.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:00.210+0000 7f89762a3700 1 -- 172.21.15.19:0/2256574823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89701031c0 msgr2=0x7f8970103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:00.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:00.210+0000 7f89762a3700 1 --2- 172.21.15.19:0/2256574823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89701031c0 0x7f8970103f90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f896400c9e0 tx=0x7f896400ccf0 comp rx=0 tx=0).stop 2024-05-09T02:08:00.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:00.210+0000 7f89762a3700 1 -- 172.21.15.19:0/2256574823 shutdown_connections 2024-05-09T02:08:00.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:00.210+0000 7f89762a3700 1 --2- 172.21.15.19:0/2256574823 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f89580b2880 0x7f89580b4d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:00.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:00.210+0000 7f89762a3700 1 --2- 172.21.15.19:0/2256574823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89701092d0 0x7f89701044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:00.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:00.210+0000 7f89762a3700 1 --2- 172.21.15.19:0/2256574823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8970102870 0x7f8970103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:00.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:00.210+0000 7f89762a3700 1 --2- 172.21.15.19:0/2256574823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89701031c0 0x7f8970103f90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:00.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:00.210+0000 7f89762a3700 1 -- 172.21.15.19:0/2256574823 >> 172.21.15.19:0/2256574823 conn(0x7f89700fe540 msgr2=0x7f89701070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:00.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:00.210+0000 7f89762a3700 1 -- 172.21.15.19:0/2256574823 shutdown_connections 2024-05-09T02:08:00.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:00.211+0000 7f89762a3700 1 -- 172.21.15.19:0/2256574823 wait complete. 2024-05-09T02:08:00.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:00 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2256574823' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T02:08:00.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:00 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2256574823' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T02:08:00.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:00 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2256574823' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T02:08:00.997 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-09T02:08:00.997 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-09T02:08:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:01.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:01 smithi019 ceph-mon[30151]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T02:08:01.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:01 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2256574823' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T02:08:01.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:01 smithi019 ceph-mon[30151]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T02:08:01.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:01 smithi059 ceph-mon[34781]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T02:08:01.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:01 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2256574823' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T02:08:01.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:01 smithi059 ceph-mon[34781]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T02:08:01.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:01 smithi196 ceph-mon[34718]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T02:08:01.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:01 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2256574823' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T02:08:01.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:01 smithi196 ceph-mon[34718]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T02:08:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.021+0000 7fd99a507700 1 -- 172.21.15.19:0/3181841553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd994102ae0 msgr2=0x7fd994102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.021+0000 7fd99a507700 1 --2- 172.21.15.19:0/3181841553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd994102ae0 0x7fd994102ef0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd9840099c0 tx=0x7fd984009cd0 comp rx=0 tx=0).stop 2024-05-09T02:08:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.021+0000 7fd99a507700 1 -- 172.21.15.19:0/3181841553 shutdown_connections 2024-05-09T02:08:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.021+0000 7fd99a507700 1 --2- 172.21.15.19:0/3181841553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd994109850 0x7fd9941111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.021+0000 7fd99a507700 1 --2- 172.21.15.19:0/3181841553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd994102ae0 0x7fd994102ef0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.021+0000 7fd99a507700 1 --2- 172.21.15.19:0/3181841553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd994108f00 0x7fd994109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:03.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.021+0000 7fd99a507700 1 -- 172.21.15.19:0/3181841553 >> 172.21.15.19:0/3181841553 conn(0x7fd9940fe540 msgr2=0x7fd994100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.021+0000 7fd99a507700 1 -- 172.21.15.19:0/3181841553 shutdown_connections 2024-05-09T02:08:03.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.021+0000 7fd99a507700 1 -- 172.21.15.19:0/3181841553 wait complete. 2024-05-09T02:08:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.022+0000 7fd99a507700 1 Processor -- start 2024-05-09T02:08:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.022+0000 7fd99a507700 1 -- start start 2024-05-09T02:08:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.022+0000 7fd99a507700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd994102ae0 0x7fd994103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.022+0000 7fd99a507700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd994108f00 0x7fd994103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.022+0000 7fd99a507700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd994109850 0x7fd9941044b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.023+0000 7fd99a507700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd994114480 con 0x7fd994102ae0 2024-05-09T02:08:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.023+0000 7fd99a507700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd994114300 con 0x7fd994108f00 2024-05-09T02:08:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.023+0000 7fd99a507700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd994114600 con 0x7fd994109850 2024-05-09T02:08:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.023+0000 7fd993fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd994102ae0 0x7fd994103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.023+0000 7fd993fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd994102ae0 0x7fd994103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44814/0 (socket says 172.21.15.19:44814) 2024-05-09T02:08:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.023+0000 7fd993fff700 1 -- 172.21.15.19:0/1205369530 learned_addr learned my addr 172.21.15.19:0/1205369530 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.023+0000 7fd98bfff700 1 --2- 172.21.15.19:0/1205369530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd994108f00 0x7fd994103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.023+0000 7fd998aa4700 1 --2- 172.21.15.19:0/1205369530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd994109850 0x7fd9941044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.023+0000 7fd993fff700 1 -- 172.21.15.19:0/1205369530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd994109850 msgr2=0x7fd9941044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.023+0000 7fd993fff700 1 --2- 172.21.15.19:0/1205369530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd994109850 0x7fd9941044b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.023+0000 7fd993fff700 1 -- 172.21.15.19:0/1205369530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd994108f00 msgr2=0x7fd994103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.023+0000 7fd993fff700 1 --2- 172.21.15.19:0/1205369530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd994108f00 0x7fd994103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.023+0000 7fd993fff700 1 -- 172.21.15.19:0/1205369530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9941a9100 con 0x7fd994102ae0 2024-05-09T02:08:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.023+0000 7fd98bfff700 1 --2- 172.21.15.19:0/1205369530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd994108f00 0x7fd994103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:08:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.023+0000 7fd993fff700 1 --2- 172.21.15.19:0/1205369530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd994102ae0 0x7fd994103a30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd9840098e0 tx=0x7fd984003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.024+0000 7fd991ffb700 1 -- 172.21.15.19:0/1205369530 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984027070 con 0x7fd994102ae0 2024-05-09T02:08:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.024+0000 7fd99a507700 1 -- 172.21.15.19:0/1205369530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9941a9390 con 0x7fd994102ae0 2024-05-09T02:08:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.024+0000 7fd991ffb700 1 -- 172.21.15.19:0/1205369530 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9840040a0 con 0x7fd994102ae0 2024-05-09T02:08:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.024+0000 7fd991ffb700 1 -- 172.21.15.19:0/1205369530 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98400f460 con 0x7fd994102ae0 2024-05-09T02:08:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.024+0000 7fd99a507700 1 -- 172.21.15.19:0/1205369530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9941a97e0 con 0x7fd994102ae0 2024-05-09T02:08:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.025+0000 7fd99a507700 1 -- 172.21.15.19:0/1205369530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9940512c0 con 0x7fd994102ae0 2024-05-09T02:08:03.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.027+0000 7fd991ffb700 1 -- 172.21.15.19:0/1205369530 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd98402a070 con 0x7fd994102ae0 2024-05-09T02:08:03.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.028+0000 7fd991ffb700 1 --2- 172.21.15.19:0/1205369530 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd9800b2880 0x7fd9800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:03.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.028+0000 7fd98bfff700 1 --2- 172.21.15.19:0/1205369530 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd9800b2880 0x7fd9800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:03.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.028+0000 7fd991ffb700 1 -- 172.21.15.19:0/1205369530 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fd9840e3500 con 0x7fd994102ae0 2024-05-09T02:08:03.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.028+0000 7fd98bfff700 1 --2- 172.21.15.19:0/1205369530 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd9800b2880 0x7fd9800b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd994101c10 tx=0x7fd97c005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:03.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.033+0000 7fd991ffb700 1 -- 172.21.15.19:0/1205369530 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd984013070 con 0x7fd994102ae0 2024-05-09T02:08:03.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:03.299+0000 7fd99a507700 1 -- 172.21.15.19:0/1205369530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fd99402e5f0 con 0x7fd994102ae0 2024-05-09T02:08:03.302 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:03 smithi019 ceph-mon[30151]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-09T02:08:03.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:03 smithi059 ceph-mon[34781]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-09T02:08:03.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:03 smithi196 ceph-mon[34718]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-09T02:08:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:04.217+0000 7fd991ffb700 1 -- 172.21.15.19:0/1205369530 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fd9840ab400 con 0x7fd994102ae0 2024-05-09T02:08:04.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:04.220+0000 7fd99a507700 1 -- 172.21.15.19:0/1205369530 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd9800b2880 msgr2=0x7fd9800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:04.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:04.220+0000 7fd99a507700 1 --2- 172.21.15.19:0/1205369530 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd9800b2880 0x7fd9800b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd994101c10 tx=0x7fd97c005f50 comp rx=0 tx=0).stop 2024-05-09T02:08:04.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:04.220+0000 7fd99a507700 1 -- 172.21.15.19:0/1205369530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd994102ae0 msgr2=0x7fd994103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:04.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:04.220+0000 7fd99a507700 1 --2- 172.21.15.19:0/1205369530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd994102ae0 0x7fd994103a30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd9840098e0 tx=0x7fd984003680 comp rx=0 tx=0).stop 2024-05-09T02:08:04.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:04.220+0000 7fd99a507700 1 -- 172.21.15.19:0/1205369530 shutdown_connections 2024-05-09T02:08:04.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:04.220+0000 7fd99a507700 1 --2- 172.21.15.19:0/1205369530 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd9800b2880 0x7fd9800b4d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:04.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:04.220+0000 7fd99a507700 1 --2- 172.21.15.19:0/1205369530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd994109850 0x7fd9941044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:04.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:04.220+0000 7fd99a507700 1 --2- 172.21.15.19:0/1205369530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd994102ae0 0x7fd994103a30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:04.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:04.220+0000 7fd99a507700 1 --2- 172.21.15.19:0/1205369530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd994108f00 0x7fd994103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:04.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:04.220+0000 7fd99a507700 1 -- 172.21.15.19:0/1205369530 >> 172.21.15.19:0/1205369530 conn(0x7fd9940fe540 msgr2=0x7fd994107170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:04.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:04.220+0000 7fd99a507700 1 -- 172.21.15.19:0/1205369530 shutdown_connections 2024-05-09T02:08:04.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:04.220+0000 7fd99a507700 1 -- 172.21.15.19:0/1205369530 wait complete. 2024-05-09T02:08:04.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:04 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1205369530' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T02:08:04.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:04 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1205369530' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T02:08:04.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:04 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1205369530' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T02:08:05.044 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-09T02:08:05.313 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:05.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:05 smithi019 ceph-mon[30151]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-09T02:08:05.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:05 smithi019 ceph-mon[30151]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T02:08:05.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:05 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1205369530' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T02:08:05.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:05 smithi019 ceph-mon[30151]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T02:08:05.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:05 smithi019 ceph-mon[30151]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T02:08:05.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:05 smithi019 ceph-mon[30151]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-05-09T02:08:05.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:05 smithi019 ceph-mon[30151]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-09T02:08:05.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:05 smithi196 ceph-mon[34718]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-09T02:08:05.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:05 smithi196 ceph-mon[34718]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T02:08:05.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:05 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1205369530' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T02:08:05.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:05 smithi196 ceph-mon[34718]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T02:08:05.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:05 smithi196 ceph-mon[34718]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T02:08:05.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:05 smithi196 ceph-mon[34718]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-05-09T02:08:05.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:05 smithi196 ceph-mon[34718]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-09T02:08:05.617 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:05 smithi059 ceph-mon[34781]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-09T02:08:05.618 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:05 smithi059 ceph-mon[34781]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T02:08:05.618 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:05 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1205369530' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T02:08:05.618 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:05 smithi059 ceph-mon[34781]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T02:08:05.618 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:05 smithi059 ceph-mon[34781]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T02:08:05.618 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:05 smithi059 ceph-mon[34781]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-05-09T02:08:05.618 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:05 smithi059 ceph-mon[34781]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-09T02:08:06.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:06 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] up:active 2024-05-09T02:08:06.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:06 smithi196 ceph-mon[34718]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T02:08:06.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:06 smithi196 ceph-mon[34718]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-05-09T02:08:06.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:06 smithi196 ceph-mon[34718]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-05-09T02:08:06.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:06 smithi196 ceph-mon[34718]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-09T02:08:06.545 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:08:06 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i[65599]: 2024-05-09T02:08:06.087+0000 7f4635e67700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-09T02:08:06.558 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:06 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] up:active 2024-05-09T02:08:06.559 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:06 smithi019 ceph-mon[30151]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T02:08:06.559 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:06 smithi019 ceph-mon[30151]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-05-09T02:08:06.559 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:06 smithi019 ceph-mon[30151]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-05-09T02:08:06.559 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:06 smithi019 ceph-mon[30151]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-09T02:08:06.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:06 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] up:active 2024-05-09T02:08:06.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:06 smithi059 ceph-mon[34781]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T02:08:06.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:06 smithi059 ceph-mon[34781]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-05-09T02:08:06.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:06 smithi059 ceph-mon[34781]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-05-09T02:08:06.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:06 smithi059 ceph-mon[34781]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-09T02:08:07.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.140+0000 7fe44e50e700 1 -- 172.21.15.19:0/1051765456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4481034e0 msgr2=0x7fe4481038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:07.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.140+0000 7fe44e50e700 1 --2- 172.21.15.19:0/1051765456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4481034e0 0x7fe4481038f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4380099c0 tx=0x7fe438009cd0 comp rx=0 tx=0).stop 2024-05-09T02:08:07.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.140+0000 7fe44e50e700 1 -- 172.21.15.19:0/1051765456 shutdown_connections 2024-05-09T02:08:07.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.140+0000 7fe44e50e700 1 --2- 172.21.15.19:0/1051765456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe448108760 0x7fe4481100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:07.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.140+0000 7fe44e50e700 1 --2- 172.21.15.19:0/1051765456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4481034e0 0x7fe4481038f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:07.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.140+0000 7fe44e50e700 1 --2- 172.21.15.19:0/1051765456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe448107e10 0x7fe448108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:07.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.140+0000 7fe44e50e700 1 -- 172.21.15.19:0/1051765456 >> 172.21.15.19:0/1051765456 conn(0x7fe4480fef30 msgr2=0x7fe448101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:07.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.140+0000 7fe44e50e700 1 -- 172.21.15.19:0/1051765456 shutdown_connections 2024-05-09T02:08:07.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.140+0000 7fe44e50e700 1 -- 172.21.15.19:0/1051765456 wait complete. 2024-05-09T02:08:07.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.141+0000 7fe44e50e700 1 Processor -- start 2024-05-09T02:08:07.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.141+0000 7fe44e50e700 1 -- start start 2024-05-09T02:08:07.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.141+0000 7fe44e50e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4481034e0 0x7fe448199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:07.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe44e50e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe448107e10 0x7fe44819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:07.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe44e50e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe448108760 0x7fe44819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:07.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe44e50e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe448113390 con 0x7fe448107e10 2024-05-09T02:08:07.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe44e50e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe448113210 con 0x7fe4481034e0 2024-05-09T02:08:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe44e50e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe448113510 con 0x7fe448108760 2024-05-09T02:08:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe4477fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe448107e10 0x7fe44819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe4477fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe448107e10 0x7fe44819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46092/0 (socket says 172.21.15.19:46092) 2024-05-09T02:08:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe4477fe700 1 -- 172.21.15.19:0/1023188845 learned_addr learned my addr 172.21.15.19:0/1023188845 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe447fff700 1 --2- 172.21.15.19:0/1023188845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4481034e0 0x7fe448199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe44caab700 1 --2- 172.21.15.19:0/1023188845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe448108760 0x7fe44819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe4477fe700 1 -- 172.21.15.19:0/1023188845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe448108760 msgr2=0x7fe44819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe4477fe700 1 --2- 172.21.15.19:0/1023188845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe448108760 0x7fe44819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe4477fe700 1 -- 172.21.15.19:0/1023188845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4481034e0 msgr2=0x7fe448199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe4477fe700 1 --2- 172.21.15.19:0/1023188845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4481034e0 0x7fe448199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe4477fe700 1 -- 172.21.15.19:0/1023188845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44819ec10 con 0x7fe448107e10 2024-05-09T02:08:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe447fff700 1 --2- 172.21.15.19:0/1023188845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4481034e0 0x7fe448199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:08:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe4477fe700 1 --2- 172.21.15.19:0/1023188845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe448107e10 0x7fe44819a2e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00e970 tx=0x7fe43c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:07.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe4457fa700 1 -- 172.21.15.19:0/1023188845 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c004a30 con 0x7fe448107e10 2024-05-09T02:08:07.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.142+0000 7fe44e50e700 1 -- 172.21.15.19:0/1023188845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44819ef00 con 0x7fe448107e10 2024-05-09T02:08:07.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.143+0000 7fe4457fa700 1 -- 172.21.15.19:0/1023188845 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe43c00cc90 con 0x7fe448107e10 2024-05-09T02:08:07.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.143+0000 7fe4457fa700 1 -- 172.21.15.19:0/1023188845 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c005610 con 0x7fe448107e10 2024-05-09T02:08:07.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.143+0000 7fe44e50e700 1 -- 172.21.15.19:0/1023188845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4481a6820 con 0x7fe448107e10 2024-05-09T02:08:07.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.144+0000 7fe44e50e700 1 -- 172.21.15.19:0/1023188845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4480512c0 con 0x7fe448107e10 2024-05-09T02:08:07.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.146+0000 7fe4457fa700 1 -- 172.21.15.19:0/1023188845 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe43c00c7e0 con 0x7fe448107e10 2024-05-09T02:08:07.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.147+0000 7fe4457fa700 1 --2- 172.21.15.19:0/1023188845 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe4300b2880 0x7fe4300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:07.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.147+0000 7fe447fff700 1 --2- 172.21.15.19:0/1023188845 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe4300b2880 0x7fe4300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:07.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.147+0000 7fe4457fa700 1 -- 172.21.15.19:0/1023188845 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fe43c0e24a0 con 0x7fe448107e10 2024-05-09T02:08:07.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.147+0000 7fe447fff700 1 --2- 172.21.15.19:0/1023188845 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe4300b2880 0x7fe4300b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe438000c00 tx=0x7fe43801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:07.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.150+0000 7fe4457fa700 1 -- 172.21.15.19:0/1023188845 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe43c054df0 con 0x7fe448107e10 2024-05-09T02:08:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:07.418+0000 7fe44e50e700 1 -- 172.21.15.19:0/1023188845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fe44819b0a0 con 0x7fe448107e10 2024-05-09T02:08:07.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:07 smithi019 ceph-mon[30151]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 4 op/s 2024-05-09T02:08:07.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:07 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] up:active 2024-05-09T02:08:07.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:07 smithi019 ceph-mon[30151]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T02:08:07.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:07 smithi019 ceph-mon[30151]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-09T02:08:07.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:07 smithi019 ceph-mon[30151]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-05-09T02:08:07.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:07 smithi019 ceph-mon[30151]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-09T02:08:07.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:07 smithi196 ceph-mon[34718]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 4 op/s 2024-05-09T02:08:07.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:07 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] up:active 2024-05-09T02:08:07.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:07 smithi196 ceph-mon[34718]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T02:08:07.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:07 smithi196 ceph-mon[34718]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-09T02:08:07.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:07 smithi196 ceph-mon[34718]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-05-09T02:08:07.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:07 smithi196 ceph-mon[34718]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-09T02:08:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:07 smithi059 ceph-mon[34781]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 4 op/s 2024-05-09T02:08:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:07 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] up:active 2024-05-09T02:08:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:07 smithi059 ceph-mon[34781]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T02:08:07.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:07 smithi059 ceph-mon[34781]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-09T02:08:07.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:07 smithi059 ceph-mon[34781]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-05-09T02:08:07.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:07 smithi059 ceph-mon[34781]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-09T02:08:08.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:08.256+0000 7fe4457fa700 1 -- 172.21.15.19:0/1023188845 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v26) ==== 121+0+0 (secure 0 0 0) 0x7fe43c0aa310 con 0x7fe448107e10 2024-05-09T02:08:08.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:08.259+0000 7fe44e50e700 1 -- 172.21.15.19:0/1023188845 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe4300b2880 msgr2=0x7fe4300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:08.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:08.259+0000 7fe44e50e700 1 --2- 172.21.15.19:0/1023188845 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe4300b2880 0x7fe4300b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe438000c00 tx=0x7fe43801a040 comp rx=0 tx=0).stop 2024-05-09T02:08:08.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:08.259+0000 7fe44e50e700 1 -- 172.21.15.19:0/1023188845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe448107e10 msgr2=0x7fe44819a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:08.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:08.259+0000 7fe44e50e700 1 --2- 172.21.15.19:0/1023188845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe448107e10 0x7fe44819a2e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00e970 tx=0x7fe43c00ec80 comp rx=0 tx=0).stop 2024-05-09T02:08:08.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:08.259+0000 7fe44e50e700 1 -- 172.21.15.19:0/1023188845 shutdown_connections 2024-05-09T02:08:08.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:08.259+0000 7fe44e50e700 1 --2- 172.21.15.19:0/1023188845 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe4300b2880 0x7fe4300b4d90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:08.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:08.259+0000 7fe44e50e700 1 --2- 172.21.15.19:0/1023188845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe448108760 0x7fe44819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:08.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:08.259+0000 7fe44e50e700 1 --2- 172.21.15.19:0/1023188845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4481034e0 0x7fe448199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:08.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:08.259+0000 7fe44e50e700 1 --2- 172.21.15.19:0/1023188845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe448107e10 0x7fe44819a2e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:08.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:08.259+0000 7fe44e50e700 1 -- 172.21.15.19:0/1023188845 >> 172.21.15.19:0/1023188845 conn(0x7fe4480fef30 msgr2=0x7fe4481063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:08.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:08.259+0000 7fe44e50e700 1 -- 172.21.15.19:0/1023188845 shutdown_connections 2024-05-09T02:08:08.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:08.259+0000 7fe44e50e700 1 -- 172.21.15.19:0/1023188845 wait complete. 2024-05-09T02:08:08.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:08 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] up:active 2024-05-09T02:08:08.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:08 smithi196 ceph-mon[34718]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T02:08:08.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:08 smithi196 ceph-mon[34718]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T02:08:08.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:08 smithi196 ceph-mon[34718]: Cluster is now healthy 2024-05-09T02:08:08.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:08 smithi196 ceph-mon[34718]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-05-09T02:08:08.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:08 smithi196 ceph-mon[34718]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-09T02:08:08.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:08 smithi196 ceph-mon[34718]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-09T02:08:08.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:08 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1023188845' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T02:08:08.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:08 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] up:active 2024-05-09T02:08:08.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:08 smithi019 ceph-mon[30151]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T02:08:08.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:08 smithi019 ceph-mon[30151]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T02:08:08.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:08 smithi019 ceph-mon[30151]: Cluster is now healthy 2024-05-09T02:08:08.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:08 smithi019 ceph-mon[30151]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-05-09T02:08:08.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:08 smithi019 ceph-mon[30151]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-09T02:08:08.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:08 smithi019 ceph-mon[30151]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-09T02:08:08.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:08 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1023188845' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T02:08:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:08 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] up:active 2024-05-09T02:08:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:08 smithi059 ceph-mon[34781]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T02:08:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:08 smithi059 ceph-mon[34781]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T02:08:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:08 smithi059 ceph-mon[34781]: Cluster is now healthy 2024-05-09T02:08:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:08 smithi059 ceph-mon[34781]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-05-09T02:08:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:08 smithi059 ceph-mon[34781]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-09T02:08:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:08 smithi059 ceph-mon[34781]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-09T02:08:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:08 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1023188845' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T02:08:08.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-09T02:08:09.247 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:09.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:09 smithi019 ceph-mon[30151]: pgmap v301: 97 pgs: 97 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 4 op/s 2024-05-09T02:08:09.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:09 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1023188845' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T02:08:09.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:09 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] up:active 2024-05-09T02:08:09.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:09 smithi019 ceph-mon[30151]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-09T02:08:09.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:09 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:08:09.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:09 smithi196 ceph-mon[34718]: pgmap v301: 97 pgs: 97 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 4 op/s 2024-05-09T02:08:09.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:09 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1023188845' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T02:08:09.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:09 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] up:active 2024-05-09T02:08:09.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:09 smithi196 ceph-mon[34718]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-09T02:08:09.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:09 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:08:09.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:09 smithi059 ceph-mon[34781]: pgmap v301: 97 pgs: 97 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 4 op/s 2024-05-09T02:08:09.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:09 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1023188845' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T02:08:09.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:09 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] up:active 2024-05-09T02:08:09.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:09 smithi059 ceph-mon[34781]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-09T02:08:09.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:09 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:08:11.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.069+0000 7f37e617d700 1 -- 172.21.15.19:0/138492774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e010e930 msgr2=0x7f37e0110e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:11.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.069+0000 7f37e617d700 1 --2- 172.21.15.19:0/138492774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e010e930 0x7f37e0110e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f37d800b0a0 tx=0x7f37d800b3b0 comp rx=0 tx=0).stop 2024-05-09T02:08:11.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.069+0000 7f37e617d700 1 -- 172.21.15.19:0/138492774 shutdown_connections 2024-05-09T02:08:11.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.069+0000 7f37e617d700 1 --2- 172.21.15.19:0/138492774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e010e930 0x7f37e0110e10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:11.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.069+0000 7f37e617d700 1 --2- 172.21.15.19:0/138492774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e0105550 0x7f37e0105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:11.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.069+0000 7f37e617d700 1 --2- 172.21.15.19:0/138492774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e0105ea0 0x7f37e010e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:11.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.069+0000 7f37e617d700 1 -- 172.21.15.19:0/138492774 >> 172.21.15.19:0/138492774 conn(0x7f37e0078e00 msgr2=0x7f37e0079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:11.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.069+0000 7f37e617d700 1 -- 172.21.15.19:0/138492774 shutdown_connections 2024-05-09T02:08:11.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.069+0000 7f37e617d700 1 -- 172.21.15.19:0/138492774 wait complete. 2024-05-09T02:08:11.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.069+0000 7f37e617d700 1 Processor -- start 2024-05-09T02:08:11.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37e617d700 1 -- start start 2024-05-09T02:08:11.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37e617d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e0105550 0x7f37e0103240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:11.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37e617d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e0105ea0 0x7f37e0103780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:11.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37e617d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e010e930 0x7f37e0103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:11.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37e617d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0114480 con 0x7f37e010e930 2024-05-09T02:08:11.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37e617d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0114300 con 0x7f37e0105ea0 2024-05-09T02:08:11.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37e617d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0114600 con 0x7f37e0105550 2024-05-09T02:08:11.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e010e930 0x7f37e0103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:11.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e010e930 0x7f37e0103cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46114/0 (socket says 172.21.15.19:46114) 2024-05-09T02:08:11.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37dffff700 1 -- 172.21.15.19:0/686133956 learned_addr learned my addr 172.21.15.19:0/686133956 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:11.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37df7fe700 1 --2- 172.21.15.19:0/686133956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e0105550 0x7f37e0103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:11.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37deffd700 1 --2- 172.21.15.19:0/686133956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e0105ea0 0x7f37e0103780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:11.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37dffff700 1 -- 172.21.15.19:0/686133956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e0105550 msgr2=0x7f37e0103240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:11.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37dffff700 1 --2- 172.21.15.19:0/686133956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e0105550 0x7f37e0103240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:11.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37dffff700 1 -- 172.21.15.19:0/686133956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e0105ea0 msgr2=0x7f37e0103780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:11.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37dffff700 1 --2- 172.21.15.19:0/686133956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e0105ea0 0x7f37e0103780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:11.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37dffff700 1 -- 172.21.15.19:0/686133956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e01a91d0 con 0x7f37e010e930 2024-05-09T02:08:11.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.070+0000 7f37deffd700 1 --2- 172.21.15.19:0/686133956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e0105ea0 0x7f37e0103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:08:11.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.071+0000 7f37dffff700 1 --2- 172.21.15.19:0/686133956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e010e930 0x7f37e0103cc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f37d800b830 tx=0x7f37d8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:11.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.071+0000 7f37dcff9700 1 -- 172.21.15.19:0/686133956 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8021070 con 0x7f37e010e930 2024-05-09T02:08:11.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.071+0000 7f37e617d700 1 -- 172.21.15.19:0/686133956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e01a9460 con 0x7f37e010e930 2024-05-09T02:08:11.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.071+0000 7f37dcff9700 1 -- 172.21.15.19:0/686133956 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37d8007e30 con 0x7f37e010e930 2024-05-09T02:08:11.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.071+0000 7f37dcff9700 1 -- 172.21.15.19:0/686133956 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d80047d0 con 0x7f37e010e930 2024-05-09T02:08:11.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.071+0000 7f37e617d700 1 -- 172.21.15.19:0/686133956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e01a99a0 con 0x7f37e010e930 2024-05-09T02:08:11.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.072+0000 7f37e617d700 1 -- 172.21.15.19:0/686133956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e00512c0 con 0x7f37e010e930 2024-05-09T02:08:11.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.074+0000 7f37dcff9700 1 -- 172.21.15.19:0/686133956 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f37d8007980 con 0x7f37e010e930 2024-05-09T02:08:11.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.074+0000 7f37dcff9700 1 --2- 172.21.15.19:0/686133956 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f37c80b2880 0x7f37c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:11.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.074+0000 7f37dcff9700 1 -- 172.21.15.19:0/686133956 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f37d800f020 con 0x7f37e010e930 2024-05-09T02:08:11.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.074+0000 7f37df7fe700 1 --2- 172.21.15.19:0/686133956 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f37c80b2880 0x7f37c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:11.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.075+0000 7f37df7fe700 1 --2- 172.21.15.19:0/686133956 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f37c80b2880 0x7f37c80b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f37d0007be0 tx=0x7f37d00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:11.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.078+0000 7f37dcff9700 1 -- 172.21.15.19:0/686133956 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f37d802a020 con 0x7f37e010e930 2024-05-09T02:08:11.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:11.344+0000 7f37e617d700 1 -- 172.21.15.19:0/686133956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f37e002e5f0 con 0x7f37e010e930 2024-05-09T02:08:11.347 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:11 smithi019 ceph-mon[30151]: pgmap v302: 97 pgs: 97 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-05-09T02:08:11.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:11 smithi196 ceph-mon[34718]: pgmap v302: 97 pgs: 97 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-05-09T02:08:11.545 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:08:11 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i[65599]: 2024-05-09T02:08:11.087+0000 7f4635e67700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-09T02:08:11.545 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:08:11 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i[65599]: 2024-05-09T02:08:11.087+0000 7f4635e67700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-09T02:08:11.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:11 smithi059 ceph-mon[34781]: pgmap v302: 97 pgs: 97 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-05-09T02:08:12.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:12.268+0000 7f37dcff9700 1 -- 172.21.15.19:0/686133956 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f37d80ab980 con 0x7f37e010e930 2024-05-09T02:08:12.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:12.272+0000 7f37e617d700 1 -- 172.21.15.19:0/686133956 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f37c80b2880 msgr2=0x7f37c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:12.272+0000 7f37e617d700 1 --2- 172.21.15.19:0/686133956 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f37c80b2880 0x7f37c80b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f37d0007be0 tx=0x7f37d00073d0 comp rx=0 tx=0).stop 2024-05-09T02:08:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:12.272+0000 7f37e617d700 1 -- 172.21.15.19:0/686133956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e010e930 msgr2=0x7f37e0103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:12.272+0000 7f37e617d700 1 --2- 172.21.15.19:0/686133956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e010e930 0x7f37e0103cc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f37d800b830 tx=0x7f37d8004ce0 comp rx=0 tx=0).stop 2024-05-09T02:08:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:12.272+0000 7f37e617d700 1 -- 172.21.15.19:0/686133956 shutdown_connections 2024-05-09T02:08:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:12.272+0000 7f37e617d700 1 --2- 172.21.15.19:0/686133956 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f37c80b2880 0x7f37c80b4d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:12.272+0000 7f37e617d700 1 --2- 172.21.15.19:0/686133956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e010e930 0x7f37e0103cc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:12.272+0000 7f37e617d700 1 --2- 172.21.15.19:0/686133956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37e0105550 0x7f37e0103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:12.272+0000 7f37e617d700 1 --2- 172.21.15.19:0/686133956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e0105ea0 0x7f37e0103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:12.272+0000 7f37e617d700 1 -- 172.21.15.19:0/686133956 >> 172.21.15.19:0/686133956 conn(0x7f37e0078e00 msgr2=0x7f37e0101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:12.272+0000 7f37e617d700 1 -- 172.21.15.19:0/686133956 shutdown_connections 2024-05-09T02:08:12.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:12.272+0000 7f37e617d700 1 -- 172.21.15.19:0/686133956 wait complete. 2024-05-09T02:08:12.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:12 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/686133956' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T02:08:12.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:12 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/686133956' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T02:08:12.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:12 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/686133956' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T02:08:13.037 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-09T02:08:13.037 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-09T02:08:13.296 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:13.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:13 smithi196 ceph-mon[34718]: pgmap v303: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-05-09T02:08:13.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:13 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/686133956' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T02:08:13.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:13 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:08:13.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:13 smithi019 ceph-mon[30151]: pgmap v303: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-05-09T02:08:13.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:13 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/686133956' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T02:08:13.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:13 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:08:13.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:13 smithi059 ceph-mon[34781]: pgmap v303: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-05-09T02:08:13.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:13 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/686133956' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T02:08:13.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:13 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:08:15.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.119+0000 7f1478843700 1 -- 172.21.15.19:0/483056433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14741034e0 msgr2=0x7f14741038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:15.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.119+0000 7f1478843700 1 --2- 172.21.15.19:0/483056433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14741034e0 0x7f14741038f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f14640099c0 tx=0x7f1464009cd0 comp rx=0 tx=0).stop 2024-05-09T02:08:15.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.119+0000 7f1478843700 1 -- 172.21.15.19:0/483056433 shutdown_connections 2024-05-09T02:08:15.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.119+0000 7f1478843700 1 --2- 172.21.15.19:0/483056433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1474108760 0x7f14741100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:15.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.119+0000 7f1478843700 1 --2- 172.21.15.19:0/483056433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14741034e0 0x7f14741038f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:15.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.119+0000 7f1478843700 1 --2- 172.21.15.19:0/483056433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1474107e10 0x7f1474108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:15.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.119+0000 7f1478843700 1 -- 172.21.15.19:0/483056433 >> 172.21.15.19:0/483056433 conn(0x7f14740fef50 msgr2=0x7f1474101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:15.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.119+0000 7f1478843700 1 -- 172.21.15.19:0/483056433 shutdown_connections 2024-05-09T02:08:15.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.119+0000 7f1478843700 1 -- 172.21.15.19:0/483056433 wait complete. 2024-05-09T02:08:15.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.120+0000 7f1478843700 1 Processor -- start 2024-05-09T02:08:15.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.120+0000 7f1478843700 1 -- start start 2024-05-09T02:08:15.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.120+0000 7f1478843700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14741034e0 0x7f147419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:15.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.120+0000 7f1478843700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1474107e10 0x7f147419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:15.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.120+0000 7f1478843700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1474108760 0x7f147419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:15.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.120+0000 7f1478843700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14741144e0 con 0x7f1474107e10 2024-05-09T02:08:15.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.120+0000 7f1478843700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1474114360 con 0x7f14741034e0 2024-05-09T02:08:15.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.120+0000 7f1478843700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1474114660 con 0x7f1474108760 2024-05-09T02:08:15.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.120+0000 7f1471d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1474107e10 0x7f147419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.120+0000 7f1471d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1474107e10 0x7f147419b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46140/0 (socket says 172.21.15.19:46140) 2024-05-09T02:08:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.120+0000 7f1471d9b700 1 -- 172.21.15.19:0/775818525 learned_addr learned my addr 172.21.15.19:0/775818525 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.121+0000 7f147259c700 1 --2- 172.21.15.19:0/775818525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14741034e0 0x7f147419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.121+0000 7f1472d9d700 1 --2- 172.21.15.19:0/775818525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1474108760 0x7f147419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.121+0000 7f1471d9b700 1 -- 172.21.15.19:0/775818525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1474108760 msgr2=0x7f147419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.121+0000 7f1471d9b700 1 --2- 172.21.15.19:0/775818525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1474108760 0x7f147419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.121+0000 7f1471d9b700 1 -- 172.21.15.19:0/775818525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14741034e0 msgr2=0x7f147419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.121+0000 7f1471d9b700 1 --2- 172.21.15.19:0/775818525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14741034e0 0x7f147419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.121+0000 7f1471d9b700 1 -- 172.21.15.19:0/775818525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147419fd00 con 0x7f1474107e10 2024-05-09T02:08:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.121+0000 7f1471d9b700 1 --2- 172.21.15.19:0/775818525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1474107e10 0x7f147419b3d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f146800d980 tx=0x7f146800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.121+0000 7f14637fe700 1 -- 172.21.15.19:0/775818525 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468017070 con 0x7f1474107e10 2024-05-09T02:08:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.121+0000 7f14637fe700 1 -- 172.21.15.19:0/775818525 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f146800fd40 con 0x7f1474107e10 2024-05-09T02:08:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.121+0000 7f1478843700 1 -- 172.21.15.19:0/775818525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147419fff0 con 0x7f1474107e10 2024-05-09T02:08:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.121+0000 7f14637fe700 1 -- 172.21.15.19:0/775818525 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146800e3b0 con 0x7f1474107e10 2024-05-09T02:08:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.121+0000 7f1478843700 1 -- 172.21.15.19:0/775818525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14741a7910 con 0x7f1474107e10 2024-05-09T02:08:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.122+0000 7f1478843700 1 -- 172.21.15.19:0/775818525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14740512c0 con 0x7f1474107e10 2024-05-09T02:08:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.124+0000 7f14637fe700 1 -- 172.21.15.19:0/775818525 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1468010610 con 0x7f1474107e10 2024-05-09T02:08:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.125+0000 7f14637fe700 1 --2- 172.21.15.19:0/775818525 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f145c0b2900 0x7f145c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.125+0000 7f147259c700 1 --2- 172.21.15.19:0/775818525 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f145c0b2900 0x7f145c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.125+0000 7f14637fe700 1 -- 172.21.15.19:0/775818525 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f14680e1a90 con 0x7f1474107e10 2024-05-09T02:08:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.126+0000 7f147259c700 1 --2- 172.21.15.19:0/775818525 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f145c0b2900 0x7f145c0b4e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1464009990 tx=0x7f14640036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.129+0000 7f14637fe700 1 -- 172.21.15.19:0/775818525 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f14680a9980 con 0x7f1474107e10 2024-05-09T02:08:15.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.375+0000 7f1478843700 1 -- 172.21.15.19:0/775818525 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f147419c190 con 0x7f145c0b2900 2024-05-09T02:08:15.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:15 smithi019 ceph-mon[30151]: pgmap v304: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-09T02:08:15.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.403+0000 7f14637fe700 1 -- 172.21.15.19:0/775818525 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f147419c190 con 0x7f145c0b2900 2024-05-09T02:08:15.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.405+0000 7f1478843700 1 -- 172.21.15.19:0/775818525 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f145c0b2900 msgr2=0x7f145c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:15.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.405+0000 7f1478843700 1 --2- 172.21.15.19:0/775818525 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f145c0b2900 0x7f145c0b4e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1464009990 tx=0x7f14640036e0 comp rx=0 tx=0).stop 2024-05-09T02:08:15.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.405+0000 7f1478843700 1 -- 172.21.15.19:0/775818525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1474107e10 msgr2=0x7f147419b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:15.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.405+0000 7f1478843700 1 --2- 172.21.15.19:0/775818525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1474107e10 0x7f147419b3d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f146800d980 tx=0x7f146800dc90 comp rx=0 tx=0).stop 2024-05-09T02:08:15.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.405+0000 7f1478843700 1 -- 172.21.15.19:0/775818525 shutdown_connections 2024-05-09T02:08:15.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.405+0000 7f1478843700 1 --2- 172.21.15.19:0/775818525 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f145c0b2900 0x7f145c0b4e10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:15.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.405+0000 7f1478843700 1 --2- 172.21.15.19:0/775818525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1474108760 0x7f147419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:15.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.405+0000 7f1478843700 1 --2- 172.21.15.19:0/775818525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14741034e0 0x7f147419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:15.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.405+0000 7f1478843700 1 --2- 172.21.15.19:0/775818525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1474107e10 0x7f147419b3d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:15.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.405+0000 7f1478843700 1 -- 172.21.15.19:0/775818525 >> 172.21.15.19:0/775818525 conn(0x7f14740fef50 msgr2=0x7f1474107190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:15.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.405+0000 7f1478843700 1 -- 172.21.15.19:0/775818525 shutdown_connections 2024-05-09T02:08:15.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:15.406+0000 7f1478843700 1 -- 172.21.15.19:0/775818525 wait complete. 2024-05-09T02:08:15.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:15 smithi196 ceph-mon[34718]: pgmap v304: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-09T02:08:15.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:15 smithi059 ceph-mon[34781]: pgmap v304: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-09T02:08:15.788 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:08:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:08:15.380+0000 7f4d2015f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:08:15.788 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:08:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:08:15.380+0000 7f4d2015f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:08:15.788 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:08:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:08:15.380+0000 7f4d2015f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:08:15.788 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:08:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:08:15.380+0000 7f4d2015f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:08:15.788 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:08:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:08:15.380+0000 7f4d2015f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:08:16.248 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-09T02:08:16.511 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:16.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:16 smithi196 ceph-mon[34718]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:08:16.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:16 smithi019 ceph-mon[30151]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:08:16.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:16 smithi059 ceph-mon[34781]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:08:17.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:17 smithi196 ceph-mon[34718]: pgmap v305: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-05-09T02:08:17.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:17 smithi196 ceph-mon[34718]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T02:08:17.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:17 smithi019 ceph-mon[30151]: pgmap v305: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-05-09T02:08:17.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:17 smithi019 ceph-mon[30151]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T02:08:17.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:17 smithi059 ceph-mon[34781]: pgmap v305: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-05-09T02:08:17.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:17 smithi059 ceph-mon[34781]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T02:08:18.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.261+0000 7f486303d700 1 -- 172.21.15.19:0/4040770991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f485c109870 msgr2=0x7f485c1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:18.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.261+0000 7f486303d700 1 --2- 172.21.15.19:0/4040770991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f485c109870 0x7f485c1111d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f485400b0d0 tx=0x7f485400b3e0 comp rx=0 tx=0).stop 2024-05-09T02:08:18.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.261+0000 7f486303d700 1 -- 172.21.15.19:0/4040770991 shutdown_connections 2024-05-09T02:08:18.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.261+0000 7f486303d700 1 --2- 172.21.15.19:0/4040770991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f485c109870 0x7f485c1111d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:18.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.261+0000 7f486303d700 1 --2- 172.21.15.19:0/4040770991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f485c1045f0 0x7f485c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:18.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.261+0000 7f486303d700 1 --2- 172.21.15.19:0/4040770991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f485c108f20 0x7f485c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:18.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.261+0000 7f486303d700 1 -- 172.21.15.19:0/4040770991 >> 172.21.15.19:0/4040770991 conn(0x7f485c100040 msgr2=0x7f485c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:18.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.261+0000 7f486303d700 1 -- 172.21.15.19:0/4040770991 shutdown_connections 2024-05-09T02:08:18.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.261+0000 7f486303d700 1 -- 172.21.15.19:0/4040770991 wait complete. 2024-05-09T02:08:18.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.262+0000 7f486303d700 1 Processor -- start 2024-05-09T02:08:18.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.262+0000 7f486303d700 1 -- start start 2024-05-09T02:08:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.262+0000 7f486303d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f485c1045f0 0x7f485c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.262+0000 7f486303d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f485c108f20 0x7f485c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.262+0000 7f486303d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f485c109870 0x7f485c1a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.262+0000 7f486303d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f485c1144a0 con 0x7f485c1045f0 2024-05-09T02:08:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.262+0000 7f486303d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f485c114320 con 0x7f485c109870 2024-05-09T02:08:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.262+0000 7f486303d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f485c114620 con 0x7f485c108f20 2024-05-09T02:08:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.263+0000 7f4860dd9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f485c1045f0 0x7f485c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.263+0000 7f4860dd9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f485c1045f0 0x7f485c19cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48360/0 (socket says 172.21.15.19:48360) 2024-05-09T02:08:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.263+0000 7f4860dd9700 1 -- 172.21.15.19:0/1067506498 learned_addr learned my addr 172.21.15.19:0/1067506498 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.263+0000 7f485bfff700 1 --2- 172.21.15.19:0/1067506498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f485c108f20 0x7f485c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.263+0000 7f48615da700 1 --2- 172.21.15.19:0/1067506498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f485c109870 0x7f485c1a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.263+0000 7f4860dd9700 1 -- 172.21.15.19:0/1067506498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f485c108f20 msgr2=0x7f485c19d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.263+0000 7f4860dd9700 1 --2- 172.21.15.19:0/1067506498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f485c108f20 0x7f485c19d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:18.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.263+0000 7f4860dd9700 1 -- 172.21.15.19:0/1067506498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f485c109870 msgr2=0x7f485c1a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:18.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.263+0000 7f4860dd9700 1 --2- 172.21.15.19:0/1067506498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f485c109870 0x7f485c1a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:18.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.263+0000 7f4860dd9700 1 -- 172.21.15.19:0/1067506498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485c1a1ea0 con 0x7f485c1045f0 2024-05-09T02:08:18.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.263+0000 7f4860dd9700 1 --2- 172.21.15.19:0/1067506498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f485c1045f0 0x7f485c19cfd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f485000d980 tx=0x7f485000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:18.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.263+0000 7f4859ffb700 1 -- 172.21.15.19:0/1067506498 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850016070 con 0x7f485c1045f0 2024-05-09T02:08:18.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.263+0000 7f486303d700 1 -- 172.21.15.19:0/1067506498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485c1a2190 con 0x7f485c1045f0 2024-05-09T02:08:18.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.263+0000 7f4859ffb700 1 -- 172.21.15.19:0/1067506498 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4850004830 con 0x7f485c1045f0 2024-05-09T02:08:18.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.263+0000 7f4859ffb700 1 -- 172.21.15.19:0/1067506498 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850005540 con 0x7f485c1045f0 2024-05-09T02:08:18.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.264+0000 7f486303d700 1 -- 172.21.15.19:0/1067506498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485c1a9ab0 con 0x7f485c1045f0 2024-05-09T02:08:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.265+0000 7f486303d700 1 -- 172.21.15.19:0/1067506498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f485c0512c0 con 0x7f485c1045f0 2024-05-09T02:08:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.266+0000 7f4859ffb700 1 -- 172.21.15.19:0/1067506498 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f485000f690 con 0x7f485c1045f0 2024-05-09T02:08:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.267+0000 7f4859ffb700 1 --2- 172.21.15.19:0/1067506498 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f484c0b2910 0x7f484c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.267+0000 7f485bfff700 1 --2- 172.21.15.19:0/1067506498 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f484c0b2910 0x7f484c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.267+0000 7f4859ffb700 1 -- 172.21.15.19:0/1067506498 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f48500e1fe0 con 0x7f485c1045f0 2024-05-09T02:08:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.267+0000 7f485bfff700 1 --2- 172.21.15.19:0/1067506498 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f484c0b2910 0x7f484c0b4e20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f485c110430 tx=0x7f4848009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:18.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.272+0000 7f4859ffb700 1 -- 172.21.15.19:0/1067506498 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f485001f020 con 0x7f485c1045f0 2024-05-09T02:08:18.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.521+0000 7f486303d700 1 -- 172.21.15.19:0/1067506498 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f485c107d90 con 0x7f484c0b2910 2024-05-09T02:08:18.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.549+0000 7f4859ffb700 1 -- 172.21.15.19:0/1067506498 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f485c107d90 con 0x7f484c0b2910 2024-05-09T02:08:18.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.551+0000 7f486303d700 1 -- 172.21.15.19:0/1067506498 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f484c0b2910 msgr2=0x7f484c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:18.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.551+0000 7f486303d700 1 --2- 172.21.15.19:0/1067506498 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f484c0b2910 0x7f484c0b4e20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f485c110430 tx=0x7f4848009260 comp rx=0 tx=0).stop 2024-05-09T02:08:18.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.551+0000 7f486303d700 1 -- 172.21.15.19:0/1067506498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f485c1045f0 msgr2=0x7f485c19cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:18.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.551+0000 7f486303d700 1 --2- 172.21.15.19:0/1067506498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f485c1045f0 0x7f485c19cfd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f485000d980 tx=0x7f485000dc90 comp rx=0 tx=0).stop 2024-05-09T02:08:18.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.551+0000 7f486303d700 1 -- 172.21.15.19:0/1067506498 shutdown_connections 2024-05-09T02:08:18.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.551+0000 7f486303d700 1 --2- 172.21.15.19:0/1067506498 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f484c0b2910 0x7f484c0b4e20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:18.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.551+0000 7f486303d700 1 --2- 172.21.15.19:0/1067506498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f485c109870 0x7f485c1a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:18.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.551+0000 7f486303d700 1 --2- 172.21.15.19:0/1067506498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f485c1045f0 0x7f485c19cfd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:18.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.551+0000 7f486303d700 1 --2- 172.21.15.19:0/1067506498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f485c108f20 0x7f485c19d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:18.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.552+0000 7f486303d700 1 -- 172.21.15.19:0/1067506498 >> 172.21.15.19:0/1067506498 conn(0x7f485c100040 msgr2=0x7f485c110ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:18.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.552+0000 7f486303d700 1 -- 172.21.15.19:0/1067506498 shutdown_connections 2024-05-09T02:08:18.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:18.552+0000 7f486303d700 1 -- 172.21.15.19:0/1067506498 wait complete. 2024-05-09T02:08:19.271 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-09T02:08:19.533 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:19.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:19 smithi196 ceph-mon[34718]: pgmap v306: 97 pgs: 97 active+clean; 585 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 19 op/s 2024-05-09T02:08:19.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:19 smithi019 ceph-mon[30151]: pgmap v306: 97 pgs: 97 active+clean; 585 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 19 op/s 2024-05-09T02:08:19.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:19 smithi059 ceph-mon[34781]: pgmap v306: 97 pgs: 97 active+clean; 585 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 19 op/s 2024-05-09T02:08:20.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:20 smithi196 ceph-mon[34718]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:08:20.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:20 smithi019 ceph-mon[30151]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:08:20.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:20 smithi059 ceph-mon[34781]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:08:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.306+0000 7f5fef525700 1 -- 172.21.15.19:0/2989215189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe8107df0 msgr2=0x7f5fe8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.306+0000 7f5fef525700 1 --2- 172.21.15.19:0/2989215189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe8107df0 0x7f5fe8108200 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80099c0 tx=0x7f5fd8009cd0 comp rx=0 tx=0).stop 2024-05-09T02:08:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.306+0000 7f5fef525700 1 -- 172.21.15.19:0/2989215189 shutdown_connections 2024-05-09T02:08:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.306+0000 7f5fef525700 1 --2- 172.21.15.19:0/2989215189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe8108740 0x7f5fe81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.306+0000 7f5fef525700 1 --2- 172.21.15.19:0/2989215189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fe81034c0 0x7f5fe81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.306+0000 7f5fef525700 1 --2- 172.21.15.19:0/2989215189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe8107df0 0x7f5fe8108200 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.306+0000 7f5fef525700 1 -- 172.21.15.19:0/2989215189 >> 172.21.15.19:0/2989215189 conn(0x7f5fe80fedf0 msgr2=0x7f5fe8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.306+0000 7f5fef525700 1 -- 172.21.15.19:0/2989215189 shutdown_connections 2024-05-09T02:08:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.306+0000 7f5fef525700 1 -- 172.21.15.19:0/2989215189 wait complete. 2024-05-09T02:08:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.307+0000 7f5fef525700 1 Processor -- start 2024-05-09T02:08:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.307+0000 7f5fef525700 1 -- start start 2024-05-09T02:08:21.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.307+0000 7f5fef525700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe81034c0 0x7f5fe819bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.307+0000 7f5fef525700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fe8107df0 0x7f5fe819c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.307+0000 7f5fef525700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe8108740 0x7f5fe819c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.307+0000 7f5fef525700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81133a0 con 0x7f5fe81034c0 2024-05-09T02:08:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.307+0000 7f5fef525700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe8113220 con 0x7f5fe8108740 2024-05-09T02:08:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.307+0000 7f5fef525700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe8113520 con 0x7f5fe8107df0 2024-05-09T02:08:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.307+0000 7f5fed2c1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe81034c0 0x7f5fe819bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.307+0000 7f5fed2c1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe81034c0 0x7f5fe819bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48378/0 (socket says 172.21.15.19:48378) 2024-05-09T02:08:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.307+0000 7f5fed2c1700 1 -- 172.21.15.19:0/4138991868 learned_addr learned my addr 172.21.15.19:0/4138991868 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.308+0000 7f5fedac2700 1 --2- 172.21.15.19:0/4138991868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe8108740 0x7f5fe819c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.308+0000 7f5fecac0700 1 --2- 172.21.15.19:0/4138991868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fe8107df0 0x7f5fe819c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.308+0000 7f5fed2c1700 1 -- 172.21.15.19:0/4138991868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fe8107df0 msgr2=0x7f5fe819c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.308+0000 7f5fed2c1700 1 --2- 172.21.15.19:0/4138991868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fe8107df0 0x7f5fe819c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.308+0000 7f5fed2c1700 1 -- 172.21.15.19:0/4138991868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe8108740 msgr2=0x7f5fe819c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:21.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.308+0000 7f5fed2c1700 1 --2- 172.21.15.19:0/4138991868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe8108740 0x7f5fe819c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:21.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.308+0000 7f5fed2c1700 1 -- 172.21.15.19:0/4138991868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe81a7030 con 0x7f5fe81034c0 2024-05-09T02:08:21.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.308+0000 7f5fedac2700 1 --2- 172.21.15.19:0/4138991868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe8108740 0x7f5fe819c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:08:21.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.308+0000 7f5fed2c1700 1 --2- 172.21.15.19:0/4138991868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe81034c0 0x7f5fe819bce0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c920 tx=0x7f5fe400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:21.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.308+0000 7f5fde7fc700 1 -- 172.21.15.19:0/4138991868 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4007b00 con 0x7f5fe81034c0 2024-05-09T02:08:21.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.308+0000 7f5fde7fc700 1 -- 172.21.15.19:0/4138991868 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe4004ce0 con 0x7f5fe81034c0 2024-05-09T02:08:21.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.308+0000 7f5fef525700 1 -- 172.21.15.19:0/4138991868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe81a7320 con 0x7f5fe81034c0 2024-05-09T02:08:21.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.308+0000 7f5fde7fc700 1 -- 172.21.15.19:0/4138991868 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4005650 con 0x7f5fe81034c0 2024-05-09T02:08:21.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.308+0000 7f5fef525700 1 -- 172.21.15.19:0/4138991868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe81a7830 con 0x7f5fe81034c0 2024-05-09T02:08:21.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.309+0000 7f5fef525700 1 -- 172.21.15.19:0/4138991868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe8104970 con 0x7f5fe81034c0 2024-05-09T02:08:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.311+0000 7f5fde7fc700 1 -- 172.21.15.19:0/4138991868 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5fe4004750 con 0x7f5fe81034c0 2024-05-09T02:08:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.312+0000 7f5fde7fc700 1 --2- 172.21.15.19:0/4138991868 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5fd40b2910 0x7f5fd40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.312+0000 7f5fecac0700 1 --2- 172.21.15.19:0/4138991868 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5fd40b2910 0x7f5fd40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.312+0000 7f5fde7fc700 1 -- 172.21.15.19:0/4138991868 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f5fe40e22c0 con 0x7f5fe81034c0 2024-05-09T02:08:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.312+0000 7f5fecac0700 1 --2- 172.21.15.19:0/4138991868 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5fd40b2910 0x7f5fd40b4e20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8000c00 tx=0x7f5fd801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:21.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.316+0000 7f5fde7fc700 1 -- 172.21.15.19:0/4138991868 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5fe4014030 con 0x7f5fe81034c0 2024-05-09T02:08:21.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:21 smithi196 ceph-mon[34718]: pgmap v307: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.7 KiB/s wr, 21 op/s 2024-05-09T02:08:21.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.564+0000 7f5fef525700 1 -- 172.21.15.19:0/4138991868 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f5fe8063c00 con 0x7f5fd40b2910 2024-05-09T02:08:21.567 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:21 smithi019 ceph-mon[30151]: pgmap v307: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.7 KiB/s wr, 21 op/s 2024-05-09T02:08:21.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.570+0000 7f5fde7fc700 1 -- 172.21.15.19:0/4138991868 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5fe8063c00 con 0x7f5fd40b2910 2024-05-09T02:08:21.572 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_0/1be20f3a-d8ea-469c-b694-a0c7ea2c21c2 2024-05-09T02:08:21.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.573+0000 7f5fef525700 1 -- 172.21.15.19:0/4138991868 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5fd40b2910 msgr2=0x7f5fd40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:21.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.573+0000 7f5fef525700 1 --2- 172.21.15.19:0/4138991868 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5fd40b2910 0x7f5fd40b4e20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8000c00 tx=0x7f5fd801a040 comp rx=0 tx=0).stop 2024-05-09T02:08:21.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.573+0000 7f5fef525700 1 -- 172.21.15.19:0/4138991868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe81034c0 msgr2=0x7f5fe819bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:21.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.573+0000 7f5fef525700 1 --2- 172.21.15.19:0/4138991868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe81034c0 0x7f5fe819bce0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c920 tx=0x7f5fe400cce0 comp rx=0 tx=0).stop 2024-05-09T02:08:21.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.573+0000 7f5fef525700 1 -- 172.21.15.19:0/4138991868 shutdown_connections 2024-05-09T02:08:21.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.573+0000 7f5fef525700 1 --2- 172.21.15.19:0/4138991868 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5fd40b2910 0x7f5fd40b4e20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:21.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.573+0000 7f5fef525700 1 --2- 172.21.15.19:0/4138991868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe8108740 0x7f5fe819c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:21.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.573+0000 7f5fef525700 1 --2- 172.21.15.19:0/4138991868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe81034c0 0x7f5fe819bce0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:21.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.573+0000 7f5fef525700 1 --2- 172.21.15.19:0/4138991868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fe8107df0 0x7f5fe819c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:21.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.573+0000 7f5fef525700 1 -- 172.21.15.19:0/4138991868 >> 172.21.15.19:0/4138991868 conn(0x7f5fe80fedf0 msgr2=0x7f5fe81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:21.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.573+0000 7f5fef525700 1 -- 172.21.15.19:0/4138991868 shutdown_connections 2024-05-09T02:08:21.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:21.573+0000 7f5fef525700 1 -- 172.21.15.19:0/4138991868 wait complete. 2024-05-09T02:08:21.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:21 smithi059 ceph-mon[34781]: pgmap v307: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.7 KiB/s wr, 21 op/s 2024-05-09T02:08:22.267 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-09T02:08:22.522 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:23.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:23 smithi019 ceph-mon[30151]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:08:23.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:23 smithi019 ceph-mon[30151]: pgmap v308: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.7 KiB/s wr, 16 op/s 2024-05-09T02:08:23.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:23 smithi059 ceph-mon[34781]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:08:23.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:23 smithi059 ceph-mon[34781]: pgmap v308: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.7 KiB/s wr, 16 op/s 2024-05-09T02:08:23.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:23 smithi196 ceph-mon[34718]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:08:23.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:23 smithi196 ceph-mon[34718]: pgmap v308: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.7 KiB/s wr, 16 op/s 2024-05-09T02:08:24.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.234+0000 7f3369e4b700 1 -- 172.21.15.19:0/2310049787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3364107df0 msgr2=0x7f3364108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.234+0000 7f3369e4b700 1 --2- 172.21.15.19:0/2310049787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3364107df0 0x7f3364108200 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3358009a40 tx=0x7f3358009d50 comp rx=0 tx=0).stop 2024-05-09T02:08:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.234+0000 7f3369e4b700 1 -- 172.21.15.19:0/2310049787 shutdown_connections 2024-05-09T02:08:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.234+0000 7f3369e4b700 1 --2- 172.21.15.19:0/2310049787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3364108740 0x7f33641100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.234+0000 7f3369e4b700 1 --2- 172.21.15.19:0/2310049787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33641034c0 0x7f33641038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.234+0000 7f3369e4b700 1 --2- 172.21.15.19:0/2310049787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3364107df0 0x7f3364108200 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.234+0000 7f3369e4b700 1 -- 172.21.15.19:0/2310049787 >> 172.21.15.19:0/2310049787 conn(0x7f33640fedf0 msgr2=0x7f3364101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.234+0000 7f3369e4b700 1 -- 172.21.15.19:0/2310049787 shutdown_connections 2024-05-09T02:08:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.234+0000 7f3369e4b700 1 -- 172.21.15.19:0/2310049787 wait complete. 2024-05-09T02:08:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.235+0000 7f3369e4b700 1 Processor -- start 2024-05-09T02:08:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.235+0000 7f3369e4b700 1 -- start start 2024-05-09T02:08:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.235+0000 7f3369e4b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33641034c0 0x7f336419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.235+0000 7f3369e4b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3364107df0 0x7f336419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.235+0000 7f3369e4b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3364108740 0x7f3364198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.235+0000 7f3369e4b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33641122d0 con 0x7f3364107df0 2024-05-09T02:08:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.235+0000 7f3369e4b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3364112150 con 0x7f33641034c0 2024-05-09T02:08:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.235+0000 7f3369e4b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3364112450 con 0x7f3364108740 2024-05-09T02:08:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.235+0000 7f3362ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3364107df0 0x7f336419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.235+0000 7f3362ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3364107df0 0x7f336419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48386/0 (socket says 172.21.15.19:48386) 2024-05-09T02:08:24.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.235+0000 7f3362ffd700 1 -- 172.21.15.19:0/555388776 learned_addr learned my addr 172.21.15.19:0/555388776 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:24.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.235+0000 7f33637fe700 1 --2- 172.21.15.19:0/555388776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33641034c0 0x7f336419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:24.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.236+0000 7f3363fff700 1 --2- 172.21.15.19:0/555388776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3364108740 0x7f3364198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:24.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.236+0000 7f3362ffd700 1 -- 172.21.15.19:0/555388776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3364108740 msgr2=0x7f3364198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:24.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.236+0000 7f3362ffd700 1 --2- 172.21.15.19:0/555388776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3364108740 0x7f3364198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:24.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.236+0000 7f3362ffd700 1 -- 172.21.15.19:0/555388776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33641034c0 msgr2=0x7f336419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:24.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.236+0000 7f3362ffd700 1 --2- 172.21.15.19:0/555388776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33641034c0 0x7f336419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:24.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.236+0000 7f3362ffd700 1 -- 172.21.15.19:0/555388776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33641990e0 con 0x7f3364107df0 2024-05-09T02:08:24.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.236+0000 7f3363fff700 1 --2- 172.21.15.19:0/555388776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3364108740 0x7f3364198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:08:24.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.236+0000 7f3362ffd700 1 --2- 172.21.15.19:0/555388776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3364107df0 0x7f336419ed50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3358018040 tx=0x7f3358003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:24.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.236+0000 7f3360ff9700 1 -- 172.21.15.19:0/555388776 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358024070 con 0x7f3364107df0 2024-05-09T02:08:24.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.236+0000 7f3369e4b700 1 -- 172.21.15.19:0/555388776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3364199370 con 0x7f3364107df0 2024-05-09T02:08:24.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.236+0000 7f3360ff9700 1 -- 172.21.15.19:0/555388776 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33580058e0 con 0x7f3364107df0 2024-05-09T02:08:24.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.236+0000 7f3360ff9700 1 -- 172.21.15.19:0/555388776 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358016430 con 0x7f3364107df0 2024-05-09T02:08:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.236+0000 7f3369e4b700 1 -- 172.21.15.19:0/555388776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33641a56a0 con 0x7f3364107df0 2024-05-09T02:08:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.238+0000 7f3369e4b700 1 -- 172.21.15.19:0/555388776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33640512c0 con 0x7f3364107df0 2024-05-09T02:08:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.240+0000 7f3360ff9700 1 -- 172.21.15.19:0/555388776 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3358027070 con 0x7f3364107df0 2024-05-09T02:08:24.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.241+0000 7f3360ff9700 1 --2- 172.21.15.19:0/555388776 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f334c0b2910 0x7f334c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:24.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.241+0000 7f33637fe700 1 --2- 172.21.15.19:0/555388776 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f334c0b2910 0x7f334c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:24.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.241+0000 7f3360ff9700 1 -- 172.21.15.19:0/555388776 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f33580e5630 con 0x7f3364107df0 2024-05-09T02:08:24.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.242+0000 7f33637fe700 1 --2- 172.21.15.19:0/555388776 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f334c0b2910 0x7f334c0b4e20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f33540060b0 tx=0x7f335400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.245+0000 7f3360ff9700 1 -- 172.21.15.19:0/555388776 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f33580ad440 con 0x7f3364107df0 2024-05-09T02:08:24.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:24.503+0000 7f3369e4b700 1 -- 172.21.15.19:0/555388776 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3364199f60 con 0x7f334c0b2910 2024-05-09T02:08:24.505 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-09T02:08:24.505 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-09T02:08:24.505 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-09T02:08:24.505 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-05-09T02:08:24.506 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-09T02:08:24.506 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-05-09T02:08:24.506 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-05-09T02:08:24.506 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-05-09T02:08:24.506 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-05-09T02:08:24.506 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:24 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-09T02:08:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-09T02:08:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-09T02:08:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-09T02:08:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-05-09T02:08:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-09T02:08:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-05-09T02:08:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-05-09T02:08:24.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-05-09T02:08:24.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-05-09T02:08:24.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:24 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-09T02:08:24.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-09T02:08:24.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-09T02:08:24.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-09T02:08:24.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-05-09T02:08:24.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-09T02:08:24.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-05-09T02:08:24.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-05-09T02:08:24.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-05-09T02:08:24.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-05-09T02:08:24.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:24 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-09T02:08:25.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:25.329+0000 7f3360ff9700 1 -- 172.21.15.19:0/555388776 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3364199f60 con 0x7f334c0b2910 2024-05-09T02:08:25.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:25.332+0000 7f3369e4b700 1 -- 172.21.15.19:0/555388776 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f334c0b2910 msgr2=0x7f334c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:25.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:25.332+0000 7f3369e4b700 1 --2- 172.21.15.19:0/555388776 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f334c0b2910 0x7f334c0b4e20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f33540060b0 tx=0x7f335400c040 comp rx=0 tx=0).stop 2024-05-09T02:08:25.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:25.332+0000 7f3369e4b700 1 -- 172.21.15.19:0/555388776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3364107df0 msgr2=0x7f336419ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:25.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:25.332+0000 7f3369e4b700 1 --2- 172.21.15.19:0/555388776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3364107df0 0x7f336419ed50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3358018040 tx=0x7f3358003a90 comp rx=0 tx=0).stop 2024-05-09T02:08:25.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:25.332+0000 7f3369e4b700 1 -- 172.21.15.19:0/555388776 shutdown_connections 2024-05-09T02:08:25.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:25.332+0000 7f3369e4b700 1 --2- 172.21.15.19:0/555388776 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f334c0b2910 0x7f334c0b4e20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:25.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:25.332+0000 7f3369e4b700 1 --2- 172.21.15.19:0/555388776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3364108740 0x7f3364198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:25.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:25.332+0000 7f3369e4b700 1 --2- 172.21.15.19:0/555388776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33641034c0 0x7f336419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:25.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:25.332+0000 7f3369e4b700 1 --2- 172.21.15.19:0/555388776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3364107df0 0x7f336419ed50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:25.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:25.332+0000 7f3369e4b700 1 -- 172.21.15.19:0/555388776 >> 172.21.15.19:0/555388776 conn(0x7f33640fedf0 msgr2=0x7f3364101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:25.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:25.333+0000 7f3369e4b700 1 -- 172.21.15.19:0/555388776 shutdown_connections 2024-05-09T02:08:25.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:25.333+0000 7f3369e4b700 1 -- 172.21.15.19:0/555388776 wait complete. 2024-05-09T02:08:25.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:25 smithi019 ceph-mon[30151]: pgmap v309: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-05-09T02:08:25.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-09T02:08:25.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-09T02:08:25.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-09T02:08:25.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-05-09T02:08:25.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-09T02:08:25.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-05-09T02:08:25.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-05-09T02:08:25.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-05-09T02:08:25.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-05-09T02:08:25.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:25 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-09T02:08:25.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:25 smithi019 ceph-mon[30151]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T02:08:25.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:25 smithi059 ceph-mon[34781]: pgmap v309: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-05-09T02:08:25.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-09T02:08:25.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-09T02:08:25.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-09T02:08:25.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-05-09T02:08:25.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-09T02:08:25.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-05-09T02:08:25.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-05-09T02:08:25.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-05-09T02:08:25.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-05-09T02:08:25.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:25 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-09T02:08:25.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:25 smithi059 ceph-mon[34781]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T02:08:25.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:25 smithi196 ceph-mon[34718]: pgmap v309: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-05-09T02:08:25.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-09T02:08:25.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-09T02:08:25.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-09T02:08:25.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-05-09T02:08:25.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-09T02:08:25.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-05-09T02:08:25.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-05-09T02:08:25.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-05-09T02:08:25.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-05-09T02:08:25.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:25 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-09T02:08:25.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:25 smithi196 ceph-mon[34718]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T02:08:26.048 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-09T02:08:26.309 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:26.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:26 smithi019 ceph-mon[30151]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:08:26.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:26 smithi019 ceph-mon[30151]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T02:08:26.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:26 smithi059 ceph-mon[34781]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:08:26.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:26 smithi059 ceph-mon[34781]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T02:08:26.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:26 smithi196 ceph-mon[34718]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:08:26.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:26 smithi196 ceph-mon[34718]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T02:08:27.446 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:27 smithi019 ceph-mon[30151]: pgmap v312: 97 pgs: 6 peering, 91 active+clean; 625 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-05-09T02:08:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:27 smithi059 ceph-mon[34781]: pgmap v312: 97 pgs: 6 peering, 91 active+clean; 625 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-05-09T02:08:27.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:27 smithi196 ceph-mon[34718]: pgmap v312: 97 pgs: 6 peering, 91 active+clean; 625 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-05-09T02:08:28.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.038+0000 7f627d376700 1 -- 172.21.15.19:0/2739029648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6278107df0 msgr2=0x7f6278108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:28.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.038+0000 7f627d376700 1 --2- 172.21.15.19:0/2739029648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6278107df0 0x7f6278108200 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f626c0099c0 tx=0x7f626c009cd0 comp rx=0 tx=0).stop 2024-05-09T02:08:28.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.038+0000 7f627d376700 1 -- 172.21.15.19:0/2739029648 shutdown_connections 2024-05-09T02:08:28.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.038+0000 7f627d376700 1 --2- 172.21.15.19:0/2739029648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6278108740 0x7f62781100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:28.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.038+0000 7f627d376700 1 --2- 172.21.15.19:0/2739029648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62781034c0 0x7f62781038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:28.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.038+0000 7f627d376700 1 --2- 172.21.15.19:0/2739029648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6278107df0 0x7f6278108200 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.038+0000 7f627d376700 1 -- 172.21.15.19:0/2739029648 >> 172.21.15.19:0/2739029648 conn(0x7f62780fedc0 msgr2=0x7f62781011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.038+0000 7f627d376700 1 -- 172.21.15.19:0/2739029648 shutdown_connections 2024-05-09T02:08:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.039+0000 7f627d376700 1 -- 172.21.15.19:0/2739029648 wait complete. 2024-05-09T02:08:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.039+0000 7f627d376700 1 Processor -- start 2024-05-09T02:08:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.040+0000 7f627d376700 1 -- start start 2024-05-09T02:08:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.040+0000 7f627d376700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62781034c0 0x7f627819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.040+0000 7f627d376700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6278107df0 0x7f627819b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.040+0000 7f627d376700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6278108740 0x7f627819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.040+0000 7f627d376700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6278113460 con 0x7f6278108740 2024-05-09T02:08:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.040+0000 7f627d376700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f62781132e0 con 0x7f6278107df0 2024-05-09T02:08:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.040+0000 7f627d376700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62781135e0 con 0x7f62781034c0 2024-05-09T02:08:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.040+0000 7f62777fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6278108740 0x7f627819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.040+0000 7f62777fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6278108740 0x7f627819f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37348/0 (socket says 172.21.15.19:37348) 2024-05-09T02:08:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.040+0000 7f62777fe700 1 -- 172.21.15.19:0/740256118 learned_addr learned my addr 172.21.15.19:0/740256118 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.040+0000 7f6276ffd700 1 --2- 172.21.15.19:0/740256118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62781034c0 0x7f627819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.040+0000 7f62767fc700 1 --2- 172.21.15.19:0/740256118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6278107df0 0x7f627819b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.041+0000 7f62777fe700 1 -- 172.21.15.19:0/740256118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62781034c0 msgr2=0x7f627819afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.041+0000 7f62777fe700 1 --2- 172.21.15.19:0/740256118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62781034c0 0x7f627819afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.041+0000 7f62777fe700 1 -- 172.21.15.19:0/740256118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6278107df0 msgr2=0x7f627819b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.041+0000 7f62777fe700 1 --2- 172.21.15.19:0/740256118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6278107df0 0x7f627819b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.041+0000 7f62777fe700 1 -- 172.21.15.19:0/740256118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627819fe70 con 0x7f6278108740 2024-05-09T02:08:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.041+0000 7f6276ffd700 1 --2- 172.21.15.19:0/740256118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62781034c0 0x7f627819afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:08:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.041+0000 7f62777fe700 1 --2- 172.21.15.19:0/740256118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6278108740 0x7f627819f7d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f627000b4a0 tx=0x7f627000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.041+0000 7f625ffff700 1 -- 172.21.15.19:0/740256118 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270015020 con 0x7f6278108740 2024-05-09T02:08:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.041+0000 7f627d376700 1 -- 172.21.15.19:0/740256118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62781a0160 con 0x7f6278108740 2024-05-09T02:08:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.041+0000 7f625ffff700 1 -- 172.21.15.19:0/740256118 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6270011ce0 con 0x7f6278108740 2024-05-09T02:08:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.041+0000 7f625ffff700 1 -- 172.21.15.19:0/740256118 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270010cb0 con 0x7f6278108740 2024-05-09T02:08:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.041+0000 7f627d376700 1 -- 172.21.15.19:0/740256118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62781a7a80 con 0x7f6278108740 2024-05-09T02:08:28.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.042+0000 7f627d376700 1 -- 172.21.15.19:0/740256118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62780512c0 con 0x7f6278108740 2024-05-09T02:08:28.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.044+0000 7f625ffff700 1 -- 172.21.15.19:0/740256118 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6270018020 con 0x7f6278108740 2024-05-09T02:08:28.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.045+0000 7f625ffff700 1 --2- 172.21.15.19:0/740256118 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f62600b2910 0x7f62600b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:28.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.045+0000 7f6276ffd700 1 --2- 172.21.15.19:0/740256118 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f62600b2910 0x7f62600b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:28.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.045+0000 7f625ffff700 1 -- 172.21.15.19:0/740256118 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f62700e2120 con 0x7f6278108740 2024-05-09T02:08:28.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.045+0000 7f6276ffd700 1 --2- 172.21.15.19:0/740256118 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f62600b2910 0x7f62600b4e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6268009fa0 tx=0x7f6268009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:28.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.048+0000 7f625ffff700 1 -- 172.21.15.19:0/740256118 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f627001f020 con 0x7f6278108740 2024-05-09T02:08:28.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.301+0000 7f627d376700 1 -- 172.21.15.19:0/740256118 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f627819c300 con 0x7f62600b2910 2024-05-09T02:08:28.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.307+0000 7f625ffff700 1 -- 172.21.15.19:0/740256118 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f627819c300 con 0x7f62600b2910 2024-05-09T02:08:28.309 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_1/3263a7e6-90e1-4c3c-b293-25c3c473ace0 2024-05-09T02:08:28.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.309+0000 7f627d376700 1 -- 172.21.15.19:0/740256118 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f62600b2910 msgr2=0x7f62600b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:28.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.309+0000 7f627d376700 1 --2- 172.21.15.19:0/740256118 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f62600b2910 0x7f62600b4e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6268009fa0 tx=0x7f6268009570 comp rx=0 tx=0).stop 2024-05-09T02:08:28.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.309+0000 7f627d376700 1 -- 172.21.15.19:0/740256118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6278108740 msgr2=0x7f627819f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:28.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.309+0000 7f627d376700 1 --2- 172.21.15.19:0/740256118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6278108740 0x7f627819f7d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f627000b4a0 tx=0x7f627000b860 comp rx=0 tx=0).stop 2024-05-09T02:08:28.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.309+0000 7f627d376700 1 -- 172.21.15.19:0/740256118 shutdown_connections 2024-05-09T02:08:28.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.309+0000 7f627d376700 1 --2- 172.21.15.19:0/740256118 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f62600b2910 0x7f62600b4e20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:28.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.309+0000 7f627d376700 1 --2- 172.21.15.19:0/740256118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6278108740 0x7f627819f7d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:28.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.309+0000 7f627d376700 1 --2- 172.21.15.19:0/740256118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62781034c0 0x7f627819afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:28.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.309+0000 7f627d376700 1 --2- 172.21.15.19:0/740256118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6278107df0 0x7f627819b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:28.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.309+0000 7f627d376700 1 -- 172.21.15.19:0/740256118 >> 172.21.15.19:0/740256118 conn(0x7f62780fedc0 msgr2=0x7f627806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:28.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.309+0000 7f627d376700 1 -- 172.21.15.19:0/740256118 shutdown_connections 2024-05-09T02:08:28.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:28.309+0000 7f627d376700 1 -- 172.21.15.19:0/740256118 wait complete. 2024-05-09T02:08:28.967 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:08:29.220 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:29.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:29 smithi019 ceph-mon[30151]: pgmap v313: 97 pgs: 6 peering, 91 active+clean; 625 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.9 KiB/s wr, 9 op/s; 13 B/s, 0 objects/s recovering 2024-05-09T02:08:29.626 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:29 smithi019 ceph-mon[30151]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:08:29.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:29 smithi059 ceph-mon[34781]: pgmap v313: 97 pgs: 6 peering, 91 active+clean; 625 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.9 KiB/s wr, 9 op/s; 13 B/s, 0 objects/s recovering 2024-05-09T02:08:29.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:29 smithi059 ceph-mon[34781]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:08:29.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:29 smithi196 ceph-mon[34718]: pgmap v313: 97 pgs: 6 peering, 91 active+clean; 625 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.9 KiB/s wr, 9 op/s; 13 B/s, 0 objects/s recovering 2024-05-09T02:08:29.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:29 smithi196 ceph-mon[34718]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:08:30.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.966+0000 7f76f911a700 1 -- 172.21.15.19:0/1500903205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4108740 msgr2=0x7f76f41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:30.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.966+0000 7f76f911a700 1 --2- 172.21.15.19:0/1500903205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4108740 0x7f76f41100a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00b0d0 tx=0x7f76ec00b3e0 comp rx=0 tx=0).stop 2024-05-09T02:08:30.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.967+0000 7f76f911a700 1 -- 172.21.15.19:0/1500903205 shutdown_connections 2024-05-09T02:08:30.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.967+0000 7f76f911a700 1 --2- 172.21.15.19:0/1500903205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4108740 0x7f76f41100a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:30.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.967+0000 7f76f911a700 1 --2- 172.21.15.19:0/1500903205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76f41034c0 0x7f76f41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:30.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.967+0000 7f76f911a700 1 --2- 172.21.15.19:0/1500903205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f4107df0 0x7f76f4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:30.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.967+0000 7f76f911a700 1 -- 172.21.15.19:0/1500903205 >> 172.21.15.19:0/1500903205 conn(0x7f76f40fedc0 msgr2=0x7f76f41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:30.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.967+0000 7f76f911a700 1 -- 172.21.15.19:0/1500903205 shutdown_connections 2024-05-09T02:08:30.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.967+0000 7f76f911a700 1 -- 172.21.15.19:0/1500903205 wait complete. 2024-05-09T02:08:30.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.968+0000 7f76f911a700 1 Processor -- start 2024-05-09T02:08:30.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.968+0000 7f76f911a700 1 -- start start 2024-05-09T02:08:30.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.968+0000 7f76f911a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f41034c0 0x7f76f4199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:30.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.968+0000 7f76f911a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4107df0 0x7f76f419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:30.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.968+0000 7f76f911a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76f4108740 0x7f76f419e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:30.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.968+0000 7f76f911a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41133e0 con 0x7f76f4107df0 2024-05-09T02:08:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.968+0000 7f76f911a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4113260 con 0x7f76f4108740 2024-05-09T02:08:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.968+0000 7f76f911a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4113560 con 0x7f76f41034c0 2024-05-09T02:08:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.968+0000 7f76f259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4107df0 0x7f76f419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.968+0000 7f76f259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4107df0 0x7f76f419a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37360/0 (socket says 172.21.15.19:37360) 2024-05-09T02:08:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.968+0000 7f76f259c700 1 -- 172.21.15.19:0/566830618 learned_addr learned my addr 172.21.15.19:0/566830618 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.968+0000 7f76f359e700 1 --2- 172.21.15.19:0/566830618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76f4108740 0x7f76f419e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.969+0000 7f76f2d9d700 1 --2- 172.21.15.19:0/566830618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f41034c0 0x7f76f4199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.969+0000 7f76f259c700 1 -- 172.21.15.19:0/566830618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f41034c0 msgr2=0x7f76f4199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.969+0000 7f76f259c700 1 --2- 172.21.15.19:0/566830618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f41034c0 0x7f76f4199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.969+0000 7f76f259c700 1 -- 172.21.15.19:0/566830618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76f4108740 msgr2=0x7f76f419e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.969+0000 7f76f259c700 1 --2- 172.21.15.19:0/566830618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76f4108740 0x7f76f419e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.969+0000 7f76f259c700 1 -- 172.21.15.19:0/566830618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f419eb50 con 0x7f76f4107df0 2024-05-09T02:08:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.969+0000 7f76f259c700 1 --2- 172.21.15.19:0/566830618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4107df0 0x7f76f419a2d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f76e800b500 tx=0x7f76e800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.969+0000 7f76dbfff700 1 -- 172.21.15.19:0/566830618 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8015020 con 0x7f76f4107df0 2024-05-09T02:08:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.969+0000 7f76dbfff700 1 -- 172.21.15.19:0/566830618 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76e8011ce0 con 0x7f76f4107df0 2024-05-09T02:08:30.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.969+0000 7f76dbfff700 1 -- 172.21.15.19:0/566830618 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8010cb0 con 0x7f76f4107df0 2024-05-09T02:08:30.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.969+0000 7f76f911a700 1 -- 172.21.15.19:0/566830618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f419ee40 con 0x7f76f4107df0 2024-05-09T02:08:30.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.969+0000 7f76f911a700 1 -- 172.21.15.19:0/566830618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f419f330 con 0x7f76f4107df0 2024-05-09T02:08:30.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.970+0000 7f76f911a700 1 -- 172.21.15.19:0/566830618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f40512c0 con 0x7f76f4107df0 2024-05-09T02:08:30.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.972+0000 7f76dbfff700 1 -- 172.21.15.19:0/566830618 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f76e8018020 con 0x7f76f4107df0 2024-05-09T02:08:30.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.973+0000 7f76dbfff700 1 --2- 172.21.15.19:0/566830618 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f76dc0b2910 0x7f76dc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:30.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.973+0000 7f76f2d9d700 1 --2- 172.21.15.19:0/566830618 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f76dc0b2910 0x7f76dc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:30.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.973+0000 7f76dbfff700 1 -- 172.21.15.19:0/566830618 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f76e80e1ee0 con 0x7f76f4107df0 2024-05-09T02:08:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.973+0000 7f76f2d9d700 1 --2- 172.21.15.19:0/566830618 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f76dc0b2910 0x7f76dc0b4e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f76e400ac50 tx=0x7f76e400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:30.977+0000 7f76dbfff700 1 -- 172.21.15.19:0/566830618 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f76e801f020 con 0x7f76f4107df0 2024-05-09T02:08:31.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:31.239+0000 7f76f911a700 1 -- 172.21.15.19:0/566830618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76f419b090 con 0x7f76f4107df0 2024-05-09T02:08:31.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:31.240+0000 7f76dbfff700 1 -- 172.21.15.19:0/566830618 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11917 (secure 0 0 0) 0x7f76e80a9b90 con 0x7f76f4107df0 2024-05-09T02:08:31.243 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:08:31.243 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:08:11.346147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24455,24457,24473,24484,24463]},"id":1}]} 2024-05-09T02:08:31.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:31.242+0000 7f76f911a700 1 -- 172.21.15.19:0/566830618 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f76dc0b2910 msgr2=0x7f76dc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:31.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:31.242+0000 7f76f911a700 1 --2- 172.21.15.19:0/566830618 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f76dc0b2910 0x7f76dc0b4e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f76e400ac50 tx=0x7f76e400a380 comp rx=0 tx=0).stop 2024-05-09T02:08:31.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:31.242+0000 7f76f911a700 1 -- 172.21.15.19:0/566830618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4107df0 msgr2=0x7f76f419a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:31.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:31.242+0000 7f76f911a700 1 --2- 172.21.15.19:0/566830618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4107df0 0x7f76f419a2d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f76e800b500 tx=0x7f76e800b810 comp rx=0 tx=0).stop 2024-05-09T02:08:31.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:31.242+0000 7f76f911a700 1 -- 172.21.15.19:0/566830618 shutdown_connections 2024-05-09T02:08:31.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:31.242+0000 7f76f911a700 1 --2- 172.21.15.19:0/566830618 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f76dc0b2910 0x7f76dc0b4e20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:31.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:31.242+0000 7f76f911a700 1 --2- 172.21.15.19:0/566830618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76f4108740 0x7f76f419e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:31.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:31.242+0000 7f76f911a700 1 --2- 172.21.15.19:0/566830618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76f41034c0 0x7f76f4199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:31.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:31.242+0000 7f76f911a700 1 --2- 172.21.15.19:0/566830618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4107df0 0x7f76f419a2d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:31.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:31.242+0000 7f76f911a700 1 -- 172.21.15.19:0/566830618 >> 172.21.15.19:0/566830618 conn(0x7f76f40fedc0 msgr2=0x7f76f4107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:31.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:31.243+0000 7f76f911a700 1 -- 172.21.15.19:0/566830618 shutdown_connections 2024-05-09T02:08:31.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:31.243+0000 7f76f911a700 1 -- 172.21.15.19:0/566830618 wait complete. 2024-05-09T02:08:31.246 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T02:08:31.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:31 smithi019 ceph-mon[30151]: pgmap v314: 97 pgs: 3 peering, 94 active+clean; 632 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.4 KiB/s wr, 14 op/s; 13 B/s, 0 objects/s recovering 2024-05-09T02:08:31.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:31 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/566830618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:08:31.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:31 smithi059 ceph-mon[34781]: pgmap v314: 97 pgs: 3 peering, 94 active+clean; 632 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.4 KiB/s wr, 14 op/s; 13 B/s, 0 objects/s recovering 2024-05-09T02:08:31.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:31 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/566830618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:08:31.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:31 smithi196 ceph-mon[34718]: pgmap v314: 97 pgs: 3 peering, 94 active+clean; 632 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.4 KiB/s wr, 14 op/s; 13 B/s, 0 objects/s recovering 2024-05-09T02:08:31.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:31 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/566830618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:08:31.955 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T02:08:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:33.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:33 smithi019 ceph-mon[30151]: pgmap v315: 97 pgs: 97 active+clean; 632 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.1 KiB/s wr, 12 op/s; 27 B/s, 1 objects/s recovering 2024-05-09T02:08:33.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:33 smithi059 ceph-mon[34781]: pgmap v315: 97 pgs: 97 active+clean; 632 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.1 KiB/s wr, 12 op/s; 27 B/s, 1 objects/s recovering 2024-05-09T02:08:33.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:33 smithi196 ceph-mon[34718]: pgmap v315: 97 pgs: 97 active+clean; 632 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.1 KiB/s wr, 12 op/s; 27 B/s, 1 objects/s recovering 2024-05-09T02:08:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.941+0000 7f848a3da700 1 -- 172.21.15.19:0/2572803004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8484107df0 msgr2=0x7f8484108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.941+0000 7f848a3da700 1 --2- 172.21.15.19:0/2572803004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8484107df0 0x7f8484108200 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f846c0099c0 tx=0x7f846c009cd0 comp rx=0 tx=0).stop 2024-05-09T02:08:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.941+0000 7f848a3da700 1 -- 172.21.15.19:0/2572803004 shutdown_connections 2024-05-09T02:08:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.941+0000 7f848a3da700 1 --2- 172.21.15.19:0/2572803004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8484108740 0x7f84841100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.941+0000 7f848a3da700 1 --2- 172.21.15.19:0/2572803004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84841034c0 0x7f84841038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.941+0000 7f848a3da700 1 --2- 172.21.15.19:0/2572803004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8484107df0 0x7f8484108200 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.941+0000 7f848a3da700 1 -- 172.21.15.19:0/2572803004 >> 172.21.15.19:0/2572803004 conn(0x7f84840fedf0 msgr2=0x7f8484101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.941+0000 7f848a3da700 1 -- 172.21.15.19:0/2572803004 shutdown_connections 2024-05-09T02:08:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.941+0000 7f848a3da700 1 -- 172.21.15.19:0/2572803004 wait complete. 2024-05-09T02:08:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f848a3da700 1 Processor -- start 2024-05-09T02:08:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f848a3da700 1 -- start start 2024-05-09T02:08:33.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f848a3da700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84841034c0 0x7f8484199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:33.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f848a3da700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8484107df0 0x7f848419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:33.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f848a3da700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8484108740 0x7f848419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:33.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f848a3da700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8484113370 con 0x7f84841034c0 2024-05-09T02:08:33.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f848a3da700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84841131f0 con 0x7f8484107df0 2024-05-09T02:08:33.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f848a3da700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84841134f0 con 0x7f8484108740 2024-05-09T02:08:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f8483fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84841034c0 0x7f8484199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f8483fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84841034c0 0x7f8484199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37386/0 (socket says 172.21.15.19:37386) 2024-05-09T02:08:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f8483fff700 1 -- 172.21.15.19:0/3253337650 learned_addr learned my addr 172.21.15.19:0/3253337650 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f847bfff700 1 --2- 172.21.15.19:0/3253337650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8484107df0 0x7f848419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f8488977700 1 --2- 172.21.15.19:0/3253337650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8484108740 0x7f848419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f8483fff700 1 -- 172.21.15.19:0/3253337650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8484108740 msgr2=0x7f848419e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f8483fff700 1 --2- 172.21.15.19:0/3253337650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8484108740 0x7f848419e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f8483fff700 1 -- 172.21.15.19:0/3253337650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8484107df0 msgr2=0x7f848419a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f8483fff700 1 --2- 172.21.15.19:0/3253337650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8484107df0 0x7f848419a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f8483fff700 1 -- 172.21.15.19:0/3253337650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848419ebc0 con 0x7f84841034c0 2024-05-09T02:08:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.942+0000 7f8488977700 1 --2- 172.21.15.19:0/3253337650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8484108740 0x7f848419e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:08:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.943+0000 7f8483fff700 1 --2- 172.21.15.19:0/3253337650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84841034c0 0x7f8484199da0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f847400d980 tx=0x7f847400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.943+0000 7f8481ffb700 1 -- 172.21.15.19:0/3253337650 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474017070 con 0x7f84841034c0 2024-05-09T02:08:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.943+0000 7f848a3da700 1 -- 172.21.15.19:0/3253337650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848419eeb0 con 0x7f84841034c0 2024-05-09T02:08:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.943+0000 7f8481ffb700 1 -- 172.21.15.19:0/3253337650 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8474004830 con 0x7f84841034c0 2024-05-09T02:08:33.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.943+0000 7f8481ffb700 1 -- 172.21.15.19:0/3253337650 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474005440 con 0x7f84841034c0 2024-05-09T02:08:33.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.943+0000 7f848a3da700 1 -- 172.21.15.19:0/3253337650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f848419f3a0 con 0x7f84841034c0 2024-05-09T02:08:33.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.944+0000 7f848a3da700 1 -- 172.21.15.19:0/3253337650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84840512c0 con 0x7f84841034c0 2024-05-09T02:08:33.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.945+0000 7f8481ffb700 1 -- 172.21.15.19:0/3253337650 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f84740055e0 con 0x7f84841034c0 2024-05-09T02:08:33.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.946+0000 7f8481ffb700 1 --2- 172.21.15.19:0/3253337650 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f84700b4b20 0x7f84700b7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:33.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.947+0000 7f847bfff700 1 --2- 172.21.15.19:0/3253337650 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f84700b4b20 0x7f84700b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:33.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.947+0000 7f8481ffb700 1 -- 172.21.15.19:0/3253337650 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f84740e2780 con 0x7f84841034c0 2024-05-09T02:08:33.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.947+0000 7f847bfff700 1 --2- 172.21.15.19:0/3253337650 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f84700b4b20 0x7f84700b7030 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f846c006010 tx=0x7f846c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:33.950+0000 7f8481ffb700 1 -- 172.21.15.19:0/3253337650 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8474020020 con 0x7f84841034c0 2024-05-09T02:08:34.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:34.166+0000 7f848a3da700 1 -- 172.21.15.19:0/3253337650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f848419b100 con 0x7f84841034c0 2024-05-09T02:08:34.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:34.168+0000 7f8481ffb700 1 -- 172.21.15.19:0/3253337650 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22698 (secure 0 0 0) 0x7f84740aa320 con 0x7f84841034c0 2024-05-09T02:08:34.171 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:08:34.171 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"fsid":"a0611f3e-0da7-11ef-bc97-c7b262605968","created":"2024-05-09T01:59:27.591343+0000","modified":"2024-05-09T02:08:25.302927+0000","last_up_change":"2024-05-09T02:06:45.565065+0000","last_in_change":"2024-05-09T02:06:24.967186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T02:02:50.018959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T02:07:43.283868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T02:07:47.110614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"996c6a3d-3e3a-451c-8789-36006e0b3dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6803","nonce":3002373576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6805","nonce":3002373576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6809","nonce":3002373576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6807","nonce":3002373576}]},"public_addr":"172.21.15.19:6803/3002373576","cluster_addr":"172.21.15.19:6805/3002373576","heartbeat_back_addr":"172.21.15.19:6809/3002373576","heartbeat_front_addr":"172.21.15.19:6807/3002373576","state":["exists","up"]},{"osd":1,"uuid":"1ac68d09-1198-4e81-86c7-e37f7c41aa5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6801","nonce":827458353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6803","nonce":827458353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6807","nonce":827458353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6805","nonce":827458353}]},"public_addr":"172.21.15.59:6801/827458353","cluster_addr":"172.21.15.59:6803/827458353","heartbeat_back_addr":"172.21.15.59:6807/827458353","heartbeat_front_addr":"172.21.15.59:6805/827458353","state":["exists","up"]},{"osd":2,"uuid":"818a0de1-04e6-471d-83df-a68833011b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6801","nonce":1659504639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6803","nonce":1659504639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6807","nonce":1659504639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6805","nonce":1659504639}]},"public_addr":"172.21.15.196:6801/1659504639","cluster_addr":"172.21.15.196:6803/1659504639","heartbeat_back_addr":"172.21.15.196:6807/1659504639","heartbeat_front_addr":"172.21.15.196:6805/1659504639","state":["exists","up"]},{"osd":3,"uuid":"7b204854-86b8-454d-aee0-8bb5775ce828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6811","nonce":2707671599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6813","nonce":2707671599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6817","nonce":2707671599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6815","nonce":2707671599}]},"public_addr":"172.21.15.19:6811/2707671599","cluster_addr":"172.21.15.19:6813/2707671599","heartbeat_back_addr":"172.21.15.19:6817/2707671599","heartbeat_front_addr":"172.21.15.19:6815/2707671599","state":["exists","up"]},{"osd":4,"uuid":"091667f1-e887-4a94-8723-6f85b676fef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6809","nonce":223037330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6811","nonce":223037330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6815","nonce":223037330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6813","nonce":223037330}]},"public_addr":"172.21.15.59:6809/223037330","cluster_addr":"172.21.15.59:6811/223037330","heartbeat_back_addr":"172.21.15.59:6815/223037330","heartbeat_front_addr":"172.21.15.59:6813/223037330","state":["exists","up"]},{"osd":5,"uuid":"2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6809","nonce":2516161898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6811","nonce":2516161898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6815","nonce":2516161898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6813","nonce":2516161898}]},"public_addr":"172.21.15.196:6809/2516161898","cluster_addr":"172.21.15.196:6811/2516161898","heartbeat_back_addr":"172.21.15.196:6815/2516161898","heartbeat_front_addr":"172.21.15.196:6813/2516161898","state":["exists","up"]},{"osd":6,"uuid":"4e352851-1279-475f-ba8b-eaf9ca73df74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6819","nonce":2150451637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6821","nonce":2150451637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6825","nonce":2150451637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6823","nonce":2150451637}]},"public_addr":"172.21.15.19:6819/2150451637","cluster_addr":"172.21.15.19:6821/2150451637","heartbeat_back_addr":"172.21.15.19:6825/2150451637","heartbeat_front_addr":"172.21.15.19:6823/2150451637","state":["exists","up"]},{"osd":7,"uuid":"83f53f59-9a60-4d87-9a8a-67e5c6f57004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6817","nonce":967902053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6819","nonce":967902053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6823","nonce":967902053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6821","nonce":967902053}]},"public_addr":"172.21.15.59:6817/967902053","cluster_addr":"172.21.15.59:6819/967902053","heartbeat_back_addr":"172.21.15.59:6823/967902053","heartbeat_front_addr":"172.21.15.59:6821/967902053","state":["exists","up"]},{"osd":8,"uuid":"a8f1bfb5-1cc8-4a65-8396-239bd781e1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6817","nonce":2946064259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6819","nonce":2946064259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6823","nonce":2946064259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6821","nonce":2946064259}]},"public_addr":"172.21.15.196:6817/2946064259","cluster_addr":"172.21.15.196:6819/2946064259","heartbeat_back_addr":"172.21.15.196:6823/2946064259","heartbeat_front_addr":"172.21.15.196:6821/2946064259","state":["exists","up"]},{"osd":9,"uuid":"73c58dc3-fcb0-4440-92b3-bf41a2e58fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6827","nonce":1735017886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6829","nonce":1735017886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6833","nonce":1735017886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6831","nonce":1735017886}]},"public_addr":"172.21.15.19:6827/1735017886","cluster_addr":"172.21.15.19:6829/1735017886","heartbeat_back_addr":"172.21.15.19:6833/1735017886","heartbeat_front_addr":"172.21.15.19:6831/1735017886","state":["exists","up"]},{"osd":10,"uuid":"8325282f-7325-4880-ab7d-b972717be9dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6825","nonce":2932899995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6827","nonce":2932899995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6831","nonce":2932899995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6829","nonce":2932899995}]},"public_addr":"172.21.15.59:6825/2932899995","cluster_addr":"172.21.15.59:6827/2932899995","heartbeat_back_addr":"172.21.15.59:6831/2932899995","heartbeat_front_addr":"172.21.15.59:6829/2932899995","state":["exists","up"]},{"osd":11,"uuid":"8b19228c-b286-43c8-a408-169782f13075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6825","nonce":2608023001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6827","nonce":2608023001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6831","nonce":2608023001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6829","nonce":2608023001}]},"public_addr":"172.21.15.196:6825/2608023001","cluster_addr":"172.21.15.196:6827/2608023001","heartbeat_back_addr":"172.21.15.196:6831/2608023001","heartbeat_front_addr":"172.21.15.196:6829/2608023001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:01:59.457748+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:23.640666+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:47.768243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:13.016746+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:39.060898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:04.508313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:31.366217+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:57.579758+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:23.177396+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:50.966399+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:16.838666+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:43.959387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3291838961":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1024133854":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1176455101":"2024-05-10T02:00:23.920230+0000","172.21.15.19:6801/2101052000":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/4018987404":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/3079777324":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6801/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/179703528":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/2101052000":"2024-05-10T02:00:23.920230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T02:08:34.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:34.171+0000 7f848a3da700 1 -- 172.21.15.19:0/3253337650 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f84700b4b20 msgr2=0x7f84700b7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:34.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:34.171+0000 7f848a3da700 1 --2- 172.21.15.19:0/3253337650 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f84700b4b20 0x7f84700b7030 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f846c006010 tx=0x7f846c000bc0 comp rx=0 tx=0).stop 2024-05-09T02:08:34.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:34.171+0000 7f848a3da700 1 -- 172.21.15.19:0/3253337650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84841034c0 msgr2=0x7f8484199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:34.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:34.171+0000 7f848a3da700 1 --2- 172.21.15.19:0/3253337650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84841034c0 0x7f8484199da0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f847400d980 tx=0x7f847400dd40 comp rx=0 tx=0).stop 2024-05-09T02:08:34.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:34.171+0000 7f848a3da700 1 -- 172.21.15.19:0/3253337650 shutdown_connections 2024-05-09T02:08:34.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:34.171+0000 7f848a3da700 1 --2- 172.21.15.19:0/3253337650 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f84700b4b20 0x7f84700b7030 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:34.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:34.171+0000 7f848a3da700 1 --2- 172.21.15.19:0/3253337650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8484108740 0x7f848419e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:34.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:34.171+0000 7f848a3da700 1 --2- 172.21.15.19:0/3253337650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84841034c0 0x7f8484199da0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:34.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:34.171+0000 7f848a3da700 1 --2- 172.21.15.19:0/3253337650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8484107df0 0x7f848419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:34.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:34.171+0000 7f848a3da700 1 -- 172.21.15.19:0/3253337650 >> 172.21.15.19:0/3253337650 conn(0x7f84840fedf0 msgr2=0x7f8484101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:34.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:34.171+0000 7f848a3da700 1 -- 172.21.15.19:0/3253337650 shutdown_connections 2024-05-09T02:08:34.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:34.171+0000 7f848a3da700 1 -- 172.21.15.19:0/3253337650 wait complete. 2024-05-09T02:08:34.480 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:34 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3253337650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:08:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:34 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3253337650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:08:34.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:34 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3253337650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:08:34.990 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T02:08:34.990 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T02:08:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:35.617 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:35 smithi059 ceph-mon[34781]: pgmap v316: 97 pgs: 97 active+clean; 632 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.4 KiB/s wr, 12 op/s; 23 B/s, 1 objects/s recovering 2024-05-09T02:08:35.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:35 smithi019 ceph-mon[30151]: pgmap v316: 97 pgs: 97 active+clean; 632 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.4 KiB/s wr, 12 op/s; 23 B/s, 1 objects/s recovering 2024-05-09T02:08:35.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:35 smithi196 ceph-mon[34718]: pgmap v316: 97 pgs: 97 active+clean; 632 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.4 KiB/s wr, 12 op/s; 23 B/s, 1 objects/s recovering 2024-05-09T02:08:36.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.980+0000 7f039e9c7700 1 -- 172.21.15.19:0/3929064535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0398107df0 msgr2=0x7f0398108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.980+0000 7f039e9c7700 1 --2- 172.21.15.19:0/3929064535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0398107df0 0x7f0398108200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f03800099c0 tx=0x7f0380009cd0 comp rx=0 tx=0).stop 2024-05-09T02:08:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.980+0000 7f039e9c7700 1 -- 172.21.15.19:0/3929064535 shutdown_connections 2024-05-09T02:08:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.980+0000 7f039e9c7700 1 --2- 172.21.15.19:0/3929064535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0398108740 0x7f03981100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.980+0000 7f039e9c7700 1 --2- 172.21.15.19:0/3929064535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03981034c0 0x7f03981038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.980+0000 7f039e9c7700 1 --2- 172.21.15.19:0/3929064535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0398107df0 0x7f0398108200 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.980+0000 7f039e9c7700 1 -- 172.21.15.19:0/3929064535 >> 172.21.15.19:0/3929064535 conn(0x7f03980fedc0 msgr2=0x7f03981011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.980+0000 7f039e9c7700 1 -- 172.21.15.19:0/3929064535 shutdown_connections 2024-05-09T02:08:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.980+0000 7f039e9c7700 1 -- 172.21.15.19:0/3929064535 wait complete. 2024-05-09T02:08:36.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.981+0000 7f039e9c7700 1 Processor -- start 2024-05-09T02:08:36.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.981+0000 7f039e9c7700 1 -- start start 2024-05-09T02:08:36.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.981+0000 7f039e9c7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03981034c0 0x7f0398199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:36.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.981+0000 7f039e9c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0398107df0 0x7f039819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:36.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.981+0000 7f039e9c7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0398108740 0x7f039819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:36.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.981+0000 7f039e9c7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03981133a0 con 0x7f0398107df0 2024-05-09T02:08:36.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.981+0000 7f039e9c7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0398113220 con 0x7f0398108740 2024-05-09T02:08:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.981+0000 7f039e9c7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0398113520 con 0x7f03981034c0 2024-05-09T02:08:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.981+0000 7f038ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0398107df0 0x7f039819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.981+0000 7f038ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0398107df0 0x7f039819a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47184/0 (socket says 172.21.15.19:47184) 2024-05-09T02:08:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.981+0000 7f038ffff700 1 -- 172.21.15.19:0/3970505780 learned_addr learned my addr 172.21.15.19:0/3970505780 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.981+0000 7f039cf64700 1 --2- 172.21.15.19:0/3970505780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0398108740 0x7f039819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.982+0000 7f0397fff700 1 --2- 172.21.15.19:0/3970505780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03981034c0 0x7f0398199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.982+0000 7f038ffff700 1 -- 172.21.15.19:0/3970505780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03981034c0 msgr2=0x7f0398199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.982+0000 7f038ffff700 1 --2- 172.21.15.19:0/3970505780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03981034c0 0x7f0398199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.982+0000 7f038ffff700 1 -- 172.21.15.19:0/3970505780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0398108740 msgr2=0x7f039819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.982+0000 7f038ffff700 1 --2- 172.21.15.19:0/3970505780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0398108740 0x7f039819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.982+0000 7f038ffff700 1 -- 172.21.15.19:0/3970505780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039819eb90 con 0x7f0398107df0 2024-05-09T02:08:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.982+0000 7f038ffff700 1 --2- 172.21.15.19:0/3970505780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0398107df0 0x7f039819a310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0380009f50 tx=0x7f03800037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.982+0000 7f0395ffb700 1 -- 172.21.15.19:0/3970505780 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380027070 con 0x7f0398107df0 2024-05-09T02:08:36.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.982+0000 7f039e9c7700 1 -- 172.21.15.19:0/3970505780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039819ee20 con 0x7f0398107df0 2024-05-09T02:08:36.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.982+0000 7f0395ffb700 1 -- 172.21.15.19:0/3970505780 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0380004090 con 0x7f0398107df0 2024-05-09T02:08:36.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.982+0000 7f0395ffb700 1 -- 172.21.15.19:0/3970505780 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380022e40 con 0x7f0398107df0 2024-05-09T02:08:36.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.982+0000 7f039e9c7700 1 -- 172.21.15.19:0/3970505780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039819f350 con 0x7f0398107df0 2024-05-09T02:08:36.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.984+0000 7f039e9c7700 1 -- 172.21.15.19:0/3970505780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03980512c0 con 0x7f0398107df0 2024-05-09T02:08:36.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.985+0000 7f0395ffb700 1 -- 172.21.15.19:0/3970505780 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f038002a070 con 0x7f0398107df0 2024-05-09T02:08:36.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.986+0000 7f0395ffb700 1 --2- 172.21.15.19:0/3970505780 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f03840b2910 0x7f03840b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:36.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.986+0000 7f0397fff700 1 --2- 172.21.15.19:0/3970505780 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f03840b2910 0x7f03840b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:36.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.986+0000 7f0395ffb700 1 -- 172.21.15.19:0/3970505780 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f03800e3180 con 0x7f0398107df0 2024-05-09T02:08:36.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.986+0000 7f0397fff700 1 --2- 172.21.15.19:0/3970505780 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f03840b2910 0x7f03840b4e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0388005fd0 tx=0x7f038800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:36.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:36.991+0000 7f0395ffb700 1 -- 172.21.15.19:0/3970505780 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0380013070 con 0x7f0398107df0 2024-05-09T02:08:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:37.199+0000 7f039e9c7700 1 -- 172.21.15.19:0/3970505780 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f039806b610 con 0x7f03840b2910 2024-05-09T02:08:37.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:37.207+0000 7f0395ffb700 1 -- 172.21.15.19:0/3970505780 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259014 (secure 0 0 0) 0x7f039806b610 con 0x7f03840b2910 2024-05-09T02:08:37.213 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:08:37.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:37.211+0000 7f039e9c7700 1 -- 172.21.15.19:0/3970505780 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f03840b2910 msgr2=0x7f03840b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:37.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:37.211+0000 7f039e9c7700 1 --2- 172.21.15.19:0/3970505780 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f03840b2910 0x7f03840b4e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0388005fd0 tx=0x7f038800a560 comp rx=0 tx=0).stop 2024-05-09T02:08:37.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:37.211+0000 7f039e9c7700 1 -- 172.21.15.19:0/3970505780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0398107df0 msgr2=0x7f039819a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:37.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:37.211+0000 7f039e9c7700 1 --2- 172.21.15.19:0/3970505780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0398107df0 0x7f039819a310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0380009f50 tx=0x7f03800037c0 comp rx=0 tx=0).stop 2024-05-09T02:08:37.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:37.211+0000 7f039e9c7700 1 -- 172.21.15.19:0/3970505780 shutdown_connections 2024-05-09T02:08:37.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:37.211+0000 7f039e9c7700 1 --2- 172.21.15.19:0/3970505780 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f03840b2910 0x7f03840b4e20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:37.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:37.211+0000 7f039e9c7700 1 --2- 172.21.15.19:0/3970505780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0398108740 0x7f039819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:37.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:37.211+0000 7f039e9c7700 1 --2- 172.21.15.19:0/3970505780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03981034c0 0x7f0398199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:37.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:37.211+0000 7f039e9c7700 1 --2- 172.21.15.19:0/3970505780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0398107df0 0x7f039819a310 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:37.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:37.211+0000 7f039e9c7700 1 -- 172.21.15.19:0/3970505780 >> 172.21.15.19:0/3970505780 conn(0x7f03980fedc0 msgr2=0x7f03981096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:37.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:37.211+0000 7f039e9c7700 1 -- 172.21.15.19:0/3970505780 shutdown_connections 2024-05-09T02:08:37.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:37.212+0000 7f039e9c7700 1 -- 172.21.15.19:0/3970505780 wait complete. 2024-05-09T02:08:37.216 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-09T02:08:37.595 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:37 smithi019 ceph-mon[30151]: pgmap v317: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 11 op/s; 20 B/s, 1 objects/s recovering 2024-05-09T02:08:37.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:37 smithi059 ceph-mon[34781]: pgmap v317: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 11 op/s; 20 B/s, 1 objects/s recovering 2024-05-09T02:08:37.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:37 smithi196 ceph-mon[34718]: pgmap v317: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 11 op/s; 20 B/s, 1 objects/s recovering 2024-05-09T02:08:37.927 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-05-09T02:08:36.027927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652996,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":380,"num_read_kb":289,"num_write":366,"num_write_kb":2281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2952064,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376720,"kb_used_data":10884,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565104,"statfs":{"total":1151940427776,"available":1151554666496,"internally_reserved":0,"allocated":11145216,"data_stored":7099308,"data_compressed":176253,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13497,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":102,"num_read_kb":77,"num_write":16,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.720111"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311161+0000","last_change":"2024-05-09T02:07:48.203957+0000","last_active":"2024-05-09T02:08:25.311161+0000","last_peered":"2024-05-09T02:08:25.311161+0000","last_clean":"2024-05-09T02:08:25.311161+0000","last_became_active":"2024-05-09T02:07:48.203354+0000","last_became_peered":"2024-05-09T02:07:48.203354+0000","last_unstale":"2024-05-09T02:08:25.311161+0000","last_undegraded":"2024-05-09T02:08:25.311161+0000","last_fullsized":"2024-05-09T02:08:25.311161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:06.169693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315459+0000","last_change":"2024-05-09T02:07:48.192885+0000","last_active":"2024-05-09T02:08:25.315459+0000","last_peered":"2024-05-09T02:08:25.315459+0000","last_clean":"2024-05-09T02:08:25.315459+0000","last_became_active":"2024-05-09T02:07:48.192476+0000","last_became_peered":"2024-05-09T02:07:48.192476+0000","last_unstale":"2024-05-09T02:08:25.315459+0000","last_undegraded":"2024-05-09T02:08:25.315459+0000","last_fullsized":"2024-05-09T02:08:25.315459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:33.710309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314654+0000","last_change":"2024-05-09T02:07:48.198684+0000","last_active":"2024-05-09T02:08:25.314654+0000","last_peered":"2024-05-09T02:08:25.314654+0000","last_clean":"2024-05-09T02:08:25.314654+0000","last_became_active":"2024-05-09T02:07:48.198344+0000","last_became_peered":"2024-05-09T02:07:48.198344+0000","last_unstale":"2024-05-09T02:08:25.314654+0000","last_undegraded":"2024-05-09T02:08:25.314654+0000","last_fullsized":"2024-05-09T02:08:25.314654+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:19.990063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317876+0000","last_change":"2024-05-09T02:07:48.189209+0000","last_active":"2024-05-09T02:08:25.317876+0000","last_peered":"2024-05-09T02:08:25.317876+0000","last_clean":"2024-05-09T02:08:25.317876+0000","last_became_active":"2024-05-09T02:07:48.188744+0000","last_became_peered":"2024-05-09T02:07:48.188744+0000","last_unstale":"2024-05-09T02:08:25.317876+0000","last_undegraded":"2024-05-09T02:08:25.317876+0000","last_fullsized":"2024-05-09T02:08:25.317876+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:03:25.216849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311030+0000","last_change":"2024-05-09T02:07:48.187897+0000","last_active":"2024-05-09T02:08:25.311030+0000","last_peered":"2024-05-09T02:08:25.311030+0000","last_clean":"2024-05-09T02:08:25.311030+0000","last_became_active":"2024-05-09T02:07:48.187476+0000","last_became_peered":"2024-05-09T02:07:48.187476+0000","last_unstale":"2024-05-09T02:08:25.311030+0000","last_undegraded":"2024-05-09T02:08:25.311030+0000","last_fullsized":"2024-05-09T02:08:25.311030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:39.366520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.307870+0000","last_change":"2024-05-09T02:07:48.195436+0000","last_active":"2024-05-09T02:08:25.307870+0000","last_peered":"2024-05-09T02:08:25.307870+0000","last_clean":"2024-05-09T02:08:25.307870+0000","last_became_active":"2024-05-09T02:07:48.194982+0000","last_became_peered":"2024-05-09T02:07:48.194982+0000","last_unstale":"2024-05-09T02:08:25.307870+0000","last_undegraded":"2024-05-09T02:08:25.307870+0000","last_fullsized":"2024-05-09T02:08:25.307870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:19.746701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313666+0000","last_change":"2024-05-09T02:07:48.198458+0000","last_active":"2024-05-09T02:08:25.313666+0000","last_peered":"2024-05-09T02:08:25.313666+0000","last_clean":"2024-05-09T02:08:25.313666+0000","last_became_active":"2024-05-09T02:07:48.198089+0000","last_became_peered":"2024-05-09T02:07:48.198089+0000","last_unstale":"2024-05-09T02:08:25.313666+0000","last_undegraded":"2024-05-09T02:08:25.313666+0000","last_fullsized":"2024-05-09T02:08:25.313666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:37:44.999438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314425+0000","last_change":"2024-05-09T02:07:48.200294+0000","last_active":"2024-05-09T02:08:25.314425+0000","last_peered":"2024-05-09T02:08:25.314425+0000","last_clean":"2024-05-09T02:08:25.314425+0000","last_became_active":"2024-05-09T02:07:48.199949+0000","last_became_peered":"2024-05-09T02:07:48.199949+0000","last_unstale":"2024-05-09T02:08:25.314425+0000","last_undegraded":"2024-05-09T02:08:25.314425+0000","last_fullsized":"2024-05-09T02:08:25.314425+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:39.263849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315463+0000","last_change":"2024-05-09T02:07:48.196282+0000","last_active":"2024-05-09T02:08:25.315463+0000","last_peered":"2024-05-09T02:08:25.315463+0000","last_clean":"2024-05-09T02:08:25.315463+0000","last_became_active":"2024-05-09T02:07:48.195887+0000","last_became_peered":"2024-05-09T02:07:48.195887+0000","last_unstale":"2024-05-09T02:08:25.315463+0000","last_undegraded":"2024-05-09T02:08:25.315463+0000","last_fullsized":"2024-05-09T02:08:25.315463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:26.081867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.320636+0000","last_change":"2024-05-09T02:08:25.320636+0000","last_active":"2024-05-09T02:08:25.320636+0000","last_peered":"2024-05-09T02:08:25.320636+0000","last_clean":"2024-05-09T02:08:25.320636+0000","last_became_active":"2024-05-09T02:08:25.320155+0000","last_became_peered":"2024-05-09T02:08:25.320155+0000","last_unstale":"2024-05-09T02:08:25.320636+0000","last_undegraded":"2024-05-09T02:08:25.320636+0000","last_fullsized":"2024-05-09T02:08:25.320636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:55.728773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314733+0000","last_change":"2024-05-09T02:07:48.197649+0000","last_active":"2024-05-09T02:08:25.314733+0000","last_peered":"2024-05-09T02:08:25.314733+0000","last_clean":"2024-05-09T02:08:25.314733+0000","last_became_active":"2024-05-09T02:07:48.196884+0000","last_became_peered":"2024-05-09T02:07:48.196884+0000","last_unstale":"2024-05-09T02:08:25.314733+0000","last_undegraded":"2024-05-09T02:08:25.314733+0000","last_fullsized":"2024-05-09T02:08:25.314733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:23.851216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318021+0000","last_change":"2024-05-09T02:07:48.193568+0000","last_active":"2024-05-09T02:08:25.318021+0000","last_peered":"2024-05-09T02:08:25.318021+0000","last_clean":"2024-05-09T02:08:25.318021+0000","last_became_active":"2024-05-09T02:07:48.193117+0000","last_became_peered":"2024-05-09T02:07:48.193117+0000","last_unstale":"2024-05-09T02:08:25.318021+0000","last_undegraded":"2024-05-09T02:08:25.318021+0000","last_fullsized":"2024-05-09T02:08:25.318021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:34.371246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309662+0000","last_change":"2024-05-09T02:07:48.196314+0000","last_active":"2024-05-09T02:08:25.309662+0000","last_peered":"2024-05-09T02:08:25.309662+0000","last_clean":"2024-05-09T02:08:25.309662+0000","last_became_active":"2024-05-09T02:07:48.195988+0000","last_became_peered":"2024-05-09T02:07:48.195988+0000","last_unstale":"2024-05-09T02:08:25.309662+0000","last_undegraded":"2024-05-09T02:08:25.309662+0000","last_fullsized":"2024-05-09T02:08:25.309662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:08.556166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314018+0000","last_change":"2024-05-09T02:07:48.192974+0000","last_active":"2024-05-09T02:08:25.314018+0000","last_peered":"2024-05-09T02:08:25.314018+0000","last_clean":"2024-05-09T02:08:25.314018+0000","last_became_active":"2024-05-09T02:07:48.192599+0000","last_became_peered":"2024-05-09T02:07:48.192599+0000","last_unstale":"2024-05-09T02:08:25.314018+0000","last_undegraded":"2024-05-09T02:08:25.314018+0000","last_fullsized":"2024-05-09T02:08:25.314018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:37.192219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314434+0000","last_change":"2024-05-09T02:07:48.193208+0000","last_active":"2024-05-09T02:08:25.314434+0000","last_peered":"2024-05-09T02:08:25.314434+0000","last_clean":"2024-05-09T02:08:25.314434+0000","last_became_active":"2024-05-09T02:07:48.192654+0000","last_became_peered":"2024-05-09T02:07:48.192654+0000","last_unstale":"2024-05-09T02:08:25.314434+0000","last_undegraded":"2024-05-09T02:08:25.314434+0000","last_fullsized":"2024-05-09T02:08:25.314434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:55:42.870372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314811+0000","last_change":"2024-05-09T02:07:48.187568+0000","last_active":"2024-05-09T02:08:25.314811+0000","last_peered":"2024-05-09T02:08:25.314811+0000","last_clean":"2024-05-09T02:08:25.314811+0000","last_became_active":"2024-05-09T02:07:48.187159+0000","last_became_peered":"2024-05-09T02:07:48.187159+0000","last_unstale":"2024-05-09T02:08:25.314811+0000","last_undegraded":"2024-05-09T02:08:25.314811+0000","last_fullsized":"2024-05-09T02:08:25.314811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:04.994094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311915+0000","last_change":"2024-05-09T02:07:48.204406+0000","last_active":"2024-05-09T02:08:25.311915+0000","last_peered":"2024-05-09T02:08:25.311915+0000","last_clean":"2024-05-09T02:08:25.311915+0000","last_became_active":"2024-05-09T02:07:48.203592+0000","last_became_peered":"2024-05-09T02:07:48.203592+0000","last_unstale":"2024-05-09T02:08:25.311915+0000","last_undegraded":"2024-05-09T02:08:25.311915+0000","last_fullsized":"2024-05-09T02:08:25.311915+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:21.655807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317492+0000","last_change":"2024-05-09T02:07:48.195326+0000","last_active":"2024-05-09T02:08:25.317492+0000","last_peered":"2024-05-09T02:08:25.317492+0000","last_clean":"2024-05-09T02:08:25.317492+0000","last_became_active":"2024-05-09T02:07:48.194969+0000","last_became_peered":"2024-05-09T02:07:48.194969+0000","last_unstale":"2024-05-09T02:08:25.317492+0000","last_undegraded":"2024-05-09T02:08:25.317492+0000","last_fullsized":"2024-05-09T02:08:25.317492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:00.269753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315957+0000","last_change":"2024-05-09T02:07:48.190706+0000","last_active":"2024-05-09T02:08:25.315957+0000","last_peered":"2024-05-09T02:08:25.315957+0000","last_clean":"2024-05-09T02:08:25.315957+0000","last_became_active":"2024-05-09T02:07:48.190171+0000","last_became_peered":"2024-05-09T02:07:48.190171+0000","last_unstale":"2024-05-09T02:08:25.315957+0000","last_undegraded":"2024-05-09T02:08:25.315957+0000","last_fullsized":"2024-05-09T02:08:25.315957+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:19:31.333618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":66,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:32.304572+0000","last_change":"2024-05-09T02:07:45.158224+0000","last_active":"2024-05-09T02:08:32.304572+0000","last_peered":"2024-05-09T02:08:32.304572+0000","last_clean":"2024-05-09T02:08:32.304572+0000","last_became_active":"2024-05-09T02:07:45.157722+0000","last_became_peered":"2024-05-09T02:07:45.157722+0000","last_unstale":"2024-05-09T02:08:32.304572+0000","last_undegraded":"2024-05-09T02:08:32.304572+0000","last_fullsized":"2024-05-09T02:08:32.304572+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:31.750699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315461+0000","last_change":"2024-05-09T02:07:48.197978+0000","last_active":"2024-05-09T02:08:25.315461+0000","last_peered":"2024-05-09T02:08:25.315461+0000","last_clean":"2024-05-09T02:08:25.315461+0000","last_became_active":"2024-05-09T02:07:48.197077+0000","last_became_peered":"2024-05-09T02:07:48.197077+0000","last_unstale":"2024-05-09T02:08:25.315461+0000","last_undegraded":"2024-05-09T02:08:25.315461+0000","last_fullsized":"2024-05-09T02:08:25.315461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:25.897904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315832+0000","last_change":"2024-05-09T02:07:45.185758+0000","last_active":"2024-05-09T02:08:25.315832+0000","last_peered":"2024-05-09T02:08:25.315832+0000","last_clean":"2024-05-09T02:08:25.315832+0000","last_became_active":"2024-05-09T02:07:45.185411+0000","last_became_peered":"2024-05-09T02:07:45.185411+0000","last_unstale":"2024-05-09T02:08:25.315832+0000","last_undegraded":"2024-05-09T02:08:25.315832+0000","last_fullsized":"2024-05-09T02:08:25.315832+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:04.410523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317321+0000","last_change":"2024-05-09T02:07:48.199794+0000","last_active":"2024-05-09T02:08:25.317321+0000","last_peered":"2024-05-09T02:08:25.317321+0000","last_clean":"2024-05-09T02:08:25.317321+0000","last_became_active":"2024-05-09T02:07:48.199453+0000","last_became_peered":"2024-05-09T02:07:48.199453+0000","last_unstale":"2024-05-09T02:08:25.317321+0000","last_undegraded":"2024-05-09T02:08:25.317321+0000","last_fullsized":"2024-05-09T02:08:25.317321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:09.735830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311011+0000","last_change":"2024-05-09T02:07:45.162221+0000","last_active":"2024-05-09T02:08:25.311011+0000","last_peered":"2024-05-09T02:08:25.311011+0000","last_clean":"2024-05-09T02:08:25.311011+0000","last_became_active":"2024-05-09T02:07:45.161748+0000","last_became_peered":"2024-05-09T02:07:45.161748+0000","last_unstale":"2024-05-09T02:08:25.311011+0000","last_undegraded":"2024-05-09T02:08:25.311011+0000","last_fullsized":"2024-05-09T02:08:25.311011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:38.837411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315182+0000","last_change":"2024-05-09T02:07:48.188741+0000","last_active":"2024-05-09T02:08:25.315182+0000","last_peered":"2024-05-09T02:08:25.315182+0000","last_clean":"2024-05-09T02:08:25.315182+0000","last_became_active":"2024-05-09T02:07:48.188329+0000","last_became_peered":"2024-05-09T02:07:48.188329+0000","last_unstale":"2024-05-09T02:08:25.315182+0000","last_undegraded":"2024-05-09T02:08:25.315182+0000","last_fullsized":"2024-05-09T02:08:25.315182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:46.773336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310908+0000","last_change":"2024-05-09T02:07:45.163573+0000","last_active":"2024-05-09T02:08:25.310908+0000","last_peered":"2024-05-09T02:08:25.310908+0000","last_clean":"2024-05-09T02:08:25.310908+0000","last_became_active":"2024-05-09T02:07:45.163162+0000","last_became_peered":"2024-05-09T02:07:45.163162+0000","last_unstale":"2024-05-09T02:08:25.310908+0000","last_undegraded":"2024-05-09T02:08:25.310908+0000","last_fullsized":"2024-05-09T02:08:25.310908+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:12.356586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318419+0000","last_change":"2024-05-09T02:07:48.203928+0000","last_active":"2024-05-09T02:08:25.318419+0000","last_peered":"2024-05-09T02:08:25.318419+0000","last_clean":"2024-05-09T02:08:25.318419+0000","last_became_active":"2024-05-09T02:07:48.203224+0000","last_became_peered":"2024-05-09T02:07:48.203224+0000","last_unstale":"2024-05-09T02:08:25.318419+0000","last_undegraded":"2024-05-09T02:08:25.318419+0000","last_fullsized":"2024-05-09T02:08:25.318419+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:00.568050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:30.303765+0000","last_change":"2024-05-09T02:07:45.160161+0000","last_active":"2024-05-09T02:08:30.303765+0000","last_peered":"2024-05-09T02:08:30.303765+0000","last_clean":"2024-05-09T02:08:30.303765+0000","last_became_active":"2024-05-09T02:07:45.159118+0000","last_became_peered":"2024-05-09T02:07:45.159118+0000","last_unstale":"2024-05-09T02:08:30.303765+0000","last_undegraded":"2024-05-09T02:08:30.303765+0000","last_fullsized":"2024-05-09T02:08:30.303765+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:04.510877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.319860+0000","last_change":"2024-05-09T02:08:25.319860+0000","last_active":"2024-05-09T02:08:25.319860+0000","last_peered":"2024-05-09T02:08:25.319860+0000","last_clean":"2024-05-09T02:08:25.319860+0000","last_became_active":"2024-05-09T02:08:25.319370+0000","last_became_peered":"2024-05-09T02:08:25.319370+0000","last_unstale":"2024-05-09T02:08:25.319860+0000","last_undegraded":"2024-05-09T02:08:25.319860+0000","last_fullsized":"2024-05-09T02:08:25.319860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:38.504057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.327069+0000","last_change":"2024-05-09T02:08:25.327069+0000","last_active":"2024-05-09T02:08:25.327069+0000","last_peered":"2024-05-09T02:08:25.327069+0000","last_clean":"2024-05-09T02:08:25.327069+0000","last_became_active":"2024-05-09T02:08:25.316629+0000","last_became_peered":"2024-05-09T02:08:25.316629+0000","last_unstale":"2024-05-09T02:08:25.327069+0000","last_undegraded":"2024-05-09T02:08:25.327069+0000","last_fullsized":"2024-05-09T02:08:25.327069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:51.028256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318643+0000","last_change":"2024-05-09T02:07:48.201056+0000","last_active":"2024-05-09T02:08:25.318643+0000","last_peered":"2024-05-09T02:08:25.318643+0000","last_clean":"2024-05-09T02:08:25.318643+0000","last_became_active":"2024-05-09T02:07:48.200671+0000","last_became_peered":"2024-05-09T02:07:48.200671+0000","last_unstale":"2024-05-09T02:08:25.318643+0000","last_undegraded":"2024-05-09T02:08:25.318643+0000","last_fullsized":"2024-05-09T02:08:25.318643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:37:16.138170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309296+0000","last_change":"2024-05-09T02:07:45.160417+0000","last_active":"2024-05-09T02:08:25.309296+0000","last_peered":"2024-05-09T02:08:25.309296+0000","last_clean":"2024-05-09T02:08:25.309296+0000","last_became_active":"2024-05-09T02:07:45.160076+0000","last_became_peered":"2024-05-09T02:07:45.160076+0000","last_unstale":"2024-05-09T02:08:25.309296+0000","last_undegraded":"2024-05-09T02:08:25.309296+0000","last_fullsized":"2024-05-09T02:08:25.309296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:53.653494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314286+0000","last_change":"2024-05-09T02:07:48.203267+0000","last_active":"2024-05-09T02:08:25.314286+0000","last_peered":"2024-05-09T02:08:25.314286+0000","last_clean":"2024-05-09T02:08:25.314286+0000","last_became_active":"2024-05-09T02:07:48.202747+0000","last_became_peered":"2024-05-09T02:07:48.202747+0000","last_unstale":"2024-05-09T02:08:25.314286+0000","last_undegraded":"2024-05-09T02:08:25.314286+0000","last_fullsized":"2024-05-09T02:08:25.314286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.954400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317820+0000","last_change":"2024-05-09T02:07:45.164713+0000","last_active":"2024-05-09T02:08:25.317820+0000","last_peered":"2024-05-09T02:08:25.317820+0000","last_clean":"2024-05-09T02:08:25.317820+0000","last_became_active":"2024-05-09T02:07:45.164324+0000","last_became_peered":"2024-05-09T02:07:45.164324+0000","last_unstale":"2024-05-09T02:08:25.317820+0000","last_undegraded":"2024-05-09T02:08:25.317820+0000","last_fullsized":"2024-05-09T02:08:25.317820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:02.481419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.316226+0000","last_change":"2024-05-09T02:07:48.190305+0000","last_active":"2024-05-09T02:08:25.316226+0000","last_peered":"2024-05-09T02:08:25.316226+0000","last_clean":"2024-05-09T02:08:25.316226+0000","last_became_active":"2024-05-09T02:07:48.189971+0000","last_became_peered":"2024-05-09T02:07:48.189971+0000","last_unstale":"2024-05-09T02:08:25.316226+0000","last_undegraded":"2024-05-09T02:08:25.316226+0000","last_fullsized":"2024-05-09T02:08:25.316226+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:23.565092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":84,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:34.308791+0000","last_change":"2024-05-09T02:07:45.164273+0000","last_active":"2024-05-09T02:08:34.308791+0000","last_peered":"2024-05-09T02:08:34.308791+0000","last_clean":"2024-05-09T02:08:34.308791+0000","last_became_active":"2024-05-09T02:07:45.163183+0000","last_became_peered":"2024-05-09T02:07:45.163183+0000","last_unstale":"2024-05-09T02:08:34.308791+0000","last_undegraded":"2024-05-09T02:08:34.308791+0000","last_fullsized":"2024-05-09T02:08:34.308791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:11.395177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315668+0000","last_change":"2024-05-09T02:07:48.202995+0000","last_active":"2024-05-09T02:08:25.315668+0000","last_peered":"2024-05-09T02:08:25.315668+0000","last_clean":"2024-05-09T02:08:25.315668+0000","last_became_active":"2024-05-09T02:07:48.200915+0000","last_became_peered":"2024-05-09T02:07:48.200915+0000","last_unstale":"2024-05-09T02:08:25.315668+0000","last_undegraded":"2024-05-09T02:08:25.315668+0000","last_fullsized":"2024-05-09T02:08:25.315668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:25.555725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315988+0000","last_change":"2024-05-09T02:07:45.166221+0000","last_active":"2024-05-09T02:08:25.315988+0000","last_peered":"2024-05-09T02:08:25.315988+0000","last_clean":"2024-05-09T02:08:25.315988+0000","last_became_active":"2024-05-09T02:07:45.165813+0000","last_became_peered":"2024-05-09T02:07:45.165813+0000","last_unstale":"2024-05-09T02:08:25.315988+0000","last_undegraded":"2024-05-09T02:08:25.315988+0000","last_fullsized":"2024-05-09T02:08:25.315988+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:01.480828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317355+0000","last_change":"2024-05-09T02:07:48.201475+0000","last_active":"2024-05-09T02:08:25.317355+0000","last_peered":"2024-05-09T02:08:25.317355+0000","last_clean":"2024-05-09T02:08:25.317355+0000","last_became_active":"2024-05-09T02:07:48.201129+0000","last_became_peered":"2024-05-09T02:07:48.201129+0000","last_unstale":"2024-05-09T02:08:25.317355+0000","last_undegraded":"2024-05-09T02:08:25.317355+0000","last_fullsized":"2024-05-09T02:08:25.317355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:51.726752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318564+0000","last_change":"2024-05-09T02:07:45.185045+0000","last_active":"2024-05-09T02:08:25.318564+0000","last_peered":"2024-05-09T02:08:25.318564+0000","last_clean":"2024-05-09T02:08:25.318564+0000","last_became_active":"2024-05-09T02:07:45.184592+0000","last_became_peered":"2024-05-09T02:07:45.184592+0000","last_unstale":"2024-05-09T02:08:25.318564+0000","last_undegraded":"2024-05-09T02:08:25.318564+0000","last_fullsized":"2024-05-09T02:08:25.318564+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:56.220022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315185+0000","last_change":"2024-05-09T02:07:48.199873+0000","last_active":"2024-05-09T02:08:25.315185+0000","last_peered":"2024-05-09T02:08:25.315185+0000","last_clean":"2024-05-09T02:08:25.315185+0000","last_became_active":"2024-05-09T02:07:48.199502+0000","last_became_peered":"2024-05-09T02:07:48.199502+0000","last_unstale":"2024-05-09T02:08:25.315185+0000","last_undegraded":"2024-05-09T02:08:25.315185+0000","last_fullsized":"2024-05-09T02:08:25.315185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:15.406904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'30","reported_seq":118,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:34.312189+0000","last_change":"2024-05-09T02:07:45.166167+0000","last_active":"2024-05-09T02:08:34.312189+0000","last_peered":"2024-05-09T02:08:34.312189+0000","last_clean":"2024-05-09T02:08:34.312189+0000","last_became_active":"2024-05-09T02:07:45.165800+0000","last_became_peered":"2024-05-09T02:07:45.165800+0000","last_unstale":"2024-05-09T02:08:34.312189+0000","last_undegraded":"2024-05-09T02:08:34.312189+0000","last_fullsized":"2024-05-09T02:08:34.312189+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:03.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55581,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":62,"num_read_kb":84,"num_write":32,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310662+0000","last_change":"2024-05-09T02:07:48.187809+0000","last_active":"2024-05-09T02:08:25.310662+0000","last_peered":"2024-05-09T02:08:25.310662+0000","last_clean":"2024-05-09T02:08:25.310662+0000","last_became_active":"2024-05-09T02:07:48.186939+0000","last_became_peered":"2024-05-09T02:07:48.186939+0000","last_unstale":"2024-05-09T02:08:25.310662+0000","last_undegraded":"2024-05-09T02:08:25.310662+0000","last_fullsized":"2024-05-09T02:08:25.310662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:13:15.600343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315617+0000","last_change":"2024-05-09T02:07:45.163756+0000","last_active":"2024-05-09T02:08:25.315617+0000","last_peered":"2024-05-09T02:08:25.315617+0000","last_clean":"2024-05-09T02:08:25.315617+0000","last_became_active":"2024-05-09T02:07:45.163388+0000","last_became_peered":"2024-05-09T02:07:45.163388+0000","last_unstale":"2024-05-09T02:08:25.315617+0000","last_undegraded":"2024-05-09T02:08:25.315617+0000","last_fullsized":"2024-05-09T02:08:25.315617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:41.937148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309348+0000","last_change":"2024-05-09T02:07:48.195218+0000","last_active":"2024-05-09T02:08:25.309348+0000","last_peered":"2024-05-09T02:08:25.309348+0000","last_clean":"2024-05-09T02:08:25.309348+0000","last_became_active":"2024-05-09T02:07:48.194895+0000","last_became_peered":"2024-05-09T02:07:48.194895+0000","last_unstale":"2024-05-09T02:08:25.309348+0000","last_undegraded":"2024-05-09T02:08:25.309348+0000","last_fullsized":"2024-05-09T02:08:25.309348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:49.086594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318180+0000","last_change":"2024-05-09T02:07:45.158039+0000","last_active":"2024-05-09T02:08:25.318180+0000","last_peered":"2024-05-09T02:08:25.318180+0000","last_clean":"2024-05-09T02:08:25.318180+0000","last_became_active":"2024-05-09T02:07:45.157255+0000","last_became_peered":"2024-05-09T02:07:45.157255+0000","last_unstale":"2024-05-09T02:08:25.318180+0000","last_undegraded":"2024-05-09T02:08:25.318180+0000","last_fullsized":"2024-05-09T02:08:25.318180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:56.513999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313958+0000","last_change":"2024-05-09T02:07:48.200150+0000","last_active":"2024-05-09T02:08:25.313958+0000","last_peered":"2024-05-09T02:08:25.313958+0000","last_clean":"2024-05-09T02:08:25.313958+0000","last_became_active":"2024-05-09T02:07:48.199336+0000","last_became_peered":"2024-05-09T02:07:48.199336+0000","last_unstale":"2024-05-09T02:08:25.313958+0000","last_undegraded":"2024-05-09T02:08:25.313958+0000","last_fullsized":"2024-05-09T02:08:25.313958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:45.699119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":58,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:29.301526+0000","last_change":"2024-05-09T02:07:45.165476+0000","last_active":"2024-05-09T02:08:29.301526+0000","last_peered":"2024-05-09T02:08:29.301526+0000","last_clean":"2024-05-09T02:08:29.301526+0000","last_became_active":"2024-05-09T02:07:45.165107+0000","last_became_peered":"2024-05-09T02:07:45.165107+0000","last_unstale":"2024-05-09T02:08:29.301526+0000","last_undegraded":"2024-05-09T02:08:29.301526+0000","last_fullsized":"2024-05-09T02:08:29.301526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:15:33.518982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313557+0000","last_change":"2024-05-09T02:07:48.193487+0000","last_active":"2024-05-09T02:08:25.313557+0000","last_peered":"2024-05-09T02:08:25.313557+0000","last_clean":"2024-05-09T02:08:25.313557+0000","last_became_active":"2024-05-09T02:07:48.193108+0000","last_became_peered":"2024-05-09T02:07:48.193108+0000","last_unstale":"2024-05-09T02:08:25.313557+0000","last_undegraded":"2024-05-09T02:08:25.313557+0000","last_fullsized":"2024-05-09T02:08:25.313557+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:11.895058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.335525+0000","last_change":"2024-05-09T02:08:25.335525+0000","last_active":"2024-05-09T02:08:25.335525+0000","last_peered":"2024-05-09T02:08:25.335525+0000","last_clean":"2024-05-09T02:08:25.335525+0000","last_became_active":"2024-05-09T02:08:25.319746+0000","last_became_peered":"2024-05-09T02:08:25.319746+0000","last_unstale":"2024-05-09T02:08:25.335525+0000","last_undegraded":"2024-05-09T02:08:25.335525+0000","last_fullsized":"2024-05-09T02:08:25.335525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:41.774066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317433+0000","last_change":"2024-05-09T02:07:48.196125+0000","last_active":"2024-05-09T02:08:25.317433+0000","last_peered":"2024-05-09T02:08:25.317433+0000","last_clean":"2024-05-09T02:08:25.317433+0000","last_became_active":"2024-05-09T02:07:48.195737+0000","last_became_peered":"2024-05-09T02:07:48.195737+0000","last_unstale":"2024-05-09T02:08:25.317433+0000","last_undegraded":"2024-05-09T02:08:25.317433+0000","last_fullsized":"2024-05-09T02:08:25.317433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:43.264720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310367+0000","last_change":"2024-05-09T02:07:45.163791+0000","last_active":"2024-05-09T02:08:25.310367+0000","last_peered":"2024-05-09T02:08:25.310367+0000","last_clean":"2024-05-09T02:08:25.310367+0000","last_became_active":"2024-05-09T02:07:45.163129+0000","last_became_peered":"2024-05-09T02:07:45.163129+0000","last_unstale":"2024-05-09T02:08:25.310367+0000","last_undegraded":"2024-05-09T02:08:25.310367+0000","last_fullsized":"2024-05-09T02:08:25.310367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:48.562850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310748+0000","last_change":"2024-05-09T02:07:48.197782+0000","last_active":"2024-05-09T02:08:25.310748+0000","last_peered":"2024-05-09T02:08:25.310748+0000","last_clean":"2024-05-09T02:08:25.310748+0000","last_became_active":"2024-05-09T02:07:48.197353+0000","last_became_peered":"2024-05-09T02:07:48.197353+0000","last_unstale":"2024-05-09T02:08:25.310748+0000","last_undegraded":"2024-05-09T02:08:25.310748+0000","last_fullsized":"2024-05-09T02:08:25.310748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:17.053571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315220+0000","last_change":"2024-05-09T02:07:45.187028+0000","last_active":"2024-05-09T02:08:25.315220+0000","last_peered":"2024-05-09T02:08:25.315220+0000","last_clean":"2024-05-09T02:08:25.315220+0000","last_became_active":"2024-05-09T02:07:45.186639+0000","last_became_peered":"2024-05-09T02:07:45.186639+0000","last_unstale":"2024-05-09T02:08:25.315220+0000","last_undegraded":"2024-05-09T02:08:25.315220+0000","last_fullsized":"2024-05-09T02:08:25.315220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:35.834374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.308184+0000","last_change":"2024-05-09T02:07:48.207150+0000","last_active":"2024-05-09T02:08:25.308184+0000","last_peered":"2024-05-09T02:08:25.308184+0000","last_clean":"2024-05-09T02:08:25.308184+0000","last_became_active":"2024-05-09T02:07:48.206797+0000","last_became_peered":"2024-05-09T02:07:48.206797+0000","last_unstale":"2024-05-09T02:08:25.308184+0000","last_undegraded":"2024-05-09T02:08:25.308184+0000","last_fullsized":"2024-05-09T02:08:25.308184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:37.328133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":59,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:33.306521+0000","last_change":"2024-05-09T02:07:45.165422+0000","last_active":"2024-05-09T02:08:33.306521+0000","last_peered":"2024-05-09T02:08:33.306521+0000","last_clean":"2024-05-09T02:08:33.306521+0000","last_became_active":"2024-05-09T02:07:45.164998+0000","last_became_peered":"2024-05-09T02:07:45.164998+0000","last_unstale":"2024-05-09T02:08:33.306521+0000","last_undegraded":"2024-05-09T02:08:33.306521+0000","last_fullsized":"2024-05-09T02:08:33.306521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:49.124323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.307961+0000","last_change":"2024-05-09T02:07:48.206538+0000","last_active":"2024-05-09T02:08:25.307961+0000","last_peered":"2024-05-09T02:08:25.307961+0000","last_clean":"2024-05-09T02:08:25.307961+0000","last_became_active":"2024-05-09T02:07:48.206061+0000","last_became_peered":"2024-05-09T02:07:48.206061+0000","last_unstale":"2024-05-09T02:08:25.307961+0000","last_undegraded":"2024-05-09T02:08:25.307961+0000","last_fullsized":"2024-05-09T02:08:25.307961+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:24.075091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318188+0000","last_change":"2024-05-09T02:07:48.202235+0000","last_active":"2024-05-09T02:08:25.318188+0000","last_peered":"2024-05-09T02:08:25.318188+0000","last_clean":"2024-05-09T02:08:25.318188+0000","last_became_active":"2024-05-09T02:07:48.200891+0000","last_became_peered":"2024-05-09T02:07:48.200891+0000","last_unstale":"2024-05-09T02:08:25.318188+0000","last_undegraded":"2024-05-09T02:08:25.318188+0000","last_fullsized":"2024-05-09T02:08:25.318188+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:39.061267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":58,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:32.304903+0000","last_change":"2024-05-09T02:07:45.184582+0000","last_active":"2024-05-09T02:08:32.304903+0000","last_peered":"2024-05-09T02:08:32.304903+0000","last_clean":"2024-05-09T02:08:32.304903+0000","last_became_active":"2024-05-09T02:07:45.184187+0000","last_became_peered":"2024-05-09T02:07:45.184187+0000","last_unstale":"2024-05-09T02:08:32.304903+0000","last_undegraded":"2024-05-09T02:08:32.304903+0000","last_fullsized":"2024-05-09T02:08:32.304903+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:11.343797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314798+0000","last_change":"2024-05-09T02:07:48.199247+0000","last_active":"2024-05-09T02:08:25.314798+0000","last_peered":"2024-05-09T02:08:25.314798+0000","last_clean":"2024-05-09T02:08:25.314798+0000","last_became_active":"2024-05-09T02:07:48.198121+0000","last_became_peered":"2024-05-09T02:07:48.198121+0000","last_unstale":"2024-05-09T02:08:25.314798+0000","last_undegraded":"2024-05-09T02:08:25.314798+0000","last_fullsized":"2024-05-09T02:08:25.314798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:45:12.885616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317973+0000","last_change":"2024-05-09T02:07:45.163750+0000","last_active":"2024-05-09T02:08:25.317973+0000","last_peered":"2024-05-09T02:08:25.317973+0000","last_clean":"2024-05-09T02:08:25.317973+0000","last_became_active":"2024-05-09T02:07:45.163039+0000","last_became_peered":"2024-05-09T02:07:45.163039+0000","last_unstale":"2024-05-09T02:08:25.317973+0000","last_undegraded":"2024-05-09T02:08:25.317973+0000","last_fullsized":"2024-05-09T02:08:25.317973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:06.794514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313720+0000","last_change":"2024-05-09T02:06:20.334734+0000","last_active":"2024-05-09T02:08:25.313720+0000","last_peered":"2024-05-09T02:08:25.313720+0000","last_clean":"2024-05-09T02:08:25.313720+0000","last_became_active":"2024-05-09T02:06:20.329198+0000","last_became_peered":"2024-05-09T02:06:20.329198+0000","last_unstale":"2024-05-09T02:08:25.313720+0000","last_undegraded":"2024-05-09T02:08:25.313720+0000","last_fullsized":"2024-05-09T02:08:25.313720+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:02:50.632016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:02:50.632016+0000","last_clean_scrub_stamp":"2024-05-09T02:02:50.632016+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:08:15.495982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.321187+0000","last_change":"2024-05-09T02:08:25.321187+0000","last_active":"2024-05-09T02:08:25.321187+0000","last_peered":"2024-05-09T02:08:25.321187+0000","last_clean":"2024-05-09T02:08:25.321187+0000","last_became_active":"2024-05-09T02:08:25.320693+0000","last_became_peered":"2024-05-09T02:08:25.320693+0000","last_unstale":"2024-05-09T02:08:25.321187+0000","last_undegraded":"2024-05-09T02:08:25.321187+0000","last_fullsized":"2024-05-09T02:08:25.321187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:00.176016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314395+0000","last_change":"2024-05-09T02:07:45.158053+0000","last_active":"2024-05-09T02:08:25.314395+0000","last_peered":"2024-05-09T02:08:25.314395+0000","last_clean":"2024-05-09T02:08:25.314395+0000","last_became_active":"2024-05-09T02:07:45.157619+0000","last_became_peered":"2024-05-09T02:07:45.157619+0000","last_unstale":"2024-05-09T02:08:25.314395+0000","last_undegraded":"2024-05-09T02:08:25.314395+0000","last_fullsized":"2024-05-09T02:08:25.314395+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:01.618475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.341770+0000","last_change":"2024-05-09T02:08:25.341377+0000","last_active":"2024-05-09T02:08:25.341770+0000","last_peered":"2024-05-09T02:08:25.341770+0000","last_clean":"2024-05-09T02:08:25.341770+0000","last_became_active":"2024-05-09T02:08:25.318683+0000","last_became_peered":"2024-05-09T02:08:25.318683+0000","last_unstale":"2024-05-09T02:08:25.341770+0000","last_undegraded":"2024-05-09T02:08:25.341770+0000","last_fullsized":"2024-05-09T02:08:25.341770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:12.456338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309370+0000","last_change":"2024-05-09T02:07:48.186892+0000","last_active":"2024-05-09T02:08:25.309370+0000","last_peered":"2024-05-09T02:08:25.309370+0000","last_clean":"2024-05-09T02:08:25.309370+0000","last_became_active":"2024-05-09T02:07:48.186543+0000","last_became_peered":"2024-05-09T02:07:48.186543+0000","last_unstale":"2024-05-09T02:08:25.309370+0000","last_undegraded":"2024-05-09T02:08:25.309370+0000","last_fullsized":"2024-05-09T02:08:25.309370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:01.210504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'14","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:28.204532+0000","last_change":"2024-05-09T02:07:45.158007+0000","last_active":"2024-05-09T02:08:28.204532+0000","last_peered":"2024-05-09T02:08:28.204532+0000","last_clean":"2024-05-09T02:08:28.204532+0000","last_became_active":"2024-05-09T02:07:45.157612+0000","last_became_peered":"2024-05-09T02:07:45.157612+0000","last_unstale":"2024-05-09T02:08:28.204532+0000","last_undegraded":"2024-05-09T02:08:28.204532+0000","last_fullsized":"2024-05-09T02:08:28.204532+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:01.526756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314093+0000","last_change":"2024-05-09T02:07:48.186065+0000","last_active":"2024-05-09T02:08:25.314093+0000","last_peered":"2024-05-09T02:08:25.314093+0000","last_clean":"2024-05-09T02:08:25.314093+0000","last_became_active":"2024-05-09T02:07:48.185746+0000","last_became_peered":"2024-05-09T02:07:48.185746+0000","last_unstale":"2024-05-09T02:08:25.314093+0000","last_undegraded":"2024-05-09T02:08:25.314093+0000","last_fullsized":"2024-05-09T02:08:25.314093+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:09.570261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317499+0000","last_change":"2024-05-09T02:07:46.005439+0000","last_active":"2024-05-09T02:08:25.317499+0000","last_peered":"2024-05-09T02:08:25.317499+0000","last_clean":"2024-05-09T02:08:25.317499+0000","last_became_active":"2024-05-09T02:07:46.005087+0000","last_became_peered":"2024-05-09T02:07:46.005087+0000","last_unstale":"2024-05-09T02:08:25.317499+0000","last_undegraded":"2024-05-09T02:08:25.317499+0000","last_fullsized":"2024-05-09T02:08:25.317499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:08.543310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315347+0000","last_change":"2024-05-09T02:07:48.194062+0000","last_active":"2024-05-09T02:08:25.315347+0000","last_peered":"2024-05-09T02:08:25.315347+0000","last_clean":"2024-05-09T02:08:25.315347+0000","last_became_active":"2024-05-09T02:07:48.193712+0000","last_became_peered":"2024-05-09T02:07:48.193712+0000","last_unstale":"2024-05-09T02:08:25.315347+0000","last_undegraded":"2024-05-09T02:08:25.315347+0000","last_fullsized":"2024-05-09T02:08:25.315347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:48.394898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315411+0000","last_change":"2024-05-09T02:07:45.184821+0000","last_active":"2024-05-09T02:08:25.315411+0000","last_peered":"2024-05-09T02:08:25.315411+0000","last_clean":"2024-05-09T02:08:25.315411+0000","last_became_active":"2024-05-09T02:07:45.184437+0000","last_became_peered":"2024-05-09T02:07:45.184437+0000","last_unstale":"2024-05-09T02:08:25.315411+0000","last_undegraded":"2024-05-09T02:08:25.315411+0000","last_fullsized":"2024-05-09T02:08:25.315411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:30.864171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309993+0000","last_change":"2024-05-09T02:07:48.187635+0000","last_active":"2024-05-09T02:08:25.309993+0000","last_peered":"2024-05-09T02:08:25.309993+0000","last_clean":"2024-05-09T02:08:25.309993+0000","last_became_active":"2024-05-09T02:07:48.187071+0000","last_became_peered":"2024-05-09T02:07:48.187071+0000","last_unstale":"2024-05-09T02:08:25.309993+0000","last_undegraded":"2024-05-09T02:08:25.309993+0000","last_fullsized":"2024-05-09T02:08:25.309993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:10.716715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315158+0000","last_change":"2024-05-09T02:07:45.162166+0000","last_active":"2024-05-09T02:08:25.315158+0000","last_peered":"2024-05-09T02:08:25.315158+0000","last_clean":"2024-05-09T02:08:25.315158+0000","last_became_active":"2024-05-09T02:07:45.161787+0000","last_became_peered":"2024-05-09T02:07:45.161787+0000","last_unstale":"2024-05-09T02:08:25.315158+0000","last_undegraded":"2024-05-09T02:08:25.315158+0000","last_fullsized":"2024-05-09T02:08:25.315158+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:17.925214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310341+0000","last_change":"2024-05-09T02:07:48.202980+0000","last_active":"2024-05-09T02:08:25.310341+0000","last_peered":"2024-05-09T02:08:25.310341+0000","last_clean":"2024-05-09T02:08:25.310341+0000","last_became_active":"2024-05-09T02:07:48.202609+0000","last_became_peered":"2024-05-09T02:07:48.202609+0000","last_unstale":"2024-05-09T02:08:25.310341+0000","last_undegraded":"2024-05-09T02:08:25.310341+0000","last_fullsized":"2024-05-09T02:08:25.310341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:59.993451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.326699+0000","last_change":"2024-05-09T02:08:25.326142+0000","last_active":"2024-05-09T02:08:25.326699+0000","last_peered":"2024-05-09T02:08:25.326699+0000","last_clean":"2024-05-09T02:08:25.326699+0000","last_became_active":"2024-05-09T02:08:25.317666+0000","last_became_peered":"2024-05-09T02:08:25.317666+0000","last_unstale":"2024-05-09T02:08:25.326699+0000","last_undegraded":"2024-05-09T02:08:25.326699+0000","last_fullsized":"2024-05-09T02:08:25.326699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:26.178211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315560+0000","last_change":"2024-05-09T02:07:48.202219+0000","last_active":"2024-05-09T02:08:25.315560+0000","last_peered":"2024-05-09T02:08:25.315560+0000","last_clean":"2024-05-09T02:08:25.315560+0000","last_became_active":"2024-05-09T02:07:48.201819+0000","last_became_peered":"2024-05-09T02:07:48.201819+0000","last_unstale":"2024-05-09T02:08:25.315560+0000","last_undegraded":"2024-05-09T02:08:25.315560+0000","last_fullsized":"2024-05-09T02:08:25.315560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:51.573359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311162+0000","last_change":"2024-05-09T02:07:45.162228+0000","last_active":"2024-05-09T02:08:25.311162+0000","last_peered":"2024-05-09T02:08:25.311162+0000","last_clean":"2024-05-09T02:08:25.311162+0000","last_became_active":"2024-05-09T02:07:45.161847+0000","last_became_peered":"2024-05-09T02:07:45.161847+0000","last_unstale":"2024-05-09T02:08:25.311162+0000","last_undegraded":"2024-05-09T02:08:25.311162+0000","last_fullsized":"2024-05-09T02:08:25.311162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:47.459709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314196+0000","last_change":"2024-05-09T02:07:48.201401+0000","last_active":"2024-05-09T02:08:25.314196+0000","last_peered":"2024-05-09T02:08:25.314196+0000","last_clean":"2024-05-09T02:08:25.314196+0000","last_became_active":"2024-05-09T02:07:48.201040+0000","last_became_peered":"2024-05-09T02:07:48.201040+0000","last_unstale":"2024-05-09T02:08:25.314196+0000","last_undegraded":"2024-05-09T02:08:25.314196+0000","last_fullsized":"2024-05-09T02:08:25.314196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:58.038690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":60,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:33.308040+0000","last_change":"2024-05-09T02:07:46.005273+0000","last_active":"2024-05-09T02:08:33.308040+0000","last_peered":"2024-05-09T02:08:33.308040+0000","last_clean":"2024-05-09T02:08:33.308040+0000","last_became_active":"2024-05-09T02:07:46.004732+0000","last_became_peered":"2024-05-09T02:07:46.004732+0000","last_unstale":"2024-05-09T02:08:33.308040+0000","last_undegraded":"2024-05-09T02:08:33.308040+0000","last_fullsized":"2024-05-09T02:08:33.308040+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:55.576955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315747+0000","last_change":"2024-05-09T02:07:48.197032+0000","last_active":"2024-05-09T02:08:25.315747+0000","last_peered":"2024-05-09T02:08:25.315747+0000","last_clean":"2024-05-09T02:08:25.315747+0000","last_became_active":"2024-05-09T02:07:48.196553+0000","last_became_peered":"2024-05-09T02:07:48.196553+0000","last_unstale":"2024-05-09T02:08:25.315747+0000","last_undegraded":"2024-05-09T02:08:25.315747+0000","last_fullsized":"2024-05-09T02:08:25.315747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:50:25.081024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318767+0000","last_change":"2024-05-09T02:07:45.166456+0000","last_active":"2024-05-09T02:08:25.318767+0000","last_peered":"2024-05-09T02:08:25.318767+0000","last_clean":"2024-05-09T02:08:25.318767+0000","last_became_active":"2024-05-09T02:07:45.166132+0000","last_became_peered":"2024-05-09T02:07:45.166132+0000","last_unstale":"2024-05-09T02:08:25.318767+0000","last_undegraded":"2024-05-09T02:08:25.318767+0000","last_fullsized":"2024-05-09T02:08:25.318767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:15.701802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.324526+0000","last_change":"2024-05-09T02:08:25.323303+0000","last_active":"2024-05-09T02:08:25.324526+0000","last_peered":"2024-05-09T02:08:25.324526+0000","last_clean":"2024-05-09T02:08:25.324526+0000","last_became_active":"2024-05-09T02:08:25.321752+0000","last_became_peered":"2024-05-09T02:08:25.321752+0000","last_unstale":"2024-05-09T02:08:25.324526+0000","last_undegraded":"2024-05-09T02:08:25.324526+0000","last_fullsized":"2024-05-09T02:08:25.324526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:17.791737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309540+0000","last_change":"2024-05-09T02:07:45.155480+0000","last_active":"2024-05-09T02:08:25.309540+0000","last_peered":"2024-05-09T02:08:25.309540+0000","last_clean":"2024-05-09T02:08:25.309540+0000","last_became_active":"2024-05-09T02:07:45.155085+0000","last_became_peered":"2024-05-09T02:07:45.155085+0000","last_unstale":"2024-05-09T02:08:25.309540+0000","last_undegraded":"2024-05-09T02:08:25.309540+0000","last_fullsized":"2024-05-09T02:08:25.309540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:05.547857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314367+0000","last_change":"2024-05-09T02:07:48.194480+0000","last_active":"2024-05-09T02:08:25.314367+0000","last_peered":"2024-05-09T02:08:25.314367+0000","last_clean":"2024-05-09T02:08:25.314367+0000","last_became_active":"2024-05-09T02:07:48.194136+0000","last_became_peered":"2024-05-09T02:07:48.194136+0000","last_unstale":"2024-05-09T02:08:25.314367+0000","last_undegraded":"2024-05-09T02:08:25.314367+0000","last_fullsized":"2024-05-09T02:08:25.314367+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:09.177762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310492+0000","last_change":"2024-05-09T02:07:48.195152+0000","last_active":"2024-05-09T02:08:25.310492+0000","last_peered":"2024-05-09T02:08:25.310492+0000","last_clean":"2024-05-09T02:08:25.310492+0000","last_became_active":"2024-05-09T02:07:48.194779+0000","last_became_peered":"2024-05-09T02:07:48.194779+0000","last_unstale":"2024-05-09T02:08:25.310492+0000","last_undegraded":"2024-05-09T02:08:25.310492+0000","last_fullsized":"2024-05-09T02:08:25.310492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:39.345452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.322235+0000","last_change":"2024-05-09T02:08:25.322235+0000","last_active":"2024-05-09T02:08:25.322235+0000","last_peered":"2024-05-09T02:08:25.322235+0000","last_clean":"2024-05-09T02:08:25.322235+0000","last_became_active":"2024-05-09T02:08:25.321674+0000","last_became_peered":"2024-05-09T02:08:25.321674+0000","last_unstale":"2024-05-09T02:08:25.322235+0000","last_undegraded":"2024-05-09T02:08:25.322235+0000","last_fullsized":"2024-05-09T02:08:25.322235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:52.671669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315466+0000","last_change":"2024-05-09T02:07:48.195710+0000","last_active":"2024-05-09T02:08:25.315466+0000","last_peered":"2024-05-09T02:08:25.315466+0000","last_clean":"2024-05-09T02:08:25.315466+0000","last_became_active":"2024-05-09T02:07:48.195332+0000","last_became_peered":"2024-05-09T02:07:48.195332+0000","last_unstale":"2024-05-09T02:08:25.315466+0000","last_undegraded":"2024-05-09T02:08:25.315466+0000","last_fullsized":"2024-05-09T02:08:25.315466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:12:17.904166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315787+0000","last_change":"2024-05-09T02:07:48.200791+0000","last_active":"2024-05-09T02:08:25.315787+0000","last_peered":"2024-05-09T02:08:25.315787+0000","last_clean":"2024-05-09T02:08:25.315787+0000","last_became_active":"2024-05-09T02:07:48.199916+0000","last_became_peered":"2024-05-09T02:07:48.199916+0000","last_unstale":"2024-05-09T02:08:25.315787+0000","last_undegraded":"2024-05-09T02:08:25.315787+0000","last_fullsized":"2024-05-09T02:08:25.315787+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:32.692375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309571+0000","last_change":"2024-05-09T02:07:48.191662+0000","last_active":"2024-05-09T02:08:25.309571+0000","last_peered":"2024-05-09T02:08:25.309571+0000","last_clean":"2024-05-09T02:08:25.309571+0000","last_became_active":"2024-05-09T02:07:48.191336+0000","last_became_peered":"2024-05-09T02:07:48.191336+0000","last_unstale":"2024-05-09T02:08:25.309571+0000","last_undegraded":"2024-05-09T02:08:25.309571+0000","last_fullsized":"2024-05-09T02:08:25.309571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:00.733611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311462+0000","last_change":"2024-05-09T02:07:48.195182+0000","last_active":"2024-05-09T02:08:25.311462+0000","last_peered":"2024-05-09T02:08:25.311462+0000","last_clean":"2024-05-09T02:08:25.311462+0000","last_became_active":"2024-05-09T02:07:48.194828+0000","last_became_peered":"2024-05-09T02:07:48.194828+0000","last_unstale":"2024-05-09T02:08:25.311462+0000","last_undegraded":"2024-05-09T02:08:25.311462+0000","last_fullsized":"2024-05-09T02:08:25.311462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:33.427409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315738+0000","last_change":"2024-05-09T02:07:48.199688+0000","last_active":"2024-05-09T02:08:25.315738+0000","last_peered":"2024-05-09T02:08:25.315738+0000","last_clean":"2024-05-09T02:08:25.315738+0000","last_became_active":"2024-05-09T02:07:48.199326+0000","last_became_peered":"2024-05-09T02:07:48.199326+0000","last_unstale":"2024-05-09T02:08:25.315738+0000","last_undegraded":"2024-05-09T02:08:25.315738+0000","last_fullsized":"2024-05-09T02:08:25.315738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:30.662738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313935+0000","last_change":"2024-05-09T02:07:48.190727+0000","last_active":"2024-05-09T02:08:25.313935+0000","last_peered":"2024-05-09T02:08:25.313935+0000","last_clean":"2024-05-09T02:08:25.313935+0000","last_became_active":"2024-05-09T02:07:48.190397+0000","last_became_peered":"2024-05-09T02:07:48.190397+0000","last_unstale":"2024-05-09T02:08:25.313935+0000","last_undegraded":"2024-05-09T02:08:25.313935+0000","last_fullsized":"2024-05-09T02:08:25.313935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:42.010669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317969+0000","last_change":"2024-05-09T02:07:48.200444+0000","last_active":"2024-05-09T02:08:25.317969+0000","last_peered":"2024-05-09T02:08:25.317969+0000","last_clean":"2024-05-09T02:08:25.317969+0000","last_became_active":"2024-05-09T02:07:48.200031+0000","last_became_peered":"2024-05-09T02:07:48.200031+0000","last_unstale":"2024-05-09T02:08:25.317969+0000","last_undegraded":"2024-05-09T02:08:25.317969+0000","last_fullsized":"2024-05-09T02:08:25.317969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:30.269792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315072+0000","last_change":"2024-05-09T02:07:48.203717+0000","last_active":"2024-05-09T02:08:25.315072+0000","last_peered":"2024-05-09T02:08:25.315072+0000","last_clean":"2024-05-09T02:08:25.315072+0000","last_became_active":"2024-05-09T02:07:48.203347+0000","last_became_peered":"2024-05-09T02:07:48.203347+0000","last_unstale":"2024-05-09T02:08:25.315072+0000","last_undegraded":"2024-05-09T02:08:25.315072+0000","last_fullsized":"2024-05-09T02:08:25.315072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:17.391504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.323029+0000","last_change":"2024-05-09T02:08:25.323029+0000","last_active":"2024-05-09T02:08:25.323029+0000","last_peered":"2024-05-09T02:08:25.323029+0000","last_clean":"2024-05-09T02:08:25.323029+0000","last_became_active":"2024-05-09T02:08:25.322524+0000","last_became_peered":"2024-05-09T02:08:25.322524+0000","last_unstale":"2024-05-09T02:08:25.323029+0000","last_undegraded":"2024-05-09T02:08:25.323029+0000","last_fullsized":"2024-05-09T02:08:25.323029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:25.017511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309673+0000","last_change":"2024-05-09T02:07:48.187802+0000","last_active":"2024-05-09T02:08:25.309673+0000","last_peered":"2024-05-09T02:08:25.309673+0000","last_clean":"2024-05-09T02:08:25.309673+0000","last_became_active":"2024-05-09T02:07:48.187478+0000","last_became_peered":"2024-05-09T02:07:48.187478+0000","last_unstale":"2024-05-09T02:08:25.309673+0000","last_undegraded":"2024-05-09T02:08:25.309673+0000","last_fullsized":"2024-05-09T02:08:25.309673+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:12.981386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310421+0000","last_change":"2024-05-09T02:07:48.192117+0000","last_active":"2024-05-09T02:08:25.310421+0000","last_peered":"2024-05-09T02:08:25.310421+0000","last_clean":"2024-05-09T02:08:25.310421+0000","last_became_active":"2024-05-09T02:07:48.191726+0000","last_became_peered":"2024-05-09T02:07:48.191726+0000","last_unstale":"2024-05-09T02:08:25.310421+0000","last_undegraded":"2024-05-09T02:08:25.310421+0000","last_fullsized":"2024-05-09T02:08:25.310421+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:07.735582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280595,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62394,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":299,"num_read_kb":220,"num_write":241,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":187182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678040,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":429664,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874268,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1076553,"data_compressed":41450,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037795,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":909312,"data_stored":429599,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234024,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":429748,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430253,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":1022378,"data_compressed":41450,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":429199,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":485693,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":430538,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":485801,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":428907,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":429563,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":1021665,"data_compressed":41450,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":57333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":57708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T02:08:37.933 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T02:08:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:38.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:38 smithi019 ceph-mon[30151]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:08:38.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:38 smithi059 ceph-mon[34781]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:08:38.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:38 smithi196 ceph-mon[34718]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:08:39.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:39 smithi019 ceph-mon[30151]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 10 op/s; 18 B/s, 1 objects/s recovering 2024-05-09T02:08:39.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:39 smithi059 ceph-mon[34781]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 10 op/s; 18 B/s, 1 objects/s recovering 2024-05-09T02:08:39.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:39 smithi196 ceph-mon[34718]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 10 op/s; 18 B/s, 1 objects/s recovering 2024-05-09T02:08:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.904+0000 7fa9f8ecb700 1 -- 172.21.15.19:0/207572965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f4107df0 msgr2=0x7fa9f4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.904+0000 7fa9f8ecb700 1 --2- 172.21.15.19:0/207572965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f4107df0 0x7fa9f4108200 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80099c0 tx=0x7fa9e8009cd0 comp rx=0 tx=0).stop 2024-05-09T02:08:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.904+0000 7fa9f8ecb700 1 -- 172.21.15.19:0/207572965 shutdown_connections 2024-05-09T02:08:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.904+0000 7fa9f8ecb700 1 --2- 172.21.15.19:0/207572965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f4108740 0x7fa9f41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.904+0000 7fa9f8ecb700 1 --2- 172.21.15.19:0/207572965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9f41034c0 0x7fa9f41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.904+0000 7fa9f8ecb700 1 --2- 172.21.15.19:0/207572965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f4107df0 0x7fa9f4108200 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.904+0000 7fa9f8ecb700 1 -- 172.21.15.19:0/207572965 >> 172.21.15.19:0/207572965 conn(0x7fa9f40fedf0 msgr2=0x7fa9f4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.904+0000 7fa9f8ecb700 1 -- 172.21.15.19:0/207572965 shutdown_connections 2024-05-09T02:08:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.904+0000 7fa9f8ecb700 1 -- 172.21.15.19:0/207572965 wait complete. 2024-05-09T02:08:39.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.905+0000 7fa9f8ecb700 1 Processor -- start 2024-05-09T02:08:39.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.905+0000 7fa9f8ecb700 1 -- start start 2024-05-09T02:08:39.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.905+0000 7fa9f8ecb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9f41034c0 0x7fa9f419af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:39.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.905+0000 7fa9f8ecb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f4107df0 0x7fa9f419b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.905+0000 7fa9f8ecb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f4108740 0x7fa9f419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.905+0000 7fa9f8ecb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f4113370 con 0x7fa9f4107df0 2024-05-09T02:08:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.905+0000 7fa9f8ecb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f41131f0 con 0x7fa9f4108740 2024-05-09T02:08:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.905+0000 7fa9f8ecb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f41134f0 con 0x7fa9f41034c0 2024-05-09T02:08:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.905+0000 7fa9f1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f4107df0 0x7fa9f419b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.905+0000 7fa9f1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f4107df0 0x7fa9f419b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47212/0 (socket says 172.21.15.19:47212) 2024-05-09T02:08:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.905+0000 7fa9f1d9b700 1 -- 172.21.15.19:0/3310883207 learned_addr learned my addr 172.21.15.19:0/3310883207 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.906+0000 7fa9f2d9d700 1 --2- 172.21.15.19:0/3310883207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f4108740 0x7fa9f419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.906+0000 7fa9f259c700 1 --2- 172.21.15.19:0/3310883207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9f41034c0 0x7fa9f419af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.906+0000 7fa9f1d9b700 1 -- 172.21.15.19:0/3310883207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9f41034c0 msgr2=0x7fa9f419af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.906+0000 7fa9f1d9b700 1 --2- 172.21.15.19:0/3310883207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9f41034c0 0x7fa9f419af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.906+0000 7fa9f1d9b700 1 -- 172.21.15.19:0/3310883207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f4108740 msgr2=0x7fa9f419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.906+0000 7fa9f1d9b700 1 --2- 172.21.15.19:0/3310883207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f4108740 0x7fa9f419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:39.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.906+0000 7fa9f1d9b700 1 -- 172.21.15.19:0/3310883207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f419fde0 con 0x7fa9f4107df0 2024-05-09T02:08:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.906+0000 7fa9f1d9b700 1 --2- 172.21.15.19:0/3310883207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f4107df0 0x7fa9f419b4b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80045e0 tx=0x7fa9e8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.906+0000 7fa9e37fe700 1 -- 172.21.15.19:0/3310883207 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8026070 con 0x7fa9f4107df0 2024-05-09T02:08:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.906+0000 7fa9e37fe700 1 -- 172.21.15.19:0/3310883207 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e800f460 con 0x7fa9f4107df0 2024-05-09T02:08:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.906+0000 7fa9f8ecb700 1 -- 172.21.15.19:0/3310883207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f41a0070 con 0x7fa9f4107df0 2024-05-09T02:08:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.906+0000 7fa9e37fe700 1 -- 172.21.15.19:0/3310883207 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8009e60 con 0x7fa9f4107df0 2024-05-09T02:08:39.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.906+0000 7fa9f8ecb700 1 -- 172.21.15.19:0/3310883207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f41a0550 con 0x7fa9f4107df0 2024-05-09T02:08:39.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.908+0000 7fa9f8ecb700 1 -- 172.21.15.19:0/3310883207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f40512c0 con 0x7fa9f4107df0 2024-05-09T02:08:39.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.909+0000 7fa9e37fe700 1 -- 172.21.15.19:0/3310883207 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa9e8029070 con 0x7fa9f4107df0 2024-05-09T02:08:39.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.910+0000 7fa9e37fe700 1 --2- 172.21.15.19:0/3310883207 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa9dc0b2910 0x7fa9dc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:39.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.910+0000 7fa9f259c700 1 --2- 172.21.15.19:0/3310883207 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa9dc0b2910 0x7fa9dc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:39.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.910+0000 7fa9e37fe700 1 -- 172.21.15.19:0/3310883207 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fa9e80e2c50 con 0x7fa9f4107df0 2024-05-09T02:08:39.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.910+0000 7fa9f259c700 1 --2- 172.21.15.19:0/3310883207 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa9dc0b2910 0x7fa9dc0b4e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009bf0 tx=0x7fa9e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:39.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:39.915+0000 7fa9e37fe700 1 -- 172.21.15.19:0/3310883207 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9e8013070 con 0x7fa9f4107df0 2024-05-09T02:08:40.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:40.127+0000 7fa9f8ecb700 1 -- 172.21.15.19:0/3310883207 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9f419c270 con 0x7fa9dc0b2910 2024-05-09T02:08:40.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:40.134+0000 7fa9e37fe700 1 -- 172.21.15.19:0/3310883207 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259015 (secure 0 0 0) 0x7fa9f419c270 con 0x7fa9dc0b2910 2024-05-09T02:08:40.141 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:08:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:40.137+0000 7fa9f8ecb700 1 -- 172.21.15.19:0/3310883207 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa9dc0b2910 msgr2=0x7fa9dc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:40.137+0000 7fa9f8ecb700 1 --2- 172.21.15.19:0/3310883207 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa9dc0b2910 0x7fa9dc0b4e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009bf0 tx=0x7fa9e4009260 comp rx=0 tx=0).stop 2024-05-09T02:08:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:40.137+0000 7fa9f8ecb700 1 -- 172.21.15.19:0/3310883207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f4107df0 msgr2=0x7fa9f419b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:40.137+0000 7fa9f8ecb700 1 --2- 172.21.15.19:0/3310883207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f4107df0 0x7fa9f419b4b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80045e0 tx=0x7fa9e8004610 comp rx=0 tx=0).stop 2024-05-09T02:08:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:40.138+0000 7fa9f8ecb700 1 -- 172.21.15.19:0/3310883207 shutdown_connections 2024-05-09T02:08:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:40.138+0000 7fa9f8ecb700 1 --2- 172.21.15.19:0/3310883207 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa9dc0b2910 0x7fa9dc0b4e20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:40.138+0000 7fa9f8ecb700 1 --2- 172.21.15.19:0/3310883207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f4108740 0x7fa9f419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:40.138+0000 7fa9f8ecb700 1 --2- 172.21.15.19:0/3310883207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9f41034c0 0x7fa9f419af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:40.138+0000 7fa9f8ecb700 1 --2- 172.21.15.19:0/3310883207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f4107df0 0x7fa9f419b4b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:40.138+0000 7fa9f8ecb700 1 -- 172.21.15.19:0/3310883207 >> 172.21.15.19:0/3310883207 conn(0x7fa9f40fedf0 msgr2=0x7fa9f4075e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:40.138+0000 7fa9f8ecb700 1 -- 172.21.15.19:0/3310883207 shutdown_connections 2024-05-09T02:08:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:40.138+0000 7fa9f8ecb700 1 -- 172.21.15.19:0/3310883207 wait complete. 2024-05-09T02:08:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-09T02:08:40.817 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-05-09T02:08:40.028996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659241,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":426,"num_read_kb":321,"num_write":370,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2952064,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376728,"kb_used_data":10892,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565096,"statfs":{"total":1151940427776,"available":1151554658304,"internally_reserved":0,"allocated":11153408,"data_stored":7111798,"data_compressed":176253,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19742,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":139,"num_read_kb":110,"num_write":20,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003410"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311161+0000","last_change":"2024-05-09T02:07:48.203957+0000","last_active":"2024-05-09T02:08:25.311161+0000","last_peered":"2024-05-09T02:08:25.311161+0000","last_clean":"2024-05-09T02:08:25.311161+0000","last_became_active":"2024-05-09T02:07:48.203354+0000","last_became_peered":"2024-05-09T02:07:48.203354+0000","last_unstale":"2024-05-09T02:08:25.311161+0000","last_undegraded":"2024-05-09T02:08:25.311161+0000","last_fullsized":"2024-05-09T02:08:25.311161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:06.169693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315459+0000","last_change":"2024-05-09T02:07:48.192885+0000","last_active":"2024-05-09T02:08:25.315459+0000","last_peered":"2024-05-09T02:08:25.315459+0000","last_clean":"2024-05-09T02:08:25.315459+0000","last_became_active":"2024-05-09T02:07:48.192476+0000","last_became_peered":"2024-05-09T02:07:48.192476+0000","last_unstale":"2024-05-09T02:08:25.315459+0000","last_undegraded":"2024-05-09T02:08:25.315459+0000","last_fullsized":"2024-05-09T02:08:25.315459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:33.710309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314654+0000","last_change":"2024-05-09T02:07:48.198684+0000","last_active":"2024-05-09T02:08:25.314654+0000","last_peered":"2024-05-09T02:08:25.314654+0000","last_clean":"2024-05-09T02:08:25.314654+0000","last_became_active":"2024-05-09T02:07:48.198344+0000","last_became_peered":"2024-05-09T02:07:48.198344+0000","last_unstale":"2024-05-09T02:08:25.314654+0000","last_undegraded":"2024-05-09T02:08:25.314654+0000","last_fullsized":"2024-05-09T02:08:25.314654+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:19.990063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317876+0000","last_change":"2024-05-09T02:07:48.189209+0000","last_active":"2024-05-09T02:08:25.317876+0000","last_peered":"2024-05-09T02:08:25.317876+0000","last_clean":"2024-05-09T02:08:25.317876+0000","last_became_active":"2024-05-09T02:07:48.188744+0000","last_became_peered":"2024-05-09T02:07:48.188744+0000","last_unstale":"2024-05-09T02:08:25.317876+0000","last_undegraded":"2024-05-09T02:08:25.317876+0000","last_fullsized":"2024-05-09T02:08:25.317876+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:03:25.216849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311030+0000","last_change":"2024-05-09T02:07:48.187897+0000","last_active":"2024-05-09T02:08:25.311030+0000","last_peered":"2024-05-09T02:08:25.311030+0000","last_clean":"2024-05-09T02:08:25.311030+0000","last_became_active":"2024-05-09T02:07:48.187476+0000","last_became_peered":"2024-05-09T02:07:48.187476+0000","last_unstale":"2024-05-09T02:08:25.311030+0000","last_undegraded":"2024-05-09T02:08:25.311030+0000","last_fullsized":"2024-05-09T02:08:25.311030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:39.366520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.307870+0000","last_change":"2024-05-09T02:07:48.195436+0000","last_active":"2024-05-09T02:08:25.307870+0000","last_peered":"2024-05-09T02:08:25.307870+0000","last_clean":"2024-05-09T02:08:25.307870+0000","last_became_active":"2024-05-09T02:07:48.194982+0000","last_became_peered":"2024-05-09T02:07:48.194982+0000","last_unstale":"2024-05-09T02:08:25.307870+0000","last_undegraded":"2024-05-09T02:08:25.307870+0000","last_fullsized":"2024-05-09T02:08:25.307870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:19.746701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313666+0000","last_change":"2024-05-09T02:07:48.198458+0000","last_active":"2024-05-09T02:08:25.313666+0000","last_peered":"2024-05-09T02:08:25.313666+0000","last_clean":"2024-05-09T02:08:25.313666+0000","last_became_active":"2024-05-09T02:07:48.198089+0000","last_became_peered":"2024-05-09T02:07:48.198089+0000","last_unstale":"2024-05-09T02:08:25.313666+0000","last_undegraded":"2024-05-09T02:08:25.313666+0000","last_fullsized":"2024-05-09T02:08:25.313666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:37:44.999438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314425+0000","last_change":"2024-05-09T02:07:48.200294+0000","last_active":"2024-05-09T02:08:25.314425+0000","last_peered":"2024-05-09T02:08:25.314425+0000","last_clean":"2024-05-09T02:08:25.314425+0000","last_became_active":"2024-05-09T02:07:48.199949+0000","last_became_peered":"2024-05-09T02:07:48.199949+0000","last_unstale":"2024-05-09T02:08:25.314425+0000","last_undegraded":"2024-05-09T02:08:25.314425+0000","last_fullsized":"2024-05-09T02:08:25.314425+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:39.263849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315463+0000","last_change":"2024-05-09T02:07:48.196282+0000","last_active":"2024-05-09T02:08:25.315463+0000","last_peered":"2024-05-09T02:08:25.315463+0000","last_clean":"2024-05-09T02:08:25.315463+0000","last_became_active":"2024-05-09T02:07:48.195887+0000","last_became_peered":"2024-05-09T02:07:48.195887+0000","last_unstale":"2024-05-09T02:08:25.315463+0000","last_undegraded":"2024-05-09T02:08:25.315463+0000","last_fullsized":"2024-05-09T02:08:25.315463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:26.081867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.320636+0000","last_change":"2024-05-09T02:08:25.320636+0000","last_active":"2024-05-09T02:08:25.320636+0000","last_peered":"2024-05-09T02:08:25.320636+0000","last_clean":"2024-05-09T02:08:25.320636+0000","last_became_active":"2024-05-09T02:08:25.320155+0000","last_became_peered":"2024-05-09T02:08:25.320155+0000","last_unstale":"2024-05-09T02:08:25.320636+0000","last_undegraded":"2024-05-09T02:08:25.320636+0000","last_fullsized":"2024-05-09T02:08:25.320636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:55.728773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314733+0000","last_change":"2024-05-09T02:07:48.197649+0000","last_active":"2024-05-09T02:08:25.314733+0000","last_peered":"2024-05-09T02:08:25.314733+0000","last_clean":"2024-05-09T02:08:25.314733+0000","last_became_active":"2024-05-09T02:07:48.196884+0000","last_became_peered":"2024-05-09T02:07:48.196884+0000","last_unstale":"2024-05-09T02:08:25.314733+0000","last_undegraded":"2024-05-09T02:08:25.314733+0000","last_fullsized":"2024-05-09T02:08:25.314733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:23.851216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318021+0000","last_change":"2024-05-09T02:07:48.193568+0000","last_active":"2024-05-09T02:08:25.318021+0000","last_peered":"2024-05-09T02:08:25.318021+0000","last_clean":"2024-05-09T02:08:25.318021+0000","last_became_active":"2024-05-09T02:07:48.193117+0000","last_became_peered":"2024-05-09T02:07:48.193117+0000","last_unstale":"2024-05-09T02:08:25.318021+0000","last_undegraded":"2024-05-09T02:08:25.318021+0000","last_fullsized":"2024-05-09T02:08:25.318021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:34.371246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309662+0000","last_change":"2024-05-09T02:07:48.196314+0000","last_active":"2024-05-09T02:08:25.309662+0000","last_peered":"2024-05-09T02:08:25.309662+0000","last_clean":"2024-05-09T02:08:25.309662+0000","last_became_active":"2024-05-09T02:07:48.195988+0000","last_became_peered":"2024-05-09T02:07:48.195988+0000","last_unstale":"2024-05-09T02:08:25.309662+0000","last_undegraded":"2024-05-09T02:08:25.309662+0000","last_fullsized":"2024-05-09T02:08:25.309662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:08.556166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314018+0000","last_change":"2024-05-09T02:07:48.192974+0000","last_active":"2024-05-09T02:08:25.314018+0000","last_peered":"2024-05-09T02:08:25.314018+0000","last_clean":"2024-05-09T02:08:25.314018+0000","last_became_active":"2024-05-09T02:07:48.192599+0000","last_became_peered":"2024-05-09T02:07:48.192599+0000","last_unstale":"2024-05-09T02:08:25.314018+0000","last_undegraded":"2024-05-09T02:08:25.314018+0000","last_fullsized":"2024-05-09T02:08:25.314018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:37.192219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314434+0000","last_change":"2024-05-09T02:07:48.193208+0000","last_active":"2024-05-09T02:08:25.314434+0000","last_peered":"2024-05-09T02:08:25.314434+0000","last_clean":"2024-05-09T02:08:25.314434+0000","last_became_active":"2024-05-09T02:07:48.192654+0000","last_became_peered":"2024-05-09T02:07:48.192654+0000","last_unstale":"2024-05-09T02:08:25.314434+0000","last_undegraded":"2024-05-09T02:08:25.314434+0000","last_fullsized":"2024-05-09T02:08:25.314434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:55:42.870372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314811+0000","last_change":"2024-05-09T02:07:48.187568+0000","last_active":"2024-05-09T02:08:25.314811+0000","last_peered":"2024-05-09T02:08:25.314811+0000","last_clean":"2024-05-09T02:08:25.314811+0000","last_became_active":"2024-05-09T02:07:48.187159+0000","last_became_peered":"2024-05-09T02:07:48.187159+0000","last_unstale":"2024-05-09T02:08:25.314811+0000","last_undegraded":"2024-05-09T02:08:25.314811+0000","last_fullsized":"2024-05-09T02:08:25.314811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:04.994094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311915+0000","last_change":"2024-05-09T02:07:48.204406+0000","last_active":"2024-05-09T02:08:25.311915+0000","last_peered":"2024-05-09T02:08:25.311915+0000","last_clean":"2024-05-09T02:08:25.311915+0000","last_became_active":"2024-05-09T02:07:48.203592+0000","last_became_peered":"2024-05-09T02:07:48.203592+0000","last_unstale":"2024-05-09T02:08:25.311915+0000","last_undegraded":"2024-05-09T02:08:25.311915+0000","last_fullsized":"2024-05-09T02:08:25.311915+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:21.655807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317492+0000","last_change":"2024-05-09T02:07:48.195326+0000","last_active":"2024-05-09T02:08:25.317492+0000","last_peered":"2024-05-09T02:08:25.317492+0000","last_clean":"2024-05-09T02:08:25.317492+0000","last_became_active":"2024-05-09T02:07:48.194969+0000","last_became_peered":"2024-05-09T02:07:48.194969+0000","last_unstale":"2024-05-09T02:08:25.317492+0000","last_undegraded":"2024-05-09T02:08:25.317492+0000","last_fullsized":"2024-05-09T02:08:25.317492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:00.269753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315957+0000","last_change":"2024-05-09T02:07:48.190706+0000","last_active":"2024-05-09T02:08:25.315957+0000","last_peered":"2024-05-09T02:08:25.315957+0000","last_clean":"2024-05-09T02:08:25.315957+0000","last_became_active":"2024-05-09T02:07:48.190171+0000","last_became_peered":"2024-05-09T02:07:48.190171+0000","last_unstale":"2024-05-09T02:08:25.315957+0000","last_undegraded":"2024-05-09T02:08:25.315957+0000","last_fullsized":"2024-05-09T02:08:25.315957+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:19:31.333618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":71,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:37.310910+0000","last_change":"2024-05-09T02:07:45.158224+0000","last_active":"2024-05-09T02:08:37.310910+0000","last_peered":"2024-05-09T02:08:37.310910+0000","last_clean":"2024-05-09T02:08:37.310910+0000","last_became_active":"2024-05-09T02:07:45.157722+0000","last_became_peered":"2024-05-09T02:07:45.157722+0000","last_unstale":"2024-05-09T02:08:37.310910+0000","last_undegraded":"2024-05-09T02:08:37.310910+0000","last_fullsized":"2024-05-09T02:08:37.310910+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:31.750699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315461+0000","last_change":"2024-05-09T02:07:48.197978+0000","last_active":"2024-05-09T02:08:25.315461+0000","last_peered":"2024-05-09T02:08:25.315461+0000","last_clean":"2024-05-09T02:08:25.315461+0000","last_became_active":"2024-05-09T02:07:48.197077+0000","last_became_peered":"2024-05-09T02:07:48.197077+0000","last_unstale":"2024-05-09T02:08:25.315461+0000","last_undegraded":"2024-05-09T02:08:25.315461+0000","last_fullsized":"2024-05-09T02:08:25.315461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:25.897904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315832+0000","last_change":"2024-05-09T02:07:45.185758+0000","last_active":"2024-05-09T02:08:25.315832+0000","last_peered":"2024-05-09T02:08:25.315832+0000","last_clean":"2024-05-09T02:08:25.315832+0000","last_became_active":"2024-05-09T02:07:45.185411+0000","last_became_peered":"2024-05-09T02:07:45.185411+0000","last_unstale":"2024-05-09T02:08:25.315832+0000","last_undegraded":"2024-05-09T02:08:25.315832+0000","last_fullsized":"2024-05-09T02:08:25.315832+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:04.410523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317321+0000","last_change":"2024-05-09T02:07:48.199794+0000","last_active":"2024-05-09T02:08:25.317321+0000","last_peered":"2024-05-09T02:08:25.317321+0000","last_clean":"2024-05-09T02:08:25.317321+0000","last_became_active":"2024-05-09T02:07:48.199453+0000","last_became_peered":"2024-05-09T02:07:48.199453+0000","last_unstale":"2024-05-09T02:08:25.317321+0000","last_undegraded":"2024-05-09T02:08:25.317321+0000","last_fullsized":"2024-05-09T02:08:25.317321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:09.735830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311011+0000","last_change":"2024-05-09T02:07:45.162221+0000","last_active":"2024-05-09T02:08:25.311011+0000","last_peered":"2024-05-09T02:08:25.311011+0000","last_clean":"2024-05-09T02:08:25.311011+0000","last_became_active":"2024-05-09T02:07:45.161748+0000","last_became_peered":"2024-05-09T02:07:45.161748+0000","last_unstale":"2024-05-09T02:08:25.311011+0000","last_undegraded":"2024-05-09T02:08:25.311011+0000","last_fullsized":"2024-05-09T02:08:25.311011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:38.837411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315182+0000","last_change":"2024-05-09T02:07:48.188741+0000","last_active":"2024-05-09T02:08:25.315182+0000","last_peered":"2024-05-09T02:08:25.315182+0000","last_clean":"2024-05-09T02:08:25.315182+0000","last_became_active":"2024-05-09T02:07:48.188329+0000","last_became_peered":"2024-05-09T02:07:48.188329+0000","last_unstale":"2024-05-09T02:08:25.315182+0000","last_undegraded":"2024-05-09T02:08:25.315182+0000","last_fullsized":"2024-05-09T02:08:25.315182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:46.773336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310908+0000","last_change":"2024-05-09T02:07:45.163573+0000","last_active":"2024-05-09T02:08:25.310908+0000","last_peered":"2024-05-09T02:08:25.310908+0000","last_clean":"2024-05-09T02:08:25.310908+0000","last_became_active":"2024-05-09T02:07:45.163162+0000","last_became_peered":"2024-05-09T02:07:45.163162+0000","last_unstale":"2024-05-09T02:08:25.310908+0000","last_undegraded":"2024-05-09T02:08:25.310908+0000","last_fullsized":"2024-05-09T02:08:25.310908+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:12.356586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318419+0000","last_change":"2024-05-09T02:07:48.203928+0000","last_active":"2024-05-09T02:08:25.318419+0000","last_peered":"2024-05-09T02:08:25.318419+0000","last_clean":"2024-05-09T02:08:25.318419+0000","last_became_active":"2024-05-09T02:07:48.203224+0000","last_became_peered":"2024-05-09T02:07:48.203224+0000","last_unstale":"2024-05-09T02:08:25.318419+0000","last_undegraded":"2024-05-09T02:08:25.318419+0000","last_fullsized":"2024-05-09T02:08:25.318419+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:00.568050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:35.310782+0000","last_change":"2024-05-09T02:07:45.160161+0000","last_active":"2024-05-09T02:08:35.310782+0000","last_peered":"2024-05-09T02:08:35.310782+0000","last_clean":"2024-05-09T02:08:35.310782+0000","last_became_active":"2024-05-09T02:07:45.159118+0000","last_became_peered":"2024-05-09T02:07:45.159118+0000","last_unstale":"2024-05-09T02:08:35.310782+0000","last_undegraded":"2024-05-09T02:08:35.310782+0000","last_fullsized":"2024-05-09T02:08:35.310782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:04.510877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.319860+0000","last_change":"2024-05-09T02:08:25.319860+0000","last_active":"2024-05-09T02:08:25.319860+0000","last_peered":"2024-05-09T02:08:25.319860+0000","last_clean":"2024-05-09T02:08:25.319860+0000","last_became_active":"2024-05-09T02:08:25.319370+0000","last_became_peered":"2024-05-09T02:08:25.319370+0000","last_unstale":"2024-05-09T02:08:25.319860+0000","last_undegraded":"2024-05-09T02:08:25.319860+0000","last_fullsized":"2024-05-09T02:08:25.319860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:38.504057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.327069+0000","last_change":"2024-05-09T02:08:25.327069+0000","last_active":"2024-05-09T02:08:25.327069+0000","last_peered":"2024-05-09T02:08:25.327069+0000","last_clean":"2024-05-09T02:08:25.327069+0000","last_became_active":"2024-05-09T02:08:25.316629+0000","last_became_peered":"2024-05-09T02:08:25.316629+0000","last_unstale":"2024-05-09T02:08:25.327069+0000","last_undegraded":"2024-05-09T02:08:25.327069+0000","last_fullsized":"2024-05-09T02:08:25.327069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:51.028256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318643+0000","last_change":"2024-05-09T02:07:48.201056+0000","last_active":"2024-05-09T02:08:25.318643+0000","last_peered":"2024-05-09T02:08:25.318643+0000","last_clean":"2024-05-09T02:08:25.318643+0000","last_became_active":"2024-05-09T02:07:48.200671+0000","last_became_peered":"2024-05-09T02:07:48.200671+0000","last_unstale":"2024-05-09T02:08:25.318643+0000","last_undegraded":"2024-05-09T02:08:25.318643+0000","last_fullsized":"2024-05-09T02:08:25.318643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:37:16.138170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309296+0000","last_change":"2024-05-09T02:07:45.160417+0000","last_active":"2024-05-09T02:08:25.309296+0000","last_peered":"2024-05-09T02:08:25.309296+0000","last_clean":"2024-05-09T02:08:25.309296+0000","last_became_active":"2024-05-09T02:07:45.160076+0000","last_became_peered":"2024-05-09T02:07:45.160076+0000","last_unstale":"2024-05-09T02:08:25.309296+0000","last_undegraded":"2024-05-09T02:08:25.309296+0000","last_fullsized":"2024-05-09T02:08:25.309296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:53.653494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314286+0000","last_change":"2024-05-09T02:07:48.203267+0000","last_active":"2024-05-09T02:08:25.314286+0000","last_peered":"2024-05-09T02:08:25.314286+0000","last_clean":"2024-05-09T02:08:25.314286+0000","last_became_active":"2024-05-09T02:07:48.202747+0000","last_became_peered":"2024-05-09T02:07:48.202747+0000","last_unstale":"2024-05-09T02:08:25.314286+0000","last_undegraded":"2024-05-09T02:08:25.314286+0000","last_fullsized":"2024-05-09T02:08:25.314286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.954400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317820+0000","last_change":"2024-05-09T02:07:45.164713+0000","last_active":"2024-05-09T02:08:25.317820+0000","last_peered":"2024-05-09T02:08:25.317820+0000","last_clean":"2024-05-09T02:08:25.317820+0000","last_became_active":"2024-05-09T02:07:45.164324+0000","last_became_peered":"2024-05-09T02:07:45.164324+0000","last_unstale":"2024-05-09T02:08:25.317820+0000","last_undegraded":"2024-05-09T02:08:25.317820+0000","last_fullsized":"2024-05-09T02:08:25.317820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:02.481419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.316226+0000","last_change":"2024-05-09T02:07:48.190305+0000","last_active":"2024-05-09T02:08:25.316226+0000","last_peered":"2024-05-09T02:08:25.316226+0000","last_clean":"2024-05-09T02:08:25.316226+0000","last_became_active":"2024-05-09T02:07:48.189971+0000","last_became_peered":"2024-05-09T02:07:48.189971+0000","last_unstale":"2024-05-09T02:08:25.316226+0000","last_undegraded":"2024-05-09T02:08:25.316226+0000","last_fullsized":"2024-05-09T02:08:25.316226+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:23.565092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":94,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:39.315034+0000","last_change":"2024-05-09T02:07:45.164273+0000","last_active":"2024-05-09T02:08:39.315034+0000","last_peered":"2024-05-09T02:08:39.315034+0000","last_clean":"2024-05-09T02:08:39.315034+0000","last_became_active":"2024-05-09T02:07:45.163183+0000","last_became_peered":"2024-05-09T02:07:45.163183+0000","last_unstale":"2024-05-09T02:08:39.315034+0000","last_undegraded":"2024-05-09T02:08:39.315034+0000","last_fullsized":"2024-05-09T02:08:39.315034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:11.395177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315668+0000","last_change":"2024-05-09T02:07:48.202995+0000","last_active":"2024-05-09T02:08:25.315668+0000","last_peered":"2024-05-09T02:08:25.315668+0000","last_clean":"2024-05-09T02:08:25.315668+0000","last_became_active":"2024-05-09T02:07:48.200915+0000","last_became_peered":"2024-05-09T02:07:48.200915+0000","last_unstale":"2024-05-09T02:08:25.315668+0000","last_undegraded":"2024-05-09T02:08:25.315668+0000","last_fullsized":"2024-05-09T02:08:25.315668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:25.555725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315988+0000","last_change":"2024-05-09T02:07:45.166221+0000","last_active":"2024-05-09T02:08:25.315988+0000","last_peered":"2024-05-09T02:08:25.315988+0000","last_clean":"2024-05-09T02:08:25.315988+0000","last_became_active":"2024-05-09T02:07:45.165813+0000","last_became_peered":"2024-05-09T02:07:45.165813+0000","last_unstale":"2024-05-09T02:08:25.315988+0000","last_undegraded":"2024-05-09T02:08:25.315988+0000","last_fullsized":"2024-05-09T02:08:25.315988+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:01.480828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317355+0000","last_change":"2024-05-09T02:07:48.201475+0000","last_active":"2024-05-09T02:08:25.317355+0000","last_peered":"2024-05-09T02:08:25.317355+0000","last_clean":"2024-05-09T02:08:25.317355+0000","last_became_active":"2024-05-09T02:07:48.201129+0000","last_became_peered":"2024-05-09T02:07:48.201129+0000","last_unstale":"2024-05-09T02:08:25.317355+0000","last_undegraded":"2024-05-09T02:08:25.317355+0000","last_fullsized":"2024-05-09T02:08:25.317355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:51.726752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318564+0000","last_change":"2024-05-09T02:07:45.185045+0000","last_active":"2024-05-09T02:08:25.318564+0000","last_peered":"2024-05-09T02:08:25.318564+0000","last_clean":"2024-05-09T02:08:25.318564+0000","last_became_active":"2024-05-09T02:07:45.184592+0000","last_became_peered":"2024-05-09T02:07:45.184592+0000","last_unstale":"2024-05-09T02:08:25.318564+0000","last_undegraded":"2024-05-09T02:08:25.318564+0000","last_fullsized":"2024-05-09T02:08:25.318564+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:56.220022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315185+0000","last_change":"2024-05-09T02:07:48.199873+0000","last_active":"2024-05-09T02:08:25.315185+0000","last_peered":"2024-05-09T02:08:25.315185+0000","last_clean":"2024-05-09T02:08:25.315185+0000","last_became_active":"2024-05-09T02:07:48.199502+0000","last_became_peered":"2024-05-09T02:07:48.199502+0000","last_unstale":"2024-05-09T02:08:25.315185+0000","last_undegraded":"2024-05-09T02:08:25.315185+0000","last_fullsized":"2024-05-09T02:08:25.315185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:15.406904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'32","reported_seq":131,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:39.320168+0000","last_change":"2024-05-09T02:07:45.166167+0000","last_active":"2024-05-09T02:08:39.320168+0000","last_peered":"2024-05-09T02:08:39.320168+0000","last_clean":"2024-05-09T02:08:39.320168+0000","last_became_active":"2024-05-09T02:07:45.165800+0000","last_became_peered":"2024-05-09T02:07:45.165800+0000","last_unstale":"2024-05-09T02:08:39.320168+0000","last_undegraded":"2024-05-09T02:08:39.320168+0000","last_fullsized":"2024-05-09T02:08:39.320168+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:03.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61826,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":73,"num_read_kb":96,"num_write":34,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310662+0000","last_change":"2024-05-09T02:07:48.187809+0000","last_active":"2024-05-09T02:08:25.310662+0000","last_peered":"2024-05-09T02:08:25.310662+0000","last_clean":"2024-05-09T02:08:25.310662+0000","last_became_active":"2024-05-09T02:07:48.186939+0000","last_became_peered":"2024-05-09T02:07:48.186939+0000","last_unstale":"2024-05-09T02:08:25.310662+0000","last_undegraded":"2024-05-09T02:08:25.310662+0000","last_fullsized":"2024-05-09T02:08:25.310662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:13:15.600343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315617+0000","last_change":"2024-05-09T02:07:45.163756+0000","last_active":"2024-05-09T02:08:25.315617+0000","last_peered":"2024-05-09T02:08:25.315617+0000","last_clean":"2024-05-09T02:08:25.315617+0000","last_became_active":"2024-05-09T02:07:45.163388+0000","last_became_peered":"2024-05-09T02:07:45.163388+0000","last_unstale":"2024-05-09T02:08:25.315617+0000","last_undegraded":"2024-05-09T02:08:25.315617+0000","last_fullsized":"2024-05-09T02:08:25.315617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:41.937148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309348+0000","last_change":"2024-05-09T02:07:48.195218+0000","last_active":"2024-05-09T02:08:25.309348+0000","last_peered":"2024-05-09T02:08:25.309348+0000","last_clean":"2024-05-09T02:08:25.309348+0000","last_became_active":"2024-05-09T02:07:48.194895+0000","last_became_peered":"2024-05-09T02:07:48.194895+0000","last_unstale":"2024-05-09T02:08:25.309348+0000","last_undegraded":"2024-05-09T02:08:25.309348+0000","last_fullsized":"2024-05-09T02:08:25.309348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:49.086594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318180+0000","last_change":"2024-05-09T02:07:45.158039+0000","last_active":"2024-05-09T02:08:25.318180+0000","last_peered":"2024-05-09T02:08:25.318180+0000","last_clean":"2024-05-09T02:08:25.318180+0000","last_became_active":"2024-05-09T02:07:45.157255+0000","last_became_peered":"2024-05-09T02:07:45.157255+0000","last_unstale":"2024-05-09T02:08:25.318180+0000","last_undegraded":"2024-05-09T02:08:25.318180+0000","last_fullsized":"2024-05-09T02:08:25.318180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:56.513999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313958+0000","last_change":"2024-05-09T02:07:48.200150+0000","last_active":"2024-05-09T02:08:25.313958+0000","last_peered":"2024-05-09T02:08:25.313958+0000","last_clean":"2024-05-09T02:08:25.313958+0000","last_became_active":"2024-05-09T02:07:48.199336+0000","last_became_peered":"2024-05-09T02:07:48.199336+0000","last_unstale":"2024-05-09T02:08:25.313958+0000","last_undegraded":"2024-05-09T02:08:25.313958+0000","last_fullsized":"2024-05-09T02:08:25.313958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:45.699119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":63,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:34.308167+0000","last_change":"2024-05-09T02:07:45.165476+0000","last_active":"2024-05-09T02:08:34.308167+0000","last_peered":"2024-05-09T02:08:34.308167+0000","last_clean":"2024-05-09T02:08:34.308167+0000","last_became_active":"2024-05-09T02:07:45.165107+0000","last_became_peered":"2024-05-09T02:07:45.165107+0000","last_unstale":"2024-05-09T02:08:34.308167+0000","last_undegraded":"2024-05-09T02:08:34.308167+0000","last_fullsized":"2024-05-09T02:08:34.308167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:15:33.518982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313557+0000","last_change":"2024-05-09T02:07:48.193487+0000","last_active":"2024-05-09T02:08:25.313557+0000","last_peered":"2024-05-09T02:08:25.313557+0000","last_clean":"2024-05-09T02:08:25.313557+0000","last_became_active":"2024-05-09T02:07:48.193108+0000","last_became_peered":"2024-05-09T02:07:48.193108+0000","last_unstale":"2024-05-09T02:08:25.313557+0000","last_undegraded":"2024-05-09T02:08:25.313557+0000","last_fullsized":"2024-05-09T02:08:25.313557+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:11.895058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.335525+0000","last_change":"2024-05-09T02:08:25.335525+0000","last_active":"2024-05-09T02:08:25.335525+0000","last_peered":"2024-05-09T02:08:25.335525+0000","last_clean":"2024-05-09T02:08:25.335525+0000","last_became_active":"2024-05-09T02:08:25.319746+0000","last_became_peered":"2024-05-09T02:08:25.319746+0000","last_unstale":"2024-05-09T02:08:25.335525+0000","last_undegraded":"2024-05-09T02:08:25.335525+0000","last_fullsized":"2024-05-09T02:08:25.335525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:41.774066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317433+0000","last_change":"2024-05-09T02:07:48.196125+0000","last_active":"2024-05-09T02:08:25.317433+0000","last_peered":"2024-05-09T02:08:25.317433+0000","last_clean":"2024-05-09T02:08:25.317433+0000","last_became_active":"2024-05-09T02:07:48.195737+0000","last_became_peered":"2024-05-09T02:07:48.195737+0000","last_unstale":"2024-05-09T02:08:25.317433+0000","last_undegraded":"2024-05-09T02:08:25.317433+0000","last_fullsized":"2024-05-09T02:08:25.317433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:43.264720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310367+0000","last_change":"2024-05-09T02:07:45.163791+0000","last_active":"2024-05-09T02:08:25.310367+0000","last_peered":"2024-05-09T02:08:25.310367+0000","last_clean":"2024-05-09T02:08:25.310367+0000","last_became_active":"2024-05-09T02:07:45.163129+0000","last_became_peered":"2024-05-09T02:07:45.163129+0000","last_unstale":"2024-05-09T02:08:25.310367+0000","last_undegraded":"2024-05-09T02:08:25.310367+0000","last_fullsized":"2024-05-09T02:08:25.310367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:48.562850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310748+0000","last_change":"2024-05-09T02:07:48.197782+0000","last_active":"2024-05-09T02:08:25.310748+0000","last_peered":"2024-05-09T02:08:25.310748+0000","last_clean":"2024-05-09T02:08:25.310748+0000","last_became_active":"2024-05-09T02:07:48.197353+0000","last_became_peered":"2024-05-09T02:07:48.197353+0000","last_unstale":"2024-05-09T02:08:25.310748+0000","last_undegraded":"2024-05-09T02:08:25.310748+0000","last_fullsized":"2024-05-09T02:08:25.310748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:17.053571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315220+0000","last_change":"2024-05-09T02:07:45.187028+0000","last_active":"2024-05-09T02:08:25.315220+0000","last_peered":"2024-05-09T02:08:25.315220+0000","last_clean":"2024-05-09T02:08:25.315220+0000","last_became_active":"2024-05-09T02:07:45.186639+0000","last_became_peered":"2024-05-09T02:07:45.186639+0000","last_unstale":"2024-05-09T02:08:25.315220+0000","last_undegraded":"2024-05-09T02:08:25.315220+0000","last_fullsized":"2024-05-09T02:08:25.315220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:35.834374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.308184+0000","last_change":"2024-05-09T02:07:48.207150+0000","last_active":"2024-05-09T02:08:25.308184+0000","last_peered":"2024-05-09T02:08:25.308184+0000","last_clean":"2024-05-09T02:08:25.308184+0000","last_became_active":"2024-05-09T02:07:48.206797+0000","last_became_peered":"2024-05-09T02:07:48.206797+0000","last_unstale":"2024-05-09T02:08:25.308184+0000","last_undegraded":"2024-05-09T02:08:25.308184+0000","last_fullsized":"2024-05-09T02:08:25.308184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:37.328133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":64,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:38.312852+0000","last_change":"2024-05-09T02:07:45.165422+0000","last_active":"2024-05-09T02:08:38.312852+0000","last_peered":"2024-05-09T02:08:38.312852+0000","last_clean":"2024-05-09T02:08:38.312852+0000","last_became_active":"2024-05-09T02:07:45.164998+0000","last_became_peered":"2024-05-09T02:07:45.164998+0000","last_unstale":"2024-05-09T02:08:38.312852+0000","last_undegraded":"2024-05-09T02:08:38.312852+0000","last_fullsized":"2024-05-09T02:08:38.312852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:49.124323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.307961+0000","last_change":"2024-05-09T02:07:48.206538+0000","last_active":"2024-05-09T02:08:25.307961+0000","last_peered":"2024-05-09T02:08:25.307961+0000","last_clean":"2024-05-09T02:08:25.307961+0000","last_became_active":"2024-05-09T02:07:48.206061+0000","last_became_peered":"2024-05-09T02:07:48.206061+0000","last_unstale":"2024-05-09T02:08:25.307961+0000","last_undegraded":"2024-05-09T02:08:25.307961+0000","last_fullsized":"2024-05-09T02:08:25.307961+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:24.075091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318188+0000","last_change":"2024-05-09T02:07:48.202235+0000","last_active":"2024-05-09T02:08:25.318188+0000","last_peered":"2024-05-09T02:08:25.318188+0000","last_clean":"2024-05-09T02:08:25.318188+0000","last_became_active":"2024-05-09T02:07:48.200891+0000","last_became_peered":"2024-05-09T02:07:48.200891+0000","last_unstale":"2024-05-09T02:08:25.318188+0000","last_undegraded":"2024-05-09T02:08:25.318188+0000","last_fullsized":"2024-05-09T02:08:25.318188+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:39.061267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":63,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:37.311344+0000","last_change":"2024-05-09T02:07:45.184582+0000","last_active":"2024-05-09T02:08:37.311344+0000","last_peered":"2024-05-09T02:08:37.311344+0000","last_clean":"2024-05-09T02:08:37.311344+0000","last_became_active":"2024-05-09T02:07:45.184187+0000","last_became_peered":"2024-05-09T02:07:45.184187+0000","last_unstale":"2024-05-09T02:08:37.311344+0000","last_undegraded":"2024-05-09T02:08:37.311344+0000","last_fullsized":"2024-05-09T02:08:37.311344+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:11.343797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314798+0000","last_change":"2024-05-09T02:07:48.199247+0000","last_active":"2024-05-09T02:08:25.314798+0000","last_peered":"2024-05-09T02:08:25.314798+0000","last_clean":"2024-05-09T02:08:25.314798+0000","last_became_active":"2024-05-09T02:07:48.198121+0000","last_became_peered":"2024-05-09T02:07:48.198121+0000","last_unstale":"2024-05-09T02:08:25.314798+0000","last_undegraded":"2024-05-09T02:08:25.314798+0000","last_fullsized":"2024-05-09T02:08:25.314798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:45:12.885616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317973+0000","last_change":"2024-05-09T02:07:45.163750+0000","last_active":"2024-05-09T02:08:25.317973+0000","last_peered":"2024-05-09T02:08:25.317973+0000","last_clean":"2024-05-09T02:08:25.317973+0000","last_became_active":"2024-05-09T02:07:45.163039+0000","last_became_peered":"2024-05-09T02:07:45.163039+0000","last_unstale":"2024-05-09T02:08:25.317973+0000","last_undegraded":"2024-05-09T02:08:25.317973+0000","last_fullsized":"2024-05-09T02:08:25.317973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:06.794514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313720+0000","last_change":"2024-05-09T02:06:20.334734+0000","last_active":"2024-05-09T02:08:25.313720+0000","last_peered":"2024-05-09T02:08:25.313720+0000","last_clean":"2024-05-09T02:08:25.313720+0000","last_became_active":"2024-05-09T02:06:20.329198+0000","last_became_peered":"2024-05-09T02:06:20.329198+0000","last_unstale":"2024-05-09T02:08:25.313720+0000","last_undegraded":"2024-05-09T02:08:25.313720+0000","last_fullsized":"2024-05-09T02:08:25.313720+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:02:50.632016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:02:50.632016+0000","last_clean_scrub_stamp":"2024-05-09T02:02:50.632016+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:08:15.495982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.321187+0000","last_change":"2024-05-09T02:08:25.321187+0000","last_active":"2024-05-09T02:08:25.321187+0000","last_peered":"2024-05-09T02:08:25.321187+0000","last_clean":"2024-05-09T02:08:25.321187+0000","last_became_active":"2024-05-09T02:08:25.320693+0000","last_became_peered":"2024-05-09T02:08:25.320693+0000","last_unstale":"2024-05-09T02:08:25.321187+0000","last_undegraded":"2024-05-09T02:08:25.321187+0000","last_fullsized":"2024-05-09T02:08:25.321187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:00.176016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314395+0000","last_change":"2024-05-09T02:07:45.158053+0000","last_active":"2024-05-09T02:08:25.314395+0000","last_peered":"2024-05-09T02:08:25.314395+0000","last_clean":"2024-05-09T02:08:25.314395+0000","last_became_active":"2024-05-09T02:07:45.157619+0000","last_became_peered":"2024-05-09T02:07:45.157619+0000","last_unstale":"2024-05-09T02:08:25.314395+0000","last_undegraded":"2024-05-09T02:08:25.314395+0000","last_fullsized":"2024-05-09T02:08:25.314395+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:01.618475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.341770+0000","last_change":"2024-05-09T02:08:25.341377+0000","last_active":"2024-05-09T02:08:25.341770+0000","last_peered":"2024-05-09T02:08:25.341770+0000","last_clean":"2024-05-09T02:08:25.341770+0000","last_became_active":"2024-05-09T02:08:25.318683+0000","last_became_peered":"2024-05-09T02:08:25.318683+0000","last_unstale":"2024-05-09T02:08:25.341770+0000","last_undegraded":"2024-05-09T02:08:25.341770+0000","last_fullsized":"2024-05-09T02:08:25.341770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:12.456338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309370+0000","last_change":"2024-05-09T02:07:48.186892+0000","last_active":"2024-05-09T02:08:25.309370+0000","last_peered":"2024-05-09T02:08:25.309370+0000","last_clean":"2024-05-09T02:08:25.309370+0000","last_became_active":"2024-05-09T02:07:48.186543+0000","last_became_peered":"2024-05-09T02:07:48.186543+0000","last_unstale":"2024-05-09T02:08:25.309370+0000","last_undegraded":"2024-05-09T02:08:25.309370+0000","last_fullsized":"2024-05-09T02:08:25.309370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:01.210504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'15","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:31.205313+0000","last_change":"2024-05-09T02:07:45.158007+0000","last_active":"2024-05-09T02:08:31.205313+0000","last_peered":"2024-05-09T02:08:31.205313+0000","last_clean":"2024-05-09T02:08:31.205313+0000","last_became_active":"2024-05-09T02:07:45.157612+0000","last_became_peered":"2024-05-09T02:07:45.157612+0000","last_unstale":"2024-05-09T02:08:31.205313+0000","last_undegraded":"2024-05-09T02:08:31.205313+0000","last_fullsized":"2024-05-09T02:08:31.205313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:01.526756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314093+0000","last_change":"2024-05-09T02:07:48.186065+0000","last_active":"2024-05-09T02:08:25.314093+0000","last_peered":"2024-05-09T02:08:25.314093+0000","last_clean":"2024-05-09T02:08:25.314093+0000","last_became_active":"2024-05-09T02:07:48.185746+0000","last_became_peered":"2024-05-09T02:07:48.185746+0000","last_unstale":"2024-05-09T02:08:25.314093+0000","last_undegraded":"2024-05-09T02:08:25.314093+0000","last_fullsized":"2024-05-09T02:08:25.314093+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:09.570261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317499+0000","last_change":"2024-05-09T02:07:46.005439+0000","last_active":"2024-05-09T02:08:25.317499+0000","last_peered":"2024-05-09T02:08:25.317499+0000","last_clean":"2024-05-09T02:08:25.317499+0000","last_became_active":"2024-05-09T02:07:46.005087+0000","last_became_peered":"2024-05-09T02:07:46.005087+0000","last_unstale":"2024-05-09T02:08:25.317499+0000","last_undegraded":"2024-05-09T02:08:25.317499+0000","last_fullsized":"2024-05-09T02:08:25.317499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:08.543310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315347+0000","last_change":"2024-05-09T02:07:48.194062+0000","last_active":"2024-05-09T02:08:25.315347+0000","last_peered":"2024-05-09T02:08:25.315347+0000","last_clean":"2024-05-09T02:08:25.315347+0000","last_became_active":"2024-05-09T02:07:48.193712+0000","last_became_peered":"2024-05-09T02:07:48.193712+0000","last_unstale":"2024-05-09T02:08:25.315347+0000","last_undegraded":"2024-05-09T02:08:25.315347+0000","last_fullsized":"2024-05-09T02:08:25.315347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:48.394898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315411+0000","last_change":"2024-05-09T02:07:45.184821+0000","last_active":"2024-05-09T02:08:25.315411+0000","last_peered":"2024-05-09T02:08:25.315411+0000","last_clean":"2024-05-09T02:08:25.315411+0000","last_became_active":"2024-05-09T02:07:45.184437+0000","last_became_peered":"2024-05-09T02:07:45.184437+0000","last_unstale":"2024-05-09T02:08:25.315411+0000","last_undegraded":"2024-05-09T02:08:25.315411+0000","last_fullsized":"2024-05-09T02:08:25.315411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:30.864171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309993+0000","last_change":"2024-05-09T02:07:48.187635+0000","last_active":"2024-05-09T02:08:25.309993+0000","last_peered":"2024-05-09T02:08:25.309993+0000","last_clean":"2024-05-09T02:08:25.309993+0000","last_became_active":"2024-05-09T02:07:48.187071+0000","last_became_peered":"2024-05-09T02:07:48.187071+0000","last_unstale":"2024-05-09T02:08:25.309993+0000","last_undegraded":"2024-05-09T02:08:25.309993+0000","last_fullsized":"2024-05-09T02:08:25.309993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:10.716715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315158+0000","last_change":"2024-05-09T02:07:45.162166+0000","last_active":"2024-05-09T02:08:25.315158+0000","last_peered":"2024-05-09T02:08:25.315158+0000","last_clean":"2024-05-09T02:08:25.315158+0000","last_became_active":"2024-05-09T02:07:45.161787+0000","last_became_peered":"2024-05-09T02:07:45.161787+0000","last_unstale":"2024-05-09T02:08:25.315158+0000","last_undegraded":"2024-05-09T02:08:25.315158+0000","last_fullsized":"2024-05-09T02:08:25.315158+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:17.925214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310341+0000","last_change":"2024-05-09T02:07:48.202980+0000","last_active":"2024-05-09T02:08:25.310341+0000","last_peered":"2024-05-09T02:08:25.310341+0000","last_clean":"2024-05-09T02:08:25.310341+0000","last_became_active":"2024-05-09T02:07:48.202609+0000","last_became_peered":"2024-05-09T02:07:48.202609+0000","last_unstale":"2024-05-09T02:08:25.310341+0000","last_undegraded":"2024-05-09T02:08:25.310341+0000","last_fullsized":"2024-05-09T02:08:25.310341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:59.993451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.326699+0000","last_change":"2024-05-09T02:08:25.326142+0000","last_active":"2024-05-09T02:08:25.326699+0000","last_peered":"2024-05-09T02:08:25.326699+0000","last_clean":"2024-05-09T02:08:25.326699+0000","last_became_active":"2024-05-09T02:08:25.317666+0000","last_became_peered":"2024-05-09T02:08:25.317666+0000","last_unstale":"2024-05-09T02:08:25.326699+0000","last_undegraded":"2024-05-09T02:08:25.326699+0000","last_fullsized":"2024-05-09T02:08:25.326699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:26.178211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315560+0000","last_change":"2024-05-09T02:07:48.202219+0000","last_active":"2024-05-09T02:08:25.315560+0000","last_peered":"2024-05-09T02:08:25.315560+0000","last_clean":"2024-05-09T02:08:25.315560+0000","last_became_active":"2024-05-09T02:07:48.201819+0000","last_became_peered":"2024-05-09T02:07:48.201819+0000","last_unstale":"2024-05-09T02:08:25.315560+0000","last_undegraded":"2024-05-09T02:08:25.315560+0000","last_fullsized":"2024-05-09T02:08:25.315560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:51.573359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311162+0000","last_change":"2024-05-09T02:07:45.162228+0000","last_active":"2024-05-09T02:08:25.311162+0000","last_peered":"2024-05-09T02:08:25.311162+0000","last_clean":"2024-05-09T02:08:25.311162+0000","last_became_active":"2024-05-09T02:07:45.161847+0000","last_became_peered":"2024-05-09T02:07:45.161847+0000","last_unstale":"2024-05-09T02:08:25.311162+0000","last_undegraded":"2024-05-09T02:08:25.311162+0000","last_fullsized":"2024-05-09T02:08:25.311162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:47.459709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314196+0000","last_change":"2024-05-09T02:07:48.201401+0000","last_active":"2024-05-09T02:08:25.314196+0000","last_peered":"2024-05-09T02:08:25.314196+0000","last_clean":"2024-05-09T02:08:25.314196+0000","last_became_active":"2024-05-09T02:07:48.201040+0000","last_became_peered":"2024-05-09T02:07:48.201040+0000","last_unstale":"2024-05-09T02:08:25.314196+0000","last_undegraded":"2024-05-09T02:08:25.314196+0000","last_fullsized":"2024-05-09T02:08:25.314196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:58.038690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":60,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:33.308040+0000","last_change":"2024-05-09T02:07:46.005273+0000","last_active":"2024-05-09T02:08:33.308040+0000","last_peered":"2024-05-09T02:08:33.308040+0000","last_clean":"2024-05-09T02:08:33.308040+0000","last_became_active":"2024-05-09T02:07:46.004732+0000","last_became_peered":"2024-05-09T02:07:46.004732+0000","last_unstale":"2024-05-09T02:08:33.308040+0000","last_undegraded":"2024-05-09T02:08:33.308040+0000","last_fullsized":"2024-05-09T02:08:33.308040+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:55.576955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315747+0000","last_change":"2024-05-09T02:07:48.197032+0000","last_active":"2024-05-09T02:08:25.315747+0000","last_peered":"2024-05-09T02:08:25.315747+0000","last_clean":"2024-05-09T02:08:25.315747+0000","last_became_active":"2024-05-09T02:07:48.196553+0000","last_became_peered":"2024-05-09T02:07:48.196553+0000","last_unstale":"2024-05-09T02:08:25.315747+0000","last_undegraded":"2024-05-09T02:08:25.315747+0000","last_fullsized":"2024-05-09T02:08:25.315747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:50:25.081024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318767+0000","last_change":"2024-05-09T02:07:45.166456+0000","last_active":"2024-05-09T02:08:25.318767+0000","last_peered":"2024-05-09T02:08:25.318767+0000","last_clean":"2024-05-09T02:08:25.318767+0000","last_became_active":"2024-05-09T02:07:45.166132+0000","last_became_peered":"2024-05-09T02:07:45.166132+0000","last_unstale":"2024-05-09T02:08:25.318767+0000","last_undegraded":"2024-05-09T02:08:25.318767+0000","last_fullsized":"2024-05-09T02:08:25.318767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:15.701802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.324526+0000","last_change":"2024-05-09T02:08:25.323303+0000","last_active":"2024-05-09T02:08:25.324526+0000","last_peered":"2024-05-09T02:08:25.324526+0000","last_clean":"2024-05-09T02:08:25.324526+0000","last_became_active":"2024-05-09T02:08:25.321752+0000","last_became_peered":"2024-05-09T02:08:25.321752+0000","last_unstale":"2024-05-09T02:08:25.324526+0000","last_undegraded":"2024-05-09T02:08:25.324526+0000","last_fullsized":"2024-05-09T02:08:25.324526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:17.791737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309540+0000","last_change":"2024-05-09T02:07:45.155480+0000","last_active":"2024-05-09T02:08:25.309540+0000","last_peered":"2024-05-09T02:08:25.309540+0000","last_clean":"2024-05-09T02:08:25.309540+0000","last_became_active":"2024-05-09T02:07:45.155085+0000","last_became_peered":"2024-05-09T02:07:45.155085+0000","last_unstale":"2024-05-09T02:08:25.309540+0000","last_undegraded":"2024-05-09T02:08:25.309540+0000","last_fullsized":"2024-05-09T02:08:25.309540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:05.547857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314367+0000","last_change":"2024-05-09T02:07:48.194480+0000","last_active":"2024-05-09T02:08:25.314367+0000","last_peered":"2024-05-09T02:08:25.314367+0000","last_clean":"2024-05-09T02:08:25.314367+0000","last_became_active":"2024-05-09T02:07:48.194136+0000","last_became_peered":"2024-05-09T02:07:48.194136+0000","last_unstale":"2024-05-09T02:08:25.314367+0000","last_undegraded":"2024-05-09T02:08:25.314367+0000","last_fullsized":"2024-05-09T02:08:25.314367+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:09.177762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310492+0000","last_change":"2024-05-09T02:07:48.195152+0000","last_active":"2024-05-09T02:08:25.310492+0000","last_peered":"2024-05-09T02:08:25.310492+0000","last_clean":"2024-05-09T02:08:25.310492+0000","last_became_active":"2024-05-09T02:07:48.194779+0000","last_became_peered":"2024-05-09T02:07:48.194779+0000","last_unstale":"2024-05-09T02:08:25.310492+0000","last_undegraded":"2024-05-09T02:08:25.310492+0000","last_fullsized":"2024-05-09T02:08:25.310492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:39.345452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.322235+0000","last_change":"2024-05-09T02:08:25.322235+0000","last_active":"2024-05-09T02:08:25.322235+0000","last_peered":"2024-05-09T02:08:25.322235+0000","last_clean":"2024-05-09T02:08:25.322235+0000","last_became_active":"2024-05-09T02:08:25.321674+0000","last_became_peered":"2024-05-09T02:08:25.321674+0000","last_unstale":"2024-05-09T02:08:25.322235+0000","last_undegraded":"2024-05-09T02:08:25.322235+0000","last_fullsized":"2024-05-09T02:08:25.322235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:52.671669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315466+0000","last_change":"2024-05-09T02:07:48.195710+0000","last_active":"2024-05-09T02:08:25.315466+0000","last_peered":"2024-05-09T02:08:25.315466+0000","last_clean":"2024-05-09T02:08:25.315466+0000","last_became_active":"2024-05-09T02:07:48.195332+0000","last_became_peered":"2024-05-09T02:07:48.195332+0000","last_unstale":"2024-05-09T02:08:25.315466+0000","last_undegraded":"2024-05-09T02:08:25.315466+0000","last_fullsized":"2024-05-09T02:08:25.315466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:12:17.904166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315787+0000","last_change":"2024-05-09T02:07:48.200791+0000","last_active":"2024-05-09T02:08:25.315787+0000","last_peered":"2024-05-09T02:08:25.315787+0000","last_clean":"2024-05-09T02:08:25.315787+0000","last_became_active":"2024-05-09T02:07:48.199916+0000","last_became_peered":"2024-05-09T02:07:48.199916+0000","last_unstale":"2024-05-09T02:08:25.315787+0000","last_undegraded":"2024-05-09T02:08:25.315787+0000","last_fullsized":"2024-05-09T02:08:25.315787+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:32.692375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309571+0000","last_change":"2024-05-09T02:07:48.191662+0000","last_active":"2024-05-09T02:08:25.309571+0000","last_peered":"2024-05-09T02:08:25.309571+0000","last_clean":"2024-05-09T02:08:25.309571+0000","last_became_active":"2024-05-09T02:07:48.191336+0000","last_became_peered":"2024-05-09T02:07:48.191336+0000","last_unstale":"2024-05-09T02:08:25.309571+0000","last_undegraded":"2024-05-09T02:08:25.309571+0000","last_fullsized":"2024-05-09T02:08:25.309571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:00.733611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311462+0000","last_change":"2024-05-09T02:07:48.195182+0000","last_active":"2024-05-09T02:08:25.311462+0000","last_peered":"2024-05-09T02:08:25.311462+0000","last_clean":"2024-05-09T02:08:25.311462+0000","last_became_active":"2024-05-09T02:07:48.194828+0000","last_became_peered":"2024-05-09T02:07:48.194828+0000","last_unstale":"2024-05-09T02:08:25.311462+0000","last_undegraded":"2024-05-09T02:08:25.311462+0000","last_fullsized":"2024-05-09T02:08:25.311462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:33.427409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315738+0000","last_change":"2024-05-09T02:07:48.199688+0000","last_active":"2024-05-09T02:08:25.315738+0000","last_peered":"2024-05-09T02:08:25.315738+0000","last_clean":"2024-05-09T02:08:25.315738+0000","last_became_active":"2024-05-09T02:07:48.199326+0000","last_became_peered":"2024-05-09T02:07:48.199326+0000","last_unstale":"2024-05-09T02:08:25.315738+0000","last_undegraded":"2024-05-09T02:08:25.315738+0000","last_fullsized":"2024-05-09T02:08:25.315738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:30.662738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313935+0000","last_change":"2024-05-09T02:07:48.190727+0000","last_active":"2024-05-09T02:08:25.313935+0000","last_peered":"2024-05-09T02:08:25.313935+0000","last_clean":"2024-05-09T02:08:25.313935+0000","last_became_active":"2024-05-09T02:07:48.190397+0000","last_became_peered":"2024-05-09T02:07:48.190397+0000","last_unstale":"2024-05-09T02:08:25.313935+0000","last_undegraded":"2024-05-09T02:08:25.313935+0000","last_fullsized":"2024-05-09T02:08:25.313935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:42.010669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317969+0000","last_change":"2024-05-09T02:07:48.200444+0000","last_active":"2024-05-09T02:08:25.317969+0000","last_peered":"2024-05-09T02:08:25.317969+0000","last_clean":"2024-05-09T02:08:25.317969+0000","last_became_active":"2024-05-09T02:07:48.200031+0000","last_became_peered":"2024-05-09T02:07:48.200031+0000","last_unstale":"2024-05-09T02:08:25.317969+0000","last_undegraded":"2024-05-09T02:08:25.317969+0000","last_fullsized":"2024-05-09T02:08:25.317969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:30.269792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315072+0000","last_change":"2024-05-09T02:07:48.203717+0000","last_active":"2024-05-09T02:08:25.315072+0000","last_peered":"2024-05-09T02:08:25.315072+0000","last_clean":"2024-05-09T02:08:25.315072+0000","last_became_active":"2024-05-09T02:07:48.203347+0000","last_became_peered":"2024-05-09T02:07:48.203347+0000","last_unstale":"2024-05-09T02:08:25.315072+0000","last_undegraded":"2024-05-09T02:08:25.315072+0000","last_fullsized":"2024-05-09T02:08:25.315072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:17.391504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.323029+0000","last_change":"2024-05-09T02:08:25.323029+0000","last_active":"2024-05-09T02:08:25.323029+0000","last_peered":"2024-05-09T02:08:25.323029+0000","last_clean":"2024-05-09T02:08:25.323029+0000","last_became_active":"2024-05-09T02:08:25.322524+0000","last_became_peered":"2024-05-09T02:08:25.322524+0000","last_unstale":"2024-05-09T02:08:25.323029+0000","last_undegraded":"2024-05-09T02:08:25.323029+0000","last_fullsized":"2024-05-09T02:08:25.323029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:25.017511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309673+0000","last_change":"2024-05-09T02:07:48.187802+0000","last_active":"2024-05-09T02:08:25.309673+0000","last_peered":"2024-05-09T02:08:25.309673+0000","last_clean":"2024-05-09T02:08:25.309673+0000","last_became_active":"2024-05-09T02:07:48.187478+0000","last_became_peered":"2024-05-09T02:07:48.187478+0000","last_unstale":"2024-05-09T02:08:25.309673+0000","last_undegraded":"2024-05-09T02:08:25.309673+0000","last_fullsized":"2024-05-09T02:08:25.309673+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:12.981386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310421+0000","last_change":"2024-05-09T02:07:48.192117+0000","last_active":"2024-05-09T02:08:25.310421+0000","last_peered":"2024-05-09T02:08:25.310421+0000","last_clean":"2024-05-09T02:08:25.310421+0000","last_became_active":"2024-05-09T02:07:48.191726+0000","last_became_peered":"2024-05-09T02:07:48.191726+0000","last_unstale":"2024-05-09T02:08:25.310421+0000","last_undegraded":"2024-05-09T02:08:25.310421+0000","last_fullsized":"2024-05-09T02:08:25.310421+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:07.735582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280595,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68639,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":345,"num_read_kb":252,"num_write":245,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":199672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678041,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":429664,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874269,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1082798,"data_compressed":41450,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037796,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":909312,"data_stored":429599,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234024,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":429748,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430254,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":1022378,"data_compressed":41450,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593779,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":429199,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":491938,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":430538,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":485801,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":428907,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":429563,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790009,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":1021665,"data_compressed":41450,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T02:08:40.822 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T02:08:40.823 INFO:tasks.cephadm:Setting up client nodes... 2024-05-09T02:08:40.823 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-09T02:08:41.079 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:41.453 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:41 smithi019 ceph-mon[30151]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.2 KiB/s wr, 13 op/s; 9 B/s, 0 objects/s recovering 2024-05-09T02:08:41.453 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:41 smithi019 ceph-mon[30151]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:08:41.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:41 smithi059 ceph-mon[34781]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.2 KiB/s wr, 13 op/s; 9 B/s, 0 objects/s recovering 2024-05-09T02:08:41.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:41 smithi059 ceph-mon[34781]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:08:41.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:41 smithi196 ceph-mon[34718]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.2 KiB/s wr, 13 op/s; 9 B/s, 0 objects/s recovering 2024-05-09T02:08:41.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:41 smithi196 ceph-mon[34718]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:08:42.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.792+0000 7f6d571b8700 1 -- 172.21.15.19:0/4104061259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d50103240 msgr2=0x7f6d50103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:42.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.792+0000 7f6d571b8700 1 --2- 172.21.15.19:0/4104061259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d50103240 0x7f6d50103650 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009990 tx=0x7f6d38009ca0 comp rx=0 tx=0).stop 2024-05-09T02:08:42.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.792+0000 7f6d571b8700 1 -- 172.21.15.19:0/4104061259 shutdown_connections 2024-05-09T02:08:42.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.792+0000 7f6d571b8700 1 --2- 172.21.15.19:0/4104061259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d50104d10 0x7f6d5010b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:42.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.792+0000 7f6d571b8700 1 --2- 172.21.15.19:0/4104061259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d50103240 0x7f6d50103650 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:42.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.792+0000 7f6d571b8700 1 --2- 172.21.15.19:0/4104061259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d50104370 0x7f6d50104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:42.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.793+0000 7f6d571b8700 1 -- 172.21.15.19:0/4104061259 >> 172.21.15.19:0/4104061259 conn(0x7f6d500fe880 msgr2=0x7f6d50100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:42.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.793+0000 7f6d571b8700 1 -- 172.21.15.19:0/4104061259 shutdown_connections 2024-05-09T02:08:42.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.793+0000 7f6d571b8700 1 -- 172.21.15.19:0/4104061259 wait complete. 2024-05-09T02:08:42.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.793+0000 7f6d571b8700 1 Processor -- start 2024-05-09T02:08:42.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.793+0000 7f6d571b8700 1 -- start start 2024-05-09T02:08:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.793+0000 7f6d571b8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d50103240 0x7f6d5006c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.793+0000 7f6d571b8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d50104370 0x7f6d5019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d571b8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d50104d10 0x7f6d501a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d571b8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5010ec80 con 0x7f6d50104d10 2024-05-09T02:08:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d571b8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5010eb00 con 0x7f6d50104370 2024-05-09T02:08:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d571b8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5010ee00 con 0x7f6d50103240 2024-05-09T02:08:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d55755700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d50104d10 0x7f6d501a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d55755700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d50104d10 0x7f6d501a1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47238/0 (socket says 172.21.15.19:47238) 2024-05-09T02:08:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d55755700 1 -- 172.21.15.19:0/1868323957 learned_addr learned my addr 172.21.15.19:0/1868323957 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d54f54700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d50103240 0x7f6d5006c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d4ffff700 1 --2- 172.21.15.19:0/1868323957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d50104370 0x7f6d5019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d55755700 1 -- 172.21.15.19:0/1868323957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d50103240 msgr2=0x7f6d5006c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d55755700 1 --2- 172.21.15.19:0/1868323957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d50103240 0x7f6d5006c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d55755700 1 -- 172.21.15.19:0/1868323957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d50104370 msgr2=0x7f6d5019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d55755700 1 --2- 172.21.15.19:0/1868323957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d50104370 0x7f6d5019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d55755700 1 -- 172.21.15.19:0/1868323957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d501a2460 con 0x7f6d50104d10 2024-05-09T02:08:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d55755700 1 --2- 172.21.15.19:0/1868323957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d50104d10 0x7f6d501a1c30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800d980 tx=0x7f6d4800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d4dffb700 1 -- 172.21.15.19:0/1868323957 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48017070 con 0x7f6d50104d10 2024-05-09T02:08:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.794+0000 7f6d571b8700 1 -- 172.21.15.19:0/1868323957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d501a2750 con 0x7f6d50104d10 2024-05-09T02:08:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.795+0000 7f6d4dffb700 1 -- 172.21.15.19:0/1868323957 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d48005020 con 0x7f6d50104d10 2024-05-09T02:08:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.795+0000 7f6d4dffb700 1 -- 172.21.15.19:0/1868323957 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4800fa10 con 0x7f6d50104d10 2024-05-09T02:08:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.795+0000 7f6d571b8700 1 -- 172.21.15.19:0/1868323957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d501a2c90 con 0x7f6d50104d10 2024-05-09T02:08:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.796+0000 7f6d571b8700 1 -- 172.21.15.19:0/1868323957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d500512c0 con 0x7f6d50104d10 2024-05-09T02:08:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.797+0000 7f6d4dffb700 1 -- 172.21.15.19:0/1868323957 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6d48004a30 con 0x7f6d50104d10 2024-05-09T02:08:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.798+0000 7f6d4dffb700 1 --2- 172.21.15.19:0/1868323957 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6d3c0b2910 0x7f6d3c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.798+0000 7f6d4dffb700 1 -- 172.21.15.19:0/1868323957 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f6d480e25f0 con 0x7f6d50104d10 2024-05-09T02:08:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.798+0000 7f6d54f54700 1 --2- 172.21.15.19:0/1868323957 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6d3c0b2910 0x7f6d3c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.799+0000 7f6d54f54700 1 --2- 172.21.15.19:0/1868323957 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6d3c0b2910 0x7f6d3c0b4e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6d380038a0 tx=0x7f6d3801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:42.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:42.802+0000 7f6d4dffb700 1 -- 172.21.15.19:0/1868323957 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d48020020 con 0x7f6d50104d10 2024-05-09T02:08:43.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:43.106+0000 7f6d571b8700 1 -- 172.21.15.19:0/1868323957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f6d5002e690 con 0x7f6d50104d10 2024-05-09T02:08:43.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:43.111+0000 7f6d4dffb700 1 -- 172.21.15.19:0/1868323957 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f6d5002e690 con 0x7f6d50104d10 2024-05-09T02:08:43.114 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-05-09T02:08:43.114 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQArMDxmXld2BhAATXHoiBxurKsRa7uHtK81yA== 2024-05-09T02:08:43.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:43.114+0000 7f6d571b8700 1 -- 172.21.15.19:0/1868323957 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6d3c0b2910 msgr2=0x7f6d3c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:43.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:43.114+0000 7f6d571b8700 1 --2- 172.21.15.19:0/1868323957 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6d3c0b2910 0x7f6d3c0b4e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6d380038a0 tx=0x7f6d3801a040 comp rx=0 tx=0).stop 2024-05-09T02:08:43.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:43.114+0000 7f6d571b8700 1 -- 172.21.15.19:0/1868323957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d50104d10 msgr2=0x7f6d501a1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:43.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:43.114+0000 7f6d571b8700 1 --2- 172.21.15.19:0/1868323957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d50104d10 0x7f6d501a1c30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800d980 tx=0x7f6d4800dc90 comp rx=0 tx=0).stop 2024-05-09T02:08:43.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:43.114+0000 7f6d571b8700 1 -- 172.21.15.19:0/1868323957 shutdown_connections 2024-05-09T02:08:43.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:43.114+0000 7f6d571b8700 1 --2- 172.21.15.19:0/1868323957 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6d3c0b2910 0x7f6d3c0b4e20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:43.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:43.114+0000 7f6d571b8700 1 --2- 172.21.15.19:0/1868323957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d50104d10 0x7f6d501a1c30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:43.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:43.114+0000 7f6d571b8700 1 --2- 172.21.15.19:0/1868323957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d50103240 0x7f6d5006c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:43.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:43.114+0000 7f6d571b8700 1 --2- 172.21.15.19:0/1868323957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d50104370 0x7f6d5019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:43.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:43.114+0000 7f6d571b8700 1 -- 172.21.15.19:0/1868323957 >> 172.21.15.19:0/1868323957 conn(0x7f6d500fe880 msgr2=0x7f6d501075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:43.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:43.114+0000 7f6d571b8700 1 -- 172.21.15.19:0/1868323957 shutdown_connections 2024-05-09T02:08:43.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:43.114+0000 7f6d571b8700 1 -- 172.21.15.19:0/1868323957 wait complete. 2024-05-09T02:08:43.478 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:43 smithi019 ceph-mon[30151]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s; 9 B/s, 0 objects/s recovering 2024-05-09T02:08:43.479 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:43 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1868323957' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T02:08:43.479 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:43 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1868323957' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T02:08:43.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:43 smithi059 ceph-mon[34781]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s; 9 B/s, 0 objects/s recovering 2024-05-09T02:08:43.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:43 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1868323957' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T02:08:43.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:43 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1868323957' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T02:08:43.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:43 smithi196 ceph-mon[34718]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s; 9 B/s, 0 objects/s recovering 2024-05-09T02:08:43.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:43 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1868323957' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T02:08:43.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:43 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1868323957' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T02:08:43.802 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T02:08:43.802 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-09T02:08:43.802 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T02:08:43.843 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T02:08:43.843 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-09T02:08:43.843 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-09T02:08:44.136 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:45.536 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:45 smithi019 ceph-mon[30151]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-09T02:08:45.536 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:45 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:08:45.587 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:45 smithi059 ceph-mon[34781]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-09T02:08:45.587 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:45 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:08:45.623 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:45 smithi196 ceph-mon[34718]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-09T02:08:45.623 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:45 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:08:45.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.847+0000 7f0ba65ea700 1 -- 172.21.15.19:0/201579863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ba0108740 msgr2=0x7f0ba01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:45.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.847+0000 7f0ba65ea700 1 --2- 172.21.15.19:0/201579863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ba0108740 0x7f0ba01100a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800b0d0 tx=0x7f0b9800b3e0 comp rx=0 tx=0).stop 2024-05-09T02:08:45.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.848+0000 7f0ba65ea700 1 -- 172.21.15.19:0/201579863 shutdown_connections 2024-05-09T02:08:45.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.848+0000 7f0ba65ea700 1 --2- 172.21.15.19:0/201579863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ba0108740 0x7f0ba01100a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:45.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.848+0000 7f0ba65ea700 1 --2- 172.21.15.19:0/201579863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ba01034c0 0x7f0ba01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:45.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.848+0000 7f0ba65ea700 1 --2- 172.21.15.19:0/201579863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba0107df0 0x7f0ba0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:45.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.848+0000 7f0ba65ea700 1 -- 172.21.15.19:0/201579863 >> 172.21.15.19:0/201579863 conn(0x7f0ba00fedc0 msgr2=0x7f0ba01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:45.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.848+0000 7f0ba65ea700 1 -- 172.21.15.19:0/201579863 shutdown_connections 2024-05-09T02:08:45.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.848+0000 7f0ba65ea700 1 -- 172.21.15.19:0/201579863 wait complete. 2024-05-09T02:08:45.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0ba65ea700 1 Processor -- start 2024-05-09T02:08:45.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0ba65ea700 1 -- start start 2024-05-09T02:08:45.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0ba65ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba01034c0 0x7f0ba019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:45.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0ba65ea700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ba0107df0 0x7f0ba019d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:45.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0ba65ea700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ba0108740 0x7f0ba01a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:45.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0ba65ea700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba0113390 con 0x7f0ba0107df0 2024-05-09T02:08:45.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0ba65ea700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba0113210 con 0x7f0ba01034c0 2024-05-09T02:08:45.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0ba65ea700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba0113510 con 0x7f0ba0108740 2024-05-09T02:08:45.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0b97fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ba0107df0 0x7f0ba019d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:45.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0b97fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ba0107df0 0x7f0ba019d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47248/0 (socket says 172.21.15.19:47248) 2024-05-09T02:08:45.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0b97fff700 1 -- 172.21.15.19:0/3906133207 learned_addr learned my addr 172.21.15.19:0/3906133207 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:45.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0ba4b87700 1 --2- 172.21.15.19:0/3906133207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ba0108740 0x7f0ba01a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:45.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0b9ffff700 1 --2- 172.21.15.19:0/3906133207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba01034c0 0x7f0ba019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:45.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0b97fff700 1 -- 172.21.15.19:0/3906133207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ba0108740 msgr2=0x7f0ba01a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:45.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0b97fff700 1 --2- 172.21.15.19:0/3906133207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ba0108740 0x7f0ba01a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:45.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0b97fff700 1 -- 172.21.15.19:0/3906133207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba01034c0 msgr2=0x7f0ba019cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:45.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0b97fff700 1 --2- 172.21.15.19:0/3906133207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba01034c0 0x7f0ba019cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:45.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0b97fff700 1 -- 172.21.15.19:0/3906133207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba01a1e90 con 0x7f0ba0107df0 2024-05-09T02:08:45.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.849+0000 7f0b9ffff700 1 --2- 172.21.15.19:0/3906133207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba01034c0 0x7f0ba019cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:08:45.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.850+0000 7f0b97fff700 1 --2- 172.21.15.19:0/3906133207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ba0107df0 0x7f0ba019d560 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800d950 tx=0x7f0b8800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:45.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.850+0000 7f0b9dffb700 1 -- 172.21.15.19:0/3906133207 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88016070 con 0x7f0ba0107df0 2024-05-09T02:08:45.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.850+0000 7f0b9dffb700 1 -- 172.21.15.19:0/3906133207 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b88004830 con 0x7f0ba0107df0 2024-05-09T02:08:45.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.850+0000 7f0ba65ea700 1 -- 172.21.15.19:0/3906133207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba01a2180 con 0x7f0ba0107df0 2024-05-09T02:08:45.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.850+0000 7f0b9dffb700 1 -- 172.21.15.19:0/3906133207 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88005440 con 0x7f0ba0107df0 2024-05-09T02:08:45.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.850+0000 7f0ba65ea700 1 -- 172.21.15.19:0/3906133207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba01a9aa0 con 0x7f0ba0107df0 2024-05-09T02:08:45.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.851+0000 7f0ba65ea700 1 -- 172.21.15.19:0/3906133207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba00512c0 con 0x7f0ba0107df0 2024-05-09T02:08:45.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.853+0000 7f0b9dffb700 1 -- 172.21.15.19:0/3906133207 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0b880055e0 con 0x7f0ba0107df0 2024-05-09T02:08:45.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.854+0000 7f0b9dffb700 1 --2- 172.21.15.19:0/3906133207 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0b8c0b2990 0x7f0b8c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:45.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.854+0000 7f0b9ffff700 1 --2- 172.21.15.19:0/3906133207 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0b8c0b2990 0x7f0b8c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:45.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.854+0000 7f0b9dffb700 1 -- 172.21.15.19:0/3906133207 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f0b880e25f0 con 0x7f0ba0107df0 2024-05-09T02:08:45.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.855+0000 7f0b9ffff700 1 --2- 172.21.15.19:0/3906133207 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0b8c0b2990 0x7f0b8c0b4ea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009cb0 tx=0x7f0b90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:45.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:45.858+0000 7f0b9dffb700 1 -- 172.21.15.19:0/3906133207 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0b8801f020 con 0x7f0ba0107df0 2024-05-09T02:08:46.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:46.184+0000 7f0ba65ea700 1 -- 172.21.15.19:0/3906133207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0ba019e320 con 0x7f0ba0107df0 2024-05-09T02:08:46.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:46.190+0000 7f0b9dffb700 1 -- 172.21.15.19:0/3906133207 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288641 (secure 0 0 0) 0x7f0ba019e320 con 0x7f0ba0107df0 2024-05-09T02:08:46.196 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:08:46.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:46.194+0000 7f0ba65ea700 1 -- 172.21.15.19:0/3906133207 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0b8c0b2990 msgr2=0x7f0b8c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:46.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:46.194+0000 7f0ba65ea700 1 --2- 172.21.15.19:0/3906133207 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0b8c0b2990 0x7f0b8c0b4ea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009cb0 tx=0x7f0b90009260 comp rx=0 tx=0).stop 2024-05-09T02:08:46.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:46.194+0000 7f0ba65ea700 1 -- 172.21.15.19:0/3906133207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ba0107df0 msgr2=0x7f0ba019d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:46.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:46.194+0000 7f0ba65ea700 1 --2- 172.21.15.19:0/3906133207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ba0107df0 0x7f0ba019d560 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800d950 tx=0x7f0b8800dc60 comp rx=0 tx=0).stop 2024-05-09T02:08:46.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:46.194+0000 7f0ba65ea700 1 -- 172.21.15.19:0/3906133207 shutdown_connections 2024-05-09T02:08:46.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:46.194+0000 7f0ba65ea700 1 --2- 172.21.15.19:0/3906133207 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f0b8c0b2990 0x7f0b8c0b4ea0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:46.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:46.194+0000 7f0ba65ea700 1 --2- 172.21.15.19:0/3906133207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ba0108740 0x7f0ba01a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:46.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:46.194+0000 7f0ba65ea700 1 --2- 172.21.15.19:0/3906133207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba01034c0 0x7f0ba019cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:46.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:46.194+0000 7f0ba65ea700 1 --2- 172.21.15.19:0/3906133207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ba0107df0 0x7f0ba019d560 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:46.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:46.194+0000 7f0ba65ea700 1 -- 172.21.15.19:0/3906133207 >> 172.21.15.19:0/3906133207 conn(0x7f0ba00fedc0 msgr2=0x7f0ba01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:46.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:46.194+0000 7f0ba65ea700 1 -- 172.21.15.19:0/3906133207 shutdown_connections 2024-05-09T02:08:46.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:46.194+0000 7f0ba65ea700 1 -- 172.21.15.19:0/3906133207 wait complete. 2024-05-09T02:08:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:46 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3906133207' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T02:08:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:46 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:46 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:46 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:46 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:46.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:46 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3906133207' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T02:08:46.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:46 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:46.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:46 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:46.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:46 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:46.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:46 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:46.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:46 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3906133207' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T02:08:46.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:46 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:46.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:46 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:46.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:46 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:46.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:46 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:47.400 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":972543870},{"type":"v1","addr":"172.21.15.19:6801","nonce":972543870}]},"active_addr":"172.21.15.19:6801/972543870","active_change":"2024-05-09T02:00:23.920355+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.19:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":3660764754}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":557070897}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":758334732}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":3573608809}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":791135479}]}]} 2024-05-09T02:08:47.406 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-09T02:08:47.406 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-09T02:08:47.407 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T02:08:47.681 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:47.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:47 smithi059 ceph-mon[34781]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-09T02:08:47.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:47 smithi196 ceph-mon[34718]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-09T02:08:47.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:47 smithi019 ceph-mon[30151]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-09T02:08:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.373+0000 7f1601346700 1 -- 172.21.15.19:0/155493391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc108200 msgr2=0x7f15fc108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.373+0000 7f1601346700 1 --2- 172.21.15.19:0/155493391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc108200 0x7f15fc108610 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009990 tx=0x7f15ec009ca0 comp rx=0 tx=0).stop 2024-05-09T02:08:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.373+0000 7f1601346700 1 -- 172.21.15.19:0/155493391 shutdown_connections 2024-05-09T02:08:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.373+0000 7f1601346700 1 --2- 172.21.15.19:0/155493391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15fc104240 0x7f15fc110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.373+0000 7f1601346700 1 --2- 172.21.15.19:0/155493391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc108200 0x7f15fc108610 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.373+0000 7f1601346700 1 --2- 172.21.15.19:0/155493391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15fc1038f0 0x7f15fc103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.373+0000 7f1601346700 1 -- 172.21.15.19:0/155493391 >> 172.21.15.19:0/155493391 conn(0x7f15fc0fe6f0 msgr2=0x7f15fc100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.373+0000 7f1601346700 1 -- 172.21.15.19:0/155493391 shutdown_connections 2024-05-09T02:08:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.373+0000 7f1601346700 1 -- 172.21.15.19:0/155493391 wait complete. 2024-05-09T02:08:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.374+0000 7f1601346700 1 Processor -- start 2024-05-09T02:08:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.374+0000 7f1601346700 1 -- start start 2024-05-09T02:08:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f1601346700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15fc1038f0 0x7f15fc19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f1601346700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15fc104240 0x7f15fc19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f1601346700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc108200 0x7f15fc199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f15fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc108200 0x7f15fc199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f1601346700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc113370 con 0x7f15fc108200 2024-05-09T02:08:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f1601346700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc1131f0 con 0x7f15fc104240 2024-05-09T02:08:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f1601346700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc1134f0 con 0x7f15fc1038f0 2024-05-09T02:08:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f15faffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15fc1038f0 0x7f15fc19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f15fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc108200 0x7f15fc199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43452/0 (socket says 172.21.15.19:43452) 2024-05-09T02:08:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f15fb7fe700 1 -- 172.21.15.19:0/2683710446 learned_addr learned my addr 172.21.15.19:0/2683710446 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f15fa7fc700 1 --2- 172.21.15.19:0/2683710446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15fc104240 0x7f15fc19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f15faffd700 1 -- 172.21.15.19:0/2683710446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15fc104240 msgr2=0x7f15fc19fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f15faffd700 1 --2- 172.21.15.19:0/2683710446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15fc104240 0x7f15fc19fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f15faffd700 1 -- 172.21.15.19:0/2683710446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc108200 msgr2=0x7f15fc199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f15faffd700 1 --2- 172.21.15.19:0/2683710446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc108200 0x7f15fc199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f15faffd700 1 -- 172.21.15.19:0/2683710446 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc19a1d0 con 0x7f15fc1038f0 2024-05-09T02:08:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.375+0000 7f15fb7fe700 1 --2- 172.21.15.19:0/2683710446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc108200 0x7f15fc199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:08:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.376+0000 7f15faffd700 1 --2- 172.21.15.19:0/2683710446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15fc1038f0 0x7f15fc19f8b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f15ec000c00 tx=0x7f15ec0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.376+0000 7f15e3fff700 1 -- 172.21.15.19:0/2683710446 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec027070 con 0x7f15fc1038f0 2024-05-09T02:08:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.376+0000 7f15e3fff700 1 -- 172.21.15.19:0/2683710446 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15ec022470 con 0x7f15fc1038f0 2024-05-09T02:08:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.376+0000 7f15e3fff700 1 -- 172.21.15.19:0/2683710446 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec017650 con 0x7f15fc1038f0 2024-05-09T02:08:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.376+0000 7f1601346700 1 -- 172.21.15.19:0/2683710446 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15fc19a460 con 0x7f15fc1038f0 2024-05-09T02:08:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.376+0000 7f1601346700 1 -- 172.21.15.19:0/2683710446 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15fc19a910 con 0x7f15fc1038f0 2024-05-09T02:08:49.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.377+0000 7f1601346700 1 -- 172.21.15.19:0/2683710446 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15fc0512c0 con 0x7f15fc1038f0 2024-05-09T02:08:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.379+0000 7f15e3fff700 1 -- 172.21.15.19:0/2683710446 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f15ec009d10 con 0x7f15fc1038f0 2024-05-09T02:08:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.380+0000 7f15e3fff700 1 --2- 172.21.15.19:0/2683710446 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f15e40b2910 0x7f15e40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.380+0000 7f15fa7fc700 1 --2- 172.21.15.19:0/2683710446 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f15e40b2910 0x7f15e40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.380+0000 7f15e3fff700 1 -- 172.21.15.19:0/2683710446 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f15ec0e2c80 con 0x7f15fc1038f0 2024-05-09T02:08:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.380+0000 7f15fa7fc700 1 --2- 172.21.15.19:0/2683710446 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f15e40b2910 0x7f15e40b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f15fc106850 tx=0x7f15f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.384+0000 7f15e3fff700 1 -- 172.21.15.19:0/2683710446 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15ec013070 con 0x7f15fc1038f0 2024-05-09T02:08:49.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:49 smithi019 ceph-mon[30151]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-09T02:08:49.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:49 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:49.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:49 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:49.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.623+0000 7f1601346700 1 -- 172.21.15.19:0/2683710446 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15fc10b180 con 0x7f15fc1038f0 2024-05-09T02:08:49.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.626+0000 7f15e3fff700 1 -- 172.21.15.19:0/2683710446 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22698 (secure 0 0 0) 0x7f15ec0aa850 con 0x7f15fc1038f0 2024-05-09T02:08:49.629 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:08:49.629 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"fsid":"a0611f3e-0da7-11ef-bc97-c7b262605968","created":"2024-05-09T01:59:27.591343+0000","modified":"2024-05-09T02:08:25.302927+0000","last_up_change":"2024-05-09T02:06:45.565065+0000","last_in_change":"2024-05-09T02:06:24.967186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T02:02:50.018959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T02:07:43.283868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T02:07:47.110614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"996c6a3d-3e3a-451c-8789-36006e0b3dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6803","nonce":3002373576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6805","nonce":3002373576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6809","nonce":3002373576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6807","nonce":3002373576}]},"public_addr":"172.21.15.19:6803/3002373576","cluster_addr":"172.21.15.19:6805/3002373576","heartbeat_back_addr":"172.21.15.19:6809/3002373576","heartbeat_front_addr":"172.21.15.19:6807/3002373576","state":["exists","up"]},{"osd":1,"uuid":"1ac68d09-1198-4e81-86c7-e37f7c41aa5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6801","nonce":827458353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6803","nonce":827458353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6807","nonce":827458353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6805","nonce":827458353}]},"public_addr":"172.21.15.59:6801/827458353","cluster_addr":"172.21.15.59:6803/827458353","heartbeat_back_addr":"172.21.15.59:6807/827458353","heartbeat_front_addr":"172.21.15.59:6805/827458353","state":["exists","up"]},{"osd":2,"uuid":"818a0de1-04e6-471d-83df-a68833011b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6801","nonce":1659504639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6803","nonce":1659504639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6807","nonce":1659504639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6805","nonce":1659504639}]},"public_addr":"172.21.15.196:6801/1659504639","cluster_addr":"172.21.15.196:6803/1659504639","heartbeat_back_addr":"172.21.15.196:6807/1659504639","heartbeat_front_addr":"172.21.15.196:6805/1659504639","state":["exists","up"]},{"osd":3,"uuid":"7b204854-86b8-454d-aee0-8bb5775ce828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6811","nonce":2707671599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6813","nonce":2707671599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6817","nonce":2707671599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6815","nonce":2707671599}]},"public_addr":"172.21.15.19:6811/2707671599","cluster_addr":"172.21.15.19:6813/2707671599","heartbeat_back_addr":"172.21.15.19:6817/2707671599","heartbeat_front_addr":"172.21.15.19:6815/2707671599","state":["exists","up"]},{"osd":4,"uuid":"091667f1-e887-4a94-8723-6f85b676fef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6809","nonce":223037330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6811","nonce":223037330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6815","nonce":223037330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6813","nonce":223037330}]},"public_addr":"172.21.15.59:6809/223037330","cluster_addr":"172.21.15.59:6811/223037330","heartbeat_back_addr":"172.21.15.59:6815/223037330","heartbeat_front_addr":"172.21.15.59:6813/223037330","state":["exists","up"]},{"osd":5,"uuid":"2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6809","nonce":2516161898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6811","nonce":2516161898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6815","nonce":2516161898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6813","nonce":2516161898}]},"public_addr":"172.21.15.196:6809/2516161898","cluster_addr":"172.21.15.196:6811/2516161898","heartbeat_back_addr":"172.21.15.196:6815/2516161898","heartbeat_front_addr":"172.21.15.196:6813/2516161898","state":["exists","up"]},{"osd":6,"uuid":"4e352851-1279-475f-ba8b-eaf9ca73df74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6819","nonce":2150451637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6821","nonce":2150451637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6825","nonce":2150451637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6823","nonce":2150451637}]},"public_addr":"172.21.15.19:6819/2150451637","cluster_addr":"172.21.15.19:6821/2150451637","heartbeat_back_addr":"172.21.15.19:6825/2150451637","heartbeat_front_addr":"172.21.15.19:6823/2150451637","state":["exists","up"]},{"osd":7,"uuid":"83f53f59-9a60-4d87-9a8a-67e5c6f57004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6817","nonce":967902053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6819","nonce":967902053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6823","nonce":967902053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6821","nonce":967902053}]},"public_addr":"172.21.15.59:6817/967902053","cluster_addr":"172.21.15.59:6819/967902053","heartbeat_back_addr":"172.21.15.59:6823/967902053","heartbeat_front_addr":"172.21.15.59:6821/967902053","state":["exists","up"]},{"osd":8,"uuid":"a8f1bfb5-1cc8-4a65-8396-239bd781e1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6817","nonce":2946064259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6819","nonce":2946064259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6823","nonce":2946064259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6821","nonce":2946064259}]},"public_addr":"172.21.15.196:6817/2946064259","cluster_addr":"172.21.15.196:6819/2946064259","heartbeat_back_addr":"172.21.15.196:6823/2946064259","heartbeat_front_addr":"172.21.15.196:6821/2946064259","state":["exists","up"]},{"osd":9,"uuid":"73c58dc3-fcb0-4440-92b3-bf41a2e58fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6827","nonce":1735017886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6829","nonce":1735017886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6833","nonce":1735017886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6831","nonce":1735017886}]},"public_addr":"172.21.15.19:6827/1735017886","cluster_addr":"172.21.15.19:6829/1735017886","heartbeat_back_addr":"172.21.15.19:6833/1735017886","heartbeat_front_addr":"172.21.15.19:6831/1735017886","state":["exists","up"]},{"osd":10,"uuid":"8325282f-7325-4880-ab7d-b972717be9dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6825","nonce":2932899995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6827","nonce":2932899995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6831","nonce":2932899995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6829","nonce":2932899995}]},"public_addr":"172.21.15.59:6825/2932899995","cluster_addr":"172.21.15.59:6827/2932899995","heartbeat_back_addr":"172.21.15.59:6831/2932899995","heartbeat_front_addr":"172.21.15.59:6829/2932899995","state":["exists","up"]},{"osd":11,"uuid":"8b19228c-b286-43c8-a408-169782f13075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6825","nonce":2608023001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6827","nonce":2608023001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6831","nonce":2608023001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6829","nonce":2608023001}]},"public_addr":"172.21.15.196:6825/2608023001","cluster_addr":"172.21.15.196:6827/2608023001","heartbeat_back_addr":"172.21.15.196:6831/2608023001","heartbeat_front_addr":"172.21.15.196:6829/2608023001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:01:59.457748+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:23.640666+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:47.768243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:13.016746+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:39.060898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:04.508313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:31.366217+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:57.579758+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:23.177396+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:50.966399+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:16.838666+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:43.959387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3291838961":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1024133854":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1176455101":"2024-05-10T02:00:23.920230+0000","172.21.15.19:6801/2101052000":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/4018987404":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/3079777324":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6801/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/179703528":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/2101052000":"2024-05-10T02:00:23.920230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T02:08:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.629+0000 7f1601346700 1 -- 172.21.15.19:0/2683710446 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f15e40b2910 msgr2=0x7f15e40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.629+0000 7f1601346700 1 --2- 172.21.15.19:0/2683710446 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f15e40b2910 0x7f15e40b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f15fc106850 tx=0x7f15f0009260 comp rx=0 tx=0).stop 2024-05-09T02:08:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.629+0000 7f1601346700 1 -- 172.21.15.19:0/2683710446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15fc1038f0 msgr2=0x7f15fc19f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.629+0000 7f1601346700 1 --2- 172.21.15.19:0/2683710446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15fc1038f0 0x7f15fc19f8b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f15ec000c00 tx=0x7f15ec0047b0 comp rx=0 tx=0).stop 2024-05-09T02:08:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.629+0000 7f1601346700 1 -- 172.21.15.19:0/2683710446 shutdown_connections 2024-05-09T02:08:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.629+0000 7f1601346700 1 --2- 172.21.15.19:0/2683710446 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f15e40b2910 0x7f15e40b4e20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.629+0000 7f1601346700 1 --2- 172.21.15.19:0/2683710446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc108200 0x7f15fc199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.629+0000 7f1601346700 1 --2- 172.21.15.19:0/2683710446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15fc1038f0 0x7f15fc19f8b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.629+0000 7f1601346700 1 --2- 172.21.15.19:0/2683710446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15fc104240 0x7f15fc19fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.629+0000 7f1601346700 1 -- 172.21.15.19:0/2683710446 >> 172.21.15.19:0/2683710446 conn(0x7f15fc0fe6f0 msgr2=0x7f15fc107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.629+0000 7f1601346700 1 -- 172.21.15.19:0/2683710446 shutdown_connections 2024-05-09T02:08:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:49.629+0000 7f1601346700 1 -- 172.21.15.19:0/2683710446 wait complete. 2024-05-09T02:08:49.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:49 smithi059 ceph-mon[34781]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-09T02:08:49.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:49 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:49.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:49 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:49.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:49 smithi196 ceph-mon[34718]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-09T02:08:49.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:49 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:49.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:49 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:50.275 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-09T02:08:50.275 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T02:08:50.540 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:50.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:08:50.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:08:50.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:50 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:50.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:50 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2683710446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:08:50.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:08:50.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:08:50.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:50 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:50.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:50 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2683710446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:08:50.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:08:50.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:08:50.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:50 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:08:50.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:50 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2683710446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:08:51.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:51 smithi019 ceph-mon[30151]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T02:08:51.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:51 smithi059 ceph-mon[34781]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T02:08:51.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:51 smithi196 ceph-mon[34718]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T02:08:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.256+0000 7f28f8c8e700 1 -- 172.21.15.19:0/1034966282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f41045f0 msgr2=0x7f28f4104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.256+0000 7f28f8c8e700 1 --2- 172.21.15.19:0/1034966282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f41045f0 0x7f28f4104a00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f28e40099c0 tx=0x7f28e4009cd0 comp rx=0 tx=0).stop 2024-05-09T02:08:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.257+0000 7f28f8c8e700 1 -- 172.21.15.19:0/1034966282 shutdown_connections 2024-05-09T02:08:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.257+0000 7f28f8c8e700 1 --2- 172.21.15.19:0/1034966282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28f4109870 0x7f28f41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.257+0000 7f28f8c8e700 1 --2- 172.21.15.19:0/1034966282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f41045f0 0x7f28f4104a00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.257+0000 7f28f8c8e700 1 --2- 172.21.15.19:0/1034966282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28f4108f20 0x7f28f4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.257+0000 7f28f8c8e700 1 -- 172.21.15.19:0/1034966282 >> 172.21.15.19:0/1034966282 conn(0x7f28f4100040 msgr2=0x7f28f4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.257+0000 7f28f8c8e700 1 -- 172.21.15.19:0/1034966282 shutdown_connections 2024-05-09T02:08:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.257+0000 7f28f8c8e700 1 -- 172.21.15.19:0/1034966282 wait complete. 2024-05-09T02:08:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.257+0000 7f28f8c8e700 1 Processor -- start 2024-05-09T02:08:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.257+0000 7f28f8c8e700 1 -- start start 2024-05-09T02:08:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f8c8e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f41045f0 0x7f28f4075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f8c8e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28f4108f20 0x7f28f4076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f8c8e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28f4109870 0x7f28f4076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f8c8e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f28f4114510 con 0x7f28f41045f0 2024-05-09T02:08:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f8c8e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f28f4114390 con 0x7f28f4108f20 2024-05-09T02:08:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f8c8e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28f4114690 con 0x7f28f4109870 2024-05-09T02:08:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f41045f0 0x7f28f4075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f41045f0 0x7f28f4075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43464/0 (socket says 172.21.15.19:43464) 2024-05-09T02:08:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f259c700 1 -- 172.21.15.19:0/844242630 learned_addr learned my addr 172.21.15.19:0/844242630 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f1d9b700 1 --2- 172.21.15.19:0/844242630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28f4108f20 0x7f28f4076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f2d9d700 1 --2- 172.21.15.19:0/844242630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28f4109870 0x7f28f4076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f259c700 1 -- 172.21.15.19:0/844242630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28f4109870 msgr2=0x7f28f4076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f259c700 1 --2- 172.21.15.19:0/844242630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28f4109870 0x7f28f4076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f259c700 1 -- 172.21.15.19:0/844242630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28f4108f20 msgr2=0x7f28f4076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f259c700 1 --2- 172.21.15.19:0/844242630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28f4108f20 0x7f28f4076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f259c700 1 -- 172.21.15.19:0/844242630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f41a9260 con 0x7f28f41045f0 2024-05-09T02:08:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f2d9d700 1 --2- 172.21.15.19:0/844242630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28f4109870 0x7f28f4076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:08:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.258+0000 7f28f259c700 1 --2- 172.21.15.19:0/844242630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f41045f0 0x7f28f4075ac0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f28e400b580 tx=0x7f28e40037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.259+0000 7f28e37fe700 1 -- 172.21.15.19:0/844242630 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4027070 con 0x7f28f41045f0 2024-05-09T02:08:52.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.259+0000 7f28e37fe700 1 -- 172.21.15.19:0/844242630 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e4004110 con 0x7f28f41045f0 2024-05-09T02:08:52.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.259+0000 7f28f8c8e700 1 -- 172.21.15.19:0/844242630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f41a94f0 con 0x7f28f41045f0 2024-05-09T02:08:52.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.259+0000 7f28e37fe700 1 -- 172.21.15.19:0/844242630 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e400f460 con 0x7f28f41045f0 2024-05-09T02:08:52.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.259+0000 7f28f8c8e700 1 -- 172.21.15.19:0/844242630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f41a9940 con 0x7f28f41045f0 2024-05-09T02:08:52.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.260+0000 7f28f8c8e700 1 -- 172.21.15.19:0/844242630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f40512c0 con 0x7f28f41045f0 2024-05-09T02:08:52.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.261+0000 7f28e37fe700 1 -- 172.21.15.19:0/844242630 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f28e402a070 con 0x7f28f41045f0 2024-05-09T02:08:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.262+0000 7f28e37fe700 1 --2- 172.21.15.19:0/844242630 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f28dc0b2910 0x7f28dc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.262+0000 7f28f1d9b700 1 --2- 172.21.15.19:0/844242630 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f28dc0b2910 0x7f28dc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.262+0000 7f28e37fe700 1 -- 172.21.15.19:0/844242630 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f28e40e37c0 con 0x7f28f41045f0 2024-05-09T02:08:52.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.263+0000 7f28f1d9b700 1 --2- 172.21.15.19:0/844242630 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f28dc0b2910 0x7f28dc0b4e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f28e8006fd0 tx=0x7f28e8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:52.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.266+0000 7f28e37fe700 1 -- 172.21.15.19:0/844242630 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28e4031080 con 0x7f28f41045f0 2024-05-09T02:08:52.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.484+0000 7f28f8c8e700 1 -- 172.21.15.19:0/844242630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28f402e7b0 con 0x7f28f41045f0 2024-05-09T02:08:52.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.486+0000 7f28e37fe700 1 -- 172.21.15.19:0/844242630 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22698 (secure 0 0 0) 0x7f28e4013070 con 0x7f28f41045f0 2024-05-09T02:08:52.489 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:08:52.489 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"fsid":"a0611f3e-0da7-11ef-bc97-c7b262605968","created":"2024-05-09T01:59:27.591343+0000","modified":"2024-05-09T02:08:25.302927+0000","last_up_change":"2024-05-09T02:06:45.565065+0000","last_in_change":"2024-05-09T02:06:24.967186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T02:02:50.018959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T02:07:43.283868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T02:07:47.110614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"996c6a3d-3e3a-451c-8789-36006e0b3dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6803","nonce":3002373576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6805","nonce":3002373576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6809","nonce":3002373576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6807","nonce":3002373576}]},"public_addr":"172.21.15.19:6803/3002373576","cluster_addr":"172.21.15.19:6805/3002373576","heartbeat_back_addr":"172.21.15.19:6809/3002373576","heartbeat_front_addr":"172.21.15.19:6807/3002373576","state":["exists","up"]},{"osd":1,"uuid":"1ac68d09-1198-4e81-86c7-e37f7c41aa5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6801","nonce":827458353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6803","nonce":827458353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6807","nonce":827458353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6805","nonce":827458353}]},"public_addr":"172.21.15.59:6801/827458353","cluster_addr":"172.21.15.59:6803/827458353","heartbeat_back_addr":"172.21.15.59:6807/827458353","heartbeat_front_addr":"172.21.15.59:6805/827458353","state":["exists","up"]},{"osd":2,"uuid":"818a0de1-04e6-471d-83df-a68833011b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6801","nonce":1659504639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6803","nonce":1659504639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6807","nonce":1659504639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6805","nonce":1659504639}]},"public_addr":"172.21.15.196:6801/1659504639","cluster_addr":"172.21.15.196:6803/1659504639","heartbeat_back_addr":"172.21.15.196:6807/1659504639","heartbeat_front_addr":"172.21.15.196:6805/1659504639","state":["exists","up"]},{"osd":3,"uuid":"7b204854-86b8-454d-aee0-8bb5775ce828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6811","nonce":2707671599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6813","nonce":2707671599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6817","nonce":2707671599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6815","nonce":2707671599}]},"public_addr":"172.21.15.19:6811/2707671599","cluster_addr":"172.21.15.19:6813/2707671599","heartbeat_back_addr":"172.21.15.19:6817/2707671599","heartbeat_front_addr":"172.21.15.19:6815/2707671599","state":["exists","up"]},{"osd":4,"uuid":"091667f1-e887-4a94-8723-6f85b676fef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6809","nonce":223037330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6811","nonce":223037330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6815","nonce":223037330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6813","nonce":223037330}]},"public_addr":"172.21.15.59:6809/223037330","cluster_addr":"172.21.15.59:6811/223037330","heartbeat_back_addr":"172.21.15.59:6815/223037330","heartbeat_front_addr":"172.21.15.59:6813/223037330","state":["exists","up"]},{"osd":5,"uuid":"2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6809","nonce":2516161898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6811","nonce":2516161898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6815","nonce":2516161898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6813","nonce":2516161898}]},"public_addr":"172.21.15.196:6809/2516161898","cluster_addr":"172.21.15.196:6811/2516161898","heartbeat_back_addr":"172.21.15.196:6815/2516161898","heartbeat_front_addr":"172.21.15.196:6813/2516161898","state":["exists","up"]},{"osd":6,"uuid":"4e352851-1279-475f-ba8b-eaf9ca73df74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6819","nonce":2150451637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6821","nonce":2150451637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6825","nonce":2150451637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6823","nonce":2150451637}]},"public_addr":"172.21.15.19:6819/2150451637","cluster_addr":"172.21.15.19:6821/2150451637","heartbeat_back_addr":"172.21.15.19:6825/2150451637","heartbeat_front_addr":"172.21.15.19:6823/2150451637","state":["exists","up"]},{"osd":7,"uuid":"83f53f59-9a60-4d87-9a8a-67e5c6f57004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6817","nonce":967902053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6819","nonce":967902053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6823","nonce":967902053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6821","nonce":967902053}]},"public_addr":"172.21.15.59:6817/967902053","cluster_addr":"172.21.15.59:6819/967902053","heartbeat_back_addr":"172.21.15.59:6823/967902053","heartbeat_front_addr":"172.21.15.59:6821/967902053","state":["exists","up"]},{"osd":8,"uuid":"a8f1bfb5-1cc8-4a65-8396-239bd781e1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6817","nonce":2946064259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6819","nonce":2946064259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6823","nonce":2946064259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6821","nonce":2946064259}]},"public_addr":"172.21.15.196:6817/2946064259","cluster_addr":"172.21.15.196:6819/2946064259","heartbeat_back_addr":"172.21.15.196:6823/2946064259","heartbeat_front_addr":"172.21.15.196:6821/2946064259","state":["exists","up"]},{"osd":9,"uuid":"73c58dc3-fcb0-4440-92b3-bf41a2e58fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6827","nonce":1735017886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6829","nonce":1735017886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6833","nonce":1735017886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6831","nonce":1735017886}]},"public_addr":"172.21.15.19:6827/1735017886","cluster_addr":"172.21.15.19:6829/1735017886","heartbeat_back_addr":"172.21.15.19:6833/1735017886","heartbeat_front_addr":"172.21.15.19:6831/1735017886","state":["exists","up"]},{"osd":10,"uuid":"8325282f-7325-4880-ab7d-b972717be9dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6825","nonce":2932899995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6827","nonce":2932899995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6831","nonce":2932899995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6829","nonce":2932899995}]},"public_addr":"172.21.15.59:6825/2932899995","cluster_addr":"172.21.15.59:6827/2932899995","heartbeat_back_addr":"172.21.15.59:6831/2932899995","heartbeat_front_addr":"172.21.15.59:6829/2932899995","state":["exists","up"]},{"osd":11,"uuid":"8b19228c-b286-43c8-a408-169782f13075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6825","nonce":2608023001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6827","nonce":2608023001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6831","nonce":2608023001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6829","nonce":2608023001}]},"public_addr":"172.21.15.196:6825/2608023001","cluster_addr":"172.21.15.196:6827/2608023001","heartbeat_back_addr":"172.21.15.196:6831/2608023001","heartbeat_front_addr":"172.21.15.196:6829/2608023001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:01:59.457748+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:23.640666+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:47.768243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:13.016746+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:39.060898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:04.508313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:31.366217+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:57.579758+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:23.177396+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:50.966399+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:16.838666+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:43.959387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3291838961":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1024133854":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1176455101":"2024-05-10T02:00:23.920230+0000","172.21.15.19:6801/2101052000":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/4018987404":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/3079777324":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6801/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/179703528":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/2101052000":"2024-05-10T02:00:23.920230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T02:08:52.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.489+0000 7f28f8c8e700 1 -- 172.21.15.19:0/844242630 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f28dc0b2910 msgr2=0x7f28dc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:52.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.489+0000 7f28f8c8e700 1 --2- 172.21.15.19:0/844242630 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f28dc0b2910 0x7f28dc0b4e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f28e8006fd0 tx=0x7f28e8008040 comp rx=0 tx=0).stop 2024-05-09T02:08:52.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.489+0000 7f28f8c8e700 1 -- 172.21.15.19:0/844242630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f41045f0 msgr2=0x7f28f4075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:52.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.489+0000 7f28f8c8e700 1 --2- 172.21.15.19:0/844242630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f41045f0 0x7f28f4075ac0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f28e400b580 tx=0x7f28e40037c0 comp rx=0 tx=0).stop 2024-05-09T02:08:52.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.489+0000 7f28f8c8e700 1 -- 172.21.15.19:0/844242630 shutdown_connections 2024-05-09T02:08:52.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.489+0000 7f28f8c8e700 1 --2- 172.21.15.19:0/844242630 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f28dc0b2910 0x7f28dc0b4e20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:52.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.489+0000 7f28f8c8e700 1 --2- 172.21.15.19:0/844242630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28f4109870 0x7f28f4076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:52.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.489+0000 7f28f8c8e700 1 --2- 172.21.15.19:0/844242630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f41045f0 0x7f28f4075ac0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:52.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.489+0000 7f28f8c8e700 1 --2- 172.21.15.19:0/844242630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28f4108f20 0x7f28f4076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:52.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.489+0000 7f28f8c8e700 1 -- 172.21.15.19:0/844242630 >> 172.21.15.19:0/844242630 conn(0x7f28f4100040 msgr2=0x7f28f410a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:52.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.489+0000 7f28f8c8e700 1 -- 172.21.15.19:0/844242630 shutdown_connections 2024-05-09T02:08:52.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:52.489+0000 7f28f8c8e700 1 -- 172.21.15.19:0/844242630 wait complete. 2024-05-09T02:08:53.211 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-09T02:08:53.211 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-09T02:08:53.212 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-09T02:08:53.212 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-09T02:08:53.212 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-09T02:08:53.213 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-09T02:08:53.213 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-09T02:08:53.214 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-09T02:08:53.214 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-09T02:08:53.214 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-09T02:08:53.215 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-09T02:08:53.215 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-09T02:08:53.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:53 smithi019 ceph-mon[30151]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:08:53.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:53 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/844242630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:08:53.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:53 smithi059 ceph-mon[34781]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:08:53.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:53 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/844242630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:08:53.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:53 smithi196 ceph-mon[34718]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:08:53.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:53 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/844242630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:08:53.862 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:53.871 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:55.079 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:55.081 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:55.083 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:55.085 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:55.098 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:55.099 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:55.099 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:55.103 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:08:55.633 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:55 smithi019 ceph-mon[30151]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:08:55.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:55 smithi059 ceph-mon[34781]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:08:55.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:55 smithi196 ceph-mon[34718]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:08:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.354+0000 7ffaa717c700 1 -- 172.21.15.19:0/2684758420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa0108760 msgr2=0x7ffaa01100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.354+0000 7ffaa717c700 1 --2- 172.21.15.19:0/2684758420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa0108760 0x7ffaa01100c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800b0d0 tx=0x7ffa9800b3e0 comp rx=0 tx=0).stop 2024-05-09T02:08:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.354+0000 7ffaa717c700 1 -- 172.21.15.19:0/2684758420 shutdown_connections 2024-05-09T02:08:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.354+0000 7ffaa717c700 1 --2- 172.21.15.19:0/2684758420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa0108760 0x7ffaa01100c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.354+0000 7ffaa717c700 1 --2- 172.21.15.19:0/2684758420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaa01034e0 0x7ffaa01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.354+0000 7ffaa717c700 1 --2- 172.21.15.19:0/2684758420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa0107e10 0x7ffaa0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.354+0000 7ffaa717c700 1 -- 172.21.15.19:0/2684758420 >> 172.21.15.19:0/2684758420 conn(0x7ffaa00fef30 msgr2=0x7ffaa0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.354+0000 7ffaa717c700 1 -- 172.21.15.19:0/2684758420 shutdown_connections 2024-05-09T02:08:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.354+0000 7ffaa717c700 1 -- 172.21.15.19:0/2684758420 wait complete. 2024-05-09T02:08:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.355+0000 7ffaa717c700 1 Processor -- start 2024-05-09T02:08:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.355+0000 7ffaa717c700 1 -- start start 2024-05-09T02:08:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.355+0000 7ffaa717c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa01034e0 0x7ffaa019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.355+0000 7ffaa717c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaa0107e10 0x7ffaa019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.355+0000 7ffaa717c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa0108760 0x7ffaa019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.355+0000 7ffaa717c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa0113460 con 0x7ffaa0108760 2024-05-09T02:08:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.355+0000 7ffaa717c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa01132e0 con 0x7ffaa01034e0 2024-05-09T02:08:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.355+0000 7ffaa717c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa01135e0 con 0x7ffaa0107e10 2024-05-09T02:08:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.355+0000 7ffaa5719700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa0108760 0x7ffaa019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.355+0000 7ffaa5719700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa0108760 0x7ffaa019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43478/0 (socket says 172.21.15.19:43478) 2024-05-09T02:08:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.355+0000 7ffaa4f18700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa01034e0 0x7ffaa019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.355+0000 7ffaa5719700 1 -- 172.21.15.19:0/3635931135 learned_addr learned my addr 172.21.15.19:0/3635931135 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:08:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.355+0000 7ffa9ffff700 1 --2- 172.21.15.19:0/3635931135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaa0107e10 0x7ffaa019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.356+0000 7ffaa5719700 1 -- 172.21.15.19:0/3635931135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaa0107e10 msgr2=0x7ffaa019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.356+0000 7ffaa5719700 1 --2- 172.21.15.19:0/3635931135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaa0107e10 0x7ffaa019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.356+0000 7ffaa5719700 1 -- 172.21.15.19:0/3635931135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa01034e0 msgr2=0x7ffaa019aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.356+0000 7ffaa5719700 1 --2- 172.21.15.19:0/3635931135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa01034e0 0x7ffaa019aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.356+0000 7ffaa5719700 1 -- 172.21.15.19:0/3635931135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa019fcc0 con 0x7ffaa0108760 2024-05-09T02:08:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.356+0000 7ffaa5719700 1 --2- 172.21.15.19:0/3635931135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa0108760 0x7ffaa019f6d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffa980079a0 tx=0x7ffa980079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.356+0000 7ffa9dffb700 1 -- 172.21.15.19:0/3635931135 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9801f070 con 0x7ffaa0108760 2024-05-09T02:08:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.356+0000 7ffaa717c700 1 -- 172.21.15.19:0/3635931135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa019ff50 con 0x7ffaa0108760 2024-05-09T02:08:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.356+0000 7ffa9dffb700 1 -- 172.21.15.19:0/3635931135 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa98003c00 con 0x7ffaa0108760 2024-05-09T02:08:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.356+0000 7ffa9dffb700 1 -- 172.21.15.19:0/3635931135 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9801b6a0 con 0x7ffaa0108760 2024-05-09T02:08:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.356+0000 7ffaa717c700 1 -- 172.21.15.19:0/3635931135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa01a0480 con 0x7ffaa0108760 2024-05-09T02:08:56.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.357+0000 7ffaa717c700 1 -- 172.21.15.19:0/3635931135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffaa0063f50 con 0x7ffaa0108760 2024-05-09T02:08:56.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.359+0000 7ffa9dffb700 1 -- 172.21.15.19:0/3635931135 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ffa98022070 con 0x7ffaa0108760 2024-05-09T02:08:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.360+0000 7ffa9dffb700 1 --2- 172.21.15.19:0/3635931135 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffa900b2910 0x7ffa900b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.360+0000 7ffaa4f18700 1 --2- 172.21.15.19:0/3635931135 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffa900b2910 0x7ffa900b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.360+0000 7ffa9dffb700 1 -- 172.21.15.19:0/3635931135 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7ffa9800f020 con 0x7ffaa0108760 2024-05-09T02:08:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.360+0000 7ffa9dffb700 1 --2- 172.21.15.19:0/3635931135 >> [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001] conn(0x7ffa900bc5a0 0x7ffa900bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:08:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.360+0000 7ffa9dffb700 1 -- 172.21.15.19:0/3635931135 --> [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa980075c0 con 0x7ffa900bc5a0 2024-05-09T02:08:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.360+0000 7ffa9dffb700 1 -- 172.21.15.19:0/3635931135 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7ffa980e3d50 con 0x7ffaa0108760 2024-05-09T02:08:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.360+0000 7ffaa4f18700 1 --2- 172.21.15.19:0/3635931135 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffa900b2910 0x7ffa900b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009a10 tx=0x7ffa94009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.360+0000 7ffa9ffff700 1 --2- 172.21.15.19:0/3635931135 >> [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001] conn(0x7ffa900bc5a0 0x7ffa900bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:08:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.361+0000 7ffa9ffff700 1 --2- 172.21.15.19:0/3635931135 >> [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001] conn(0x7ffa900bc5a0 0x7ffa900bea90 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:08:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.362+0000 7ffa9dffb700 1 -- 172.21.15.19:0/3635931135 <== osd.11 v2:172.21.15.196:6824/2608023001 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa980075c0 con 0x7ffa900bc5a0 2024-05-09T02:08:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.378+0000 7ffaa717c700 1 -- 172.21.15.19:0/3635931135 --> [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffaa0064380 con 0x7ffa900bc5a0 2024-05-09T02:08:56.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.379+0000 7ffa9dffb700 1 -- 172.21.15.19:0/3635931135 <== osd.11 v2:172.21.15.196:6824/2608023001 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffaa0064380 con 0x7ffa900bc5a0 2024-05-09T02:08:56.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.379+0000 7ffaa717c700 1 -- 172.21.15.19:0/3635931135 >> [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001] conn(0x7ffa900bc5a0 msgr2=0x7ffa900bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:56.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.379+0000 7ffaa717c700 1 --2- 172.21.15.19:0/3635931135 >> [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001] conn(0x7ffa900bc5a0 0x7ffa900bea90 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:08:56.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.379+0000 7ffaa717c700 1 -- 172.21.15.19:0/3635931135 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffa900b2910 msgr2=0x7ffa900b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:56.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.379+0000 7ffaa717c700 1 --2- 172.21.15.19:0/3635931135 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffa900b2910 0x7ffa900b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009a10 tx=0x7ffa94009190 comp rx=0 tx=0).stop 2024-05-09T02:08:56.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.379+0000 7ffaa717c700 1 -- 172.21.15.19:0/3635931135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa0108760 msgr2=0x7ffaa019f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:08:56.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.379+0000 7ffaa717c700 1 --2- 172.21.15.19:0/3635931135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa0108760 0x7ffaa019f6d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffa980079a0 tx=0x7ffa980079d0 comp rx=0 tx=0).stop 2024-05-09T02:08:56.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.379+0000 7ffaa5719700 1 -- 172.21.15.19:0/3635931135 reap_dead start 2024-05-09T02:08:56.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.380+0000 7ffaa717c700 1 -- 172.21.15.19:0/3635931135 shutdown_connections 2024-05-09T02:08:56.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.380+0000 7ffaa717c700 1 -- 172.21.15.19:0/3635931135 >> 172.21.15.19:0/3635931135 conn(0x7ffaa00fef30 msgr2=0x7ffaa006c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:08:56.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.380+0000 7ffaa717c700 1 -- 172.21.15.19:0/3635931135 shutdown_connections 2024-05-09T02:08:56.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:08:56.380+0000 7ffaa717c700 1 -- 172.21.15.19:0/3635931135 wait complete. 2024-05-09T02:08:57.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:57 smithi019 ceph-mon[30151]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:08:57.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:57 smithi059 ceph-mon[34781]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:08:57.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:57 smithi196 ceph-mon[34718]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:08:59.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:08:59 smithi019 ceph-mon[30151]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:08:59.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:08:59 smithi059 ceph-mon[34781]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:08:59.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:08:59 smithi196 ceph-mon[34718]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:01.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:01 smithi019 ceph-mon[30151]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:09:01.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:01 smithi059 ceph-mon[34781]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:09:01.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:01 smithi196 ceph-mon[34718]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:09:02.138 INFO:teuthology.orchestra.run.smithi019.stdout:304942678046 2024-05-09T02:09:02.138 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-09T02:09:03.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.099+0000 7f11584be700 1 -- 172.21.15.19:0/989125222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1150104650 msgr2=0x7f1150104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:03.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.099+0000 7f11584be700 1 --2- 172.21.15.19:0/989125222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1150104650 0x7f1150104a60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f114c0099c0 tx=0x7f114c009cd0 comp rx=0 tx=0).stop 2024-05-09T02:09:03.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.099+0000 7f11584be700 1 -- 172.21.15.19:0/989125222 shutdown_connections 2024-05-09T02:09:03.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.099+0000 7f11584be700 1 --2- 172.21.15.19:0/989125222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1150109880 0x7f11501111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:03.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.099+0000 7f11584be700 1 --2- 172.21.15.19:0/989125222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1150104650 0x7f1150104a60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:03.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.099+0000 7f11584be700 1 --2- 172.21.15.19:0/989125222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1150108f30 0x7f1150109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:03.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.099+0000 7f11584be700 1 -- 172.21.15.19:0/989125222 >> 172.21.15.19:0/989125222 conn(0x7f1150079f00 msgr2=0x7f115007a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:03.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.099+0000 7f11584be700 1 -- 172.21.15.19:0/989125222 shutdown_connections 2024-05-09T02:09:03.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.099+0000 7f11584be700 1 -- 172.21.15.19:0/989125222 wait complete. 2024-05-09T02:09:03.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.099+0000 7f11584be700 1 Processor -- start 2024-05-09T02:09:03.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.099+0000 7f11584be700 1 -- start start 2024-05-09T02:09:03.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f11584be700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1150104650 0x7f1150075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:03.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f11584be700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1150108f30 0x7f1150076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:03.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f11584be700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1150109880 0x7f1150076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:03.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f11584be700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11501144b0 con 0x7f1150109880 2024-05-09T02:09:03.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f11584be700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1150114330 con 0x7f1150104650 2024-05-09T02:09:03.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f11584be700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1150114630 con 0x7f1150108f30 2024-05-09T02:09:03.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f1156a5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1150109880 0x7f1150076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:03.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f1156a5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1150109880 0x7f1150076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34978/0 (socket says 172.21.15.19:34978) 2024-05-09T02:09:03.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f1156a5b700 1 -- 172.21.15.19:0/606700980 learned_addr learned my addr 172.21.15.19:0/606700980 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:03.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f115625a700 1 --2- 172.21.15.19:0/606700980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1150104650 0x7f1150075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f1155a59700 1 --2- 172.21.15.19:0/606700980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1150108f30 0x7f1150076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f1156a5b700 1 -- 172.21.15.19:0/606700980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1150108f30 msgr2=0x7f1150076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f1156a5b700 1 --2- 172.21.15.19:0/606700980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1150108f30 0x7f1150076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f1156a5b700 1 -- 172.21.15.19:0/606700980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1150104650 msgr2=0x7f1150075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f1156a5b700 1 --2- 172.21.15.19:0/606700980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1150104650 0x7f1150075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f1156a5b700 1 -- 172.21.15.19:0/606700980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11501a9240 con 0x7f1150109880 2024-05-09T02:09:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f1155a59700 1 --2- 172.21.15.19:0/606700980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1150108f30 0x7f1150076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:09:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.100+0000 7f1156a5b700 1 --2- 172.21.15.19:0/606700980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1150109880 0x7f1150076540 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1148007ee0 tx=0x7f114800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.101+0000 7f11477fe700 1 -- 172.21.15.19:0/606700980 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114800dbd0 con 0x7f1150109880 2024-05-09T02:09:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.101+0000 7f11477fe700 1 -- 172.21.15.19:0/606700980 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114800dd70 con 0x7f1150109880 2024-05-09T02:09:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.101+0000 7f11584be700 1 -- 172.21.15.19:0/606700980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11501a9530 con 0x7f1150109880 2024-05-09T02:09:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.101+0000 7f11477fe700 1 -- 172.21.15.19:0/606700980 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148005990 con 0x7f1150109880 2024-05-09T02:09:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.101+0000 7f11584be700 1 -- 172.21.15.19:0/606700980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11501a9a70 con 0x7f1150109880 2024-05-09T02:09:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.101+0000 7f11584be700 1 -- 172.21.15.19:0/606700980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1150063ed0 con 0x7f1150109880 2024-05-09T02:09:03.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.103+0000 7f11477fe700 1 -- 172.21.15.19:0/606700980 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f114801a020 con 0x7f1150109880 2024-05-09T02:09:03.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.104+0000 7f11477fe700 1 --2- 172.21.15.19:0/606700980 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f113c0b2990 0x7f113c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:03.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.105+0000 7f115625a700 1 --2- 172.21.15.19:0/606700980 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f113c0b2990 0x7f113c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:03.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.105+0000 7f11477fe700 1 -- 172.21.15.19:0/606700980 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f11480e24d0 con 0x7f1150109880 2024-05-09T02:09:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.105+0000 7f11477fe700 1 --2- 172.21.15.19:0/606700980 >> [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576] conn(0x7f113c0bc620 0x7f113c0beb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.105+0000 7f11477fe700 1 -- 172.21.15.19:0/606700980 --> [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f113c0bf1e0 con 0x7f113c0bc620 2024-05-09T02:09:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.105+0000 7f11477fe700 1 -- 172.21.15.19:0/606700980 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f11480e2920 con 0x7f1150109880 2024-05-09T02:09:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.105+0000 7f1155a59700 1 --2- 172.21.15.19:0/606700980 >> [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576] conn(0x7f113c0bc620 0x7f113c0beb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.105+0000 7f115625a700 1 --2- 172.21.15.19:0/606700980 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f113c0b2990 0x7f113c0b4ea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f114c009990 tx=0x7f114c0036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.106+0000 7f1155a59700 1 --2- 172.21.15.19:0/606700980 >> [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576] conn(0x7f113c0bc620 0x7f113c0beb10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:03.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.106+0000 7f11477fe700 1 -- 172.21.15.19:0/606700980 <== osd.0 v2:172.21.15.19:6802/3002373576 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f113c0bf1e0 con 0x7f113c0bc620 2024-05-09T02:09:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.123+0000 7f11584be700 1 -- 172.21.15.19:0/606700980 --> [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f11500642d0 con 0x7f113c0bc620 2024-05-09T02:09:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.123+0000 7f11477fe700 1 -- 172.21.15.19:0/606700980 <== osd.0 v2:172.21.15.19:6802/3002373576 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f11500642d0 con 0x7f113c0bc620 2024-05-09T02:09:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.124+0000 7f11584be700 1 -- 172.21.15.19:0/606700980 >> [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576] conn(0x7f113c0bc620 msgr2=0x7f113c0beb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.124+0000 7f11584be700 1 --2- 172.21.15.19:0/606700980 >> [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576] conn(0x7f113c0bc620 0x7f113c0beb10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.124+0000 7f11584be700 1 -- 172.21.15.19:0/606700980 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f113c0b2990 msgr2=0x7f113c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.124+0000 7f11584be700 1 --2- 172.21.15.19:0/606700980 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f113c0b2990 0x7f113c0b4ea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f114c009990 tx=0x7f114c0036e0 comp rx=0 tx=0).stop 2024-05-09T02:09:03.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.124+0000 7f11584be700 1 -- 172.21.15.19:0/606700980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1150109880 msgr2=0x7f1150076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:03.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.124+0000 7f11584be700 1 --2- 172.21.15.19:0/606700980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1150109880 0x7f1150076540 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1148007ee0 tx=0x7f114800d3b0 comp rx=0 tx=0).stop 2024-05-09T02:09:03.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.124+0000 7f1156a5b700 1 -- 172.21.15.19:0/606700980 reap_dead start 2024-05-09T02:09:03.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.124+0000 7f11584be700 1 -- 172.21.15.19:0/606700980 shutdown_connections 2024-05-09T02:09:03.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.124+0000 7f11584be700 1 -- 172.21.15.19:0/606700980 >> 172.21.15.19:0/606700980 conn(0x7f1150079f00 msgr2=0x7f11501020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:03.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.124+0000 7f11584be700 1 -- 172.21.15.19:0/606700980 shutdown_connections 2024-05-09T02:09:03.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:03.124+0000 7f11584be700 1 -- 172.21.15.19:0/606700980 wait complete. 2024-05-09T02:09:03.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:03 smithi059 ceph-mon[34781]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:03.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:03 smithi196 ceph-mon[34718]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:03.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:03 smithi019 ceph-mon[30151]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:05.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.049+0000 7fc4f4aec700 1 -- 172.21.15.19:0/292601436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4f0108740 msgr2=0x7fc4f01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:05.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.049+0000 7fc4f4aec700 1 --2- 172.21.15.19:0/292601436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4f0108740 0x7fc4f01100a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800b0d0 tx=0x7fc4e800b3e0 comp rx=0 tx=0).stop 2024-05-09T02:09:05.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.049+0000 7fc4f4aec700 1 -- 172.21.15.19:0/292601436 shutdown_connections 2024-05-09T02:09:05.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.049+0000 7fc4f4aec700 1 --2- 172.21.15.19:0/292601436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4f0108740 0x7fc4f01100a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:05.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.049+0000 7fc4f4aec700 1 --2- 172.21.15.19:0/292601436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4f01034c0 0x7fc4f01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.049+0000 7fc4f4aec700 1 --2- 172.21.15.19:0/292601436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f0107df0 0x7fc4f0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.049+0000 7fc4f4aec700 1 -- 172.21.15.19:0/292601436 >> 172.21.15.19:0/292601436 conn(0x7fc4f00fedf0 msgr2=0x7fc4f0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.049+0000 7fc4f4aec700 1 -- 172.21.15.19:0/292601436 shutdown_connections 2024-05-09T02:09:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.049+0000 7fc4f4aec700 1 -- 172.21.15.19:0/292601436 wait complete. 2024-05-09T02:09:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.050+0000 7fc4f4aec700 1 Processor -- start 2024-05-09T02:09:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.050+0000 7fc4f4aec700 1 -- start start 2024-05-09T02:09:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4f4aec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f01034c0 0x7fc4f019bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4f4aec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4f0107df0 0x7fc4f019c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4f4aec700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4f0108740 0x7fc4f019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4f4aec700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01133e0 con 0x7fc4f0107df0 2024-05-09T02:09:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4f4aec700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f0113260 con 0x7fc4f01034c0 2024-05-09T02:09:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4f4aec700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f0113560 con 0x7fc4f0108740 2024-05-09T02:09:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4edd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4f0107df0 0x7fc4f019c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4edd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4f0107df0 0x7fc4f019c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34988/0 (socket says 172.21.15.19:34988) 2024-05-09T02:09:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4edd9b700 1 -- 172.21.15.19:0/1744398285 learned_addr learned my addr 172.21.15.19:0/1744398285 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4eed9d700 1 --2- 172.21.15.19:0/1744398285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4f0108740 0x7fc4f019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4ee59c700 1 --2- 172.21.15.19:0/1744398285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f01034c0 0x7fc4f019bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4edd9b700 1 -- 172.21.15.19:0/1744398285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4f0108740 msgr2=0x7fc4f019c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4edd9b700 1 --2- 172.21.15.19:0/1744398285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4f0108740 0x7fc4f019c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4edd9b700 1 -- 172.21.15.19:0/1744398285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f01034c0 msgr2=0x7fc4f019bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4edd9b700 1 --2- 172.21.15.19:0/1744398285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f01034c0 0x7fc4f019bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4edd9b700 1 -- 172.21.15.19:0/1744398285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f0199a90 con 0x7fc4f0107df0 2024-05-09T02:09:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.051+0000 7fc4edd9b700 1 --2- 172.21.15.19:0/1744398285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4f0107df0 0x7fc4f019c1e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800b500 tx=0x7fc4d800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.052+0000 7fc4e77fe700 1 -- 172.21.15.19:0/1744398285 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8015020 con 0x7fc4f0107df0 2024-05-09T02:09:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.052+0000 7fc4e77fe700 1 -- 172.21.15.19:0/1744398285 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4d8011ce0 con 0x7fc4f0107df0 2024-05-09T02:09:05.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.052+0000 7fc4e77fe700 1 -- 172.21.15.19:0/1744398285 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8010cb0 con 0x7fc4f0107df0 2024-05-09T02:09:05.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.052+0000 7fc4f4aec700 1 -- 172.21.15.19:0/1744398285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f0199d80 con 0x7fc4f0107df0 2024-05-09T02:09:05.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.052+0000 7fc4f4aec700 1 -- 172.21.15.19:0/1744398285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f019a270 con 0x7fc4f0107df0 2024-05-09T02:09:05.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.053+0000 7fc4f4aec700 1 -- 172.21.15.19:0/1744398285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc4f0063ed0 con 0x7fc4f0107df0 2024-05-09T02:09:05.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.055+0000 7fc4e77fe700 1 -- 172.21.15.19:0/1744398285 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc4d8018020 con 0x7fc4f0107df0 2024-05-09T02:09:05.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.056+0000 7fc4e77fe700 1 --2- 172.21.15.19:0/1744398285 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc4dc0b2910 0x7fc4dc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:05.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.056+0000 7fc4e77fe700 1 -- 172.21.15.19:0/1744398285 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc4d80e2050 con 0x7fc4f0107df0 2024-05-09T02:09:05.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.056+0000 7fc4ee59c700 1 --2- 172.21.15.19:0/1744398285 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc4dc0b2910 0x7fc4dc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:05.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.056+0000 7fc4e77fe700 1 --2- 172.21.15.19:0/1744398285 >> [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639] conn(0x7fc4dc0bc5a0 0x7fc4dc0bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:05.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.056+0000 7fc4e77fe700 1 -- 172.21.15.19:0/1744398285 --> [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4d800fec0 con 0x7fc4dc0bc5a0 2024-05-09T02:09:05.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.056+0000 7fc4e77fe700 1 -- 172.21.15.19:0/1744398285 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fc4d80e24a0 con 0x7fc4f0107df0 2024-05-09T02:09:05.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.057+0000 7fc4eed9d700 1 --2- 172.21.15.19:0/1744398285 >> [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639] conn(0x7fc4dc0bc5a0 0x7fc4dc0bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:05.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.057+0000 7fc4ee59c700 1 --2- 172.21.15.19:0/1744398285 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc4dc0b2910 0x7fc4dc0b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0005950 tx=0x7fc4e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:05.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.057+0000 7fc4eed9d700 1 --2- 172.21.15.19:0/1744398285 >> [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639] conn(0x7fc4dc0bc5a0 0x7fc4dc0bea90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:05.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.058+0000 7fc4e77fe700 1 -- 172.21.15.19:0/1744398285 <== osd.2 v2:172.21.15.196:6800/1659504639 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4d800fec0 con 0x7fc4dc0bc5a0 2024-05-09T02:09:05.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.075+0000 7fc4f4aec700 1 -- 172.21.15.19:0/1744398285 --> [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc4f0064350 con 0x7fc4dc0bc5a0 2024-05-09T02:09:05.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.075+0000 7fc4e77fe700 1 -- 172.21.15.19:0/1744398285 <== osd.2 v2:172.21.15.196:6800/1659504639 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc4f0064350 con 0x7fc4dc0bc5a0 2024-05-09T02:09:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.075+0000 7fc4f4aec700 1 -- 172.21.15.19:0/1744398285 >> [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639] conn(0x7fc4dc0bc5a0 msgr2=0x7fc4dc0bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.075+0000 7fc4f4aec700 1 --2- 172.21.15.19:0/1744398285 >> [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639] conn(0x7fc4dc0bc5a0 0x7fc4dc0bea90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.076+0000 7fc4f4aec700 1 -- 172.21.15.19:0/1744398285 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc4dc0b2910 msgr2=0x7fc4dc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:05.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.076+0000 7fc4f4aec700 1 --2- 172.21.15.19:0/1744398285 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc4dc0b2910 0x7fc4dc0b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0005950 tx=0x7fc4e000b410 comp rx=0 tx=0).stop 2024-05-09T02:09:05.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.076+0000 7fc4f4aec700 1 -- 172.21.15.19:0/1744398285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4f0107df0 msgr2=0x7fc4f019c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:05.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.076+0000 7fc4f4aec700 1 --2- 172.21.15.19:0/1744398285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4f0107df0 0x7fc4f019c1e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800b500 tx=0x7fc4d800b810 comp rx=0 tx=0).stop 2024-05-09T02:09:05.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.076+0000 7fc4eed9d700 1 -- 172.21.15.19:0/1744398285 reap_dead start 2024-05-09T02:09:05.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.076+0000 7fc4f4aec700 1 -- 172.21.15.19:0/1744398285 shutdown_connections 2024-05-09T02:09:05.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.076+0000 7fc4f4aec700 1 -- 172.21.15.19:0/1744398285 >> 172.21.15.19:0/1744398285 conn(0x7fc4f00fedf0 msgr2=0x7fc4f0107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:05.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.076+0000 7fc4f4aec700 1 -- 172.21.15.19:0/1744398285 shutdown_connections 2024-05-09T02:09:05.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:05.076+0000 7fc4f4aec700 1 -- 172.21.15.19:0/1744398285 wait complete. 2024-05-09T02:09:05.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:05 smithi019 ceph-mon[30151]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:09:05.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:05 smithi059 ceph-mon[34781]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:09:05.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:05 smithi196 ceph-mon[34718]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:09:07.149 INFO:teuthology.orchestra.run.smithi019.stdout:34359738456 2024-05-09T02:09:07.149 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-09T02:09:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:07 smithi059 ceph-mon[34781]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:09:07.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:07 smithi196 ceph-mon[34718]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:09:07.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:07 smithi019 ceph-mon[30151]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:09:08.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.887+0000 7ffad268b700 1 -- 172.21.15.19:0/2269460041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffacc109860 msgr2=0x7ffacc1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:08.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.887+0000 7ffad268b700 1 --2- 172.21.15.19:0/2269460041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffacc109860 0x7ffacc1111c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffac400b0d0 tx=0x7ffac400b3e0 comp rx=0 tx=0).stop 2024-05-09T02:09:08.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.887+0000 7ffad268b700 1 -- 172.21.15.19:0/2269460041 shutdown_connections 2024-05-09T02:09:08.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.887+0000 7ffad268b700 1 --2- 172.21.15.19:0/2269460041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffacc109860 0x7ffacc1111c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:08.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.887+0000 7ffad268b700 1 --2- 172.21.15.19:0/2269460041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffacc1045e0 0x7ffacc1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:08.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.887+0000 7ffad268b700 1 --2- 172.21.15.19:0/2269460041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffacc108f10 0x7ffacc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:08.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.887+0000 7ffad268b700 1 -- 172.21.15.19:0/2269460041 >> 172.21.15.19:0/2269460041 conn(0x7ffacc0ffef0 msgr2=0x7ffacc102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:08.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.887+0000 7ffad268b700 1 -- 172.21.15.19:0/2269460041 shutdown_connections 2024-05-09T02:09:08.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.887+0000 7ffad268b700 1 -- 172.21.15.19:0/2269460041 wait complete. 2024-05-09T02:09:08.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.888+0000 7ffad268b700 1 Processor -- start 2024-05-09T02:09:08.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.888+0000 7ffad268b700 1 -- start start 2024-05-09T02:09:08.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffad268b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffacc1045e0 0x7ffacc19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:08.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffad268b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffacc108f10 0x7ffacc19b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:08.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffad268b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffacc109860 0x7ffacc19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:08.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffad268b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc114580 con 0x7ffacc109860 2024-05-09T02:09:08.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffad268b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc114400 con 0x7ffacc1045e0 2024-05-09T02:09:08.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffad268b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc114700 con 0x7ffacc108f10 2024-05-09T02:09:08.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffad0c28700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffacc109860 0x7ffacc19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:08.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffad0c28700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffacc109860 0x7ffacc19f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35682/0 (socket says 172.21.15.19:35682) 2024-05-09T02:09:08.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffad0c28700 1 -- 172.21.15.19:0/3718384335 learned_addr learned my addr 172.21.15.19:0/3718384335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:08.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffacbfff700 1 --2- 172.21.15.19:0/3718384335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffacc1045e0 0x7ffacc19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:08.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffacb7fe700 1 --2- 172.21.15.19:0/3718384335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffacc108f10 0x7ffacc19b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:08.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffad0c28700 1 -- 172.21.15.19:0/3718384335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffacc108f10 msgr2=0x7ffacc19b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:08.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffad0c28700 1 --2- 172.21.15.19:0/3718384335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffacc108f10 0x7ffacc19b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:08.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffad0c28700 1 -- 172.21.15.19:0/3718384335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffacc1045e0 msgr2=0x7ffacc19af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:08.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffad0c28700 1 --2- 172.21.15.19:0/3718384335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffacc1045e0 0x7ffacc19af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:08.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffad0c28700 1 -- 172.21.15.19:0/3718384335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffacc19fe00 con 0x7ffacc109860 2024-05-09T02:09:08.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffacb7fe700 1 --2- 172.21.15.19:0/3718384335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffacc108f10 0x7ffacc19b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:09:08.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.889+0000 7ffad0c28700 1 --2- 172.21.15.19:0/3718384335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffacc109860 0x7ffacc19f760 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009fd0 tx=0x7ffac4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:08.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.890+0000 7ffac97fa700 1 -- 172.21.15.19:0/3718384335 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac401f070 con 0x7ffacc109860 2024-05-09T02:09:08.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.890+0000 7ffac97fa700 1 -- 172.21.15.19:0/3718384335 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac4003c00 con 0x7ffacc109860 2024-05-09T02:09:08.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.890+0000 7ffad268b700 1 -- 172.21.15.19:0/3718384335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffacc1a0090 con 0x7ffacc109860 2024-05-09T02:09:08.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.890+0000 7ffac97fa700 1 -- 172.21.15.19:0/3718384335 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac401b7b0 con 0x7ffacc109860 2024-05-09T02:09:08.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.890+0000 7ffad268b700 1 -- 172.21.15.19:0/3718384335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffacc1a7a10 con 0x7ffacc109860 2024-05-09T02:09:08.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.890+0000 7ffad268b700 1 -- 172.21.15.19:0/3718384335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffacc1a05c0 con 0x7ffacc109860 2024-05-09T02:09:08.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.893+0000 7ffac97fa700 1 -- 172.21.15.19:0/3718384335 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ffac4022070 con 0x7ffacc109860 2024-05-09T02:09:08.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.893+0000 7ffac97fa700 1 --2- 172.21.15.19:0/3718384335 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffab40b2990 0x7ffab40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:08.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.893+0000 7ffacbfff700 1 --2- 172.21.15.19:0/3718384335 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffab40b2990 0x7ffab40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:08.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.894+0000 7ffac97fa700 1 -- 172.21.15.19:0/3718384335 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7ffac400f020 con 0x7ffacc109860 2024-05-09T02:09:08.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.894+0000 7ffac97fa700 1 --2- 172.21.15.19:0/3718384335 >> [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259] conn(0x7ffab40bc620 0x7ffab40beb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:08.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.894+0000 7ffac97fa700 1 -- 172.21.15.19:0/3718384335 --> [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffab40bf1e0 con 0x7ffab40bc620 2024-05-09T02:09:08.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.894+0000 7ffac97fa700 1 -- 172.21.15.19:0/3718384335 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7ffac40e3d70 con 0x7ffacc109860 2024-05-09T02:09:08.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.894+0000 7ffacbfff700 1 --2- 172.21.15.19:0/3718384335 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffab40b2990 0x7ffab40b4ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009a10 tx=0x7ffabc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:08.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.894+0000 7ffacb7fe700 1 --2- 172.21.15.19:0/3718384335 >> [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259] conn(0x7ffab40bc620 0x7ffab40beb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:08.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.894+0000 7ffacb7fe700 1 --2- 172.21.15.19:0/3718384335 >> [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259] conn(0x7ffab40bc620 0x7ffab40beb10 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:08.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.895+0000 7ffac97fa700 1 -- 172.21.15.19:0/3718384335 <== osd.8 v2:172.21.15.196:6816/2946064259 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffab40bf1e0 con 0x7ffab40bc620 2024-05-09T02:09:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.912+0000 7ffad268b700 1 -- 172.21.15.19:0/3718384335 --> [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffacc064190 con 0x7ffab40bc620 2024-05-09T02:09:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.913+0000 7ffac97fa700 1 -- 172.21.15.19:0/3718384335 <== osd.8 v2:172.21.15.196:6816/2946064259 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffacc064190 con 0x7ffab40bc620 2024-05-09T02:09:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.913+0000 7ffad268b700 1 -- 172.21.15.19:0/3718384335 >> [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259] conn(0x7ffab40bc620 msgr2=0x7ffab40beb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.913+0000 7ffad268b700 1 --2- 172.21.15.19:0/3718384335 >> [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259] conn(0x7ffab40bc620 0x7ffab40beb10 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.913+0000 7ffad268b700 1 -- 172.21.15.19:0/3718384335 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffab40b2990 msgr2=0x7ffab40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.913+0000 7ffad268b700 1 --2- 172.21.15.19:0/3718384335 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffab40b2990 0x7ffab40b4ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009a10 tx=0x7ffabc009190 comp rx=0 tx=0).stop 2024-05-09T02:09:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.913+0000 7ffad268b700 1 -- 172.21.15.19:0/3718384335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffacc109860 msgr2=0x7ffacc19f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.913+0000 7ffad268b700 1 --2- 172.21.15.19:0/3718384335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffacc109860 0x7ffacc19f760 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009fd0 tx=0x7ffac4007a20 comp rx=0 tx=0).stop 2024-05-09T02:09:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.913+0000 7ffad0c28700 1 -- 172.21.15.19:0/3718384335 reap_dead start 2024-05-09T02:09:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.913+0000 7ffad268b700 1 -- 172.21.15.19:0/3718384335 shutdown_connections 2024-05-09T02:09:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.913+0000 7ffad268b700 1 -- 172.21.15.19:0/3718384335 >> 172.21.15.19:0/3718384335 conn(0x7ffacc0ffef0 msgr2=0x7ffacc1086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.913+0000 7ffad268b700 1 -- 172.21.15.19:0/3718384335 shutdown_connections 2024-05-09T02:09:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:08.913+0000 7ffad268b700 1 -- 172.21.15.19:0/3718384335 wait complete. 2024-05-09T02:09:09.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.138+0000 7f8eb2692700 1 -- 172.21.15.19:0/2408042477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eac108f00 msgr2=0x7f8eac109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.138+0000 7f8eb2692700 1 --2- 172.21.15.19:0/2408042477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eac108f00 0x7f8eac109310 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8e940099c0 tx=0x7f8e94009cd0 comp rx=0 tx=0).stop 2024-05-09T02:09:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.138+0000 7f8eb2692700 1 -- 172.21.15.19:0/2408042477 shutdown_connections 2024-05-09T02:09:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.138+0000 7f8eb2692700 1 --2- 172.21.15.19:0/2408042477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eac109850 0x7f8eac1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.138+0000 7f8eb2692700 1 --2- 172.21.15.19:0/2408042477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8eac1045d0 0x7f8eac1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.138+0000 7f8eb2692700 1 --2- 172.21.15.19:0/2408042477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eac108f00 0x7f8eac109310 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.138+0000 7f8eb2692700 1 -- 172.21.15.19:0/2408042477 >> 172.21.15.19:0/2408042477 conn(0x7f8eac0ffea0 msgr2=0x7f8eac1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.138+0000 7f8eb2692700 1 -- 172.21.15.19:0/2408042477 shutdown_connections 2024-05-09T02:09:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.138+0000 7f8eb2692700 1 -- 172.21.15.19:0/2408042477 wait complete. 2024-05-09T02:09:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.139+0000 7f8eb2692700 1 Processor -- start 2024-05-09T02:09:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.139+0000 7f8eb2692700 1 -- start start 2024-05-09T02:09:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.139+0000 7f8eb2692700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eac1045d0 0x7f8eac19aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.139+0000 7f8eb2692700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8eac108f00 0x7f8eac19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:09.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.139+0000 7f8eb2692700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eac109850 0x7f8eac19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:09.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.139+0000 7f8eb2692700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac1144b0 con 0x7f8eac109850 2024-05-09T02:09:09.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.139+0000 7f8eb2692700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac114330 con 0x7f8eac108f00 2024-05-09T02:09:09.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.139+0000 7f8eb2692700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac114630 con 0x7f8eac1045d0 2024-05-09T02:09:09.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.139+0000 7f8eb0c2f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eac109850 0x7f8eac19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:09.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.139+0000 7f8eb0c2f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eac109850 0x7f8eac19f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35698/0 (socket says 172.21.15.19:35698) 2024-05-09T02:09:09.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.139+0000 7f8eb0c2f700 1 -- 172.21.15.19:0/2537681967 learned_addr learned my addr 172.21.15.19:0/2537681967 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:09.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.139+0000 7f8eabfff700 1 --2- 172.21.15.19:0/2537681967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8eac108f00 0x7f8eac19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:09.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.139+0000 7f8ea3fff700 1 --2- 172.21.15.19:0/2537681967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eac1045d0 0x7f8eac19aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:09.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.140+0000 7f8eb0c2f700 1 -- 172.21.15.19:0/2537681967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eac1045d0 msgr2=0x7f8eac19aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:09.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.140+0000 7f8eb0c2f700 1 --2- 172.21.15.19:0/2537681967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eac1045d0 0x7f8eac19aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:09.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.140+0000 7f8eb0c2f700 1 -- 172.21.15.19:0/2537681967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8eac108f00 msgr2=0x7f8eac19b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:09.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.140+0000 7f8eb0c2f700 1 --2- 172.21.15.19:0/2537681967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8eac108f00 0x7f8eac19b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:09.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.140+0000 7f8eb0c2f700 1 -- 172.21.15.19:0/2537681967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eac19fdb0 con 0x7f8eac109850 2024-05-09T02:09:09.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.140+0000 7f8eb0c2f700 1 --2- 172.21.15.19:0/2537681967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eac109850 0x7f8eac19f710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4007ee0 tx=0x7f8ea400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:09.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.140+0000 7f8ea9ffb700 1 -- 172.21.15.19:0/2537681967 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4004bb0 con 0x7f8eac109850 2024-05-09T02:09:09.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.140+0000 7f8ea9ffb700 1 -- 172.21.15.19:0/2537681967 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ea4004d50 con 0x7f8eac109850 2024-05-09T02:09:09.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.140+0000 7f8eb2692700 1 -- 172.21.15.19:0/2537681967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eac1a00a0 con 0x7f8eac109850 2024-05-09T02:09:09.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.140+0000 7f8ea9ffb700 1 -- 172.21.15.19:0/2537681967 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4005920 con 0x7f8eac109850 2024-05-09T02:09:09.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.140+0000 7f8eb2692700 1 -- 172.21.15.19:0/2537681967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eac1a79c0 con 0x7f8eac109850 2024-05-09T02:09:09.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.141+0000 7f8eb2692700 1 -- 172.21.15.19:0/2537681967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8eac063e90 con 0x7f8eac109850 2024-05-09T02:09:09.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.143+0000 7f8ea9ffb700 1 -- 172.21.15.19:0/2537681967 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8ea401a020 con 0x7f8eac109850 2024-05-09T02:09:09.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.144+0000 7f8ea9ffb700 1 --2- 172.21.15.19:0/2537681967 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f8e980b2910 0x7f8e980b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.144+0000 7f8ea3fff700 1 --2- 172.21.15.19:0/2537681967 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f8e980b2910 0x7f8e980b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.144+0000 7f8ea9ffb700 1 -- 172.21.15.19:0/2537681967 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f8ea40e23f0 con 0x7f8eac109850 2024-05-09T02:09:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.144+0000 7f8ea9ffb700 1 --2- 172.21.15.19:0/2537681967 >> [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637] conn(0x7f8e980bc5a0 0x7f8e980bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.144+0000 7f8ea9ffb700 1 -- 172.21.15.19:0/2537681967 --> [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e980bf160 con 0x7f8e980bc5a0 2024-05-09T02:09:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.144+0000 7f8ea9ffb700 1 -- 172.21.15.19:0/2537681967 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f8ea40e2840 con 0x7f8eac109850 2024-05-09T02:09:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.144+0000 7f8eabfff700 1 --2- 172.21.15.19:0/2537681967 >> [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637] conn(0x7f8e980bc5a0 0x7f8e980bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.144+0000 7f8ea3fff700 1 --2- 172.21.15.19:0/2537681967 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f8e980b2910 0x7f8e980b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c005950 tx=0x7f8e9c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.145+0000 7f8eabfff700 1 --2- 172.21.15.19:0/2537681967 >> [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637] conn(0x7f8e980bc5a0 0x7f8e980bea90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:09.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.145+0000 7f8ea9ffb700 1 -- 172.21.15.19:0/2537681967 <== osd.6 v2:172.21.15.19:6818/2150451637 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e980bf160 con 0x7f8e980bc5a0 2024-05-09T02:09:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.162+0000 7f8eb2692700 1 -- 172.21.15.19:0/2537681967 --> [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8eac064290 con 0x7f8e980bc5a0 2024-05-09T02:09:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.162+0000 7f8ea9ffb700 1 -- 172.21.15.19:0/2537681967 <== osd.6 v2:172.21.15.19:6818/2150451637 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8eac064290 con 0x7f8e980bc5a0 2024-05-09T02:09:09.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.162+0000 7f8eb2692700 1 -- 172.21.15.19:0/2537681967 >> [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637] conn(0x7f8e980bc5a0 msgr2=0x7f8e980bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:09.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.162+0000 7f8eb2692700 1 --2- 172.21.15.19:0/2537681967 >> [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637] conn(0x7f8e980bc5a0 0x7f8e980bea90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:09.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.162+0000 7f8eb2692700 1 -- 172.21.15.19:0/2537681967 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f8e980b2910 msgr2=0x7f8e980b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.162+0000 7f8eb2692700 1 --2- 172.21.15.19:0/2537681967 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f8e980b2910 0x7f8e980b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c005950 tx=0x7f8e9c00a300 comp rx=0 tx=0).stop 2024-05-09T02:09:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.162+0000 7f8eb2692700 1 -- 172.21.15.19:0/2537681967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eac109850 msgr2=0x7f8eac19f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.162+0000 7f8eb2692700 1 --2- 172.21.15.19:0/2537681967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eac109850 0x7f8eac19f710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4007ee0 tx=0x7f8ea400d3b0 comp rx=0 tx=0).stop 2024-05-09T02:09:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.162+0000 7f8eb0c2f700 1 -- 172.21.15.19:0/2537681967 reap_dead start 2024-05-09T02:09:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.162+0000 7f8eb2692700 1 -- 172.21.15.19:0/2537681967 shutdown_connections 2024-05-09T02:09:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.163+0000 7f8eb2692700 1 -- 172.21.15.19:0/2537681967 >> 172.21.15.19:0/2537681967 conn(0x7f8eac0ffea0 msgr2=0x7f8eac1086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.163+0000 7f8eb2692700 1 -- 172.21.15.19:0/2537681967 shutdown_connections 2024-05-09T02:09:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:09.163+0000 7f8eb2692700 1 -- 172.21.15.19:0/2537681967 wait complete. 2024-05-09T02:09:09.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:09 smithi019 ceph-mon[30151]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:09.689 INFO:teuthology.orchestra.run.smithi019.stdout:77309411406 2024-05-09T02:09:09.690 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-09T02:09:09.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:09 smithi059 ceph-mon[34781]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:09.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:09 smithi196 ceph-mon[34718]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:10.689 INFO:teuthology.orchestra.run.smithi019.stdout:231928234031 2024-05-09T02:09:10.689 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-09T02:09:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.571+0000 7fb043961700 1 -- 172.21.15.19:0/2739119495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c1034c0 msgr2=0x7fb03c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:11.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.571+0000 7fb043961700 1 --2- 172.21.15.19:0/2739119495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c1034c0 0x7fb03c1038d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb0380099c0 tx=0x7fb038009cd0 comp rx=0 tx=0).stop 2024-05-09T02:09:11.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.571+0000 7fb043961700 1 -- 172.21.15.19:0/2739119495 shutdown_connections 2024-05-09T02:09:11.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.571+0000 7fb043961700 1 --2- 172.21.15.19:0/2739119495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb03c108740 0x7fb03c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:11.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.571+0000 7fb043961700 1 --2- 172.21.15.19:0/2739119495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c1034c0 0x7fb03c1038d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.571+0000 7fb043961700 1 --2- 172.21.15.19:0/2739119495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb03c107df0 0x7fb03c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.571+0000 7fb043961700 1 -- 172.21.15.19:0/2739119495 >> 172.21.15.19:0/2739119495 conn(0x7fb03c0fedc0 msgr2=0x7fb03c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.571+0000 7fb043961700 1 -- 172.21.15.19:0/2739119495 shutdown_connections 2024-05-09T02:09:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.571+0000 7fb043961700 1 -- 172.21.15.19:0/2739119495 wait complete. 2024-05-09T02:09:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb043961700 1 Processor -- start 2024-05-09T02:09:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb043961700 1 -- start start 2024-05-09T02:09:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb043961700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c1034c0 0x7fb03c19bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb043961700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb03c107df0 0x7fb03c19c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb043961700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb03c108740 0x7fb03c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb043961700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c113370 con 0x7fb03c1034c0 2024-05-09T02:09:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb043961700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c1131f0 con 0x7fb03c107df0 2024-05-09T02:09:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb043961700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c1134f0 con 0x7fb03c108740 2024-05-09T02:09:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb0416fd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c1034c0 0x7fb03c19bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb0416fd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c1034c0 0x7fb03c19bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35668/0 (socket says 172.21.15.19:35668) 2024-05-09T02:09:11.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb0416fd700 1 -- 172.21.15.19:0/2755985477 learned_addr learned my addr 172.21.15.19:0/2755985477 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:11.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb041efe700 1 --2- 172.21.15.19:0/2755985477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb03c108740 0x7fb03c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:11.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb0416fd700 1 -- 172.21.15.19:0/2755985477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb03c108740 msgr2=0x7fb03c19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:11.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb0416fd700 1 --2- 172.21.15.19:0/2755985477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb03c108740 0x7fb03c19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb0416fd700 1 -- 172.21.15.19:0/2755985477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb03c107df0 msgr2=0x7fb03c19c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb040efc700 1 --2- 172.21.15.19:0/2755985477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb03c107df0 0x7fb03c19c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb0416fd700 1 --2- 172.21.15.19:0/2755985477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb03c107df0 0x7fb03c19c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.572+0000 7fb0416fd700 1 -- 172.21.15.19:0/2755985477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03c1a6f80 con 0x7fb03c1034c0 2024-05-09T02:09:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.573+0000 7fb0416fd700 1 --2- 172.21.15.19:0/2755985477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c1034c0 0x7fb03c19bc10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb038006010 tx=0x7fb0380045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.573+0000 7fb0327fc700 1 -- 172.21.15.19:0/2755985477 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038026070 con 0x7fb03c1034c0 2024-05-09T02:09:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.573+0000 7fb0327fc700 1 -- 172.21.15.19:0/2755985477 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03800f460 con 0x7fb03c1034c0 2024-05-09T02:09:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.573+0000 7fb043961700 1 -- 172.21.15.19:0/2755985477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03c1a7210 con 0x7fb03c1034c0 2024-05-09T02:09:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.573+0000 7fb0327fc700 1 -- 172.21.15.19:0/2755985477 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038009e60 con 0x7fb03c1034c0 2024-05-09T02:09:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.573+0000 7fb043961700 1 -- 172.21.15.19:0/2755985477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03c1a7660 con 0x7fb03c1034c0 2024-05-09T02:09:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.574+0000 7fb043961700 1 -- 172.21.15.19:0/2755985477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb03c063ed0 con 0x7fb03c1034c0 2024-05-09T02:09:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.576+0000 7fb0327fc700 1 -- 172.21.15.19:0/2755985477 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb038029070 con 0x7fb03c1034c0 2024-05-09T02:09:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.576+0000 7fb0327fc700 1 --2- 172.21.15.19:0/2755985477 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb0280b2910 0x7fb0280b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.576+0000 7fb040efc700 1 --2- 172.21.15.19:0/2755985477 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb0280b2910 0x7fb0280b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.577+0000 7fb0327fc700 1 -- 172.21.15.19:0/2755985477 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fb0380e3110 con 0x7fb03c1034c0 2024-05-09T02:09:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.577+0000 7fb0327fc700 1 --2- 172.21.15.19:0/2755985477 >> [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599] conn(0x7fb0280bc5a0 0x7fb0280bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.577+0000 7fb0327fc700 1 -- 172.21.15.19:0/2755985477 --> [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0280bf160 con 0x7fb0280bc5a0 2024-05-09T02:09:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.577+0000 7fb0327fc700 1 -- 172.21.15.19:0/2755985477 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fb0380e3560 con 0x7fb03c1034c0 2024-05-09T02:09:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.577+0000 7fb041efe700 1 --2- 172.21.15.19:0/2755985477 >> [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599] conn(0x7fb0280bc5a0 0x7fb0280bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.577+0000 7fb040efc700 1 --2- 172.21.15.19:0/2755985477 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb0280b2910 0x7fb0280b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb03c106550 tx=0x7fb02c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.577+0000 7fb041efe700 1 --2- 172.21.15.19:0/2755985477 >> [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599] conn(0x7fb0280bc5a0 0x7fb0280bea90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.578+0000 7fb0327fc700 1 -- 172.21.15.19:0/2755985477 <== osd.3 v2:172.21.15.19:6810/2707671599 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0280bf160 con 0x7fb0280bc5a0 2024-05-09T02:09:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.594+0000 7fb043961700 1 -- 172.21.15.19:0/2755985477 --> [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb03c064350 con 0x7fb0280bc5a0 2024-05-09T02:09:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.595+0000 7fb0327fc700 1 -- 172.21.15.19:0/2755985477 <== osd.3 v2:172.21.15.19:6810/2707671599 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb03c064350 con 0x7fb0280bc5a0 2024-05-09T02:09:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.595+0000 7fb043961700 1 -- 172.21.15.19:0/2755985477 >> [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599] conn(0x7fb0280bc5a0 msgr2=0x7fb0280bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.595+0000 7fb043961700 1 --2- 172.21.15.19:0/2755985477 >> [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599] conn(0x7fb0280bc5a0 0x7fb0280bea90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.595+0000 7fb043961700 1 -- 172.21.15.19:0/2755985477 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb0280b2910 msgr2=0x7fb0280b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.595+0000 7fb043961700 1 --2- 172.21.15.19:0/2755985477 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb0280b2910 0x7fb0280b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb03c106550 tx=0x7fb02c009260 comp rx=0 tx=0).stop 2024-05-09T02:09:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.595+0000 7fb043961700 1 -- 172.21.15.19:0/2755985477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c1034c0 msgr2=0x7fb03c19bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.595+0000 7fb043961700 1 --2- 172.21.15.19:0/2755985477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c1034c0 0x7fb03c19bc10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb038006010 tx=0x7fb0380045e0 comp rx=0 tx=0).stop 2024-05-09T02:09:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.595+0000 7fb041efe700 1 -- 172.21.15.19:0/2755985477 reap_dead start 2024-05-09T02:09:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.595+0000 7fb043961700 1 -- 172.21.15.19:0/2755985477 shutdown_connections 2024-05-09T02:09:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.595+0000 7fb043961700 1 -- 172.21.15.19:0/2755985477 >> 172.21.15.19:0/2755985477 conn(0x7fb03c0fedc0 msgr2=0x7fb03c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.595+0000 7fb043961700 1 -- 172.21.15.19:0/2755985477 shutdown_connections 2024-05-09T02:09:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:07.595+0000 7fb043961700 1 -- 172.21.15.19:0/2755985477 wait complete. 2024-05-09T02:09:11.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:11 smithi019 ceph-mon[30151]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:09:11.629 INFO:teuthology.orchestra.run.smithi019.stdout:111669149770 2024-05-09T02:09:11.629 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-09T02:09:11.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:11 smithi059 ceph-mon[34781]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:09:11.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:11 smithi196 ceph-mon[34718]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:09:11.997 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.344+0000 7fa6e6535700 1 -- 172.21.15.19:0/2066602071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6e0110b80 msgr2=0x7fa6e0112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.344+0000 7fa6e6535700 1 --2- 172.21.15.19:0/2066602071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6e0110b80 0x7fa6e0112fe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800b0a0 tx=0x7fa6d800b3b0 comp rx=0 tx=0).stop 2024-05-09T02:09:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.344+0000 7fa6e6535700 1 -- 172.21.15.19:0/2066602071 shutdown_connections 2024-05-09T02:09:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.344+0000 7fa6e6535700 1 --2- 172.21.15.19:0/2066602071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6e0110b80 0x7fa6e0112fe0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.344+0000 7fa6e6535700 1 --2- 172.21.15.19:0/2066602071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e0076e40 0x7fa6e0077250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.344+0000 7fa6e6535700 1 --2- 172.21.15.19:0/2066602071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e0077790 0x7fa6e0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.344+0000 7fa6e6535700 1 -- 172.21.15.19:0/2066602071 >> 172.21.15.19:0/2066602071 conn(0x7fa6e00ffea0 msgr2=0x7fa6e01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:13.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.344+0000 7fa6e6535700 1 -- 172.21.15.19:0/2066602071 shutdown_connections 2024-05-09T02:09:13.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.344+0000 7fa6e6535700 1 -- 172.21.15.19:0/2066602071 wait complete. 2024-05-09T02:09:13.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.345+0000 7fa6e6535700 1 Processor -- start 2024-05-09T02:09:13.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.345+0000 7fa6e6535700 1 -- start start 2024-05-09T02:09:13.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.345+0000 7fa6e6535700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6e0076e40 0x7fa6e019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:13.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.345+0000 7fa6e6535700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e0077790 0x7fa6e019d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:13.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.345+0000 7fa6e6535700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e0110b80 0x7fa6e01a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:13.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.345+0000 7fa6e6535700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e0116660 con 0x7fa6e0076e40 2024-05-09T02:09:13.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.345+0000 7fa6e6535700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e01164e0 con 0x7fa6e0077790 2024-05-09T02:09:13.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.345+0000 7fa6e6535700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e01167e0 con 0x7fa6e0110b80 2024-05-09T02:09:13.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6e0076e40 0x7fa6e019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:13.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6e0076e40 0x7fa6e019cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35716/0 (socket says 172.21.15.19:35716) 2024-05-09T02:09:13.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6dffff700 1 -- 172.21.15.19:0/1004092343 learned_addr learned my addr 172.21.15.19:0/1004092343 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:13.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6d7fff700 1 --2- 172.21.15.19:0/1004092343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e0077790 0x7fa6e019d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:13.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6e4ad2700 1 --2- 172.21.15.19:0/1004092343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e0110b80 0x7fa6e01a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:13.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6dffff700 1 -- 172.21.15.19:0/1004092343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e0110b80 msgr2=0x7fa6e01a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:13.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6dffff700 1 --2- 172.21.15.19:0/1004092343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e0110b80 0x7fa6e01a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:13.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6dffff700 1 -- 172.21.15.19:0/1004092343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e0077790 msgr2=0x7fa6e019d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:13.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6dffff700 1 --2- 172.21.15.19:0/1004092343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e0077790 0x7fa6e019d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:13.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6dffff700 1 -- 172.21.15.19:0/1004092343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e01a1e70 con 0x7fa6e0076e40 2024-05-09T02:09:13.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6e4ad2700 1 --2- 172.21.15.19:0/1004092343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e0110b80 0x7fa6e01a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:09:13.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6dffff700 1 --2- 172.21.15.19:0/1004092343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6e0076e40 0x7fa6e019cff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000d950 tx=0x7fa6d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:13.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6ddffb700 1 -- 172.21.15.19:0/1004092343 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0016070 con 0x7fa6e0076e40 2024-05-09T02:09:13.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6e6535700 1 -- 172.21.15.19:0/1004092343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e01a2160 con 0x7fa6e0076e40 2024-05-09T02:09:13.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6ddffb700 1 -- 172.21.15.19:0/1004092343 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6d0009df0 con 0x7fa6e0076e40 2024-05-09T02:09:13.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6ddffb700 1 -- 172.21.15.19:0/1004092343 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d000f3c0 con 0x7fa6e0076e40 2024-05-09T02:09:13.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.346+0000 7fa6e6535700 1 -- 172.21.15.19:0/1004092343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e01a9a80 con 0x7fa6e0076e40 2024-05-09T02:09:13.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.347+0000 7fa6e6535700 1 -- 172.21.15.19:0/1004092343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa6e0063f50 con 0x7fa6e0076e40 2024-05-09T02:09:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.349+0000 7fa6ddffb700 1 -- 172.21.15.19:0/1004092343 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa6d0019070 con 0x7fa6e0076e40 2024-05-09T02:09:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.350+0000 7fa6ddffb700 1 --2- 172.21.15.19:0/1004092343 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa6cc0b2910 0x7fa6cc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.350+0000 7fa6d7fff700 1 --2- 172.21.15.19:0/1004092343 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa6cc0b2910 0x7fa6cc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.350+0000 7fa6ddffb700 1 -- 172.21.15.19:0/1004092343 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fa6d00e2570 con 0x7fa6e0076e40 2024-05-09T02:09:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.350+0000 7fa6ddffb700 1 --2- 172.21.15.19:0/1004092343 >> [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330] conn(0x7fa6cc0bc5a0 0x7fa6cc0bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.350+0000 7fa6ddffb700 1 -- 172.21.15.19:0/1004092343 --> [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa6cc0bf160 con 0x7fa6cc0bc5a0 2024-05-09T02:09:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.350+0000 7fa6ddffb700 1 -- 172.21.15.19:0/1004092343 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fa6d00e29c0 con 0x7fa6e0076e40 2024-05-09T02:09:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.350+0000 7fa6d7fff700 1 --2- 172.21.15.19:0/1004092343 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa6cc0b2910 0x7fa6cc0b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0103350 tx=0x7fa6c800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:13.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.350+0000 7fa6e4ad2700 1 --2- 172.21.15.19:0/1004092343 >> [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330] conn(0x7fa6cc0bc5a0 0x7fa6cc0bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:13.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.351+0000 7fa6e4ad2700 1 --2- 172.21.15.19:0/1004092343 >> [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330] conn(0x7fa6cc0bc5a0 0x7fa6cc0bea90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:13.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.352+0000 7fa6ddffb700 1 -- 172.21.15.19:0/1004092343 <== osd.4 v2:172.21.15.59:6808/223037330 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa6cc0bf160 con 0x7fa6cc0bc5a0 2024-05-09T02:09:13.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.369+0000 7fa6e6535700 1 -- 172.21.15.19:0/1004092343 --> [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa6e0064350 con 0x7fa6cc0bc5a0 2024-05-09T02:09:13.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.369+0000 7fa6ddffb700 1 -- 172.21.15.19:0/1004092343 <== osd.4 v2:172.21.15.59:6808/223037330 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa6e0064350 con 0x7fa6cc0bc5a0 2024-05-09T02:09:13.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.370+0000 7fa6e6535700 1 -- 172.21.15.19:0/1004092343 >> [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330] conn(0x7fa6cc0bc5a0 msgr2=0x7fa6cc0bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:13.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.370+0000 7fa6e6535700 1 --2- 172.21.15.19:0/1004092343 >> [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330] conn(0x7fa6cc0bc5a0 0x7fa6cc0bea90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.370+0000 7fa6e6535700 1 -- 172.21.15.19:0/1004092343 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa6cc0b2910 msgr2=0x7fa6cc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.370+0000 7fa6e6535700 1 --2- 172.21.15.19:0/1004092343 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa6cc0b2910 0x7fa6cc0b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0103350 tx=0x7fa6c800b410 comp rx=0 tx=0).stop 2024-05-09T02:09:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.370+0000 7fa6e6535700 1 -- 172.21.15.19:0/1004092343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6e0076e40 msgr2=0x7fa6e019cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.370+0000 7fa6e6535700 1 --2- 172.21.15.19:0/1004092343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6e0076e40 0x7fa6e019cff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000d950 tx=0x7fa6d000dc60 comp rx=0 tx=0).stop 2024-05-09T02:09:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.370+0000 7fa6e4ad2700 1 -- 172.21.15.19:0/1004092343 reap_dead start 2024-05-09T02:09:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.370+0000 7fa6e6535700 1 -- 172.21.15.19:0/1004092343 shutdown_connections 2024-05-09T02:09:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.370+0000 7fa6e6535700 1 -- 172.21.15.19:0/1004092343 >> 172.21.15.19:0/1004092343 conn(0x7fa6e00ffea0 msgr2=0x7fa6e0100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.370+0000 7fa6e6535700 1 -- 172.21.15.19:0/1004092343 shutdown_connections 2024-05-09T02:09:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:13.370+0000 7fa6e6535700 1 -- 172.21.15.19:0/1004092343 wait complete. 2024-05-09T02:09:13.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:13 smithi019 ceph-mon[30151]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:13.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:13 smithi059 ceph-mon[34781]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:13.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:13 smithi196 ceph-mon[34718]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:14.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.207+0000 7fc54ec67700 1 -- 172.21.15.19:0/11193157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5481045f0 msgr2=0x7fc548104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:14.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.207+0000 7fc54ec67700 1 --2- 172.21.15.19:0/11193157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5481045f0 0x7fc548104a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc5440099c0 tx=0x7fc544009cd0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.207+0000 7fc54ec67700 1 -- 172.21.15.19:0/11193157 shutdown_connections 2024-05-09T02:09:14.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.207+0000 7fc54ec67700 1 --2- 172.21.15.19:0/11193157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc548109870 0x7fc5481111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.207+0000 7fc54ec67700 1 --2- 172.21.15.19:0/11193157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5481045f0 0x7fc548104a00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.207+0000 7fc54ec67700 1 --2- 172.21.15.19:0/11193157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc548108f20 0x7fc548109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.207+0000 7fc54ec67700 1 -- 172.21.15.19:0/11193157 >> 172.21.15.19:0/11193157 conn(0x7fc5480fffe0 msgr2=0x7fc548102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:14.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.207+0000 7fc54ec67700 1 -- 172.21.15.19:0/11193157 shutdown_connections 2024-05-09T02:09:14.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.207+0000 7fc54ec67700 1 -- 172.21.15.19:0/11193157 wait complete. 2024-05-09T02:09:14.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54ec67700 1 Processor -- start 2024-05-09T02:09:14.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54ec67700 1 -- start start 2024-05-09T02:09:14.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54ec67700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5481045f0 0x7fc54819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:14.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54ec67700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc548108f20 0x7fc54819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:14.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54ec67700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548109870 0x7fc54819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:14.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54ec67700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481144f0 con 0x7fc548109870 2024-05-09T02:09:14.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54ec67700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc548114370 con 0x7fc548108f20 2024-05-09T02:09:14.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54ec67700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc548114670 con 0x7fc5481045f0 2024-05-09T02:09:14.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54d204700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548109870 0x7fc54819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:14.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54d204700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548109870 0x7fc54819f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35730/0 (socket says 172.21.15.19:35730) 2024-05-09T02:09:14.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54d204700 1 -- 172.21.15.19:0/3103782545 learned_addr learned my addr 172.21.15.19:0/3103782545 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:14.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54ca03700 1 --2- 172.21.15.19:0/3103782545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5481045f0 0x7fc54819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:14.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc53ffff700 1 --2- 172.21.15.19:0/3103782545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc548108f20 0x7fc54819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:14.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54d204700 1 -- 172.21.15.19:0/3103782545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5481045f0 msgr2=0x7fc54819aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:14.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54d204700 1 --2- 172.21.15.19:0/3103782545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5481045f0 0x7fc54819aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54d204700 1 -- 172.21.15.19:0/3103782545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc548108f20 msgr2=0x7fc54819b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:14.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54d204700 1 --2- 172.21.15.19:0/3103782545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc548108f20 0x7fc54819b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.208+0000 7fc54d204700 1 -- 172.21.15.19:0/3103782545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54819fd70 con 0x7fc548109870 2024-05-09T02:09:14.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.209+0000 7fc54d204700 1 --2- 172.21.15.19:0/3103782545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548109870 0x7fc54819f6d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc54000ef30 tx=0x7fc54000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:14.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.209+0000 7fc53dffb700 1 -- 172.21.15.19:0/3103782545 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5400092e0 con 0x7fc548109870 2024-05-09T02:09:14.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.209+0000 7fc53dffb700 1 -- 172.21.15.19:0/3103782545 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc540004bb0 con 0x7fc548109870 2024-05-09T02:09:14.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.209+0000 7fc54ec67700 1 -- 172.21.15.19:0/3103782545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5481a0000 con 0x7fc548109870 2024-05-09T02:09:14.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.209+0000 7fc53dffb700 1 -- 172.21.15.19:0/3103782545 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540004020 con 0x7fc548109870 2024-05-09T02:09:14.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.209+0000 7fc54ec67700 1 -- 172.21.15.19:0/3103782545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5481a7980 con 0x7fc548109870 2024-05-09T02:09:14.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.210+0000 7fc54ec67700 1 -- 172.21.15.19:0/3103782545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc548063f50 con 0x7fc548109870 2024-05-09T02:09:14.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.212+0000 7fc53dffb700 1 -- 172.21.15.19:0/3103782545 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc54001b030 con 0x7fc548109870 2024-05-09T02:09:14.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.213+0000 7fc53dffb700 1 --2- 172.21.15.19:0/3103782545 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5300b2990 0x7fc5300b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:14.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.213+0000 7fc54ca03700 1 --2- 172.21.15.19:0/3103782545 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5300b2990 0x7fc5300b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:14.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.213+0000 7fc53dffb700 1 -- 172.21.15.19:0/3103782545 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc5400e2770 con 0x7fc548109870 2024-05-09T02:09:14.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.213+0000 7fc53dffb700 1 --2- 172.21.15.19:0/3103782545 >> [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898] conn(0x7fc5300bc620 0x7fc5300beb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:14.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.213+0000 7fc53dffb700 1 -- 172.21.15.19:0/3103782545 --> [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc5300bf1e0 con 0x7fc5300bc620 2024-05-09T02:09:14.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.213+0000 7fc53dffb700 1 -- 172.21.15.19:0/3103782545 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fc5400e2b20 con 0x7fc548109870 2024-05-09T02:09:14.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.213+0000 7fc54ca03700 1 --2- 172.21.15.19:0/3103782545 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5300b2990 0x7fc5300b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc544000c00 tx=0x7fc54401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:14.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.213+0000 7fc53ffff700 1 --2- 172.21.15.19:0/3103782545 >> [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898] conn(0x7fc5300bc620 0x7fc5300beb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:14.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.214+0000 7fc53ffff700 1 --2- 172.21.15.19:0/3103782545 >> [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898] conn(0x7fc5300bc620 0x7fc5300beb10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:14.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.216+0000 7fc53dffb700 1 -- 172.21.15.19:0/3103782545 <== osd.5 v2:172.21.15.196:6808/2516161898 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc5300bf1e0 con 0x7fc5300bc620 2024-05-09T02:09:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.232+0000 7fc54ec67700 1 -- 172.21.15.19:0/3103782545 --> [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc5480643d0 con 0x7fc5300bc620 2024-05-09T02:09:14.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.233+0000 7fc53dffb700 1 -- 172.21.15.19:0/3103782545 <== osd.5 v2:172.21.15.196:6808/2516161898 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc5480643d0 con 0x7fc5300bc620 2024-05-09T02:09:14.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.233+0000 7fc54ec67700 1 -- 172.21.15.19:0/3103782545 >> [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898] conn(0x7fc5300bc620 msgr2=0x7fc5300beb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:14.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.233+0000 7fc54ec67700 1 --2- 172.21.15.19:0/3103782545 >> [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898] conn(0x7fc5300bc620 0x7fc5300beb10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.233+0000 7fc54ec67700 1 -- 172.21.15.19:0/3103782545 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5300b2990 msgr2=0x7fc5300b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:14.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.233+0000 7fc54ec67700 1 --2- 172.21.15.19:0/3103782545 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5300b2990 0x7fc5300b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc544000c00 tx=0x7fc54401a040 comp rx=0 tx=0).stop 2024-05-09T02:09:14.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.233+0000 7fc54ec67700 1 -- 172.21.15.19:0/3103782545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548109870 msgr2=0x7fc54819f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:14.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.233+0000 7fc54ec67700 1 --2- 172.21.15.19:0/3103782545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc548109870 0x7fc54819f6d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc54000ef30 tx=0x7fc54000c3a0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.233+0000 7fc54d204700 1 -- 172.21.15.19:0/3103782545 reap_dead start 2024-05-09T02:09:14.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.233+0000 7fc54ec67700 1 -- 172.21.15.19:0/3103782545 shutdown_connections 2024-05-09T02:09:14.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.233+0000 7fc54ec67700 1 -- 172.21.15.19:0/3103782545 >> 172.21.15.19:0/3103782545 conn(0x7fc5480fffe0 msgr2=0x7fc548107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.233+0000 7fc54ec67700 1 -- 172.21.15.19:0/3103782545 shutdown_connections 2024-05-09T02:09:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.233+0000 7fc54ec67700 1 -- 172.21.15.19:0/3103782545 wait complete. 2024-05-09T02:09:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.428+0000 7fc6977c4700 1 -- 172.21.15.19:0/2101717033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6901034c0 msgr2=0x7fc6901038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.428+0000 7fc6977c4700 1 --2- 172.21.15.19:0/2101717033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6901034c0 0x7fc6901038d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0099c0 tx=0x7fc68c009cd0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.428+0000 7fc6977c4700 1 -- 172.21.15.19:0/2101717033 shutdown_connections 2024-05-09T02:09:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.428+0000 7fc6977c4700 1 --2- 172.21.15.19:0/2101717033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc690108740 0x7fc6901100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.428+0000 7fc6977c4700 1 --2- 172.21.15.19:0/2101717033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6901034c0 0x7fc6901038d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.428+0000 7fc6977c4700 1 --2- 172.21.15.19:0/2101717033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc690107df0 0x7fc690108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.428+0000 7fc6977c4700 1 -- 172.21.15.19:0/2101717033 >> 172.21.15.19:0/2101717033 conn(0x7fc6900fedc0 msgr2=0x7fc6901011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.428+0000 7fc6977c4700 1 -- 172.21.15.19:0/2101717033 shutdown_connections 2024-05-09T02:09:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.428+0000 7fc6977c4700 1 -- 172.21.15.19:0/2101717033 wait complete. 2024-05-09T02:09:14.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.429+0000 7fc6977c4700 1 Processor -- start 2024-05-09T02:09:14.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.429+0000 7fc6977c4700 1 -- start start 2024-05-09T02:09:14.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.429+0000 7fc6977c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6901034c0 0x7fc69019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:14.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.429+0000 7fc6977c4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc690107df0 0x7fc69019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:14.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.429+0000 7fc6977c4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc690108740 0x7fc690199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:14.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.429+0000 7fc6977c4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc690113370 con 0x7fc6901034c0 2024-05-09T02:09:14.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.429+0000 7fc6977c4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc6901131f0 con 0x7fc690108740 2024-05-09T02:09:14.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.429+0000 7fc6977c4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6901134f0 con 0x7fc690107df0 2024-05-09T02:09:14.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.429+0000 7fc695560700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6901034c0 0x7fc69019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:14.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.429+0000 7fc695560700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6901034c0 0x7fc69019f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35754/0 (socket says 172.21.15.19:35754) 2024-05-09T02:09:14.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.429+0000 7fc695560700 1 -- 172.21.15.19:0/145598588 learned_addr learned my addr 172.21.15.19:0/145598588 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:14.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.430+0000 7fc694d5f700 1 --2- 172.21.15.19:0/145598588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc690107df0 0x7fc69019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:14.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.430+0000 7fc695d61700 1 --2- 172.21.15.19:0/145598588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc690108740 0x7fc690199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:14.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.430+0000 7fc695560700 1 -- 172.21.15.19:0/145598588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc690107df0 msgr2=0x7fc69019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:14.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.430+0000 7fc695560700 1 --2- 172.21.15.19:0/145598588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc690107df0 0x7fc69019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.430+0000 7fc695560700 1 -- 172.21.15.19:0/145598588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc690108740 msgr2=0x7fc690199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:14.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.430+0000 7fc695560700 1 --2- 172.21.15.19:0/145598588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc690108740 0x7fc690199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.430+0000 7fc695560700 1 -- 172.21.15.19:0/145598588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69019a160 con 0x7fc6901034c0 2024-05-09T02:09:14.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.430+0000 7fc695560700 1 --2- 172.21.15.19:0/145598588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6901034c0 0x7fc69019f8b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0098e0 tx=0x7fc68c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:14.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.430+0000 7fc6867fc700 1 -- 172.21.15.19:0/145598588 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c027070 con 0x7fc6901034c0 2024-05-09T02:09:14.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.430+0000 7fc6867fc700 1 -- 172.21.15.19:0/145598588 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc68c003fe0 con 0x7fc6901034c0 2024-05-09T02:09:14.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.430+0000 7fc6977c4700 1 -- 172.21.15.19:0/145598588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69019a3f0 con 0x7fc6901034c0 2024-05-09T02:09:14.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.430+0000 7fc6867fc700 1 -- 172.21.15.19:0/145598588 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c022d30 con 0x7fc6901034c0 2024-05-09T02:09:14.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.430+0000 7fc6977c4700 1 -- 172.21.15.19:0/145598588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69019a8d0 con 0x7fc6901034c0 2024-05-09T02:09:14.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.432+0000 7fc67bfff700 1 -- 172.21.15.19:0/145598588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc670001030 con 0x7fc6901034c0 2024-05-09T02:09:14.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.434+0000 7fc6867fc700 1 -- 172.21.15.19:0/145598588 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc68c02a070 con 0x7fc6901034c0 2024-05-09T02:09:14.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.434+0000 7fc6867fc700 1 --2- 172.21.15.19:0/145598588 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc67c0b2910 0x7fc67c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:14.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.435+0000 7fc694d5f700 1 --2- 172.21.15.19:0/145598588 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc67c0b2910 0x7fc67c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:14.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.435+0000 7fc6867fc700 1 -- 172.21.15.19:0/145598588 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc68c0e34b0 con 0x7fc6901034c0 2024-05-09T02:09:14.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.435+0000 7fc6867fc700 1 --2- 172.21.15.19:0/145598588 >> [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053] conn(0x7fc67c0bc5a0 0x7fc67c0bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:14.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.435+0000 7fc6867fc700 1 -- 172.21.15.19:0/145598588 --> [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc68c019ea0 con 0x7fc67c0bc5a0 2024-05-09T02:09:14.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.435+0000 7fc6867fc700 1 -- 172.21.15.19:0/145598588 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fc68c0e3900 con 0x7fc6901034c0 2024-05-09T02:09:14.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.435+0000 7fc694d5f700 1 --2- 172.21.15.19:0/145598588 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc67c0b2910 0x7fc67c0b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc6901063e0 tx=0x7fc68000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:14.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.435+0000 7fc695d61700 1 --2- 172.21.15.19:0/145598588 >> [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053] conn(0x7fc67c0bc5a0 0x7fc67c0bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:14.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.436+0000 7fc695d61700 1 --2- 172.21.15.19:0/145598588 >> [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053] conn(0x7fc67c0bc5a0 0x7fc67c0bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:14.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.436+0000 7fc6867fc700 1 -- 172.21.15.19:0/145598588 <== osd.7 v2:172.21.15.59:6816/967902053 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc68c019ea0 con 0x7fc67c0bc5a0 2024-05-09T02:09:14.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.454+0000 7fc67bfff700 1 -- 172.21.15.19:0/145598588 --> [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc670002e60 con 0x7fc67c0bc5a0 2024-05-09T02:09:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.455+0000 7fc6867fc700 1 -- 172.21.15.19:0/145598588 <== osd.7 v2:172.21.15.59:6816/967902053 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc670002e60 con 0x7fc67c0bc5a0 2024-05-09T02:09:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.455+0000 7fc67bfff700 1 -- 172.21.15.19:0/145598588 >> [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053] conn(0x7fc67c0bc5a0 msgr2=0x7fc67c0bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.455+0000 7fc67bfff700 1 --2- 172.21.15.19:0/145598588 >> [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053] conn(0x7fc67c0bc5a0 0x7fc67c0bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.455+0000 7fc67bfff700 1 -- 172.21.15.19:0/145598588 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc67c0b2910 msgr2=0x7fc67c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.455+0000 7fc67bfff700 1 --2- 172.21.15.19:0/145598588 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc67c0b2910 0x7fc67c0b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc6901063e0 tx=0x7fc68000a380 comp rx=0 tx=0).stop 2024-05-09T02:09:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.455+0000 7fc67bfff700 1 -- 172.21.15.19:0/145598588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6901034c0 msgr2=0x7fc69019f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.455+0000 7fc67bfff700 1 --2- 172.21.15.19:0/145598588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6901034c0 0x7fc69019f8b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0098e0 tx=0x7fc68c0057c0 comp rx=0 tx=0).stop 2024-05-09T02:09:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.455+0000 7fc695d61700 1 -- 172.21.15.19:0/145598588 reap_dead start 2024-05-09T02:09:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.455+0000 7fc67bfff700 1 -- 172.21.15.19:0/145598588 shutdown_connections 2024-05-09T02:09:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.455+0000 7fc67bfff700 1 -- 172.21.15.19:0/145598588 >> 172.21.15.19:0/145598588 conn(0x7fc6900fedc0 msgr2=0x7fc690101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.455+0000 7fc67bfff700 1 -- 172.21.15.19:0/145598588 shutdown_connections 2024-05-09T02:09:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:14.455+0000 7fc67bfff700 1 -- 172.21.15.19:0/145598588 wait complete. 2024-05-09T02:09:15.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.594+0000 7fe5169a8700 1 -- 172.21.15.19:0/2409449888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe510108740 msgr2=0x7fe5101100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:15.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.594+0000 7fe5169a8700 1 --2- 172.21.15.19:0/2409449888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe510108740 0x7fe5101100a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe50800b0d0 tx=0x7fe50800b3e0 comp rx=0 tx=0).stop 2024-05-09T02:09:15.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.594+0000 7fe5169a8700 1 -- 172.21.15.19:0/2409449888 shutdown_connections 2024-05-09T02:09:15.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.594+0000 7fe5169a8700 1 --2- 172.21.15.19:0/2409449888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe510108740 0x7fe5101100a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:15.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.594+0000 7fe5169a8700 1 --2- 172.21.15.19:0/2409449888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5101034c0 0x7fe5101038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:15.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.594+0000 7fe5169a8700 1 --2- 172.21.15.19:0/2409449888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe510107df0 0x7fe510108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:15.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.594+0000 7fe5169a8700 1 -- 172.21.15.19:0/2409449888 >> 172.21.15.19:0/2409449888 conn(0x7fe5100fedc0 msgr2=0x7fe5101011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:15.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.594+0000 7fe5169a8700 1 -- 172.21.15.19:0/2409449888 shutdown_connections 2024-05-09T02:09:15.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.594+0000 7fe5169a8700 1 -- 172.21.15.19:0/2409449888 wait complete. 2024-05-09T02:09:15.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.595+0000 7fe5169a8700 1 Processor -- start 2024-05-09T02:09:15.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.595+0000 7fe5169a8700 1 -- start start 2024-05-09T02:09:15.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe5169a8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5101034c0 0x7fe51019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:15.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe5169a8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe510107df0 0x7fe51019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:15.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe5169a8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe510108740 0x7fe51019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe5169a8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe510113370 con 0x7fe5101034c0 2024-05-09T02:09:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe5169a8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5101131f0 con 0x7fe510108740 2024-05-09T02:09:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe5169a8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5101134f0 con 0x7fe510107df0 2024-05-09T02:09:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe50ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5101034c0 0x7fe51019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe50ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5101034c0 0x7fe51019bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35768/0 (socket says 172.21.15.19:35768) 2024-05-09T02:09:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe50ffff700 1 -- 172.21.15.19:0/3486156626 learned_addr learned my addr 172.21.15.19:0/3486156626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe514f45700 1 --2- 172.21.15.19:0/3486156626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe510108740 0x7fe51019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe50f7fe700 1 --2- 172.21.15.19:0/3486156626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe510107df0 0x7fe51019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe50ffff700 1 -- 172.21.15.19:0/3486156626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe510107df0 msgr2=0x7fe51019c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe50ffff700 1 --2- 172.21.15.19:0/3486156626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe510107df0 0x7fe51019c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe50ffff700 1 -- 172.21.15.19:0/3486156626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe510108740 msgr2=0x7fe51019c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe50ffff700 1 --2- 172.21.15.19:0/3486156626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe510108740 0x7fe51019c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe50ffff700 1 -- 172.21.15.19:0/3486156626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5101a6ed0 con 0x7fe5101034c0 2024-05-09T02:09:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe514f45700 1 --2- 172.21.15.19:0/3486156626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe510108740 0x7fe51019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:09:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.596+0000 7fe50ffff700 1 --2- 172.21.15.19:0/3486156626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5101034c0 0x7fe51019bc10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe50000c9b0 tx=0x7fe50000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.597+0000 7fe50d7fa700 1 -- 172.21.15.19:0/3486156626 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe500007b00 con 0x7fe5101034c0 2024-05-09T02:09:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.597+0000 7fe5169a8700 1 -- 172.21.15.19:0/3486156626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5101a71c0 con 0x7fe5101034c0 2024-05-09T02:09:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.597+0000 7fe50d7fa700 1 -- 172.21.15.19:0/3486156626 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe500007ca0 con 0x7fe5101034c0 2024-05-09T02:09:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.597+0000 7fe50d7fa700 1 -- 172.21.15.19:0/3486156626 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe500018730 con 0x7fe5101034c0 2024-05-09T02:09:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.597+0000 7fe5169a8700 1 -- 172.21.15.19:0/3486156626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5101a7750 con 0x7fe5101034c0 2024-05-09T02:09:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.598+0000 7fe5169a8700 1 -- 172.21.15.19:0/3486156626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe510063ed0 con 0x7fe5101034c0 2024-05-09T02:09:15.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.600+0000 7fe50d7fa700 1 -- 172.21.15.19:0/3486156626 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe50000f450 con 0x7fe5101034c0 2024-05-09T02:09:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.600+0000 7fe50d7fa700 1 --2- 172.21.15.19:0/3486156626 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe4f80b2910 0x7fe4f80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.600+0000 7fe50f7fe700 1 --2- 172.21.15.19:0/3486156626 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe4f80b2910 0x7fe4f80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.600+0000 7fe50d7fa700 1 -- 172.21.15.19:0/3486156626 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fe5000e2ad0 con 0x7fe5101034c0 2024-05-09T02:09:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.601+0000 7fe50d7fa700 1 --2- 172.21.15.19:0/3486156626 >> [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995] conn(0x7fe4f80bc5a0 0x7fe4f80bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.601+0000 7fe50d7fa700 1 -- 172.21.15.19:0/3486156626 --> [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe500014ec0 con 0x7fe4f80bc5a0 2024-05-09T02:09:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.601+0000 7fe50d7fa700 1 -- 172.21.15.19:0/3486156626 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fe5000e2f20 con 0x7fe5101034c0 2024-05-09T02:09:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.601+0000 7fe50f7fe700 1 --2- 172.21.15.19:0/3486156626 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe4f80b2910 0x7fe4f80b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe510102360 tx=0x7fe504009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:15.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.601+0000 7fe514f45700 1 --2- 172.21.15.19:0/3486156626 >> [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995] conn(0x7fe4f80bc5a0 0x7fe4f80bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:15.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.602+0000 7fe514f45700 1 --2- 172.21.15.19:0/3486156626 >> [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995] conn(0x7fe4f80bc5a0 0x7fe4f80bea90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:15.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.603+0000 7fe50d7fa700 1 -- 172.21.15.19:0/3486156626 <== osd.10 v2:172.21.15.59:6824/2932899995 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe500014ec0 con 0x7fe4f80bc5a0 2024-05-09T02:09:15.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.619+0000 7fe5169a8700 1 -- 172.21.15.19:0/3486156626 --> [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe510064300 con 0x7fe4f80bc5a0 2024-05-09T02:09:15.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.620+0000 7fe50d7fa700 1 -- 172.21.15.19:0/3486156626 <== osd.10 v2:172.21.15.59:6824/2932899995 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe510064300 con 0x7fe4f80bc5a0 2024-05-09T02:09:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.620+0000 7fe5169a8700 1 -- 172.21.15.19:0/3486156626 >> [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995] conn(0x7fe4f80bc5a0 msgr2=0x7fe4f80bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.620+0000 7fe5169a8700 1 --2- 172.21.15.19:0/3486156626 >> [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995] conn(0x7fe4f80bc5a0 0x7fe4f80bea90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.620+0000 7fe5169a8700 1 -- 172.21.15.19:0/3486156626 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe4f80b2910 msgr2=0x7fe4f80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.620+0000 7fe5169a8700 1 --2- 172.21.15.19:0/3486156626 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe4f80b2910 0x7fe4f80b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe510102360 tx=0x7fe504009260 comp rx=0 tx=0).stop 2024-05-09T02:09:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.620+0000 7fe5169a8700 1 -- 172.21.15.19:0/3486156626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5101034c0 msgr2=0x7fe51019bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.620+0000 7fe5169a8700 1 --2- 172.21.15.19:0/3486156626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5101034c0 0x7fe51019bc10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe50000c9b0 tx=0x7fe50000cd70 comp rx=0 tx=0).stop 2024-05-09T02:09:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.620+0000 7fe514f45700 1 -- 172.21.15.19:0/3486156626 reap_dead start 2024-05-09T02:09:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.620+0000 7fe5169a8700 1 -- 172.21.15.19:0/3486156626 shutdown_connections 2024-05-09T02:09:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.620+0000 7fe5169a8700 1 -- 172.21.15.19:0/3486156626 >> 172.21.15.19:0/3486156626 conn(0x7fe5100fedc0 msgr2=0x7fe510107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.620+0000 7fe5169a8700 1 -- 172.21.15.19:0/3486156626 shutdown_connections 2024-05-09T02:09:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:15.620+0000 7fe5169a8700 1 -- 172.21.15.19:0/3486156626 wait complete. 2024-05-09T02:09:15.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:15 smithi059 ceph-mon[34781]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:09:15.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:15 smithi196 ceph-mon[34718]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:09:15.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:15 smithi019 ceph-mon[30151]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:09:16.357 INFO:teuthology.orchestra.run.smithi019.stdout:206158430262 2024-05-09T02:09:16.357 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-09T02:09:17.037 INFO:teuthology.orchestra.run.smithi019.stdout:184683593786 2024-05-09T02:09:17.037 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-09T02:09:17.313 INFO:teuthology.orchestra.run.smithi019.stdout:137438953541 2024-05-09T02:09:17.313 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-09T02:09:17.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.536+0000 7fb260ad5700 1 -- 172.21.15.19:0/401608304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb25c107e10 msgr2=0x7fb25c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:17.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.536+0000 7fb260ad5700 1 --2- 172.21.15.19:0/401608304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb25c107e10 0x7fb25c108220 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb2500099c0 tx=0x7fb250009cd0 comp rx=0 tx=0).stop 2024-05-09T02:09:17.539 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:17 smithi019 ceph-mon[30151]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-09T02:09:17.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.536+0000 7fb260ad5700 1 -- 172.21.15.19:0/401608304 shutdown_connections 2024-05-09T02:09:17.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.536+0000 7fb260ad5700 1 --2- 172.21.15.19:0/401608304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb25c108760 0x7fb25c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:17.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.536+0000 7fb260ad5700 1 --2- 172.21.15.19:0/401608304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb25c1034e0 0x7fb25c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:17.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.536+0000 7fb260ad5700 1 --2- 172.21.15.19:0/401608304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb25c107e10 0x7fb25c108220 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:17.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.536+0000 7fb260ad5700 1 -- 172.21.15.19:0/401608304 >> 172.21.15.19:0/401608304 conn(0x7fb25c0fef30 msgr2=0x7fb25c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:17.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.537+0000 7fb260ad5700 1 -- 172.21.15.19:0/401608304 shutdown_connections 2024-05-09T02:09:17.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.537+0000 7fb260ad5700 1 -- 172.21.15.19:0/401608304 wait complete. 2024-05-09T02:09:17.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.538+0000 7fb260ad5700 1 Processor -- start 2024-05-09T02:09:17.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.538+0000 7fb260ad5700 1 -- start start 2024-05-09T02:09:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.538+0000 7fb260ad5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb25c1034e0 0x7fb25c199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.538+0000 7fb260ad5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb25c107e10 0x7fb25c19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.538+0000 7fb260ad5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb25c108760 0x7fb25c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.538+0000 7fb260ad5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c1133e0 con 0x7fb25c108760 2024-05-09T02:09:17.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.538+0000 7fb260ad5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c113260 con 0x7fb25c107e10 2024-05-09T02:09:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.538+0000 7fb260ad5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c113560 con 0x7fb25c1034e0 2024-05-09T02:09:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.538+0000 7fb25ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb25c108760 0x7fb25c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.539+0000 7fb25ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb25c108760 0x7fb25c19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47128/0 (socket says 172.21.15.19:47128) 2024-05-09T02:09:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.539+0000 7fb25ad9d700 1 -- 172.21.15.19:0/3970614586 learned_addr learned my addr 172.21.15.19:0/3970614586 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.539+0000 7fb259d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb25c107e10 0x7fb25c19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.539+0000 7fb25a59c700 1 --2- 172.21.15.19:0/3970614586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb25c1034e0 0x7fb25c199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.539+0000 7fb259d9b700 1 -- 172.21.15.19:0/3970614586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb25c1034e0 msgr2=0x7fb25c199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.539+0000 7fb259d9b700 1 --2- 172.21.15.19:0/3970614586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb25c1034e0 0x7fb25c199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.539+0000 7fb259d9b700 1 -- 172.21.15.19:0/3970614586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb25c108760 msgr2=0x7fb25c19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.539+0000 7fb259d9b700 1 --2- 172.21.15.19:0/3970614586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb25c108760 0x7fb25c19e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.539+0000 7fb259d9b700 1 -- 172.21.15.19:0/3970614586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25c19ed00 con 0x7fb25c107e10 2024-05-09T02:09:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.539+0000 7fb25ad9d700 1 --2- 172.21.15.19:0/3970614586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb25c108760 0x7fb25c19e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:09:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.539+0000 7fb259d9b700 1 --2- 172.21.15.19:0/3970614586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb25c107e10 0x7fb25c19a3d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb25000b580 tx=0x7fb2500047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:17.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.540+0000 7fb24b7fe700 1 -- 172.21.15.19:0/3970614586 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb250027070 con 0x7fb25c107e10 2024-05-09T02:09:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.540+0000 7fb24b7fe700 1 -- 172.21.15.19:0/3970614586 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb250003f20 con 0x7fb25c107e10 2024-05-09T02:09:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.540+0000 7fb260ad5700 1 -- 172.21.15.19:0/3970614586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25c19ef90 con 0x7fb25c107e10 2024-05-09T02:09:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.540+0000 7fb24b7fe700 1 -- 172.21.15.19:0/3970614586 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb250022bc0 con 0x7fb25c107e10 2024-05-09T02:09:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.540+0000 7fb260ad5700 1 -- 172.21.15.19:0/3970614586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb25c19f440 con 0x7fb25c107e10 2024-05-09T02:09:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.541+0000 7fb260ad5700 1 -- 172.21.15.19:0/3970614586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb25c06c310 con 0x7fb25c107e10 2024-05-09T02:09:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.542+0000 7fb24b7fe700 1 -- 172.21.15.19:0/3970614586 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb25002a070 con 0x7fb25c107e10 2024-05-09T02:09:17.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.543+0000 7fb24b7fe700 1 --2- 172.21.15.19:0/3970614586 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb2440b2910 0x7fb2440b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:17.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.543+0000 7fb24b7fe700 1 -- 172.21.15.19:0/3970614586 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fb2500e3e10 con 0x7fb25c107e10 2024-05-09T02:09:17.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.543+0000 7fb25a59c700 1 --2- 172.21.15.19:0/3970614586 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb2440b2910 0x7fb2440b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:17.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.543+0000 7fb24b7fe700 1 --2- 172.21.15.19:0/3970614586 >> [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353] conn(0x7fb2440bc5a0 0x7fb2440bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:17.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.543+0000 7fb24b7fe700 1 -- 172.21.15.19:0/3970614586 --> [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb250010ea0 con 0x7fb2440bc5a0 2024-05-09T02:09:17.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.543+0000 7fb24b7fe700 1 -- 172.21.15.19:0/3970614586 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fb2500e4260 con 0x7fb25c107e10 2024-05-09T02:09:17.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.544+0000 7fb25ad9d700 1 --2- 172.21.15.19:0/3970614586 >> [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353] conn(0x7fb2440bc5a0 0x7fb2440bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:17.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.544+0000 7fb25a59c700 1 --2- 172.21.15.19:0/3970614586 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb2440b2910 0x7fb2440b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb24c005950 tx=0x7fb24c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:17.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.544+0000 7fb25ad9d700 1 --2- 172.21.15.19:0/3970614586 >> [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353] conn(0x7fb2440bc5a0 0x7fb2440bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:17.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.545+0000 7fb24b7fe700 1 -- 172.21.15.19:0/3970614586 <== osd.1 v2:172.21.15.59:6800/827458353 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb250010ea0 con 0x7fb2440bc5a0 2024-05-09T02:09:17.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.563+0000 7fb260ad5700 1 -- 172.21.15.19:0/3970614586 --> [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb25c06c790 con 0x7fb2440bc5a0 2024-05-09T02:09:17.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.563+0000 7fb24b7fe700 1 -- 172.21.15.19:0/3970614586 <== osd.1 v2:172.21.15.59:6800/827458353 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb25c06c790 con 0x7fb2440bc5a0 2024-05-09T02:09:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.563+0000 7fb260ad5700 1 -- 172.21.15.19:0/3970614586 >> [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353] conn(0x7fb2440bc5a0 msgr2=0x7fb2440bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.563+0000 7fb260ad5700 1 --2- 172.21.15.19:0/3970614586 >> [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353] conn(0x7fb2440bc5a0 0x7fb2440bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.564+0000 7fb260ad5700 1 -- 172.21.15.19:0/3970614586 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb2440b2910 msgr2=0x7fb2440b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.564+0000 7fb260ad5700 1 --2- 172.21.15.19:0/3970614586 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb2440b2910 0x7fb2440b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb24c005950 tx=0x7fb24c00b410 comp rx=0 tx=0).stop 2024-05-09T02:09:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.564+0000 7fb260ad5700 1 -- 172.21.15.19:0/3970614586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb25c107e10 msgr2=0x7fb25c19a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.564+0000 7fb260ad5700 1 --2- 172.21.15.19:0/3970614586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb25c107e10 0x7fb25c19a3d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb25000b580 tx=0x7fb2500047b0 comp rx=0 tx=0).stop 2024-05-09T02:09:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.564+0000 7fb25ad9d700 1 -- 172.21.15.19:0/3970614586 reap_dead start 2024-05-09T02:09:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.564+0000 7fb260ad5700 1 -- 172.21.15.19:0/3970614586 shutdown_connections 2024-05-09T02:09:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.564+0000 7fb260ad5700 1 -- 172.21.15.19:0/3970614586 >> 172.21.15.19:0/3970614586 conn(0x7fb25c0fef30 msgr2=0x7fb25c1096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.564+0000 7fb260ad5700 1 -- 172.21.15.19:0/3970614586 shutdown_connections 2024-05-09T02:09:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:17.564+0000 7fb260ad5700 1 -- 172.21.15.19:0/3970614586 wait complete. 2024-05-09T02:09:17.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:17 smithi059 ceph-mon[34781]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-09T02:09:17.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:17 smithi196 ceph-mon[34718]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-09T02:09:18.005 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:18.771 INFO:teuthology.orchestra.run.smithi019.stdout:55834574934 2024-05-09T02:09:18.771 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-09T02:09:18.909 INFO:teuthology.orchestra.run.smithi019.stdout:158913790017 2024-05-09T02:09:18.909 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-09T02:09:19.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:19 smithi059 ceph-mon[34781]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-09T02:09:19.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:19 smithi019 ceph-mon[30151]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-09T02:09:19.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:19 smithi196 ceph-mon[34718]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-09T02:09:19.992 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:19.992 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:19.993 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:20.498 INFO:teuthology.orchestra.run.smithi019.stdout:279172874277 2024-05-09T02:09:20.498 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-09T02:09:21.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.177+0000 7fe16d4f4700 1 -- 172.21.15.19:0/2692533940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe168107df0 msgr2=0x7fe168108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:21.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.177+0000 7fe16d4f4700 1 --2- 172.21.15.19:0/2692533940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe168107df0 0x7fe168108200 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0099c0 tx=0x7fe15c009cd0 comp rx=0 tx=0).stop 2024-05-09T02:09:21.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.177+0000 7fe16d4f4700 1 -- 172.21.15.19:0/2692533940 shutdown_connections 2024-05-09T02:09:21.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.177+0000 7fe16d4f4700 1 --2- 172.21.15.19:0/2692533940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe168108740 0x7fe1681100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:21.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.177+0000 7fe16d4f4700 1 --2- 172.21.15.19:0/2692533940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1681034c0 0x7fe1681038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:21.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.177+0000 7fe16d4f4700 1 --2- 172.21.15.19:0/2692533940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe168107df0 0x7fe168108200 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:21.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.177+0000 7fe16d4f4700 1 -- 172.21.15.19:0/2692533940 >> 172.21.15.19:0/2692533940 conn(0x7fe1680fedc0 msgr2=0x7fe1681011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:21.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.177+0000 7fe16d4f4700 1 -- 172.21.15.19:0/2692533940 shutdown_connections 2024-05-09T02:09:21.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.177+0000 7fe16d4f4700 1 -- 172.21.15.19:0/2692533940 wait complete. 2024-05-09T02:09:21.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.178+0000 7fe16d4f4700 1 Processor -- start 2024-05-09T02:09:21.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.178+0000 7fe16d4f4700 1 -- start start 2024-05-09T02:09:21.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe16d4f4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1681034c0 0x7fe16819d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:21.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe16d4f4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe168107df0 0x7fe16819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:21.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe16d4f4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe168108740 0x7fe1681a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:21.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe16d4f4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe168112320 con 0x7fe168108740 2024-05-09T02:09:21.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe16d4f4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681121a0 con 0x7fe1681034c0 2024-05-09T02:09:21.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe16d4f4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681124a0 con 0x7fe168107df0 2024-05-09T02:09:21.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe1677fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe168108740 0x7fe1681a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:21.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe1677fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe168108740 0x7fe1681a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47148/0 (socket says 172.21.15.19:47148) 2024-05-09T02:09:21.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe1677fe700 1 -- 172.21.15.19:0/1500466251 learned_addr learned my addr 172.21.15.19:0/1500466251 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:21.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe1667fc700 1 --2- 172.21.15.19:0/1500466251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe168107df0 0x7fe16819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:21.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe166ffd700 1 --2- 172.21.15.19:0/1500466251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1681034c0 0x7fe16819d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:21.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe1677fe700 1 -- 172.21.15.19:0/1500466251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe168107df0 msgr2=0x7fe16819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:21.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe1677fe700 1 --2- 172.21.15.19:0/1500466251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe168107df0 0x7fe16819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:21.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe1677fe700 1 -- 172.21.15.19:0/1500466251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1681034c0 msgr2=0x7fe16819d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:21.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe1677fe700 1 --2- 172.21.15.19:0/1500466251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1681034c0 0x7fe16819d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:21.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.179+0000 7fe1677fe700 1 -- 172.21.15.19:0/1500466251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1681a1f50 con 0x7fe168108740 2024-05-09T02:09:21.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.180+0000 7fe1677fe700 1 --2- 172.21.15.19:0/1500466251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe168108740 0x7fe1681a18b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe16000ada0 tx=0x7fe16000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:21.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.180+0000 7fe14ffff700 1 -- 172.21.15.19:0/1500466251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160017020 con 0x7fe168108740 2024-05-09T02:09:21.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.180+0000 7fe16d4f4700 1 -- 172.21.15.19:0/1500466251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1681a2240 con 0x7fe168108740 2024-05-09T02:09:21.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.180+0000 7fe14ffff700 1 -- 172.21.15.19:0/1500466251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe16000cd80 con 0x7fe168108740 2024-05-09T02:09:21.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.180+0000 7fe14ffff700 1 -- 172.21.15.19:0/1500466251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160005260 con 0x7fe168108740 2024-05-09T02:09:21.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.180+0000 7fe16d4f4700 1 -- 172.21.15.19:0/1500466251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1681a9b60 con 0x7fe168108740 2024-05-09T02:09:21.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.181+0000 7fe16d4f4700 1 -- 172.21.15.19:0/1500466251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe168063f50 con 0x7fe168108740 2024-05-09T02:09:21.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.183+0000 7fe14ffff700 1 -- 172.21.15.19:0/1500466251 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe16000c8d0 con 0x7fe168108740 2024-05-09T02:09:21.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.183+0000 7fe14ffff700 1 --2- 172.21.15.19:0/1500466251 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe1500b2900 0x7fe1500b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:21.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.184+0000 7fe14ffff700 1 -- 172.21.15.19:0/1500466251 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fe1600e27e0 con 0x7fe168108740 2024-05-09T02:09:21.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.184+0000 7fe166ffd700 1 --2- 172.21.15.19:0/1500466251 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe1500b2900 0x7fe1500b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:21.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.184+0000 7fe14ffff700 1 --2- 172.21.15.19:0/1500466251 >> [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886] conn(0x7fe1500bc590 0x7fe1500bea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:21.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.184+0000 7fe14ffff700 1 -- 172.21.15.19:0/1500466251 --> [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe16000bea0 con 0x7fe1500bc590 2024-05-09T02:09:21.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.184+0000 7fe14ffff700 1 -- 172.21.15.19:0/1500466251 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fe1600e2c30 con 0x7fe168108740 2024-05-09T02:09:21.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.184+0000 7fe1667fc700 1 --2- 172.21.15.19:0/1500466251 >> [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886] conn(0x7fe1500bc590 0x7fe1500bea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:21.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.184+0000 7fe166ffd700 1 --2- 172.21.15.19:0/1500466251 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe1500b2900 0x7fe1500b4e10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe158005950 tx=0x7fe15800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:21.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.184+0000 7fe1667fc700 1 --2- 172.21.15.19:0/1500466251 >> [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886] conn(0x7fe1500bc590 0x7fe1500bea80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:21.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.185+0000 7fe14ffff700 1 -- 172.21.15.19:0/1500466251 <== osd.9 v2:172.21.15.19:6826/1735017886 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe16000bea0 con 0x7fe1500bc590 2024-05-09T02:09:21.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.202+0000 7fe16d4f4700 1 -- 172.21.15.19:0/1500466251 --> [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe1680641a0 con 0x7fe1500bc590 2024-05-09T02:09:21.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.203+0000 7fe14ffff700 1 -- 172.21.15.19:0/1500466251 <== osd.9 v2:172.21.15.19:6826/1735017886 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe1680641a0 con 0x7fe1500bc590 2024-05-09T02:09:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.203+0000 7fe16d4f4700 1 -- 172.21.15.19:0/1500466251 >> [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886] conn(0x7fe1500bc590 msgr2=0x7fe1500bea80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.203+0000 7fe16d4f4700 1 --2- 172.21.15.19:0/1500466251 >> [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886] conn(0x7fe1500bc590 0x7fe1500bea80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.203+0000 7fe16d4f4700 1 -- 172.21.15.19:0/1500466251 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe1500b2900 msgr2=0x7fe1500b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.203+0000 7fe16d4f4700 1 --2- 172.21.15.19:0/1500466251 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe1500b2900 0x7fe1500b4e10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe158005950 tx=0x7fe15800a300 comp rx=0 tx=0).stop 2024-05-09T02:09:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.203+0000 7fe16d4f4700 1 -- 172.21.15.19:0/1500466251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe168108740 msgr2=0x7fe1681a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.203+0000 7fe16d4f4700 1 --2- 172.21.15.19:0/1500466251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe168108740 0x7fe1681a18b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe16000ada0 tx=0x7fe16000c330 comp rx=0 tx=0).stop 2024-05-09T02:09:21.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.203+0000 7fe1677fe700 1 -- 172.21.15.19:0/1500466251 reap_dead start 2024-05-09T02:09:21.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.203+0000 7fe16d4f4700 1 -- 172.21.15.19:0/1500466251 shutdown_connections 2024-05-09T02:09:21.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.203+0000 7fe16d4f4700 1 -- 172.21.15.19:0/1500466251 >> 172.21.15.19:0/1500466251 conn(0x7fe1680fedc0 msgr2=0x7fe168101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:21.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.203+0000 7fe16d4f4700 1 -- 172.21.15.19:0/1500466251 shutdown_connections 2024-05-09T02:09:21.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:21.203+0000 7fe16d4f4700 1 -- 172.21.15.19:0/1500466251 wait complete. 2024-05-09T02:09:21.511 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:21 smithi019 ceph-mon[30151]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:09:21.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:21 smithi059 ceph-mon[34781]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:09:21.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:21 smithi196 ceph-mon[34718]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:09:22.011 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:22.040 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:23.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:23 smithi019 ceph-mon[30151]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-05-09T02:09:23.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:23 smithi059 ceph-mon[34781]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-05-09T02:09:23.735 INFO:teuthology.orchestra.run.smithi019.stdout:257698037805 2024-05-09T02:09:23.735 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-09T02:09:23.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:23 smithi196 ceph-mon[34718]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-05-09T02:09:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:25.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:25 smithi019 ceph-mon[30151]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:09:25.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:25 smithi059 ceph-mon[34781]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:09:25.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:25 smithi196 ceph-mon[34718]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:09:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.357+0000 7f39bed7c700 1 -- 172.21.15.19:0/2523304081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39b81034e0 msgr2=0x7f39b81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.357+0000 7f39bed7c700 1 --2- 172.21.15.19:0/2523304081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39b81034e0 0x7f39b81038f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0099c0 tx=0x7f39ac009cd0 comp rx=0 tx=0).stop 2024-05-09T02:09:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.357+0000 7f39bed7c700 1 -- 172.21.15.19:0/2523304081 shutdown_connections 2024-05-09T02:09:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.357+0000 7f39bed7c700 1 --2- 172.21.15.19:0/2523304081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b8108760 0x7f39b81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.357+0000 7f39bed7c700 1 --2- 172.21.15.19:0/2523304081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39b81034e0 0x7f39b81038f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.357+0000 7f39bed7c700 1 --2- 172.21.15.19:0/2523304081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b8107e10 0x7f39b8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.357+0000 7f39bed7c700 1 -- 172.21.15.19:0/2523304081 >> 172.21.15.19:0/2523304081 conn(0x7f39b80fef30 msgr2=0x7f39b8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.357+0000 7f39bed7c700 1 -- 172.21.15.19:0/2523304081 shutdown_connections 2024-05-09T02:09:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.357+0000 7f39bed7c700 1 -- 172.21.15.19:0/2523304081 wait complete. 2024-05-09T02:09:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39bed7c700 1 Processor -- start 2024-05-09T02:09:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39bed7c700 1 -- start start 2024-05-09T02:09:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39bed7c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b81034e0 0x7f39b819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39bed7c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b8107e10 0x7f39b819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39bed7c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39b8108760 0x7f39b819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39bed7c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39b8113420 con 0x7f39b8108760 2024-05-09T02:09:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39bed7c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39b81132a0 con 0x7f39b8107e10 2024-05-09T02:09:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39bed7c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39b81135a0 con 0x7f39b81034e0 2024-05-09T02:09:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39bd319700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39b8108760 0x7f39b819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39bd319700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39b8108760 0x7f39b819f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47162/0 (socket says 172.21.15.19:47162) 2024-05-09T02:09:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39bd319700 1 -- 172.21.15.19:0/3766212369 learned_addr learned my addr 172.21.15.19:0/3766212369 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39bcb18700 1 --2- 172.21.15.19:0/3766212369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b81034e0 0x7f39b819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39b7fff700 1 --2- 172.21.15.19:0/3766212369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b8107e10 0x7f39b819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39bd319700 1 -- 172.21.15.19:0/3766212369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b81034e0 msgr2=0x7f39b819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39bd319700 1 --2- 172.21.15.19:0/3766212369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b81034e0 0x7f39b819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.358+0000 7f39bd319700 1 -- 172.21.15.19:0/3766212369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b8107e10 msgr2=0x7f39b819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.359+0000 7f39bd319700 1 --2- 172.21.15.19:0/3766212369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b8107e10 0x7f39b819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.359+0000 7f39bd319700 1 -- 172.21.15.19:0/3766212369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b819fd30 con 0x7f39b8108760 2024-05-09T02:09:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.359+0000 7f39b7fff700 1 --2- 172.21.15.19:0/3766212369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b8107e10 0x7f39b819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:09:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.359+0000 7f39bd319700 1 --2- 172.21.15.19:0/3766212369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39b8108760 0x7f39b819f690 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f39b0007ee0 tx=0x7f39b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.359+0000 7f39b5ffb700 1 -- 172.21.15.19:0/3766212369 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b000dbd0 con 0x7f39b8108760 2024-05-09T02:09:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.359+0000 7f39b5ffb700 1 -- 172.21.15.19:0/3766212369 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39b000dd70 con 0x7f39b8108760 2024-05-09T02:09:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.359+0000 7f39bed7c700 1 -- 172.21.15.19:0/3766212369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b81a0020 con 0x7f39b8108760 2024-05-09T02:09:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.359+0000 7f39b5ffb700 1 -- 172.21.15.19:0/3766212369 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0005990 con 0x7f39b8108760 2024-05-09T02:09:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.359+0000 7f39bed7c700 1 -- 172.21.15.19:0/3766212369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b81a7940 con 0x7f39b8108760 2024-05-09T02:09:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.360+0000 7f39bed7c700 1 -- 172.21.15.19:0/3766212369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b80512c0 con 0x7f39b8108760 2024-05-09T02:09:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.362+0000 7f39b5ffb700 1 -- 172.21.15.19:0/3766212369 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f39b001a020 con 0x7f39b8108760 2024-05-09T02:09:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.362+0000 7f39b5ffb700 1 --2- 172.21.15.19:0/3766212369 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f39a80b2910 0x7f39a80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.363+0000 7f39bcb18700 1 --2- 172.21.15.19:0/3766212369 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f39a80b2910 0x7f39a80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.363+0000 7f39b5ffb700 1 -- 172.21.15.19:0/3766212369 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f39b00e2210 con 0x7f39b8108760 2024-05-09T02:09:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.363+0000 7f39bcb18700 1 --2- 172.21.15.19:0/3766212369 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f39a80b2910 0x7f39a80b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009990 tx=0x7f39ac0036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.366+0000 7f39b5ffb700 1 -- 172.21.15.19:0/3766212369 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f39b0010070 con 0x7f39b8108760 2024-05-09T02:09:26.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.586+0000 7f39bed7c700 1 -- 172.21.15.19:0/3766212369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f39b802e5f0 con 0x7f39b8108760 2024-05-09T02:09:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.586+0000 7f39b5ffb700 1 -- 172.21.15.19:0/3766212369 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f39b00a9db0 con 0x7f39b8108760 2024-05-09T02:09:26.588 INFO:teuthology.orchestra.run.smithi019.stdout:304942678051 2024-05-09T02:09:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.589+0000 7f39bed7c700 1 -- 172.21.15.19:0/3766212369 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f39a80b2910 msgr2=0x7f39a80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.589+0000 7f39bed7c700 1 --2- 172.21.15.19:0/3766212369 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f39a80b2910 0x7f39a80b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009990 tx=0x7f39ac0036e0 comp rx=0 tx=0).stop 2024-05-09T02:09:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.589+0000 7f39bed7c700 1 -- 172.21.15.19:0/3766212369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39b8108760 msgr2=0x7f39b819f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.589+0000 7f39bed7c700 1 --2- 172.21.15.19:0/3766212369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39b8108760 0x7f39b819f690 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f39b0007ee0 tx=0x7f39b000d3b0 comp rx=0 tx=0).stop 2024-05-09T02:09:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.589+0000 7f39bed7c700 1 -- 172.21.15.19:0/3766212369 shutdown_connections 2024-05-09T02:09:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.589+0000 7f39bed7c700 1 --2- 172.21.15.19:0/3766212369 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f39a80b2910 0x7f39a80b4e20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.589+0000 7f39bed7c700 1 --2- 172.21.15.19:0/3766212369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39b8108760 0x7f39b819f690 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.589+0000 7f39bed7c700 1 --2- 172.21.15.19:0/3766212369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b81034e0 0x7f39b819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.589+0000 7f39bed7c700 1 --2- 172.21.15.19:0/3766212369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b8107e10 0x7f39b819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.589+0000 7f39bed7c700 1 -- 172.21.15.19:0/3766212369 >> 172.21.15.19:0/3766212369 conn(0x7f39b80fef30 msgr2=0x7f39b806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.589+0000 7f39bed7c700 1 -- 172.21.15.19:0/3766212369 shutdown_connections 2024-05-09T02:09:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:26.589+0000 7f39bed7c700 1 -- 172.21.15.19:0/3766212369 wait complete. 2024-05-09T02:09:27.574 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:27 smithi019 ceph-mon[30151]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:09:27.575 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:27 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3766212369' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T02:09:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:27 smithi059 ceph-mon[34781]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:09:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:27 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3766212369' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T02:09:27.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:27 smithi196 ceph-mon[34718]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:09:27.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:27 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3766212369' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T02:09:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:28.649 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678046 got 304942678051 for osd.11 2024-05-09T02:09:28.649 DEBUG:teuthology.parallel:result is None 2024-05-09T02:09:29.061 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:29.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:29 smithi019 ceph-mon[30151]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-05-09T02:09:29.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:29 smithi059 ceph-mon[34781]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-05-09T02:09:29.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:29 smithi196 ceph-mon[34718]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-05-09T02:09:31.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:31 smithi059 ceph-mon[34781]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-05-09T02:09:31.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:31 smithi196 ceph-mon[34718]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-05-09T02:09:31.817 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:31 smithi019 ceph-mon[30151]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-05-09T02:09:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.293+0000 7f5a859e6700 1 -- 172.21.15.19:0/2600481166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a801034c0 msgr2=0x7f5a801038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.293+0000 7f5a859e6700 1 --2- 172.21.15.19:0/2600481166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a801034c0 0x7f5a801038d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800b0d0 tx=0x7f5a7800b3e0 comp rx=0 tx=0).stop 2024-05-09T02:09:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.293+0000 7f5a859e6700 1 -- 172.21.15.19:0/2600481166 shutdown_connections 2024-05-09T02:09:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.293+0000 7f5a859e6700 1 --2- 172.21.15.19:0/2600481166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a80108740 0x7f5a801100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.293+0000 7f5a859e6700 1 --2- 172.21.15.19:0/2600481166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a801034c0 0x7f5a801038d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.293+0000 7f5a859e6700 1 --2- 172.21.15.19:0/2600481166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a80107df0 0x7f5a80108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.293+0000 7f5a859e6700 1 -- 172.21.15.19:0/2600481166 >> 172.21.15.19:0/2600481166 conn(0x7f5a800fedc0 msgr2=0x7f5a801011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.294+0000 7f5a859e6700 1 -- 172.21.15.19:0/2600481166 shutdown_connections 2024-05-09T02:09:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.294+0000 7f5a859e6700 1 -- 172.21.15.19:0/2600481166 wait complete. 2024-05-09T02:09:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.294+0000 7f5a859e6700 1 Processor -- start 2024-05-09T02:09:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.294+0000 7f5a859e6700 1 -- start start 2024-05-09T02:09:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a859e6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a801034c0 0x7f5a8019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a859e6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a80107df0 0x7f5a8019c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a859e6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a80108740 0x7f5a8019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a859e6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a80113370 con 0x7f5a80108740 2024-05-09T02:09:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a859e6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a801131f0 con 0x7f5a80107df0 2024-05-09T02:09:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a859e6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a801134f0 con 0x7f5a801034c0 2024-05-09T02:09:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a7f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a80108740 0x7f5a8019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a7f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a80108740 0x7f5a8019c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36166/0 (socket says 172.21.15.19:36166) 2024-05-09T02:09:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a7f7fe700 1 -- 172.21.15.19:0/1592342944 learned_addr learned my addr 172.21.15.19:0/1592342944 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a7e7fc700 1 --2- 172.21.15.19:0/1592342944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a80107df0 0x7f5a8019c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a7effd700 1 --2- 172.21.15.19:0/1592342944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a801034c0 0x7f5a8019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a7f7fe700 1 -- 172.21.15.19:0/1592342944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a801034c0 msgr2=0x7f5a8019bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a7f7fe700 1 --2- 172.21.15.19:0/1592342944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a801034c0 0x7f5a8019bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a7f7fe700 1 -- 172.21.15.19:0/1592342944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a80107df0 msgr2=0x7f5a8019c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a7f7fe700 1 --2- 172.21.15.19:0/1592342944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a80107df0 0x7f5a8019c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a7f7fe700 1 -- 172.21.15.19:0/1592342944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a80199b40 con 0x7f5a80108740 2024-05-09T02:09:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a7effd700 1 --2- 172.21.15.19:0/1592342944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a801034c0 0x7f5a8019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:09:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.295+0000 7f5a7f7fe700 1 --2- 172.21.15.19:0/1592342944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a80108740 0x7f5a8019c770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000e940 tx=0x7f5a7000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.296+0000 7f5a849e4700 1 -- 172.21.15.19:0/1592342944 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000cbd0 con 0x7f5a80108740 2024-05-09T02:09:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.296+0000 7f5a859e6700 1 -- 172.21.15.19:0/1592342944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a80199e30 con 0x7f5a80108740 2024-05-09T02:09:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.296+0000 7f5a849e4700 1 -- 172.21.15.19:0/1592342944 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a7000ee90 con 0x7f5a80108740 2024-05-09T02:09:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.296+0000 7f5a849e4700 1 -- 172.21.15.19:0/1592342944 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70005590 con 0x7f5a80108740 2024-05-09T02:09:32.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.296+0000 7f5a859e6700 1 -- 172.21.15.19:0/1592342944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a8019a320 con 0x7f5a80108740 2024-05-09T02:09:32.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.297+0000 7f5a859e6700 1 -- 172.21.15.19:0/1592342944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a800512c0 con 0x7f5a80108740 2024-05-09T02:09:32.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.298+0000 7f5a849e4700 1 -- 172.21.15.19:0/1592342944 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5a70004750 con 0x7f5a80108740 2024-05-09T02:09:32.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.299+0000 7f5a849e4700 1 --2- 172.21.15.19:0/1592342944 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5a6c0b2910 0x7f5a6c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:32.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.299+0000 7f5a7effd700 1 --2- 172.21.15.19:0/1592342944 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5a6c0b2910 0x7f5a6c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:32.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.299+0000 7f5a849e4700 1 -- 172.21.15.19:0/1592342944 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f5a700e2830 con 0x7f5a80108740 2024-05-09T02:09:32.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.300+0000 7f5a7effd700 1 --2- 172.21.15.19:0/1592342944 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5a6c0b2910 0x7f5a6c0b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009fd0 tx=0x7f5a7800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.303+0000 7f5a849e4700 1 -- 172.21.15.19:0/1592342944 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5a70054df0 con 0x7f5a80108740 2024-05-09T02:09:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.521+0000 7f5a859e6700 1 -- 172.21.15.19:0/1592342944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5a8006bc00 con 0x7f5a80108740 2024-05-09T02:09:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.522+0000 7f5a849e4700 1 -- 172.21.15.19:0/1592342944 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5a700aa3d0 con 0x7f5a80108740 2024-05-09T02:09:32.523 INFO:teuthology.orchestra.run.smithi019.stdout:77309411412 2024-05-09T02:09:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.524+0000 7f5a859e6700 1 -- 172.21.15.19:0/1592342944 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5a6c0b2910 msgr2=0x7f5a6c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.524+0000 7f5a859e6700 1 --2- 172.21.15.19:0/1592342944 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5a6c0b2910 0x7f5a6c0b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009fd0 tx=0x7f5a7800b040 comp rx=0 tx=0).stop 2024-05-09T02:09:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.524+0000 7f5a859e6700 1 -- 172.21.15.19:0/1592342944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a80108740 msgr2=0x7f5a8019c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.524+0000 7f5a859e6700 1 --2- 172.21.15.19:0/1592342944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a80108740 0x7f5a8019c770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000e940 tx=0x7f5a7000ec50 comp rx=0 tx=0).stop 2024-05-09T02:09:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.524+0000 7f5a859e6700 1 -- 172.21.15.19:0/1592342944 shutdown_connections 2024-05-09T02:09:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.524+0000 7f5a859e6700 1 --2- 172.21.15.19:0/1592342944 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5a6c0b2910 0x7f5a6c0b4e20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.524+0000 7f5a859e6700 1 --2- 172.21.15.19:0/1592342944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a80108740 0x7f5a8019c770 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.524+0000 7f5a859e6700 1 --2- 172.21.15.19:0/1592342944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a801034c0 0x7f5a8019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.524+0000 7f5a859e6700 1 --2- 172.21.15.19:0/1592342944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a80107df0 0x7f5a8019c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.524+0000 7f5a859e6700 1 -- 172.21.15.19:0/1592342944 >> 172.21.15.19:0/1592342944 conn(0x7f5a800fedc0 msgr2=0x7f5a801096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.525+0000 7f5a859e6700 1 -- 172.21.15.19:0/1592342944 shutdown_connections 2024-05-09T02:09:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:32.525+0000 7f5a859e6700 1 -- 172.21.15.19:0/1592342944 wait complete. 2024-05-09T02:09:33.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:33 smithi019 ceph-mon[30151]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:33.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:33 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1592342944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T02:09:33.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:33 smithi059 ceph-mon[34781]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:33.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:33 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1592342944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T02:09:33.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:33 smithi196 ceph-mon[34718]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:33.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:33 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1592342944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T02:09:34.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.298+0000 7fc986d63700 1 -- 172.21.15.19:0/1913556428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9801039e0 msgr2=0x7fc980103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:34.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.298+0000 7fc986d63700 1 --2- 172.21.15.19:0/1913556428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9801039e0 0x7fc980103df0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc968009990 tx=0x7fc968009ca0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.298+0000 7fc986d63700 1 -- 172.21.15.19:0/1913556428 shutdown_connections 2024-05-09T02:09:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.298+0000 7fc986d63700 1 --2- 172.21.15.19:0/1913556428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc980104330 0x7fc980110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.298+0000 7fc986d63700 1 --2- 172.21.15.19:0/1913556428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9801092d0 0x7fc9801096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.298+0000 7fc986d63700 1 --2- 172.21.15.19:0/1913556428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9801039e0 0x7fc980103df0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.298+0000 7fc986d63700 1 -- 172.21.15.19:0/1913556428 >> 172.21.15.19:0/1913556428 conn(0x7fc9800fe520 msgr2=0x7fc980100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.298+0000 7fc986d63700 1 -- 172.21.15.19:0/1913556428 shutdown_connections 2024-05-09T02:09:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.298+0000 7fc986d63700 1 -- 172.21.15.19:0/1913556428 wait complete. 2024-05-09T02:09:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.299+0000 7fc986d63700 1 Processor -- start 2024-05-09T02:09:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.299+0000 7fc986d63700 1 -- start start 2024-05-09T02:09:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.299+0000 7fc986d63700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9801039e0 0x7fc98019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.299+0000 7fc986d63700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc980104330 0x7fc98019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.299+0000 7fc986d63700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9801092d0 0x7fc98019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.299+0000 7fc986d63700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9801144a0 con 0x7fc9801092d0 2024-05-09T02:09:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.299+0000 7fc986d63700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc980114320 con 0x7fc980104330 2024-05-09T02:09:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.299+0000 7fc986d63700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc980114620 con 0x7fc9801039e0 2024-05-09T02:09:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.299+0000 7fc985300700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9801092d0 0x7fc98019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:34.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.299+0000 7fc985300700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9801092d0 0x7fc98019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36182/0 (socket says 172.21.15.19:36182) 2024-05-09T02:09:34.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.299+0000 7fc985300700 1 -- 172.21.15.19:0/12151240 learned_addr learned my addr 172.21.15.19:0/12151240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:34.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.299+0000 7fc97ffff700 1 --2- 172.21.15.19:0/12151240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc980104330 0x7fc98019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:34.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.300+0000 7fc984aff700 1 --2- 172.21.15.19:0/12151240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9801039e0 0x7fc98019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:34.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.300+0000 7fc985300700 1 -- 172.21.15.19:0/12151240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9801039e0 msgr2=0x7fc98019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:34.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.300+0000 7fc985300700 1 --2- 172.21.15.19:0/12151240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9801039e0 0x7fc98019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.300+0000 7fc985300700 1 -- 172.21.15.19:0/12151240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc980104330 msgr2=0x7fc98019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:34.009 INFO:teuthology.orchestra.run.smithi019.stdout:111669149775 2024-05-09T02:09:34.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.300+0000 7fc985300700 1 --2- 172.21.15.19:0/12151240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc980104330 0x7fc98019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.300+0000 7fc985300700 1 -- 172.21.15.19:0/12151240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98019fd40 con 0x7fc9801092d0 2024-05-09T02:09:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.300+0000 7fc985300700 1 --2- 172.21.15.19:0/12151240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9801092d0 0x7fc98019f6a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc978007ee0 tx=0x7fc97800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.301+0000 7fc97dffb700 1 -- 172.21.15.19:0/12151240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978004db0 con 0x7fc9801092d0 2024-05-09T02:09:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.301+0000 7fc97dffb700 1 -- 172.21.15.19:0/12151240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc978012550 con 0x7fc9801092d0 2024-05-09T02:09:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.301+0000 7fc986d63700 1 -- 172.21.15.19:0/12151240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9801a0030 con 0x7fc9801092d0 2024-05-09T02:09:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.301+0000 7fc97dffb700 1 -- 172.21.15.19:0/12151240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97801b6e0 con 0x7fc9801092d0 2024-05-09T02:09:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.301+0000 7fc986d63700 1 -- 172.21.15.19:0/12151240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9801109d0 con 0x7fc9801092d0 2024-05-09T02:09:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.302+0000 7fc986d63700 1 -- 172.21.15.19:0/12151240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9801049f0 con 0x7fc9801092d0 2024-05-09T02:09:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.305+0000 7fc97dffb700 1 -- 172.21.15.19:0/12151240 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc978012a20 con 0x7fc9801092d0 2024-05-09T02:09:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.305+0000 7fc97dffb700 1 --2- 172.21.15.19:0/12151240 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc96c0b2990 0x7fc96c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.306+0000 7fc984aff700 1 --2- 172.21.15.19:0/12151240 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc96c0b2990 0x7fc96c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.306+0000 7fc97dffb700 1 -- 172.21.15.19:0/12151240 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc9780e5dc0 con 0x7fc9801092d0 2024-05-09T02:09:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.306+0000 7fc984aff700 1 --2- 172.21.15.19:0/12151240 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc96c0b2990 0x7fc96c0b4ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc970005950 tx=0x7fc9700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.309+0000 7fc97dffb700 1 -- 172.21.15.19:0/12151240 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc9780adb00 con 0x7fc9801092d0 2024-05-09T02:09:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.532+0000 7fc986d63700 1 -- 172.21.15.19:0/12151240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc98019c1d0 con 0x7fc9801092d0 2024-05-09T02:09:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.533+0000 7fc97dffb700 1 -- 172.21.15.19:0/12151240 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc9780ad400 con 0x7fc9801092d0 2024-05-09T02:09:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.535+0000 7fc986d63700 1 -- 172.21.15.19:0/12151240 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc96c0b2990 msgr2=0x7fc96c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.535+0000 7fc986d63700 1 --2- 172.21.15.19:0/12151240 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc96c0b2990 0x7fc96c0b4ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc970005950 tx=0x7fc9700058e0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.535+0000 7fc986d63700 1 -- 172.21.15.19:0/12151240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9801092d0 msgr2=0x7fc98019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.535+0000 7fc986d63700 1 --2- 172.21.15.19:0/12151240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9801092d0 0x7fc98019f6a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc978007ee0 tx=0x7fc97800d3b0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.535+0000 7fc986d63700 1 -- 172.21.15.19:0/12151240 shutdown_connections 2024-05-09T02:09:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.535+0000 7fc986d63700 1 --2- 172.21.15.19:0/12151240 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc96c0b2990 0x7fc96c0b4ea0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.535+0000 7fc986d63700 1 --2- 172.21.15.19:0/12151240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9801092d0 0x7fc98019f6a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.536+0000 7fc986d63700 1 --2- 172.21.15.19:0/12151240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9801039e0 0x7fc98019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.536+0000 7fc986d63700 1 --2- 172.21.15.19:0/12151240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc980104330 0x7fc98019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.536+0000 7fc986d63700 1 -- 172.21.15.19:0/12151240 >> 172.21.15.19:0/12151240 conn(0x7fc9800fe520 msgr2=0x7fc980100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:34.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.536+0000 7fc986d63700 1 -- 172.21.15.19:0/12151240 shutdown_connections 2024-05-09T02:09:34.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.536+0000 7fc986d63700 1 -- 172.21.15.19:0/12151240 wait complete. 2024-05-09T02:09:34.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.419+0000 7fdb811e4700 1 -- 172.21.15.19:0/3938414177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb7c109850 msgr2=0x7fdb7c1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:34.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.419+0000 7fdb811e4700 1 --2- 172.21.15.19:0/3938414177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb7c109850 0x7fdb7c1111b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdb740099c0 tx=0x7fdb74009cd0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.419+0000 7fdb811e4700 1 -- 172.21.15.19:0/3938414177 shutdown_connections 2024-05-09T02:09:34.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.419+0000 7fdb811e4700 1 --2- 172.21.15.19:0/3938414177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb7c109850 0x7fdb7c1111b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.419+0000 7fdb811e4700 1 --2- 172.21.15.19:0/3938414177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c1045d0 0x7fdb7c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.419+0000 7fdb811e4700 1 --2- 172.21.15.19:0/3938414177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c108f00 0x7fdb7c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.419+0000 7fdb811e4700 1 -- 172.21.15.19:0/3938414177 >> 172.21.15.19:0/3938414177 conn(0x7fdb7c0ffea0 msgr2=0x7fdb7c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:34.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.419+0000 7fdb811e4700 1 -- 172.21.15.19:0/3938414177 shutdown_connections 2024-05-09T02:09:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.419+0000 7fdb811e4700 1 -- 172.21.15.19:0/3938414177 wait complete. 2024-05-09T02:09:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.421+0000 7fdb811e4700 1 Processor -- start 2024-05-09T02:09:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.421+0000 7fdb811e4700 1 -- start start 2024-05-09T02:09:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.421+0000 7fdb811e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb7c1045d0 0x7fdb7c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.421+0000 7fdb811e4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c108f00 0x7fdb7c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.421+0000 7fdb811e4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c109850 0x7fdb7c19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.421+0000 7fdb811e4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c114480 con 0x7fdb7c1045d0 2024-05-09T02:09:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.421+0000 7fdb811e4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c114300 con 0x7fdb7c108f00 2024-05-09T02:09:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.421+0000 7fdb811e4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c114600 con 0x7fdb7c109850 2024-05-09T02:09:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.421+0000 7fdb7ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb7c1045d0 0x7fdb7c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.421+0000 7fdb7ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb7c1045d0 0x7fdb7c19ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36204/0 (socket says 172.21.15.19:36204) 2024-05-09T02:09:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.421+0000 7fdb7ad9d700 1 -- 172.21.15.19:0/128605208 learned_addr learned my addr 172.21.15.19:0/128605208 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.421+0000 7fdb7a59c700 1 --2- 172.21.15.19:0/128605208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c108f00 0x7fdb7c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.422+0000 7fdb7b59e700 1 --2- 172.21.15.19:0/128605208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c109850 0x7fdb7c19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.422+0000 7fdb7ad9d700 1 -- 172.21.15.19:0/128605208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c109850 msgr2=0x7fdb7c19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.422+0000 7fdb7ad9d700 1 --2- 172.21.15.19:0/128605208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c109850 0x7fdb7c19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.422+0000 7fdb7ad9d700 1 -- 172.21.15.19:0/128605208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c108f00 msgr2=0x7fdb7c19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.422+0000 7fdb7ad9d700 1 --2- 172.21.15.19:0/128605208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c108f00 0x7fdb7c19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.422+0000 7fdb7ad9d700 1 -- 172.21.15.19:0/128605208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7c19fc20 con 0x7fdb7c1045d0 2024-05-09T02:09:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.422+0000 7fdb7ad9d700 1 --2- 172.21.15.19:0/128605208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb7c1045d0 0x7fdb7c19ad50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0060b0 tx=0x7fdb6c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.423+0000 7fdb63fff700 1 -- 172.21.15.19:0/128605208 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c01a070 con 0x7fdb7c1045d0 2024-05-09T02:09:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.423+0000 7fdb63fff700 1 -- 172.21.15.19:0/128605208 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb6c00de20 con 0x7fdb7c1045d0 2024-05-09T02:09:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.423+0000 7fdb811e4700 1 -- 172.21.15.19:0/128605208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb7c19ff10 con 0x7fdb7c1045d0 2024-05-09T02:09:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.423+0000 7fdb63fff700 1 -- 172.21.15.19:0/128605208 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c014720 con 0x7fdb7c1045d0 2024-05-09T02:09:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.423+0000 7fdb811e4700 1 -- 172.21.15.19:0/128605208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb7c1a7830 con 0x7fdb7c1045d0 2024-05-09T02:09:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.424+0000 7fdb811e4700 1 -- 172.21.15.19:0/128605208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb7c0512c0 con 0x7fdb7c1045d0 2024-05-09T02:09:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.427+0000 7fdb63fff700 1 -- 172.21.15.19:0/128605208 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdb6c01d070 con 0x7fdb7c1045d0 2024-05-09T02:09:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.428+0000 7fdb63fff700 1 --2- 172.21.15.19:0/128605208 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fdb640b2990 0x7fdb640b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.428+0000 7fdb7a59c700 1 --2- 172.21.15.19:0/128605208 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fdb640b2990 0x7fdb640b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.428+0000 7fdb63fff700 1 -- 172.21.15.19:0/128605208 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fdb6c0e2130 con 0x7fdb7c1045d0 2024-05-09T02:09:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.428+0000 7fdb7a59c700 1 --2- 172.21.15.19:0/128605208 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fdb640b2990 0x7fdb640b4ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb70005950 tx=0x7fdb700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.430+0000 7fdb63fff700 1 -- 172.21.15.19:0/128605208 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdb6c010070 con 0x7fdb7c1045d0 2024-05-09T02:09:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.654+0000 7fdb811e4700 1 -- 172.21.15.19:0/128605208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fdb7c02e5f0 con 0x7fdb7c1045d0 2024-05-09T02:09:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.654+0000 7fdb63fff700 1 -- 172.21.15.19:0/128605208 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdb6c0a9cd0 con 0x7fdb7c1045d0 2024-05-09T02:09:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.656+0000 7fdb811e4700 1 -- 172.21.15.19:0/128605208 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fdb640b2990 msgr2=0x7fdb640b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.656+0000 7fdb811e4700 1 --2- 172.21.15.19:0/128605208 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fdb640b2990 0x7fdb640b4ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb70005950 tx=0x7fdb700058e0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.656+0000 7fdb811e4700 1 -- 172.21.15.19:0/128605208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb7c1045d0 msgr2=0x7fdb7c19ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:34.574 INFO:teuthology.orchestra.run.smithi019.stdout:231928234036 2024-05-09T02:09:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.656+0000 7fdb811e4700 1 --2- 172.21.15.19:0/128605208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb7c1045d0 0x7fdb7c19ad50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0060b0 tx=0x7fdb6c007f90 comp rx=0 tx=0).stop 2024-05-09T02:09:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.656+0000 7fdb811e4700 1 -- 172.21.15.19:0/128605208 shutdown_connections 2024-05-09T02:09:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.656+0000 7fdb811e4700 1 --2- 172.21.15.19:0/128605208 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fdb640b2990 0x7fdb640b4ea0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.656+0000 7fdb811e4700 1 --2- 172.21.15.19:0/128605208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c109850 0x7fdb7c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.656+0000 7fdb811e4700 1 --2- 172.21.15.19:0/128605208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb7c1045d0 0x7fdb7c19ad50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.656+0000 7fdb811e4700 1 --2- 172.21.15.19:0/128605208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c108f00 0x7fdb7c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.656+0000 7fdb811e4700 1 -- 172.21.15.19:0/128605208 >> 172.21.15.19:0/128605208 conn(0x7fdb7c0ffea0 msgr2=0x7fdb7c108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.657+0000 7fdb811e4700 1 -- 172.21.15.19:0/128605208 shutdown_connections 2024-05-09T02:09:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:33.657+0000 7fdb811e4700 1 -- 172.21.15.19:0/128605208 wait complete. 2024-05-09T02:09:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:34 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/12151240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T02:09:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:34 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/128605208' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T02:09:34.722 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:34 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/12151240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T02:09:34.722 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:34 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/128605208' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T02:09:34.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:34 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/12151240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T02:09:34.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:34 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/128605208' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T02:09:35.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:35 smithi196 ceph-mon[34718]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T02:09:35.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:35 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/694228315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T02:09:35.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:35 smithi059 ceph-mon[34781]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T02:09:35.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:35 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/694228315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T02:09:35.719 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:35 smithi019 ceph-mon[30151]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T02:09:35.719 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:35 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/694228315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T02:09:35.775 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411406 got 77309411412 for osd.2 2024-05-09T02:09:35.775 DEBUG:teuthology.parallel:result is None 2024-05-09T02:09:36.540 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234031 got 231928234036 for osd.8 2024-05-09T02:09:36.540 DEBUG:teuthology.parallel:result is None 2024-05-09T02:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.097+0000 7fa962e71700 1 -- 172.21.15.19:0/453397680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c1034c0 msgr2=0x7fa95c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.097+0000 7fa962e71700 1 --2- 172.21.15.19:0/453397680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c1034c0 0x7fa95c1038d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa9500099c0 tx=0x7fa950009cd0 comp rx=0 tx=0).stop 2024-05-09T02:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.097+0000 7fa962e71700 1 -- 172.21.15.19:0/453397680 shutdown_connections 2024-05-09T02:09:36.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.097+0000 7fa962e71700 1 --2- 172.21.15.19:0/453397680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95c108740 0x7fa95c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.097+0000 7fa962e71700 1 --2- 172.21.15.19:0/453397680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c1034c0 0x7fa95c1038d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.097+0000 7fa962e71700 1 --2- 172.21.15.19:0/453397680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa95c107df0 0x7fa95c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.097+0000 7fa962e71700 1 -- 172.21.15.19:0/453397680 >> 172.21.15.19:0/453397680 conn(0x7fa95c0fedc0 msgr2=0x7fa95c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.097+0000 7fa962e71700 1 -- 172.21.15.19:0/453397680 shutdown_connections 2024-05-09T02:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.097+0000 7fa962e71700 1 -- 172.21.15.19:0/453397680 wait complete. 2024-05-09T02:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.098+0000 7fa962e71700 1 Processor -- start 2024-05-09T02:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.098+0000 7fa962e71700 1 -- start start 2024-05-09T02:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.098+0000 7fa962e71700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa95c1034c0 0x7fa95c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.098+0000 7fa962e71700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c107df0 0x7fa95c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.098+0000 7fa962e71700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95c108740 0x7fa95c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.098+0000 7fa962e71700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c1133f0 con 0x7fa95c107df0 2024-05-09T02:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.098+0000 7fa962e71700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c113270 con 0x7fa95c108740 2024-05-09T02:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.098+0000 7fa962e71700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c113570 con 0x7fa95c1034c0 2024-05-09T02:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.098+0000 7fa95bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c107df0 0x7fa95c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.098+0000 7fa95bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c107df0 0x7fa95c19b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36224/0 (socket says 172.21.15.19:36224) 2024-05-09T02:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.098+0000 7fa95bfff700 1 -- 172.21.15.19:0/694228315 learned_addr learned my addr 172.21.15.19:0/694228315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.099+0000 7fa960c0d700 1 --2- 172.21.15.19:0/694228315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa95c1034c0 0x7fa95c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.099+0000 7fa96140e700 1 --2- 172.21.15.19:0/694228315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95c108740 0x7fa95c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.099+0000 7fa95bfff700 1 -- 172.21.15.19:0/694228315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa95c1034c0 msgr2=0x7fa95c19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.099+0000 7fa95bfff700 1 --2- 172.21.15.19:0/694228315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa95c1034c0 0x7fa95c19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.099+0000 7fa95bfff700 1 -- 172.21.15.19:0/694228315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95c108740 msgr2=0x7fa95c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.099+0000 7fa95bfff700 1 --2- 172.21.15.19:0/694228315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95c108740 0x7fa95c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.099+0000 7fa95bfff700 1 -- 172.21.15.19:0/694228315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95c19fd20 con 0x7fa95c107df0 2024-05-09T02:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.099+0000 7fa960c0d700 1 --2- 172.21.15.19:0/694228315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa95c1034c0 0x7fa95c19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.099+0000 7fa95bfff700 1 --2- 172.21.15.19:0/694228315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c107df0 0x7fa95c19b3f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa94800d980 tx=0x7fa94800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:36.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.099+0000 7fa959ffb700 1 -- 172.21.15.19:0/694228315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948017070 con 0x7fa95c107df0 2024-05-09T02:09:36.615 INFO:teuthology.orchestra.run.smithi019.stdout:184683593791 2024-05-09T02:09:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.099+0000 7fa962e71700 1 -- 172.21.15.19:0/694228315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95c1a0010 con 0x7fa95c107df0 2024-05-09T02:09:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.099+0000 7fa959ffb700 1 -- 172.21.15.19:0/694228315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa948010460 con 0x7fa95c107df0 2024-05-09T02:09:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.099+0000 7fa959ffb700 1 -- 172.21.15.19:0/694228315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94800de60 con 0x7fa95c107df0 2024-05-09T02:09:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.099+0000 7fa962e71700 1 -- 172.21.15.19:0/694228315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa95c1a7930 con 0x7fa95c107df0 2024-05-09T02:09:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.100+0000 7fa962e71700 1 -- 172.21.15.19:0/694228315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa95c0512c0 con 0x7fa95c107df0 2024-05-09T02:09:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.102+0000 7fa959ffb700 1 -- 172.21.15.19:0/694228315 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa948010610 con 0x7fa95c107df0 2024-05-09T02:09:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.103+0000 7fa959ffb700 1 --2- 172.21.15.19:0/694228315 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa94c0b2910 0x7fa94c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.103+0000 7fa960c0d700 1 --2- 172.21.15.19:0/694228315 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa94c0b2910 0x7fa94c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.103+0000 7fa959ffb700 1 -- 172.21.15.19:0/694228315 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fa9480e1f80 con 0x7fa95c107df0 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.103+0000 7fa960c0d700 1 --2- 172.21.15.19:0/694228315 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa94c0b2910 0x7fa94c0b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa950000c00 tx=0x7fa95001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.108+0000 7fa959ffb700 1 -- 172.21.15.19:0/694228315 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9480a9b20 con 0x7fa95c107df0 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.324+0000 7fa962e71700 1 -- 172.21.15.19:0/694228315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa95c02e5f0 con 0x7fa95c107df0 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.325+0000 7fa959ffb700 1 -- 172.21.15.19:0/694228315 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa9480a9420 con 0x7fa95c107df0 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.327+0000 7fa962e71700 1 -- 172.21.15.19:0/694228315 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa94c0b2910 msgr2=0x7fa94c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.327+0000 7fa962e71700 1 --2- 172.21.15.19:0/694228315 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa94c0b2910 0x7fa94c0b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa950000c00 tx=0x7fa95001a040 comp rx=0 tx=0).stop 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.327+0000 7fa962e71700 1 -- 172.21.15.19:0/694228315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c107df0 msgr2=0x7fa95c19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.327+0000 7fa962e71700 1 --2- 172.21.15.19:0/694228315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c107df0 0x7fa95c19b3f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa94800d980 tx=0x7fa94800dd40 comp rx=0 tx=0).stop 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.327+0000 7fa962e71700 1 -- 172.21.15.19:0/694228315 shutdown_connections 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.327+0000 7fa962e71700 1 --2- 172.21.15.19:0/694228315 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fa94c0b2910 0x7fa94c0b4e20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.327+0000 7fa962e71700 1 --2- 172.21.15.19:0/694228315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95c108740 0x7fa95c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.327+0000 7fa962e71700 1 --2- 172.21.15.19:0/694228315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa95c1034c0 0x7fa95c19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.327+0000 7fa962e71700 1 --2- 172.21.15.19:0/694228315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c107df0 0x7fa95c19b3f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.327+0000 7fa962e71700 1 -- 172.21.15.19:0/694228315 >> 172.21.15.19:0/694228315 conn(0x7fa95c0fedc0 msgr2=0x7fa95c101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.327+0000 7fa962e71700 1 -- 172.21.15.19:0/694228315 shutdown_connections 2024-05-09T02:09:36.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:35.327+0000 7fa962e71700 1 -- 172.21.15.19:0/694228315 wait complete. 2024-05-09T02:09:37.227 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593786 got 184683593791 for osd.6 2024-05-09T02:09:37.227 DEBUG:teuthology.parallel:result is None 2024-05-09T02:09:37.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:37 smithi059 ceph-mon[34781]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:09:37.772 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:37 smithi019 ceph-mon[30151]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:09:37.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:37 smithi196 ceph-mon[34718]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:09:37.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.893+0000 7f10ff293700 1 -- 172.21.15.19:0/825512884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f8107e10 msgr2=0x7f10f8108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:37.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.893+0000 7f10ff293700 1 --2- 172.21.15.19:0/825512884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f8107e10 0x7f10f8108220 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0099c0 tx=0x7f10ec009cd0 comp rx=0 tx=0).stop 2024-05-09T02:09:37.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.893+0000 7f10ff293700 1 -- 172.21.15.19:0/825512884 shutdown_connections 2024-05-09T02:09:37.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.893+0000 7f10ff293700 1 --2- 172.21.15.19:0/825512884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10f8108760 0x7f10f81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:37.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.893+0000 7f10ff293700 1 --2- 172.21.15.19:0/825512884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f81034e0 0x7f10f81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:37.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.894+0000 7f10ff293700 1 --2- 172.21.15.19:0/825512884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f8107e10 0x7f10f8108220 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:37.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.894+0000 7f10ff293700 1 -- 172.21.15.19:0/825512884 >> 172.21.15.19:0/825512884 conn(0x7f10f80fef30 msgr2=0x7f10f8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:37.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.894+0000 7f10ff293700 1 -- 172.21.15.19:0/825512884 shutdown_connections 2024-05-09T02:09:37.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.894+0000 7f10ff293700 1 -- 172.21.15.19:0/825512884 wait complete. 2024-05-09T02:09:37.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.894+0000 7f10ff293700 1 Processor -- start 2024-05-09T02:09:37.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.894+0000 7f10ff293700 1 -- start start 2024-05-09T02:09:37.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10ff293700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f81034e0 0x7f10f819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:37.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10ff293700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f8107e10 0x7f10f819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:37.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10ff293700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10f8108760 0x7f10f819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:37.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10ff293700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10f8113460 con 0x7f10f81034e0 2024-05-09T02:09:37.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10ff293700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81132e0 con 0x7f10f8108760 2024-05-09T02:09:37.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10ff293700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81135e0 con 0x7f10f8107e10 2024-05-09T02:09:37.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10fd02f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f81034e0 0x7f10f819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:37.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10fd02f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f81034e0 0x7f10f819af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43220/0 (socket says 172.21.15.19:43220) 2024-05-09T02:09:37.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10fd02f700 1 -- 172.21.15.19:0/4011718292 learned_addr learned my addr 172.21.15.19:0/4011718292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:37.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10fc82e700 1 --2- 172.21.15.19:0/4011718292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f8107e10 0x7f10f819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:37.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10fd830700 1 --2- 172.21.15.19:0/4011718292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10f8108760 0x7f10f819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:37.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10fd02f700 1 -- 172.21.15.19:0/4011718292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f8107e10 msgr2=0x7f10f819b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:37.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10fd02f700 1 --2- 172.21.15.19:0/4011718292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f8107e10 0x7f10f819b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:37.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10fd02f700 1 -- 172.21.15.19:0/4011718292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10f8108760 msgr2=0x7f10f819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:37.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10fd02f700 1 --2- 172.21.15.19:0/4011718292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10f8108760 0x7f10f819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:37.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10fd02f700 1 -- 172.21.15.19:0/4011718292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f819fe10 con 0x7f10f81034e0 2024-05-09T02:09:37.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10fc82e700 1 --2- 172.21.15.19:0/4011718292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f8107e10 0x7f10f819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:09:37.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.895+0000 7f10fd02f700 1 --2- 172.21.15.19:0/4011718292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f81034e0 0x7f10f819af40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f10e800ba50 tx=0x7f10e800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:37.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.896+0000 7f10f67fc700 1 -- 172.21.15.19:0/4011718292 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e8019070 con 0x7f10f81034e0 2024-05-09T02:09:37.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.896+0000 7f10ff293700 1 -- 172.21.15.19:0/4011718292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f81a0100 con 0x7f10f81034e0 2024-05-09T02:09:37.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.896+0000 7f10f67fc700 1 -- 172.21.15.19:0/4011718292 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10e8014440 con 0x7f10f81034e0 2024-05-09T02:09:37.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.896+0000 7f10f67fc700 1 -- 172.21.15.19:0/4011718292 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e80075d0 con 0x7f10f81034e0 2024-05-09T02:09:37.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.896+0000 7f10ff293700 1 -- 172.21.15.19:0/4011718292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f81a7a20 con 0x7f10f81034e0 2024-05-09T02:09:37.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.897+0000 7f10ff293700 1 -- 172.21.15.19:0/4011718292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f80512c0 con 0x7f10f81034e0 2024-05-09T02:09:37.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.898+0000 7f10f67fc700 1 -- 172.21.15.19:0/4011718292 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f10e801c070 con 0x7f10f81034e0 2024-05-09T02:09:37.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.899+0000 7f10f67fc700 1 --2- 172.21.15.19:0/4011718292 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f10e00b2910 0x7f10e00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:37.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.900+0000 7f10fc82e700 1 --2- 172.21.15.19:0/4011718292 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f10e00b2910 0x7f10e00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:37.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.900+0000 7f10f67fc700 1 -- 172.21.15.19:0/4011718292 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f10e80e1c80 con 0x7f10f81034e0 2024-05-09T02:09:37.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.900+0000 7f10fc82e700 1 --2- 172.21.15.19:0/4011718292 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f10e00b2910 0x7f10e00b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f10ec000c00 tx=0x7f10ec01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:37.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:37.903+0000 7f10f67fc700 1 -- 172.21.15.19:0/4011718292 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10e80a9930 con 0x7f10f81034e0 2024-05-09T02:09:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:38.120+0000 7f10ff293700 1 -- 172.21.15.19:0/4011718292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f10f819c2a0 con 0x7f10f81034e0 2024-05-09T02:09:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:38.120+0000 7f10f67fc700 1 -- 172.21.15.19:0/4011718292 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f10e80a9230 con 0x7f10f81034e0 2024-05-09T02:09:38.123 INFO:teuthology.orchestra.run.smithi019.stdout:34359738462 2024-05-09T02:09:38.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:38.122+0000 7f10ff293700 1 -- 172.21.15.19:0/4011718292 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f10e00b2910 msgr2=0x7f10e00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:38.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:38.122+0000 7f10ff293700 1 --2- 172.21.15.19:0/4011718292 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f10e00b2910 0x7f10e00b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f10ec000c00 tx=0x7f10ec01a040 comp rx=0 tx=0).stop 2024-05-09T02:09:38.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:38.122+0000 7f10ff293700 1 -- 172.21.15.19:0/4011718292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f81034e0 msgr2=0x7f10f819af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:38.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:38.122+0000 7f10ff293700 1 --2- 172.21.15.19:0/4011718292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f81034e0 0x7f10f819af40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f10e800ba50 tx=0x7f10e800be10 comp rx=0 tx=0).stop 2024-05-09T02:09:38.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:38.123+0000 7f10ff293700 1 -- 172.21.15.19:0/4011718292 shutdown_connections 2024-05-09T02:09:38.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:38.123+0000 7f10ff293700 1 --2- 172.21.15.19:0/4011718292 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f10e00b2910 0x7f10e00b4e20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:38.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:38.123+0000 7f10ff293700 1 --2- 172.21.15.19:0/4011718292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10f8108760 0x7f10f819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:38.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:38.123+0000 7f10ff293700 1 --2- 172.21.15.19:0/4011718292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f81034e0 0x7f10f819af40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:38.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:38.123+0000 7f10ff293700 1 --2- 172.21.15.19:0/4011718292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f8107e10 0x7f10f819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:38.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:38.123+0000 7f10ff293700 1 -- 172.21.15.19:0/4011718292 >> 172.21.15.19:0/4011718292 conn(0x7f10f80fef30 msgr2=0x7f10f806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:38.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:38.123+0000 7f10ff293700 1 -- 172.21.15.19:0/4011718292 shutdown_connections 2024-05-09T02:09:38.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:38.123+0000 7f10ff293700 1 -- 172.21.15.19:0/4011718292 wait complete. 2024-05-09T02:09:38.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:38 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/4011718292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T02:09:38.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:38 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/4011718292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T02:09:38.724 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149770 got 111669149775 for osd.3 2024-05-09T02:09:38.724 DEBUG:teuthology.parallel:result is None 2024-05-09T02:09:38.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:38 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/4011718292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T02:09:39.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:39 smithi059 ceph-mon[34781]: pgmap v348: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:39.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:39 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2879832258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T02:09:39.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:39 smithi196 ceph-mon[34718]: pgmap v348: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:39.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:39 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2879832258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T02:09:39.807 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:39 smithi019 ceph-mon[30151]: pgmap v348: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:39.807 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:39 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2879832258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T02:09:40.129 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738456 got 34359738462 for osd.0 2024-05-09T02:09:40.129 DEBUG:teuthology.parallel:result is None 2024-05-09T02:09:40.895 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:40.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.899+0000 7f36f619d700 1 -- 172.21.15.19:0/1734195303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f0108740 msgr2=0x7f36f01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:40.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.899+0000 7f36f619d700 1 --2- 172.21.15.19:0/1734195303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f0108740 0x7f36f01100a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b0d0 tx=0x7f36e800b3e0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.899+0000 7f36f619d700 1 -- 172.21.15.19:0/1734195303 shutdown_connections 2024-05-09T02:09:40.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.899+0000 7f36f619d700 1 --2- 172.21.15.19:0/1734195303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f0108740 0x7f36f01100a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.899+0000 7f36f619d700 1 --2- 172.21.15.19:0/1734195303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f01034c0 0x7f36f01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.899+0000 7f36f619d700 1 --2- 172.21.15.19:0/1734195303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f0107df0 0x7f36f0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.899+0000 7f36f619d700 1 -- 172.21.15.19:0/1734195303 >> 172.21.15.19:0/1734195303 conn(0x7f36f00fedc0 msgr2=0x7f36f01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:40.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.899+0000 7f36f619d700 1 -- 172.21.15.19:0/1734195303 shutdown_connections 2024-05-09T02:09:40.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.899+0000 7f36f619d700 1 -- 172.21.15.19:0/1734195303 wait complete. 2024-05-09T02:09:40.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.899+0000 7f36f619d700 1 Processor -- start 2024-05-09T02:09:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.900+0000 7f36f619d700 1 -- start start 2024-05-09T02:09:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.900+0000 7f36f619d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f01034c0 0x7f36f00ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.900+0000 7f36f619d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f0107df0 0x7f36f00ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.900+0000 7f36f619d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f0108740 0x7f36f00ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.900+0000 7f36f619d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01134b0 con 0x7f36f0107df0 2024-05-09T02:09:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.900+0000 7f36f619d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0113330 con 0x7f36f01034c0 2024-05-09T02:09:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.900+0000 7f36f619d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0113630 con 0x7f36f0108740 2024-05-09T02:09:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.900+0000 7f36e6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f0107df0 0x7f36f00ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.900+0000 7f36e6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f0107df0 0x7f36f00ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43260/0 (socket says 172.21.15.19:43260) 2024-05-09T02:09:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.900+0000 7f36e6ffd700 1 -- 172.21.15.19:0/2397397167 learned_addr learned my addr 172.21.15.19:0/2397397167 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.900+0000 7f36effff700 1 --2- 172.21.15.19:0/2397397167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f0108740 0x7f36f00ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.900+0000 7f36ef7fe700 1 --2- 172.21.15.19:0/2397397167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f01034c0 0x7f36f00ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.901+0000 7f36e6ffd700 1 -- 172.21.15.19:0/2397397167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f0108740 msgr2=0x7f36f00ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.901+0000 7f36e6ffd700 1 --2- 172.21.15.19:0/2397397167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f0108740 0x7f36f00ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.901+0000 7f36e6ffd700 1 -- 172.21.15.19:0/2397397167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f01034c0 msgr2=0x7f36f00ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:40.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.901+0000 7f36e6ffd700 1 --2- 172.21.15.19:0/2397397167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f01034c0 0x7f36f00ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.901+0000 7f36e6ffd700 1 -- 172.21.15.19:0/2397397167 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f01a92f0 con 0x7f36f0107df0 2024-05-09T02:09:40.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.901+0000 7f36e6ffd700 1 --2- 172.21.15.19:0/2397397167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f0107df0 0x7f36f00ff7e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f36d800b500 tx=0x7f36d800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:40.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.901+0000 7f36ed7fa700 1 -- 172.21.15.19:0/2397397167 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8015020 con 0x7f36f0107df0 2024-05-09T02:09:40.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.901+0000 7f36ed7fa700 1 -- 172.21.15.19:0/2397397167 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d8011ce0 con 0x7f36f0107df0 2024-05-09T02:09:40.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.901+0000 7f36f619d700 1 -- 172.21.15.19:0/2397397167 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f01a95e0 con 0x7f36f0107df0 2024-05-09T02:09:40.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.901+0000 7f36ed7fa700 1 -- 172.21.15.19:0/2397397167 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8010cb0 con 0x7f36f0107df0 2024-05-09T02:09:40.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.901+0000 7f36f619d700 1 -- 172.21.15.19:0/2397397167 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f01a9b20 con 0x7f36f0107df0 2024-05-09T02:09:40.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.902+0000 7f36f619d700 1 -- 172.21.15.19:0/2397397167 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f00512c0 con 0x7f36f0107df0 2024-05-09T02:09:40.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.905+0000 7f36ed7fa700 1 -- 172.21.15.19:0/2397397167 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f36d8018020 con 0x7f36f0107df0 2024-05-09T02:09:40.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.906+0000 7f36ed7fa700 1 --2- 172.21.15.19:0/2397397167 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f36dc0b2910 0x7f36dc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:40.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.906+0000 7f36ef7fe700 1 --2- 172.21.15.19:0/2397397167 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f36dc0b2910 0x7f36dc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:40.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.906+0000 7f36ed7fa700 1 -- 172.21.15.19:0/2397397167 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f36d80e1e90 con 0x7f36f0107df0 2024-05-09T02:09:40.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.906+0000 7f36ef7fe700 1 --2- 172.21.15.19:0/2397397167 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f36dc0b2910 0x7f36dc0b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009a10 tx=0x7f36e0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:40.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:40.910+0000 7f36ed7fa700 1 -- 172.21.15.19:0/2397397167 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36d801f020 con 0x7f36f0107df0 2024-05-09T02:09:40.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.174+0000 7f9091a8e700 1 -- 172.21.15.19:0/3172020496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f908c105e30 msgr2=0x7f908c10e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:40.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.174+0000 7f9091a8e700 1 --2- 172.21.15.19:0/3172020496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f908c105e30 0x7f908c10e380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9080009990 tx=0x7f9080009ca0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.174+0000 7f9091a8e700 1 -- 172.21.15.19:0/3172020496 shutdown_connections 2024-05-09T02:09:40.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.174+0000 7f9091a8e700 1 --2- 172.21.15.19:0/3172020496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f908c10e910 0x7f908c110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.174+0000 7f9091a8e700 1 --2- 172.21.15.19:0/3172020496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f908c1054e0 0x7f908c1058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.174+0000 7f9091a8e700 1 --2- 172.21.15.19:0/3172020496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f908c105e30 0x7f908c10e380 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.174+0000 7f9091a8e700 1 -- 172.21.15.19:0/3172020496 >> 172.21.15.19:0/3172020496 conn(0x7f908c0fef30 msgr2=0x7f908c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:40.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.174+0000 7f9091a8e700 1 -- 172.21.15.19:0/3172020496 shutdown_connections 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.174+0000 7f9091a8e700 1 -- 172.21.15.19:0/3172020496 wait complete. 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.175+0000 7f9091a8e700 1 Processor -- start 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.175+0000 7f9091a8e700 1 -- start start 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.175+0000 7f9091a8e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f908c1054e0 0x7f908c19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.175+0000 7f9091a8e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f908c105e30 0x7f908c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.175+0000 7f9091a8e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f908c10e910 0x7f908c19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.175+0000 7f9091a8e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f908c114470 con 0x7f908c105e30 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.175+0000 7f9091a8e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f908c1142f0 con 0x7f908c1054e0 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.175+0000 7f9091a8e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f908c1145f0 con 0x7f908c10e910 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.175+0000 7f908affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f908c105e30 0x7f908c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.175+0000 7f908affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f908c105e30 0x7f908c19b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43250/0 (socket says 172.21.15.19:43250) 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.175+0000 7f908affd700 1 -- 172.21.15.19:0/2879832258 learned_addr learned my addr 172.21.15.19:0/2879832258 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.175+0000 7f908bfff700 1 --2- 172.21.15.19:0/2879832258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f908c10e910 0x7f908c19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.175+0000 7f908b7fe700 1 --2- 172.21.15.19:0/2879832258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f908c1054e0 0x7f908c19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.176+0000 7f908affd700 1 -- 172.21.15.19:0/2879832258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f908c10e910 msgr2=0x7f908c19f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.176+0000 7f908affd700 1 --2- 172.21.15.19:0/2879832258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f908c10e910 0x7f908c19f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.176+0000 7f908affd700 1 -- 172.21.15.19:0/2879832258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f908c1054e0 msgr2=0x7f908c19ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:40.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.176+0000 7f908affd700 1 --2- 172.21.15.19:0/2879832258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f908c1054e0 0x7f908c19ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.176+0000 7f908affd700 1 -- 172.21.15.19:0/2879832258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c19fd30 con 0x7f908c105e30 2024-05-09T02:09:40.985 INFO:teuthology.orchestra.run.smithi019.stdout:55834574938 2024-05-09T02:09:40.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.176+0000 7f908affd700 1 --2- 172.21.15.19:0/2879832258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f908c105e30 0x7f908c19b400 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9080000c00 tx=0x7f90800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:40.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.176+0000 7f9088ff9700 1 -- 172.21.15.19:0/2879832258 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080027070 con 0x7f908c105e30 2024-05-09T02:09:40.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.176+0000 7f9091a8e700 1 -- 172.21.15.19:0/2879832258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908c19ffc0 con 0x7f908c105e30 2024-05-09T02:09:40.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.176+0000 7f9088ff9700 1 -- 172.21.15.19:0/2879832258 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9080003f20 con 0x7f908c105e30 2024-05-09T02:09:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.176+0000 7f9088ff9700 1 -- 172.21.15.19:0/2879832258 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080022cf0 con 0x7f908c105e30 2024-05-09T02:09:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.176+0000 7f9091a8e700 1 -- 172.21.15.19:0/2879832258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f908c1a04a0 con 0x7f908c105e30 2024-05-09T02:09:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.177+0000 7f9091a8e700 1 -- 172.21.15.19:0/2879832258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f908c0512c0 con 0x7f908c105e30 2024-05-09T02:09:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.179+0000 7f9088ff9700 1 -- 172.21.15.19:0/2879832258 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f908002a070 con 0x7f908c105e30 2024-05-09T02:09:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.180+0000 7f9088ff9700 1 --2- 172.21.15.19:0/2879832258 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f90740b2910 0x7f90740b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.180+0000 7f908b7fe700 1 --2- 172.21.15.19:0/2879832258 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f90740b2910 0x7f90740b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.180+0000 7f9088ff9700 1 -- 172.21.15.19:0/2879832258 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f90800e3610 con 0x7f908c105e30 2024-05-09T02:09:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.181+0000 7f908b7fe700 1 --2- 172.21.15.19:0/2879832258 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f90740b2910 0x7f90740b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f907c005950 tx=0x7f907c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.184+0000 7f9088ff9700 1 -- 172.21.15.19:0/2879832258 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9080013070 con 0x7f908c105e30 2024-05-09T02:09:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.403+0000 7f9091a8e700 1 -- 172.21.15.19:0/2879832258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f908c02e5f0 con 0x7f908c105e30 2024-05-09T02:09:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.403+0000 7f9088ff9700 1 -- 172.21.15.19:0/2879832258 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f90800ab1b0 con 0x7f908c105e30 2024-05-09T02:09:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.406+0000 7f9091a8e700 1 -- 172.21.15.19:0/2879832258 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f90740b2910 msgr2=0x7f90740b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.406+0000 7f9091a8e700 1 --2- 172.21.15.19:0/2879832258 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f90740b2910 0x7f90740b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f907c005950 tx=0x7f907c0058e0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.406+0000 7f9091a8e700 1 -- 172.21.15.19:0/2879832258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f908c105e30 msgr2=0x7f908c19b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.406+0000 7f9091a8e700 1 --2- 172.21.15.19:0/2879832258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f908c105e30 0x7f908c19b400 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9080000c00 tx=0x7f90800047b0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.406+0000 7f9091a8e700 1 -- 172.21.15.19:0/2879832258 shutdown_connections 2024-05-09T02:09:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.406+0000 7f9091a8e700 1 --2- 172.21.15.19:0/2879832258 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f90740b2910 0x7f90740b4e20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.406+0000 7f9091a8e700 1 --2- 172.21.15.19:0/2879832258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f908c10e910 0x7f908c19f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.406+0000 7f9091a8e700 1 --2- 172.21.15.19:0/2879832258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f908c1054e0 0x7f908c19ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.406+0000 7f9091a8e700 1 --2- 172.21.15.19:0/2879832258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f908c105e30 0x7f908c19b400 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.406+0000 7f9091a8e700 1 -- 172.21.15.19:0/2879832258 >> 172.21.15.19:0/2879832258 conn(0x7f908c0fef30 msgr2=0x7f908c0ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.406+0000 7f9091a8e700 1 -- 172.21.15.19:0/2879832258 shutdown_connections 2024-05-09T02:09:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:39.406+0000 7f9091a8e700 1 -- 172.21.15.19:0/2879832258 wait complete. 2024-05-09T02:09:41.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.141+0000 7f36f619d700 1 -- 172.21.15.19:0/2397397167 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f36f002e5f0 con 0x7f36f0107df0 2024-05-09T02:09:41.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.141+0000 7f36ed7fa700 1 -- 172.21.15.19:0/2397397167 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f36d80a9b40 con 0x7f36f0107df0 2024-05-09T02:09:41.144 INFO:teuthology.orchestra.run.smithi019.stdout:206158430267 2024-05-09T02:09:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.143+0000 7f36f619d700 1 -- 172.21.15.19:0/2397397167 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f36dc0b2910 msgr2=0x7f36dc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.143+0000 7f36f619d700 1 --2- 172.21.15.19:0/2397397167 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f36dc0b2910 0x7f36dc0b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009a10 tx=0x7f36e0009190 comp rx=0 tx=0).stop 2024-05-09T02:09:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.143+0000 7f36f619d700 1 -- 172.21.15.19:0/2397397167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f0107df0 msgr2=0x7f36f00ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.144+0000 7f36f619d700 1 --2- 172.21.15.19:0/2397397167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f0107df0 0x7f36f00ff7e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f36d800b500 tx=0x7f36d800b810 comp rx=0 tx=0).stop 2024-05-09T02:09:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.144+0000 7f36f619d700 1 -- 172.21.15.19:0/2397397167 shutdown_connections 2024-05-09T02:09:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.144+0000 7f36f619d700 1 --2- 172.21.15.19:0/2397397167 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f36dc0b2910 0x7f36dc0b4e20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:41.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.144+0000 7f36f619d700 1 --2- 172.21.15.19:0/2397397167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f0108740 0x7f36f00ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:41.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.144+0000 7f36f619d700 1 --2- 172.21.15.19:0/2397397167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f01034c0 0x7f36f00ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:41.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.144+0000 7f36f619d700 1 --2- 172.21.15.19:0/2397397167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f0107df0 0x7f36f00ff7e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:41.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.144+0000 7f36f619d700 1 -- 172.21.15.19:0/2397397167 >> 172.21.15.19:0/2397397167 conn(0x7f36f00fedc0 msgr2=0x7f36f006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:41.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.144+0000 7f36f619d700 1 -- 172.21.15.19:0/2397397167 shutdown_connections 2024-05-09T02:09:41.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.144+0000 7f36f619d700 1 -- 172.21.15.19:0/2397397167 wait complete. 2024-05-09T02:09:41.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:41 smithi019 ceph-mon[30151]: pgmap v349: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:09:41.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:41 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2397397167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T02:09:41.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:41 smithi059 ceph-mon[34781]: pgmap v349: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:09:41.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:41 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2397397167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T02:09:41.733 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574934 got 55834574938 for osd.1 2024-05-09T02:09:41.734 DEBUG:teuthology.parallel:result is None 2024-05-09T02:09:41.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:41 smithi196 ceph-mon[34718]: pgmap v349: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:09:41.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:41 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2397397167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T02:09:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.534+0000 7f43f9c49700 1 -- 172.21.15.19:0/4162366983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f41034c0 msgr2=0x7f43f41038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.534+0000 7f43f9c49700 1 --2- 172.21.15.19:0/4162366983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f41034c0 0x7f43f41038d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f43e40099c0 tx=0x7f43e4009cd0 comp rx=0 tx=0).stop 2024-05-09T02:09:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.534+0000 7f43f9c49700 1 -- 172.21.15.19:0/4162366983 shutdown_connections 2024-05-09T02:09:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.534+0000 7f43f9c49700 1 --2- 172.21.15.19:0/4162366983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f4108740 0x7f43f41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.535+0000 7f43f9c49700 1 --2- 172.21.15.19:0/4162366983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f41034c0 0x7f43f41038d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.535+0000 7f43f9c49700 1 --2- 172.21.15.19:0/4162366983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43f4107df0 0x7f43f4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.535+0000 7f43f9c49700 1 -- 172.21.15.19:0/4162366983 >> 172.21.15.19:0/4162366983 conn(0x7f43f40fedf0 msgr2=0x7f43f4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.535+0000 7f43f9c49700 1 -- 172.21.15.19:0/4162366983 shutdown_connections 2024-05-09T02:09:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.535+0000 7f43f9c49700 1 -- 172.21.15.19:0/4162366983 wait complete. 2024-05-09T02:09:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.535+0000 7f43f9c49700 1 Processor -- start 2024-05-09T02:09:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.535+0000 7f43f9c49700 1 -- start start 2024-05-09T02:09:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f9c49700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43f41034c0 0x7f43f419d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f9c49700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f4107df0 0x7f43f419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f9c49700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f4108740 0x7f43f41a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f9c49700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43f41144e0 con 0x7f43f4108740 2024-05-09T02:09:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f9c49700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43f4114360 con 0x7f43f4107df0 2024-05-09T02:09:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f9c49700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43f4114660 con 0x7f43f41034c0 2024-05-09T02:09:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f4108740 0x7f43f41a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f4108740 0x7f43f41a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43272/0 (socket says 172.21.15.19:43272) 2024-05-09T02:09:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f3fff700 1 -- 172.21.15.19:0/512906747 learned_addr learned my addr 172.21.15.19:0/512906747 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:42.246 INFO:teuthology.orchestra.run.smithi019.stdout:158913790022 2024-05-09T02:09:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f37fe700 1 --2- 172.21.15.19:0/512906747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43f41034c0 0x7f43f419d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f2ffd700 1 --2- 172.21.15.19:0/512906747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f4107df0 0x7f43f419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f3fff700 1 -- 172.21.15.19:0/512906747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43f41034c0 msgr2=0x7f43f419d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f3fff700 1 --2- 172.21.15.19:0/512906747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43f41034c0 0x7f43f419d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f3fff700 1 -- 172.21.15.19:0/512906747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f4107df0 msgr2=0x7f43f419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f3fff700 1 --2- 172.21.15.19:0/512906747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f4107df0 0x7f43f419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f3fff700 1 -- 172.21.15.19:0/512906747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f41a1f10 con 0x7f43f4108740 2024-05-09T02:09:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.536+0000 7f43f3fff700 1 --2- 172.21.15.19:0/512906747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f4108740 0x7f43f41a1870 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00b970 tx=0x7f43ec00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.537+0000 7f43f0ff9700 1 -- 172.21.15.19:0/512906747 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec019070 con 0x7f43f4108740 2024-05-09T02:09:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.537+0000 7f43f9c49700 1 -- 172.21.15.19:0/512906747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f41a2200 con 0x7f43f4108740 2024-05-09T02:09:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.537+0000 7f43f0ff9700 1 -- 172.21.15.19:0/512906747 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43ec00cde0 con 0x7f43f4108740 2024-05-09T02:09:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.537+0000 7f43f0ff9700 1 -- 172.21.15.19:0/512906747 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec007370 con 0x7f43f4108740 2024-05-09T02:09:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.537+0000 7f43f9c49700 1 -- 172.21.15.19:0/512906747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f41a9b20 con 0x7f43f4108740 2024-05-09T02:09:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.538+0000 7f43f9c49700 1 -- 172.21.15.19:0/512906747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f40512c0 con 0x7f43f4108740 2024-05-09T02:09:42.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.539+0000 7f43f0ff9700 1 -- 172.21.15.19:0/512906747 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f43ec01c070 con 0x7f43f4108740 2024-05-09T02:09:42.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.540+0000 7f43f0ff9700 1 --2- 172.21.15.19:0/512906747 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f43dc0b2910 0x7f43dc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:42.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.540+0000 7f43f37fe700 1 --2- 172.21.15.19:0/512906747 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f43dc0b2910 0x7f43dc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:42.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.541+0000 7f43f0ff9700 1 -- 172.21.15.19:0/512906747 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f43ec0e1ec0 con 0x7f43f4108740 2024-05-09T02:09:42.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.541+0000 7f43f37fe700 1 --2- 172.21.15.19:0/512906747 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f43dc0b2910 0x7f43dc0b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009990 tx=0x7f43e40036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:42.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.544+0000 7f43f0ff9700 1 -- 172.21.15.19:0/512906747 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f43ec023070 con 0x7f43f4108740 2024-05-09T02:09:42.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.775+0000 7f43f9c49700 1 -- 172.21.15.19:0/512906747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f43f41a9cf0 con 0x7f43f4108740 2024-05-09T02:09:42.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.775+0000 7f43f0ff9700 1 -- 172.21.15.19:0/512906747 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f43ec0a9b70 con 0x7f43f4108740 2024-05-09T02:09:42.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.778+0000 7f43f9c49700 1 -- 172.21.15.19:0/512906747 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f43dc0b2910 msgr2=0x7f43dc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:42.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.778+0000 7f43f9c49700 1 --2- 172.21.15.19:0/512906747 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f43dc0b2910 0x7f43dc0b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009990 tx=0x7f43e40036e0 comp rx=0 tx=0).stop 2024-05-09T02:09:42.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.778+0000 7f43f9c49700 1 -- 172.21.15.19:0/512906747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f4108740 msgr2=0x7f43f41a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:42.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.778+0000 7f43f9c49700 1 --2- 172.21.15.19:0/512906747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f4108740 0x7f43f41a1870 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00b970 tx=0x7f43ec00bc80 comp rx=0 tx=0).stop 2024-05-09T02:09:42.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.778+0000 7f43f9c49700 1 -- 172.21.15.19:0/512906747 shutdown_connections 2024-05-09T02:09:42.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.778+0000 7f43f9c49700 1 --2- 172.21.15.19:0/512906747 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f43dc0b2910 0x7f43dc0b4e20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:42.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.778+0000 7f43f9c49700 1 --2- 172.21.15.19:0/512906747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f4108740 0x7f43f41a1870 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:42.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.778+0000 7f43f9c49700 1 --2- 172.21.15.19:0/512906747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43f41034c0 0x7f43f419d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:42.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.778+0000 7f43f9c49700 1 --2- 172.21.15.19:0/512906747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f4107df0 0x7f43f419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:42.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.778+0000 7f43f9c49700 1 -- 172.21.15.19:0/512906747 >> 172.21.15.19:0/512906747 conn(0x7f43f40fedf0 msgr2=0x7f43f410f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:42.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.778+0000 7f43f9c49700 1 -- 172.21.15.19:0/512906747 shutdown_connections 2024-05-09T02:09:42.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:41.778+0000 7f43f9c49700 1 -- 172.21.15.19:0/512906747 wait complete. 2024-05-09T02:09:42.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:42 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/512906747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T02:09:42.829 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:42 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/512906747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T02:09:42.885 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790017 got 158913790022 for osd.5 2024-05-09T02:09:42.885 DEBUG:teuthology.parallel:result is None 2024-05-09T02:09:42.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:42 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/512906747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T02:09:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.681+0000 7fb043f47700 1 -- 172.21.15.19:0/3878412159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c10e8f0 msgr2=0x7fb03c110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.681+0000 7fb043f47700 1 --2- 172.21.15.19:0/3878412159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c10e8f0 0x7fb03c110dd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb03400b0a0 tx=0x7fb03400b3b0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.681+0000 7fb043f47700 1 -- 172.21.15.19:0/3878412159 shutdown_connections 2024-05-09T02:09:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.681+0000 7fb043f47700 1 --2- 172.21.15.19:0/3878412159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c10e8f0 0x7fb03c110dd0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.681+0000 7fb043f47700 1 --2- 172.21.15.19:0/3878412159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb03c105480 0x7fb03c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.681+0000 7fb043f47700 1 --2- 172.21.15.19:0/3878412159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb03c105dd0 0x7fb03c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.681+0000 7fb043f47700 1 -- 172.21.15.19:0/3878412159 >> 172.21.15.19:0/3878412159 conn(0x7fb03c0fef30 msgr2=0x7fb03c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:43.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.681+0000 7fb043f47700 1 -- 172.21.15.19:0/3878412159 shutdown_connections 2024-05-09T02:09:43.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.681+0000 7fb043f47700 1 -- 172.21.15.19:0/3878412159 wait complete. 2024-05-09T02:09:43.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.682+0000 7fb043f47700 1 Processor -- start 2024-05-09T02:09:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.682+0000 7fb043f47700 1 -- start start 2024-05-09T02:09:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb043f47700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb03c105480 0x7fb03c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb043f47700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb03c105dd0 0x7fb03c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb043f47700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c10e8f0 0x7fb03c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb043f47700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c114450 con 0x7fb03c10e8f0 2024-05-09T02:09:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb043f47700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c1142d0 con 0x7fb03c105480 2024-05-09T02:09:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb043f47700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c1145d0 con 0x7fb03c105dd0 2024-05-09T02:09:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb0424e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c10e8f0 0x7fb03c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb0424e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c10e8f0 0x7fb03c19f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43300/0 (socket says 172.21.15.19:43300) 2024-05-09T02:09:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb0424e4700 1 -- 172.21.15.19:0/1491794397 learned_addr learned my addr 172.21.15.19:0/1491794397 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb041ce3700 1 --2- 172.21.15.19:0/1491794397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb03c105480 0x7fb03c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb0414e2700 1 --2- 172.21.15.19:0/1491794397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb03c105dd0 0x7fb03c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb0424e4700 1 -- 172.21.15.19:0/1491794397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb03c105dd0 msgr2=0x7fb03c19b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb0424e4700 1 --2- 172.21.15.19:0/1491794397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb03c105dd0 0x7fb03c19b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb0424e4700 1 -- 172.21.15.19:0/1491794397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb03c105480 msgr2=0x7fb03c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb0424e4700 1 --2- 172.21.15.19:0/1491794397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb03c105480 0x7fb03c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb0424e4700 1 -- 172.21.15.19:0/1491794397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03c19fc90 con 0x7fb03c10e8f0 2024-05-09T02:09:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.683+0000 7fb0424e4700 1 --2- 172.21.15.19:0/1491794397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c10e8f0 0x7fb03c19f5f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb0340062a0 tx=0x7fb034004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.684+0000 7fb032ffd700 1 -- 172.21.15.19:0/1491794397 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034021070 con 0x7fb03c10e8f0 2024-05-09T02:09:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.684+0000 7fb032ffd700 1 -- 172.21.15.19:0/1491794397 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb034007e30 con 0x7fb03c10e8f0 2024-05-09T02:09:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.684+0000 7fb043f47700 1 -- 172.21.15.19:0/1491794397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03c19ff20 con 0x7fb03c10e8f0 2024-05-09T02:09:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.684+0000 7fb032ffd700 1 -- 172.21.15.19:0/1491794397 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0340046d0 con 0x7fb03c10e8f0 2024-05-09T02:09:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.684+0000 7fb043f47700 1 -- 172.21.15.19:0/1491794397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03c1a78a0 con 0x7fb03c10e8f0 2024-05-09T02:09:43.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.685+0000 7fb043f47700 1 -- 172.21.15.19:0/1491794397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb03c0512c0 con 0x7fb03c10e8f0 2024-05-09T02:09:43.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.687+0000 7fb032ffd700 1 -- 172.21.15.19:0/1491794397 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb034007980 con 0x7fb03c10e8f0 2024-05-09T02:09:43.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.687+0000 7fb032ffd700 1 --2- 172.21.15.19:0/1491794397 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb0280b2910 0x7fb0280b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.687+0000 7fb041ce3700 1 --2- 172.21.15.19:0/1491794397 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb0280b2910 0x7fb0280b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.687+0000 7fb032ffd700 1 -- 172.21.15.19:0/1491794397 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fb03400f020 con 0x7fb03c10e8f0 2024-05-09T02:09:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.688+0000 7fb041ce3700 1 --2- 172.21.15.19:0/1491794397 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb0280b2910 0x7fb0280b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb038005950 tx=0x7fb03800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:43.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.691+0000 7fb032ffd700 1 -- 172.21.15.19:0/1491794397 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb03402a070 con 0x7fb03c10e8f0 2024-05-09T02:09:43.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.568+0000 7f2277d63700 1 -- 172.21.15.19:0/589186534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2270108740 msgr2=0x7f22701100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:43.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.568+0000 7f2277d63700 1 --2- 172.21.15.19:0/589186534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2270108740 0x7f22701100a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f226800b0d0 tx=0x7f226800b3e0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.568+0000 7f2277d63700 1 -- 172.21.15.19:0/589186534 shutdown_connections 2024-05-09T02:09:43.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.568+0000 7f2277d63700 1 --2- 172.21.15.19:0/589186534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2270108740 0x7f22701100a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.568+0000 7f2277d63700 1 --2- 172.21.15.19:0/589186534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22701034c0 0x7f22701038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.568+0000 7f2277d63700 1 --2- 172.21.15.19:0/589186534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2270107df0 0x7f2270108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.568+0000 7f2277d63700 1 -- 172.21.15.19:0/589186534 >> 172.21.15.19:0/589186534 conn(0x7f22700fedc0 msgr2=0x7f22701011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:43.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.568+0000 7f2277d63700 1 -- 172.21.15.19:0/589186534 shutdown_connections 2024-05-09T02:09:43.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.568+0000 7f2277d63700 1 -- 172.21.15.19:0/589186534 wait complete. 2024-05-09T02:09:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.569+0000 7f2277d63700 1 Processor -- start 2024-05-09T02:09:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.569+0000 7f2277d63700 1 -- start start 2024-05-09T02:09:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.569+0000 7f2277d63700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22701034c0 0x7f22700ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.569+0000 7f2277d63700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2270107df0 0x7f22700ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.569+0000 7f2277d63700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2270108740 0x7f22700ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.569+0000 7f2277d63700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2270113370 con 0x7f2270108740 2024-05-09T02:09:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.569+0000 7f2277d63700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22701131f0 con 0x7f2270107df0 2024-05-09T02:09:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.569+0000 7f2277d63700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22701134f0 con 0x7f22701034c0 2024-05-09T02:09:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.569+0000 7f2276300700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2270108740 0x7f22700ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.569+0000 7f2276300700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2270108740 0x7f22700ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43284/0 (socket says 172.21.15.19:43284) 2024-05-09T02:09:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.569+0000 7f2276300700 1 -- 172.21.15.19:0/1528919755 learned_addr learned my addr 172.21.15.19:0/1528919755 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.569+0000 7f2275aff700 1 --2- 172.21.15.19:0/1528919755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22701034c0 0x7f22700ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.569+0000 7f22752fe700 1 --2- 172.21.15.19:0/1528919755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2270107df0 0x7f22700ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.570+0000 7f2276300700 1 -- 172.21.15.19:0/1528919755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22701034c0 msgr2=0x7f22700ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.570+0000 7f2276300700 1 --2- 172.21.15.19:0/1528919755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22701034c0 0x7f22700ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.570+0000 7f2276300700 1 -- 172.21.15.19:0/1528919755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2270107df0 msgr2=0x7f22700ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.570+0000 7f2276300700 1 --2- 172.21.15.19:0/1528919755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2270107df0 0x7f22700ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.570+0000 7f2276300700 1 -- 172.21.15.19:0/1528919755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22701a90c0 con 0x7f2270108740 2024-05-09T02:09:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.570+0000 7f2276300700 1 --2- 172.21.15.19:0/1528919755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2270108740 0x7f22700ffd20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f22680095c0 tx=0x7f2268007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.570+0000 7f2266ffd700 1 -- 172.21.15.19:0/1528919755 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226801f070 con 0x7f2270108740 2024-05-09T02:09:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.570+0000 7f2277d63700 1 -- 172.21.15.19:0/1528919755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22701a9350 con 0x7f2270108740 2024-05-09T02:09:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.570+0000 7f2266ffd700 1 -- 172.21.15.19:0/1528919755 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2268007e20 con 0x7f2270108740 2024-05-09T02:09:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.570+0000 7f2266ffd700 1 -- 172.21.15.19:0/1528919755 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226801b4a0 con 0x7f2270108740 2024-05-09T02:09:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.570+0000 7f2277d63700 1 -- 172.21.15.19:0/1528919755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22701a9800 con 0x7f2270108740 2024-05-09T02:09:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.571+0000 7f2277d63700 1 -- 172.21.15.19:0/1528919755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22700512c0 con 0x7f2270108740 2024-05-09T02:09:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.573+0000 7f2266ffd700 1 -- 172.21.15.19:0/1528919755 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2268022070 con 0x7f2270108740 2024-05-09T02:09:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.574+0000 7f2266ffd700 1 --2- 172.21.15.19:0/1528919755 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f225c0b2910 0x7f225c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.574+0000 7f2275aff700 1 --2- 172.21.15.19:0/1528919755 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f225c0b2910 0x7f225c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.574+0000 7f2266ffd700 1 -- 172.21.15.19:0/1528919755 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f226800f020 con 0x7f2270108740 2024-05-09T02:09:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.574+0000 7f2275aff700 1 --2- 172.21.15.19:0/1528919755 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f225c0b2910 0x7f225c0b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f226c00ac50 tx=0x7f226c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.578+0000 7f2266ffd700 1 -- 172.21.15.19:0/1528919755 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2268029020 con 0x7f2270108740 2024-05-09T02:09:43.786 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:43 smithi019 ceph-mon[30151]: pgmap v350: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:43.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:43 smithi196 ceph-mon[34718]: pgmap v350: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.798+0000 7f2277d63700 1 -- 172.21.15.19:0/1528919755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f227002e5f0 con 0x7f2270108740 2024-05-09T02:09:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.799+0000 7f2266ffd700 1 -- 172.21.15.19:0/1528919755 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f22680ab590 con 0x7f2270108740 2024-05-09T02:09:43.801 INFO:teuthology.orchestra.run.smithi019.stdout:279172874282 2024-05-09T02:09:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.801+0000 7f2277d63700 1 -- 172.21.15.19:0/1528919755 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f225c0b2910 msgr2=0x7f225c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.801+0000 7f2277d63700 1 --2- 172.21.15.19:0/1528919755 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f225c0b2910 0x7f225c0b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f226c00ac50 tx=0x7f226c00a380 comp rx=0 tx=0).stop 2024-05-09T02:09:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.801+0000 7f2277d63700 1 -- 172.21.15.19:0/1528919755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2270108740 msgr2=0x7f22700ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.801+0000 7f2277d63700 1 --2- 172.21.15.19:0/1528919755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2270108740 0x7f22700ffd20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f22680095c0 tx=0x7f2268007a20 comp rx=0 tx=0).stop 2024-05-09T02:09:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.801+0000 7f2277d63700 1 -- 172.21.15.19:0/1528919755 shutdown_connections 2024-05-09T02:09:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.801+0000 7f2277d63700 1 --2- 172.21.15.19:0/1528919755 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f225c0b2910 0x7f225c0b4e20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.801+0000 7f2277d63700 1 --2- 172.21.15.19:0/1528919755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2270108740 0x7f22700ffd20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.801+0000 7f2277d63700 1 --2- 172.21.15.19:0/1528919755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22701034c0 0x7f22700ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.801+0000 7f2277d63700 1 --2- 172.21.15.19:0/1528919755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2270107df0 0x7f22700ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.801+0000 7f2277d63700 1 -- 172.21.15.19:0/1528919755 >> 172.21.15.19:0/1528919755 conn(0x7f22700fedc0 msgr2=0x7f227006b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.801+0000 7f2277d63700 1 -- 172.21.15.19:0/1528919755 shutdown_connections 2024-05-09T02:09:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.801+0000 7f2277d63700 1 -- 172.21.15.19:0/1528919755 wait complete. 2024-05-09T02:09:43.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.908+0000 7fb043f47700 1 -- 172.21.15.19:0/1491794397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb03c19c120 con 0x7fb03c10e8f0 2024-05-09T02:09:43.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.909+0000 7fb032ffd700 1 -- 172.21.15.19:0/1491794397 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb034004870 con 0x7fb03c10e8f0 2024-05-09T02:09:43.912 INFO:teuthology.orchestra.run.smithi019.stdout:137438953547 2024-05-09T02:09:43.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.911+0000 7fb043f47700 1 -- 172.21.15.19:0/1491794397 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb0280b2910 msgr2=0x7fb0280b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:43.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.911+0000 7fb043f47700 1 --2- 172.21.15.19:0/1491794397 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb0280b2910 0x7fb0280b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb038005950 tx=0x7fb03800b410 comp rx=0 tx=0).stop 2024-05-09T02:09:43.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.911+0000 7fb043f47700 1 -- 172.21.15.19:0/1491794397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c10e8f0 msgr2=0x7fb03c19f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:43.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.911+0000 7fb043f47700 1 --2- 172.21.15.19:0/1491794397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c10e8f0 0x7fb03c19f5f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb0340062a0 tx=0x7fb034004c80 comp rx=0 tx=0).stop 2024-05-09T02:09:43.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.911+0000 7fb043f47700 1 -- 172.21.15.19:0/1491794397 shutdown_connections 2024-05-09T02:09:43.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.911+0000 7fb043f47700 1 --2- 172.21.15.19:0/1491794397 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fb0280b2910 0x7fb0280b4e20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.911+0000 7fb043f47700 1 --2- 172.21.15.19:0/1491794397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb03c10e8f0 0x7fb03c19f5f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.911+0000 7fb043f47700 1 --2- 172.21.15.19:0/1491794397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb03c105480 0x7fb03c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.911+0000 7fb043f47700 1 --2- 172.21.15.19:0/1491794397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb03c105dd0 0x7fb03c19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:43.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.911+0000 7fb043f47700 1 -- 172.21.15.19:0/1491794397 >> 172.21.15.19:0/1491794397 conn(0x7fb03c0fef30 msgr2=0x7fb03c0ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:43.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.912+0000 7fb043f47700 1 -- 172.21.15.19:0/1491794397 shutdown_connections 2024-05-09T02:09:43.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:43.912+0000 7fb043f47700 1 -- 172.21.15.19:0/1491794397 wait complete. 2024-05-09T02:09:43.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:43 smithi059 ceph-mon[34781]: pgmap v350: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:44.450 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430262 got 206158430267 for osd.7 2024-05-09T02:09:44.450 DEBUG:teuthology.parallel:result is None 2024-05-09T02:09:44.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:44 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1528919755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T02:09:44.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:44 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1491794397' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T02:09:44.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:44 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1528919755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T02:09:44.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:44 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1491794397' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T02:09:44.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:44 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1528919755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T02:09:44.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:44 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1491794397' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T02:09:45.079 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874277 got 279172874282 for osd.10 2024-05-09T02:09:45.079 DEBUG:teuthology.parallel:result is None 2024-05-09T02:09:45.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:45 smithi019 ceph-mon[30151]: pgmap v351: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:09:45.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:45 smithi196 ceph-mon[34718]: pgmap v351: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:09:45.922 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953541 got 137438953547 for osd.4 2024-05-09T02:09:45.922 DEBUG:teuthology.parallel:result is None 2024-05-09T02:09:45.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:45 smithi059 ceph-mon[34781]: pgmap v351: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:09:46.750 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:46 smithi019 ceph-mon[30151]: pgmap v352: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:09:46.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:46 smithi196 ceph-mon[34718]: pgmap v352: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:09:46.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:46 smithi059 ceph-mon[34781]: pgmap v352: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:09:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.338+0000 7f89118f6700 1 -- 172.21.15.19:0/228468448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890c108780 msgr2=0x7f890c1100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.338+0000 7f89118f6700 1 --2- 172.21.15.19:0/228468448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890c108780 0x7f890c1100e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f890400b0d0 tx=0x7f890400b3e0 comp rx=0 tx=0).stop 2024-05-09T02:09:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.338+0000 7f89118f6700 1 -- 172.21.15.19:0/228468448 shutdown_connections 2024-05-09T02:09:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.338+0000 7f89118f6700 1 --2- 172.21.15.19:0/228468448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890c108780 0x7f890c1100e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.338+0000 7f89118f6700 1 --2- 172.21.15.19:0/228468448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c103550 0x7f890c103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.338+0000 7f89118f6700 1 --2- 172.21.15.19:0/228468448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f890c107e30 0x7f890c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.338+0000 7f89118f6700 1 -- 172.21.15.19:0/228468448 >> 172.21.15.19:0/228468448 conn(0x7f890c078e00 msgr2=0x7f890c079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.338+0000 7f89118f6700 1 -- 172.21.15.19:0/228468448 shutdown_connections 2024-05-09T02:09:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.338+0000 7f89118f6700 1 -- 172.21.15.19:0/228468448 wait complete. 2024-05-09T02:09:47.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.338+0000 7f89118f6700 1 Processor -- start 2024-05-09T02:09:47.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f89118f6700 1 -- start start 2024-05-09T02:09:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f89118f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890c103550 0x7f890c19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f89118f6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c107e30 0x7f890c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f89118f6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f890c108780 0x7f890c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f89118f6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1133b0 con 0x7f890c103550 2024-05-09T02:09:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f89118f6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f890c113230 con 0x7f890c107e30 2024-05-09T02:09:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f89118f6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f890c113530 con 0x7f890c108780 2024-05-09T02:09:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f890affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890c103550 0x7f890c19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f890affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890c103550 0x7f890c19ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47348/0 (socket says 172.21.15.19:47348) 2024-05-09T02:09:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f890affd700 1 -- 172.21.15.19:0/2323364066 learned_addr learned my addr 172.21.15.19:0/2323364066 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f890b7fe700 1 --2- 172.21.15.19:0/2323364066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f890c108780 0x7f890c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f890a7fc700 1 --2- 172.21.15.19:0/2323364066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c107e30 0x7f890c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f890affd700 1 -- 172.21.15.19:0/2323364066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f890c108780 msgr2=0x7f890c19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f890affd700 1 --2- 172.21.15.19:0/2323364066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f890c108780 0x7f890c19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f890affd700 1 -- 172.21.15.19:0/2323364066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c107e30 msgr2=0x7f890c19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.339+0000 7f890affd700 1 --2- 172.21.15.19:0/2323364066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c107e30 0x7f890c19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.340+0000 7f890affd700 1 -- 172.21.15.19:0/2323364066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c19fc90 con 0x7f890c103550 2024-05-09T02:09:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.340+0000 7f890affd700 1 --2- 172.21.15.19:0/2323364066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890c103550 0x7f890c19ae70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00d950 tx=0x7f88fc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.340+0000 7f89108f4700 1 -- 172.21.15.19:0/2323364066 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc016070 con 0x7f890c103550 2024-05-09T02:09:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.340+0000 7f89118f6700 1 -- 172.21.15.19:0/2323364066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890c19ff80 con 0x7f890c103550 2024-05-09T02:09:47.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.340+0000 7f89108f4700 1 -- 172.21.15.19:0/2323364066 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88fc00fd40 con 0x7f890c103550 2024-05-09T02:09:47.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.340+0000 7f89108f4700 1 -- 172.21.15.19:0/2323364066 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc00e3b0 con 0x7f890c103550 2024-05-09T02:09:47.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.340+0000 7f89118f6700 1 -- 172.21.15.19:0/2323364066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890c1a04c0 con 0x7f890c103550 2024-05-09T02:09:47.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.341+0000 7f89118f6700 1 -- 172.21.15.19:0/2323364066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f890c0512c0 con 0x7f890c103550 2024-05-09T02:09:47.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.343+0000 7f89108f4700 1 -- 172.21.15.19:0/2323364066 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f88fc019070 con 0x7f890c103550 2024-05-09T02:09:47.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.344+0000 7f89108f4700 1 --2- 172.21.15.19:0/2323364066 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f88f40b2910 0x7f88f40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:47.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.344+0000 7f890a7fc700 1 --2- 172.21.15.19:0/2323364066 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f88f40b2910 0x7f88f40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:47.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.344+0000 7f89108f4700 1 -- 172.21.15.19:0/2323364066 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f88fc0e1e10 con 0x7f890c103550 2024-05-09T02:09:47.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.344+0000 7f890a7fc700 1 --2- 172.21.15.19:0/2323364066 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f88f40b2910 0x7f88f40b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f890c06b970 tx=0x7f890000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:47.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.348+0000 7f89108f4700 1 -- 172.21.15.19:0/2323364066 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f88fc0a99b0 con 0x7f890c103550 2024-05-09T02:09:47.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.562+0000 7f89118f6700 1 -- 172.21.15.19:0/2323364066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f890c02e5f0 con 0x7f890c103550 2024-05-09T02:09:47.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.562+0000 7f89108f4700 1 -- 172.21.15.19:0/2323364066 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f88fc0a92b0 con 0x7f890c103550 2024-05-09T02:09:47.565 INFO:teuthology.orchestra.run.smithi019.stdout:257698037810 2024-05-09T02:09:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.565+0000 7f89118f6700 1 -- 172.21.15.19:0/2323364066 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f88f40b2910 msgr2=0x7f88f40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.565+0000 7f89118f6700 1 --2- 172.21.15.19:0/2323364066 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f88f40b2910 0x7f88f40b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f890c06b970 tx=0x7f890000a380 comp rx=0 tx=0).stop 2024-05-09T02:09:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.565+0000 7f89118f6700 1 -- 172.21.15.19:0/2323364066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890c103550 msgr2=0x7f890c19ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.565+0000 7f89118f6700 1 --2- 172.21.15.19:0/2323364066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890c103550 0x7f890c19ae70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00d950 tx=0x7f88fc00dc60 comp rx=0 tx=0).stop 2024-05-09T02:09:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.565+0000 7f89118f6700 1 -- 172.21.15.19:0/2323364066 shutdown_connections 2024-05-09T02:09:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.565+0000 7f89118f6700 1 --2- 172.21.15.19:0/2323364066 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f88f40b2910 0x7f88f40b4e20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.565+0000 7f89118f6700 1 --2- 172.21.15.19:0/2323364066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f890c108780 0x7f890c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.565+0000 7f89118f6700 1 --2- 172.21.15.19:0/2323364066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890c103550 0x7f890c19ae70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.565+0000 7f89118f6700 1 --2- 172.21.15.19:0/2323364066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c107e30 0x7f890c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.565+0000 7f89118f6700 1 -- 172.21.15.19:0/2323364066 >> 172.21.15.19:0/2323364066 conn(0x7f890c078e00 msgr2=0x7f890c075d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.565+0000 7f89118f6700 1 -- 172.21.15.19:0/2323364066 shutdown_connections 2024-05-09T02:09:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:47.565+0000 7f89118f6700 1 -- 172.21.15.19:0/2323364066 wait complete. 2024-05-09T02:09:47.891 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:47 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2323364066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T02:09:47.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:47 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2323364066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T02:09:48.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:47 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2323364066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T02:09:48.225 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037805 got 257698037810 for osd.9 2024-05-09T02:09:48.226 DEBUG:teuthology.parallel:result is None 2024-05-09T02:09:48.226 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T02:09:48.227 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T02:09:48.481 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:48.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:48 smithi019 ceph-mon[30151]: pgmap v353: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:48.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:48 smithi059 ceph-mon[34781]: pgmap v353: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:49.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:48 smithi196 ceph-mon[34718]: pgmap v353: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:50.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:49 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:09:50.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.180+0000 7fc6ebf62700 1 -- 172.21.15.19:0/2496115353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e4107df0 msgr2=0x7fc6e4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:50.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.180+0000 7fc6ebf62700 1 --2- 172.21.15.19:0/2496115353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e4107df0 0x7fc6e4108200 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40099c0 tx=0x7fc6d4009cd0 comp rx=0 tx=0).stop 2024-05-09T02:09:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.180+0000 7fc6ebf62700 1 -- 172.21.15.19:0/2496115353 shutdown_connections 2024-05-09T02:09:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.180+0000 7fc6ebf62700 1 --2- 172.21.15.19:0/2496115353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6e4108740 0x7fc6e41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.180+0000 7fc6ebf62700 1 --2- 172.21.15.19:0/2496115353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6e41034c0 0x7fc6e41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.180+0000 7fc6ebf62700 1 --2- 172.21.15.19:0/2496115353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e4107df0 0x7fc6e4108200 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.180+0000 7fc6ebf62700 1 -- 172.21.15.19:0/2496115353 >> 172.21.15.19:0/2496115353 conn(0x7fc6e40fedf0 msgr2=0x7fc6e4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.181+0000 7fc6ebf62700 1 -- 172.21.15.19:0/2496115353 shutdown_connections 2024-05-09T02:09:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.181+0000 7fc6ebf62700 1 -- 172.21.15.19:0/2496115353 wait complete. 2024-05-09T02:09:50.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.181+0000 7fc6ebf62700 1 Processor -- start 2024-05-09T02:09:50.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.181+0000 7fc6ebf62700 1 -- start start 2024-05-09T02:09:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6ebf62700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e41034c0 0x7fc6e419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6ebf62700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6e4107df0 0x7fc6e419c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6ebf62700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6e4108740 0x7fc6e419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6ebf62700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e4113350 con 0x7fc6e41034c0 2024-05-09T02:09:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6ebf62700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e41131d0 con 0x7fc6e4107df0 2024-05-09T02:09:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6ebf62700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e41134d0 con 0x7fc6e4108740 2024-05-09T02:09:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6e9cfe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e41034c0 0x7fc6e419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6e9cfe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e41034c0 0x7fc6e419bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47364/0 (socket says 172.21.15.19:47364) 2024-05-09T02:09:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6e9cfe700 1 -- 172.21.15.19:0/966043260 learned_addr learned my addr 172.21.15.19:0/966043260 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6e94fd700 1 --2- 172.21.15.19:0/966043260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6e4107df0 0x7fc6e419c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6ea4ff700 1 --2- 172.21.15.19:0/966043260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6e4108740 0x7fc6e419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6e9cfe700 1 -- 172.21.15.19:0/966043260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6e4108740 msgr2=0x7fc6e419c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6e9cfe700 1 --2- 172.21.15.19:0/966043260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6e4108740 0x7fc6e419c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6e9cfe700 1 -- 172.21.15.19:0/966043260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6e4107df0 msgr2=0x7fc6e419c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6e9cfe700 1 --2- 172.21.15.19:0/966043260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6e4107df0 0x7fc6e419c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6e9cfe700 1 -- 172.21.15.19:0/966043260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e41a7000 con 0x7fc6e41034c0 2024-05-09T02:09:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.182+0000 7fc6e9cfe700 1 --2- 172.21.15.19:0/966043260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e41034c0 0x7fc6e419bcb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000e9a0 tx=0x7fc6e000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.183+0000 7fc6daffd700 1 -- 172.21.15.19:0/966043260 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000ca10 con 0x7fc6e41034c0 2024-05-09T02:09:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.183+0000 7fc6ebf62700 1 -- 172.21.15.19:0/966043260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e41a72f0 con 0x7fc6e41034c0 2024-05-09T02:09:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.183+0000 7fc6daffd700 1 -- 172.21.15.19:0/966043260 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e000cbb0 con 0x7fc6e41034c0 2024-05-09T02:09:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.183+0000 7fc6daffd700 1 -- 172.21.15.19:0/966043260 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0018680 con 0x7fc6e41034c0 2024-05-09T02:09:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.183+0000 7fc6ebf62700 1 -- 172.21.15.19:0/966043260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e41a7800 con 0x7fc6e41034c0 2024-05-09T02:09:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.184+0000 7fc6ebf62700 1 -- 172.21.15.19:0/966043260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e40512c0 con 0x7fc6e41034c0 2024-05-09T02:09:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.186+0000 7fc6daffd700 1 -- 172.21.15.19:0/966043260 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc6e0010430 con 0x7fc6e41034c0 2024-05-09T02:09:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.186+0000 7fc6daffd700 1 --2- 172.21.15.19:0/966043260 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc6d00b2910 0x7fc6d00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.186+0000 7fc6e94fd700 1 --2- 172.21.15.19:0/966043260 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc6d00b2910 0x7fc6d00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.187+0000 7fc6daffd700 1 -- 172.21.15.19:0/966043260 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc6e00e2870 con 0x7fc6e41034c0 2024-05-09T02:09:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.187+0000 7fc6e94fd700 1 --2- 172.21.15.19:0/966043260 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc6d00b2910 0x7fc6d00b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4000c00 tx=0x7fc6d401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.195+0000 7fc6daffd700 1 -- 172.21.15.19:0/966043260 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc6e00aa410 con 0x7fc6e41034c0 2024-05-09T02:09:50.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:49 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:09:50.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:49 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:09:50.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.438+0000 7fc6ebf62700 1 -- 172.21.15.19:0/966043260 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc6e4063c00 con 0x7fc6d00b2910 2024-05-09T02:09:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.452+0000 7fc6daffd700 1 -- 172.21.15.19:0/966043260 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259026 (secure 0 0 0) 0x7fc6d40673f0 con 0x7fc6d00b2910 2024-05-09T02:09:50.460 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:09:50.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.456+0000 7fc6ebf62700 1 -- 172.21.15.19:0/966043260 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc6d00b2910 msgr2=0x7fc6d00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:50.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.456+0000 7fc6ebf62700 1 --2- 172.21.15.19:0/966043260 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc6d00b2910 0x7fc6d00b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4000c00 tx=0x7fc6d401a040 comp rx=0 tx=0).stop 2024-05-09T02:09:50.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.456+0000 7fc6ebf62700 1 -- 172.21.15.19:0/966043260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e41034c0 msgr2=0x7fc6e419bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:50.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.456+0000 7fc6ebf62700 1 --2- 172.21.15.19:0/966043260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e41034c0 0x7fc6e419bcb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000e9a0 tx=0x7fc6e000ecb0 comp rx=0 tx=0).stop 2024-05-09T02:09:50.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.456+0000 7fc6ebf62700 1 -- 172.21.15.19:0/966043260 shutdown_connections 2024-05-09T02:09:50.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.456+0000 7fc6ebf62700 1 --2- 172.21.15.19:0/966043260 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc6d00b2910 0x7fc6d00b4e20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:50.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.456+0000 7fc6ebf62700 1 --2- 172.21.15.19:0/966043260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6e4108740 0x7fc6e419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:50.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.456+0000 7fc6ebf62700 1 --2- 172.21.15.19:0/966043260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6e41034c0 0x7fc6e419bcb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:50.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.456+0000 7fc6ebf62700 1 --2- 172.21.15.19:0/966043260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6e4107df0 0x7fc6e419c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:50.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.456+0000 7fc6ebf62700 1 -- 172.21.15.19:0/966043260 >> 172.21.15.19:0/966043260 conn(0x7fc6e40fedf0 msgr2=0x7fc6e4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:50.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.456+0000 7fc6ebf62700 1 -- 172.21.15.19:0/966043260 shutdown_connections 2024-05-09T02:09:50.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:50.456+0000 7fc6ebf62700 1 -- 172.21.15.19:0/966043260 wait complete. 2024-05-09T02:09:50.463 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-09T02:09:50.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:50 smithi059 ceph-mon[34781]: pgmap v354: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:09:50.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:50 smithi059 ceph-mon[34781]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:09:51.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:50 smithi196 ceph-mon[34718]: pgmap v354: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:09:51.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:50 smithi196 ceph-mon[34718]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:09:51.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:50 smithi019 ceph-mon[30151]: pgmap v354: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:09:51.064 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:50 smithi019 ceph-mon[30151]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:09:51.844 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-05-09T02:09:50.064834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659241,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1126,"num_read_kb":671,"num_write":370,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2952064,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376732,"kb_used_data":10896,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565092,"statfs":{"total":1151940427776,"available":1151554654208,"internally_reserved":0,"allocated":11157504,"data_stored":7118043,"data_compressed":176253,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003315"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311161+0000","last_change":"2024-05-09T02:07:48.203957+0000","last_active":"2024-05-09T02:08:25.311161+0000","last_peered":"2024-05-09T02:08:25.311161+0000","last_clean":"2024-05-09T02:08:25.311161+0000","last_became_active":"2024-05-09T02:07:48.203354+0000","last_became_peered":"2024-05-09T02:07:48.203354+0000","last_unstale":"2024-05-09T02:08:25.311161+0000","last_undegraded":"2024-05-09T02:08:25.311161+0000","last_fullsized":"2024-05-09T02:08:25.311161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:06.169693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315459+0000","last_change":"2024-05-09T02:07:48.192885+0000","last_active":"2024-05-09T02:08:25.315459+0000","last_peered":"2024-05-09T02:08:25.315459+0000","last_clean":"2024-05-09T02:08:25.315459+0000","last_became_active":"2024-05-09T02:07:48.192476+0000","last_became_peered":"2024-05-09T02:07:48.192476+0000","last_unstale":"2024-05-09T02:08:25.315459+0000","last_undegraded":"2024-05-09T02:08:25.315459+0000","last_fullsized":"2024-05-09T02:08:25.315459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:33.710309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314654+0000","last_change":"2024-05-09T02:07:48.198684+0000","last_active":"2024-05-09T02:08:25.314654+0000","last_peered":"2024-05-09T02:08:25.314654+0000","last_clean":"2024-05-09T02:08:25.314654+0000","last_became_active":"2024-05-09T02:07:48.198344+0000","last_became_peered":"2024-05-09T02:07:48.198344+0000","last_unstale":"2024-05-09T02:08:25.314654+0000","last_undegraded":"2024-05-09T02:08:25.314654+0000","last_fullsized":"2024-05-09T02:08:25.314654+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:19.990063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317876+0000","last_change":"2024-05-09T02:07:48.189209+0000","last_active":"2024-05-09T02:08:25.317876+0000","last_peered":"2024-05-09T02:08:25.317876+0000","last_clean":"2024-05-09T02:08:25.317876+0000","last_became_active":"2024-05-09T02:07:48.188744+0000","last_became_peered":"2024-05-09T02:07:48.188744+0000","last_unstale":"2024-05-09T02:08:25.317876+0000","last_undegraded":"2024-05-09T02:08:25.317876+0000","last_fullsized":"2024-05-09T02:08:25.317876+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:03:25.216849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311030+0000","last_change":"2024-05-09T02:07:48.187897+0000","last_active":"2024-05-09T02:08:25.311030+0000","last_peered":"2024-05-09T02:08:25.311030+0000","last_clean":"2024-05-09T02:08:25.311030+0000","last_became_active":"2024-05-09T02:07:48.187476+0000","last_became_peered":"2024-05-09T02:07:48.187476+0000","last_unstale":"2024-05-09T02:08:25.311030+0000","last_undegraded":"2024-05-09T02:08:25.311030+0000","last_fullsized":"2024-05-09T02:08:25.311030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:39.366520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.307870+0000","last_change":"2024-05-09T02:07:48.195436+0000","last_active":"2024-05-09T02:08:25.307870+0000","last_peered":"2024-05-09T02:08:25.307870+0000","last_clean":"2024-05-09T02:08:25.307870+0000","last_became_active":"2024-05-09T02:07:48.194982+0000","last_became_peered":"2024-05-09T02:07:48.194982+0000","last_unstale":"2024-05-09T02:08:25.307870+0000","last_undegraded":"2024-05-09T02:08:25.307870+0000","last_fullsized":"2024-05-09T02:08:25.307870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:19.746701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313666+0000","last_change":"2024-05-09T02:07:48.198458+0000","last_active":"2024-05-09T02:08:25.313666+0000","last_peered":"2024-05-09T02:08:25.313666+0000","last_clean":"2024-05-09T02:08:25.313666+0000","last_became_active":"2024-05-09T02:07:48.198089+0000","last_became_peered":"2024-05-09T02:07:48.198089+0000","last_unstale":"2024-05-09T02:08:25.313666+0000","last_undegraded":"2024-05-09T02:08:25.313666+0000","last_fullsized":"2024-05-09T02:08:25.313666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:37:44.999438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314425+0000","last_change":"2024-05-09T02:07:48.200294+0000","last_active":"2024-05-09T02:08:25.314425+0000","last_peered":"2024-05-09T02:08:25.314425+0000","last_clean":"2024-05-09T02:08:25.314425+0000","last_became_active":"2024-05-09T02:07:48.199949+0000","last_became_peered":"2024-05-09T02:07:48.199949+0000","last_unstale":"2024-05-09T02:08:25.314425+0000","last_undegraded":"2024-05-09T02:08:25.314425+0000","last_fullsized":"2024-05-09T02:08:25.314425+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:39.263849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315463+0000","last_change":"2024-05-09T02:07:48.196282+0000","last_active":"2024-05-09T02:08:25.315463+0000","last_peered":"2024-05-09T02:08:25.315463+0000","last_clean":"2024-05-09T02:08:25.315463+0000","last_became_active":"2024-05-09T02:07:48.195887+0000","last_became_peered":"2024-05-09T02:07:48.195887+0000","last_unstale":"2024-05-09T02:08:25.315463+0000","last_undegraded":"2024-05-09T02:08:25.315463+0000","last_fullsized":"2024-05-09T02:08:25.315463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:26.081867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.320636+0000","last_change":"2024-05-09T02:08:25.320636+0000","last_active":"2024-05-09T02:08:25.320636+0000","last_peered":"2024-05-09T02:08:25.320636+0000","last_clean":"2024-05-09T02:08:25.320636+0000","last_became_active":"2024-05-09T02:08:25.320155+0000","last_became_peered":"2024-05-09T02:08:25.320155+0000","last_unstale":"2024-05-09T02:08:25.320636+0000","last_undegraded":"2024-05-09T02:08:25.320636+0000","last_fullsized":"2024-05-09T02:08:25.320636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:55.728773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314733+0000","last_change":"2024-05-09T02:07:48.197649+0000","last_active":"2024-05-09T02:08:25.314733+0000","last_peered":"2024-05-09T02:08:25.314733+0000","last_clean":"2024-05-09T02:08:25.314733+0000","last_became_active":"2024-05-09T02:07:48.196884+0000","last_became_peered":"2024-05-09T02:07:48.196884+0000","last_unstale":"2024-05-09T02:08:25.314733+0000","last_undegraded":"2024-05-09T02:08:25.314733+0000","last_fullsized":"2024-05-09T02:08:25.314733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:23.851216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318021+0000","last_change":"2024-05-09T02:07:48.193568+0000","last_active":"2024-05-09T02:08:25.318021+0000","last_peered":"2024-05-09T02:08:25.318021+0000","last_clean":"2024-05-09T02:08:25.318021+0000","last_became_active":"2024-05-09T02:07:48.193117+0000","last_became_peered":"2024-05-09T02:07:48.193117+0000","last_unstale":"2024-05-09T02:08:25.318021+0000","last_undegraded":"2024-05-09T02:08:25.318021+0000","last_fullsized":"2024-05-09T02:08:25.318021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:34.371246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309662+0000","last_change":"2024-05-09T02:07:48.196314+0000","last_active":"2024-05-09T02:08:25.309662+0000","last_peered":"2024-05-09T02:08:25.309662+0000","last_clean":"2024-05-09T02:08:25.309662+0000","last_became_active":"2024-05-09T02:07:48.195988+0000","last_became_peered":"2024-05-09T02:07:48.195988+0000","last_unstale":"2024-05-09T02:08:25.309662+0000","last_undegraded":"2024-05-09T02:08:25.309662+0000","last_fullsized":"2024-05-09T02:08:25.309662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:08.556166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314018+0000","last_change":"2024-05-09T02:07:48.192974+0000","last_active":"2024-05-09T02:08:25.314018+0000","last_peered":"2024-05-09T02:08:25.314018+0000","last_clean":"2024-05-09T02:08:25.314018+0000","last_became_active":"2024-05-09T02:07:48.192599+0000","last_became_peered":"2024-05-09T02:07:48.192599+0000","last_unstale":"2024-05-09T02:08:25.314018+0000","last_undegraded":"2024-05-09T02:08:25.314018+0000","last_fullsized":"2024-05-09T02:08:25.314018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:37.192219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314434+0000","last_change":"2024-05-09T02:07:48.193208+0000","last_active":"2024-05-09T02:08:25.314434+0000","last_peered":"2024-05-09T02:08:25.314434+0000","last_clean":"2024-05-09T02:08:25.314434+0000","last_became_active":"2024-05-09T02:07:48.192654+0000","last_became_peered":"2024-05-09T02:07:48.192654+0000","last_unstale":"2024-05-09T02:08:25.314434+0000","last_undegraded":"2024-05-09T02:08:25.314434+0000","last_fullsized":"2024-05-09T02:08:25.314434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:55:42.870372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314811+0000","last_change":"2024-05-09T02:07:48.187568+0000","last_active":"2024-05-09T02:08:25.314811+0000","last_peered":"2024-05-09T02:08:25.314811+0000","last_clean":"2024-05-09T02:08:25.314811+0000","last_became_active":"2024-05-09T02:07:48.187159+0000","last_became_peered":"2024-05-09T02:07:48.187159+0000","last_unstale":"2024-05-09T02:08:25.314811+0000","last_undegraded":"2024-05-09T02:08:25.314811+0000","last_fullsized":"2024-05-09T02:08:25.314811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:04.994094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311915+0000","last_change":"2024-05-09T02:07:48.204406+0000","last_active":"2024-05-09T02:08:25.311915+0000","last_peered":"2024-05-09T02:08:25.311915+0000","last_clean":"2024-05-09T02:08:25.311915+0000","last_became_active":"2024-05-09T02:07:48.203592+0000","last_became_peered":"2024-05-09T02:07:48.203592+0000","last_unstale":"2024-05-09T02:08:25.311915+0000","last_undegraded":"2024-05-09T02:08:25.311915+0000","last_fullsized":"2024-05-09T02:08:25.311915+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:21.655807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317492+0000","last_change":"2024-05-09T02:07:48.195326+0000","last_active":"2024-05-09T02:08:25.317492+0000","last_peered":"2024-05-09T02:08:25.317492+0000","last_clean":"2024-05-09T02:08:25.317492+0000","last_became_active":"2024-05-09T02:07:48.194969+0000","last_became_peered":"2024-05-09T02:07:48.194969+0000","last_unstale":"2024-05-09T02:08:25.317492+0000","last_undegraded":"2024-05-09T02:08:25.317492+0000","last_fullsized":"2024-05-09T02:08:25.317492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:00.269753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315957+0000","last_change":"2024-05-09T02:07:48.190706+0000","last_active":"2024-05-09T02:08:25.315957+0000","last_peered":"2024-05-09T02:08:25.315957+0000","last_clean":"2024-05-09T02:08:25.315957+0000","last_became_active":"2024-05-09T02:07:48.190171+0000","last_became_peered":"2024-05-09T02:07:48.190171+0000","last_unstale":"2024-05-09T02:08:25.315957+0000","last_undegraded":"2024-05-09T02:08:25.315957+0000","last_fullsized":"2024-05-09T02:08:25.315957+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:19:31.333618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":141,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:47.408648+0000","last_change":"2024-05-09T02:07:45.158224+0000","last_active":"2024-05-09T02:09:47.408648+0000","last_peered":"2024-05-09T02:09:47.408648+0000","last_clean":"2024-05-09T02:09:47.408648+0000","last_became_active":"2024-05-09T02:07:45.157722+0000","last_became_peered":"2024-05-09T02:07:45.157722+0000","last_unstale":"2024-05-09T02:09:47.408648+0000","last_undegraded":"2024-05-09T02:09:47.408648+0000","last_fullsized":"2024-05-09T02:09:47.408648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:31.750699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315461+0000","last_change":"2024-05-09T02:07:48.197978+0000","last_active":"2024-05-09T02:08:25.315461+0000","last_peered":"2024-05-09T02:08:25.315461+0000","last_clean":"2024-05-09T02:08:25.315461+0000","last_became_active":"2024-05-09T02:07:48.197077+0000","last_became_peered":"2024-05-09T02:07:48.197077+0000","last_unstale":"2024-05-09T02:08:25.315461+0000","last_undegraded":"2024-05-09T02:08:25.315461+0000","last_fullsized":"2024-05-09T02:08:25.315461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:25.897904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315892+0000","last_change":"2024-05-09T02:07:45.185758+0000","last_active":"2024-05-09T02:08:25.315892+0000","last_peered":"2024-05-09T02:08:25.315892+0000","last_clean":"2024-05-09T02:08:25.315892+0000","last_became_active":"2024-05-09T02:07:45.185411+0000","last_became_peered":"2024-05-09T02:07:45.185411+0000","last_unstale":"2024-05-09T02:08:25.315892+0000","last_undegraded":"2024-05-09T02:08:25.315892+0000","last_fullsized":"2024-05-09T02:08:25.315892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:04.410523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317321+0000","last_change":"2024-05-09T02:07:48.199794+0000","last_active":"2024-05-09T02:08:25.317321+0000","last_peered":"2024-05-09T02:08:25.317321+0000","last_clean":"2024-05-09T02:08:25.317321+0000","last_became_active":"2024-05-09T02:07:48.199453+0000","last_became_peered":"2024-05-09T02:07:48.199453+0000","last_unstale":"2024-05-09T02:08:25.317321+0000","last_undegraded":"2024-05-09T02:08:25.317321+0000","last_fullsized":"2024-05-09T02:08:25.317321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:09.735830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311011+0000","last_change":"2024-05-09T02:07:45.162221+0000","last_active":"2024-05-09T02:08:25.311011+0000","last_peered":"2024-05-09T02:08:25.311011+0000","last_clean":"2024-05-09T02:08:25.311011+0000","last_became_active":"2024-05-09T02:07:45.161748+0000","last_became_peered":"2024-05-09T02:07:45.161748+0000","last_unstale":"2024-05-09T02:08:25.311011+0000","last_undegraded":"2024-05-09T02:08:25.311011+0000","last_fullsized":"2024-05-09T02:08:25.311011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:38.837411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315182+0000","last_change":"2024-05-09T02:07:48.188741+0000","last_active":"2024-05-09T02:08:25.315182+0000","last_peered":"2024-05-09T02:08:25.315182+0000","last_clean":"2024-05-09T02:08:25.315182+0000","last_became_active":"2024-05-09T02:07:48.188329+0000","last_became_peered":"2024-05-09T02:07:48.188329+0000","last_unstale":"2024-05-09T02:08:25.315182+0000","last_undegraded":"2024-05-09T02:08:25.315182+0000","last_fullsized":"2024-05-09T02:08:25.315182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:46.773336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310908+0000","last_change":"2024-05-09T02:07:45.163573+0000","last_active":"2024-05-09T02:08:25.310908+0000","last_peered":"2024-05-09T02:08:25.310908+0000","last_clean":"2024-05-09T02:08:25.310908+0000","last_became_active":"2024-05-09T02:07:45.163162+0000","last_became_peered":"2024-05-09T02:07:45.163162+0000","last_unstale":"2024-05-09T02:08:25.310908+0000","last_undegraded":"2024-05-09T02:08:25.310908+0000","last_fullsized":"2024-05-09T02:08:25.310908+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:12.356586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318419+0000","last_change":"2024-05-09T02:07:48.203928+0000","last_active":"2024-05-09T02:08:25.318419+0000","last_peered":"2024-05-09T02:08:25.318419+0000","last_clean":"2024-05-09T02:08:25.318419+0000","last_became_active":"2024-05-09T02:07:48.203224+0000","last_became_peered":"2024-05-09T02:07:48.203224+0000","last_unstale":"2024-05-09T02:08:25.318419+0000","last_undegraded":"2024-05-09T02:08:25.318419+0000","last_fullsized":"2024-05-09T02:08:25.318419+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:00.568050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":132,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:45.408163+0000","last_change":"2024-05-09T02:07:45.160161+0000","last_active":"2024-05-09T02:09:45.408163+0000","last_peered":"2024-05-09T02:09:45.408163+0000","last_clean":"2024-05-09T02:09:45.408163+0000","last_became_active":"2024-05-09T02:07:45.159118+0000","last_became_peered":"2024-05-09T02:07:45.159118+0000","last_unstale":"2024-05-09T02:09:45.408163+0000","last_undegraded":"2024-05-09T02:09:45.408163+0000","last_fullsized":"2024-05-09T02:09:45.408163+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:04.510877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.319860+0000","last_change":"2024-05-09T02:08:25.319860+0000","last_active":"2024-05-09T02:08:25.319860+0000","last_peered":"2024-05-09T02:08:25.319860+0000","last_clean":"2024-05-09T02:08:25.319860+0000","last_became_active":"2024-05-09T02:08:25.319370+0000","last_became_peered":"2024-05-09T02:08:25.319370+0000","last_unstale":"2024-05-09T02:08:25.319860+0000","last_undegraded":"2024-05-09T02:08:25.319860+0000","last_fullsized":"2024-05-09T02:08:25.319860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:38.504057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.327069+0000","last_change":"2024-05-09T02:08:25.327069+0000","last_active":"2024-05-09T02:08:25.327069+0000","last_peered":"2024-05-09T02:08:25.327069+0000","last_clean":"2024-05-09T02:08:25.327069+0000","last_became_active":"2024-05-09T02:08:25.316629+0000","last_became_peered":"2024-05-09T02:08:25.316629+0000","last_unstale":"2024-05-09T02:08:25.327069+0000","last_undegraded":"2024-05-09T02:08:25.327069+0000","last_fullsized":"2024-05-09T02:08:25.327069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:51.028256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318643+0000","last_change":"2024-05-09T02:07:48.201056+0000","last_active":"2024-05-09T02:08:25.318643+0000","last_peered":"2024-05-09T02:08:25.318643+0000","last_clean":"2024-05-09T02:08:25.318643+0000","last_became_active":"2024-05-09T02:07:48.200671+0000","last_became_peered":"2024-05-09T02:07:48.200671+0000","last_unstale":"2024-05-09T02:08:25.318643+0000","last_undegraded":"2024-05-09T02:08:25.318643+0000","last_fullsized":"2024-05-09T02:08:25.318643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:37:16.138170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309296+0000","last_change":"2024-05-09T02:07:45.160417+0000","last_active":"2024-05-09T02:08:25.309296+0000","last_peered":"2024-05-09T02:08:25.309296+0000","last_clean":"2024-05-09T02:08:25.309296+0000","last_became_active":"2024-05-09T02:07:45.160076+0000","last_became_peered":"2024-05-09T02:07:45.160076+0000","last_unstale":"2024-05-09T02:08:25.309296+0000","last_undegraded":"2024-05-09T02:08:25.309296+0000","last_fullsized":"2024-05-09T02:08:25.309296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:53.653494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314286+0000","last_change":"2024-05-09T02:07:48.203267+0000","last_active":"2024-05-09T02:08:25.314286+0000","last_peered":"2024-05-09T02:08:25.314286+0000","last_clean":"2024-05-09T02:08:25.314286+0000","last_became_active":"2024-05-09T02:07:48.202747+0000","last_became_peered":"2024-05-09T02:07:48.202747+0000","last_unstale":"2024-05-09T02:08:25.314286+0000","last_undegraded":"2024-05-09T02:08:25.314286+0000","last_fullsized":"2024-05-09T02:08:25.314286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.954400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317820+0000","last_change":"2024-05-09T02:07:45.164713+0000","last_active":"2024-05-09T02:08:25.317820+0000","last_peered":"2024-05-09T02:08:25.317820+0000","last_clean":"2024-05-09T02:08:25.317820+0000","last_became_active":"2024-05-09T02:07:45.164324+0000","last_became_peered":"2024-05-09T02:07:45.164324+0000","last_unstale":"2024-05-09T02:08:25.317820+0000","last_undegraded":"2024-05-09T02:08:25.317820+0000","last_fullsized":"2024-05-09T02:08:25.317820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:02.481419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.316226+0000","last_change":"2024-05-09T02:07:48.190305+0000","last_active":"2024-05-09T02:08:25.316226+0000","last_peered":"2024-05-09T02:08:25.316226+0000","last_clean":"2024-05-09T02:08:25.316226+0000","last_became_active":"2024-05-09T02:07:48.189971+0000","last_became_peered":"2024-05-09T02:07:48.189971+0000","last_unstale":"2024-05-09T02:08:25.316226+0000","last_undegraded":"2024-05-09T02:08:25.316226+0000","last_fullsized":"2024-05-09T02:08:25.316226+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:23.565092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":234,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:49.412746+0000","last_change":"2024-05-09T02:07:45.164273+0000","last_active":"2024-05-09T02:09:49.412746+0000","last_peered":"2024-05-09T02:09:49.412746+0000","last_clean":"2024-05-09T02:09:49.412746+0000","last_became_active":"2024-05-09T02:07:45.163183+0000","last_became_peered":"2024-05-09T02:07:45.163183+0000","last_unstale":"2024-05-09T02:09:49.412746+0000","last_undegraded":"2024-05-09T02:09:49.412746+0000","last_fullsized":"2024-05-09T02:09:49.412746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:11.395177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315668+0000","last_change":"2024-05-09T02:07:48.202995+0000","last_active":"2024-05-09T02:08:25.315668+0000","last_peered":"2024-05-09T02:08:25.315668+0000","last_clean":"2024-05-09T02:08:25.315668+0000","last_became_active":"2024-05-09T02:07:48.200915+0000","last_became_peered":"2024-05-09T02:07:48.200915+0000","last_unstale":"2024-05-09T02:08:25.315668+0000","last_undegraded":"2024-05-09T02:08:25.315668+0000","last_fullsized":"2024-05-09T02:08:25.315668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:25.555725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315988+0000","last_change":"2024-05-09T02:07:45.166221+0000","last_active":"2024-05-09T02:08:25.315988+0000","last_peered":"2024-05-09T02:08:25.315988+0000","last_clean":"2024-05-09T02:08:25.315988+0000","last_became_active":"2024-05-09T02:07:45.165813+0000","last_became_peered":"2024-05-09T02:07:45.165813+0000","last_unstale":"2024-05-09T02:08:25.315988+0000","last_undegraded":"2024-05-09T02:08:25.315988+0000","last_fullsized":"2024-05-09T02:08:25.315988+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:01.480828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317355+0000","last_change":"2024-05-09T02:07:48.201475+0000","last_active":"2024-05-09T02:08:25.317355+0000","last_peered":"2024-05-09T02:08:25.317355+0000","last_clean":"2024-05-09T02:08:25.317355+0000","last_became_active":"2024-05-09T02:07:48.201129+0000","last_became_peered":"2024-05-09T02:07:48.201129+0000","last_unstale":"2024-05-09T02:08:25.317355+0000","last_undegraded":"2024-05-09T02:08:25.317355+0000","last_fullsized":"2024-05-09T02:08:25.317355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:51.726752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318564+0000","last_change":"2024-05-09T02:07:45.185045+0000","last_active":"2024-05-09T02:08:25.318564+0000","last_peered":"2024-05-09T02:08:25.318564+0000","last_clean":"2024-05-09T02:08:25.318564+0000","last_became_active":"2024-05-09T02:07:45.184592+0000","last_became_peered":"2024-05-09T02:07:45.184592+0000","last_unstale":"2024-05-09T02:08:25.318564+0000","last_undegraded":"2024-05-09T02:08:25.318564+0000","last_fullsized":"2024-05-09T02:08:25.318564+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:56.220022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315185+0000","last_change":"2024-05-09T02:07:48.199873+0000","last_active":"2024-05-09T02:08:25.315185+0000","last_peered":"2024-05-09T02:08:25.315185+0000","last_clean":"2024-05-09T02:08:25.315185+0000","last_became_active":"2024-05-09T02:07:48.199502+0000","last_became_peered":"2024-05-09T02:07:48.199502+0000","last_unstale":"2024-05-09T02:08:25.315185+0000","last_undegraded":"2024-05-09T02:08:25.315185+0000","last_fullsized":"2024-05-09T02:08:25.315185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:15.406904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'32","reported_seq":271,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:49.412710+0000","last_change":"2024-05-09T02:07:45.166167+0000","last_active":"2024-05-09T02:09:49.412710+0000","last_peered":"2024-05-09T02:09:49.412710+0000","last_clean":"2024-05-09T02:09:49.412710+0000","last_became_active":"2024-05-09T02:07:45.165800+0000","last_became_peered":"2024-05-09T02:07:45.165800+0000","last_unstale":"2024-05-09T02:09:49.412710+0000","last_undegraded":"2024-05-09T02:09:49.412710+0000","last_fullsized":"2024-05-09T02:09:49.412710+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:03.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61826,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":213,"num_read_kb":166,"num_write":34,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310662+0000","last_change":"2024-05-09T02:07:48.187809+0000","last_active":"2024-05-09T02:08:25.310662+0000","last_peered":"2024-05-09T02:08:25.310662+0000","last_clean":"2024-05-09T02:08:25.310662+0000","last_became_active":"2024-05-09T02:07:48.186939+0000","last_became_peered":"2024-05-09T02:07:48.186939+0000","last_unstale":"2024-05-09T02:08:25.310662+0000","last_undegraded":"2024-05-09T02:08:25.310662+0000","last_fullsized":"2024-05-09T02:08:25.310662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:13:15.600343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315617+0000","last_change":"2024-05-09T02:07:45.163756+0000","last_active":"2024-05-09T02:08:25.315617+0000","last_peered":"2024-05-09T02:08:25.315617+0000","last_clean":"2024-05-09T02:08:25.315617+0000","last_became_active":"2024-05-09T02:07:45.163388+0000","last_became_peered":"2024-05-09T02:07:45.163388+0000","last_unstale":"2024-05-09T02:08:25.315617+0000","last_undegraded":"2024-05-09T02:08:25.315617+0000","last_fullsized":"2024-05-09T02:08:25.315617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:41.937148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309348+0000","last_change":"2024-05-09T02:07:48.195218+0000","last_active":"2024-05-09T02:08:25.309348+0000","last_peered":"2024-05-09T02:08:25.309348+0000","last_clean":"2024-05-09T02:08:25.309348+0000","last_became_active":"2024-05-09T02:07:48.194895+0000","last_became_peered":"2024-05-09T02:07:48.194895+0000","last_unstale":"2024-05-09T02:08:25.309348+0000","last_undegraded":"2024-05-09T02:08:25.309348+0000","last_fullsized":"2024-05-09T02:08:25.309348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:49.086594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318180+0000","last_change":"2024-05-09T02:07:45.158039+0000","last_active":"2024-05-09T02:08:25.318180+0000","last_peered":"2024-05-09T02:08:25.318180+0000","last_clean":"2024-05-09T02:08:25.318180+0000","last_became_active":"2024-05-09T02:07:45.157255+0000","last_became_peered":"2024-05-09T02:07:45.157255+0000","last_unstale":"2024-05-09T02:08:25.318180+0000","last_undegraded":"2024-05-09T02:08:25.318180+0000","last_fullsized":"2024-05-09T02:08:25.318180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:56.513999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313958+0000","last_change":"2024-05-09T02:07:48.200150+0000","last_active":"2024-05-09T02:08:25.313958+0000","last_peered":"2024-05-09T02:08:25.313958+0000","last_clean":"2024-05-09T02:08:25.313958+0000","last_became_active":"2024-05-09T02:07:48.199336+0000","last_became_peered":"2024-05-09T02:07:48.199336+0000","last_unstale":"2024-05-09T02:08:25.313958+0000","last_undegraded":"2024-05-09T02:08:25.313958+0000","last_fullsized":"2024-05-09T02:08:25.313958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:45.699119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":133,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:44.405856+0000","last_change":"2024-05-09T02:07:45.165476+0000","last_active":"2024-05-09T02:09:44.405856+0000","last_peered":"2024-05-09T02:09:44.405856+0000","last_clean":"2024-05-09T02:09:44.405856+0000","last_became_active":"2024-05-09T02:07:45.165107+0000","last_became_peered":"2024-05-09T02:07:45.165107+0000","last_unstale":"2024-05-09T02:09:44.405856+0000","last_undegraded":"2024-05-09T02:09:44.405856+0000","last_fullsized":"2024-05-09T02:09:44.405856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:15:33.518982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313557+0000","last_change":"2024-05-09T02:07:48.193487+0000","last_active":"2024-05-09T02:08:25.313557+0000","last_peered":"2024-05-09T02:08:25.313557+0000","last_clean":"2024-05-09T02:08:25.313557+0000","last_became_active":"2024-05-09T02:07:48.193108+0000","last_became_peered":"2024-05-09T02:07:48.193108+0000","last_unstale":"2024-05-09T02:08:25.313557+0000","last_undegraded":"2024-05-09T02:08:25.313557+0000","last_fullsized":"2024-05-09T02:08:25.313557+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:11.895058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.335525+0000","last_change":"2024-05-09T02:08:25.335525+0000","last_active":"2024-05-09T02:08:25.335525+0000","last_peered":"2024-05-09T02:08:25.335525+0000","last_clean":"2024-05-09T02:08:25.335525+0000","last_became_active":"2024-05-09T02:08:25.319746+0000","last_became_peered":"2024-05-09T02:08:25.319746+0000","last_unstale":"2024-05-09T02:08:25.335525+0000","last_undegraded":"2024-05-09T02:08:25.335525+0000","last_fullsized":"2024-05-09T02:08:25.335525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:41.774066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317433+0000","last_change":"2024-05-09T02:07:48.196125+0000","last_active":"2024-05-09T02:08:25.317433+0000","last_peered":"2024-05-09T02:08:25.317433+0000","last_clean":"2024-05-09T02:08:25.317433+0000","last_became_active":"2024-05-09T02:07:48.195737+0000","last_became_peered":"2024-05-09T02:07:48.195737+0000","last_unstale":"2024-05-09T02:08:25.317433+0000","last_undegraded":"2024-05-09T02:08:25.317433+0000","last_fullsized":"2024-05-09T02:08:25.317433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:43.264720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310367+0000","last_change":"2024-05-09T02:07:45.163791+0000","last_active":"2024-05-09T02:08:25.310367+0000","last_peered":"2024-05-09T02:08:25.310367+0000","last_clean":"2024-05-09T02:08:25.310367+0000","last_became_active":"2024-05-09T02:07:45.163129+0000","last_became_peered":"2024-05-09T02:07:45.163129+0000","last_unstale":"2024-05-09T02:08:25.310367+0000","last_undegraded":"2024-05-09T02:08:25.310367+0000","last_fullsized":"2024-05-09T02:08:25.310367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:48.562850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310748+0000","last_change":"2024-05-09T02:07:48.197782+0000","last_active":"2024-05-09T02:08:25.310748+0000","last_peered":"2024-05-09T02:08:25.310748+0000","last_clean":"2024-05-09T02:08:25.310748+0000","last_became_active":"2024-05-09T02:07:48.197353+0000","last_became_peered":"2024-05-09T02:07:48.197353+0000","last_unstale":"2024-05-09T02:08:25.310748+0000","last_undegraded":"2024-05-09T02:08:25.310748+0000","last_fullsized":"2024-05-09T02:08:25.310748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:17.053571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315220+0000","last_change":"2024-05-09T02:07:45.187028+0000","last_active":"2024-05-09T02:08:25.315220+0000","last_peered":"2024-05-09T02:08:25.315220+0000","last_clean":"2024-05-09T02:08:25.315220+0000","last_became_active":"2024-05-09T02:07:45.186639+0000","last_became_peered":"2024-05-09T02:07:45.186639+0000","last_unstale":"2024-05-09T02:08:25.315220+0000","last_undegraded":"2024-05-09T02:08:25.315220+0000","last_fullsized":"2024-05-09T02:08:25.315220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:35.834374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.308184+0000","last_change":"2024-05-09T02:07:48.207150+0000","last_active":"2024-05-09T02:08:25.308184+0000","last_peered":"2024-05-09T02:08:25.308184+0000","last_clean":"2024-05-09T02:08:25.308184+0000","last_became_active":"2024-05-09T02:07:48.206797+0000","last_became_peered":"2024-05-09T02:07:48.206797+0000","last_unstale":"2024-05-09T02:08:25.308184+0000","last_undegraded":"2024-05-09T02:08:25.308184+0000","last_fullsized":"2024-05-09T02:08:25.308184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:37.328133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":135,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:49.412069+0000","last_change":"2024-05-09T02:07:45.165422+0000","last_active":"2024-05-09T02:09:49.412069+0000","last_peered":"2024-05-09T02:09:49.412069+0000","last_clean":"2024-05-09T02:09:49.412069+0000","last_became_active":"2024-05-09T02:07:45.164998+0000","last_became_peered":"2024-05-09T02:07:45.164998+0000","last_unstale":"2024-05-09T02:09:49.412069+0000","last_undegraded":"2024-05-09T02:09:49.412069+0000","last_fullsized":"2024-05-09T02:09:49.412069+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:49.124323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.307961+0000","last_change":"2024-05-09T02:07:48.206538+0000","last_active":"2024-05-09T02:08:25.307961+0000","last_peered":"2024-05-09T02:08:25.307961+0000","last_clean":"2024-05-09T02:08:25.307961+0000","last_became_active":"2024-05-09T02:07:48.206061+0000","last_became_peered":"2024-05-09T02:07:48.206061+0000","last_unstale":"2024-05-09T02:08:25.307961+0000","last_undegraded":"2024-05-09T02:08:25.307961+0000","last_fullsized":"2024-05-09T02:08:25.307961+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:24.075091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318188+0000","last_change":"2024-05-09T02:07:48.202235+0000","last_active":"2024-05-09T02:08:25.318188+0000","last_peered":"2024-05-09T02:08:25.318188+0000","last_clean":"2024-05-09T02:08:25.318188+0000","last_became_active":"2024-05-09T02:07:48.200891+0000","last_became_peered":"2024-05-09T02:07:48.200891+0000","last_unstale":"2024-05-09T02:08:25.318188+0000","last_undegraded":"2024-05-09T02:08:25.318188+0000","last_fullsized":"2024-05-09T02:08:25.318188+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:39.061267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":133,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:47.409077+0000","last_change":"2024-05-09T02:07:45.184582+0000","last_active":"2024-05-09T02:09:47.409077+0000","last_peered":"2024-05-09T02:09:47.409077+0000","last_clean":"2024-05-09T02:09:47.409077+0000","last_became_active":"2024-05-09T02:07:45.184187+0000","last_became_peered":"2024-05-09T02:07:45.184187+0000","last_unstale":"2024-05-09T02:09:47.409077+0000","last_undegraded":"2024-05-09T02:09:47.409077+0000","last_fullsized":"2024-05-09T02:09:47.409077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:11.343797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314798+0000","last_change":"2024-05-09T02:07:48.199247+0000","last_active":"2024-05-09T02:08:25.314798+0000","last_peered":"2024-05-09T02:08:25.314798+0000","last_clean":"2024-05-09T02:08:25.314798+0000","last_became_active":"2024-05-09T02:07:48.198121+0000","last_became_peered":"2024-05-09T02:07:48.198121+0000","last_unstale":"2024-05-09T02:08:25.314798+0000","last_undegraded":"2024-05-09T02:08:25.314798+0000","last_fullsized":"2024-05-09T02:08:25.314798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:45:12.885616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317973+0000","last_change":"2024-05-09T02:07:45.163750+0000","last_active":"2024-05-09T02:08:25.317973+0000","last_peered":"2024-05-09T02:08:25.317973+0000","last_clean":"2024-05-09T02:08:25.317973+0000","last_became_active":"2024-05-09T02:07:45.163039+0000","last_became_peered":"2024-05-09T02:07:45.163039+0000","last_unstale":"2024-05-09T02:08:25.317973+0000","last_undegraded":"2024-05-09T02:08:25.317973+0000","last_fullsized":"2024-05-09T02:08:25.317973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:06.794514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":100,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314191+0000","last_change":"2024-05-09T02:06:20.334734+0000","last_active":"2024-05-09T02:08:25.314191+0000","last_peered":"2024-05-09T02:08:25.314191+0000","last_clean":"2024-05-09T02:08:25.314191+0000","last_became_active":"2024-05-09T02:06:20.329198+0000","last_became_peered":"2024-05-09T02:06:20.329198+0000","last_unstale":"2024-05-09T02:08:25.314191+0000","last_undegraded":"2024-05-09T02:08:25.314191+0000","last_fullsized":"2024-05-09T02:08:25.314191+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:02:50.632016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:02:50.632016+0000","last_clean_scrub_stamp":"2024-05-09T02:02:50.632016+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:08:15.495982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.321187+0000","last_change":"2024-05-09T02:08:25.321187+0000","last_active":"2024-05-09T02:08:25.321187+0000","last_peered":"2024-05-09T02:08:25.321187+0000","last_clean":"2024-05-09T02:08:25.321187+0000","last_became_active":"2024-05-09T02:08:25.320693+0000","last_became_peered":"2024-05-09T02:08:25.320693+0000","last_unstale":"2024-05-09T02:08:25.321187+0000","last_undegraded":"2024-05-09T02:08:25.321187+0000","last_fullsized":"2024-05-09T02:08:25.321187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:00.176016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314470+0000","last_change":"2024-05-09T02:07:45.158053+0000","last_active":"2024-05-09T02:08:25.314470+0000","last_peered":"2024-05-09T02:08:25.314470+0000","last_clean":"2024-05-09T02:08:25.314470+0000","last_became_active":"2024-05-09T02:07:45.157619+0000","last_became_peered":"2024-05-09T02:07:45.157619+0000","last_unstale":"2024-05-09T02:08:25.314470+0000","last_undegraded":"2024-05-09T02:08:25.314470+0000","last_fullsized":"2024-05-09T02:08:25.314470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:01.618475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.341770+0000","last_change":"2024-05-09T02:08:25.341377+0000","last_active":"2024-05-09T02:08:25.341770+0000","last_peered":"2024-05-09T02:08:25.341770+0000","last_clean":"2024-05-09T02:08:25.341770+0000","last_became_active":"2024-05-09T02:08:25.318683+0000","last_became_peered":"2024-05-09T02:08:25.318683+0000","last_unstale":"2024-05-09T02:08:25.341770+0000","last_undegraded":"2024-05-09T02:08:25.341770+0000","last_fullsized":"2024-05-09T02:08:25.341770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:12.456338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309370+0000","last_change":"2024-05-09T02:07:48.186892+0000","last_active":"2024-05-09T02:08:25.309370+0000","last_peered":"2024-05-09T02:08:25.309370+0000","last_clean":"2024-05-09T02:08:25.309370+0000","last_became_active":"2024-05-09T02:07:48.186543+0000","last_became_peered":"2024-05-09T02:07:48.186543+0000","last_unstale":"2024-05-09T02:08:25.309370+0000","last_undegraded":"2024-05-09T02:08:25.309370+0000","last_fullsized":"2024-05-09T02:08:25.309370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:01.210504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'15","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:31.205313+0000","last_change":"2024-05-09T02:07:45.158007+0000","last_active":"2024-05-09T02:08:31.205313+0000","last_peered":"2024-05-09T02:08:31.205313+0000","last_clean":"2024-05-09T02:08:31.205313+0000","last_became_active":"2024-05-09T02:07:45.157612+0000","last_became_peered":"2024-05-09T02:07:45.157612+0000","last_unstale":"2024-05-09T02:08:31.205313+0000","last_undegraded":"2024-05-09T02:08:31.205313+0000","last_fullsized":"2024-05-09T02:08:31.205313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:01.526756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314093+0000","last_change":"2024-05-09T02:07:48.186065+0000","last_active":"2024-05-09T02:08:25.314093+0000","last_peered":"2024-05-09T02:08:25.314093+0000","last_clean":"2024-05-09T02:08:25.314093+0000","last_became_active":"2024-05-09T02:07:48.185746+0000","last_became_peered":"2024-05-09T02:07:48.185746+0000","last_unstale":"2024-05-09T02:08:25.314093+0000","last_undegraded":"2024-05-09T02:08:25.314093+0000","last_fullsized":"2024-05-09T02:08:25.314093+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:09.570261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317499+0000","last_change":"2024-05-09T02:07:46.005439+0000","last_active":"2024-05-09T02:08:25.317499+0000","last_peered":"2024-05-09T02:08:25.317499+0000","last_clean":"2024-05-09T02:08:25.317499+0000","last_became_active":"2024-05-09T02:07:46.005087+0000","last_became_peered":"2024-05-09T02:07:46.005087+0000","last_unstale":"2024-05-09T02:08:25.317499+0000","last_undegraded":"2024-05-09T02:08:25.317499+0000","last_fullsized":"2024-05-09T02:08:25.317499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:08.543310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315347+0000","last_change":"2024-05-09T02:07:48.194062+0000","last_active":"2024-05-09T02:08:25.315347+0000","last_peered":"2024-05-09T02:08:25.315347+0000","last_clean":"2024-05-09T02:08:25.315347+0000","last_became_active":"2024-05-09T02:07:48.193712+0000","last_became_peered":"2024-05-09T02:07:48.193712+0000","last_unstale":"2024-05-09T02:08:25.315347+0000","last_undegraded":"2024-05-09T02:08:25.315347+0000","last_fullsized":"2024-05-09T02:08:25.315347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:48.394898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315411+0000","last_change":"2024-05-09T02:07:45.184821+0000","last_active":"2024-05-09T02:08:25.315411+0000","last_peered":"2024-05-09T02:08:25.315411+0000","last_clean":"2024-05-09T02:08:25.315411+0000","last_became_active":"2024-05-09T02:07:45.184437+0000","last_became_peered":"2024-05-09T02:07:45.184437+0000","last_unstale":"2024-05-09T02:08:25.315411+0000","last_undegraded":"2024-05-09T02:08:25.315411+0000","last_fullsized":"2024-05-09T02:08:25.315411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:30.864171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309993+0000","last_change":"2024-05-09T02:07:48.187635+0000","last_active":"2024-05-09T02:08:25.309993+0000","last_peered":"2024-05-09T02:08:25.309993+0000","last_clean":"2024-05-09T02:08:25.309993+0000","last_became_active":"2024-05-09T02:07:48.187071+0000","last_became_peered":"2024-05-09T02:07:48.187071+0000","last_unstale":"2024-05-09T02:08:25.309993+0000","last_undegraded":"2024-05-09T02:08:25.309993+0000","last_fullsized":"2024-05-09T02:08:25.309993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:10.716715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315158+0000","last_change":"2024-05-09T02:07:45.162166+0000","last_active":"2024-05-09T02:08:25.315158+0000","last_peered":"2024-05-09T02:08:25.315158+0000","last_clean":"2024-05-09T02:08:25.315158+0000","last_became_active":"2024-05-09T02:07:45.161787+0000","last_became_peered":"2024-05-09T02:07:45.161787+0000","last_unstale":"2024-05-09T02:08:25.315158+0000","last_undegraded":"2024-05-09T02:08:25.315158+0000","last_fullsized":"2024-05-09T02:08:25.315158+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:17.925214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310341+0000","last_change":"2024-05-09T02:07:48.202980+0000","last_active":"2024-05-09T02:08:25.310341+0000","last_peered":"2024-05-09T02:08:25.310341+0000","last_clean":"2024-05-09T02:08:25.310341+0000","last_became_active":"2024-05-09T02:07:48.202609+0000","last_became_peered":"2024-05-09T02:07:48.202609+0000","last_unstale":"2024-05-09T02:08:25.310341+0000","last_undegraded":"2024-05-09T02:08:25.310341+0000","last_fullsized":"2024-05-09T02:08:25.310341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:59.993451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.326699+0000","last_change":"2024-05-09T02:08:25.326142+0000","last_active":"2024-05-09T02:08:25.326699+0000","last_peered":"2024-05-09T02:08:25.326699+0000","last_clean":"2024-05-09T02:08:25.326699+0000","last_became_active":"2024-05-09T02:08:25.317666+0000","last_became_peered":"2024-05-09T02:08:25.317666+0000","last_unstale":"2024-05-09T02:08:25.326699+0000","last_undegraded":"2024-05-09T02:08:25.326699+0000","last_fullsized":"2024-05-09T02:08:25.326699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:26.178211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315560+0000","last_change":"2024-05-09T02:07:48.202219+0000","last_active":"2024-05-09T02:08:25.315560+0000","last_peered":"2024-05-09T02:08:25.315560+0000","last_clean":"2024-05-09T02:08:25.315560+0000","last_became_active":"2024-05-09T02:07:48.201819+0000","last_became_peered":"2024-05-09T02:07:48.201819+0000","last_unstale":"2024-05-09T02:08:25.315560+0000","last_undegraded":"2024-05-09T02:08:25.315560+0000","last_fullsized":"2024-05-09T02:08:25.315560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:51.573359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311162+0000","last_change":"2024-05-09T02:07:45.162228+0000","last_active":"2024-05-09T02:08:25.311162+0000","last_peered":"2024-05-09T02:08:25.311162+0000","last_clean":"2024-05-09T02:08:25.311162+0000","last_became_active":"2024-05-09T02:07:45.161847+0000","last_became_peered":"2024-05-09T02:07:45.161847+0000","last_unstale":"2024-05-09T02:08:25.311162+0000","last_undegraded":"2024-05-09T02:08:25.311162+0000","last_fullsized":"2024-05-09T02:08:25.311162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:47.459709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314196+0000","last_change":"2024-05-09T02:07:48.201401+0000","last_active":"2024-05-09T02:08:25.314196+0000","last_peered":"2024-05-09T02:08:25.314196+0000","last_clean":"2024-05-09T02:08:25.314196+0000","last_became_active":"2024-05-09T02:07:48.201040+0000","last_became_peered":"2024-05-09T02:07:48.201040+0000","last_unstale":"2024-05-09T02:08:25.314196+0000","last_undegraded":"2024-05-09T02:08:25.314196+0000","last_fullsized":"2024-05-09T02:08:25.314196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:58.038690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":130,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:43.405348+0000","last_change":"2024-05-09T02:07:46.005273+0000","last_active":"2024-05-09T02:09:43.405348+0000","last_peered":"2024-05-09T02:09:43.405348+0000","last_clean":"2024-05-09T02:09:43.405348+0000","last_became_active":"2024-05-09T02:07:46.004732+0000","last_became_peered":"2024-05-09T02:07:46.004732+0000","last_unstale":"2024-05-09T02:09:43.405348+0000","last_undegraded":"2024-05-09T02:09:43.405348+0000","last_fullsized":"2024-05-09T02:09:43.405348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:55.576955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315747+0000","last_change":"2024-05-09T02:07:48.197032+0000","last_active":"2024-05-09T02:08:25.315747+0000","last_peered":"2024-05-09T02:08:25.315747+0000","last_clean":"2024-05-09T02:08:25.315747+0000","last_became_active":"2024-05-09T02:07:48.196553+0000","last_became_peered":"2024-05-09T02:07:48.196553+0000","last_unstale":"2024-05-09T02:08:25.315747+0000","last_undegraded":"2024-05-09T02:08:25.315747+0000","last_fullsized":"2024-05-09T02:08:25.315747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:50:25.081024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318767+0000","last_change":"2024-05-09T02:07:45.166456+0000","last_active":"2024-05-09T02:08:25.318767+0000","last_peered":"2024-05-09T02:08:25.318767+0000","last_clean":"2024-05-09T02:08:25.318767+0000","last_became_active":"2024-05-09T02:07:45.166132+0000","last_became_peered":"2024-05-09T02:07:45.166132+0000","last_unstale":"2024-05-09T02:08:25.318767+0000","last_undegraded":"2024-05-09T02:08:25.318767+0000","last_fullsized":"2024-05-09T02:08:25.318767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:15.701802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.324526+0000","last_change":"2024-05-09T02:08:25.323303+0000","last_active":"2024-05-09T02:08:25.324526+0000","last_peered":"2024-05-09T02:08:25.324526+0000","last_clean":"2024-05-09T02:08:25.324526+0000","last_became_active":"2024-05-09T02:08:25.321752+0000","last_became_peered":"2024-05-09T02:08:25.321752+0000","last_unstale":"2024-05-09T02:08:25.324526+0000","last_undegraded":"2024-05-09T02:08:25.324526+0000","last_fullsized":"2024-05-09T02:08:25.324526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:17.791737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309540+0000","last_change":"2024-05-09T02:07:45.155480+0000","last_active":"2024-05-09T02:08:25.309540+0000","last_peered":"2024-05-09T02:08:25.309540+0000","last_clean":"2024-05-09T02:08:25.309540+0000","last_became_active":"2024-05-09T02:07:45.155085+0000","last_became_peered":"2024-05-09T02:07:45.155085+0000","last_unstale":"2024-05-09T02:08:25.309540+0000","last_undegraded":"2024-05-09T02:08:25.309540+0000","last_fullsized":"2024-05-09T02:08:25.309540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:05.547857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314367+0000","last_change":"2024-05-09T02:07:48.194480+0000","last_active":"2024-05-09T02:08:25.314367+0000","last_peered":"2024-05-09T02:08:25.314367+0000","last_clean":"2024-05-09T02:08:25.314367+0000","last_became_active":"2024-05-09T02:07:48.194136+0000","last_became_peered":"2024-05-09T02:07:48.194136+0000","last_unstale":"2024-05-09T02:08:25.314367+0000","last_undegraded":"2024-05-09T02:08:25.314367+0000","last_fullsized":"2024-05-09T02:08:25.314367+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:09.177762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310492+0000","last_change":"2024-05-09T02:07:48.195152+0000","last_active":"2024-05-09T02:08:25.310492+0000","last_peered":"2024-05-09T02:08:25.310492+0000","last_clean":"2024-05-09T02:08:25.310492+0000","last_became_active":"2024-05-09T02:07:48.194779+0000","last_became_peered":"2024-05-09T02:07:48.194779+0000","last_unstale":"2024-05-09T02:08:25.310492+0000","last_undegraded":"2024-05-09T02:08:25.310492+0000","last_fullsized":"2024-05-09T02:08:25.310492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:39.345452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.322235+0000","last_change":"2024-05-09T02:08:25.322235+0000","last_active":"2024-05-09T02:08:25.322235+0000","last_peered":"2024-05-09T02:08:25.322235+0000","last_clean":"2024-05-09T02:08:25.322235+0000","last_became_active":"2024-05-09T02:08:25.321674+0000","last_became_peered":"2024-05-09T02:08:25.321674+0000","last_unstale":"2024-05-09T02:08:25.322235+0000","last_undegraded":"2024-05-09T02:08:25.322235+0000","last_fullsized":"2024-05-09T02:08:25.322235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:52.671669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315466+0000","last_change":"2024-05-09T02:07:48.195710+0000","last_active":"2024-05-09T02:08:25.315466+0000","last_peered":"2024-05-09T02:08:25.315466+0000","last_clean":"2024-05-09T02:08:25.315466+0000","last_became_active":"2024-05-09T02:07:48.195332+0000","last_became_peered":"2024-05-09T02:07:48.195332+0000","last_unstale":"2024-05-09T02:08:25.315466+0000","last_undegraded":"2024-05-09T02:08:25.315466+0000","last_fullsized":"2024-05-09T02:08:25.315466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:12:17.904166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315787+0000","last_change":"2024-05-09T02:07:48.200791+0000","last_active":"2024-05-09T02:08:25.315787+0000","last_peered":"2024-05-09T02:08:25.315787+0000","last_clean":"2024-05-09T02:08:25.315787+0000","last_became_active":"2024-05-09T02:07:48.199916+0000","last_became_peered":"2024-05-09T02:07:48.199916+0000","last_unstale":"2024-05-09T02:08:25.315787+0000","last_undegraded":"2024-05-09T02:08:25.315787+0000","last_fullsized":"2024-05-09T02:08:25.315787+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:32.692375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309571+0000","last_change":"2024-05-09T02:07:48.191662+0000","last_active":"2024-05-09T02:08:25.309571+0000","last_peered":"2024-05-09T02:08:25.309571+0000","last_clean":"2024-05-09T02:08:25.309571+0000","last_became_active":"2024-05-09T02:07:48.191336+0000","last_became_peered":"2024-05-09T02:07:48.191336+0000","last_unstale":"2024-05-09T02:08:25.309571+0000","last_undegraded":"2024-05-09T02:08:25.309571+0000","last_fullsized":"2024-05-09T02:08:25.309571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:00.733611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311462+0000","last_change":"2024-05-09T02:07:48.195182+0000","last_active":"2024-05-09T02:08:25.311462+0000","last_peered":"2024-05-09T02:08:25.311462+0000","last_clean":"2024-05-09T02:08:25.311462+0000","last_became_active":"2024-05-09T02:07:48.194828+0000","last_became_peered":"2024-05-09T02:07:48.194828+0000","last_unstale":"2024-05-09T02:08:25.311462+0000","last_undegraded":"2024-05-09T02:08:25.311462+0000","last_fullsized":"2024-05-09T02:08:25.311462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:33.427409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315738+0000","last_change":"2024-05-09T02:07:48.199688+0000","last_active":"2024-05-09T02:08:25.315738+0000","last_peered":"2024-05-09T02:08:25.315738+0000","last_clean":"2024-05-09T02:08:25.315738+0000","last_became_active":"2024-05-09T02:07:48.199326+0000","last_became_peered":"2024-05-09T02:07:48.199326+0000","last_unstale":"2024-05-09T02:08:25.315738+0000","last_undegraded":"2024-05-09T02:08:25.315738+0000","last_fullsized":"2024-05-09T02:08:25.315738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:30.662738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313935+0000","last_change":"2024-05-09T02:07:48.190727+0000","last_active":"2024-05-09T02:08:25.313935+0000","last_peered":"2024-05-09T02:08:25.313935+0000","last_clean":"2024-05-09T02:08:25.313935+0000","last_became_active":"2024-05-09T02:07:48.190397+0000","last_became_peered":"2024-05-09T02:07:48.190397+0000","last_unstale":"2024-05-09T02:08:25.313935+0000","last_undegraded":"2024-05-09T02:08:25.313935+0000","last_fullsized":"2024-05-09T02:08:25.313935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:42.010669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317969+0000","last_change":"2024-05-09T02:07:48.200444+0000","last_active":"2024-05-09T02:08:25.317969+0000","last_peered":"2024-05-09T02:08:25.317969+0000","last_clean":"2024-05-09T02:08:25.317969+0000","last_became_active":"2024-05-09T02:07:48.200031+0000","last_became_peered":"2024-05-09T02:07:48.200031+0000","last_unstale":"2024-05-09T02:08:25.317969+0000","last_undegraded":"2024-05-09T02:08:25.317969+0000","last_fullsized":"2024-05-09T02:08:25.317969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:30.269792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315072+0000","last_change":"2024-05-09T02:07:48.203717+0000","last_active":"2024-05-09T02:08:25.315072+0000","last_peered":"2024-05-09T02:08:25.315072+0000","last_clean":"2024-05-09T02:08:25.315072+0000","last_became_active":"2024-05-09T02:07:48.203347+0000","last_became_peered":"2024-05-09T02:07:48.203347+0000","last_unstale":"2024-05-09T02:08:25.315072+0000","last_undegraded":"2024-05-09T02:08:25.315072+0000","last_fullsized":"2024-05-09T02:08:25.315072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:17.391504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.323029+0000","last_change":"2024-05-09T02:08:25.323029+0000","last_active":"2024-05-09T02:08:25.323029+0000","last_peered":"2024-05-09T02:08:25.323029+0000","last_clean":"2024-05-09T02:08:25.323029+0000","last_became_active":"2024-05-09T02:08:25.322524+0000","last_became_peered":"2024-05-09T02:08:25.322524+0000","last_unstale":"2024-05-09T02:08:25.323029+0000","last_undegraded":"2024-05-09T02:08:25.323029+0000","last_fullsized":"2024-05-09T02:08:25.323029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:25.017511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309673+0000","last_change":"2024-05-09T02:07:48.187802+0000","last_active":"2024-05-09T02:08:25.309673+0000","last_peered":"2024-05-09T02:08:25.309673+0000","last_clean":"2024-05-09T02:08:25.309673+0000","last_became_active":"2024-05-09T02:07:48.187478+0000","last_became_peered":"2024-05-09T02:07:48.187478+0000","last_unstale":"2024-05-09T02:08:25.309673+0000","last_undegraded":"2024-05-09T02:08:25.309673+0000","last_fullsized":"2024-05-09T02:08:25.309673+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:12.981386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310421+0000","last_change":"2024-05-09T02:07:48.192117+0000","last_active":"2024-05-09T02:08:25.310421+0000","last_peered":"2024-05-09T02:08:25.310421+0000","last_clean":"2024-05-09T02:08:25.310421+0000","last_became_active":"2024-05-09T02:07:48.191726+0000","last_became_peered":"2024-05-09T02:07:48.191726+0000","last_unstale":"2024-05-09T02:08:25.310421+0000","last_undegraded":"2024-05-09T02:08:25.310421+0000","last_fullsized":"2024-05-09T02:08:25.310421+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:07.735582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280595,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68639,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1045,"num_read_kb":602,"num_write":245,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678056,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":429664,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874284,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1082798,"data_compressed":41450,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037811,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":909312,"data_stored":429599,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":429748,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430269,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":1022378,"data_compressed":41450,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593794,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":429199,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":491938,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":430538,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":492046,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":428907,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953548,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":429563,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":1021665,"data_compressed":41450,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T02:09:51.849 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T02:09:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:52.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:52.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:52.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:52.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:52 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:52.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:52 smithi019 ceph-mon[30151]: pgmap v355: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:52 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:52 smithi059 ceph-mon[34781]: pgmap v355: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:52 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:52 smithi196 ceph-mon[34718]: pgmap v355: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:09:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.896+0000 7f857ac73700 1 -- 172.21.15.19:0/3932446589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8574107df0 msgr2=0x7f8574108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.896+0000 7f857ac73700 1 --2- 172.21.15.19:0/3932446589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8574107df0 0x7f8574108200 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f85600099c0 tx=0x7f8560009cd0 comp rx=0 tx=0).stop 2024-05-09T02:09:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.896+0000 7f857ac73700 1 -- 172.21.15.19:0/3932446589 shutdown_connections 2024-05-09T02:09:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.896+0000 7f857ac73700 1 --2- 172.21.15.19:0/3932446589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8574108740 0x7f85741100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.896+0000 7f857ac73700 1 --2- 172.21.15.19:0/3932446589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85741034c0 0x7f85741038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.896+0000 7f857ac73700 1 --2- 172.21.15.19:0/3932446589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8574107df0 0x7f8574108200 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.896+0000 7f857ac73700 1 -- 172.21.15.19:0/3932446589 >> 172.21.15.19:0/3932446589 conn(0x7f85740fedc0 msgr2=0x7f85741011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.896+0000 7f857ac73700 1 -- 172.21.15.19:0/3932446589 shutdown_connections 2024-05-09T02:09:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.896+0000 7f857ac73700 1 -- 172.21.15.19:0/3932446589 wait complete. 2024-05-09T02:09:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.897+0000 7f857ac73700 1 Processor -- start 2024-05-09T02:09:53.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.897+0000 7f857ac73700 1 -- start start 2024-05-09T02:09:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.897+0000 7f857ac73700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85741034c0 0x7f857419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.897+0000 7f857ac73700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8574107df0 0x7f857419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.897+0000 7f857ac73700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8574108740 0x7f857419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.897+0000 7f857ac73700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85741133a0 con 0x7f85741034c0 2024-05-09T02:09:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.897+0000 7f857ac73700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8574113220 con 0x7f8574107df0 2024-05-09T02:09:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.897+0000 7f857ac73700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8574113520 con 0x7f8574108740 2024-05-09T02:09:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.897+0000 7f8578a0f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85741034c0 0x7f857419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.897+0000 7f8578a0f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85741034c0 0x7f857419ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47384/0 (socket says 172.21.15.19:47384) 2024-05-09T02:09:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.897+0000 7f8578a0f700 1 -- 172.21.15.19:0/1779921513 learned_addr learned my addr 172.21.15.19:0/1779921513 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.897+0000 7f8579210700 1 --2- 172.21.15.19:0/1779921513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8574108740 0x7f857419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.897+0000 7f8573fff700 1 --2- 172.21.15.19:0/1779921513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8574107df0 0x7f857419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.898+0000 7f8578a0f700 1 -- 172.21.15.19:0/1779921513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8574108740 msgr2=0x7f857419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.898+0000 7f8578a0f700 1 --2- 172.21.15.19:0/1779921513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8574108740 0x7f857419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.898+0000 7f8578a0f700 1 -- 172.21.15.19:0/1779921513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8574107df0 msgr2=0x7f857419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.898+0000 7f8578a0f700 1 --2- 172.21.15.19:0/1779921513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8574107df0 0x7f857419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.898+0000 7f8578a0f700 1 -- 172.21.15.19:0/1779921513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857419fcd0 con 0x7f85741034c0 2024-05-09T02:09:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.898+0000 7f8578a0f700 1 --2- 172.21.15.19:0/1779921513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85741034c0 0x7f857419ae00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f856800e9a0 tx=0x7f856800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.898+0000 7f8571ffb700 1 -- 172.21.15.19:0/1779921513 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856800cbd0 con 0x7f85741034c0 2024-05-09T02:09:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.898+0000 7f857ac73700 1 -- 172.21.15.19:0/1779921513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857419ffc0 con 0x7f85741034c0 2024-05-09T02:09:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.898+0000 7f8571ffb700 1 -- 172.21.15.19:0/1779921513 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f856800cd70 con 0x7f85741034c0 2024-05-09T02:09:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.898+0000 7f8571ffb700 1 -- 172.21.15.19:0/1779921513 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568018680 con 0x7f85741034c0 2024-05-09T02:09:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.898+0000 7f857ac73700 1 -- 172.21.15.19:0/1779921513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85741a78e0 con 0x7f85741034c0 2024-05-09T02:09:53.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.900+0000 7f857ac73700 1 -- 172.21.15.19:0/1779921513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85740512c0 con 0x7f85741034c0 2024-05-09T02:09:53.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.901+0000 7f8571ffb700 1 -- 172.21.15.19:0/1779921513 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8568010430 con 0x7f85741034c0 2024-05-09T02:09:53.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.902+0000 7f8571ffb700 1 --2- 172.21.15.19:0/1779921513 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f85640b2990 0x7f85640b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:53.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.902+0000 7f8573fff700 1 --2- 172.21.15.19:0/1779921513 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f85640b2990 0x7f85640b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:53.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.902+0000 7f8571ffb700 1 -- 172.21.15.19:0/1779921513 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f85680e25a0 con 0x7f85741034c0 2024-05-09T02:09:53.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.903+0000 7f8573fff700 1 --2- 172.21.15.19:0/1779921513 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f85640b2990 0x7f85640b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8560000c00 tx=0x7f856001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:53.907+0000 7f8571ffb700 1 -- 172.21.15.19:0/1779921513 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f85680aa140 con 0x7f85741034c0 2024-05-09T02:09:54.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:54.184+0000 7f857ac73700 1 -- 172.21.15.19:0/1779921513 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85741013e0 con 0x7f85640b2990 2024-05-09T02:09:54.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:54.192+0000 7f8571ffb700 1 -- 172.21.15.19:0/1779921513 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259027 (secure 0 0 0) 0x7f85600663f0 con 0x7f85640b2990 2024-05-09T02:09:54.198 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:09:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:54.196+0000 7f857ac73700 1 -- 172.21.15.19:0/1779921513 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f85640b2990 msgr2=0x7f85640b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:54.196+0000 7f857ac73700 1 --2- 172.21.15.19:0/1779921513 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f85640b2990 0x7f85640b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8560000c00 tx=0x7f856001a040 comp rx=0 tx=0).stop 2024-05-09T02:09:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:54.196+0000 7f857ac73700 1 -- 172.21.15.19:0/1779921513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85741034c0 msgr2=0x7f857419ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:54.196+0000 7f857ac73700 1 --2- 172.21.15.19:0/1779921513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85741034c0 0x7f857419ae00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f856800e9a0 tx=0x7f856800ecb0 comp rx=0 tx=0).stop 2024-05-09T02:09:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:54.196+0000 7f857ac73700 1 -- 172.21.15.19:0/1779921513 shutdown_connections 2024-05-09T02:09:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:54.196+0000 7f857ac73700 1 --2- 172.21.15.19:0/1779921513 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f85640b2990 0x7f85640b4ea0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:54.196+0000 7f857ac73700 1 --2- 172.21.15.19:0/1779921513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8574108740 0x7f857419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:54.196+0000 7f857ac73700 1 --2- 172.21.15.19:0/1779921513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85741034c0 0x7f857419ae00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:54.196+0000 7f857ac73700 1 --2- 172.21.15.19:0/1779921513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8574107df0 0x7f857419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:54.196+0000 7f857ac73700 1 -- 172.21.15.19:0/1779921513 >> 172.21.15.19:0/1779921513 conn(0x7f85740fedc0 msgr2=0x7f85741096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:54.196+0000 7f857ac73700 1 -- 172.21.15.19:0/1779921513 shutdown_connections 2024-05-09T02:09:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:54.196+0000 7f857ac73700 1 -- 172.21.15.19:0/1779921513 wait complete. 2024-05-09T02:09:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-09T02:09:54.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:54 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:54.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:54 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:54.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:54 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:09:54.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:54 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:09:54.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:54 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:54.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:54 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:54.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:54 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:54 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:09:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:54 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:09:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:54 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:54 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:54 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:54 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:09:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:54 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:09:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:54 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:09:54.898 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-05-09T02:09:54.066041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659241,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1151,"num_read_kb":681,"num_write":370,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2952064,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376732,"kb_used_data":10896,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565092,"statfs":{"total":1151940427776,"available":1151554654208,"internally_reserved":0,"allocated":11157504,"data_stored":7118043,"data_compressed":176253,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003435"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311161+0000","last_change":"2024-05-09T02:07:48.203957+0000","last_active":"2024-05-09T02:08:25.311161+0000","last_peered":"2024-05-09T02:08:25.311161+0000","last_clean":"2024-05-09T02:08:25.311161+0000","last_became_active":"2024-05-09T02:07:48.203354+0000","last_became_peered":"2024-05-09T02:07:48.203354+0000","last_unstale":"2024-05-09T02:08:25.311161+0000","last_undegraded":"2024-05-09T02:08:25.311161+0000","last_fullsized":"2024-05-09T02:08:25.311161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:06.169693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315507+0000","last_change":"2024-05-09T02:07:48.192885+0000","last_active":"2024-05-09T02:08:25.315507+0000","last_peered":"2024-05-09T02:08:25.315507+0000","last_clean":"2024-05-09T02:08:25.315507+0000","last_became_active":"2024-05-09T02:07:48.192476+0000","last_became_peered":"2024-05-09T02:07:48.192476+0000","last_unstale":"2024-05-09T02:08:25.315507+0000","last_undegraded":"2024-05-09T02:08:25.315507+0000","last_fullsized":"2024-05-09T02:08:25.315507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:33.710309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314707+0000","last_change":"2024-05-09T02:07:48.198684+0000","last_active":"2024-05-09T02:08:25.314707+0000","last_peered":"2024-05-09T02:08:25.314707+0000","last_clean":"2024-05-09T02:08:25.314707+0000","last_became_active":"2024-05-09T02:07:48.198344+0000","last_became_peered":"2024-05-09T02:07:48.198344+0000","last_unstale":"2024-05-09T02:08:25.314707+0000","last_undegraded":"2024-05-09T02:08:25.314707+0000","last_fullsized":"2024-05-09T02:08:25.314707+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:19.990063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317946+0000","last_change":"2024-05-09T02:07:48.189209+0000","last_active":"2024-05-09T02:08:25.317946+0000","last_peered":"2024-05-09T02:08:25.317946+0000","last_clean":"2024-05-09T02:08:25.317946+0000","last_became_active":"2024-05-09T02:07:48.188744+0000","last_became_peered":"2024-05-09T02:07:48.188744+0000","last_unstale":"2024-05-09T02:08:25.317946+0000","last_undegraded":"2024-05-09T02:08:25.317946+0000","last_fullsized":"2024-05-09T02:08:25.317946+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:03:25.216849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311030+0000","last_change":"2024-05-09T02:07:48.187897+0000","last_active":"2024-05-09T02:08:25.311030+0000","last_peered":"2024-05-09T02:08:25.311030+0000","last_clean":"2024-05-09T02:08:25.311030+0000","last_became_active":"2024-05-09T02:07:48.187476+0000","last_became_peered":"2024-05-09T02:07:48.187476+0000","last_unstale":"2024-05-09T02:08:25.311030+0000","last_undegraded":"2024-05-09T02:08:25.311030+0000","last_fullsized":"2024-05-09T02:08:25.311030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:39.366520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.307870+0000","last_change":"2024-05-09T02:07:48.195436+0000","last_active":"2024-05-09T02:08:25.307870+0000","last_peered":"2024-05-09T02:08:25.307870+0000","last_clean":"2024-05-09T02:08:25.307870+0000","last_became_active":"2024-05-09T02:07:48.194982+0000","last_became_peered":"2024-05-09T02:07:48.194982+0000","last_unstale":"2024-05-09T02:08:25.307870+0000","last_undegraded":"2024-05-09T02:08:25.307870+0000","last_fullsized":"2024-05-09T02:08:25.307870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:19.746701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313666+0000","last_change":"2024-05-09T02:07:48.198458+0000","last_active":"2024-05-09T02:08:25.313666+0000","last_peered":"2024-05-09T02:08:25.313666+0000","last_clean":"2024-05-09T02:08:25.313666+0000","last_became_active":"2024-05-09T02:07:48.198089+0000","last_became_peered":"2024-05-09T02:07:48.198089+0000","last_unstale":"2024-05-09T02:08:25.313666+0000","last_undegraded":"2024-05-09T02:08:25.313666+0000","last_fullsized":"2024-05-09T02:08:25.313666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:37:44.999438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314425+0000","last_change":"2024-05-09T02:07:48.200294+0000","last_active":"2024-05-09T02:08:25.314425+0000","last_peered":"2024-05-09T02:08:25.314425+0000","last_clean":"2024-05-09T02:08:25.314425+0000","last_became_active":"2024-05-09T02:07:48.199949+0000","last_became_peered":"2024-05-09T02:07:48.199949+0000","last_unstale":"2024-05-09T02:08:25.314425+0000","last_undegraded":"2024-05-09T02:08:25.314425+0000","last_fullsized":"2024-05-09T02:08:25.314425+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:39.263849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315463+0000","last_change":"2024-05-09T02:07:48.196282+0000","last_active":"2024-05-09T02:08:25.315463+0000","last_peered":"2024-05-09T02:08:25.315463+0000","last_clean":"2024-05-09T02:08:25.315463+0000","last_became_active":"2024-05-09T02:07:48.195887+0000","last_became_peered":"2024-05-09T02:07:48.195887+0000","last_unstale":"2024-05-09T02:08:25.315463+0000","last_undegraded":"2024-05-09T02:08:25.315463+0000","last_fullsized":"2024-05-09T02:08:25.315463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:26.081867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.320636+0000","last_change":"2024-05-09T02:08:25.320636+0000","last_active":"2024-05-09T02:08:25.320636+0000","last_peered":"2024-05-09T02:08:25.320636+0000","last_clean":"2024-05-09T02:08:25.320636+0000","last_became_active":"2024-05-09T02:08:25.320155+0000","last_became_peered":"2024-05-09T02:08:25.320155+0000","last_unstale":"2024-05-09T02:08:25.320636+0000","last_undegraded":"2024-05-09T02:08:25.320636+0000","last_fullsized":"2024-05-09T02:08:25.320636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:55.728773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314733+0000","last_change":"2024-05-09T02:07:48.197649+0000","last_active":"2024-05-09T02:08:25.314733+0000","last_peered":"2024-05-09T02:08:25.314733+0000","last_clean":"2024-05-09T02:08:25.314733+0000","last_became_active":"2024-05-09T02:07:48.196884+0000","last_became_peered":"2024-05-09T02:07:48.196884+0000","last_unstale":"2024-05-09T02:08:25.314733+0000","last_undegraded":"2024-05-09T02:08:25.314733+0000","last_fullsized":"2024-05-09T02:08:25.314733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:23.851216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318021+0000","last_change":"2024-05-09T02:07:48.193568+0000","last_active":"2024-05-09T02:08:25.318021+0000","last_peered":"2024-05-09T02:08:25.318021+0000","last_clean":"2024-05-09T02:08:25.318021+0000","last_became_active":"2024-05-09T02:07:48.193117+0000","last_became_peered":"2024-05-09T02:07:48.193117+0000","last_unstale":"2024-05-09T02:08:25.318021+0000","last_undegraded":"2024-05-09T02:08:25.318021+0000","last_fullsized":"2024-05-09T02:08:25.318021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:34.371246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309662+0000","last_change":"2024-05-09T02:07:48.196314+0000","last_active":"2024-05-09T02:08:25.309662+0000","last_peered":"2024-05-09T02:08:25.309662+0000","last_clean":"2024-05-09T02:08:25.309662+0000","last_became_active":"2024-05-09T02:07:48.195988+0000","last_became_peered":"2024-05-09T02:07:48.195988+0000","last_unstale":"2024-05-09T02:08:25.309662+0000","last_undegraded":"2024-05-09T02:08:25.309662+0000","last_fullsized":"2024-05-09T02:08:25.309662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:08.556166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314018+0000","last_change":"2024-05-09T02:07:48.192974+0000","last_active":"2024-05-09T02:08:25.314018+0000","last_peered":"2024-05-09T02:08:25.314018+0000","last_clean":"2024-05-09T02:08:25.314018+0000","last_became_active":"2024-05-09T02:07:48.192599+0000","last_became_peered":"2024-05-09T02:07:48.192599+0000","last_unstale":"2024-05-09T02:08:25.314018+0000","last_undegraded":"2024-05-09T02:08:25.314018+0000","last_fullsized":"2024-05-09T02:08:25.314018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:37.192219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314434+0000","last_change":"2024-05-09T02:07:48.193208+0000","last_active":"2024-05-09T02:08:25.314434+0000","last_peered":"2024-05-09T02:08:25.314434+0000","last_clean":"2024-05-09T02:08:25.314434+0000","last_became_active":"2024-05-09T02:07:48.192654+0000","last_became_peered":"2024-05-09T02:07:48.192654+0000","last_unstale":"2024-05-09T02:08:25.314434+0000","last_undegraded":"2024-05-09T02:08:25.314434+0000","last_fullsized":"2024-05-09T02:08:25.314434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:55:42.870372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314811+0000","last_change":"2024-05-09T02:07:48.187568+0000","last_active":"2024-05-09T02:08:25.314811+0000","last_peered":"2024-05-09T02:08:25.314811+0000","last_clean":"2024-05-09T02:08:25.314811+0000","last_became_active":"2024-05-09T02:07:48.187159+0000","last_became_peered":"2024-05-09T02:07:48.187159+0000","last_unstale":"2024-05-09T02:08:25.314811+0000","last_undegraded":"2024-05-09T02:08:25.314811+0000","last_fullsized":"2024-05-09T02:08:25.314811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:04.994094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311915+0000","last_change":"2024-05-09T02:07:48.204406+0000","last_active":"2024-05-09T02:08:25.311915+0000","last_peered":"2024-05-09T02:08:25.311915+0000","last_clean":"2024-05-09T02:08:25.311915+0000","last_became_active":"2024-05-09T02:07:48.203592+0000","last_became_peered":"2024-05-09T02:07:48.203592+0000","last_unstale":"2024-05-09T02:08:25.311915+0000","last_undegraded":"2024-05-09T02:08:25.311915+0000","last_fullsized":"2024-05-09T02:08:25.311915+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:21.655807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317492+0000","last_change":"2024-05-09T02:07:48.195326+0000","last_active":"2024-05-09T02:08:25.317492+0000","last_peered":"2024-05-09T02:08:25.317492+0000","last_clean":"2024-05-09T02:08:25.317492+0000","last_became_active":"2024-05-09T02:07:48.194969+0000","last_became_peered":"2024-05-09T02:07:48.194969+0000","last_unstale":"2024-05-09T02:08:25.317492+0000","last_undegraded":"2024-05-09T02:08:25.317492+0000","last_fullsized":"2024-05-09T02:08:25.317492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:00.269753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315957+0000","last_change":"2024-05-09T02:07:48.190706+0000","last_active":"2024-05-09T02:08:25.315957+0000","last_peered":"2024-05-09T02:08:25.315957+0000","last_clean":"2024-05-09T02:08:25.315957+0000","last_became_active":"2024-05-09T02:07:48.190171+0000","last_became_peered":"2024-05-09T02:07:48.190171+0000","last_unstale":"2024-05-09T02:08:25.315957+0000","last_undegraded":"2024-05-09T02:08:25.315957+0000","last_fullsized":"2024-05-09T02:08:25.315957+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:19:31.333618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":147,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:53.416417+0000","last_change":"2024-05-09T02:07:45.158224+0000","last_active":"2024-05-09T02:09:53.416417+0000","last_peered":"2024-05-09T02:09:53.416417+0000","last_clean":"2024-05-09T02:09:53.416417+0000","last_became_active":"2024-05-09T02:07:45.157722+0000","last_became_peered":"2024-05-09T02:07:45.157722+0000","last_unstale":"2024-05-09T02:09:53.416417+0000","last_undegraded":"2024-05-09T02:09:53.416417+0000","last_fullsized":"2024-05-09T02:09:53.416417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:31.750699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315513+0000","last_change":"2024-05-09T02:07:48.197978+0000","last_active":"2024-05-09T02:08:25.315513+0000","last_peered":"2024-05-09T02:08:25.315513+0000","last_clean":"2024-05-09T02:08:25.315513+0000","last_became_active":"2024-05-09T02:07:48.197077+0000","last_became_peered":"2024-05-09T02:07:48.197077+0000","last_unstale":"2024-05-09T02:08:25.315513+0000","last_undegraded":"2024-05-09T02:08:25.315513+0000","last_fullsized":"2024-05-09T02:08:25.315513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:25.897904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315892+0000","last_change":"2024-05-09T02:07:45.185758+0000","last_active":"2024-05-09T02:08:25.315892+0000","last_peered":"2024-05-09T02:08:25.315892+0000","last_clean":"2024-05-09T02:08:25.315892+0000","last_became_active":"2024-05-09T02:07:45.185411+0000","last_became_peered":"2024-05-09T02:07:45.185411+0000","last_unstale":"2024-05-09T02:08:25.315892+0000","last_undegraded":"2024-05-09T02:08:25.315892+0000","last_fullsized":"2024-05-09T02:08:25.315892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:04.410523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317384+0000","last_change":"2024-05-09T02:07:48.199794+0000","last_active":"2024-05-09T02:08:25.317384+0000","last_peered":"2024-05-09T02:08:25.317384+0000","last_clean":"2024-05-09T02:08:25.317384+0000","last_became_active":"2024-05-09T02:07:48.199453+0000","last_became_peered":"2024-05-09T02:07:48.199453+0000","last_unstale":"2024-05-09T02:08:25.317384+0000","last_undegraded":"2024-05-09T02:08:25.317384+0000","last_fullsized":"2024-05-09T02:08:25.317384+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:09.735830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311011+0000","last_change":"2024-05-09T02:07:45.162221+0000","last_active":"2024-05-09T02:08:25.311011+0000","last_peered":"2024-05-09T02:08:25.311011+0000","last_clean":"2024-05-09T02:08:25.311011+0000","last_became_active":"2024-05-09T02:07:45.161748+0000","last_became_peered":"2024-05-09T02:07:45.161748+0000","last_unstale":"2024-05-09T02:08:25.311011+0000","last_undegraded":"2024-05-09T02:08:25.311011+0000","last_fullsized":"2024-05-09T02:08:25.311011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:38.837411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315182+0000","last_change":"2024-05-09T02:07:48.188741+0000","last_active":"2024-05-09T02:08:25.315182+0000","last_peered":"2024-05-09T02:08:25.315182+0000","last_clean":"2024-05-09T02:08:25.315182+0000","last_became_active":"2024-05-09T02:07:48.188329+0000","last_became_peered":"2024-05-09T02:07:48.188329+0000","last_unstale":"2024-05-09T02:08:25.315182+0000","last_undegraded":"2024-05-09T02:08:25.315182+0000","last_fullsized":"2024-05-09T02:08:25.315182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:46.773336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310908+0000","last_change":"2024-05-09T02:07:45.163573+0000","last_active":"2024-05-09T02:08:25.310908+0000","last_peered":"2024-05-09T02:08:25.310908+0000","last_clean":"2024-05-09T02:08:25.310908+0000","last_became_active":"2024-05-09T02:07:45.163162+0000","last_became_peered":"2024-05-09T02:07:45.163162+0000","last_unstale":"2024-05-09T02:08:25.310908+0000","last_undegraded":"2024-05-09T02:08:25.310908+0000","last_fullsized":"2024-05-09T02:08:25.310908+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:12.356586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318419+0000","last_change":"2024-05-09T02:07:48.203928+0000","last_active":"2024-05-09T02:08:25.318419+0000","last_peered":"2024-05-09T02:08:25.318419+0000","last_clean":"2024-05-09T02:08:25.318419+0000","last_became_active":"2024-05-09T02:07:48.203224+0000","last_became_peered":"2024-05-09T02:07:48.203224+0000","last_unstale":"2024-05-09T02:08:25.318419+0000","last_undegraded":"2024-05-09T02:08:25.318419+0000","last_fullsized":"2024-05-09T02:08:25.318419+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:00.568050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":137,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:50.414603+0000","last_change":"2024-05-09T02:07:45.160161+0000","last_active":"2024-05-09T02:09:50.414603+0000","last_peered":"2024-05-09T02:09:50.414603+0000","last_clean":"2024-05-09T02:09:50.414603+0000","last_became_active":"2024-05-09T02:07:45.159118+0000","last_became_peered":"2024-05-09T02:07:45.159118+0000","last_unstale":"2024-05-09T02:09:50.414603+0000","last_undegraded":"2024-05-09T02:09:50.414603+0000","last_fullsized":"2024-05-09T02:09:50.414603+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:04.510877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.319860+0000","last_change":"2024-05-09T02:08:25.319860+0000","last_active":"2024-05-09T02:08:25.319860+0000","last_peered":"2024-05-09T02:08:25.319860+0000","last_clean":"2024-05-09T02:08:25.319860+0000","last_became_active":"2024-05-09T02:08:25.319370+0000","last_became_peered":"2024-05-09T02:08:25.319370+0000","last_unstale":"2024-05-09T02:08:25.319860+0000","last_undegraded":"2024-05-09T02:08:25.319860+0000","last_fullsized":"2024-05-09T02:08:25.319860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:38.504057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.327069+0000","last_change":"2024-05-09T02:08:25.327069+0000","last_active":"2024-05-09T02:08:25.327069+0000","last_peered":"2024-05-09T02:08:25.327069+0000","last_clean":"2024-05-09T02:08:25.327069+0000","last_became_active":"2024-05-09T02:08:25.316629+0000","last_became_peered":"2024-05-09T02:08:25.316629+0000","last_unstale":"2024-05-09T02:08:25.327069+0000","last_undegraded":"2024-05-09T02:08:25.327069+0000","last_fullsized":"2024-05-09T02:08:25.327069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:51.028256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318643+0000","last_change":"2024-05-09T02:07:48.201056+0000","last_active":"2024-05-09T02:08:25.318643+0000","last_peered":"2024-05-09T02:08:25.318643+0000","last_clean":"2024-05-09T02:08:25.318643+0000","last_became_active":"2024-05-09T02:07:48.200671+0000","last_became_peered":"2024-05-09T02:07:48.200671+0000","last_unstale":"2024-05-09T02:08:25.318643+0000","last_undegraded":"2024-05-09T02:08:25.318643+0000","last_fullsized":"2024-05-09T02:08:25.318643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:37:16.138170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309296+0000","last_change":"2024-05-09T02:07:45.160417+0000","last_active":"2024-05-09T02:08:25.309296+0000","last_peered":"2024-05-09T02:08:25.309296+0000","last_clean":"2024-05-09T02:08:25.309296+0000","last_became_active":"2024-05-09T02:07:45.160076+0000","last_became_peered":"2024-05-09T02:07:45.160076+0000","last_unstale":"2024-05-09T02:08:25.309296+0000","last_undegraded":"2024-05-09T02:08:25.309296+0000","last_fullsized":"2024-05-09T02:08:25.309296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:53.653494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314286+0000","last_change":"2024-05-09T02:07:48.203267+0000","last_active":"2024-05-09T02:08:25.314286+0000","last_peered":"2024-05-09T02:08:25.314286+0000","last_clean":"2024-05-09T02:08:25.314286+0000","last_became_active":"2024-05-09T02:07:48.202747+0000","last_became_peered":"2024-05-09T02:07:48.202747+0000","last_unstale":"2024-05-09T02:08:25.314286+0000","last_undegraded":"2024-05-09T02:08:25.314286+0000","last_fullsized":"2024-05-09T02:08:25.314286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.954400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317820+0000","last_change":"2024-05-09T02:07:45.164713+0000","last_active":"2024-05-09T02:08:25.317820+0000","last_peered":"2024-05-09T02:08:25.317820+0000","last_clean":"2024-05-09T02:08:25.317820+0000","last_became_active":"2024-05-09T02:07:45.164324+0000","last_became_peered":"2024-05-09T02:07:45.164324+0000","last_unstale":"2024-05-09T02:08:25.317820+0000","last_undegraded":"2024-05-09T02:08:25.317820+0000","last_fullsized":"2024-05-09T02:08:25.317820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:02.481419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.316226+0000","last_change":"2024-05-09T02:07:48.190305+0000","last_active":"2024-05-09T02:08:25.316226+0000","last_peered":"2024-05-09T02:08:25.316226+0000","last_clean":"2024-05-09T02:08:25.316226+0000","last_became_active":"2024-05-09T02:07:48.189971+0000","last_became_peered":"2024-05-09T02:07:48.189971+0000","last_unstale":"2024-05-09T02:08:25.316226+0000","last_undegraded":"2024-05-09T02:08:25.316226+0000","last_fullsized":"2024-05-09T02:08:25.316226+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:23.565092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":234,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:49.412746+0000","last_change":"2024-05-09T02:07:45.164273+0000","last_active":"2024-05-09T02:09:49.412746+0000","last_peered":"2024-05-09T02:09:49.412746+0000","last_clean":"2024-05-09T02:09:49.412746+0000","last_became_active":"2024-05-09T02:07:45.163183+0000","last_became_peered":"2024-05-09T02:07:45.163183+0000","last_unstale":"2024-05-09T02:09:49.412746+0000","last_undegraded":"2024-05-09T02:09:49.412746+0000","last_fullsized":"2024-05-09T02:09:49.412746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:11.395177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315668+0000","last_change":"2024-05-09T02:07:48.202995+0000","last_active":"2024-05-09T02:08:25.315668+0000","last_peered":"2024-05-09T02:08:25.315668+0000","last_clean":"2024-05-09T02:08:25.315668+0000","last_became_active":"2024-05-09T02:07:48.200915+0000","last_became_peered":"2024-05-09T02:07:48.200915+0000","last_unstale":"2024-05-09T02:08:25.315668+0000","last_undegraded":"2024-05-09T02:08:25.315668+0000","last_fullsized":"2024-05-09T02:08:25.315668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:25.555725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315988+0000","last_change":"2024-05-09T02:07:45.166221+0000","last_active":"2024-05-09T02:08:25.315988+0000","last_peered":"2024-05-09T02:08:25.315988+0000","last_clean":"2024-05-09T02:08:25.315988+0000","last_became_active":"2024-05-09T02:07:45.165813+0000","last_became_peered":"2024-05-09T02:07:45.165813+0000","last_unstale":"2024-05-09T02:08:25.315988+0000","last_undegraded":"2024-05-09T02:08:25.315988+0000","last_fullsized":"2024-05-09T02:08:25.315988+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:01.480828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317355+0000","last_change":"2024-05-09T02:07:48.201475+0000","last_active":"2024-05-09T02:08:25.317355+0000","last_peered":"2024-05-09T02:08:25.317355+0000","last_clean":"2024-05-09T02:08:25.317355+0000","last_became_active":"2024-05-09T02:07:48.201129+0000","last_became_peered":"2024-05-09T02:07:48.201129+0000","last_unstale":"2024-05-09T02:08:25.317355+0000","last_undegraded":"2024-05-09T02:08:25.317355+0000","last_fullsized":"2024-05-09T02:08:25.317355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:51.726752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318564+0000","last_change":"2024-05-09T02:07:45.185045+0000","last_active":"2024-05-09T02:08:25.318564+0000","last_peered":"2024-05-09T02:08:25.318564+0000","last_clean":"2024-05-09T02:08:25.318564+0000","last_became_active":"2024-05-09T02:07:45.184592+0000","last_became_peered":"2024-05-09T02:07:45.184592+0000","last_unstale":"2024-05-09T02:08:25.318564+0000","last_undegraded":"2024-05-09T02:08:25.318564+0000","last_fullsized":"2024-05-09T02:08:25.318564+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:56.220022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315185+0000","last_change":"2024-05-09T02:07:48.199873+0000","last_active":"2024-05-09T02:08:25.315185+0000","last_peered":"2024-05-09T02:08:25.315185+0000","last_clean":"2024-05-09T02:08:25.315185+0000","last_became_active":"2024-05-09T02:07:48.199502+0000","last_became_peered":"2024-05-09T02:07:48.199502+0000","last_unstale":"2024-05-09T02:08:25.315185+0000","last_undegraded":"2024-05-09T02:08:25.315185+0000","last_fullsized":"2024-05-09T02:08:25.315185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:15.406904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'32","reported_seq":271,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:49.412710+0000","last_change":"2024-05-09T02:07:45.166167+0000","last_active":"2024-05-09T02:09:49.412710+0000","last_peered":"2024-05-09T02:09:49.412710+0000","last_clean":"2024-05-09T02:09:49.412710+0000","last_became_active":"2024-05-09T02:07:45.165800+0000","last_became_peered":"2024-05-09T02:07:45.165800+0000","last_unstale":"2024-05-09T02:09:49.412710+0000","last_undegraded":"2024-05-09T02:09:49.412710+0000","last_fullsized":"2024-05-09T02:09:49.412710+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:03.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61826,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":213,"num_read_kb":166,"num_write":34,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310662+0000","last_change":"2024-05-09T02:07:48.187809+0000","last_active":"2024-05-09T02:08:25.310662+0000","last_peered":"2024-05-09T02:08:25.310662+0000","last_clean":"2024-05-09T02:08:25.310662+0000","last_became_active":"2024-05-09T02:07:48.186939+0000","last_became_peered":"2024-05-09T02:07:48.186939+0000","last_unstale":"2024-05-09T02:08:25.310662+0000","last_undegraded":"2024-05-09T02:08:25.310662+0000","last_fullsized":"2024-05-09T02:08:25.310662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:13:15.600343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315617+0000","last_change":"2024-05-09T02:07:45.163756+0000","last_active":"2024-05-09T02:08:25.315617+0000","last_peered":"2024-05-09T02:08:25.315617+0000","last_clean":"2024-05-09T02:08:25.315617+0000","last_became_active":"2024-05-09T02:07:45.163388+0000","last_became_peered":"2024-05-09T02:07:45.163388+0000","last_unstale":"2024-05-09T02:08:25.315617+0000","last_undegraded":"2024-05-09T02:08:25.315617+0000","last_fullsized":"2024-05-09T02:08:25.315617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:41.937148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309348+0000","last_change":"2024-05-09T02:07:48.195218+0000","last_active":"2024-05-09T02:08:25.309348+0000","last_peered":"2024-05-09T02:08:25.309348+0000","last_clean":"2024-05-09T02:08:25.309348+0000","last_became_active":"2024-05-09T02:07:48.194895+0000","last_became_peered":"2024-05-09T02:07:48.194895+0000","last_unstale":"2024-05-09T02:08:25.309348+0000","last_undegraded":"2024-05-09T02:08:25.309348+0000","last_fullsized":"2024-05-09T02:08:25.309348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:49.086594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318180+0000","last_change":"2024-05-09T02:07:45.158039+0000","last_active":"2024-05-09T02:08:25.318180+0000","last_peered":"2024-05-09T02:08:25.318180+0000","last_clean":"2024-05-09T02:08:25.318180+0000","last_became_active":"2024-05-09T02:07:45.157255+0000","last_became_peered":"2024-05-09T02:07:45.157255+0000","last_unstale":"2024-05-09T02:08:25.318180+0000","last_undegraded":"2024-05-09T02:08:25.318180+0000","last_fullsized":"2024-05-09T02:08:25.318180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:56.513999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313958+0000","last_change":"2024-05-09T02:07:48.200150+0000","last_active":"2024-05-09T02:08:25.313958+0000","last_peered":"2024-05-09T02:08:25.313958+0000","last_clean":"2024-05-09T02:08:25.313958+0000","last_became_active":"2024-05-09T02:07:48.199336+0000","last_became_peered":"2024-05-09T02:07:48.199336+0000","last_unstale":"2024-05-09T02:08:25.313958+0000","last_undegraded":"2024-05-09T02:08:25.313958+0000","last_fullsized":"2024-05-09T02:08:25.313958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:45.699119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":138,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:49.412576+0000","last_change":"2024-05-09T02:07:45.165476+0000","last_active":"2024-05-09T02:09:49.412576+0000","last_peered":"2024-05-09T02:09:49.412576+0000","last_clean":"2024-05-09T02:09:49.412576+0000","last_became_active":"2024-05-09T02:07:45.165107+0000","last_became_peered":"2024-05-09T02:07:45.165107+0000","last_unstale":"2024-05-09T02:09:49.412576+0000","last_undegraded":"2024-05-09T02:09:49.412576+0000","last_fullsized":"2024-05-09T02:09:49.412576+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:15:33.518982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313557+0000","last_change":"2024-05-09T02:07:48.193487+0000","last_active":"2024-05-09T02:08:25.313557+0000","last_peered":"2024-05-09T02:08:25.313557+0000","last_clean":"2024-05-09T02:08:25.313557+0000","last_became_active":"2024-05-09T02:07:48.193108+0000","last_became_peered":"2024-05-09T02:07:48.193108+0000","last_unstale":"2024-05-09T02:08:25.313557+0000","last_undegraded":"2024-05-09T02:08:25.313557+0000","last_fullsized":"2024-05-09T02:08:25.313557+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:11.895058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.335525+0000","last_change":"2024-05-09T02:08:25.335525+0000","last_active":"2024-05-09T02:08:25.335525+0000","last_peered":"2024-05-09T02:08:25.335525+0000","last_clean":"2024-05-09T02:08:25.335525+0000","last_became_active":"2024-05-09T02:08:25.319746+0000","last_became_peered":"2024-05-09T02:08:25.319746+0000","last_unstale":"2024-05-09T02:08:25.335525+0000","last_undegraded":"2024-05-09T02:08:25.335525+0000","last_fullsized":"2024-05-09T02:08:25.335525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:41.774066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317433+0000","last_change":"2024-05-09T02:07:48.196125+0000","last_active":"2024-05-09T02:08:25.317433+0000","last_peered":"2024-05-09T02:08:25.317433+0000","last_clean":"2024-05-09T02:08:25.317433+0000","last_became_active":"2024-05-09T02:07:48.195737+0000","last_became_peered":"2024-05-09T02:07:48.195737+0000","last_unstale":"2024-05-09T02:08:25.317433+0000","last_undegraded":"2024-05-09T02:08:25.317433+0000","last_fullsized":"2024-05-09T02:08:25.317433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:43.264720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310367+0000","last_change":"2024-05-09T02:07:45.163791+0000","last_active":"2024-05-09T02:08:25.310367+0000","last_peered":"2024-05-09T02:08:25.310367+0000","last_clean":"2024-05-09T02:08:25.310367+0000","last_became_active":"2024-05-09T02:07:45.163129+0000","last_became_peered":"2024-05-09T02:07:45.163129+0000","last_unstale":"2024-05-09T02:08:25.310367+0000","last_undegraded":"2024-05-09T02:08:25.310367+0000","last_fullsized":"2024-05-09T02:08:25.310367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:48.562850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310748+0000","last_change":"2024-05-09T02:07:48.197782+0000","last_active":"2024-05-09T02:08:25.310748+0000","last_peered":"2024-05-09T02:08:25.310748+0000","last_clean":"2024-05-09T02:08:25.310748+0000","last_became_active":"2024-05-09T02:07:48.197353+0000","last_became_peered":"2024-05-09T02:07:48.197353+0000","last_unstale":"2024-05-09T02:08:25.310748+0000","last_undegraded":"2024-05-09T02:08:25.310748+0000","last_fullsized":"2024-05-09T02:08:25.310748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:17.053571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315292+0000","last_change":"2024-05-09T02:07:45.187028+0000","last_active":"2024-05-09T02:08:25.315292+0000","last_peered":"2024-05-09T02:08:25.315292+0000","last_clean":"2024-05-09T02:08:25.315292+0000","last_became_active":"2024-05-09T02:07:45.186639+0000","last_became_peered":"2024-05-09T02:07:45.186639+0000","last_unstale":"2024-05-09T02:08:25.315292+0000","last_undegraded":"2024-05-09T02:08:25.315292+0000","last_fullsized":"2024-05-09T02:08:25.315292+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:35.834374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.308184+0000","last_change":"2024-05-09T02:07:48.207150+0000","last_active":"2024-05-09T02:08:25.308184+0000","last_peered":"2024-05-09T02:08:25.308184+0000","last_clean":"2024-05-09T02:08:25.308184+0000","last_became_active":"2024-05-09T02:07:48.206797+0000","last_became_peered":"2024-05-09T02:07:48.206797+0000","last_unstale":"2024-05-09T02:08:25.308184+0000","last_undegraded":"2024-05-09T02:08:25.308184+0000","last_fullsized":"2024-05-09T02:08:25.308184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:37.328133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":135,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:49.412069+0000","last_change":"2024-05-09T02:07:45.165422+0000","last_active":"2024-05-09T02:09:49.412069+0000","last_peered":"2024-05-09T02:09:49.412069+0000","last_clean":"2024-05-09T02:09:49.412069+0000","last_became_active":"2024-05-09T02:07:45.164998+0000","last_became_peered":"2024-05-09T02:07:45.164998+0000","last_unstale":"2024-05-09T02:09:49.412069+0000","last_undegraded":"2024-05-09T02:09:49.412069+0000","last_fullsized":"2024-05-09T02:09:49.412069+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:49.124323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.307961+0000","last_change":"2024-05-09T02:07:48.206538+0000","last_active":"2024-05-09T02:08:25.307961+0000","last_peered":"2024-05-09T02:08:25.307961+0000","last_clean":"2024-05-09T02:08:25.307961+0000","last_became_active":"2024-05-09T02:07:48.206061+0000","last_became_peered":"2024-05-09T02:07:48.206061+0000","last_unstale":"2024-05-09T02:08:25.307961+0000","last_undegraded":"2024-05-09T02:08:25.307961+0000","last_fullsized":"2024-05-09T02:08:25.307961+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:24.075091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318270+0000","last_change":"2024-05-09T02:07:48.202235+0000","last_active":"2024-05-09T02:08:25.318270+0000","last_peered":"2024-05-09T02:08:25.318270+0000","last_clean":"2024-05-09T02:08:25.318270+0000","last_became_active":"2024-05-09T02:07:48.200891+0000","last_became_peered":"2024-05-09T02:07:48.200891+0000","last_unstale":"2024-05-09T02:08:25.318270+0000","last_undegraded":"2024-05-09T02:08:25.318270+0000","last_fullsized":"2024-05-09T02:08:25.318270+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:39.061267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":139,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:53.416660+0000","last_change":"2024-05-09T02:07:45.184582+0000","last_active":"2024-05-09T02:09:53.416660+0000","last_peered":"2024-05-09T02:09:53.416660+0000","last_clean":"2024-05-09T02:09:53.416660+0000","last_became_active":"2024-05-09T02:07:45.184187+0000","last_became_peered":"2024-05-09T02:07:45.184187+0000","last_unstale":"2024-05-09T02:09:53.416660+0000","last_undegraded":"2024-05-09T02:09:53.416660+0000","last_fullsized":"2024-05-09T02:09:53.416660+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:11.343797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314798+0000","last_change":"2024-05-09T02:07:48.199247+0000","last_active":"2024-05-09T02:08:25.314798+0000","last_peered":"2024-05-09T02:08:25.314798+0000","last_clean":"2024-05-09T02:08:25.314798+0000","last_became_active":"2024-05-09T02:07:48.198121+0000","last_became_peered":"2024-05-09T02:07:48.198121+0000","last_unstale":"2024-05-09T02:08:25.314798+0000","last_undegraded":"2024-05-09T02:08:25.314798+0000","last_fullsized":"2024-05-09T02:08:25.314798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:45:12.885616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318037+0000","last_change":"2024-05-09T02:07:45.163750+0000","last_active":"2024-05-09T02:08:25.318037+0000","last_peered":"2024-05-09T02:08:25.318037+0000","last_clean":"2024-05-09T02:08:25.318037+0000","last_became_active":"2024-05-09T02:07:45.163039+0000","last_became_peered":"2024-05-09T02:07:45.163039+0000","last_unstale":"2024-05-09T02:08:25.318037+0000","last_undegraded":"2024-05-09T02:08:25.318037+0000","last_fullsized":"2024-05-09T02:08:25.318037+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:06.794514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":100,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314191+0000","last_change":"2024-05-09T02:06:20.334734+0000","last_active":"2024-05-09T02:08:25.314191+0000","last_peered":"2024-05-09T02:08:25.314191+0000","last_clean":"2024-05-09T02:08:25.314191+0000","last_became_active":"2024-05-09T02:06:20.329198+0000","last_became_peered":"2024-05-09T02:06:20.329198+0000","last_unstale":"2024-05-09T02:08:25.314191+0000","last_undegraded":"2024-05-09T02:08:25.314191+0000","last_fullsized":"2024-05-09T02:08:25.314191+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:02:50.632016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:02:50.632016+0000","last_clean_scrub_stamp":"2024-05-09T02:02:50.632016+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:08:15.495982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.322091+0000","last_change":"2024-05-09T02:08:25.321187+0000","last_active":"2024-05-09T02:08:25.322091+0000","last_peered":"2024-05-09T02:08:25.322091+0000","last_clean":"2024-05-09T02:08:25.322091+0000","last_became_active":"2024-05-09T02:08:25.320693+0000","last_became_peered":"2024-05-09T02:08:25.320693+0000","last_unstale":"2024-05-09T02:08:25.322091+0000","last_undegraded":"2024-05-09T02:08:25.322091+0000","last_fullsized":"2024-05-09T02:08:25.322091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:00.176016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314470+0000","last_change":"2024-05-09T02:07:45.158053+0000","last_active":"2024-05-09T02:08:25.314470+0000","last_peered":"2024-05-09T02:08:25.314470+0000","last_clean":"2024-05-09T02:08:25.314470+0000","last_became_active":"2024-05-09T02:07:45.157619+0000","last_became_peered":"2024-05-09T02:07:45.157619+0000","last_unstale":"2024-05-09T02:08:25.314470+0000","last_undegraded":"2024-05-09T02:08:25.314470+0000","last_fullsized":"2024-05-09T02:08:25.314470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:01.618475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.341770+0000","last_change":"2024-05-09T02:08:25.341377+0000","last_active":"2024-05-09T02:08:25.341770+0000","last_peered":"2024-05-09T02:08:25.341770+0000","last_clean":"2024-05-09T02:08:25.341770+0000","last_became_active":"2024-05-09T02:08:25.318683+0000","last_became_peered":"2024-05-09T02:08:25.318683+0000","last_unstale":"2024-05-09T02:08:25.341770+0000","last_undegraded":"2024-05-09T02:08:25.341770+0000","last_fullsized":"2024-05-09T02:08:25.341770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:12.456338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309451+0000","last_change":"2024-05-09T02:07:48.186892+0000","last_active":"2024-05-09T02:08:25.309451+0000","last_peered":"2024-05-09T02:08:25.309451+0000","last_clean":"2024-05-09T02:08:25.309451+0000","last_became_active":"2024-05-09T02:07:48.186543+0000","last_became_peered":"2024-05-09T02:07:48.186543+0000","last_unstale":"2024-05-09T02:08:25.309451+0000","last_undegraded":"2024-05-09T02:08:25.309451+0000","last_fullsized":"2024-05-09T02:08:25.309451+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:01.210504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'15","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:31.205313+0000","last_change":"2024-05-09T02:07:45.158007+0000","last_active":"2024-05-09T02:08:31.205313+0000","last_peered":"2024-05-09T02:08:31.205313+0000","last_clean":"2024-05-09T02:08:31.205313+0000","last_became_active":"2024-05-09T02:07:45.157612+0000","last_became_peered":"2024-05-09T02:07:45.157612+0000","last_unstale":"2024-05-09T02:08:31.205313+0000","last_undegraded":"2024-05-09T02:08:31.205313+0000","last_fullsized":"2024-05-09T02:08:31.205313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:01.526756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314093+0000","last_change":"2024-05-09T02:07:48.186065+0000","last_active":"2024-05-09T02:08:25.314093+0000","last_peered":"2024-05-09T02:08:25.314093+0000","last_clean":"2024-05-09T02:08:25.314093+0000","last_became_active":"2024-05-09T02:07:48.185746+0000","last_became_peered":"2024-05-09T02:07:48.185746+0000","last_unstale":"2024-05-09T02:08:25.314093+0000","last_undegraded":"2024-05-09T02:08:25.314093+0000","last_fullsized":"2024-05-09T02:08:25.314093+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:09.570261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317499+0000","last_change":"2024-05-09T02:07:46.005439+0000","last_active":"2024-05-09T02:08:25.317499+0000","last_peered":"2024-05-09T02:08:25.317499+0000","last_clean":"2024-05-09T02:08:25.317499+0000","last_became_active":"2024-05-09T02:07:46.005087+0000","last_became_peered":"2024-05-09T02:07:46.005087+0000","last_unstale":"2024-05-09T02:08:25.317499+0000","last_undegraded":"2024-05-09T02:08:25.317499+0000","last_fullsized":"2024-05-09T02:08:25.317499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:08.543310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315347+0000","last_change":"2024-05-09T02:07:48.194062+0000","last_active":"2024-05-09T02:08:25.315347+0000","last_peered":"2024-05-09T02:08:25.315347+0000","last_clean":"2024-05-09T02:08:25.315347+0000","last_became_active":"2024-05-09T02:07:48.193712+0000","last_became_peered":"2024-05-09T02:07:48.193712+0000","last_unstale":"2024-05-09T02:08:25.315347+0000","last_undegraded":"2024-05-09T02:08:25.315347+0000","last_fullsized":"2024-05-09T02:08:25.315347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:48.394898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315411+0000","last_change":"2024-05-09T02:07:45.184821+0000","last_active":"2024-05-09T02:08:25.315411+0000","last_peered":"2024-05-09T02:08:25.315411+0000","last_clean":"2024-05-09T02:08:25.315411+0000","last_became_active":"2024-05-09T02:07:45.184437+0000","last_became_peered":"2024-05-09T02:07:45.184437+0000","last_unstale":"2024-05-09T02:08:25.315411+0000","last_undegraded":"2024-05-09T02:08:25.315411+0000","last_fullsized":"2024-05-09T02:08:25.315411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:30.864171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309993+0000","last_change":"2024-05-09T02:07:48.187635+0000","last_active":"2024-05-09T02:08:25.309993+0000","last_peered":"2024-05-09T02:08:25.309993+0000","last_clean":"2024-05-09T02:08:25.309993+0000","last_became_active":"2024-05-09T02:07:48.187071+0000","last_became_peered":"2024-05-09T02:07:48.187071+0000","last_unstale":"2024-05-09T02:08:25.309993+0000","last_undegraded":"2024-05-09T02:08:25.309993+0000","last_fullsized":"2024-05-09T02:08:25.309993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:10.716715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315158+0000","last_change":"2024-05-09T02:07:45.162166+0000","last_active":"2024-05-09T02:08:25.315158+0000","last_peered":"2024-05-09T02:08:25.315158+0000","last_clean":"2024-05-09T02:08:25.315158+0000","last_became_active":"2024-05-09T02:07:45.161787+0000","last_became_peered":"2024-05-09T02:07:45.161787+0000","last_unstale":"2024-05-09T02:08:25.315158+0000","last_undegraded":"2024-05-09T02:08:25.315158+0000","last_fullsized":"2024-05-09T02:08:25.315158+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:17.925214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310341+0000","last_change":"2024-05-09T02:07:48.202980+0000","last_active":"2024-05-09T02:08:25.310341+0000","last_peered":"2024-05-09T02:08:25.310341+0000","last_clean":"2024-05-09T02:08:25.310341+0000","last_became_active":"2024-05-09T02:07:48.202609+0000","last_became_peered":"2024-05-09T02:07:48.202609+0000","last_unstale":"2024-05-09T02:08:25.310341+0000","last_undegraded":"2024-05-09T02:08:25.310341+0000","last_fullsized":"2024-05-09T02:08:25.310341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:59.993451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.326699+0000","last_change":"2024-05-09T02:08:25.326142+0000","last_active":"2024-05-09T02:08:25.326699+0000","last_peered":"2024-05-09T02:08:25.326699+0000","last_clean":"2024-05-09T02:08:25.326699+0000","last_became_active":"2024-05-09T02:08:25.317666+0000","last_became_peered":"2024-05-09T02:08:25.317666+0000","last_unstale":"2024-05-09T02:08:25.326699+0000","last_undegraded":"2024-05-09T02:08:25.326699+0000","last_fullsized":"2024-05-09T02:08:25.326699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:26.178211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315560+0000","last_change":"2024-05-09T02:07:48.202219+0000","last_active":"2024-05-09T02:08:25.315560+0000","last_peered":"2024-05-09T02:08:25.315560+0000","last_clean":"2024-05-09T02:08:25.315560+0000","last_became_active":"2024-05-09T02:07:48.201819+0000","last_became_peered":"2024-05-09T02:07:48.201819+0000","last_unstale":"2024-05-09T02:08:25.315560+0000","last_undegraded":"2024-05-09T02:08:25.315560+0000","last_fullsized":"2024-05-09T02:08:25.315560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:51.573359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311162+0000","last_change":"2024-05-09T02:07:45.162228+0000","last_active":"2024-05-09T02:08:25.311162+0000","last_peered":"2024-05-09T02:08:25.311162+0000","last_clean":"2024-05-09T02:08:25.311162+0000","last_became_active":"2024-05-09T02:07:45.161847+0000","last_became_peered":"2024-05-09T02:07:45.161847+0000","last_unstale":"2024-05-09T02:08:25.311162+0000","last_undegraded":"2024-05-09T02:08:25.311162+0000","last_fullsized":"2024-05-09T02:08:25.311162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:47.459709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314196+0000","last_change":"2024-05-09T02:07:48.201401+0000","last_active":"2024-05-09T02:08:25.314196+0000","last_peered":"2024-05-09T02:08:25.314196+0000","last_clean":"2024-05-09T02:08:25.314196+0000","last_became_active":"2024-05-09T02:07:48.201040+0000","last_became_peered":"2024-05-09T02:07:48.201040+0000","last_unstale":"2024-05-09T02:08:25.314196+0000","last_undegraded":"2024-05-09T02:08:25.314196+0000","last_fullsized":"2024-05-09T02:08:25.314196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:58.038690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":135,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:09:48.412003+0000","last_change":"2024-05-09T02:07:46.005273+0000","last_active":"2024-05-09T02:09:48.412003+0000","last_peered":"2024-05-09T02:09:48.412003+0000","last_clean":"2024-05-09T02:09:48.412003+0000","last_became_active":"2024-05-09T02:07:46.004732+0000","last_became_peered":"2024-05-09T02:07:46.004732+0000","last_unstale":"2024-05-09T02:09:48.412003+0000","last_undegraded":"2024-05-09T02:09:48.412003+0000","last_fullsized":"2024-05-09T02:09:48.412003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:55.576955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315747+0000","last_change":"2024-05-09T02:07:48.197032+0000","last_active":"2024-05-09T02:08:25.315747+0000","last_peered":"2024-05-09T02:08:25.315747+0000","last_clean":"2024-05-09T02:08:25.315747+0000","last_became_active":"2024-05-09T02:07:48.196553+0000","last_became_peered":"2024-05-09T02:07:48.196553+0000","last_unstale":"2024-05-09T02:08:25.315747+0000","last_undegraded":"2024-05-09T02:08:25.315747+0000","last_fullsized":"2024-05-09T02:08:25.315747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:50:25.081024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.318767+0000","last_change":"2024-05-09T02:07:45.166456+0000","last_active":"2024-05-09T02:08:25.318767+0000","last_peered":"2024-05-09T02:08:25.318767+0000","last_clean":"2024-05-09T02:08:25.318767+0000","last_became_active":"2024-05-09T02:07:45.166132+0000","last_became_peered":"2024-05-09T02:07:45.166132+0000","last_unstale":"2024-05-09T02:08:25.318767+0000","last_undegraded":"2024-05-09T02:08:25.318767+0000","last_fullsized":"2024-05-09T02:08:25.318767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:15.701802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.324526+0000","last_change":"2024-05-09T02:08:25.323303+0000","last_active":"2024-05-09T02:08:25.324526+0000","last_peered":"2024-05-09T02:08:25.324526+0000","last_clean":"2024-05-09T02:08:25.324526+0000","last_became_active":"2024-05-09T02:08:25.321752+0000","last_became_peered":"2024-05-09T02:08:25.321752+0000","last_unstale":"2024-05-09T02:08:25.324526+0000","last_undegraded":"2024-05-09T02:08:25.324526+0000","last_fullsized":"2024-05-09T02:08:25.324526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:17.791737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309540+0000","last_change":"2024-05-09T02:07:45.155480+0000","last_active":"2024-05-09T02:08:25.309540+0000","last_peered":"2024-05-09T02:08:25.309540+0000","last_clean":"2024-05-09T02:08:25.309540+0000","last_became_active":"2024-05-09T02:07:45.155085+0000","last_became_peered":"2024-05-09T02:07:45.155085+0000","last_unstale":"2024-05-09T02:08:25.309540+0000","last_undegraded":"2024-05-09T02:08:25.309540+0000","last_fullsized":"2024-05-09T02:08:25.309540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:05.547857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.314367+0000","last_change":"2024-05-09T02:07:48.194480+0000","last_active":"2024-05-09T02:08:25.314367+0000","last_peered":"2024-05-09T02:08:25.314367+0000","last_clean":"2024-05-09T02:08:25.314367+0000","last_became_active":"2024-05-09T02:07:48.194136+0000","last_became_peered":"2024-05-09T02:07:48.194136+0000","last_unstale":"2024-05-09T02:08:25.314367+0000","last_undegraded":"2024-05-09T02:08:25.314367+0000","last_fullsized":"2024-05-09T02:08:25.314367+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:09.177762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310492+0000","last_change":"2024-05-09T02:07:48.195152+0000","last_active":"2024-05-09T02:08:25.310492+0000","last_peered":"2024-05-09T02:08:25.310492+0000","last_clean":"2024-05-09T02:08:25.310492+0000","last_became_active":"2024-05-09T02:07:48.194779+0000","last_became_peered":"2024-05-09T02:07:48.194779+0000","last_unstale":"2024-05-09T02:08:25.310492+0000","last_undegraded":"2024-05-09T02:08:25.310492+0000","last_fullsized":"2024-05-09T02:08:25.310492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:39.345452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.322235+0000","last_change":"2024-05-09T02:08:25.322235+0000","last_active":"2024-05-09T02:08:25.322235+0000","last_peered":"2024-05-09T02:08:25.322235+0000","last_clean":"2024-05-09T02:08:25.322235+0000","last_became_active":"2024-05-09T02:08:25.321674+0000","last_became_peered":"2024-05-09T02:08:25.321674+0000","last_unstale":"2024-05-09T02:08:25.322235+0000","last_undegraded":"2024-05-09T02:08:25.322235+0000","last_fullsized":"2024-05-09T02:08:25.322235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:52.671669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315466+0000","last_change":"2024-05-09T02:07:48.195710+0000","last_active":"2024-05-09T02:08:25.315466+0000","last_peered":"2024-05-09T02:08:25.315466+0000","last_clean":"2024-05-09T02:08:25.315466+0000","last_became_active":"2024-05-09T02:07:48.195332+0000","last_became_peered":"2024-05-09T02:07:48.195332+0000","last_unstale":"2024-05-09T02:08:25.315466+0000","last_undegraded":"2024-05-09T02:08:25.315466+0000","last_fullsized":"2024-05-09T02:08:25.315466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:12:17.904166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315843+0000","last_change":"2024-05-09T02:07:48.200791+0000","last_active":"2024-05-09T02:08:25.315843+0000","last_peered":"2024-05-09T02:08:25.315843+0000","last_clean":"2024-05-09T02:08:25.315843+0000","last_became_active":"2024-05-09T02:07:48.199916+0000","last_became_peered":"2024-05-09T02:07:48.199916+0000","last_unstale":"2024-05-09T02:08:25.315843+0000","last_undegraded":"2024-05-09T02:08:25.315843+0000","last_fullsized":"2024-05-09T02:08:25.315843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:32.692375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309571+0000","last_change":"2024-05-09T02:07:48.191662+0000","last_active":"2024-05-09T02:08:25.309571+0000","last_peered":"2024-05-09T02:08:25.309571+0000","last_clean":"2024-05-09T02:08:25.309571+0000","last_became_active":"2024-05-09T02:07:48.191336+0000","last_became_peered":"2024-05-09T02:07:48.191336+0000","last_unstale":"2024-05-09T02:08:25.309571+0000","last_undegraded":"2024-05-09T02:08:25.309571+0000","last_fullsized":"2024-05-09T02:08:25.309571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:00.733611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.311462+0000","last_change":"2024-05-09T02:07:48.195182+0000","last_active":"2024-05-09T02:08:25.311462+0000","last_peered":"2024-05-09T02:08:25.311462+0000","last_clean":"2024-05-09T02:08:25.311462+0000","last_became_active":"2024-05-09T02:07:48.194828+0000","last_became_peered":"2024-05-09T02:07:48.194828+0000","last_unstale":"2024-05-09T02:08:25.311462+0000","last_undegraded":"2024-05-09T02:08:25.311462+0000","last_fullsized":"2024-05-09T02:08:25.311462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:33.427409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315738+0000","last_change":"2024-05-09T02:07:48.199688+0000","last_active":"2024-05-09T02:08:25.315738+0000","last_peered":"2024-05-09T02:08:25.315738+0000","last_clean":"2024-05-09T02:08:25.315738+0000","last_became_active":"2024-05-09T02:07:48.199326+0000","last_became_peered":"2024-05-09T02:07:48.199326+0000","last_unstale":"2024-05-09T02:08:25.315738+0000","last_undegraded":"2024-05-09T02:08:25.315738+0000","last_fullsized":"2024-05-09T02:08:25.315738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:30.662738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.313935+0000","last_change":"2024-05-09T02:07:48.190727+0000","last_active":"2024-05-09T02:08:25.313935+0000","last_peered":"2024-05-09T02:08:25.313935+0000","last_clean":"2024-05-09T02:08:25.313935+0000","last_became_active":"2024-05-09T02:07:48.190397+0000","last_became_peered":"2024-05-09T02:07:48.190397+0000","last_unstale":"2024-05-09T02:08:25.313935+0000","last_undegraded":"2024-05-09T02:08:25.313935+0000","last_fullsized":"2024-05-09T02:08:25.313935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:42.010669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.317969+0000","last_change":"2024-05-09T02:07:48.200444+0000","last_active":"2024-05-09T02:08:25.317969+0000","last_peered":"2024-05-09T02:08:25.317969+0000","last_clean":"2024-05-09T02:08:25.317969+0000","last_became_active":"2024-05-09T02:07:48.200031+0000","last_became_peered":"2024-05-09T02:07:48.200031+0000","last_unstale":"2024-05-09T02:08:25.317969+0000","last_undegraded":"2024-05-09T02:08:25.317969+0000","last_fullsized":"2024-05-09T02:08:25.317969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:30.269792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.315072+0000","last_change":"2024-05-09T02:07:48.203717+0000","last_active":"2024-05-09T02:08:25.315072+0000","last_peered":"2024-05-09T02:08:25.315072+0000","last_clean":"2024-05-09T02:08:25.315072+0000","last_became_active":"2024-05-09T02:07:48.203347+0000","last_became_peered":"2024-05-09T02:07:48.203347+0000","last_unstale":"2024-05-09T02:08:25.315072+0000","last_undegraded":"2024-05-09T02:08:25.315072+0000","last_fullsized":"2024-05-09T02:08:25.315072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:17.391504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.323029+0000","last_change":"2024-05-09T02:08:25.323029+0000","last_active":"2024-05-09T02:08:25.323029+0000","last_peered":"2024-05-09T02:08:25.323029+0000","last_clean":"2024-05-09T02:08:25.323029+0000","last_became_active":"2024-05-09T02:08:25.322524+0000","last_became_peered":"2024-05-09T02:08:25.322524+0000","last_unstale":"2024-05-09T02:08:25.323029+0000","last_undegraded":"2024-05-09T02:08:25.323029+0000","last_fullsized":"2024-05-09T02:08:25.323029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:25.017511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.309673+0000","last_change":"2024-05-09T02:07:48.187802+0000","last_active":"2024-05-09T02:08:25.309673+0000","last_peered":"2024-05-09T02:08:25.309673+0000","last_clean":"2024-05-09T02:08:25.309673+0000","last_became_active":"2024-05-09T02:07:48.187478+0000","last_became_peered":"2024-05-09T02:07:48.187478+0000","last_unstale":"2024-05-09T02:08:25.309673+0000","last_undegraded":"2024-05-09T02:08:25.309673+0000","last_fullsized":"2024-05-09T02:08:25.309673+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:12.981386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-09T02:08:25.310421+0000","last_change":"2024-05-09T02:07:48.192117+0000","last_active":"2024-05-09T02:08:25.310421+0000","last_peered":"2024-05-09T02:08:25.310421+0000","last_clean":"2024-05-09T02:08:25.310421+0000","last_became_active":"2024-05-09T02:07:48.191726+0000","last_became_peered":"2024-05-09T02:07:48.191726+0000","last_unstale":"2024-05-09T02:08:25.310421+0000","last_undegraded":"2024-05-09T02:08:25.310421+0000","last_fullsized":"2024-05-09T02:08:25.310421+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:07.735582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280595,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68639,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1070,"num_read_kb":612,"num_write":245,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678057,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":429664,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874284,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1082798,"data_compressed":41450,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037812,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":909312,"data_stored":429599,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234040,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":429748,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430269,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":1022378,"data_compressed":41450,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593795,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":429199,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":491938,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":430538,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":492046,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149779,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":428907,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953549,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":429563,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":1021665,"data_compressed":41450,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T02:09:54.902 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T02:09:54.902 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T02:09:54.902 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-09T02:09:54.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-09T02:09:55.156 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:55.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:55 smithi019 ceph-mon[30151]: pgmap v356: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:09:55.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:55 smithi019 ceph-mon[30151]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:09:55.556 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:55 smithi019 ceph-mon[30151]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T02:09:55.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:55 smithi196 ceph-mon[34718]: pgmap v356: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:09:55.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:55 smithi196 ceph-mon[34718]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:09:55.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:55 smithi196 ceph-mon[34718]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T02:09:55.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:55 smithi059 ceph-mon[34781]: pgmap v356: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:09:55.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:55 smithi059 ceph-mon[34781]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:09:55.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:55 smithi059 ceph-mon[34781]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T02:09:56.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:56 smithi019 ceph-mon[30151]: pgmap v357: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T02:09:56.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:56 smithi196 ceph-mon[34718]: pgmap v357: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T02:09:56.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.897+0000 7f060c889700 1 -- 172.21.15.19:0/1785720485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0608107df0 msgr2=0x7f0608108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:56.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.897+0000 7f060c889700 1 --2- 172.21.15.19:0/1785720485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0608107df0 0x7f0608108200 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0099c0 tx=0x7f05fc009cd0 comp rx=0 tx=0).stop 2024-05-09T02:09:56.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.897+0000 7f060c889700 1 -- 172.21.15.19:0/1785720485 shutdown_connections 2024-05-09T02:09:56.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.897+0000 7f060c889700 1 --2- 172.21.15.19:0/1785720485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608108740 0x7f06081100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:56.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.897+0000 7f060c889700 1 --2- 172.21.15.19:0/1785720485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06081034c0 0x7f06081038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:56.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.897+0000 7f060c889700 1 --2- 172.21.15.19:0/1785720485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0608107df0 0x7f0608108200 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:56.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.897+0000 7f060c889700 1 -- 172.21.15.19:0/1785720485 >> 172.21.15.19:0/1785720485 conn(0x7f06080fedf0 msgr2=0x7f0608101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:56.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.897+0000 7f060c889700 1 -- 172.21.15.19:0/1785720485 shutdown_connections 2024-05-09T02:09:56.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.898+0000 7f060c889700 1 -- 172.21.15.19:0/1785720485 wait complete. 2024-05-09T02:09:56.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.898+0000 7f060c889700 1 Processor -- start 2024-05-09T02:09:56.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.898+0000 7f060c889700 1 -- start start 2024-05-09T02:09:56.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f060c889700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06081034c0 0x7f0608199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:56.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f060c889700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0608107df0 0x7f060819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:56.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f060c889700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608108740 0x7f060819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:56.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f060c889700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0608113350 con 0x7f0608107df0 2024-05-09T02:09:56.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f060c889700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06081131d0 con 0x7f0608108740 2024-05-09T02:09:56.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f060c889700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06081134d0 con 0x7f06081034c0 2024-05-09T02:09:56.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f0605d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0608107df0 0x7f060819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:56.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f0605d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0608107df0 0x7f060819a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39532/0 (socket says 172.21.15.19:39532) 2024-05-09T02:09:56.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f0605d9b700 1 -- 172.21.15.19:0/1804361908 learned_addr learned my addr 172.21.15.19:0/1804361908 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:56.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f060659c700 1 --2- 172.21.15.19:0/1804361908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06081034c0 0x7f0608199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:56.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f0606d9d700 1 --2- 172.21.15.19:0/1804361908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608108740 0x7f060819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:56.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f0605d9b700 1 -- 172.21.15.19:0/1804361908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06081034c0 msgr2=0x7f0608199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:56.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f0605d9b700 1 --2- 172.21.15.19:0/1804361908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06081034c0 0x7f0608199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:56.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f0605d9b700 1 -- 172.21.15.19:0/1804361908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608108740 msgr2=0x7f060819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:56.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f0605d9b700 1 --2- 172.21.15.19:0/1804361908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608108740 0x7f060819e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:56.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f0605d9b700 1 -- 172.21.15.19:0/1804361908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060819ec50 con 0x7f0608107df0 2024-05-09T02:09:56.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f0606d9d700 1 --2- 172.21.15.19:0/1804361908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608108740 0x7f060819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:09:56.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.899+0000 7f0605d9b700 1 --2- 172.21.15.19:0/1804361908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0608107df0 0x7f060819a320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f05fc006010 tx=0x7f05fc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:56.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.900+0000 7f05f77fe700 1 -- 172.21.15.19:0/1804361908 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc027070 con 0x7f0608107df0 2024-05-09T02:09:56.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.900+0000 7f060c889700 1 -- 172.21.15.19:0/1804361908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060819eee0 con 0x7f0608107df0 2024-05-09T02:09:56.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.900+0000 7f05f77fe700 1 -- 172.21.15.19:0/1804361908 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05fc003f20 con 0x7f0608107df0 2024-05-09T02:09:56.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.900+0000 7f05f77fe700 1 -- 172.21.15.19:0/1804361908 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc022cd0 con 0x7f0608107df0 2024-05-09T02:09:56.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.900+0000 7f060c889700 1 -- 172.21.15.19:0/1804361908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060819f3c0 con 0x7f0608107df0 2024-05-09T02:09:56.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.901+0000 7f060c889700 1 -- 172.21.15.19:0/1804361908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06080512c0 con 0x7f0608107df0 2024-05-09T02:09:56.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.902+0000 7f05f77fe700 1 -- 172.21.15.19:0/1804361908 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f05fc02a070 con 0x7f0608107df0 2024-05-09T02:09:56.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.903+0000 7f05f77fe700 1 --2- 172.21.15.19:0/1804361908 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f05f00b2880 0x7f05f00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:56.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.903+0000 7f060659c700 1 --2- 172.21.15.19:0/1804361908 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f05f00b2880 0x7f05f00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:56.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.903+0000 7f05f77fe700 1 -- 172.21.15.19:0/1804361908 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f05fc0e34e0 con 0x7f0608107df0 2024-05-09T02:09:56.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.904+0000 7f060659c700 1 --2- 172.21.15.19:0/1804361908 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f05f00b2880 0x7f05f00b4d90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f05f8005950 tx=0x7f05f800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:56.907+0000 7f05f77fe700 1 -- 172.21.15.19:0/1804361908 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f05fc031080 con 0x7f0608107df0 2024-05-09T02:09:56.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:56 smithi059 ceph-mon[34781]: pgmap v357: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T02:09:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:57.171+0000 7f060c889700 1 -- 172.21.15.19:0/1804361908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f060819b0e0 con 0x7f0608107df0 2024-05-09T02:09:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:57.171+0000 7f05f77fe700 1 -- 172.21.15.19:0/1804361908 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f05fc0b3070 con 0x7f0608107df0 2024-05-09T02:09:57.174 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:09:57.174 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T02:09:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:57.174+0000 7f060c889700 1 -- 172.21.15.19:0/1804361908 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f05f00b2880 msgr2=0x7f05f00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:57.174+0000 7f060c889700 1 --2- 172.21.15.19:0/1804361908 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f05f00b2880 0x7f05f00b4d90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f05f8005950 tx=0x7f05f800b410 comp rx=0 tx=0).stop 2024-05-09T02:09:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:57.174+0000 7f060c889700 1 -- 172.21.15.19:0/1804361908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0608107df0 msgr2=0x7f060819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:57.174+0000 7f060c889700 1 --2- 172.21.15.19:0/1804361908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0608107df0 0x7f060819a320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f05fc006010 tx=0x7f05fc0047b0 comp rx=0 tx=0).stop 2024-05-09T02:09:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:57.174+0000 7f060c889700 1 -- 172.21.15.19:0/1804361908 shutdown_connections 2024-05-09T02:09:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:57.174+0000 7f060c889700 1 --2- 172.21.15.19:0/1804361908 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f05f00b2880 0x7f05f00b4d90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:57.174+0000 7f060c889700 1 --2- 172.21.15.19:0/1804361908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608108740 0x7f060819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:57.174+0000 7f060c889700 1 --2- 172.21.15.19:0/1804361908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06081034c0 0x7f0608199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:57.174+0000 7f060c889700 1 --2- 172.21.15.19:0/1804361908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0608107df0 0x7f060819a320 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:57.174+0000 7f060c889700 1 -- 172.21.15.19:0/1804361908 >> 172.21.15.19:0/1804361908 conn(0x7f06080fedf0 msgr2=0x7f06081096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:57.174+0000 7f060c889700 1 -- 172.21.15.19:0/1804361908 shutdown_connections 2024-05-09T02:09:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:57.174+0000 7f060c889700 1 -- 172.21.15.19:0/1804361908 wait complete. 2024-05-09T02:09:57.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:57 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1804361908' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T02:09:57.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:57 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1804361908' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T02:09:57.890 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-09T02:09:57.891 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:09:57.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:57 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1804361908' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T02:09:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:09:58.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:09:58 smithi196 ceph-mon[34718]: pgmap v358: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-09T02:09:58.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:09:58 smithi019 ceph-mon[30151]: pgmap v358: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-09T02:09:58.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:09:58 smithi059 ceph-mon[34781]: pgmap v358: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-09T02:09:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.897+0000 7fe73d633700 1 -- 172.21.15.19:0/2369668208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe738108760 msgr2=0x7fe7381100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.897+0000 7fe73d633700 1 --2- 172.21.15.19:0/2369668208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe738108760 0x7fe7381100c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe73000b0d0 tx=0x7fe73000b3e0 comp rx=0 tx=0).stop 2024-05-09T02:09:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.897+0000 7fe73d633700 1 -- 172.21.15.19:0/2369668208 shutdown_connections 2024-05-09T02:09:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.897+0000 7fe73d633700 1 --2- 172.21.15.19:0/2369668208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe738108760 0x7fe7381100c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.897+0000 7fe73d633700 1 --2- 172.21.15.19:0/2369668208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7381034e0 0x7fe7381038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.897+0000 7fe73d633700 1 --2- 172.21.15.19:0/2369668208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe738107e10 0x7fe738108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.897+0000 7fe73d633700 1 -- 172.21.15.19:0/2369668208 >> 172.21.15.19:0/2369668208 conn(0x7fe7380fef30 msgr2=0x7fe738101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:09:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.897+0000 7fe73d633700 1 -- 172.21.15.19:0/2369668208 shutdown_connections 2024-05-09T02:09:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.897+0000 7fe73d633700 1 -- 172.21.15.19:0/2369668208 wait complete. 2024-05-09T02:09:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.898+0000 7fe73d633700 1 Processor -- start 2024-05-09T02:09:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.898+0000 7fe73d633700 1 -- start start 2024-05-09T02:09:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.898+0000 7fe73d633700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7381034e0 0x7fe73819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.898+0000 7fe73d633700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe738107e10 0x7fe73819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.898+0000 7fe73d633700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe738108760 0x7fe73819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.898+0000 7fe73d633700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe738113390 con 0x7fe738107e10 2024-05-09T02:09:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.898+0000 7fe73d633700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe738113210 con 0x7fe738108760 2024-05-09T02:09:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.898+0000 7fe73d633700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe738113510 con 0x7fe7381034e0 2024-05-09T02:09:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.898+0000 7fe7367fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe738107e10 0x7fe73819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.898+0000 7fe7367fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe738107e10 0x7fe73819b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39550/0 (socket says 172.21.15.19:39550) 2024-05-09T02:09:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.898+0000 7fe7367fc700 1 -- 172.21.15.19:0/927569540 learned_addr learned my addr 172.21.15.19:0/927569540 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:09:59.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.899+0000 7fe736ffd700 1 --2- 172.21.15.19:0/927569540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7381034e0 0x7fe73819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:59.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.899+0000 7fe7377fe700 1 --2- 172.21.15.19:0/927569540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe738108760 0x7fe73819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:59.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.899+0000 7fe7367fc700 1 -- 172.21.15.19:0/927569540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7381034e0 msgr2=0x7fe73819aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:59.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.899+0000 7fe7367fc700 1 --2- 172.21.15.19:0/927569540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7381034e0 0x7fe73819aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:59.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.899+0000 7fe7367fc700 1 -- 172.21.15.19:0/927569540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe738108760 msgr2=0x7fe73819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:09:59.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.899+0000 7fe7367fc700 1 --2- 172.21.15.19:0/927569540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe738108760 0x7fe73819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:09:59.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.899+0000 7fe7367fc700 1 -- 172.21.15.19:0/927569540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73819fd80 con 0x7fe738107e10 2024-05-09T02:09:59.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.899+0000 7fe7367fc700 1 --2- 172.21.15.19:0/927569540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe738107e10 0x7fe73819b450 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00d980 tx=0x7fe72c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:59.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.899+0000 7fe71ffff700 1 -- 172.21.15.19:0/927569540 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c017070 con 0x7fe738107e10 2024-05-09T02:09:59.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.899+0000 7fe73d633700 1 -- 172.21.15.19:0/927569540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7381a0070 con 0x7fe738107e10 2024-05-09T02:09:59.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.899+0000 7fe71ffff700 1 -- 172.21.15.19:0/927569540 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe72c00fd40 con 0x7fe738107e10 2024-05-09T02:09:59.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.899+0000 7fe71ffff700 1 -- 172.21.15.19:0/927569540 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c00e3b0 con 0x7fe738107e10 2024-05-09T02:09:59.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.899+0000 7fe73d633700 1 -- 172.21.15.19:0/927569540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7381a7990 con 0x7fe738107e10 2024-05-09T02:09:59.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.900+0000 7fe73d633700 1 -- 172.21.15.19:0/927569540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7380512c0 con 0x7fe738107e10 2024-05-09T02:09:59.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.902+0000 7fe71ffff700 1 -- 172.21.15.19:0/927569540 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe72c010610 con 0x7fe738107e10 2024-05-09T02:09:59.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.903+0000 7fe71ffff700 1 --2- 172.21.15.19:0/927569540 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe7200b2880 0x7fe7200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:09:59.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.903+0000 7fe736ffd700 1 --2- 172.21.15.19:0/927569540 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe7200b2880 0x7fe7200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:09:59.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.903+0000 7fe71ffff700 1 -- 172.21.15.19:0/927569540 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fe72c0e1bf0 con 0x7fe738107e10 2024-05-09T02:09:59.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.903+0000 7fe736ffd700 1 --2- 172.21.15.19:0/927569540 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe7200b2880 0x7fe7200b4d90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe728005950 tx=0x7fe72800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:09:59.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:09:59.906+0000 7fe71ffff700 1 -- 172.21.15.19:0/927569540 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe72c0a98f0 con 0x7fe738107e10 2024-05-09T02:10:00.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:00.167+0000 7fe73d633700 1 -- 172.21.15.19:0/927569540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe73802e5f0 con 0x7fe738107e10 2024-05-09T02:10:00.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:00.168+0000 7fe71ffff700 1 -- 172.21.15.19:0/927569540 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11917 (secure 0 0 0) 0x7fe72c0a91f0 con 0x7fe738107e10 2024-05-09T02:10:00.170 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:10:00.171 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:08:11.346147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24455,24457,24473,24484,24463]},"id":1}]} 2024-05-09T02:10:00.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:00.170+0000 7fe73d633700 1 -- 172.21.15.19:0/927569540 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe7200b2880 msgr2=0x7fe7200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:00.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:00.170+0000 7fe73d633700 1 --2- 172.21.15.19:0/927569540 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe7200b2880 0x7fe7200b4d90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe728005950 tx=0x7fe72800b410 comp rx=0 tx=0).stop 2024-05-09T02:10:00.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:00.170+0000 7fe73d633700 1 -- 172.21.15.19:0/927569540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe738107e10 msgr2=0x7fe73819b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:00.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:00.170+0000 7fe73d633700 1 --2- 172.21.15.19:0/927569540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe738107e10 0x7fe73819b450 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00d980 tx=0x7fe72c00dc90 comp rx=0 tx=0).stop 2024-05-09T02:10:00.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:00.170+0000 7fe73d633700 1 -- 172.21.15.19:0/927569540 shutdown_connections 2024-05-09T02:10:00.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:00.170+0000 7fe73d633700 1 --2- 172.21.15.19:0/927569540 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fe7200b2880 0x7fe7200b4d90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:00.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:00.170+0000 7fe73d633700 1 --2- 172.21.15.19:0/927569540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe738108760 0x7fe73819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:00.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:00.170+0000 7fe73d633700 1 --2- 172.21.15.19:0/927569540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7381034e0 0x7fe73819aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:00.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:00.170+0000 7fe73d633700 1 --2- 172.21.15.19:0/927569540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe738107e10 0x7fe73819b450 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:00.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:00.170+0000 7fe73d633700 1 -- 172.21.15.19:0/927569540 >> 172.21.15.19:0/927569540 conn(0x7fe7380fef30 msgr2=0x7fe73806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:00.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:00.170+0000 7fe73d633700 1 -- 172.21.15.19:0/927569540 shutdown_connections 2024-05-09T02:10:00.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:00.171+0000 7fe73d633700 1 -- 172.21.15.19:0/927569540 wait complete. 2024-05-09T02:10:00.174 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T02:10:00.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:00 smithi019 ceph-mon[30151]: overall HEALTH_OK 2024-05-09T02:10:00.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:00 smithi059 ceph-mon[34781]: overall HEALTH_OK 2024-05-09T02:10:00.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:00 smithi196 ceph-mon[34718]: overall HEALTH_OK 2024-05-09T02:10:00.841 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:10:01.059 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:01 smithi019 ceph-mon[30151]: pgmap v359: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T02:10:01.060 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:01 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/927569540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:10:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:01.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:01 smithi059 ceph-mon[34781]: pgmap v359: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T02:10:01.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:01 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/927569540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:10:01.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:01 smithi196 ceph-mon[34718]: pgmap v359: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T02:10:01.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:01 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/927569540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:10:03.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.079+0000 7f1558c80700 1 -- 172.21.15.19:0/3308281803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15541034c0 msgr2=0x7f15541038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:03.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.079+0000 7f1558c80700 1 --2- 172.21.15.19:0/3308281803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15541034c0 0x7f15541038d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f154400b0d0 tx=0x7f154400b3e0 comp rx=0 tx=0).stop 2024-05-09T02:10:03.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.080+0000 7f1558c80700 1 -- 172.21.15.19:0/3308281803 shutdown_connections 2024-05-09T02:10:03.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.080+0000 7f1558c80700 1 --2- 172.21.15.19:0/3308281803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1554108740 0x7f15541100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:03.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.080+0000 7f1558c80700 1 --2- 172.21.15.19:0/3308281803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15541034c0 0x7f15541038d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:03.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.080+0000 7f1558c80700 1 --2- 172.21.15.19:0/3308281803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1554107df0 0x7f1554108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:03.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.080+0000 7f1558c80700 1 -- 172.21.15.19:0/3308281803 >> 172.21.15.19:0/3308281803 conn(0x7f15540fedf0 msgr2=0x7f1554101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:03.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.080+0000 7f1558c80700 1 -- 172.21.15.19:0/3308281803 shutdown_connections 2024-05-09T02:10:03.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.080+0000 7f1558c80700 1 -- 172.21.15.19:0/3308281803 wait complete. 2024-05-09T02:10:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.081+0000 7f1558c80700 1 Processor -- start 2024-05-09T02:10:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.081+0000 7f1558c80700 1 -- start start 2024-05-09T02:10:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.081+0000 7f1558c80700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15541034c0 0x7f155419d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.081+0000 7f1558c80700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554107df0 0x7f155419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.081+0000 7f1558c80700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1554108740 0x7f15541a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.081+0000 7f1558c80700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15541144e0 con 0x7f1554107df0 2024-05-09T02:10:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.081+0000 7f1558c80700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1554114360 con 0x7f1554108740 2024-05-09T02:10:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.081+0000 7f1558c80700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1554114660 con 0x7f15541034c0 2024-05-09T02:10:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.081+0000 7f1551d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554107df0 0x7f155419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.081+0000 7f1551d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554107df0 0x7f155419d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39570/0 (socket says 172.21.15.19:39570) 2024-05-09T02:10:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.081+0000 7f1551d9b700 1 -- 172.21.15.19:0/3485232535 learned_addr learned my addr 172.21.15.19:0/3485232535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.082+0000 7f1552d9d700 1 --2- 172.21.15.19:0/3485232535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1554108740 0x7f15541a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.082+0000 7f155259c700 1 --2- 172.21.15.19:0/3485232535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15541034c0 0x7f155419d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.082+0000 7f1551d9b700 1 -- 172.21.15.19:0/3485232535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15541034c0 msgr2=0x7f155419d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.082+0000 7f1551d9b700 1 --2- 172.21.15.19:0/3485232535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15541034c0 0x7f155419d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.082+0000 7f1551d9b700 1 -- 172.21.15.19:0/3485232535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1554108740 msgr2=0x7f15541a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.082+0000 7f1551d9b700 1 --2- 172.21.15.19:0/3485232535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1554108740 0x7f15541a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.082+0000 7f1551d9b700 1 -- 172.21.15.19:0/3485232535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15541a1eb0 con 0x7f1554107df0 2024-05-09T02:10:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.082+0000 7f1551d9b700 1 --2- 172.21.15.19:0/3485232535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554107df0 0x7f155419d630 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f153c00c9e0 tx=0x7f153c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.082+0000 7f154b7fe700 1 -- 172.21.15.19:0/3485232535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c007aa0 con 0x7f1554107df0 2024-05-09T02:10:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.082+0000 7f1558c80700 1 -- 172.21.15.19:0/3485232535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15541a21a0 con 0x7f1554107df0 2024-05-09T02:10:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.082+0000 7f154b7fe700 1 -- 172.21.15.19:0/3485232535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f153c007c40 con 0x7f1554107df0 2024-05-09T02:10:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.082+0000 7f1558c80700 1 -- 172.21.15.19:0/3485232535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15541a9b70 con 0x7f1554107df0 2024-05-09T02:10:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.082+0000 7f154b7fe700 1 -- 172.21.15.19:0/3485232535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c00fe60 con 0x7f1554107df0 2024-05-09T02:10:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.084+0000 7f1558c80700 1 -- 172.21.15.19:0/3485232535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15540512c0 con 0x7f1554107df0 2024-05-09T02:10:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.085+0000 7f154b7fe700 1 -- 172.21.15.19:0/3485232535 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f153c00f450 con 0x7f1554107df0 2024-05-09T02:10:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.086+0000 7f154b7fe700 1 --2- 172.21.15.19:0/3485232535 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f15400b2880 0x7f15400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.086+0000 7f155259c700 1 --2- 172.21.15.19:0/3485232535 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f15400b2880 0x7f15400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.086+0000 7f154b7fe700 1 -- 172.21.15.19:0/3485232535 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f153c0e2150 con 0x7f1554107df0 2024-05-09T02:10:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.086+0000 7f155259c700 1 --2- 172.21.15.19:0/3485232535 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f15400b2880 0x7f15400b4d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1544009900 tx=0x7f154400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.091+0000 7f154b7fe700 1 -- 172.21.15.19:0/3485232535 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f153c014030 con 0x7f1554107df0 2024-05-09T02:10:03.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.356+0000 7f1558c80700 1 -- 172.21.15.19:0/3485232535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f155419e3f0 con 0x7f1554107df0 2024-05-09T02:10:03.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:03 smithi019 ceph-mon[30151]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T02:10:03.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.357+0000 7f154b7fe700 1 -- 172.21.15.19:0/3485232535 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11917 (secure 0 0 0) 0x7f153c0a9e50 con 0x7f1554107df0 2024-05-09T02:10:03.360 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:10:03.360 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:08:11.346147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24455,24457,24473,24484,24463]},"id":1}]} 2024-05-09T02:10:03.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.360+0000 7f1558c80700 1 -- 172.21.15.19:0/3485232535 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f15400b2880 msgr2=0x7f15400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:03.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.360+0000 7f1558c80700 1 --2- 172.21.15.19:0/3485232535 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f15400b2880 0x7f15400b4d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1544009900 tx=0x7f154400b040 comp rx=0 tx=0).stop 2024-05-09T02:10:03.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.360+0000 7f1558c80700 1 -- 172.21.15.19:0/3485232535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554107df0 msgr2=0x7f155419d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:03.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.360+0000 7f1558c80700 1 --2- 172.21.15.19:0/3485232535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554107df0 0x7f155419d630 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f153c00c9e0 tx=0x7f153c00ccf0 comp rx=0 tx=0).stop 2024-05-09T02:10:03.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.360+0000 7f1558c80700 1 -- 172.21.15.19:0/3485232535 shutdown_connections 2024-05-09T02:10:03.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.360+0000 7f1558c80700 1 --2- 172.21.15.19:0/3485232535 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f15400b2880 0x7f15400b4d90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:03.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.360+0000 7f1558c80700 1 --2- 172.21.15.19:0/3485232535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1554108740 0x7f15541a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:03.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.360+0000 7f1558c80700 1 --2- 172.21.15.19:0/3485232535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15541034c0 0x7f155419d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:03.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.360+0000 7f1558c80700 1 --2- 172.21.15.19:0/3485232535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554107df0 0x7f155419d630 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:03.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.360+0000 7f1558c80700 1 -- 172.21.15.19:0/3485232535 >> 172.21.15.19:0/3485232535 conn(0x7f15540fedf0 msgr2=0x7f15541096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:03.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.360+0000 7f1558c80700 1 -- 172.21.15.19:0/3485232535 shutdown_connections 2024-05-09T02:10:03.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:03.360+0000 7f1558c80700 1 -- 172.21.15.19:0/3485232535 wait complete. 2024-05-09T02:10:03.364 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T02:10:03.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:03 smithi059 ceph-mon[34781]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T02:10:03.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:03 smithi196 ceph-mon[34718]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T02:10:04.055 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T02:10:04.315 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:04.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:04 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3485232535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:10:04.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:04 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3485232535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:10:04.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:04 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3485232535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:10:05.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:05 smithi196 ceph-mon[34718]: pgmap v361: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T02:10:05.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:05 smithi059 ceph-mon[34781]: pgmap v361: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T02:10:05.467 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:05 smithi019 ceph-mon[30151]: pgmap v361: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T02:10:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.067+0000 7fae9a7cb700 1 -- 172.21.15.19:0/2367959136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 msgr2=0x7fae941100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.067+0000 7fae9a7cb700 1 --2- 172.21.15.19:0/2367959136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae941100a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00b0d0 tx=0x7fae8c00b3e0 comp rx=0 tx=0).stop 2024-05-09T02:10:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.067+0000 7fae9a7cb700 1 -- 172.21.15.19:0/2367959136 shutdown_connections 2024-05-09T02:10:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.067+0000 7fae9a7cb700 1 --2- 172.21.15.19:0/2367959136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae941100a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.067+0000 7fae9a7cb700 1 --2- 172.21.15.19:0/2367959136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae941034c0 0x7fae941038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.067+0000 7fae9a7cb700 1 --2- 172.21.15.19:0/2367959136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae94107df0 0x7fae94108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.067+0000 7fae9a7cb700 1 -- 172.21.15.19:0/2367959136 >> 172.21.15.19:0/2367959136 conn(0x7fae940fedf0 msgr2=0x7fae94101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.067+0000 7fae9a7cb700 1 -- 172.21.15.19:0/2367959136 shutdown_connections 2024-05-09T02:10:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.067+0000 7fae9a7cb700 1 -- 172.21.15.19:0/2367959136 wait complete. 2024-05-09T02:10:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.068+0000 7fae9a7cb700 1 Processor -- start 2024-05-09T02:10:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.068+0000 7fae9a7cb700 1 -- start start 2024-05-09T02:10:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae9a7cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae941034c0 0x7fae9419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae9a7cb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae94107df0 0x7fae9419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae9a7cb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae94199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae9a7cb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae94113370 con 0x7fae94108740 2024-05-09T02:10:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae9a7cb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae941131f0 con 0x7fae941034c0 2024-05-09T02:10:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae9a7cb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae941134f0 con 0x7fae94107df0 2024-05-09T02:10:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae98d68700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae94199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae98d68700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae94199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39592/0 (socket says 172.21.15.19:39592) 2024-05-09T02:10:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae98d68700 1 -- 172.21.15.19:0/2842147351 learned_addr learned my addr 172.21.15.19:0/2842147351 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae93fff700 1 --2- 172.21.15.19:0/2842147351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae941034c0 0x7fae9419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae937fe700 1 --2- 172.21.15.19:0/2842147351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae94107df0 0x7fae9419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae98d68700 1 -- 172.21.15.19:0/2842147351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae94107df0 msgr2=0x7fae9419fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae98d68700 1 --2- 172.21.15.19:0/2842147351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae94107df0 0x7fae9419fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae98d68700 1 -- 172.21.15.19:0/2842147351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae941034c0 msgr2=0x7fae9419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae98d68700 1 --2- 172.21.15.19:0/2842147351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae941034c0 0x7fae9419f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae98d68700 1 -- 172.21.15.19:0/2842147351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9419a1d0 con 0x7fae94108740 2024-05-09T02:10:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.069+0000 7fae98d68700 1 --2- 172.21.15.19:0/2842147351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae94199af0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0095c0 tx=0x7fae8c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.070+0000 7fae917fa700 1 -- 172.21.15.19:0/2842147351 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c01f070 con 0x7fae94108740 2024-05-09T02:10:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.070+0000 7fae917fa700 1 -- 172.21.15.19:0/2842147351 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae8c003c00 con 0x7fae94108740 2024-05-09T02:10:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.070+0000 7fae9a7cb700 1 -- 172.21.15.19:0/2842147351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9419a460 con 0x7fae94108740 2024-05-09T02:10:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.070+0000 7fae917fa700 1 -- 172.21.15.19:0/2842147351 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c01b6a0 con 0x7fae94108740 2024-05-09T02:10:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.070+0000 7fae9a7cb700 1 -- 172.21.15.19:0/2842147351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae940759f0 con 0x7fae94108740 2024-05-09T02:10:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.071+0000 7fae9a7cb700 1 -- 172.21.15.19:0/2842147351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae94104970 con 0x7fae94108740 2024-05-09T02:10:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.072+0000 7fae917fa700 1 -- 172.21.15.19:0/2842147351 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fae8c022070 con 0x7fae94108740 2024-05-09T02:10:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.073+0000 7fae917fa700 1 --2- 172.21.15.19:0/2842147351 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fae7c0b2880 0x7fae7c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.073+0000 7fae93fff700 1 --2- 172.21.15.19:0/2842147351 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fae7c0b2880 0x7fae7c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.073+0000 7fae917fa700 1 -- 172.21.15.19:0/2842147351 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fae8c00f020 con 0x7fae94108740 2024-05-09T02:10:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.074+0000 7fae93fff700 1 --2- 172.21.15.19:0/2842147351 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fae7c0b2880 0x7fae7c0b4d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fae9419ac30 tx=0x7fae840073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:06.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.077+0000 7fae917fa700 1 -- 172.21.15.19:0/2842147351 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae8c029020 con 0x7fae94108740 2024-05-09T02:10:06.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.293+0000 7fae9a7cb700 1 -- 172.21.15.19:0/2842147351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae9419afe0 con 0x7fae94108740 2024-05-09T02:10:06.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.295+0000 7fae917fa700 1 -- 172.21.15.19:0/2842147351 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22698 (secure 0 0 0) 0x7fae8c0ab6a0 con 0x7fae94108740 2024-05-09T02:10:06.298 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:10:06.298 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"fsid":"a0611f3e-0da7-11ef-bc97-c7b262605968","created":"2024-05-09T01:59:27.591343+0000","modified":"2024-05-09T02:08:25.302927+0000","last_up_change":"2024-05-09T02:06:45.565065+0000","last_in_change":"2024-05-09T02:06:24.967186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T02:02:50.018959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T02:07:43.283868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T02:07:47.110614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"996c6a3d-3e3a-451c-8789-36006e0b3dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6803","nonce":3002373576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6805","nonce":3002373576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6809","nonce":3002373576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6807","nonce":3002373576}]},"public_addr":"172.21.15.19:6803/3002373576","cluster_addr":"172.21.15.19:6805/3002373576","heartbeat_back_addr":"172.21.15.19:6809/3002373576","heartbeat_front_addr":"172.21.15.19:6807/3002373576","state":["exists","up"]},{"osd":1,"uuid":"1ac68d09-1198-4e81-86c7-e37f7c41aa5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6801","nonce":827458353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6803","nonce":827458353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6807","nonce":827458353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6805","nonce":827458353}]},"public_addr":"172.21.15.59:6801/827458353","cluster_addr":"172.21.15.59:6803/827458353","heartbeat_back_addr":"172.21.15.59:6807/827458353","heartbeat_front_addr":"172.21.15.59:6805/827458353","state":["exists","up"]},{"osd":2,"uuid":"818a0de1-04e6-471d-83df-a68833011b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6801","nonce":1659504639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6803","nonce":1659504639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6807","nonce":1659504639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6805","nonce":1659504639}]},"public_addr":"172.21.15.196:6801/1659504639","cluster_addr":"172.21.15.196:6803/1659504639","heartbeat_back_addr":"172.21.15.196:6807/1659504639","heartbeat_front_addr":"172.21.15.196:6805/1659504639","state":["exists","up"]},{"osd":3,"uuid":"7b204854-86b8-454d-aee0-8bb5775ce828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6811","nonce":2707671599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6813","nonce":2707671599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6817","nonce":2707671599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6815","nonce":2707671599}]},"public_addr":"172.21.15.19:6811/2707671599","cluster_addr":"172.21.15.19:6813/2707671599","heartbeat_back_addr":"172.21.15.19:6817/2707671599","heartbeat_front_addr":"172.21.15.19:6815/2707671599","state":["exists","up"]},{"osd":4,"uuid":"091667f1-e887-4a94-8723-6f85b676fef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6809","nonce":223037330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6811","nonce":223037330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6815","nonce":223037330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6813","nonce":223037330}]},"public_addr":"172.21.15.59:6809/223037330","cluster_addr":"172.21.15.59:6811/223037330","heartbeat_back_addr":"172.21.15.59:6815/223037330","heartbeat_front_addr":"172.21.15.59:6813/223037330","state":["exists","up"]},{"osd":5,"uuid":"2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6809","nonce":2516161898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6811","nonce":2516161898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6815","nonce":2516161898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6813","nonce":2516161898}]},"public_addr":"172.21.15.196:6809/2516161898","cluster_addr":"172.21.15.196:6811/2516161898","heartbeat_back_addr":"172.21.15.196:6815/2516161898","heartbeat_front_addr":"172.21.15.196:6813/2516161898","state":["exists","up"]},{"osd":6,"uuid":"4e352851-1279-475f-ba8b-eaf9ca73df74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6819","nonce":2150451637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6821","nonce":2150451637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6825","nonce":2150451637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6823","nonce":2150451637}]},"public_addr":"172.21.15.19:6819/2150451637","cluster_addr":"172.21.15.19:6821/2150451637","heartbeat_back_addr":"172.21.15.19:6825/2150451637","heartbeat_front_addr":"172.21.15.19:6823/2150451637","state":["exists","up"]},{"osd":7,"uuid":"83f53f59-9a60-4d87-9a8a-67e5c6f57004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6817","nonce":967902053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6819","nonce":967902053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6823","nonce":967902053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6821","nonce":967902053}]},"public_addr":"172.21.15.59:6817/967902053","cluster_addr":"172.21.15.59:6819/967902053","heartbeat_back_addr":"172.21.15.59:6823/967902053","heartbeat_front_addr":"172.21.15.59:6821/967902053","state":["exists","up"]},{"osd":8,"uuid":"a8f1bfb5-1cc8-4a65-8396-239bd781e1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6817","nonce":2946064259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6819","nonce":2946064259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6823","nonce":2946064259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6821","nonce":2946064259}]},"public_addr":"172.21.15.196:6817/2946064259","cluster_addr":"172.21.15.196:6819/2946064259","heartbeat_back_addr":"172.21.15.196:6823/2946064259","heartbeat_front_addr":"172.21.15.196:6821/2946064259","state":["exists","up"]},{"osd":9,"uuid":"73c58dc3-fcb0-4440-92b3-bf41a2e58fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6827","nonce":1735017886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6829","nonce":1735017886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6833","nonce":1735017886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6831","nonce":1735017886}]},"public_addr":"172.21.15.19:6827/1735017886","cluster_addr":"172.21.15.19:6829/1735017886","heartbeat_back_addr":"172.21.15.19:6833/1735017886","heartbeat_front_addr":"172.21.15.19:6831/1735017886","state":["exists","up"]},{"osd":10,"uuid":"8325282f-7325-4880-ab7d-b972717be9dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6825","nonce":2932899995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6827","nonce":2932899995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6831","nonce":2932899995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6829","nonce":2932899995}]},"public_addr":"172.21.15.59:6825/2932899995","cluster_addr":"172.21.15.59:6827/2932899995","heartbeat_back_addr":"172.21.15.59:6831/2932899995","heartbeat_front_addr":"172.21.15.59:6829/2932899995","state":["exists","up"]},{"osd":11,"uuid":"8b19228c-b286-43c8-a408-169782f13075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6825","nonce":2608023001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6827","nonce":2608023001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6831","nonce":2608023001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6829","nonce":2608023001}]},"public_addr":"172.21.15.196:6825/2608023001","cluster_addr":"172.21.15.196:6827/2608023001","heartbeat_back_addr":"172.21.15.196:6831/2608023001","heartbeat_front_addr":"172.21.15.196:6829/2608023001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:01:59.457748+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:23.640666+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:47.768243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:13.016746+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:39.060898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:04.508313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:31.366217+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:57.579758+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:23.177396+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:50.966399+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:16.838666+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:43.959387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3291838961":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1024133854":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1176455101":"2024-05-10T02:00:23.920230+0000","172.21.15.19:6801/2101052000":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/4018987404":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/3079777324":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6801/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/179703528":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/2101052000":"2024-05-10T02:00:23.920230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T02:10:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.297+0000 7fae9a7cb700 1 -- 172.21.15.19:0/2842147351 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fae7c0b2880 msgr2=0x7fae7c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.297+0000 7fae9a7cb700 1 --2- 172.21.15.19:0/2842147351 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fae7c0b2880 0x7fae7c0b4d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fae9419ac30 tx=0x7fae840073d0 comp rx=0 tx=0).stop 2024-05-09T02:10:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.297+0000 7fae9a7cb700 1 -- 172.21.15.19:0/2842147351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 msgr2=0x7fae94199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.297+0000 7fae9a7cb700 1 --2- 172.21.15.19:0/2842147351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae94199af0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0095c0 tx=0x7fae8c007a20 comp rx=0 tx=0).stop 2024-05-09T02:10:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.297+0000 7fae9a7cb700 1 -- 172.21.15.19:0/2842147351 shutdown_connections 2024-05-09T02:10:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.297+0000 7fae9a7cb700 1 --2- 172.21.15.19:0/2842147351 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fae7c0b2880 0x7fae7c0b4d90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.297+0000 7fae9a7cb700 1 --2- 172.21.15.19:0/2842147351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94108740 0x7fae94199af0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.297+0000 7fae9a7cb700 1 --2- 172.21.15.19:0/2842147351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae941034c0 0x7fae9419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.297+0000 7fae9a7cb700 1 --2- 172.21.15.19:0/2842147351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae94107df0 0x7fae9419fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.297+0000 7fae9a7cb700 1 -- 172.21.15.19:0/2842147351 >> 172.21.15.19:0/2842147351 conn(0x7fae940fedf0 msgr2=0x7fae94107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.298+0000 7fae9a7cb700 1 -- 172.21.15.19:0/2842147351 shutdown_connections 2024-05-09T02:10:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:06.298+0000 7fae9a7cb700 1 -- 172.21.15.19:0/2842147351 wait complete. 2024-05-09T02:10:07.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:10:07.289 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:07.412 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:07 smithi019 ceph-mon[30151]: pgmap v362: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T02:10:07.412 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:07 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2842147351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:10:07.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:07 smithi059 ceph-mon[34781]: pgmap v362: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T02:10:07.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:07 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2842147351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:10:07.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:07 smithi196 ceph-mon[34718]: pgmap v362: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T02:10:07.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:07 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2842147351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:10:09.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.022+0000 7f4bfb1db700 1 -- 172.21.15.19:0/2014369864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bf4108740 msgr2=0x7f4bf41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:09.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.022+0000 7f4bfb1db700 1 --2- 172.21.15.19:0/2014369864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bf4108740 0x7f4bf41100a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00b0d0 tx=0x7f4bec00b3e0 comp rx=0 tx=0).stop 2024-05-09T02:10:09.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.022+0000 7f4bfb1db700 1 -- 172.21.15.19:0/2014369864 shutdown_connections 2024-05-09T02:10:09.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.022+0000 7f4bfb1db700 1 --2- 172.21.15.19:0/2014369864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bf4108740 0x7f4bf41100a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:09.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.022+0000 7f4bfb1db700 1 --2- 172.21.15.19:0/2014369864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf41034c0 0x7f4bf41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:09.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.022+0000 7f4bfb1db700 1 --2- 172.21.15.19:0/2014369864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf4107df0 0x7f4bf4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.022+0000 7f4bfb1db700 1 -- 172.21.15.19:0/2014369864 >> 172.21.15.19:0/2014369864 conn(0x7f4bf40fedf0 msgr2=0x7f4bf4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.022+0000 7f4bfb1db700 1 -- 172.21.15.19:0/2014369864 shutdown_connections 2024-05-09T02:10:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.022+0000 7f4bfb1db700 1 -- 172.21.15.19:0/2014369864 wait complete. 2024-05-09T02:10:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.023+0000 7f4bfb1db700 1 Processor -- start 2024-05-09T02:10:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.023+0000 7f4bfb1db700 1 -- start start 2024-05-09T02:10:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.023+0000 7f4bfb1db700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf41034c0 0x7f4bf40ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.023+0000 7f4bfb1db700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf4107df0 0x7f4bf40ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.023+0000 7f4bfb1db700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bf4108740 0x7f4bf40ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.023+0000 7f4bfb1db700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4113370 con 0x7f4bf4108740 2024-05-09T02:10:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.023+0000 7f4bfb1db700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf41131f0 con 0x7f4bf4107df0 2024-05-09T02:10:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.023+0000 7f4bfb1db700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf41134f0 con 0x7f4bf41034c0 2024-05-09T02:10:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.024+0000 7f4bf9778700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bf4108740 0x7f4bf40ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.024+0000 7f4bf9778700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bf4108740 0x7f4bf40ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60850/0 (socket says 172.21.15.19:60850) 2024-05-09T02:10:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.024+0000 7f4bf9778700 1 -- 172.21.15.19:0/3856938017 learned_addr learned my addr 172.21.15.19:0/3856938017 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.024+0000 7f4bf3fff700 1 --2- 172.21.15.19:0/3856938017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf4107df0 0x7f4bf40ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.024+0000 7f4bf8f77700 1 --2- 172.21.15.19:0/3856938017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf41034c0 0x7f4bf40ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.024+0000 7f4bf9778700 1 -- 172.21.15.19:0/3856938017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf41034c0 msgr2=0x7f4bf40ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.024+0000 7f4bf9778700 1 --2- 172.21.15.19:0/3856938017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf41034c0 0x7f4bf40ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.024+0000 7f4bf9778700 1 -- 172.21.15.19:0/3856938017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf4107df0 msgr2=0x7f4bf40ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.024+0000 7f4bf9778700 1 --2- 172.21.15.19:0/3856938017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf4107df0 0x7f4bf40ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.024+0000 7f4bf9778700 1 -- 172.21.15.19:0/3856938017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf41a90c0 con 0x7f4bf4108740 2024-05-09T02:10:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.024+0000 7f4bf8f77700 1 --2- 172.21.15.19:0/3856938017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf41034c0 0x7f4bf40ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:10:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.024+0000 7f4bf9778700 1 --2- 172.21.15.19:0/3856938017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bf4108740 0x7f4bf40ffd50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0095c0 tx=0x7f4bec007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.024+0000 7f4bf1ffb700 1 -- 172.21.15.19:0/3856938017 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec01f070 con 0x7f4bf4108740 2024-05-09T02:10:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.024+0000 7f4bfb1db700 1 -- 172.21.15.19:0/3856938017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf41a9350 con 0x7f4bf4108740 2024-05-09T02:10:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.024+0000 7f4bf1ffb700 1 -- 172.21.15.19:0/3856938017 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bec003c00 con 0x7f4bf4108740 2024-05-09T02:10:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.025+0000 7f4bf1ffb700 1 -- 172.21.15.19:0/3856938017 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec01b6a0 con 0x7f4bf4108740 2024-05-09T02:10:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.025+0000 7f4bfb1db700 1 -- 172.21.15.19:0/3856938017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf41a97b0 con 0x7f4bf4108740 2024-05-09T02:10:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.026+0000 7f4bfb1db700 1 -- 172.21.15.19:0/3856938017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf40512c0 con 0x7f4bf4108740 2024-05-09T02:10:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.028+0000 7f4bf1ffb700 1 -- 172.21.15.19:0/3856938017 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4bec022070 con 0x7f4bf4108740 2024-05-09T02:10:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.029+0000 7f4bf1ffb700 1 --2- 172.21.15.19:0/3856938017 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f4be40b2880 0x7f4be40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.029+0000 7f4bf8f77700 1 --2- 172.21.15.19:0/3856938017 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f4be40b2880 0x7f4be40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.029+0000 7f4bf1ffb700 1 -- 172.21.15.19:0/3856938017 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f4bec00f020 con 0x7f4bf4108740 2024-05-09T02:10:09.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.029+0000 7f4bf8f77700 1 --2- 172.21.15.19:0/3856938017 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f4be40b2880 0x7f4be40b4d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4be8005fd0 tx=0x7f4be8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.034+0000 7f4bf1ffb700 1 -- 172.21.15.19:0/3856938017 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4bec029020 con 0x7f4bf4108740 2024-05-09T02:10:09.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.297+0000 7f4bfb1db700 1 -- 172.21.15.19:0/3856938017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bf41a9ae0 con 0x7f4bf4108740 2024-05-09T02:10:09.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:09 smithi019 ceph-mon[30151]: pgmap v363: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T02:10:09.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.298+0000 7f4bf1ffb700 1 -- 172.21.15.19:0/3856938017 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11917 (secure 0 0 0) 0x7f4bec0ab260 con 0x7f4bf4108740 2024-05-09T02:10:09.300 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:10:09.301 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:08:11.346147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24455,24457,24473,24484,24463]},"id":1}]} 2024-05-09T02:10:09.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.300+0000 7f4bfb1db700 1 -- 172.21.15.19:0/3856938017 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f4be40b2880 msgr2=0x7f4be40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:09.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.300+0000 7f4bfb1db700 1 --2- 172.21.15.19:0/3856938017 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f4be40b2880 0x7f4be40b4d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4be8005fd0 tx=0x7f4be8005e30 comp rx=0 tx=0).stop 2024-05-09T02:10:09.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.300+0000 7f4bfb1db700 1 -- 172.21.15.19:0/3856938017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bf4108740 msgr2=0x7f4bf40ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:09.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.300+0000 7f4bfb1db700 1 --2- 172.21.15.19:0/3856938017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bf4108740 0x7f4bf40ffd50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0095c0 tx=0x7f4bec007a20 comp rx=0 tx=0).stop 2024-05-09T02:10:09.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.300+0000 7f4bfb1db700 1 -- 172.21.15.19:0/3856938017 shutdown_connections 2024-05-09T02:10:09.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.300+0000 7f4bfb1db700 1 --2- 172.21.15.19:0/3856938017 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f4be40b2880 0x7f4be40b4d90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:09.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.300+0000 7f4bfb1db700 1 --2- 172.21.15.19:0/3856938017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bf4108740 0x7f4bf40ffd50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:09.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.300+0000 7f4bfb1db700 1 --2- 172.21.15.19:0/3856938017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf41034c0 0x7f4bf40ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:09.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.300+0000 7f4bfb1db700 1 --2- 172.21.15.19:0/3856938017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf4107df0 0x7f4bf40ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:09.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.300+0000 7f4bfb1db700 1 -- 172.21.15.19:0/3856938017 >> 172.21.15.19:0/3856938017 conn(0x7f4bf40fedf0 msgr2=0x7f4bf4107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:09.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.300+0000 7f4bfb1db700 1 -- 172.21.15.19:0/3856938017 shutdown_connections 2024-05-09T02:10:09.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:09.300+0000 7f4bfb1db700 1 -- 172.21.15.19:0/3856938017 wait complete. 2024-05-09T02:10:09.304 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T02:10:09.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:09 smithi059 ceph-mon[34781]: pgmap v363: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T02:10:09.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:09 smithi196 ceph-mon[34718]: pgmap v363: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T02:10:09.963 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T02:07:50.316306+0000', 'modified': '2024-05-09T02:08:11.346147+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24484, 'mds_1': 24473, 'mds_2': 24463, 'mds_3': 24457, 'mds_4': 24455}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14604': {'gid': 14604, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6839/810391856', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 810391856}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 810391856}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14616': {'gid': 14616, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6841/3162604494', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 3162604494}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 3162604494}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6833/2040289281', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6832', 'nonce': 2040289281}, {'type': 'v1', 'addr': '172.21.15.59:6833', 'nonce': 2040289281}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/416853517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 416853517}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 416853517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6835/978263217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 978263217}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 978263217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.59:6837/2268419175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 2268419175}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2268419175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.196:6835/442694937', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 442694937}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 442694937}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.196:6837/1033177921', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 1033177921}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 1033177921}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.59:6839/1206568409', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6838', 'nonce': 1206568409}, {'type': 'v1', 'addr': '172.21.15.59:6839', 'nonce': 1206568409}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.196:6839/89436664', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 89436664}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 89436664}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24484, 'qdb_cluster': [24455, 24457, 24473, 24484, 24463]} 2024-05-09T02:10:09.963 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-09T02:10:09.963 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-09T02:10:10.226 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:10.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:10 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3856938017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:10:10.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:10 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3856938017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:10:10.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:10 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3856938017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:10:11.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:11 smithi059 ceph-mon[34781]: pgmap v364: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-09T02:10:11.531 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:11 smithi019 ceph-mon[30151]: pgmap v364: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-09T02:10:11.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:11 smithi196 ceph-mon[34718]: pgmap v364: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-09T02:10:11.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.934+0000 7f12f7cfc700 1 -- 172.21.15.19:0/3072224658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f01045f0 msgr2=0x7f12f0104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:11.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.934+0000 7f12f7cfc700 1 --2- 172.21.15.19:0/3072224658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f01045f0 0x7f12f0104a00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0099c0 tx=0x7f12ec009cd0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.934+0000 7f12f7cfc700 1 -- 172.21.15.19:0/3072224658 shutdown_connections 2024-05-09T02:10:11.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.934+0000 7f12f7cfc700 1 --2- 172.21.15.19:0/3072224658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12f0109870 0x7f12f01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.934+0000 7f12f7cfc700 1 --2- 172.21.15.19:0/3072224658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f01045f0 0x7f12f0104a00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.934+0000 7f12f7cfc700 1 --2- 172.21.15.19:0/3072224658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f0108f20 0x7f12f0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.934+0000 7f12f7cfc700 1 -- 172.21.15.19:0/3072224658 >> 172.21.15.19:0/3072224658 conn(0x7f12f00fffe0 msgr2=0x7f12f0102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:11.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.934+0000 7f12f7cfc700 1 -- 172.21.15.19:0/3072224658 shutdown_connections 2024-05-09T02:10:11.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.934+0000 7f12f7cfc700 1 -- 172.21.15.19:0/3072224658 wait complete. 2024-05-09T02:10:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.935+0000 7f12f7cfc700 1 Processor -- start 2024-05-09T02:10:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.935+0000 7f12f7cfc700 1 -- start start 2024-05-09T02:10:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.935+0000 7f12f7cfc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f01045f0 0x7f12f019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.935+0000 7f12f7cfc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f0108f20 0x7f12f019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.935+0000 7f12f7cfc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12f0109870 0x7f12f019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.935+0000 7f12f7cfc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01144a0 con 0x7f12f01045f0 2024-05-09T02:10:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.935+0000 7f12f7cfc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f12f0114320 con 0x7f12f0109870 2024-05-09T02:10:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.935+0000 7f12f7cfc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12f0114620 con 0x7f12f0108f20 2024-05-09T02:10:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.935+0000 7f12f5a98700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f01045f0 0x7f12f019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.935+0000 7f12f5a98700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f01045f0 0x7f12f019ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60876/0 (socket says 172.21.15.19:60876) 2024-05-09T02:10:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.935+0000 7f12f5a98700 1 -- 172.21.15.19:0/2211873443 learned_addr learned my addr 172.21.15.19:0/2211873443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.935+0000 7f12f5297700 1 --2- 172.21.15.19:0/2211873443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f0108f20 0x7f12f019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.935+0000 7f12f6299700 1 --2- 172.21.15.19:0/2211873443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12f0109870 0x7f12f019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.935+0000 7f12f5a98700 1 -- 172.21.15.19:0/2211873443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f0108f20 msgr2=0x7f12f019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.936+0000 7f12f5a98700 1 --2- 172.21.15.19:0/2211873443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f0108f20 0x7f12f019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.936+0000 7f12f5a98700 1 -- 172.21.15.19:0/2211873443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12f0109870 msgr2=0x7f12f019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.936+0000 7f12f5a98700 1 --2- 172.21.15.19:0/2211873443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12f0109870 0x7f12f019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.936+0000 7f12f5a98700 1 -- 172.21.15.19:0/2211873443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f019fcf0 con 0x7f12f01045f0 2024-05-09T02:10:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.936+0000 7f12f5a98700 1 --2- 172.21.15.19:0/2211873443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f01045f0 0x7f12f019ae70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00b580 tx=0x7f12ec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.936+0000 7f12e6ffd700 1 -- 172.21.15.19:0/2211873443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec027070 con 0x7f12f01045f0 2024-05-09T02:10:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.936+0000 7f12e6ffd700 1 -- 172.21.15.19:0/2211873443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12ec003f20 con 0x7f12f01045f0 2024-05-09T02:10:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.936+0000 7f12f7cfc700 1 -- 172.21.15.19:0/2211873443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f019ff80 con 0x7f12f01045f0 2024-05-09T02:10:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.936+0000 7f12e6ffd700 1 -- 172.21.15.19:0/2211873443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec022d70 con 0x7f12f01045f0 2024-05-09T02:10:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.936+0000 7f12f7cfc700 1 -- 172.21.15.19:0/2211873443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f01a0460 con 0x7f12f01045f0 2024-05-09T02:10:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.939+0000 7f12e6ffd700 1 -- 172.21.15.19:0/2211873443 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f12ec022500 con 0x7f12f01045f0 2024-05-09T02:10:11.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.939+0000 7f12e6ffd700 1 --2- 172.21.15.19:0/2211873443 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f12dc0b2880 0x7f12dc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.940+0000 7f12f5297700 1 --2- 172.21.15.19:0/2211873443 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f12dc0b2880 0x7f12dc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.940+0000 7f12e6ffd700 1 -- 172.21.15.19:0/2211873443 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f12ec0e2f20 con 0x7f12f01045f0 2024-05-09T02:10:11.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.940+0000 7f1306712b80 1 Processor -- start 2024-05-09T02:10:11.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.940+0000 7f12f5297700 1 --2- 172.21.15.19:0/2211873443 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f12dc0b2880 0x7f12dc0b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f12f0103510 tx=0x7f12e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:11.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.940+0000 7f1306712b80 1 -- start start 2024-05-09T02:10:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.940+0000 7f1306712b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558625344cb0 0x55862544ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.940+0000 7f1306712b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586252ca450 0x55862544bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.940+0000 7f1306712b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558625450190 0x5586254505a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.940+0000 7f1306712b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55862536a050 con 0x5586252ca450 2024-05-09T02:10:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.940+0000 7f1306712b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55862530f950 con 0x558625344cb0 2024-05-09T02:10:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.940+0000 7f1306712b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558625366010 con 0x558625450190 2024-05-09T02:10:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.941+0000 7f12f5297700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586252ca450 0x55862544bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.941+0000 7f12f5297700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586252ca450 0x55862544bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60888/0 (socket says 172.21.15.19:60888) 2024-05-09T02:10:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.941+0000 7f12f5297700 1 -- 172.21.15.19:0/551675022 learned_addr learned my addr 172.21.15.19:0/551675022 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.941+0000 7f12f5a98700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558625344cb0 0x55862544ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.941+0000 7f12f6299700 1 --2- 172.21.15.19:0/551675022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558625450190 0x5586254505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.941+0000 7f12f5297700 1 -- 172.21.15.19:0/551675022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558625450190 msgr2=0x5586254505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.941+0000 7f12f5297700 1 --2- 172.21.15.19:0/551675022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558625450190 0x5586254505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.941+0000 7f12f5297700 1 -- 172.21.15.19:0/551675022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558625344cb0 msgr2=0x55862544ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.941+0000 7f12f5297700 1 --2- 172.21.15.19:0/551675022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558625344cb0 0x55862544ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.941+0000 7f12f5297700 1 -- 172.21.15.19:0/551675022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586253004b0 con 0x5586252ca450 2024-05-09T02:10:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.941+0000 7f12f5297700 1 --2- 172.21.15.19:0/551675022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586252ca450 0x55862544bfe0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f12e0010f40 tx=0x7f12e0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278f926c676e84be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.941+0000 7f12dbfff700 1 -- 172.21.15.19:0/551675022 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e000be60 con 0x5586252ca450 2024-05-09T02:10:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.941+0000 7f12dbfff700 1 -- 172.21.15.19:0/551675022 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e0011db0 con 0x5586252ca450 2024-05-09T02:10:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.941+0000 7f12dbfff700 1 -- 172.21.15.19:0/551675022 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e001c720 con 0x5586252ca450 2024-05-09T02:10:11.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.942+0000 7f1306712b80 1 -- 172.21.15.19:0/551675022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586252ca450 msgr2=0x55862544bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:11.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.942+0000 7f1306712b80 1 --2- 172.21.15.19:0/551675022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586252ca450 0x55862544bfe0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f12e0010f40 tx=0x7f12e0013000 comp rx=0 tx=0).stop 2024-05-09T02:10:11.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.942+0000 7f1306712b80 1 -- 172.21.15.19:0/551675022 shutdown_connections 2024-05-09T02:10:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.942+0000 7f1306712b80 1 --2- 172.21.15.19:0/551675022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558625450190 0x5586254505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.942+0000 7f1306712b80 1 --2- 172.21.15.19:0/551675022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558625344cb0 0x55862544ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.942+0000 7f1306712b80 1 --2- 172.21.15.19:0/551675022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586252ca450 0x55862544bfe0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.942+0000 7f1306712b80 1 -- 172.21.15.19:0/551675022 >> 172.21.15.19:0/551675022 conn(0x5586252cdf20 msgr2=0x5586254421d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.942+0000 7f1306712b80 1 -- 172.21.15.19:0/551675022 shutdown_connections 2024-05-09T02:10:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.942+0000 7f1306712b80 1 -- 172.21.15.19:0/551675022 wait complete. 2024-05-09T02:10:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.943+0000 7f1306712b80 1 -- start start 2024-05-09T02:10:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.943+0000 7f1306712b80 1 Processor -- start 2024-05-09T02:10:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.943+0000 7f1306712b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586252ca450 0x5586254553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.943+0000 7f1306712b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625344cb0 0x558625455930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.943+0000 7f1306712b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558625450190 0x558625455e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.943+0000 7f1306712b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558625450eb0 con 0x558625344cb0 2024-05-09T02:10:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.943+0000 7f1306712b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558625304ad0 con 0x558625450190 2024-05-09T02:10:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.943+0000 7f1306712b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558625458420 con 0x5586252ca450 2024-05-09T02:10:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f1306712b80 10 client.0 _ll_register_callbacks cb 0x5586251db670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:10:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f12f5297700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625344cb0 0x558625455930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f12f5297700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625344cb0 0x558625455930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60894/0 (socket says 172.21.15.19:60894) 2024-05-09T02:10:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f12f5297700 1 -- 172.21.15.19:0/358173506 learned_addr learned my addr 172.21.15.19:0/358173506 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f12f5a98700 1 --2- 172.21.15.19:0/358173506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586252ca450 0x5586254553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f12f6299700 1 --2- 172.21.15.19:0/358173506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558625450190 0x558625455e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f12f5297700 1 -- 172.21.15.19:0/358173506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586252ca450 msgr2=0x5586254553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f12f5297700 1 --2- 172.21.15.19:0/358173506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586252ca450 0x5586254553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f12f5297700 1 -- 172.21.15.19:0/358173506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558625450190 msgr2=0x558625455e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f12f5297700 1 --2- 172.21.15.19:0/358173506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558625450190 0x558625455e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f12f5297700 1 -- 172.21.15.19:0/358173506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586251e5440 con 0x558625344cb0 2024-05-09T02:10:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f12f5a98700 1 --2- 172.21.15.19:0/358173506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586252ca450 0x5586254553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:10:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f12f5297700 1 --2- 172.21.15.19:0/358173506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625344cb0 0x558625455930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f12e0011a00 tx=0x7f12e0005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc162ae6765fcc5f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f12d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:10:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f12d97fa700 1 -- 172.21.15.19:0/358173506 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0011bb0 con 0x558625344cb0 2024-05-09T02:10:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f1306712b80 1 -- 172.21.15.19:0/358173506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e0011db0 con 0x558625344cb0 2024-05-09T02:10:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.944+0000 7f12d97fa700 1 -- 172.21.15.19:0/358173506 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f12e00046d0 con 0x558625344cb0 2024-05-09T02:10:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.945+0000 7f12d97fa700 1 -- 172.21.15.19:0/358173506 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f12e00048d0 con 0x558625344cb0 2024-05-09T02:10:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.945+0000 7f1306712b80 10 client.14970 fetch_fsmap learned FSMap version 28 2024-05-09T02:10:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.945+0000 7f1306712b80 1 -- 172.21.15.19:0/358173506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558625462d80 con 0x558625344cb0 2024-05-09T02:10:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.945+0000 7f12d97fa700 1 -- 172.21.15.19:0/358173506 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e0011db0 con 0x558625344cb0 2024-05-09T02:10:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.945+0000 7f12d97fa700 1 -- 172.21.15.19:0/358173506 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f12e003c420 con 0x558625344cb0 2024-05-09T02:10:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.945+0000 7f1306712b80 10 client.14970 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:10:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.945+0000 7f1306712b80 10 client.14970 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T02:10:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.945+0000 7f1306712b80 20 client.14970 populate_metadata read hostname 'smithi019' 2024-05-09T02:10:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.945+0000 7f1306712b80 1 --2- 172.21.15.19:0/358173506 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5586254650a0 0x558625466510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.945+0000 7f1306712b80 4 client.14970 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:10:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.945+0000 7f1306712b80 1 -- 172.21.15.19:0/358173506 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558625377a30 con 0x5586254650a0 2024-05-09T02:10:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.946+0000 7f12f5a98700 1 --2- 172.21.15.19:0/358173506 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5586254650a0 0x558625466510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.946+0000 7f12f5a98700 1 --2- 172.21.15.19:0/358173506 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5586254650a0 0x558625466510 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=30f513440fb4132c server_cookie=d8b9a0d3a2f86814 in_seq=0 out_seq=0 2024-05-09T02:10:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.946+0000 7f12d97fa700 10 client.14970 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:10:11.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.947+0000 7f12d97fa700 1 -- 172.21.15.19:0/358173506 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558625377a30 con 0x5586254650a0 2024-05-09T02:10:11.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.947+0000 7f12d97fa700 10 client.14970 handle_command_reply: tid=0 2024-05-09T02:10:11.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.947+0000 7f1306712b80 1 client.14970 shutdown 2024-05-09T02:10:11.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.947+0000 7f1306712b80 1 -- 172.21.15.19:0/358173506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625344cb0 msgr2=0x558625455930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:11.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.947+0000 7f1306712b80 1 --2- 172.21.15.19:0/358173506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625344cb0 0x558625455930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f12e0011a00 tx=0x7f12e0005250 comp rx=0 tx=0).stop 2024-05-09T02:10:11.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.947+0000 7f1306712b80 1 -- 172.21.15.19:0/358173506 shutdown_connections 2024-05-09T02:10:11.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.947+0000 7f1306712b80 1 --2- 172.21.15.19:0/358173506 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5586254650a0 0x558625466510 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.947+0000 7f1306712b80 1 --2- 172.21.15.19:0/358173506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558625450190 0x558625455e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.947+0000 7f1306712b80 1 --2- 172.21.15.19:0/358173506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586252ca450 0x5586254553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.947+0000 7f1306712b80 1 --2- 172.21.15.19:0/358173506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625344cb0 0x558625455930 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.947+0000 7f12d97fa700 0 client.14970 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:10:11.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.947+0000 7f1306712b80 1 -- 172.21.15.19:0/358173506 >> 172.21.15.19:0/358173506 conn(0x5586252cdf20 msgr2=0x55862543bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:11.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.947+0000 7f1306712b80 1 -- 172.21.15.19:0/358173506 shutdown_connections 2024-05-09T02:10:11.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.947+0000 7f1306712b80 1 -- 172.21.15.19:0/358173506 wait complete. 2024-05-09T02:10:11.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.947+0000 7f1306712b80 20 client.14970 trim_cache size 0 max 16384 2024-05-09T02:10:11.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.969+0000 7f1306712b80 1 Processor -- start 2024-05-09T02:10:11.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.969+0000 7f1306712b80 1 -- start start 2024-05-09T02:10:11.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.969+0000 7f1306712b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586252ca450 0x558625442240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.969+0000 7f1306712b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558625344cb0 0x558625461420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.969+0000 7f1306712b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625450190 0x558625455cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.969+0000 7f1306712b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558625464850 con 0x558625450190 2024-05-09T02:10:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.969+0000 7f1306712b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558625464f20 con 0x558625344cb0 2024-05-09T02:10:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.969+0000 7f1306712b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5586254655f0 con 0x5586252ca450 2024-05-09T02:10:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.969+0000 7f12f6299700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625450190 0x558625455cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.969+0000 7f12f6299700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625450190 0x558625455cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60896/0 (socket says 172.21.15.19:60896) 2024-05-09T02:10:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.969+0000 7f12f6299700 1 -- 172.21.15.19:0/234607419 learned_addr learned my addr 172.21.15.19:0/234607419 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.969+0000 7f12f5a98700 1 --2- 172.21.15.19:0/234607419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586252ca450 0x558625442240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.969+0000 7f12f5297700 1 --2- 172.21.15.19:0/234607419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558625344cb0 0x558625461420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.969+0000 7f12f6299700 1 -- 172.21.15.19:0/234607419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586252ca450 msgr2=0x558625442240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.970+0000 7f12f6299700 1 --2- 172.21.15.19:0/234607419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586252ca450 0x558625442240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.970+0000 7f12f6299700 1 -- 172.21.15.19:0/234607419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558625344cb0 msgr2=0x558625461420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:11.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.970+0000 7f12f6299700 1 --2- 172.21.15.19:0/234607419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558625344cb0 0x558625461420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.970+0000 7f12f6299700 1 -- 172.21.15.19:0/234607419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558625443e50 con 0x558625450190 2024-05-09T02:10:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.970+0000 7f12f6299700 1 --2- 172.21.15.19:0/234607419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625450190 0x558625455cb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009ec0 tx=0x7f12e80077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91da5a4ee58bfe4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.970+0000 7f12db7fe700 1 -- 172.21.15.19:0/234607419 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e801b070 con 0x558625450190 2024-05-09T02:10:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.970+0000 7f12db7fe700 1 -- 172.21.15.19:0/234607419 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e8017af0 con 0x558625450190 2024-05-09T02:10:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.970+0000 7f12db7fe700 1 -- 172.21.15.19:0/234607419 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8016a40 con 0x558625450190 2024-05-09T02:10:11.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.971+0000 7f1306712b80 1 -- 172.21.15.19:0/234607419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625450190 msgr2=0x558625455cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:11.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.971+0000 7f1306712b80 1 --2- 172.21.15.19:0/234607419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625450190 0x558625455cb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009ec0 tx=0x7f12e80077e0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.971+0000 7f1306712b80 1 -- 172.21.15.19:0/234607419 shutdown_connections 2024-05-09T02:10:11.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.971+0000 7f1306712b80 1 --2- 172.21.15.19:0/234607419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625450190 0x558625455cb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.971+0000 7f1306712b80 1 --2- 172.21.15.19:0/234607419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586252ca450 0x558625442240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.971+0000 7f1306712b80 1 --2- 172.21.15.19:0/234607419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558625344cb0 0x558625461420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.971+0000 7f1306712b80 1 -- 172.21.15.19:0/234607419 >> 172.21.15.19:0/234607419 conn(0x5586252cdf20 msgr2=0x558625453ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:11.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.971+0000 7f1306712b80 1 -- 172.21.15.19:0/234607419 shutdown_connections 2024-05-09T02:10:11.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.971+0000 7f1306712b80 1 -- 172.21.15.19:0/234607419 wait complete. 2024-05-09T02:10:11.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.971+0000 7f1306712b80 1 -- start start 2024-05-09T02:10:11.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.971+0000 7f1306712b80 1 Processor -- start 2024-05-09T02:10:11.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f1306712b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586252ca450 0x558625469860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f1306712b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558625344cb0 0x558625469da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f1306712b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625450190 0x55862546dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f1306712b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558625466390 con 0x558625450190 2024-05-09T02:10:11.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f1306712b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558625465cc0 con 0x5586252ca450 2024-05-09T02:10:11.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f1306712b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558625466a60 con 0x558625344cb0 2024-05-09T02:10:11.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f1306712b80 10 client.0 _ll_register_callbacks cb 0x5586251db670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:10:11.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f12f6299700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625450190 0x55862546dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f12f6299700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625450190 0x55862546dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60904/0 (socket says 172.21.15.19:60904) 2024-05-09T02:10:11.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f12f6299700 1 -- 172.21.15.19:0/1432875917 learned_addr learned my addr 172.21.15.19:0/1432875917 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:11.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f12f5a98700 1 --2- 172.21.15.19:0/1432875917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586252ca450 0x558625469860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f12f5297700 1 --2- 172.21.15.19:0/1432875917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558625344cb0 0x558625469da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f12f6299700 1 -- 172.21.15.19:0/1432875917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558625344cb0 msgr2=0x558625469da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:11.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f12f6299700 1 --2- 172.21.15.19:0/1432875917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558625344cb0 0x558625469da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f12f6299700 1 -- 172.21.15.19:0/1432875917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586252ca450 msgr2=0x558625469860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:11.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f12f6299700 1 --2- 172.21.15.19:0/1432875917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586252ca450 0x558625469860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f12f6299700 1 -- 172.21.15.19:0/1432875917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55862546a650 con 0x558625450190 2024-05-09T02:10:11.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f12f6299700 1 --2- 172.21.15.19:0/1432875917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625450190 0x55862546dfd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f12e8007fd0 tx=0x7f12e8017ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c24d8aabd89325 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:11.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.972+0000 7f12d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:10:11.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.973+0000 7f12d97fa700 1 -- 172.21.15.19:0/1432875917 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e801b050 con 0x558625450190 2024-05-09T02:10:11.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.973+0000 7f12d97fa700 1 -- 172.21.15.19:0/1432875917 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f12e8017850 con 0x558625450190 2024-05-09T02:10:11.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.973+0000 7f1306712b80 1 -- 172.21.15.19:0/1432875917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e8017af0 con 0x558625450190 2024-05-09T02:10:11.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.973+0000 7f12d97fa700 1 -- 172.21.15.19:0/1432875917 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f12e80166c0 con 0x558625450190 2024-05-09T02:10:11.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.973+0000 7f1306712b80 10 client.14976 fetch_fsmap learned FSMap version 28 2024-05-09T02:10:11.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.973+0000 7f1306712b80 1 -- 172.21.15.19:0/1432875917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586254d0a80 con 0x558625450190 2024-05-09T02:10:11.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.973+0000 7f12d97fa700 1 -- 172.21.15.19:0/1432875917 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e8017af0 con 0x558625450190 2024-05-09T02:10:11.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.973+0000 7f12d97fa700 1 -- 172.21.15.19:0/1432875917 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f12e8028070 con 0x558625450190 2024-05-09T02:10:11.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.973+0000 7f1306712b80 10 client.14976 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:10:11.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.973+0000 7f1306712b80 10 client.14976 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T02:10:11.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.973+0000 7f1306712b80 20 client.14976 populate_metadata read hostname 'smithi019' 2024-05-09T02:10:11.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.973+0000 7f1306712b80 1 --2- 172.21.15.19:0/1432875917 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5586254d2a90 0x5586254d3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:11.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.973+0000 7f1306712b80 4 client.14976 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T02:10:11.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.973+0000 7f1306712b80 1 -- 172.21.15.19:0/1432875917 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "status"}) -- 0x558625371ad0 con 0x5586254d2a90 2024-05-09T02:10:11.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.974+0000 7f12f5a98700 1 --2- 172.21.15.19:0/1432875917 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5586254d2a90 0x5586254d3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:11.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.974+0000 7f12f5a98700 1 --2- 172.21.15.19:0/1432875917 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5586254d2a90 0x5586254d3f00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c4a8decec2d76d67 server_cookie=213e986a2416f20a in_seq=0 out_seq=0 2024-05-09T02:10:11.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.974+0000 7f12d97fa700 10 client.14976 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:10:11.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.974+0000 7f12d97fa700 1 -- 172.21.15.19:0/1432875917 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x558625371ad0 con 0x5586254d2a90 2024-05-09T02:10:11.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.974+0000 7f12d97fa700 10 client.14976 handle_command_reply: tid=0 2024-05-09T02:10:11.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.974+0000 7f1306712b80 1 client.14976 shutdown 2024-05-09T02:10:11.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.974+0000 7f1306712b80 1 -- 172.21.15.19:0/1432875917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625450190 msgr2=0x55862546dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:11.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.974+0000 7f1306712b80 1 --2- 172.21.15.19:0/1432875917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625450190 0x55862546dfd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f12e8007fd0 tx=0x7f12e8017ca0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.974+0000 7f1306712b80 1 -- 172.21.15.19:0/1432875917 shutdown_connections 2024-05-09T02:10:11.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.974+0000 7f1306712b80 1 --2- 172.21.15.19:0/1432875917 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5586254d2a90 0x5586254d3f00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.974+0000 7f1306712b80 1 --2- 172.21.15.19:0/1432875917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558625450190 0x55862546dfd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.974+0000 7f1306712b80 1 --2- 172.21.15.19:0/1432875917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586252ca450 0x558625469860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.974+0000 7f1306712b80 1 --2- 172.21.15.19:0/1432875917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558625344cb0 0x558625469da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.974+0000 7f12d97fa700 0 client.14976 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:10:11.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.974+0000 7f1306712b80 1 -- 172.21.15.19:0/1432875917 >> 172.21.15.19:0/1432875917 conn(0x5586252cdf20 msgr2=0x558625455cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:11.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.975+0000 7f1306712b80 1 -- 172.21.15.19:0/1432875917 shutdown_connections 2024-05-09T02:10:11.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.975+0000 7f1306712b80 1 -- 172.21.15.19:0/1432875917 wait complete. 2024-05-09T02:10:11.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.975+0000 7f1306712b80 20 client.14976 trim_cache size 0 max 16384 2024-05-09T02:10:11.982 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:10:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "a0611f3e-0da7-11ef-bc97-c7b262605968", 2024-05-09T02:10:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-05-09T02:10:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24455, 2024-05-09T02:10:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T02:10:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T02:10:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T02:10:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 124.722477353, 2024-05-09T02:10:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-05-09T02:10:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 81, 2024-05-09T02:10:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 79, 2024-05-09T02:10:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 169.121298516 2024-05-09T02:10:11.983 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:10:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.975+0000 7f12f7cfc700 1 -- 172.21.15.19:0/2211873443 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f12dc0b2880 msgr2=0x7f12dc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:11.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.975+0000 7f12f7cfc700 1 --2- 172.21.15.19:0/2211873443 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f12dc0b2880 0x7f12dc0b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f12f0103510 tx=0x7f12e000b410 comp rx=0 tx=0).stop 2024-05-09T02:10:11.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.975+0000 7f12f7cfc700 1 -- 172.21.15.19:0/2211873443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f01045f0 msgr2=0x7f12f019ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:11.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.975+0000 7f12f7cfc700 1 --2- 172.21.15.19:0/2211873443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f01045f0 0x7f12f019ae70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00b580 tx=0x7f12ec0047b0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.975+0000 7f12f7cfc700 1 -- 172.21.15.19:0/2211873443 shutdown_connections 2024-05-09T02:10:11.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.975+0000 7f12f7cfc700 1 --2- 172.21.15.19:0/2211873443 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f12dc0b2880 0x7f12dc0b4d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.975+0000 7f12f7cfc700 1 --2- 172.21.15.19:0/2211873443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12f0109870 0x7f12f019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.975+0000 7f12f7cfc700 1 --2- 172.21.15.19:0/2211873443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12f01045f0 0x7f12f019ae70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.975+0000 7f12f7cfc700 1 --2- 172.21.15.19:0/2211873443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f0108f20 0x7f12f019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:11.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.975+0000 7f12f7cfc700 1 -- 172.21.15.19:0/2211873443 >> 172.21.15.19:0/2211873443 conn(0x7f12f00fffe0 msgr2=0x7f12f0108260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:11.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.975+0000 7f12f7cfc700 1 -- 172.21.15.19:0/2211873443 shutdown_connections 2024-05-09T02:10:11.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:11.975+0000 7f12f7cfc700 1 -- 172.21.15.19:0/2211873443 wait complete. 2024-05-09T02:10:12.671 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-09T02:10:12.924 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:13.303 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:13 smithi019 ceph-mon[30151]: pgmap v365: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:13.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:13 smithi059 ceph-mon[34781]: pgmap v365: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:13.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:13 smithi196 ceph-mon[34718]: pgmap v365: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.718+0000 7fc5fed68700 1 -- 172.21.15.19:0/2649271452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f8109860 msgr2=0x7fc5f81111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.718+0000 7fc5fed68700 1 --2- 172.21.15.19:0/2649271452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f8109860 0x7fc5f81111c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000b0d0 tx=0x7fc5f000b3e0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.718+0000 7fc5fed68700 1 -- 172.21.15.19:0/2649271452 shutdown_connections 2024-05-09T02:10:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.718+0000 7fc5fed68700 1 --2- 172.21.15.19:0/2649271452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f8109860 0x7fc5f81111c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.718+0000 7fc5fed68700 1 --2- 172.21.15.19:0/2649271452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f81045e0 0x7fc5f81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.718+0000 7fc5fed68700 1 --2- 172.21.15.19:0/2649271452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f8108f10 0x7fc5f8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.718+0000 7fc5fed68700 1 -- 172.21.15.19:0/2649271452 >> 172.21.15.19:0/2649271452 conn(0x7fc5f80ffe70 msgr2=0x7fc5f81022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.718+0000 7fc5fed68700 1 -- 172.21.15.19:0/2649271452 shutdown_connections 2024-05-09T02:10:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.718+0000 7fc5fed68700 1 -- 172.21.15.19:0/2649271452 wait complete. 2024-05-09T02:10:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.719+0000 7fc5fed68700 1 Processor -- start 2024-05-09T02:10:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.719+0000 7fc5fed68700 1 -- start start 2024-05-09T02:10:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.719+0000 7fc5fed68700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f81045e0 0x7fc5f819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.719+0000 7fc5fed68700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f8108f10 0x7fc5f819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.719+0000 7fc5fed68700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f8109860 0x7fc5f819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.719+0000 7fc5fed68700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f8114490 con 0x7fc5f81045e0 2024-05-09T02:10:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.719+0000 7fc5fed68700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f8114310 con 0x7fc5f8109860 2024-05-09T02:10:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.719+0000 7fc5fed68700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f8114610 con 0x7fc5f8108f10 2024-05-09T02:10:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.719+0000 7fc5fcb04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f81045e0 0x7fc5f819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.719+0000 7fc5fcb04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f81045e0 0x7fc5f819ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60910/0 (socket says 172.21.15.19:60910) 2024-05-09T02:10:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.719+0000 7fc5fcb04700 1 -- 172.21.15.19:0/1153051785 learned_addr learned my addr 172.21.15.19:0/1153051785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.719+0000 7fc5fd305700 1 --2- 172.21.15.19:0/1153051785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f8109860 0x7fc5f819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.720+0000 7fc5effff700 1 --2- 172.21.15.19:0/1153051785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f8108f10 0x7fc5f819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.720+0000 7fc5fcb04700 1 -- 172.21.15.19:0/1153051785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f8108f10 msgr2=0x7fc5f819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.720+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/1153051785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f8108f10 0x7fc5f819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.720+0000 7fc5fcb04700 1 -- 172.21.15.19:0/1153051785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f8109860 msgr2=0x7fc5f819f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.720+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/1153051785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f8109860 0x7fc5f819f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.720+0000 7fc5fcb04700 1 -- 172.21.15.19:0/1153051785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f819fc30 con 0x7fc5f81045e0 2024-05-09T02:10:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.720+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/1153051785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f81045e0 0x7fc5f819ad60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400d950 tx=0x7fc5f400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.720+0000 7fc5edffb700 1 -- 172.21.15.19:0/1153051785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4016070 con 0x7fc5f81045e0 2024-05-09T02:10:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.720+0000 7fc5fed68700 1 -- 172.21.15.19:0/1153051785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f819ff20 con 0x7fc5f81045e0 2024-05-09T02:10:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.720+0000 7fc5edffb700 1 -- 172.21.15.19:0/1153051785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5f4009df0 con 0x7fc5f81045e0 2024-05-09T02:10:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.720+0000 7fc5edffb700 1 -- 172.21.15.19:0/1153051785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f400f3c0 con 0x7fc5f81045e0 2024-05-09T02:10:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.721+0000 7fc5fed68700 1 -- 172.21.15.19:0/1153051785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f81a7840 con 0x7fc5f81045e0 2024-05-09T02:10:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.723+0000 7fc5edffb700 1 -- 172.21.15.19:0/1153051785 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc5f4019070 con 0x7fc5f81045e0 2024-05-09T02:10:14.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.724+0000 7fc5edffb700 1 --2- 172.21.15.19:0/1153051785 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5e00b2880 0x7fc5e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.724+0000 7fc5effff700 1 --2- 172.21.15.19:0/1153051785 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5e00b2880 0x7fc5e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.724+0000 7fc5edffb700 1 -- 172.21.15.19:0/1153051785 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc5f40e1eb0 con 0x7fc5f81045e0 2024-05-09T02:10:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.725+0000 7fc5effff700 1 --2- 172.21.15.19:0/1153051785 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5e00b2880 0x7fc5e00b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8103830 tx=0x7fc5e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.725+0000 7fc60d77eb80 1 Processor -- start 2024-05-09T02:10:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.725+0000 7fc60d77eb80 1 -- start start 2024-05-09T02:10:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.725+0000 7fc60d77eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd2a8cb0 0x5583fd3b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.725+0000 7fc60d77eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd22e450 0x5583fd3b1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.725+0000 7fc60d77eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd3ab3b0 0x5583fd3ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.725+0000 7fc60d77eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5583fd2ce050 con 0x5583fd3ab3b0 2024-05-09T02:10:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.725+0000 7fc60d77eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5583fd273950 con 0x5583fd2a8cb0 2024-05-09T02:10:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.725+0000 7fc60d77eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5583fd2ca010 con 0x5583fd22e450 2024-05-09T02:10:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.725+0000 7fc5fd305700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd3ab3b0 0x5583fd3ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.725+0000 7fc5fcb04700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd2a8cb0 0x5583fd3b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.725+0000 7fc5fcb04700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd2a8cb0 0x5583fd3b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:36300/0 (socket says 172.21.15.19:36300) 2024-05-09T02:10:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.725+0000 7fc5fcb04700 1 -- 172.21.15.19:0/3175050497 learned_addr learned my addr 172.21.15.19:0/3175050497 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.725+0000 7fc5effff700 1 --2- 172.21.15.19:0/3175050497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd22e450 0x5583fd3b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.726+0000 7fc5fcb04700 1 -- 172.21.15.19:0/3175050497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd22e450 msgr2=0x5583fd3b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.726+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/3175050497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd22e450 0x5583fd3b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.726+0000 7fc5fcb04700 1 -- 172.21.15.19:0/3175050497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd3ab3b0 msgr2=0x5583fd3ab7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.726+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/3175050497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd3ab3b0 0x5583fd3ab7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.726+0000 7fc5fcb04700 1 -- 172.21.15.19:0/3175050497 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583fd2644b0 con 0x5583fd2a8cb0 2024-05-09T02:10:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.726+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/3175050497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd2a8cb0 0x5583fd3b11f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40a9150 tx=0x7fc5f40a9bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68d02bffd2b22a1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.726+0000 7fc5deffd700 1 -- 172.21.15.19:0/3175050497 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4020070 con 0x5583fd2a8cb0 2024-05-09T02:10:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.726+0000 7fc5deffd700 1 -- 172.21.15.19:0/3175050497 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5f40a9450 con 0x5583fd2a8cb0 2024-05-09T02:10:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.726+0000 7fc5deffd700 1 -- 172.21.15.19:0/3175050497 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40b1e20 con 0x5583fd2a8cb0 2024-05-09T02:10:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.727+0000 7fc60d77eb80 1 -- 172.21.15.19:0/3175050497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd2a8cb0 msgr2=0x5583fd3b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.727+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/3175050497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd2a8cb0 0x5583fd3b11f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40a9150 tx=0x7fc5f40a9bd0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.727+0000 7fc60d77eb80 1 -- 172.21.15.19:0/3175050497 shutdown_connections 2024-05-09T02:10:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.727+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/3175050497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd3ab3b0 0x5583fd3ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.727+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/3175050497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd2a8cb0 0x5583fd3b11f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.727+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/3175050497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd22e450 0x5583fd3b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.727+0000 7fc60d77eb80 1 -- 172.21.15.19:0/3175050497 >> 172.21.15.19:0/3175050497 conn(0x5583fd231f20 msgr2=0x5583fd3a61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.728+0000 7fc60d77eb80 1 -- 172.21.15.19:0/3175050497 shutdown_connections 2024-05-09T02:10:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.728+0000 7fc60d77eb80 1 -- 172.21.15.19:0/3175050497 wait complete. 2024-05-09T02:10:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.728+0000 7fc60d77eb80 1 -- start start 2024-05-09T02:10:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.728+0000 7fc60d77eb80 1 Processor -- start 2024-05-09T02:10:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc60d77eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd22e450 0x5583fd3b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc60d77eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd2a8cb0 0x5583fd3b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc60d77eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd3ab3b0 0x5583fd3b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc60d77eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5583fd3ac0b0 con 0x5583fd22e450 2024-05-09T02:10:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc60d77eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5583fd268ad0 con 0x5583fd3ab3b0 2024-05-09T02:10:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc60d77eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5583fd3c1180 con 0x5583fd2a8cb0 2024-05-09T02:10:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc5fcb04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd22e450 0x5583fd3b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc5fcb04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd22e450 0x5583fd3b5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60920/0 (socket says 172.21.15.19:60920) 2024-05-09T02:10:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc5fcb04700 1 -- 172.21.15.19:0/558291500 learned_addr learned my addr 172.21.15.19:0/558291500 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc60d77eb80 10 client.0 _ll_register_callbacks cb 0x5583fd13f670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:10:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc5effff700 1 --2- 172.21.15.19:0/558291500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd2a8cb0 0x5583fd3b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc5fd305700 1 --2- 172.21.15.19:0/558291500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd3ab3b0 0x5583fd3b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc5fcb04700 1 -- 172.21.15.19:0/558291500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd2a8cb0 msgr2=0x5583fd3b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/558291500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd2a8cb0 0x5583fd3b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc5fcb04700 1 -- 172.21.15.19:0/558291500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd3ab3b0 msgr2=0x5583fd3b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.729+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/558291500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd3ab3b0 0x5583fd3b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.730+0000 7fc5fcb04700 1 -- 172.21.15.19:0/558291500 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583fd149440 con 0x5583fd22e450 2024-05-09T02:10:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.730+0000 7fc5fd305700 1 --2- 172.21.15.19:0/558291500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd3ab3b0 0x5583fd3b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:10:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.730+0000 7fc5effff700 1 --2- 172.21.15.19:0/558291500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd2a8cb0 0x5583fd3b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:10:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.730+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/558291500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd22e450 0x5583fd3b5030 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40a5450 tx=0x7fc5f40b22b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610926283b2d2b57 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.730+0000 7fc5d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:10:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.730+0000 7fc5d7fff700 1 -- 172.21.15.19:0/558291500 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4020070 con 0x5583fd22e450 2024-05-09T02:10:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.730+0000 7fc5d7fff700 1 -- 172.21.15.19:0/558291500 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc5f40b1e20 con 0x5583fd22e450 2024-05-09T02:10:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.730+0000 7fc60d77eb80 1 -- 172.21.15.19:0/558291500 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5f40a9450 con 0x5583fd22e450 2024-05-09T02:10:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.730+0000 7fc5d7fff700 1 -- 172.21.15.19:0/558291500 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc5f406e070 con 0x5583fd22e450 2024-05-09T02:10:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.730+0000 7fc60d77eb80 10 client.14988 fetch_fsmap learned FSMap version 28 2024-05-09T02:10:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.730+0000 7fc60d77eb80 1 -- 172.21.15.19:0/558291500 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5583fd3c2ab0 con 0x5583fd22e450 2024-05-09T02:10:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.730+0000 7fc5d7fff700 1 -- 172.21.15.19:0/558291500 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5f40a9450 con 0x5583fd22e450 2024-05-09T02:10:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.731+0000 7fc5d7fff700 1 -- 172.21.15.19:0/558291500 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc5f40ca300 con 0x5583fd22e450 2024-05-09T02:10:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.731+0000 7fc60d77eb80 10 client.14988 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:10:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.731+0000 7fc60d77eb80 10 client.14988 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T02:10:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.731+0000 7fc60d77eb80 20 client.14988 populate_metadata read hostname 'smithi019' 2024-05-09T02:10:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.731+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/558291500 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5583fd3c4cb0 0x5583fd3c6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.731+0000 7fc60d77eb80 4 client.14988 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:10:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.731+0000 7fc60d77eb80 1 -- 172.21.15.19:0/558291500 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583fd2dba30 con 0x5583fd3c4cb0 2024-05-09T02:10:14.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.731+0000 7fc5effff700 1 --2- 172.21.15.19:0/558291500 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5583fd3c4cb0 0x5583fd3c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.732+0000 7fc5effff700 1 --2- 172.21.15.19:0/558291500 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5583fd3c4cb0 0x5583fd3c6120 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cafaebe91343cd0c server_cookie=ee912b533f25c5d6 in_seq=0 out_seq=0 2024-05-09T02:10:14.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.732+0000 7fc5d7fff700 10 client.14988 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:10:14.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.732+0000 7fc5d7fff700 1 -- 172.21.15.19:0/558291500 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583fd2dba30 con 0x5583fd3c4cb0 2024-05-09T02:10:14.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.732+0000 7fc5d7fff700 10 client.14988 handle_command_reply: tid=0 2024-05-09T02:10:14.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.732+0000 7fc60d77eb80 1 client.14988 shutdown 2024-05-09T02:10:14.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.733+0000 7fc60d77eb80 1 -- 172.21.15.19:0/558291500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd22e450 msgr2=0x5583fd3b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:14.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.733+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/558291500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd22e450 0x5583fd3b5030 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40a5450 tx=0x7fc5f40b22b0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.733+0000 7fc60d77eb80 1 -- 172.21.15.19:0/558291500 shutdown_connections 2024-05-09T02:10:14.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.733+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/558291500 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5583fd3c4cb0 0x5583fd3c6120 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.733+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/558291500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd3ab3b0 0x5583fd3b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.733+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/558291500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd22e450 0x5583fd3b5030 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.733+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/558291500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd2a8cb0 0x5583fd3b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.733+0000 7fc60d77eb80 1 -- 172.21.15.19:0/558291500 >> 172.21.15.19:0/558291500 conn(0x5583fd231f20 msgr2=0x5583fd39ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:14.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.733+0000 7fc5d7fff700 0 client.14988 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:10:14.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.733+0000 7fc60d77eb80 1 -- 172.21.15.19:0/558291500 shutdown_connections 2024-05-09T02:10:14.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.733+0000 7fc60d77eb80 1 -- 172.21.15.19:0/558291500 wait complete. 2024-05-09T02:10:14.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.733+0000 7fc60d77eb80 20 client.14988 trim_cache size 0 max 16384 2024-05-09T02:10:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc60d77eb80 1 Processor -- start 2024-05-09T02:10:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc60d77eb80 1 -- start start 2024-05-09T02:10:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc60d77eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd22e450 0x5583fd3b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc60d77eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd2a8cb0 0x5583fd3c10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc60d77eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd3ab3b0 0x5583fd3cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc60d77eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5583fd3c4460 con 0x5583fd22e450 2024-05-09T02:10:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc60d77eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5583fd3c4b30 con 0x5583fd2a8cb0 2024-05-09T02:10:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc60d77eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5583fd3c5200 con 0x5583fd3ab3b0 2024-05-09T02:10:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc5fcb04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd22e450 0x5583fd3b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc5fcb04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd22e450 0x5583fd3b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60924/0 (socket says 172.21.15.19:60924) 2024-05-09T02:10:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc5fcb04700 1 -- 172.21.15.19:0/4157099906 learned_addr learned my addr 172.21.15.19:0/4157099906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc5fd305700 1 --2- 172.21.15.19:0/4157099906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd3ab3b0 0x5583fd3cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc5effff700 1 --2- 172.21.15.19:0/4157099906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd2a8cb0 0x5583fd3c10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc5fcb04700 1 -- 172.21.15.19:0/4157099906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd3ab3b0 msgr2=0x5583fd3cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/4157099906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd3ab3b0 0x5583fd3cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc5fcb04700 1 -- 172.21.15.19:0/4157099906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd2a8cb0 msgr2=0x5583fd3c10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/4157099906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd2a8cb0 0x5583fd3c10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc5fcb04700 1 -- 172.21.15.19:0/4157099906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583fd3a2980 con 0x5583fd22e450 2024-05-09T02:10:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.755+0000 7fc5effff700 1 --2- 172.21.15.19:0/4157099906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd2a8cb0 0x5583fd3c10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:10:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.756+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/4157099906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd22e450 0x5583fd3b4350 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400b2f0 tx=0x7fc5f40a9bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e434d103010348 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.756+0000 7fc5de7fc700 1 -- 172.21.15.19:0/4157099906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4020070 con 0x5583fd22e450 2024-05-09T02:10:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.756+0000 7fc5de7fc700 1 -- 172.21.15.19:0/4157099906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5f40a9450 con 0x5583fd22e450 2024-05-09T02:10:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.756+0000 7fc5de7fc700 1 -- 172.21.15.19:0/4157099906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40ca330 con 0x5583fd22e450 2024-05-09T02:10:14.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.757+0000 7fc60d77eb80 1 -- 172.21.15.19:0/4157099906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd22e450 msgr2=0x5583fd3b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.757+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/4157099906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd22e450 0x5583fd3b4350 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400b2f0 tx=0x7fc5f40a9bd0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.757+0000 7fc60d77eb80 1 -- 172.21.15.19:0/4157099906 shutdown_connections 2024-05-09T02:10:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.757+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/4157099906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd3ab3b0 0x5583fd3cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.757+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/4157099906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd22e450 0x5583fd3b4350 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.757+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/4157099906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd2a8cb0 0x5583fd3c10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.757+0000 7fc60d77eb80 1 -- 172.21.15.19:0/4157099906 >> 172.21.15.19:0/4157099906 conn(0x5583fd231f20 msgr2=0x5583fd3b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.757+0000 7fc60d77eb80 1 -- 172.21.15.19:0/4157099906 shutdown_connections 2024-05-09T02:10:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.757+0000 7fc60d77eb80 1 -- 172.21.15.19:0/4157099906 wait complete. 2024-05-09T02:10:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.757+0000 7fc60d77eb80 1 -- start start 2024-05-09T02:10:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.757+0000 7fc60d77eb80 1 Processor -- start 2024-05-09T02:10:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc60d77eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd22e450 0x5583fd3cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc60d77eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd2a8cb0 0x5583fd3cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc60d77eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd3ab3b0 0x5583fd3c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc60d77eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5583fd3c5fa0 con 0x5583fd2a8cb0 2024-05-09T02:10:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc60d77eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5583fd3c58d0 con 0x5583fd22e450 2024-05-09T02:10:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc60d77eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5583fd3c6670 con 0x5583fd3ab3b0 2024-05-09T02:10:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc60d77eb80 10 client.0 _ll_register_callbacks cb 0x5583fd13f670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:10:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc5effff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd2a8cb0 0x5583fd3cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc5effff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd2a8cb0 0x5583fd3cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60936/0 (socket says 172.21.15.19:60936) 2024-05-09T02:10:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc5effff700 1 -- 172.21.15.19:0/3442966852 learned_addr learned my addr 172.21.15.19:0/3442966852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc5fd305700 1 --2- 172.21.15.19:0/3442966852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd3ab3b0 0x5583fd3c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/3442966852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd22e450 0x5583fd3cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc5effff700 1 -- 172.21.15.19:0/3442966852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd3ab3b0 msgr2=0x5583fd3c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc5effff700 1 --2- 172.21.15.19:0/3442966852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd3ab3b0 0x5583fd3c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc5effff700 1 -- 172.21.15.19:0/3442966852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd22e450 msgr2=0x5583fd3cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc5effff700 1 --2- 172.21.15.19:0/3442966852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd22e450 0x5583fd3cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc5effff700 1 -- 172.21.15.19:0/3442966852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583fd3c9140 con 0x5583fd2a8cb0 2024-05-09T02:10:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.758+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/3442966852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd22e450 0x5583fd3cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:10:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.759+0000 7fc5effff700 1 --2- 172.21.15.19:0/3442966852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd2a8cb0 0x5583fd3cfb80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009a90 tx=0x7fc5e80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f33f653fc4589a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.759+0000 7fc5d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:10:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.759+0000 7fc5d7fff700 1 -- 172.21.15.19:0/3442966852 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e802a830 con 0x5583fd2a8cb0 2024-05-09T02:10:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.759+0000 7fc5d7fff700 1 -- 172.21.15.19:0/3442966852 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc5e802a9d0 con 0x5583fd2a8cb0 2024-05-09T02:10:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.759+0000 7fc60d77eb80 1 -- 172.21.15.19:0/3442966852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5f40a9450 con 0x5583fd2a8cb0 2024-05-09T02:10:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.759+0000 7fc5d7fff700 1 -- 172.21.15.19:0/3442966852 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc5e801f430 con 0x5583fd2a8cb0 2024-05-09T02:10:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.759+0000 7fc60d77eb80 10 client.14994 fetch_fsmap learned FSMap version 28 2024-05-09T02:10:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.759+0000 7fc60d77eb80 1 -- 172.21.15.19:0/3442966852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5583fd3cdb50 con 0x5583fd2a8cb0 2024-05-09T02:10:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.759+0000 7fc5d7fff700 1 -- 172.21.15.19:0/3442966852 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5f40a9450 con 0x5583fd2a8cb0 2024-05-09T02:10:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.760+0000 7fc5d7fff700 1 -- 172.21.15.19:0/3442966852 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc5e8033520 con 0x5583fd2a8cb0 2024-05-09T02:10:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.760+0000 7fc60d77eb80 10 client.14994 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:10:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.760+0000 7fc60d77eb80 10 client.14994 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T02:10:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.760+0000 7fc60d77eb80 20 client.14994 populate_metadata read hostname 'smithi019' 2024-05-09T02:10:14.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.760+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/3442966852 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5583fd436460 0x5583fd4378d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:14.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.760+0000 7fc60d77eb80 4 client.14994 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T02:10:14.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.760+0000 7fc60d77eb80 1 -- 172.21.15.19:0/3442966852 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "status"}) -- 0x5583fd2d5ad0 con 0x5583fd436460 2024-05-09T02:10:14.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.760+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/3442966852 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5583fd436460 0x5583fd4378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:14.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.760+0000 7fc5d7fff700 10 client.14994 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:10:14.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.760+0000 7fc5fcb04700 1 --2- 172.21.15.19:0/3442966852 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5583fd436460 0x5583fd4378d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=16a8cb766ce8cca6 server_cookie=a2b3ea3698de4b56 in_seq=0 out_seq=0 2024-05-09T02:10:14.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.760+0000 7fc5d7fff700 1 -- 172.21.15.19:0/3442966852 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5583fd2d5ad0 con 0x5583fd436460 2024-05-09T02:10:14.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.760+0000 7fc5d7fff700 10 client.14994 handle_command_reply: tid=0 2024-05-09T02:10:14.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.760+0000 7fc60d77eb80 1 client.14994 shutdown 2024-05-09T02:10:14.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc60d77eb80 1 -- 172.21.15.19:0/3442966852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd2a8cb0 msgr2=0x5583fd3cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:14.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/3442966852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd2a8cb0 0x5583fd3cfb80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009a90 tx=0x7fc5e80047c0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc60d77eb80 1 -- 172.21.15.19:0/3442966852 shutdown_connections 2024-05-09T02:10:14.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/3442966852 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5583fd436460 0x5583fd4378d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/3442966852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fd3ab3b0 0x5583fd3c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/3442966852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583fd22e450 0x5583fd3cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc60d77eb80 1 --2- 172.21.15.19:0/3442966852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583fd2a8cb0 0x5583fd3cfb80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc60d77eb80 1 -- 172.21.15.19:0/3442966852 >> 172.21.15.19:0/3442966852 conn(0x5583fd231f20 msgr2=0x5583fd3b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc5d7fff700 0 client.14994 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:10:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc60d77eb80 1 -- 172.21.15.19:0/3442966852 shutdown_connections 2024-05-09T02:10:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc60d77eb80 1 -- 172.21.15.19:0/3442966852 wait complete. 2024-05-09T02:10:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc60d77eb80 20 client.14994 trim_cache size 0 max 16384 2024-05-09T02:10:14.768 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:10:14.768 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "a0611f3e-0da7-11ef-bc97-c7b262605968", 2024-05-09T02:10:14.768 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-05-09T02:10:14.768 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24457, 2024-05-09T02:10:14.768 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T02:10:14.769 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T02:10:14.769 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T02:10:14.769 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 128.517508647, 2024-05-09T02:10:14.769 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-05-09T02:10:14.769 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 81, 2024-05-09T02:10:14.769 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 79, 2024-05-09T02:10:14.769 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 177.95112195999999 2024-05-09T02:10:14.769 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:10:14.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc5fed68700 1 -- 172.21.15.19:0/1153051785 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5e00b2880 msgr2=0x7fc5e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:14.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc5fed68700 1 --2- 172.21.15.19:0/1153051785 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5e00b2880 0x7fc5e00b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8103830 tx=0x7fc5e8009260 comp rx=0 tx=0).stop 2024-05-09T02:10:14.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc5fed68700 1 -- 172.21.15.19:0/1153051785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f81045e0 msgr2=0x7fc5f819ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:14.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.761+0000 7fc5fed68700 1 --2- 172.21.15.19:0/1153051785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f81045e0 0x7fc5f819ad60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400d950 tx=0x7fc5f400dc60 comp rx=0 tx=0).stop 2024-05-09T02:10:14.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.762+0000 7fc5fed68700 1 -- 172.21.15.19:0/1153051785 shutdown_connections 2024-05-09T02:10:14.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.762+0000 7fc5fed68700 1 --2- 172.21.15.19:0/1153051785 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5e00b2880 0x7fc5e00b4d90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.762+0000 7fc5fed68700 1 --2- 172.21.15.19:0/1153051785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f8109860 0x7fc5f819f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.762+0000 7fc5fed68700 1 --2- 172.21.15.19:0/1153051785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5f81045e0 0x7fc5f819ad60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.762+0000 7fc5fed68700 1 --2- 172.21.15.19:0/1153051785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f8108f10 0x7fc5f819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:14.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.762+0000 7fc5fed68700 1 -- 172.21.15.19:0/1153051785 >> 172.21.15.19:0/1153051785 conn(0x7fc5f80ffe70 msgr2=0x7fc5f8108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:14.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.762+0000 7fc5fed68700 1 -- 172.21.15.19:0/1153051785 shutdown_connections 2024-05-09T02:10:14.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:14.762+0000 7fc5fed68700 1 -- 172.21.15.19:0/1153051785 wait complete. 2024-05-09T02:10:15.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:15 smithi019 ceph-mon[30151]: pgmap v366: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:10:15.463 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-09T02:10:15.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:15 smithi059 ceph-mon[34781]: pgmap v366: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:10:15.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:15 smithi196 ceph-mon[34718]: pgmap v366: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:10:15.717 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:17.391 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:17 smithi019 ceph-mon[30151]: pgmap v367: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:10:17.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.387+0000 7f6f69e79700 1 -- 172.21.15.19:0/224327983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f64107df0 msgr2=0x7f6f64108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:17.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.387+0000 7f6f69e79700 1 --2- 172.21.15.19:0/224327983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f64107df0 0x7f6f64108200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6f580099c0 tx=0x7f6f58009cd0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.387+0000 7f6f69e79700 1 -- 172.21.15.19:0/224327983 shutdown_connections 2024-05-09T02:10:17.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.387+0000 7f6f69e79700 1 --2- 172.21.15.19:0/224327983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f64108740 0x7f6f641100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.387+0000 7f6f69e79700 1 --2- 172.21.15.19:0/224327983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f641034c0 0x7f6f641038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.387+0000 7f6f69e79700 1 --2- 172.21.15.19:0/224327983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f64107df0 0x7f6f64108200 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.387+0000 7f6f69e79700 1 -- 172.21.15.19:0/224327983 >> 172.21.15.19:0/224327983 conn(0x7f6f640fedf0 msgr2=0x7f6f64101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:17.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.387+0000 7f6f69e79700 1 -- 172.21.15.19:0/224327983 shutdown_connections 2024-05-09T02:10:17.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.387+0000 7f6f69e79700 1 -- 172.21.15.19:0/224327983 wait complete. 2024-05-09T02:10:17.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.388+0000 7f6f69e79700 1 Processor -- start 2024-05-09T02:10:17.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.388+0000 7f6f69e79700 1 -- start start 2024-05-09T02:10:17.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.388+0000 7f6f69e79700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f641034c0 0x7f6f6419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f69e79700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f64107df0 0x7f6f6419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f69e79700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f64108740 0x7f6f6419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f69e79700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6f64113350 con 0x7f6f641034c0 2024-05-09T02:10:17.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f69e79700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f641131d0 con 0x7f6f64108740 2024-05-09T02:10:17.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f69e79700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f641134d0 con 0x7f6f64107df0 2024-05-09T02:10:17.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f637fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f641034c0 0x7f6f6419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f637fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f641034c0 0x7f6f6419aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35818/0 (socket says 172.21.15.19:35818) 2024-05-09T02:10:17.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f637fe700 1 -- 172.21.15.19:0/630506411 learned_addr learned my addr 172.21.15.19:0/630506411 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:17.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f64107df0 0x7f6f6419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f63fff700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f64108740 0x7f6f6419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f637fe700 1 -- 172.21.15.19:0/630506411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f64107df0 msgr2=0x7f6f6419b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:17.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f637fe700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f64107df0 0x7f6f6419b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f637fe700 1 -- 172.21.15.19:0/630506411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f64108740 msgr2=0x7f6f6419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:17.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f637fe700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f64108740 0x7f6f6419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f637fe700 1 -- 172.21.15.19:0/630506411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6419fda0 con 0x7f6f641034c0 2024-05-09T02:10:17.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f63fff700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f64108740 0x7f6f6419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:10:17.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f64107df0 0x7f6f6419b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:10:17.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.389+0000 7f6f637fe700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f641034c0 0x7f6f6419aed0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400d950 tx=0x7f6f5400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:17.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.390+0000 7f6f60ff9700 1 -- 172.21.15.19:0/630506411 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54016070 con 0x7f6f641034c0 2024-05-09T02:10:17.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.390+0000 7f6f69e79700 1 -- 172.21.15.19:0/630506411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f641a0090 con 0x7f6f641034c0 2024-05-09T02:10:17.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.390+0000 7f6f60ff9700 1 -- 172.21.15.19:0/630506411 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f54010460 con 0x7f6f641034c0 2024-05-09T02:10:17.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.390+0000 7f6f60ff9700 1 -- 172.21.15.19:0/630506411 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5400f620 con 0x7f6f641034c0 2024-05-09T02:10:17.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.390+0000 7f6f69e79700 1 -- 172.21.15.19:0/630506411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f641a79b0 con 0x7f6f641034c0 2024-05-09T02:10:17.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.392+0000 7f6f60ff9700 1 -- 172.21.15.19:0/630506411 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6f54019070 con 0x7f6f641034c0 2024-05-09T02:10:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f60ff9700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6f4c0b2900 0x7f6f4c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f7888fb80 1 Processor -- start 2024-05-09T02:10:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6f4c0b2900 0x7f6f4c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f7888fb80 1 -- start start 2024-05-09T02:10:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f60ff9700 1 -- 172.21.15.19:0/630506411 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f6f540e2270 con 0x7f6f641034c0 2024-05-09T02:10:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f7888fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f7888fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba22f450 0x55b5ba3b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f7888fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba3ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f7888fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5ba2cf050 con 0x55b5ba22f450 2024-05-09T02:10:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f7888fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b5ba274950 con 0x55b5ba2a9cb0 2024-05-09T02:10:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f7888fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b5ba2cb010 con 0x55b5ba3ac3b0 2024-05-09T02:10:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f62ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba22f450 0x55b5ba3b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f62ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba22f450 0x55b5ba3b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35822/0 (socket says 172.21.15.19:35822) 2024-05-09T02:10:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f62ffd700 1 -- 172.21.15.19:0/181653925 learned_addr learned my addr 172.21.15.19:0/181653925 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f637fe700 1 --2- 172.21.15.19:0/181653925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.394+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6f4c0b2900 0x7f6f4c0b4e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f58000c00 tx=0x7f6f5801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.395+0000 7f6f63fff700 1 --2- 172.21.15.19:0/181653925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba3ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.395+0000 7f6f62ffd700 1 -- 172.21.15.19:0/181653925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba3ac3b0 msgr2=0x55b5ba3ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.395+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/181653925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba3ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.395+0000 7f6f62ffd700 1 -- 172.21.15.19:0/181653925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba2a9cb0 msgr2=0x55b5ba3b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.395+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/181653925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.395+0000 7f6f62ffd700 1 -- 172.21.15.19:0/181653925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5ba2654b0 con 0x55b5ba22f450 2024-05-09T02:10:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.395+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/181653925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba22f450 0x55b5ba3b2730 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6f580099c0 tx=0x7f6f5801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b4fb5e2af47e70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.395+0000 7f6f49ffb700 1 -- 172.21.15.19:0/181653925 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58003860 con 0x55b5ba22f450 2024-05-09T02:10:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.395+0000 7f6f49ffb700 1 -- 172.21.15.19:0/181653925 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f58005030 con 0x55b5ba22f450 2024-05-09T02:10:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.395+0000 7f6f49ffb700 1 -- 172.21.15.19:0/181653925 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800fa60 con 0x55b5ba22f450 2024-05-09T02:10:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.396+0000 7f6f7888fb80 1 -- 172.21.15.19:0/181653925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba22f450 msgr2=0x55b5ba3b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.396+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/181653925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba22f450 0x55b5ba3b2730 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6f580099c0 tx=0x7f6f5801e000 comp rx=0 tx=0).stop 2024-05-09T02:10:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.396+0000 7f6f7888fb80 1 -- 172.21.15.19:0/181653925 shutdown_connections 2024-05-09T02:10:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.396+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/181653925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba3ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.396+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/181653925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.396+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/181653925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba22f450 0x55b5ba3b2730 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.396+0000 7f6f7888fb80 1 -- 172.21.15.19:0/181653925 >> 172.21.15.19:0/181653925 conn(0x55b5ba232f20 msgr2=0x55b5ba3a71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.396+0000 7f6f7888fb80 1 -- 172.21.15.19:0/181653925 shutdown_connections 2024-05-09T02:10:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.396+0000 7f6f7888fb80 1 -- 172.21.15.19:0/181653925 wait complete. 2024-05-09T02:10:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.397+0000 7f6f7888fb80 1 -- start start 2024-05-09T02:10:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.397+0000 7f6f7888fb80 1 Processor -- start 2024-05-09T02:10:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.397+0000 7f6f7888fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba22f450 0x55b5ba3b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.397+0000 7f6f7888fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.397+0000 7f6f7888fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba3b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.397+0000 7f6f7888fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5ba3ad0b0 con 0x55b5ba22f450 2024-05-09T02:10:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.397+0000 7f6f7888fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b5ba269ad0 con 0x55b5ba2a9cb0 2024-05-09T02:10:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.397+0000 7f6f7888fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b5ba3c22c0 con 0x55b5ba3ac3b0 2024-05-09T02:10:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.397+0000 7f6f637fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba22f450 0x55b5ba3b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.397+0000 7f6f7888fb80 10 client.0 _ll_register_callbacks cb 0x55b5ba140670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:10:17.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.397+0000 7f6f637fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba22f450 0x55b5ba3b60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35832/0 (socket says 172.21.15.19:35832) 2024-05-09T02:10:17.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.397+0000 7f6f637fe700 1 -- 172.21.15.19:0/3757217618 learned_addr learned my addr 172.21.15.19:0/3757217618 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:17.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.397+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/3757217618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f63fff700 1 --2- 172.21.15.19:0/3757217618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba3b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f637fe700 1 -- 172.21.15.19:0/3757217618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba3ac3b0 msgr2=0x55b5ba3b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:17.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f637fe700 1 --2- 172.21.15.19:0/3757217618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba3b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f637fe700 1 -- 172.21.15.19:0/3757217618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba2a9cb0 msgr2=0x55b5ba3b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:17.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f637fe700 1 --2- 172.21.15.19:0/3757217618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f637fe700 1 -- 172.21.15.19:0/3757217618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5ba14a440 con 0x55b5ba22f450 2024-05-09T02:10:17.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f63fff700 1 --2- 172.21.15.19:0/3757217618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba3b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:10:17.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f637fe700 1 --2- 172.21.15.19:0/3757217618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba22f450 0x55b5ba3b60f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400b2f0 tx=0x7f6f54010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac269f9286d4d59f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:17.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:10:17.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f42ffd700 1 -- 172.21.15.19:0/3757217618 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54020070 con 0x55b5ba22f450 2024-05-09T02:10:17.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f42ffd700 1 -- 172.21.15.19:0/3757217618 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f6f540a5aa0 con 0x55b5ba22f450 2024-05-09T02:10:17.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f7888fb80 1 -- 172.21.15.19:0/3757217618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f58005030 con 0x55b5ba22f450 2024-05-09T02:10:17.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f42ffd700 1 -- 172.21.15.19:0/3757217618 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f540adbc0 con 0x55b5ba22f450 2024-05-09T02:10:17.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f7888fb80 10 client.15006 fetch_fsmap learned FSMap version 28 2024-05-09T02:10:17.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f7888fb80 1 -- 172.21.15.19:0/3757217618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b5ba3c3cb0 con 0x55b5ba22f450 2024-05-09T02:10:17.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.398+0000 7f6f42ffd700 1 -- 172.21.15.19:0/3757217618 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f58005030 con 0x55b5ba22f450 2024-05-09T02:10:17.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.399+0000 7f6f42ffd700 1 -- 172.21.15.19:0/3757217618 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6f540b2270 con 0x55b5ba22f450 2024-05-09T02:10:17.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.399+0000 7f6f7888fb80 10 client.15006 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:10:17.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.399+0000 7f6f7888fb80 10 client.15006 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T02:10:17.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.399+0000 7f6f7888fb80 20 client.15006 populate_metadata read hostname 'smithi019' 2024-05-09T02:10:17.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.399+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/3757217618 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b5ba3c6300 0x55b5ba3c7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.399+0000 7f6f7888fb80 4 client.15006 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:10:17.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.399+0000 7f6f7888fb80 1 -- 172.21.15.19:0/3757217618 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5ba2dca30 con 0x55b5ba3c6300 2024-05-09T02:10:17.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.399+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/3757217618 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b5ba3c6300 0x55b5ba3c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.399+0000 7f6f42ffd700 10 client.15006 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:10:17.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.399+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/3757217618 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b5ba3c6300 0x55b5ba3c7770 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a698308188d30202 server_cookie=a6ce1dc9b0d57109 in_seq=0 out_seq=0 2024-05-09T02:10:17.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.400+0000 7f6f42ffd700 1 -- 172.21.15.19:0/3757217618 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b5ba2dca30 con 0x55b5ba3c6300 2024-05-09T02:10:17.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.400+0000 7f6f42ffd700 10 client.15006 handle_command_reply: tid=0 2024-05-09T02:10:17.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.400+0000 7f6f7888fb80 1 client.15006 shutdown 2024-05-09T02:10:17.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.400+0000 7f6f7888fb80 1 -- 172.21.15.19:0/3757217618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba22f450 msgr2=0x55b5ba3b60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:17.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.400+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/3757217618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba22f450 0x55b5ba3b60f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400b2f0 tx=0x7f6f54010b60 comp rx=0 tx=0).stop 2024-05-09T02:10:17.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.401+0000 7f6f7888fb80 1 -- 172.21.15.19:0/3757217618 shutdown_connections 2024-05-09T02:10:17.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.401+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/3757217618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba3b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.401+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/3757217618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba22f450 0x55b5ba3b60f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.401+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/3757217618 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b5ba3c6300 0x55b5ba3c7770 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.401+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/3757217618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.401+0000 7f6f7888fb80 1 -- 172.21.15.19:0/3757217618 >> 172.21.15.19:0/3757217618 conn(0x55b5ba232f20 msgr2=0x55b5ba3a0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:17.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.401+0000 7f6f42ffd700 0 client.15006 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:10:17.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.401+0000 7f6f7888fb80 1 -- 172.21.15.19:0/3757217618 shutdown_connections 2024-05-09T02:10:17.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.401+0000 7f6f7888fb80 1 -- 172.21.15.19:0/3757217618 wait complete. 2024-05-09T02:10:17.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.401+0000 7f6f7888fb80 20 client.15006 trim_cache size 0 max 16384 2024-05-09T02:10:17.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.422+0000 7f6f7888fb80 1 Processor -- start 2024-05-09T02:10:17.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.422+0000 7f6f7888fb80 1 -- start start 2024-05-09T02:10:17.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.422+0000 7f6f7888fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba22f450 0x55b5ba3a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.422+0000 7f6f7888fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3c2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.422+0000 7f6f7888fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba3b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.422+0000 7f6f7888fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5ba3c5ab0 con 0x55b5ba2a9cb0 2024-05-09T02:10:17.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.422+0000 7f6f7888fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b5ba3c6180 con 0x55b5ba3ac3b0 2024-05-09T02:10:17.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.422+0000 7f6f7888fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b5ba3c6850 con 0x55b5ba22f450 2024-05-09T02:10:17.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.422+0000 7f6f62ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.422+0000 7f6f62ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35842/0 (socket says 172.21.15.19:35842) 2024-05-09T02:10:17.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.422+0000 7f6f62ffd700 1 -- 172.21.15.19:0/1109192536 learned_addr learned my addr 172.21.15.19:0/1109192536 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:17.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.422+0000 7f6f637fe700 1 --2- 172.21.15.19:0/1109192536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba22f450 0x55b5ba3a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.422+0000 7f6f63fff700 1 --2- 172.21.15.19:0/1109192536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba3b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.423+0000 7f6f62ffd700 1 -- 172.21.15.19:0/1109192536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba22f450 msgr2=0x55b5ba3a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:17.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.423+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/1109192536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba22f450 0x55b5ba3a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.423+0000 7f6f62ffd700 1 -- 172.21.15.19:0/1109192536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba3ac3b0 msgr2=0x55b5ba3b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:17.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.423+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/1109192536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba3b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.423+0000 7f6f62ffd700 1 -- 172.21.15.19:0/1109192536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5ba3a3980 con 0x55b5ba2a9cb0 2024-05-09T02:10:17.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.423+0000 7f6f637fe700 1 --2- 172.21.15.19:0/1109192536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba22f450 0x55b5ba3a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:10:17.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.423+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/1109192536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3c2180 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55b5ba3a8c40 tx=0x7f6f58005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5638afd4ee5e319c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:17.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.423+0000 7f6f497fa700 1 -- 172.21.15.19:0/1109192536 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58005070 con 0x55b5ba2a9cb0 2024-05-09T02:10:17.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.423+0000 7f6f497fa700 1 -- 172.21.15.19:0/1109192536 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f58039e50 con 0x55b5ba2a9cb0 2024-05-09T02:10:17.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.423+0000 7f6f497fa700 1 -- 172.21.15.19:0/1109192536 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800f740 con 0x55b5ba2a9cb0 2024-05-09T02:10:17.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.424+0000 7f6f7888fb80 1 -- 172.21.15.19:0/1109192536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba2a9cb0 msgr2=0x55b5ba3c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:17.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.424+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/1109192536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3c2180 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55b5ba3a8c40 tx=0x7f6f58005790 comp rx=0 tx=0).stop 2024-05-09T02:10:17.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.424+0000 7f6f7888fb80 1 -- 172.21.15.19:0/1109192536 shutdown_connections 2024-05-09T02:10:17.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.424+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/1109192536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba3b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.424+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/1109192536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba22f450 0x55b5ba3a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.424+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/1109192536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3c2180 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.424+0000 7f6f7888fb80 1 -- 172.21.15.19:0/1109192536 >> 172.21.15.19:0/1109192536 conn(0x55b5ba232f20 msgr2=0x55b5ba3a0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:17.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.424+0000 7f6f7888fb80 1 -- 172.21.15.19:0/1109192536 shutdown_connections 2024-05-09T02:10:17.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.424+0000 7f6f7888fb80 1 -- 172.21.15.19:0/1109192536 wait complete. 2024-05-09T02:10:17.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.424+0000 7f6f7888fb80 1 -- start start 2024-05-09T02:10:17.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f7888fb80 1 Processor -- start 2024-05-09T02:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f7888fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba22f450 0x55b5ba3ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f7888fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f7888fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba430770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f7888fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5ba3c75f0 con 0x55b5ba3ac3b0 2024-05-09T02:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f7888fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b5ba3c6f20 con 0x55b5ba22f450 2024-05-09T02:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f7888fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b5ba3c7cc0 con 0x55b5ba2a9cb0 2024-05-09T02:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f7888fb80 10 client.0 _ll_register_callbacks cb 0x55b5ba140670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f63fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba430770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f63fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba430770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35846/0 (socket says 172.21.15.19:35846) 2024-05-09T02:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f63fff700 1 -- 172.21.15.19:0/2066161629 learned_addr learned my addr 172.21.15.19:0/2066161629 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f637fe700 1 --2- 172.21.15.19:0/2066161629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba22f450 0x55b5ba3ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/2066161629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f637fe700 1 -- 172.21.15.19:0/2066161629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba2a9cb0 msgr2=0x55b5ba3cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f637fe700 1 --2- 172.21.15.19:0/2066161629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f637fe700 1 -- 172.21.15.19:0/2066161629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba3ac3b0 msgr2=0x55b5ba430770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f637fe700 1 --2- 172.21.15.19:0/2066161629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba430770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f637fe700 1 -- 172.21.15.19:0/2066161629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5ba3cd840 con 0x55b5ba22f450 2024-05-09T02:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.425+0000 7f6f63fff700 1 --2- 172.21.15.19:0/2066161629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba430770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.426+0000 7f6f637fe700 1 --2- 172.21.15.19:0/2066161629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba22f450 0x55b5ba3ccad0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f540a97a0 tx=0x7f6f540a9160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af6d1f918cb555b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.426+0000 7f6f42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.426+0000 7f6f42ffd700 1 -- 172.21.15.19:0/2066161629 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54020070 con 0x55b5ba22f450 2024-05-09T02:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.426+0000 7f6f7888fb80 1 -- 172.21.15.19:0/2066161629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f58039e50 con 0x55b5ba22f450 2024-05-09T02:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.426+0000 7f6f42ffd700 1 -- 172.21.15.19:0/2066161629 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f6f540a9a70 con 0x55b5ba22f450 2024-05-09T02:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.426+0000 7f6f42ffd700 1 -- 172.21.15.19:0/2066161629 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f540b23f0 con 0x55b5ba22f450 2024-05-09T02:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.426+0000 7f6f7888fb80 10 client.24776 fetch_fsmap learned FSMap version 28 2024-05-09T02:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.426+0000 7f6f7888fb80 1 -- 172.21.15.19:0/2066161629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b5ba431c70 con 0x55b5ba22f450 2024-05-09T02:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.426+0000 7f6f42ffd700 1 -- 172.21.15.19:0/2066161629 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f58039e50 con 0x55b5ba22f450 2024-05-09T02:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.427+0000 7f6f42ffd700 1 -- 172.21.15.19:0/2066161629 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6f540910b0 con 0x55b5ba22f450 2024-05-09T02:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.427+0000 7f6f7888fb80 10 client.24776 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.427+0000 7f6f7888fb80 10 client.24776 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T02:10:17.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.427+0000 7f6f7888fb80 20 client.24776 populate_metadata read hostname 'smithi019' 2024-05-09T02:10:17.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.427+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/2066161629 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b5ba433f40 0x55b5ba4353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:17.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.427+0000 7f6f7888fb80 4 client.24776 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T02:10:17.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.427+0000 7f6f7888fb80 1 -- 172.21.15.19:0/2066161629 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "status"}) -- 0x55b5ba2d6ad0 con 0x55b5ba433f40 2024-05-09T02:10:17.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.427+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/2066161629 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b5ba433f40 0x55b5ba4353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:17.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f62ffd700 1 --2- 172.21.15.19:0/2066161629 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b5ba433f40 0x55b5ba4353b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=791f6ea918b83c56 server_cookie=7cadc777774f972d in_seq=0 out_seq=0 2024-05-09T02:10:17.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f42ffd700 10 client.24776 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:10:17.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f42ffd700 1 -- 172.21.15.19:0/2066161629 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b5ba2d6ad0 con 0x55b5ba433f40 2024-05-09T02:10:17.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f42ffd700 10 client.24776 handle_command_reply: tid=0 2024-05-09T02:10:17.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f7888fb80 1 client.24776 shutdown 2024-05-09T02:10:17.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f7888fb80 1 -- 172.21.15.19:0/2066161629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba22f450 msgr2=0x55b5ba3ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:17.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/2066161629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba22f450 0x55b5ba3ccad0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f540a97a0 tx=0x7f6f540a9160 comp rx=0 tx=0).stop 2024-05-09T02:10:17.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f7888fb80 1 -- 172.21.15.19:0/2066161629 shutdown_connections 2024-05-09T02:10:17.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/2066161629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5ba3ac3b0 0x55b5ba430770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/2066161629 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b5ba433f40 0x55b5ba4353b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/2066161629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5ba22f450 0x55b5ba3ccad0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f7888fb80 1 --2- 172.21.15.19:0/2066161629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5ba2a9cb0 0x55b5ba3cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f42ffd700 0 client.24776 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:10:17.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f7888fb80 1 -- 172.21.15.19:0/2066161629 >> 172.21.15.19:0/2066161629 conn(0x55b5ba232f20 msgr2=0x55b5ba3a1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:17.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f7888fb80 1 -- 172.21.15.19:0/2066161629 shutdown_connections 2024-05-09T02:10:17.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f7888fb80 1 -- 172.21.15.19:0/2066161629 wait complete. 2024-05-09T02:10:17.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.428+0000 7f6f7888fb80 20 client.24776 trim_cache size 0 max 16384 2024-05-09T02:10:17.435 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:10:17.435 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "a0611f3e-0da7-11ef-bc97-c7b262605968", 2024-05-09T02:10:17.435 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-05-09T02:10:17.435 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24463, 2024-05-09T02:10:17.435 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T02:10:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T02:10:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T02:10:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 132.197781969, 2024-05-09T02:10:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-05-09T02:10:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 81, 2024-05-09T02:10:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 79, 2024-05-09T02:10:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 177.64613467300001 2024-05-09T02:10:17.436 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:10:17.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.429+0000 7f6f69e79700 1 -- 172.21.15.19:0/630506411 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6f4c0b2900 msgr2=0x7f6f4c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:17.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.429+0000 7f6f69e79700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6f4c0b2900 0x7f6f4c0b4e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f58000c00 tx=0x7f6f5801a040 comp rx=0 tx=0).stop 2024-05-09T02:10:17.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.429+0000 7f6f69e79700 1 -- 172.21.15.19:0/630506411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f641034c0 msgr2=0x7f6f6419aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:17.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.429+0000 7f6f69e79700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f641034c0 0x7f6f6419aed0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400d950 tx=0x7f6f5400dd10 comp rx=0 tx=0).stop 2024-05-09T02:10:17.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.429+0000 7f6f69e79700 1 -- 172.21.15.19:0/630506411 shutdown_connections 2024-05-09T02:10:17.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.429+0000 7f6f69e79700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6f4c0b2900 0x7f6f4c0b4e10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.429+0000 7f6f69e79700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f64108740 0x7f6f6419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.429+0000 7f6f69e79700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f641034c0 0x7f6f6419aed0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.429+0000 7f6f69e79700 1 --2- 172.21.15.19:0/630506411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f64107df0 0x7f6f6419b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:17.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.429+0000 7f6f69e79700 1 -- 172.21.15.19:0/630506411 >> 172.21.15.19:0/630506411 conn(0x7f6f640fedf0 msgr2=0x7f6f641096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:17.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.429+0000 7f6f69e79700 1 -- 172.21.15.19:0/630506411 shutdown_connections 2024-05-09T02:10:17.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:17.429+0000 7f6f69e79700 1 -- 172.21.15.19:0/630506411 wait complete. 2024-05-09T02:10:17.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:17 smithi059 ceph-mon[34781]: pgmap v367: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:10:17.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:17 smithi196 ceph-mon[34718]: pgmap v367: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:10:18.196 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T02:10:18.454 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:19.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:19 smithi059 ceph-mon[34781]: pgmap v368: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:19.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:19 smithi196 ceph-mon[34718]: pgmap v368: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:19.561 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:19 smithi019 ceph-mon[30151]: pgmap v368: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:20.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.149+0000 7f66398bb700 1 -- 172.21.15.19:0/3678161579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66341045d0 msgr2=0x7f66341049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:20.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.149+0000 7f66398bb700 1 --2- 172.21.15.19:0/3678161579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66341045d0 0x7f66341049e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f661c0099c0 tx=0x7f661c009cd0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.149+0000 7f66398bb700 1 -- 172.21.15.19:0/3678161579 shutdown_connections 2024-05-09T02:10:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.149+0000 7f66398bb700 1 --2- 172.21.15.19:0/3678161579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6634109850 0x7f66341111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.149+0000 7f66398bb700 1 --2- 172.21.15.19:0/3678161579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66341045d0 0x7f66341049e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.149+0000 7f66398bb700 1 --2- 172.21.15.19:0/3678161579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6634108f00 0x7f6634109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.149+0000 7f66398bb700 1 -- 172.21.15.19:0/3678161579 >> 172.21.15.19:0/3678161579 conn(0x7f66340ffea0 msgr2=0x7f66341022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.150+0000 7f66398bb700 1 -- 172.21.15.19:0/3678161579 shutdown_connections 2024-05-09T02:10:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.150+0000 7f66398bb700 1 -- 172.21.15.19:0/3678161579 wait complete. 2024-05-09T02:10:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.151+0000 7f66398bb700 1 Processor -- start 2024-05-09T02:10:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.151+0000 7f66398bb700 1 -- start start 2024-05-09T02:10:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f66398bb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66341045d0 0x7f663419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f66398bb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6634108f00 0x7f663419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f66398bb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6634109850 0x7f663419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f66398bb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f66341144d0 con 0x7f6634109850 2024-05-09T02:10:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f66398bb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6634114350 con 0x7f66341045d0 2024-05-09T02:10:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f66398bb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6634114650 con 0x7f6634108f00 2024-05-09T02:10:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f66337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6634109850 0x7f663419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f66337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6634109850 0x7f663419f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35876/0 (socket says 172.21.15.19:35876) 2024-05-09T02:10:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f66337fe700 1 -- 172.21.15.19:0/4119782668 learned_addr learned my addr 172.21.15.19:0/4119782668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f6632ffd700 1 --2- 172.21.15.19:0/4119782668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66341045d0 0x7f663419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f66327fc700 1 --2- 172.21.15.19:0/4119782668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6634108f00 0x7f663419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f66337fe700 1 -- 172.21.15.19:0/4119782668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6634108f00 msgr2=0x7f663419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f66337fe700 1 --2- 172.21.15.19:0/4119782668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6634108f00 0x7f663419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f66337fe700 1 -- 172.21.15.19:0/4119782668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66341045d0 msgr2=0x7f663419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f66337fe700 1 --2- 172.21.15.19:0/4119782668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66341045d0 0x7f663419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.152+0000 7f66337fe700 1 -- 172.21.15.19:0/4119782668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663419fc20 con 0x7f6634109850 2024-05-09T02:10:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.153+0000 7f66337fe700 1 --2- 172.21.15.19:0/4119782668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6634109850 0x7f663419f630 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f662c007d70 tx=0x7f662c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.153+0000 7f66388b9700 1 -- 172.21.15.19:0/4119782668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c016020 con 0x7f6634109850 2024-05-09T02:10:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.153+0000 7f66388b9700 1 -- 172.21.15.19:0/4119782668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f662c00ac90 con 0x7f6634109850 2024-05-09T02:10:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.153+0000 7f66398bb700 1 -- 172.21.15.19:0/4119782668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663419ff10 con 0x7f6634109850 2024-05-09T02:10:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.153+0000 7f66388b9700 1 -- 172.21.15.19:0/4119782668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c0115e0 con 0x7f6634109850 2024-05-09T02:10:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.153+0000 7f66398bb700 1 -- 172.21.15.19:0/4119782668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66341a0400 con 0x7f6634109850 2024-05-09T02:10:20.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.156+0000 7f66388b9700 1 -- 172.21.15.19:0/4119782668 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f662c019020 con 0x7f6634109850 2024-05-09T02:10:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f66482d1b80 1 Processor -- start 2024-05-09T02:10:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f66388b9700 1 --2- 172.21.15.19:0/4119782668 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f66200b2880 0x7f66200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f66482d1b80 1 -- start start 2024-05-09T02:10:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f66327fc700 1 --2- 172.21.15.19:0/4119782668 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f66200b2880 0x7f66200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f66388b9700 1 -- 172.21.15.19:0/4119782668 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f662c0e21b0 con 0x7f6634109850 2024-05-09T02:10:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f66482d1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575db9cbcb0 0x5575dbad2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f66482d1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 0x5575dbace3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f66482d1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575dbace9d0 0x5575dbae3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f66482d1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5575db9f1050 con 0x5575db9cbcb0 2024-05-09T02:10:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f66482d1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5575db996950 con 0x5575dbace9d0 2024-05-09T02:10:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f66482d1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5575db9ed010 con 0x5575db951450 2024-05-09T02:10:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f6632ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575db9cbcb0 0x5575dbad2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f6632ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575db9cbcb0 0x5575dbad2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35886/0 (socket says 172.21.15.19:35886) 2024-05-09T02:10:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f6632ffd700 1 -- 172.21.15.19:0/2095108552 learned_addr learned my addr 172.21.15.19:0/2095108552 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f66327fc700 1 --2- 172.21.15.19:0/4119782668 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f66200b2880 0x7f66200b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6634107d70 tx=0x7f662400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f66337fe700 1 --2- 172.21.15.19:0/2095108552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575dbace9d0 0x5575dbae3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.157+0000 7f66327fc700 1 --2- 172.21.15.19:0/2095108552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 0x5575dbace3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.158+0000 7f6632ffd700 1 -- 172.21.15.19:0/2095108552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 msgr2=0x5575dbace3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.158+0000 7f6632ffd700 1 --2- 172.21.15.19:0/2095108552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 0x5575dbace3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.158+0000 7f6632ffd700 1 -- 172.21.15.19:0/2095108552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575dbace9d0 msgr2=0x5575dbae3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.158+0000 7f6632ffd700 1 --2- 172.21.15.19:0/2095108552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575dbace9d0 0x5575dbae3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.158+0000 7f6632ffd700 1 -- 172.21.15.19:0/2095108552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575db9874b0 con 0x5575db9cbcb0 2024-05-09T02:10:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.158+0000 7f66327fc700 1 --2- 172.21.15.19:0/2095108552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 0x5575dbace3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:10:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.158+0000 7f6632ffd700 1 --2- 172.21.15.19:0/2095108552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575db9cbcb0 0x5575dbad2250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f661c005b40 tx=0x7f661c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05f580f25a1d307 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:20.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.158+0000 7f6629ffb700 1 -- 172.21.15.19:0/2095108552 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c003c80 con 0x5575db9cbcb0 2024-05-09T02:10:20.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.158+0000 7f6629ffb700 1 -- 172.21.15.19:0/2095108552 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f661c003e20 con 0x5575db9cbcb0 2024-05-09T02:10:20.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.158+0000 7f6629ffb700 1 -- 172.21.15.19:0/2095108552 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c00bea0 con 0x5575db9cbcb0 2024-05-09T02:10:20.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.159+0000 7f66482d1b80 1 -- 172.21.15.19:0/2095108552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575db9cbcb0 msgr2=0x5575dbad2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:20.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.159+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2095108552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575db9cbcb0 0x5575dbad2250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f661c005b40 tx=0x7f661c003910 comp rx=0 tx=0).stop 2024-05-09T02:10:20.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.159+0000 7f66482d1b80 1 -- 172.21.15.19:0/2095108552 shutdown_connections 2024-05-09T02:10:20.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.159+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2095108552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575dbace9d0 0x5575dbae3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.159+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2095108552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575db9cbcb0 0x5575dbad2250 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.159+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2095108552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 0x5575dbace3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.159+0000 7f66482d1b80 1 -- 172.21.15.19:0/2095108552 >> 172.21.15.19:0/2095108552 conn(0x5575db954f20 msgr2=0x5575dbac91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.159+0000 7f66482d1b80 1 -- 172.21.15.19:0/2095108552 shutdown_connections 2024-05-09T02:10:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.159+0000 7f66482d1b80 1 -- 172.21.15.19:0/2095108552 wait complete. 2024-05-09T02:10:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.159+0000 7f66482d1b80 1 -- start start 2024-05-09T02:10:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.160+0000 7f66482d1b80 1 Processor -- start 2024-05-09T02:10:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.160+0000 7f66482d1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 0x5575dbae0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.160+0000 7f66482d1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575db9cbcb0 0x5575dbae09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.160+0000 7f66482d1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575dbace9d0 0x5575dbae0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.160+0000 7f66482d1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5575dbae3560 con 0x5575db9cbcb0 2024-05-09T02:10:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.160+0000 7f66482d1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5575db98bad0 con 0x5575dbace9d0 2024-05-09T02:10:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.160+0000 7f66482d1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5575dbae36e0 con 0x5575db951450 2024-05-09T02:10:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.160+0000 7f66327fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575db9cbcb0 0x5575dbae09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.160+0000 7f66482d1b80 10 client.0 _ll_register_callbacks cb 0x5575db862670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:10:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.160+0000 7f66327fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575db9cbcb0 0x5575dbae09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35888/0 (socket says 172.21.15.19:35888) 2024-05-09T02:10:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.160+0000 7f66327fc700 1 -- 172.21.15.19:0/2578512740 learned_addr learned my addr 172.21.15.19:0/2578512740 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.160+0000 7f66337fe700 1 --2- 172.21.15.19:0/2578512740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575dbace9d0 0x5575dbae0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.160+0000 7f6632ffd700 1 --2- 172.21.15.19:0/2578512740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 0x5575dbae0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.161+0000 7f66337fe700 1 -- 172.21.15.19:0/2578512740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 msgr2=0x5575dbae0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.161+0000 7f66337fe700 1 --2- 172.21.15.19:0/2578512740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 0x5575dbae0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.161+0000 7f66337fe700 1 -- 172.21.15.19:0/2578512740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575db9cbcb0 msgr2=0x5575dbae09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.161+0000 7f66337fe700 1 --2- 172.21.15.19:0/2578512740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575db9cbcb0 0x5575dbae09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.161+0000 7f66337fe700 1 -- 172.21.15.19:0/2578512740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575db86c440 con 0x5575dbace9d0 2024-05-09T02:10:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.161+0000 7f66327fc700 1 --2- 172.21.15.19:0/2578512740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575db9cbcb0 0x5575dbae09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:10:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.161+0000 7f66337fe700 1 --2- 172.21.15.19:0/2578512740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575dbace9d0 0x5575dbae0ee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f662c0a9450 tx=0x7f662c0129f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43ab48abb5b41015 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.161+0000 7f6612ffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:10:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.161+0000 7f6612ffd700 1 -- 172.21.15.19:0/2578512740 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c0c3060 con 0x5575dbace9d0 2024-05-09T02:10:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.161+0000 7f66482d1b80 1 -- 172.21.15.19:0/2578512740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f661c003e20 con 0x5575dbace9d0 2024-05-09T02:10:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.161+0000 7f6612ffd700 1 -- 172.21.15.19:0/2578512740 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f662c0912f0 con 0x5575dbace9d0 2024-05-09T02:10:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.161+0000 7f6612ffd700 1 -- 172.21.15.19:0/2578512740 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f662c0a8ec0 con 0x5575dbace9d0 2024-05-09T02:10:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.162+0000 7f66482d1b80 10 client.24785 fetch_fsmap learned FSMap version 28 2024-05-09T02:10:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.162+0000 7f66482d1b80 1 -- 172.21.15.19:0/2578512740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5575dbaee0d0 con 0x5575dbace9d0 2024-05-09T02:10:20.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.162+0000 7f6612ffd700 1 -- 172.21.15.19:0/2578512740 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f661c003e20 con 0x5575dbace9d0 2024-05-09T02:10:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.162+0000 7f6612ffd700 1 -- 172.21.15.19:0/2578512740 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f662c0beba0 con 0x5575dbace9d0 2024-05-09T02:10:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.162+0000 7f66482d1b80 10 client.24785 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:10:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.162+0000 7f66482d1b80 10 client.24785 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T02:10:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.162+0000 7f66482d1b80 20 client.24785 populate_metadata read hostname 'smithi019' 2024-05-09T02:10:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.162+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2578512740 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5575dbaf0770 0x5575dbaf1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.162+0000 7f66482d1b80 4 client.24785 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.162+0000 7f66482d1b80 1 -- 172.21.15.19:0/2578512740 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575db9fea30 con 0x5575dbaf0770 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.162+0000 7f6632ffd700 1 --2- 172.21.15.19:0/2578512740 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5575dbaf0770 0x5575dbaf1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.163+0000 7f6632ffd700 1 --2- 172.21.15.19:0/2578512740 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5575dbaf0770 0x5575dbaf1be0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9090141b83f24247 server_cookie=a3881f4fed617069 in_seq=0 out_seq=0 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.163+0000 7f6612ffd700 10 client.24785 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.164+0000 7f6612ffd700 1 -- 172.21.15.19:0/2578512740 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5575db9fea30 con 0x5575dbaf0770 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.164+0000 7f6612ffd700 10 client.24785 handle_command_reply: tid=0 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.164+0000 7f66482d1b80 1 client.24785 shutdown 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.164+0000 7f66482d1b80 1 -- 172.21.15.19:0/2578512740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575dbace9d0 msgr2=0x5575dbae0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.164+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2578512740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575dbace9d0 0x5575dbae0ee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f662c0a9450 tx=0x7f662c0129f0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.164+0000 7f66482d1b80 1 -- 172.21.15.19:0/2578512740 shutdown_connections 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.164+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2578512740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575dbace9d0 0x5575dbae0ee0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.164+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2578512740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 0x5575dbae0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.164+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2578512740 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5575dbaf0770 0x5575dbaf1be0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.164+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2578512740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575db9cbcb0 0x5575dbae09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.164+0000 7f66482d1b80 1 -- 172.21.15.19:0/2578512740 >> 172.21.15.19:0/2578512740 conn(0x5575db954f20 msgr2=0x5575dbacb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.164+0000 7f6612ffd700 0 client.24785 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:10:20.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.164+0000 7f66482d1b80 1 -- 172.21.15.19:0/2578512740 shutdown_connections 2024-05-09T02:10:20.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.164+0000 7f66482d1b80 1 -- 172.21.15.19:0/2578512740 wait complete. 2024-05-09T02:10:20.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.164+0000 7f66482d1b80 20 client.24785 trim_cache size 0 max 16384 2024-05-09T02:10:20.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.189+0000 7f66482d1b80 1 Processor -- start 2024-05-09T02:10:20.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.189+0000 7f66482d1b80 1 -- start start 2024-05-09T02:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.189+0000 7f66482d1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575db951450 0x5575dbaf00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.189+0000 7f66482d1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db9cbcb0 0x5575dbaf1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.189+0000 7f66482d1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575dbace9d0 0x5575dbaec940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.189+0000 7f66482d1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5575dbaeff20 con 0x5575dbace9d0 2024-05-09T02:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.189+0000 7f66482d1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5575dbaf05f0 con 0x5575db951450 2024-05-09T02:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.189+0000 7f66482d1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5575dbaf0cc0 con 0x5575db9cbcb0 2024-05-09T02:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.189+0000 7f66337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575dbace9d0 0x5575dbaec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.189+0000 7f66337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575dbace9d0 0x5575dbaec940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35904/0 (socket says 172.21.15.19:35904) 2024-05-09T02:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.189+0000 7f66337fe700 1 -- 172.21.15.19:0/339444180 learned_addr learned my addr 172.21.15.19:0/339444180 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.189+0000 7f6632ffd700 1 --2- 172.21.15.19:0/339444180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575db951450 0x5575dbaf00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.189+0000 7f66327fc700 1 --2- 172.21.15.19:0/339444180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db9cbcb0 0x5575dbaf1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.190+0000 7f66337fe700 1 -- 172.21.15.19:0/339444180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db9cbcb0 msgr2=0x5575dbaf1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.190+0000 7f66337fe700 1 --2- 172.21.15.19:0/339444180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db9cbcb0 0x5575dbaf1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.190+0000 7f66337fe700 1 -- 172.21.15.19:0/339444180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575db951450 msgr2=0x5575dbaf00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.190+0000 7f66337fe700 1 --2- 172.21.15.19:0/339444180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575db951450 0x5575dbaf00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.190+0000 7f66337fe700 1 -- 172.21.15.19:0/339444180 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575dbae2d00 con 0x5575dbace9d0 2024-05-09T02:10:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.190+0000 7f66337fe700 1 --2- 172.21.15.19:0/339444180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575dbace9d0 0x5575dbaec940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f662c018ff0 tx=0x7f662c007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6969e1b5a759586 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.190+0000 7f66297fa700 1 -- 172.21.15.19:0/339444180 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c0cc0b0 con 0x5575dbace9d0 2024-05-09T02:10:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.190+0000 7f66297fa700 1 -- 172.21.15.19:0/339444180 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f662c00ae40 con 0x5575dbace9d0 2024-05-09T02:10:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.190+0000 7f66297fa700 1 -- 172.21.15.19:0/339444180 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c0beba0 con 0x5575dbace9d0 2024-05-09T02:10:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.191+0000 7f66482d1b80 1 -- 172.21.15.19:0/339444180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575dbace9d0 msgr2=0x5575dbaec940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.191+0000 7f66482d1b80 1 --2- 172.21.15.19:0/339444180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575dbace9d0 0x5575dbaec940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f662c018ff0 tx=0x7f662c007c50 comp rx=0 tx=0).stop 2024-05-09T02:10:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.191+0000 7f66482d1b80 1 -- 172.21.15.19:0/339444180 shutdown_connections 2024-05-09T02:10:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.191+0000 7f66482d1b80 1 --2- 172.21.15.19:0/339444180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575dbace9d0 0x5575dbaec940 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.191+0000 7f66482d1b80 1 --2- 172.21.15.19:0/339444180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575db951450 0x5575dbaf00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.191+0000 7f66482d1b80 1 --2- 172.21.15.19:0/339444180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db9cbcb0 0x5575dbaf1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.191+0000 7f66482d1b80 1 -- 172.21.15.19:0/339444180 >> 172.21.15.19:0/339444180 conn(0x5575db954f20 msgr2=0x5575dbaded50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.192+0000 7f66482d1b80 1 -- 172.21.15.19:0/339444180 shutdown_connections 2024-05-09T02:10:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.192+0000 7f66482d1b80 1 -- 172.21.15.19:0/339444180 wait complete. 2024-05-09T02:10:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.192+0000 7f66482d1b80 1 -- start start 2024-05-09T02:10:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.192+0000 7f66482d1b80 1 Processor -- start 2024-05-09T02:10:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.193+0000 7f66482d1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 0x5575dbafb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.193+0000 7f66482d1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575db9cbcb0 0x5575dbafb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.193+0000 7f66482d1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575dbace9d0 0x5575dbaf4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.193+0000 7f66482d1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5575dbaf1a60 con 0x5575dbace9d0 2024-05-09T02:10:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.193+0000 7f66482d1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5575dbaf1390 con 0x5575db9cbcb0 2024-05-09T02:10:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.193+0000 7f66482d1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5575dbaf2130 con 0x5575db951450 2024-05-09T02:10:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.193+0000 7f66482d1b80 10 client.0 _ll_register_callbacks cb 0x5575db862670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:10:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.193+0000 7f66337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575dbace9d0 0x5575dbaf4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.193+0000 7f6632ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 0x5575dbafb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.193+0000 7f66337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575dbace9d0 0x5575dbaf4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35916/0 (socket says 172.21.15.19:35916) 2024-05-09T02:10:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.193+0000 7f66337fe700 1 -- 172.21.15.19:0/2519555947 learned_addr learned my addr 172.21.15.19:0/2519555947 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.193+0000 7f66327fc700 1 --2- 172.21.15.19:0/2519555947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575db9cbcb0 0x5575dbafb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.194+0000 7f66337fe700 1 -- 172.21.15.19:0/2519555947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 msgr2=0x5575dbafb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.194+0000 7f66337fe700 1 --2- 172.21.15.19:0/2519555947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 0x5575dbafb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.194+0000 7f66337fe700 1 -- 172.21.15.19:0/2519555947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575db9cbcb0 msgr2=0x5575dbafb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.194+0000 7f66337fe700 1 --2- 172.21.15.19:0/2519555947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575db9cbcb0 0x5575dbafb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.194+0000 7f66337fe700 1 -- 172.21.15.19:0/2519555947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575dbaf4ab0 con 0x5575dbace9d0 2024-05-09T02:10:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.194+0000 7f66337fe700 1 --2- 172.21.15.19:0/2519555947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575dbace9d0 0x5575dbaf4400 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f662c0a5320 tx=0x7f662c0129f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef17cae149b5d112 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.194+0000 7f6612ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:10:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.194+0000 7f6612ffd700 1 -- 172.21.15.19:0/2519555947 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c083e60 con 0x5575dbace9d0 2024-05-09T02:10:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.194+0000 7f66482d1b80 1 -- 172.21.15.19:0/2519555947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5575db9f8ad0 con 0x5575dbace9d0 2024-05-09T02:10:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.194+0000 7f6612ffd700 1 -- 172.21.15.19:0/2519555947 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f662c0a5d40 con 0x5575dbace9d0 2024-05-09T02:10:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.194+0000 7f6612ffd700 1 -- 172.21.15.19:0/2519555947 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f662c0c3020 con 0x5575dbace9d0 2024-05-09T02:10:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.194+0000 7f66482d1b80 10 client.15030 fetch_fsmap learned FSMap version 28 2024-05-09T02:10:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.194+0000 7f66482d1b80 1 -- 172.21.15.19:0/2519555947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5575dbaf93b0 con 0x5575dbace9d0 2024-05-09T02:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.195+0000 7f6612ffd700 1 -- 172.21.15.19:0/2519555947 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5575db9f8ad0 con 0x5575dbace9d0 2024-05-09T02:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.195+0000 7f6612ffd700 1 -- 172.21.15.19:0/2519555947 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f662c0a8420 con 0x5575dbace9d0 2024-05-09T02:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.195+0000 7f66482d1b80 10 client.15030 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.195+0000 7f66482d1b80 10 client.15030 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T02:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.195+0000 7f66482d1b80 20 client.15030 populate_metadata read hostname 'smithi019' 2024-05-09T02:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.195+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2519555947 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5575dbb5d490 0x5575dbb5e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.195+0000 7f66482d1b80 4 client.15030 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T02:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.195+0000 7f66482d1b80 1 -- 172.21.15.19:0/2519555947 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "status"}) -- 0x5575db86c920 con 0x5575dbb5d490 2024-05-09T02:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.196+0000 7f6632ffd700 1 --2- 172.21.15.19:0/2519555947 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5575dbb5d490 0x5575dbb5e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.196+0000 7f6632ffd700 1 --2- 172.21.15.19:0/2519555947 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5575dbb5d490 0x5575dbb5e900 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e1360761bf6e642 server_cookie=3afe8b794ef70fa in_seq=0 out_seq=0 2024-05-09T02:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.196+0000 7f6612ffd700 10 client.15030 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.196+0000 7f6612ffd700 1 -- 172.21.15.19:0/2519555947 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5575db86c920 con 0x5575dbb5d490 2024-05-09T02:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.196+0000 7f6612ffd700 10 client.15030 handle_command_reply: tid=0 2024-05-09T02:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.196+0000 7f66482d1b80 1 client.15030 shutdown 2024-05-09T02:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.197+0000 7f66482d1b80 1 -- 172.21.15.19:0/2519555947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575dbace9d0 msgr2=0x5575dbaf4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.197+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2519555947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575dbace9d0 0x5575dbaf4400 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f662c0a5320 tx=0x7f662c0129f0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.197+0000 7f66482d1b80 1 -- 172.21.15.19:0/2519555947 shutdown_connections 2024-05-09T02:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.197+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2519555947 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5575dbb5d490 0x5575dbb5e900 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.197+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2519555947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575dbace9d0 0x5575dbaf4400 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.197+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2519555947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575db951450 0x5575dbafb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.197+0000 7f66482d1b80 1 --2- 172.21.15.19:0/2519555947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5575db9cbcb0 0x5575dbafb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.197+0000 7f6612ffd700 0 client.15030 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.197+0000 7f66482d1b80 1 -- 172.21.15.19:0/2519555947 >> 172.21.15.19:0/2519555947 conn(0x5575db954f20 msgr2=0x5575dbacd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:20.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.197+0000 7f66482d1b80 1 -- 172.21.15.19:0/2519555947 shutdown_connections 2024-05-09T02:10:20.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.197+0000 7f66482d1b80 1 -- 172.21.15.19:0/2519555947 wait complete. 2024-05-09T02:10:20.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.197+0000 7f66482d1b80 20 client.15030 trim_cache size 0 max 16384 2024-05-09T02:10:20.202 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:10:20.202 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "a0611f3e-0da7-11ef-bc97-c7b262605968", 2024-05-09T02:10:20.202 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-05-09T02:10:20.202 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24473, 2024-05-09T02:10:20.202 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T02:10:20.202 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T02:10:20.202 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T02:10:20.203 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 135.97169246600001, 2024-05-09T02:10:20.203 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-05-09T02:10:20.203 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 81, 2024-05-09T02:10:20.203 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 79, 2024-05-09T02:10:20.203 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 164.14588169999999 2024-05-09T02:10:20.203 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:10:20.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.198+0000 7f66398bb700 1 -- 172.21.15.19:0/4119782668 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f66200b2880 msgr2=0x7f66200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:20.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.198+0000 7f66398bb700 1 --2- 172.21.15.19:0/4119782668 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f66200b2880 0x7f66200b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6634107d70 tx=0x7f662400a380 comp rx=0 tx=0).stop 2024-05-09T02:10:20.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.198+0000 7f66398bb700 1 -- 172.21.15.19:0/4119782668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6634109850 msgr2=0x7f663419f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:20.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.198+0000 7f66398bb700 1 --2- 172.21.15.19:0/4119782668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6634109850 0x7f663419f630 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f662c007d70 tx=0x7f662c00a340 comp rx=0 tx=0).stop 2024-05-09T02:10:20.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.198+0000 7f66398bb700 1 -- 172.21.15.19:0/4119782668 shutdown_connections 2024-05-09T02:10:20.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.198+0000 7f66398bb700 1 --2- 172.21.15.19:0/4119782668 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f66200b2880 0x7f66200b4d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.198+0000 7f66398bb700 1 --2- 172.21.15.19:0/4119782668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6634109850 0x7f663419f630 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.198+0000 7f66398bb700 1 --2- 172.21.15.19:0/4119782668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66341045d0 0x7f663419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.198+0000 7f66398bb700 1 --2- 172.21.15.19:0/4119782668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6634108f00 0x7f663419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:20.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.198+0000 7f66398bb700 1 -- 172.21.15.19:0/4119782668 >> 172.21.15.19:0/4119782668 conn(0x7f66340ffea0 msgr2=0x7f66341086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:20.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.198+0000 7f66398bb700 1 -- 172.21.15.19:0/4119782668 shutdown_connections 2024-05-09T02:10:20.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:20.199+0000 7f66398bb700 1 -- 172.21.15.19:0/4119782668 wait complete. 2024-05-09T02:10:20.891 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T02:10:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:21.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:21 smithi019 ceph-mon[30151]: pgmap v369: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:10:21.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:21 smithi059 ceph-mon[34781]: pgmap v369: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:10:21.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:21 smithi196 ceph-mon[34718]: pgmap v369: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:10:22.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.844+0000 7ff48fc42700 1 -- 172.21.15.19:0/2951000458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff488109850 msgr2=0x7ff4881111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:22.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.844+0000 7ff48fc42700 1 --2- 172.21.15.19:0/2951000458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff488109850 0x7ff4881111b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff48000b0d0 tx=0x7ff48000b3e0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.844+0000 7ff48fc42700 1 -- 172.21.15.19:0/2951000458 shutdown_connections 2024-05-09T02:10:22.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.844+0000 7ff48fc42700 1 --2- 172.21.15.19:0/2951000458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff488109850 0x7ff4881111b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.844+0000 7ff48fc42700 1 --2- 172.21.15.19:0/2951000458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4881045d0 0x7ff4881049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.844+0000 7ff48fc42700 1 --2- 172.21.15.19:0/2951000458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff488108f00 0x7ff488109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.844+0000 7ff48fc42700 1 -- 172.21.15.19:0/2951000458 >> 172.21.15.19:0/2951000458 conn(0x7ff4880ffea0 msgr2=0x7ff4881022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:22.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.844+0000 7ff48fc42700 1 -- 172.21.15.19:0/2951000458 shutdown_connections 2024-05-09T02:10:22.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.844+0000 7ff48fc42700 1 -- 172.21.15.19:0/2951000458 wait complete. 2024-05-09T02:10:22.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.845+0000 7ff48fc42700 1 Processor -- start 2024-05-09T02:10:22.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.845+0000 7ff48fc42700 1 -- start start 2024-05-09T02:10:22.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.845+0000 7ff48fc42700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4881045d0 0x7ff48819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.845+0000 7ff48fc42700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff488108f00 0x7ff48819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.845+0000 7ff48fc42700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff488109850 0x7ff48819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.845+0000 7ff48fc42700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff488114500 con 0x7ff4881045d0 2024-05-09T02:10:22.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.845+0000 7ff48fc42700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff488114380 con 0x7ff488108f00 2024-05-09T02:10:22.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.845+0000 7ff48fc42700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff488114680 con 0x7ff488109850 2024-05-09T02:10:22.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff48d9de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4881045d0 0x7ff48819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff48d9de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4881045d0 0x7ff48819ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35934/0 (socket says 172.21.15.19:35934) 2024-05-09T02:10:22.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff48d9de700 1 -- 172.21.15.19:0/1760803537 learned_addr learned my addr 172.21.15.19:0/1760803537 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:22.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff48e1df700 1 --2- 172.21.15.19:0/1760803537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff488109850 0x7ff48819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff48d1dd700 1 --2- 172.21.15.19:0/1760803537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff488108f00 0x7ff48819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff48d9de700 1 -- 172.21.15.19:0/1760803537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff488109850 msgr2=0x7ff48819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:22.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff48d9de700 1 --2- 172.21.15.19:0/1760803537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff488109850 0x7ff48819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff48d9de700 1 -- 172.21.15.19:0/1760803537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff488108f00 msgr2=0x7ff48819b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:22.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff48d9de700 1 --2- 172.21.15.19:0/1760803537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff488108f00 0x7ff48819b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff48d9de700 1 -- 172.21.15.19:0/1760803537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48819fd60 con 0x7ff4881045d0 2024-05-09T02:10:22.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff48d1dd700 1 --2- 172.21.15.19:0/1760803537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff488108f00 0x7ff48819b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:10:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff48d9de700 1 --2- 172.21.15.19:0/1760803537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4881045d0 0x7ff48819ae90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff48400b530 tx=0x7ff48400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff47effd700 1 -- 172.21.15.19:0/1760803537 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484015020 con 0x7ff4881045d0 2024-05-09T02:10:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff47effd700 1 -- 172.21.15.19:0/1760803537 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff484011ce0 con 0x7ff4881045d0 2024-05-09T02:10:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff48fc42700 1 -- 172.21.15.19:0/1760803537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4881a0050 con 0x7ff4881045d0 2024-05-09T02:10:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.846+0000 7ff47effd700 1 -- 172.21.15.19:0/1760803537 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484010e40 con 0x7ff4881045d0 2024-05-09T02:10:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.847+0000 7ff48fc42700 1 -- 172.21.15.19:0/1760803537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff488076b00 con 0x7ff4881045d0 2024-05-09T02:10:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.849+0000 7ff47effd700 1 -- 172.21.15.19:0/1760803537 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff484018020 con 0x7ff4881045d0 2024-05-09T02:10:22.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.850+0000 7ff47effd700 1 --2- 172.21.15.19:0/1760803537 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff4740b2880 0x7ff4740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.850+0000 7ff48d1dd700 1 --2- 172.21.15.19:0/1760803537 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff4740b2880 0x7ff4740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.850+0000 7ff47effd700 1 -- 172.21.15.19:0/1760803537 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7ff4840e1ae0 con 0x7ff4881045d0 2024-05-09T02:10:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.850+0000 7ff49e658b80 1 Processor -- start 2024-05-09T02:10:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.850+0000 7ff48d1dd700 1 --2- 172.21.15.19:0/1760803537 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff4740b2880 0x7ff4740b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff478005950 tx=0x7ff4780058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.850+0000 7ff49e658b80 1 -- start start 2024-05-09T02:10:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff49e658b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503f34110 0x55e503fcd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff49e658b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fcd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff49e658b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503fc7650 0x55e503fc7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff49e658b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e503ee9fa0 con 0x55e503e4a450 2024-05-09T02:10:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff49e658b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e503e887a0 con 0x55e503f34110 2024-05-09T02:10:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff49e658b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e503ee5f60 con 0x55e503fc7650 2024-05-09T02:10:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff48d1dd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fcd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff48d1dd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fcd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35942/0 (socket says 172.21.15.19:35942) 2024-05-09T02:10:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff48d1dd700 1 -- 172.21.15.19:0/3644353794 learned_addr learned my addr 172.21.15.19:0/3644353794 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff48d9de700 1 --2- 172.21.15.19:0/3644353794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503f34110 0x55e503fcd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff48e1df700 1 --2- 172.21.15.19:0/3644353794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503fc7650 0x55e503fc7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff48d1dd700 1 -- 172.21.15.19:0/3644353794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503fc7650 msgr2=0x55e503fc7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff48d1dd700 1 --2- 172.21.15.19:0/3644353794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503fc7650 0x55e503fc7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff48d1dd700 1 -- 172.21.15.19:0/3644353794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503f34110 msgr2=0x55e503fcd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff48d1dd700 1 --2- 172.21.15.19:0/3644353794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503f34110 0x55e503fcd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff48d1dd700 1 -- 172.21.15.19:0/3644353794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e503e80320 con 0x55e503e4a450 2024-05-09T02:10:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.851+0000 7ff48d1dd700 1 --2- 172.21.15.19:0/3644353794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fcd9e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff47800b5d0 tx=0x7ff478012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2b0b429f36a618 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.852+0000 7ff473fff700 1 -- 172.21.15.19:0/3644353794 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478003880 con 0x55e503e4a450 2024-05-09T02:10:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.852+0000 7ff473fff700 1 -- 172.21.15.19:0/3644353794 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff478024070 con 0x55e503e4a450 2024-05-09T02:10:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.852+0000 7ff473fff700 1 -- 172.21.15.19:0/3644353794 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47801fcf0 con 0x55e503e4a450 2024-05-09T02:10:22.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.853+0000 7ff49e658b80 1 -- 172.21.15.19:0/3644353794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 msgr2=0x55e503fcd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:22.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.853+0000 7ff49e658b80 1 --2- 172.21.15.19:0/3644353794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fcd9e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff47800b5d0 tx=0x7ff478012000 comp rx=0 tx=0).stop 2024-05-09T02:10:22.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.853+0000 7ff49e658b80 1 -- 172.21.15.19:0/3644353794 shutdown_connections 2024-05-09T02:10:22.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.853+0000 7ff49e658b80 1 --2- 172.21.15.19:0/3644353794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503fc7650 0x55e503fc7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.853+0000 7ff49e658b80 1 --2- 172.21.15.19:0/3644353794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503f34110 0x55e503fcd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.853+0000 7ff49e658b80 1 --2- 172.21.15.19:0/3644353794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fcd9e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.853+0000 7ff49e658b80 1 -- 172.21.15.19:0/3644353794 >> 172.21.15.19:0/3644353794 conn(0x55e503eaa650 msgr2=0x55e503fc23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:22.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.853+0000 7ff49e658b80 1 -- 172.21.15.19:0/3644353794 shutdown_connections 2024-05-09T02:10:22.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.853+0000 7ff49e658b80 1 -- 172.21.15.19:0/3644353794 wait complete. 2024-05-09T02:10:22.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.853+0000 7ff49e658b80 1 -- start start 2024-05-09T02:10:22.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.853+0000 7ff49e658b80 1 Processor -- start 2024-05-09T02:10:22.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff49e658b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503e4a450 0x55e503fd11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff49e658b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503f34110 0x55e503fd16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff49e658b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503fc7650 0x55e503fd1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff49e658b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e503d65b50 con 0x55e503f34110 2024-05-09T02:10:22.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff49e658b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e503ea4100 con 0x55e503e4a450 2024-05-09T02:10:22.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff49e658b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e503fd4180 con 0x55e503fc7650 2024-05-09T02:10:22.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff49e658b80 10 client.0 _ll_register_callbacks cb 0x55e503d5b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:10:22.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff48d1dd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503f34110 0x55e503fd16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff48d1dd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503f34110 0x55e503fd16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35952/0 (socket says 172.21.15.19:35952) 2024-05-09T02:10:22.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff48d1dd700 1 -- 172.21.15.19:0/455652022 learned_addr learned my addr 172.21.15.19:0/455652022 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:22.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff48e1df700 1 --2- 172.21.15.19:0/455652022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503fc7650 0x55e503fd1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff48d9de700 1 --2- 172.21.15.19:0/455652022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503e4a450 0x55e503fd11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff48d1dd700 1 -- 172.21.15.19:0/455652022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503fc7650 msgr2=0x55e503fd1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:22.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff48d1dd700 1 --2- 172.21.15.19:0/455652022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503fc7650 0x55e503fd1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff48d1dd700 1 -- 172.21.15.19:0/455652022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503e4a450 msgr2=0x55e503fd11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:22.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff48d1dd700 1 --2- 172.21.15.19:0/455652022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503e4a450 0x55e503fd11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff48d1dd700 1 -- 172.21.15.19:0/455652022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e503d65440 con 0x55e503f34110 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff48d9de700 1 --2- 172.21.15.19:0/455652022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503e4a450 0x55e503fd11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff48d1dd700 1 --2- 172.21.15.19:0/455652022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503f34110 0x55e503fd16f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff47801f850 tx=0x7ff47801f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16af54c3470ee3ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.854+0000 7ff4717fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.855+0000 7ff4717fa700 1 -- 172.21.15.19:0/455652022 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478003680 con 0x55e503f34110 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.855+0000 7ff4717fa700 1 -- 172.21.15.19:0/455652022 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7ff478003820 con 0x55e503f34110 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.855+0000 7ff49e658b80 1 -- 172.21.15.19:0/455652022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff478024070 con 0x55e503f34110 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.855+0000 7ff4717fa700 1 -- 172.21.15.19:0/455652022 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff478003a20 con 0x55e503f34110 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.855+0000 7ff49e658b80 10 client.15042 fetch_fsmap learned FSMap version 28 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.855+0000 7ff49e658b80 1 -- 172.21.15.19:0/455652022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e503fdec20 con 0x55e503f34110 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.855+0000 7ff4717fa700 1 -- 172.21.15.19:0/455652022 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff478024070 con 0x55e503f34110 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.855+0000 7ff4717fa700 1 -- 172.21.15.19:0/455652022 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff478032070 con 0x55e503f34110 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.855+0000 7ff49e658b80 10 client.15042 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.855+0000 7ff49e658b80 10 client.15042 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.855+0000 7ff49e658b80 20 client.15042 populate_metadata read hostname 'smithi019' 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.855+0000 7ff49e658b80 1 --2- 172.21.15.19:0/455652022 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e503fe0f90 0x55e503fe2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.855+0000 7ff49e658b80 4 client.15042 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:10:22.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.855+0000 7ff49e658b80 1 -- 172.21.15.19:0/455652022 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e503ef7980 con 0x55e503fe0f90 2024-05-09T02:10:22.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.856+0000 7ff48d9de700 1 --2- 172.21.15.19:0/455652022 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e503fe0f90 0x55e503fe2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.856+0000 7ff48d9de700 1 --2- 172.21.15.19:0/455652022 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e503fe0f90 0x55e503fe2400 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98c52b3a0e1f9503 server_cookie=d439af9cdd8d8490 in_seq=0 out_seq=0 2024-05-09T02:10:22.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.856+0000 7ff4717fa700 10 client.15042 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:10:22.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.857+0000 7ff4717fa700 1 -- 172.21.15.19:0/455652022 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e503ef7980 con 0x55e503fe0f90 2024-05-09T02:10:22.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.857+0000 7ff4717fa700 10 client.15042 handle_command_reply: tid=0 2024-05-09T02:10:22.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.857+0000 7ff49e658b80 1 client.15042 shutdown 2024-05-09T02:10:22.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.857+0000 7ff49e658b80 1 -- 172.21.15.19:0/455652022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503f34110 msgr2=0x55e503fd16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:22.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.857+0000 7ff49e658b80 1 --2- 172.21.15.19:0/455652022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503f34110 0x55e503fd16f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff47801f850 tx=0x7ff47801f690 comp rx=0 tx=0).stop 2024-05-09T02:10:22.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.857+0000 7ff49e658b80 1 -- 172.21.15.19:0/455652022 shutdown_connections 2024-05-09T02:10:22.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.857+0000 7ff49e658b80 1 --2- 172.21.15.19:0/455652022 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e503fe0f90 0x55e503fe2400 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.857+0000 7ff49e658b80 1 --2- 172.21.15.19:0/455652022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503fc7650 0x55e503fd1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.857+0000 7ff49e658b80 1 --2- 172.21.15.19:0/455652022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503e4a450 0x55e503fd11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.857+0000 7ff49e658b80 1 --2- 172.21.15.19:0/455652022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503f34110 0x55e503fd16f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.857+0000 7ff4717fa700 0 client.15042 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:10:22.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.857+0000 7ff49e658b80 1 -- 172.21.15.19:0/455652022 >> 172.21.15.19:0/455652022 conn(0x55e503eaa650 msgr2=0x55e503fc2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:22.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.857+0000 7ff49e658b80 1 -- 172.21.15.19:0/455652022 shutdown_connections 2024-05-09T02:10:22.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.857+0000 7ff49e658b80 1 -- 172.21.15.19:0/455652022 wait complete. 2024-05-09T02:10:22.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.857+0000 7ff49e658b80 20 client.15042 trim_cache size 0 max 16384 2024-05-09T02:10:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff49e658b80 1 Processor -- start 2024-05-09T02:10:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff49e658b80 1 -- start start 2024-05-09T02:10:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff49e658b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fcc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff49e658b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503f34110 0x55e503fcc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff49e658b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503fc7650 0x55e503fdc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff49e658b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e503fe0740 con 0x55e503e4a450 2024-05-09T02:10:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff49e658b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e503fe0e10 con 0x55e503f34110 2024-05-09T02:10:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff49e658b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e503fe14e0 con 0x55e503fc7650 2024-05-09T02:10:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff48d9de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fcc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff48d9de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fcc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35964/0 (socket says 172.21.15.19:35964) 2024-05-09T02:10:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff48d9de700 1 -- 172.21.15.19:0/2296216618 learned_addr learned my addr 172.21.15.19:0/2296216618 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff48d9de700 1 -- 172.21.15.19:0/2296216618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503fc7650 msgr2=0x55e503fdc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff48d9de700 1 --2- 172.21.15.19:0/2296216618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503fc7650 0x55e503fdc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff48d9de700 1 -- 172.21.15.19:0/2296216618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503f34110 msgr2=0x55e503fcc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff48d9de700 1 --2- 172.21.15.19:0/2296216618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503f34110 0x55e503fcc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff48d9de700 1 -- 172.21.15.19:0/2296216618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e503fd4110 con 0x55e503e4a450 2024-05-09T02:10:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.879+0000 7ff48d9de700 1 --2- 172.21.15.19:0/2296216618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fcc2a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff48400bc40 tx=0x7ff4840108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40df663d11e60dc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.880+0000 7ff4737fe700 1 -- 172.21.15.19:0/2296216618 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4840a90e0 con 0x55e503e4a450 2024-05-09T02:10:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.880+0000 7ff4737fe700 1 -- 172.21.15.19:0/2296216618 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4840ad490 con 0x55e503e4a450 2024-05-09T02:10:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.880+0000 7ff4737fe700 1 -- 172.21.15.19:0/2296216618 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4840b1ab0 con 0x55e503e4a450 2024-05-09T02:10:22.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.880+0000 7ff49e658b80 1 -- 172.21.15.19:0/2296216618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 msgr2=0x55e503fcc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:22.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.880+0000 7ff49e658b80 1 --2- 172.21.15.19:0/2296216618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fcc2a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff48400bc40 tx=0x7ff4840108e0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.880+0000 7ff49e658b80 1 -- 172.21.15.19:0/2296216618 shutdown_connections 2024-05-09T02:10:22.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.880+0000 7ff49e658b80 1 --2- 172.21.15.19:0/2296216618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503fc7650 0x55e503fdc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.880+0000 7ff49e658b80 1 --2- 172.21.15.19:0/2296216618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fcc2a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.880+0000 7ff49e658b80 1 --2- 172.21.15.19:0/2296216618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503f34110 0x55e503fcc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.880+0000 7ff49e658b80 1 -- 172.21.15.19:0/2296216618 >> 172.21.15.19:0/2296216618 conn(0x55e503eaa650 msgr2=0x55e503fcfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:22.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.881+0000 7ff49e658b80 1 -- 172.21.15.19:0/2296216618 shutdown_connections 2024-05-09T02:10:22.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.881+0000 7ff49e658b80 1 -- 172.21.15.19:0/2296216618 wait complete. 2024-05-09T02:10:22.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.881+0000 7ff49e658b80 1 -- start start 2024-05-09T02:10:22.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.881+0000 7ff49e658b80 1 Processor -- start 2024-05-09T02:10:22.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.881+0000 7ff49e658b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.881+0000 7ff49e658b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503f34110 0x55e503fcec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.881+0000 7ff49e658b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503fc7650 0x55e503fd2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff49e658b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e503fe2280 con 0x55e503e4a450 2024-05-09T02:10:22.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff49e658b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e503fe1bb0 con 0x55e503fc7650 2024-05-09T02:10:22.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff49e658b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e503fe2950 con 0x55e503f34110 2024-05-09T02:10:22.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff49e658b80 10 client.0 _ll_register_callbacks cb 0x55e503d5b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:10:22.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff48d9de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff48d9de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35966/0 (socket says 172.21.15.19:35966) 2024-05-09T02:10:22.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff48d9de700 1 -- 172.21.15.19:0/299358579 learned_addr learned my addr 172.21.15.19:0/299358579 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:22.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff48d1dd700 1 --2- 172.21.15.19:0/299358579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503f34110 0x55e503fcec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff48e1df700 1 --2- 172.21.15.19:0/299358579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503fc7650 0x55e503fd2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff48d9de700 1 -- 172.21.15.19:0/299358579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503f34110 msgr2=0x55e503fcec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:22.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff48d9de700 1 --2- 172.21.15.19:0/299358579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503f34110 0x55e503fcec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff48d9de700 1 -- 172.21.15.19:0/299358579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503fc7650 msgr2=0x55e503fd2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:22.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff48d9de700 1 --2- 172.21.15.19:0/299358579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503fc7650 0x55e503fd2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff48d9de700 1 -- 172.21.15.19:0/299358579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e503fd2c90 con 0x55e503e4a450 2024-05-09T02:10:22.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff48d9de700 1 --2- 172.21.15.19:0/299358579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fce750 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff4840df590 tx=0x7ff4840b1fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4347deee39db8668 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:22.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff4717fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:10:22.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff4717fa700 1 -- 172.21.15.19:0/299358579 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4840b1ab0 con 0x55e503e4a450 2024-05-09T02:10:22.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff4717fa700 1 -- 172.21.15.19:0/299358579 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7ff4840b1c50 con 0x55e503e4a450 2024-05-09T02:10:22.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.882+0000 7ff49e658b80 1 -- 172.21.15.19:0/299358579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4840ad490 con 0x55e503e4a450 2024-05-09T02:10:22.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.883+0000 7ff4717fa700 1 -- 172.21.15.19:0/299358579 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff484011830 con 0x55e503e4a450 2024-05-09T02:10:22.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.883+0000 7ff49e658b80 10 client.15048 fetch_fsmap learned FSMap version 28 2024-05-09T02:10:22.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.883+0000 7ff49e658b80 1 -- 172.21.15.19:0/299358579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e50404cb60 con 0x55e503e4a450 2024-05-09T02:10:22.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.883+0000 7ff4717fa700 1 -- 172.21.15.19:0/299358579 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4840ad490 con 0x55e503e4a450 2024-05-09T02:10:22.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.883+0000 7ff4717fa700 1 -- 172.21.15.19:0/299358579 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff4840a7db0 con 0x55e503e4a450 2024-05-09T02:10:22.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.883+0000 7ff49e658b80 10 client.15048 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:10:22.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.883+0000 7ff49e658b80 10 client.15048 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T02:10:22.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.883+0000 7ff49e658b80 20 client.15048 populate_metadata read hostname 'smithi019' 2024-05-09T02:10:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.883+0000 7ff49e658b80 1 --2- 172.21.15.19:0/299358579 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e50404e370 0x55e50404fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.883+0000 7ff49e658b80 4 client.15048 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T02:10:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.883+0000 7ff49e658b80 1 -- 172.21.15.19:0/299358579 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "status"}) -- 0x55e503ef1a20 con 0x55e50404e370 2024-05-09T02:10:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.883+0000 7ff48d1dd700 1 --2- 172.21.15.19:0/299358579 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e50404e370 0x55e50404fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.884+0000 7ff48d1dd700 1 --2- 172.21.15.19:0/299358579 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e50404e370 0x55e50404fe90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8b8b2a168d01872 server_cookie=d410851440e35f71 in_seq=0 out_seq=0 2024-05-09T02:10:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.884+0000 7ff4717fa700 10 client.15048 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:10:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.884+0000 7ff4717fa700 1 -- 172.21.15.19:0/299358579 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e503ef1a20 con 0x55e50404e370 2024-05-09T02:10:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.884+0000 7ff4717fa700 10 client.15048 handle_command_reply: tid=0 2024-05-09T02:10:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.884+0000 7ff49e658b80 1 client.15048 shutdown 2024-05-09T02:10:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.884+0000 7ff49e658b80 1 -- 172.21.15.19:0/299358579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 msgr2=0x55e503fce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.884+0000 7ff49e658b80 1 --2- 172.21.15.19:0/299358579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fce750 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff4840df590 tx=0x7ff4840b1fc0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.884+0000 7ff49e658b80 1 -- 172.21.15.19:0/299358579 shutdown_connections 2024-05-09T02:10:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.884+0000 7ff49e658b80 1 --2- 172.21.15.19:0/299358579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e503fc7650 0x55e503fd2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.884+0000 7ff49e658b80 1 --2- 172.21.15.19:0/299358579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e503e4a450 0x55e503fce750 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.884+0000 7ff49e658b80 1 --2- 172.21.15.19:0/299358579 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e50404e370 0x55e50404fe90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.884+0000 7ff49e658b80 1 --2- 172.21.15.19:0/299358579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e503f34110 0x55e503fcec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.884+0000 7ff4717fa700 0 client.15048 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:10:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.884+0000 7ff49e658b80 1 -- 172.21.15.19:0/299358579 >> 172.21.15.19:0/299358579 conn(0x55e503eaa650 msgr2=0x55e503fdc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.884+0000 7ff49e658b80 1 -- 172.21.15.19:0/299358579 shutdown_connections 2024-05-09T02:10:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.885+0000 7ff49e658b80 1 -- 172.21.15.19:0/299358579 wait complete. 2024-05-09T02:10:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.885+0000 7ff49e658b80 20 client.15048 trim_cache size 0 max 16384 2024-05-09T02:10:22.891 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:10:22.891 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "a0611f3e-0da7-11ef-bc97-c7b262605968", 2024-05-09T02:10:22.891 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-05-09T02:10:22.891 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24484, 2024-05-09T02:10:22.892 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T02:10:22.892 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T02:10:22.892 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T02:10:22.892 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 151.69186392399999, 2024-05-09T02:10:22.892 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-05-09T02:10:22.892 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 81, 2024-05-09T02:10:22.892 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 79, 2024-05-09T02:10:22.892 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 177.17157161200001 2024-05-09T02:10:22.892 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:10:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.885+0000 7ff48fc42700 1 -- 172.21.15.19:0/1760803537 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff4740b2880 msgr2=0x7ff4740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.885+0000 7ff48fc42700 1 --2- 172.21.15.19:0/1760803537 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff4740b2880 0x7ff4740b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff478005950 tx=0x7ff4780058e0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.885+0000 7ff48fc42700 1 -- 172.21.15.19:0/1760803537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4881045d0 msgr2=0x7ff48819ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.885+0000 7ff48fc42700 1 --2- 172.21.15.19:0/1760803537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4881045d0 0x7ff48819ae90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff48400b530 tx=0x7ff48400b840 comp rx=0 tx=0).stop 2024-05-09T02:10:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.885+0000 7ff48fc42700 1 -- 172.21.15.19:0/1760803537 shutdown_connections 2024-05-09T02:10:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.885+0000 7ff48fc42700 1 --2- 172.21.15.19:0/1760803537 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff4740b2880 0x7ff4740b4d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.885+0000 7ff48fc42700 1 --2- 172.21.15.19:0/1760803537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff488109850 0x7ff48819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.885+0000 7ff48fc42700 1 --2- 172.21.15.19:0/1760803537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4881045d0 0x7ff48819ae90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.885+0000 7ff48fc42700 1 --2- 172.21.15.19:0/1760803537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff488108f00 0x7ff48819b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.885+0000 7ff48fc42700 1 -- 172.21.15.19:0/1760803537 >> 172.21.15.19:0/1760803537 conn(0x7ff4880ffea0 msgr2=0x7ff488108640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.885+0000 7ff48fc42700 1 -- 172.21.15.19:0/1760803537 shutdown_connections 2024-05-09T02:10:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:22.885+0000 7ff48fc42700 1 -- 172.21.15.19:0/1760803537 wait complete. 2024-05-09T02:10:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:23 smithi059 ceph-mon[34781]: pgmap v370: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:23.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:23 smithi196 ceph-mon[34718]: pgmap v370: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:23.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:23 smithi019 ceph-mon[30151]: pgmap v370: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:23.706 INFO:tasks.cephadm:Setup complete, yielding 2024-05-09T02:10:23.706 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T02:10:23.716 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T02:10:23.716 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-09T02:10:23.974 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:25.445 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:25 smithi019 ceph-mon[30151]: pgmap v371: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:10:25.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:25 smithi059 ceph-mon[34781]: pgmap v371: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:10:25.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:25 smithi196 ceph-mon[34718]: pgmap v371: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:10:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.688+0000 7f488370e700 1 -- 172.21.15.19:0/3288817484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f487c104d10 msgr2=0x7f487c10b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.688+0000 7f488370e700 1 --2- 172.21.15.19:0/3288817484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f487c104d10 0x7f487c10b7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f487400b0a0 tx=0x7f487400b3b0 comp rx=0 tx=0).stop 2024-05-09T02:10:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.688+0000 7f488370e700 1 -- 172.21.15.19:0/3288817484 shutdown_connections 2024-05-09T02:10:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.688+0000 7f488370e700 1 --2- 172.21.15.19:0/3288817484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f487c104d10 0x7f487c10b7a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.688+0000 7f488370e700 1 --2- 172.21.15.19:0/3288817484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c1037d0 0x7f487c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.688+0000 7f488370e700 1 --2- 172.21.15.19:0/3288817484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f487c1041b0 0x7f487c104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.688+0000 7f488370e700 1 -- 172.21.15.19:0/3288817484 >> 172.21.15.19:0/3288817484 conn(0x7f487c0fe020 msgr2=0x7f487c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.688+0000 7f488370e700 1 -- 172.21.15.19:0/3288817484 shutdown_connections 2024-05-09T02:10:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.688+0000 7f488370e700 1 -- 172.21.15.19:0/3288817484 wait complete. 2024-05-09T02:10:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.689+0000 7f488370e700 1 Processor -- start 2024-05-09T02:10:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.689+0000 7f488370e700 1 -- start start 2024-05-09T02:10:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.689+0000 7f488370e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f487c1037d0 0x7f487c19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:25.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.689+0000 7f488370e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f487c1041b0 0x7f487c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:25.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.689+0000 7f488370e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c104d10 0x7f487c1a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:25.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.689+0000 7f488370e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f487c10fde0 con 0x7f487c1041b0 2024-05-09T02:10:25.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.689+0000 7f488370e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f487c10fc60 con 0x7f487c104d10 2024-05-09T02:10:25.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.689+0000 7f488370e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f487c10ff60 con 0x7f487c1037d0 2024-05-09T02:10:25.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.689+0000 7f4880ca9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f487c1041b0 0x7f487c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:25.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.690+0000 7f4880ca9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f487c1041b0 0x7f487c19ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35978/0 (socket says 172.21.15.19:35978) 2024-05-09T02:10:25.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.690+0000 7f4880ca9700 1 -- 172.21.15.19:0/273305729 learned_addr learned my addr 172.21.15.19:0/273305729 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:25.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.690+0000 7f48814aa700 1 --2- 172.21.15.19:0/273305729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f487c1037d0 0x7f487c19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:25.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.690+0000 7f4881cab700 1 --2- 172.21.15.19:0/273305729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c104d10 0x7f487c1a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:25.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.690+0000 7f4880ca9700 1 -- 172.21.15.19:0/273305729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f487c1037d0 msgr2=0x7f487c19a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:25.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.690+0000 7f4880ca9700 1 --2- 172.21.15.19:0/273305729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f487c1037d0 0x7f487c19a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:25.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.690+0000 7f4880ca9700 1 -- 172.21.15.19:0/273305729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c104d10 msgr2=0x7f487c1a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:25.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.690+0000 7f4880ca9700 1 --2- 172.21.15.19:0/273305729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c104d10 0x7f487c1a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:25.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.690+0000 7f4880ca9700 1 -- 172.21.15.19:0/273305729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487c1a23a0 con 0x7f487c1041b0 2024-05-09T02:10:25.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.690+0000 7f4880ca9700 1 --2- 172.21.15.19:0/273305729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f487c1041b0 0x7f487c19ad80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f486c00d950 tx=0x7f486c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:25.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.690+0000 7f48727fc700 1 -- 172.21.15.19:0/273305729 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c016070 con 0x7f487c1041b0 2024-05-09T02:10:25.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.690+0000 7f48727fc700 1 -- 172.21.15.19:0/273305729 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f486c00fd40 con 0x7f487c1041b0 2024-05-09T02:10:25.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.690+0000 7f48727fc700 1 -- 172.21.15.19:0/273305729 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c00e3b0 con 0x7f487c1041b0 2024-05-09T02:10:25.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.690+0000 7f488370e700 1 -- 172.21.15.19:0/273305729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487c1a2690 con 0x7f487c1041b0 2024-05-09T02:10:25.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.690+0000 7f488370e700 1 -- 172.21.15.19:0/273305729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487c1a2c20 con 0x7f487c1041b0 2024-05-09T02:10:25.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.691+0000 7f488370e700 1 -- 172.21.15.19:0/273305729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f487c0512c0 con 0x7f487c1041b0 2024-05-09T02:10:25.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.693+0000 7f48727fc700 1 -- 172.21.15.19:0/273305729 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f486c019070 con 0x7f487c1041b0 2024-05-09T02:10:25.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.694+0000 7f48727fc700 1 --2- 172.21.15.19:0/273305729 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f48680b2900 0x7f48680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:25.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.694+0000 7f48814aa700 1 --2- 172.21.15.19:0/273305729 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f48680b2900 0x7f48680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:25.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.694+0000 7f48727fc700 1 -- 172.21.15.19:0/273305729 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f486c0e1bf0 con 0x7f487c1041b0 2024-05-09T02:10:25.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.695+0000 7f48814aa700 1 --2- 172.21.15.19:0/273305729 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f48680b2900 0x7f48680b4e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f487c06c0c0 tx=0x7f48780073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:25.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.698+0000 7f48727fc700 1 -- 172.21.15.19:0/273305729 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f486c0a9890 con 0x7f487c1041b0 2024-05-09T02:10:25.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.928+0000 7f488370e700 1 -- 172.21.15.19:0/273305729 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f487c063c00 con 0x7f48680b2900 2024-05-09T02:10:25.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.929+0000 7f48727fc700 1 -- 172.21.15.19:0/273305729 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f487c063c00 con 0x7f48680b2900 2024-05-09T02:10:25.931 INFO:teuthology.orchestra.run.smithi019.stdout:Backend: cephadm 2024-05-09T02:10:25.932 INFO:teuthology.orchestra.run.smithi019.stdout:Available: Yes 2024-05-09T02:10:25.932 INFO:teuthology.orchestra.run.smithi019.stdout:Paused: No 2024-05-09T02:10:25.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.931+0000 7f488370e700 1 -- 172.21.15.19:0/273305729 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f48680b2900 msgr2=0x7f48680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:25.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.931+0000 7f488370e700 1 --2- 172.21.15.19:0/273305729 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f48680b2900 0x7f48680b4e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f487c06c0c0 tx=0x7f48780073d0 comp rx=0 tx=0).stop 2024-05-09T02:10:25.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.931+0000 7f488370e700 1 -- 172.21.15.19:0/273305729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f487c1041b0 msgr2=0x7f487c19ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:25.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.931+0000 7f488370e700 1 --2- 172.21.15.19:0/273305729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f487c1041b0 0x7f487c19ad80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f486c00d950 tx=0x7f486c00dc60 comp rx=0 tx=0).stop 2024-05-09T02:10:25.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.931+0000 7f488370e700 1 -- 172.21.15.19:0/273305729 shutdown_connections 2024-05-09T02:10:25.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.931+0000 7f488370e700 1 --2- 172.21.15.19:0/273305729 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f48680b2900 0x7f48680b4e10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:25.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.931+0000 7f488370e700 1 --2- 172.21.15.19:0/273305729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c104d10 0x7f487c1a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:25.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.931+0000 7f488370e700 1 --2- 172.21.15.19:0/273305729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f487c1037d0 0x7f487c19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:25.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.931+0000 7f488370e700 1 --2- 172.21.15.19:0/273305729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f487c1041b0 0x7f487c19ad80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:25.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.931+0000 7f488370e700 1 -- 172.21.15.19:0/273305729 >> 172.21.15.19:0/273305729 conn(0x7f487c0fe020 msgr2=0x7f487c107780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:25.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.932+0000 7f488370e700 1 -- 172.21.15.19:0/273305729 shutdown_connections 2024-05-09T02:10:25.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:25.932+0000 7f488370e700 1 -- 172.21.15.19:0/273305729 wait complete. 2024-05-09T02:10:26.769 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-09T02:10:27.029 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:27.437 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:27 smithi019 ceph-mon[30151]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:10:27.438 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:27 smithi019 ceph-mon[30151]: pgmap v372: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:10:27.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:27 smithi059 ceph-mon[34781]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:10:27.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:27 smithi059 ceph-mon[34781]: pgmap v372: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:27 smithi196 ceph-mon[34718]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:10:27.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:27 smithi196 ceph-mon[34718]: pgmap v372: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:10:28.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.863+0000 7f54e11da700 1 -- 172.21.15.19:0/2115016822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54dc104d10 msgr2=0x7f54dc10b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.863+0000 7f54e11da700 1 --2- 172.21.15.19:0/2115016822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54dc104d10 0x7f54dc10b5c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f54d400b0a0 tx=0x7f54d400b3b0 comp rx=0 tx=0).stop 2024-05-09T02:10:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.863+0000 7f54e11da700 1 -- 172.21.15.19:0/2115016822 shutdown_connections 2024-05-09T02:10:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.863+0000 7f54e11da700 1 --2- 172.21.15.19:0/2115016822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54dc104d10 0x7f54dc10b5c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.863+0000 7f54e11da700 1 --2- 172.21.15.19:0/2115016822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54dc103240 0x7f54dc103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.863+0000 7f54e11da700 1 --2- 172.21.15.19:0/2115016822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54dc104370 0x7f54dc104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.863+0000 7f54e11da700 1 -- 172.21.15.19:0/2115016822 >> 172.21.15.19:0/2115016822 conn(0x7f54dc0fe880 msgr2=0x7f54dc100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.863+0000 7f54e11da700 1 -- 172.21.15.19:0/2115016822 shutdown_connections 2024-05-09T02:10:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.863+0000 7f54e11da700 1 -- 172.21.15.19:0/2115016822 wait complete. 2024-05-09T02:10:28.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.864+0000 7f54e11da700 1 Processor -- start 2024-05-09T02:10:28.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.864+0000 7f54e11da700 1 -- start start 2024-05-09T02:10:28.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.864+0000 7f54e11da700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54dc103240 0x7f54dc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:28.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.864+0000 7f54e11da700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54dc104370 0x7f54dc19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:28.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.864+0000 7f54e11da700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54dc104d10 0x7f54dc1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:28.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.864+0000 7f54e11da700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f54dc10ed40 con 0x7f54dc104370 2024-05-09T02:10:28.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.864+0000 7f54e11da700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f54dc10ebc0 con 0x7f54dc104d10 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.864+0000 7f54e11da700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54dc10eec0 con 0x7f54dc103240 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.864+0000 7f54da59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54dc104370 0x7f54dc19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.864+0000 7f54dad9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54dc103240 0x7f54dc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.864+0000 7f54da59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54dc104370 0x7f54dc19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56936/0 (socket says 172.21.15.19:56936) 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.864+0000 7f54da59c700 1 -- 172.21.15.19:0/1935195032 learned_addr learned my addr 172.21.15.19:0/1935195032 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.864+0000 7f54db59e700 1 --2- 172.21.15.19:0/1935195032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54dc104d10 0x7f54dc1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.865+0000 7f54da59c700 1 -- 172.21.15.19:0/1935195032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54dc103240 msgr2=0x7f54dc06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.865+0000 7f54da59c700 1 --2- 172.21.15.19:0/1935195032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54dc103240 0x7f54dc06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.865+0000 7f54da59c700 1 -- 172.21.15.19:0/1935195032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54dc104d10 msgr2=0x7f54dc1a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.865+0000 7f54da59c700 1 --2- 172.21.15.19:0/1935195032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54dc104d10 0x7f54dc1a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.865+0000 7f54da59c700 1 -- 172.21.15.19:0/1935195032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54dc1a2480 con 0x7f54dc104370 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.865+0000 7f54da59c700 1 --2- 172.21.15.19:0/1935195032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54dc104370 0x7f54dc19f6c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f54d000b530 tx=0x7f54d000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.865+0000 7f54c3fff700 1 -- 172.21.15.19:0/1935195032 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0015020 con 0x7f54dc104370 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.865+0000 7f54c3fff700 1 -- 172.21.15.19:0/1935195032 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d0011ce0 con 0x7f54dc104370 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.865+0000 7f54e11da700 1 -- 172.21.15.19:0/1935195032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54dc1a2770 con 0x7f54dc104370 2024-05-09T02:10:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.865+0000 7f54c3fff700 1 -- 172.21.15.19:0/1935195032 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0010dd0 con 0x7f54dc104370 2024-05-09T02:10:28.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.865+0000 7f54e11da700 1 -- 172.21.15.19:0/1935195032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54dc1a2d30 con 0x7f54dc104370 2024-05-09T02:10:28.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.866+0000 7f54e11da700 1 -- 172.21.15.19:0/1935195032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54dc0512c0 con 0x7f54dc104370 2024-05-09T02:10:28.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.868+0000 7f54c3fff700 1 -- 172.21.15.19:0/1935195032 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f54d0018020 con 0x7f54dc104370 2024-05-09T02:10:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.868+0000 7f54c3fff700 1 --2- 172.21.15.19:0/1935195032 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f54c40b2880 0x7f54c40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.868+0000 7f54dad9d700 1 --2- 172.21.15.19:0/1935195032 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f54c40b2880 0x7f54c40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.869+0000 7f54c3fff700 1 -- 172.21.15.19:0/1935195032 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f54d00e18f0 con 0x7f54dc104370 2024-05-09T02:10:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.869+0000 7f54dad9d700 1 --2- 172.21.15.19:0/1935195032 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f54c40b2880 0x7f54c40b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f54dc104270 tx=0x7f54cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:28.873+0000 7f54c3fff700 1 -- 172.21.15.19:0/1935195032 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f54d001f020 con 0x7f54dc104370 2024-05-09T02:10:29.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:29.095+0000 7f54e11da700 1 -- 172.21.15.19:0/1935195032 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f54dc063c00 con 0x7f54c40b2880 2024-05-09T02:10:29.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:29.109+0000 7f54c3fff700 1 -- 172.21.15.19:0/1935195032 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f54dc063c00 con 0x7f54c40b2880 2024-05-09T02:10:29.112 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T02:10:29.112 INFO:teuthology.orchestra.run.smithi019.stdout:mds.a smithi019 running (2m) 35s ago 2m 86.2M - 19.0.0-3594-gc9da4049 db794d706f49 3dc436e93cf4 2024-05-09T02:10:29.112 INFO:teuthology.orchestra.run.smithi019.stdout:mds.b smithi059 running (2m) 37s ago 2m 87.2M - 19.0.0-3594-gc9da4049 db794d706f49 de12d771cb43 2024-05-09T02:10:29.112 INFO:teuthology.orchestra.run.smithi019.stdout:mds.c smithi196 running (3m) 37s ago 3m 87.1M - 19.0.0-3594-gc9da4049 db794d706f49 1b60ffd11ad0 2024-05-09T02:10:29.112 INFO:teuthology.orchestra.run.smithi019.stdout:mds.d smithi019 running (3m) 35s ago 3m 82.0M - 19.0.0-3594-gc9da4049 db794d706f49 40e01ca93a14 2024-05-09T02:10:29.112 INFO:teuthology.orchestra.run.smithi019.stdout:mds.e smithi059 running (3m) 37s ago 3m 90.5M - 19.0.0-3594-gc9da4049 db794d706f49 f355fbdce2f6 2024-05-09T02:10:29.112 INFO:teuthology.orchestra.run.smithi019.stdout:mds.f smithi196 running (3m) 37s ago 3m 87.0M - 19.0.0-3594-gc9da4049 db794d706f49 c43f3f422142 2024-05-09T02:10:29.112 INFO:teuthology.orchestra.run.smithi019.stdout:mds.g smithi019 running (3m) 35s ago 3m 91.5M - 19.0.0-3594-gc9da4049 db794d706f49 39eab8a6e571 2024-05-09T02:10:29.112 INFO:teuthology.orchestra.run.smithi019.stdout:mds.h smithi059 running (3m) 37s ago 3m 89.0M - 19.0.0-3594-gc9da4049 db794d706f49 3c406c8ba9fd 2024-05-09T02:10:29.113 INFO:teuthology.orchestra.run.smithi019.stdout:mds.i smithi196 running (3m) 37s ago 3m 93.7M - 19.0.0-3594-gc9da4049 db794d706f49 6e8623f5a409 2024-05-09T02:10:29.113 INFO:teuthology.orchestra.run.smithi019.stdout:mds.j smithi019 running (3m) 35s ago 3m 81.9M - 19.0.0-3594-gc9da4049 db794d706f49 49737d7ffb42 2024-05-09T02:10:29.113 INFO:teuthology.orchestra.run.smithi019.stdout:mds.k smithi059 running (3m) 37s ago 3m 91.3M - 19.0.0-3594-gc9da4049 db794d706f49 23ac5e3c1872 2024-05-09T02:10:29.113 INFO:teuthology.orchestra.run.smithi019.stdout:mds.l smithi196 running (3m) 37s ago 3m 85.9M - 19.0.0-3594-gc9da4049 db794d706f49 c0f206047bfa 2024-05-09T02:10:29.113 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.x smithi019 *:9283,8765 running (10m) 35s ago 10m 598M - 19.0.0-3594-gc9da4049 db794d706f49 fb9ba98e5788 2024-05-09T02:10:29.113 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.y smithi059 *:8443,8765 running (8m) 37s ago 8m 537M - 19.0.0-3594-gc9da4049 db794d706f49 f0c89ea92ac1 2024-05-09T02:10:29.113 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.z smithi196 *:8443,8765 running (8m) 37s ago 8m 537M - 19.0.0-3594-gc9da4049 db794d706f49 8fab2740a03d 2024-05-09T02:10:29.113 INFO:teuthology.orchestra.run.smithi019.stdout:mon.a smithi019 running (10m) 35s ago 11m 121M 2048M 19.0.0-3594-gc9da4049 db794d706f49 4d8bf39103f3 2024-05-09T02:10:29.113 INFO:teuthology.orchestra.run.smithi019.stdout:mon.b smithi059 running (9m) 37s ago 9m 110M 2048M 19.0.0-3594-gc9da4049 db794d706f49 e28aece55a38 2024-05-09T02:10:29.113 INFO:teuthology.orchestra.run.smithi019.stdout:mon.c smithi196 running (9m) 37s ago 9m 106M 2048M 19.0.0-3594-gc9da4049 db794d706f49 228fb8b25695 2024-05-09T02:10:29.113 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (8m) 35s ago 8m 126M 4273M 19.0.0-3594-gc9da4049 db794d706f49 e195ade32bd5 2024-05-09T02:10:29.113 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi059 running (8m) 37s ago 8m 122M 4273M 19.0.0-3594-gc9da4049 db794d706f49 59eb3914594b 2024-05-09T02:10:29.113 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi196 running (7m) 37s ago 7m 124M 4272M 19.0.0-3594-gc9da4049 db794d706f49 d874e54741ad 2024-05-09T02:10:29.113 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi019 running (7m) 35s ago 7m 124M 4273M 19.0.0-3594-gc9da4049 db794d706f49 fbc073cc6829 2024-05-09T02:10:29.114 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi059 running (6m) 37s ago 6m 124M 4273M 19.0.0-3594-gc9da4049 db794d706f49 7311b3fc9fb2 2024-05-09T02:10:29.114 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi196 running (6m) 37s ago 6m 126M 4272M 19.0.0-3594-gc9da4049 db794d706f49 dc8ebbcaf939 2024-05-09T02:10:29.114 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 smithi019 running (6m) 35s ago 6m 112M 4273M 19.0.0-3594-gc9da4049 db794d706f49 03802a5580d4 2024-05-09T02:10:29.114 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 smithi059 running (5m) 37s ago 5m 123M 4273M 19.0.0-3594-gc9da4049 db794d706f49 1f63947673b0 2024-05-09T02:10:29.114 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 smithi196 running (5m) 37s ago 5m 111M 4272M 19.0.0-3594-gc9da4049 db794d706f49 2352e114b9ce 2024-05-09T02:10:29.114 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 smithi019 running (4m) 35s ago 4m 118M 4273M 19.0.0-3594-gc9da4049 db794d706f49 47857d55cbe0 2024-05-09T02:10:29.114 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 smithi059 running (4m) 37s ago 4m 113M 4273M 19.0.0-3594-gc9da4049 db794d706f49 7f08b240316d 2024-05-09T02:10:29.114 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 smithi196 running (3m) 37s ago 3m 119M 4272M 19.0.0-3594-gc9da4049 db794d706f49 d91b37888210 2024-05-09T02:10:29.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:29.111+0000 7f54e11da700 1 -- 172.21.15.19:0/1935195032 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f54c40b2880 msgr2=0x7f54c40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:29.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:29.111+0000 7f54e11da700 1 --2- 172.21.15.19:0/1935195032 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f54c40b2880 0x7f54c40b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f54dc104270 tx=0x7f54cc00b410 comp rx=0 tx=0).stop 2024-05-09T02:10:29.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:29.111+0000 7f54e11da700 1 -- 172.21.15.19:0/1935195032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54dc104370 msgr2=0x7f54dc19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:29.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:29.111+0000 7f54e11da700 1 --2- 172.21.15.19:0/1935195032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54dc104370 0x7f54dc19f6c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f54d000b530 tx=0x7f54d000b840 comp rx=0 tx=0).stop 2024-05-09T02:10:29.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:29.112+0000 7f54e11da700 1 -- 172.21.15.19:0/1935195032 shutdown_connections 2024-05-09T02:10:29.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:29.112+0000 7f54e11da700 1 --2- 172.21.15.19:0/1935195032 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f54c40b2880 0x7f54c40b4d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:29.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:29.112+0000 7f54e11da700 1 --2- 172.21.15.19:0/1935195032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54dc104d10 0x7f54dc1a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:29.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:29.112+0000 7f54e11da700 1 --2- 172.21.15.19:0/1935195032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54dc103240 0x7f54dc06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:29.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:29.112+0000 7f54e11da700 1 --2- 172.21.15.19:0/1935195032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54dc104370 0x7f54dc19f6c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:29.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:29.112+0000 7f54e11da700 1 -- 172.21.15.19:0/1935195032 >> 172.21.15.19:0/1935195032 conn(0x7f54dc0fe880 msgr2=0x7f54dc1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:29.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:29.112+0000 7f54e11da700 1 -- 172.21.15.19:0/1935195032 shutdown_connections 2024-05-09T02:10:29.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:29.112+0000 7f54e11da700 1 -- 172.21.15.19:0/1935195032 wait complete. 2024-05-09T02:10:29.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:29 smithi059 ceph-mon[34781]: pgmap v373: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:29.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:29 smithi019 ceph-mon[30151]: pgmap v373: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:29.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:29 smithi196 ceph-mon[34718]: pgmap v373: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:29.935 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-09T02:10:30.190 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:30.456 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:30 smithi019 ceph-mon[30151]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:10:30.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:30 smithi059 ceph-mon[34781]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:10:30.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:30 smithi196 ceph-mon[34718]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:10:31.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:31 smithi059 ceph-mon[34781]: pgmap v374: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:10:31.506 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:31 smithi019 ceph-mon[30151]: pgmap v374: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:10:31.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:31 smithi196 ceph-mon[34718]: pgmap v374: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:10:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.915+0000 7f1f31eac700 1 -- 172.21.15.19:0/1492503227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f2c105480 msgr2=0x7f1f2c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.915+0000 7f1f31eac700 1 --2- 172.21.15.19:0/1492503227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f2c105480 0x7f1f2c105890 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009990 tx=0x7f1f14009ca0 comp rx=0 tx=0).stop 2024-05-09T02:10:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.915+0000 7f1f31eac700 1 -- 172.21.15.19:0/1492503227 shutdown_connections 2024-05-09T02:10:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.915+0000 7f1f31eac700 1 --2- 172.21.15.19:0/1492503227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f2c105e20 0x7f1f2c10c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.915+0000 7f1f31eac700 1 --2- 172.21.15.19:0/1492503227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2c104350 0x7f1f2c104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.915+0000 7f1f31eac700 1 --2- 172.21.15.19:0/1492503227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f2c105480 0x7f1f2c105890 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.915+0000 7f1f31eac700 1 -- 172.21.15.19:0/1492503227 >> 172.21.15.19:0/1492503227 conn(0x7f1f2c0ff990 msgr2=0x7f1f2c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.915+0000 7f1f31eac700 1 -- 172.21.15.19:0/1492503227 shutdown_connections 2024-05-09T02:10:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.915+0000 7f1f31eac700 1 -- 172.21.15.19:0/1492503227 wait complete. 2024-05-09T02:10:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.916+0000 7f1f31eac700 1 Processor -- start 2024-05-09T02:10:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.916+0000 7f1f31eac700 1 -- start start 2024-05-09T02:10:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.916+0000 7f1f31eac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f2c104350 0x7f1f2c19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.916+0000 7f1f31eac700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2c105480 0x7f1f2c19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.916+0000 7f1f31eac700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f2c105e20 0x7f1f2c1a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.916+0000 7f1f31eac700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c10fdc0 con 0x7f1f2c104350 2024-05-09T02:10:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.916+0000 7f1f31eac700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c10fc40 con 0x7f1f2c105480 2024-05-09T02:10:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.916+0000 7f1f31eac700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c10ff40 con 0x7f1f2c105e20 2024-05-09T02:10:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.916+0000 7f1f2b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f2c104350 0x7f1f2c19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.916+0000 7f1f2b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f2c104350 0x7f1f2c19a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56946/0 (socket says 172.21.15.19:56946) 2024-05-09T02:10:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.916+0000 7f1f2b7fe700 1 -- 172.21.15.19:0/2439514174 learned_addr learned my addr 172.21.15.19:0/2439514174 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.917+0000 7f1f22ffd700 1 --2- 172.21.15.19:0/2439514174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2c105480 0x7f1f2c19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.917+0000 7f1f2bfff700 1 --2- 172.21.15.19:0/2439514174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f2c105e20 0x7f1f2c1a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.917+0000 7f1f2b7fe700 1 -- 172.21.15.19:0/2439514174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f2c105e20 msgr2=0x7f1f2c1a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.917+0000 7f1f2b7fe700 1 --2- 172.21.15.19:0/2439514174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f2c105e20 0x7f1f2c1a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.917+0000 7f1f2b7fe700 1 -- 172.21.15.19:0/2439514174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2c105480 msgr2=0x7f1f2c19ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.917+0000 7f1f2b7fe700 1 --2- 172.21.15.19:0/2439514174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2c105480 0x7f1f2c19ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.917+0000 7f1f2b7fe700 1 -- 172.21.15.19:0/2439514174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2c1a2410 con 0x7f1f2c104350 2024-05-09T02:10:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.917+0000 7f1f22ffd700 1 --2- 172.21.15.19:0/2439514174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2c105480 0x7f1f2c19ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:10:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.917+0000 7f1f2b7fe700 1 --2- 172.21.15.19:0/2439514174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f2c104350 0x7f1f2c19a940 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00d950 tx=0x7f1f1c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.917+0000 7f1f297fa700 1 -- 172.21.15.19:0/2439514174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c017070 con 0x7f1f2c104350 2024-05-09T02:10:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.917+0000 7f1f297fa700 1 -- 172.21.15.19:0/2439514174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f1c00de30 con 0x7f1f2c104350 2024-05-09T02:10:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.917+0000 7f1f31eac700 1 -- 172.21.15.19:0/2439514174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2c1a2700 con 0x7f1f2c104350 2024-05-09T02:10:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.917+0000 7f1f297fa700 1 -- 172.21.15.19:0/2439514174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c003b10 con 0x7f1f2c104350 2024-05-09T02:10:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.917+0000 7f1f31eac700 1 -- 172.21.15.19:0/2439514174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f2c1a2c90 con 0x7f1f2c104350 2024-05-09T02:10:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.918+0000 7f1f31eac700 1 -- 172.21.15.19:0/2439514174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f2c0512c0 con 0x7f1f2c104350 2024-05-09T02:10:31.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.920+0000 7f1f297fa700 1 -- 172.21.15.19:0/2439514174 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1f1c003cb0 con 0x7f1f2c104350 2024-05-09T02:10:31.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.921+0000 7f1f297fa700 1 --2- 172.21.15.19:0/2439514174 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1f180b2900 0x7f1f180b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:31.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.921+0000 7f1f22ffd700 1 --2- 172.21.15.19:0/2439514174 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1f180b2900 0x7f1f180b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:31.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.921+0000 7f1f297fa700 1 -- 172.21.15.19:0/2439514174 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f1f1c0e25e0 con 0x7f1f2c104350 2024-05-09T02:10:31.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.922+0000 7f1f22ffd700 1 --2- 172.21.15.19:0/2439514174 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1f180b2900 0x7f1f180b4e10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1f14003800 tx=0x7f1f1401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:31.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:31.925+0000 7f1f297fa700 1 -- 172.21.15.19:0/2439514174 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f1c020030 con 0x7f1f2c104350 2024-05-09T02:10:32.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:32.151+0000 7f1f31eac700 1 -- 172.21.15.19:0/2439514174 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f1f2c063c00 con 0x7f1f180b2900 2024-05-09T02:10:32.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:32.157+0000 7f1f297fa700 1 -- 172.21.15.19:0/2439514174 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f1f2c063c00 con 0x7f1f180b2900 2024-05-09T02:10:32.159 INFO:teuthology.orchestra.run.smithi019.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-09T02:10:32.160 INFO:teuthology.orchestra.run.smithi019.stdout:mds.all 12/12 40s ago 3m smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-05-09T02:10:32.160 INFO:teuthology.orchestra.run.smithi019.stdout:mgr 3/3 40s ago 9m smithi019=x;smithi059=y;smithi196=z;count:3 2024-05-09T02:10:32.160 INFO:teuthology.orchestra.run.smithi019.stdout:mon 3/3 40s ago 9m smithi019:172.21.15.19=a;smithi059:172.21.15.59=b;smithi196:172.21.15.196=c;count:3 2024-05-09T02:10:32.160 INFO:teuthology.orchestra.run.smithi019.stdout:osd 12 40s ago - 2024-05-09T02:10:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:32.159+0000 7f1f31eac700 1 -- 172.21.15.19:0/2439514174 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1f180b2900 msgr2=0x7f1f180b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:32.159+0000 7f1f31eac700 1 --2- 172.21.15.19:0/2439514174 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1f180b2900 0x7f1f180b4e10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1f14003800 tx=0x7f1f1401a040 comp rx=0 tx=0).stop 2024-05-09T02:10:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:32.159+0000 7f1f31eac700 1 -- 172.21.15.19:0/2439514174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f2c104350 msgr2=0x7f1f2c19a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:32.159+0000 7f1f31eac700 1 --2- 172.21.15.19:0/2439514174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f2c104350 0x7f1f2c19a940 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00d950 tx=0x7f1f1c00dd10 comp rx=0 tx=0).stop 2024-05-09T02:10:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:32.159+0000 7f1f31eac700 1 -- 172.21.15.19:0/2439514174 shutdown_connections 2024-05-09T02:10:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:32.159+0000 7f1f31eac700 1 --2- 172.21.15.19:0/2439514174 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f1f180b2900 0x7f1f180b4e10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:32.159+0000 7f1f31eac700 1 --2- 172.21.15.19:0/2439514174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f2c105e20 0x7f1f2c1a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:32.159+0000 7f1f31eac700 1 --2- 172.21.15.19:0/2439514174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f2c104350 0x7f1f2c19a940 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:32.159+0000 7f1f31eac700 1 --2- 172.21.15.19:0/2439514174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2c105480 0x7f1f2c19ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:32.159+0000 7f1f31eac700 1 -- 172.21.15.19:0/2439514174 >> 172.21.15.19:0/2439514174 conn(0x7f1f2c0ff990 msgr2=0x7f1f2c1086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:32.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:32.159+0000 7f1f31eac700 1 -- 172.21.15.19:0/2439514174 shutdown_connections 2024-05-09T02:10:32.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:32.159+0000 7f1f31eac700 1 -- 172.21.15.19:0/2439514174 wait complete. 2024-05-09T02:10:32.985 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-09T02:10:33.241 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:33.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:33 smithi019 ceph-mon[30151]: pgmap v375: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:33.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:33 smithi019 ceph-mon[30151]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:10:33.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:33 smithi059 ceph-mon[34781]: pgmap v375: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:33.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:33 smithi059 ceph-mon[34781]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:10:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:33 smithi196 ceph-mon[34718]: pgmap v375: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:33.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:33 smithi196 ceph-mon[34718]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:10:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.003+0000 7f5a55e50700 1 -- 172.21.15.19:0/2689467446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a50104d20 msgr2=0x7f5a5010b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.003+0000 7f5a55e50700 1 --2- 172.21.15.19:0/2689467446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a50104d20 0x7f5a5010b5d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800b0a0 tx=0x7f5a4800b3b0 comp rx=0 tx=0).stop 2024-05-09T02:10:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.003+0000 7f5a55e50700 1 -- 172.21.15.19:0/2689467446 shutdown_connections 2024-05-09T02:10:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.003+0000 7f5a55e50700 1 --2- 172.21.15.19:0/2689467446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a50104d20 0x7f5a5010b5d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.003+0000 7f5a55e50700 1 --2- 172.21.15.19:0/2689467446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a50103250 0x7f5a50103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.003+0000 7f5a55e50700 1 --2- 172.21.15.19:0/2689467446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a50104380 0x7f5a50104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.003+0000 7f5a55e50700 1 -- 172.21.15.19:0/2689467446 >> 172.21.15.19:0/2689467446 conn(0x7f5a500fe890 msgr2=0x7f5a50100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.004+0000 7f5a55e50700 1 -- 172.21.15.19:0/2689467446 shutdown_connections 2024-05-09T02:10:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.004+0000 7f5a55e50700 1 -- 172.21.15.19:0/2689467446 wait complete. 2024-05-09T02:10:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.004+0000 7f5a55e50700 1 Processor -- start 2024-05-09T02:10:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.004+0000 7f5a55e50700 1 -- start start 2024-05-09T02:10:35.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a55e50700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a50103250 0x7f5a5019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:35.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a55e50700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a50104380 0x7f5a5019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:35.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a55e50700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a50104d20 0x7f5a501a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:35.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a55e50700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5010ecb0 con 0x7f5a50104d20 2024-05-09T02:10:35.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a55e50700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5010eb30 con 0x7f5a50104380 2024-05-09T02:10:35.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a55e50700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5010ee30 con 0x7f5a50103250 2024-05-09T02:10:35.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a4ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a50104d20 0x7f5a501a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:35.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a4ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a50104d20 0x7f5a501a1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56952/0 (socket says 172.21.15.19:56952) 2024-05-09T02:10:35.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a4ffff700 1 -- 172.21.15.19:0/1720278634 learned_addr learned my addr 172.21.15.19:0/1720278634 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:35.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a4effd700 1 --2- 172.21.15.19:0/1720278634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a50104380 0x7f5a5019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:35.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a4f7fe700 1 --2- 172.21.15.19:0/1720278634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a50103250 0x7f5a5019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:35.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a4ffff700 1 -- 172.21.15.19:0/1720278634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a50103250 msgr2=0x7f5a5019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:35.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a4ffff700 1 --2- 172.21.15.19:0/1720278634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a50103250 0x7f5a5019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:35.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a4ffff700 1 -- 172.21.15.19:0/1720278634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a50104380 msgr2=0x7f5a5019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:35.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a4ffff700 1 --2- 172.21.15.19:0/1720278634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a50104380 0x7f5a5019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:35.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.005+0000 7f5a4ffff700 1 -- 172.21.15.19:0/1720278634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a501a2360 con 0x7f5a50104d20 2024-05-09T02:10:35.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.006+0000 7f5a4ffff700 1 --2- 172.21.15.19:0/1720278634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a50104d20 0x7f5a501a1d70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800b070 tx=0x7f5a48004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:35.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.006+0000 7f5a4cff9700 1 -- 172.21.15.19:0/1720278634 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48021070 con 0x7f5a50104d20 2024-05-09T02:10:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.006+0000 7f5a55e50700 1 -- 172.21.15.19:0/1720278634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a501a25f0 con 0x7f5a50104d20 2024-05-09T02:10:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.006+0000 7f5a4cff9700 1 -- 172.21.15.19:0/1720278634 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a48007c80 con 0x7f5a50104d20 2024-05-09T02:10:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.006+0000 7f5a4cff9700 1 -- 172.21.15.19:0/1720278634 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48005280 con 0x7f5a50104d20 2024-05-09T02:10:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.006+0000 7f5a55e50700 1 -- 172.21.15.19:0/1720278634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a501a2b80 con 0x7f5a50104d20 2024-05-09T02:10:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.007+0000 7f5a55e50700 1 -- 172.21.15.19:0/1720278634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a500512c0 con 0x7f5a50104d20 2024-05-09T02:10:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.009+0000 7f5a4cff9700 1 -- 172.21.15.19:0/1720278634 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5a48024070 con 0x7f5a50104d20 2024-05-09T02:10:35.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.009+0000 7f5a4cff9700 1 --2- 172.21.15.19:0/1720278634 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5a3c0b2900 0x7f5a3c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:35.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.010+0000 7f5a4f7fe700 1 --2- 172.21.15.19:0/1720278634 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5a3c0b2900 0x7f5a3c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:35.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.010+0000 7f5a4cff9700 1 -- 172.21.15.19:0/1720278634 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f5a4800f020 con 0x7f5a50104d20 2024-05-09T02:10:35.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.010+0000 7f5a4f7fe700 1 --2- 172.21.15.19:0/1720278634 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5a3c0b2900 0x7f5a3c0b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5a50104280 tx=0x7f5a3800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.017+0000 7f5a4cff9700 1 -- 172.21.15.19:0/1720278634 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5a4802b020 con 0x7f5a50104d20 2024-05-09T02:10:35.241 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:35 smithi019 ceph-mon[30151]: pgmap v376: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:10:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.239+0000 7f5a55e50700 1 -- 172.21.15.19:0/1720278634 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f5a501a2e10 con 0x7f5a3c0b2900 2024-05-09T02:10:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.240+0000 7f5a4cff9700 1 -- 172.21.15.19:0/1720278634 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f5a501a2e10 con 0x7f5a3c0b2900 2024-05-09T02:10:35.243 INFO:teuthology.orchestra.run.smithi019.stdout:HOST ADDR LABELS STATUS 2024-05-09T02:10:35.243 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 172.21.15.19 2024-05-09T02:10:35.243 INFO:teuthology.orchestra.run.smithi019.stdout:smithi059 172.21.15.59 2024-05-09T02:10:35.243 INFO:teuthology.orchestra.run.smithi019.stdout:smithi196 172.21.15.196 2024-05-09T02:10:35.243 INFO:teuthology.orchestra.run.smithi019.stdout:3 hosts in cluster 2024-05-09T02:10:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.243+0000 7f5a55e50700 1 -- 172.21.15.19:0/1720278634 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5a3c0b2900 msgr2=0x7f5a3c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.243+0000 7f5a55e50700 1 --2- 172.21.15.19:0/1720278634 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5a3c0b2900 0x7f5a3c0b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5a50104280 tx=0x7f5a3800a3b0 comp rx=0 tx=0).stop 2024-05-09T02:10:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.243+0000 7f5a55e50700 1 -- 172.21.15.19:0/1720278634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a50104d20 msgr2=0x7f5a501a1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.243+0000 7f5a55e50700 1 --2- 172.21.15.19:0/1720278634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a50104d20 0x7f5a501a1d70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800b070 tx=0x7f5a48004e40 comp rx=0 tx=0).stop 2024-05-09T02:10:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.243+0000 7f5a55e50700 1 -- 172.21.15.19:0/1720278634 shutdown_connections 2024-05-09T02:10:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.243+0000 7f5a55e50700 1 --2- 172.21.15.19:0/1720278634 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f5a3c0b2900 0x7f5a3c0b4e10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.243+0000 7f5a55e50700 1 --2- 172.21.15.19:0/1720278634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a50104d20 0x7f5a501a1d70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.243+0000 7f5a55e50700 1 --2- 172.21.15.19:0/1720278634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a50103250 0x7f5a5019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.243+0000 7f5a55e50700 1 --2- 172.21.15.19:0/1720278634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a50104380 0x7f5a5019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.243+0000 7f5a55e50700 1 -- 172.21.15.19:0/1720278634 >> 172.21.15.19:0/1720278634 conn(0x7f5a500fe890 msgr2=0x7f5a50100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.243+0000 7f5a55e50700 1 -- 172.21.15.19:0/1720278634 shutdown_connections 2024-05-09T02:10:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:35.243+0000 7f5a55e50700 1 -- 172.21.15.19:0/1720278634 wait complete. 2024-05-09T02:10:35.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:35 smithi059 ceph-mon[34781]: pgmap v376: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:10:35.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:35 smithi196 ceph-mon[34718]: pgmap v376: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:10:36.091 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-09T02:10:36.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:36 smithi019 ceph-mon[30151]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:10:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:36.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:36 smithi059 ceph-mon[34781]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:10:36.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:36 smithi196 ceph-mon[34718]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:10:37.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:37 smithi059 ceph-mon[34781]: pgmap v377: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:10:37.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:37 smithi196 ceph-mon[34718]: pgmap v377: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:10:37.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:37 smithi019 ceph-mon[30151]: pgmap v377: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:10:38.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.231+0000 7f122a9f9700 1 -- 172.21.15.19:0/2938244965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224102ca0 msgr2=0x7f122410c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:38.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.231+0000 7f122a9f9700 1 --2- 172.21.15.19:0/2938244965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224102ca0 0x7f122410c8c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f121c00b0a0 tx=0x7f121c00b3b0 comp rx=0 tx=0).stop 2024-05-09T02:10:38.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.231+0000 7f122a9f9700 1 -- 172.21.15.19:0/2938244965 shutdown_connections 2024-05-09T02:10:38.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.231+0000 7f122a9f9700 1 --2- 172.21.15.19:0/2938244965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224102ca0 0x7f122410c8c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:38.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.231+0000 7f122a9f9700 1 --2- 172.21.15.19:0/2938244965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1224105900 0x7f1224105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:38.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.231+0000 7f122a9f9700 1 --2- 172.21.15.19:0/2938244965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1224102350 0x7f1224102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:38.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.231+0000 7f122a9f9700 1 -- 172.21.15.19:0/2938244965 >> 172.21.15.19:0/2938244965 conn(0x7f12240fe020 msgr2=0x7f12241004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:38.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.231+0000 7f122a9f9700 1 -- 172.21.15.19:0/2938244965 shutdown_connections 2024-05-09T02:10:38.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.231+0000 7f122a9f9700 1 -- 172.21.15.19:0/2938244965 wait complete. 2024-05-09T02:10:38.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f122a9f9700 1 Processor -- start 2024-05-09T02:10:38.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f122a9f9700 1 -- start start 2024-05-09T02:10:38.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f122a9f9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1224102350 0x7f122419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:38.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f122a9f9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1224102ca0 0x7f122419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:38.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f122a9f9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224105900 0x7f12241a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:38.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f122a9f9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f122410fdc0 con 0x7f1224105900 2024-05-09T02:10:38.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f122a9f9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f122410fc40 con 0x7f1224102ca0 2024-05-09T02:10:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f122a9f9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f122410ff40 con 0x7f1224102350 2024-05-09T02:10:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f1228f96700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224105900 0x7f12241a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f1228f96700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224105900 0x7f12241a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45832/0 (socket says 172.21.15.19:45832) 2024-05-09T02:10:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f1228f96700 1 -- 172.21.15.19:0/3396110905 learned_addr learned my addr 172.21.15.19:0/3396110905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f1223fff700 1 --2- 172.21.15.19:0/3396110905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1224102350 0x7f122419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f121bfff700 1 --2- 172.21.15.19:0/3396110905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1224102ca0 0x7f122419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f1228f96700 1 -- 172.21.15.19:0/3396110905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1224102350 msgr2=0x7f122419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f1228f96700 1 --2- 172.21.15.19:0/3396110905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1224102350 0x7f122419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f1228f96700 1 -- 172.21.15.19:0/3396110905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1224102ca0 msgr2=0x7f122419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f1228f96700 1 --2- 172.21.15.19:0/3396110905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1224102ca0 0x7f122419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.232+0000 7f1228f96700 1 -- 172.21.15.19:0/3396110905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12241a2340 con 0x7f1224105900 2024-05-09T02:10:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.233+0000 7f1228f96700 1 --2- 172.21.15.19:0/3396110905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224105900 0x7f12241a1d50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f121c0062a0 tx=0x7f121c004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.233+0000 7f1221ffb700 1 -- 172.21.15.19:0/3396110905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c021070 con 0x7f1224105900 2024-05-09T02:10:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.233+0000 7f122a9f9700 1 -- 172.21.15.19:0/3396110905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12241a25d0 con 0x7f1224105900 2024-05-09T02:10:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.233+0000 7f1221ffb700 1 -- 172.21.15.19:0/3396110905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f121c007c80 con 0x7f1224105900 2024-05-09T02:10:38.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.233+0000 7f1221ffb700 1 -- 172.21.15.19:0/3396110905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c004850 con 0x7f1224105900 2024-05-09T02:10:38.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.233+0000 7f122a9f9700 1 -- 172.21.15.19:0/3396110905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12241a2b60 con 0x7f1224105900 2024-05-09T02:10:38.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.234+0000 7f122a9f9700 1 -- 172.21.15.19:0/3396110905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12240512c0 con 0x7f1224105900 2024-05-09T02:10:38.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.236+0000 7f1221ffb700 1 -- 172.21.15.19:0/3396110905 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f121c003a00 con 0x7f1224105900 2024-05-09T02:10:38.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.236+0000 7f1221ffb700 1 --2- 172.21.15.19:0/3396110905 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f12100b2880 0x7f12100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:38.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.236+0000 7f1223fff700 1 --2- 172.21.15.19:0/3396110905 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f12100b2880 0x7f12100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:38.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.237+0000 7f1221ffb700 1 -- 172.21.15.19:0/3396110905 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f121c00f020 con 0x7f1224105900 2024-05-09T02:10:38.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.237+0000 7f1223fff700 1 --2- 172.21.15.19:0/3396110905 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f12100b2880 0x7f12100b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f122406c0a0 tx=0x7f121400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:38.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.240+0000 7f1221ffb700 1 -- 172.21.15.19:0/3396110905 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f121c02a020 con 0x7f1224105900 2024-05-09T02:10:38.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.463+0000 7f122a9f9700 1 -- 172.21.15.19:0/3396110905 --> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f1224063c00 con 0x7f12100b2880 2024-05-09T02:10:38.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.464+0000 7f1221ffb700 1 -- 172.21.15.19:0/3396110905 <== mgr.14152 v2:172.21.15.19:6800/972543870 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f1224063c00 con 0x7f12100b2880 2024-05-09T02:10:38.467 INFO:teuthology.orchestra.run.smithi019.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-09T02:10:38.467 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330002G400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-09T02:10:38.467 INFO:teuthology.orchestra.run.smithi019.stdout:smithi059 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009L400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-09T02:10:38.467 INFO:teuthology.orchestra.run.smithi019.stdout:smithi196 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-09T02:10:38.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.467+0000 7f122a9f9700 1 -- 172.21.15.19:0/3396110905 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f12100b2880 msgr2=0x7f12100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:38.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.467+0000 7f122a9f9700 1 --2- 172.21.15.19:0/3396110905 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f12100b2880 0x7f12100b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f122406c0a0 tx=0x7f121400b410 comp rx=0 tx=0).stop 2024-05-09T02:10:38.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.467+0000 7f122a9f9700 1 -- 172.21.15.19:0/3396110905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224105900 msgr2=0x7f12241a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:38.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.467+0000 7f122a9f9700 1 --2- 172.21.15.19:0/3396110905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224105900 0x7f12241a1d50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f121c0062a0 tx=0x7f121c004e40 comp rx=0 tx=0).stop 2024-05-09T02:10:38.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.467+0000 7f122a9f9700 1 -- 172.21.15.19:0/3396110905 shutdown_connections 2024-05-09T02:10:38.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.467+0000 7f122a9f9700 1 --2- 172.21.15.19:0/3396110905 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f12100b2880 0x7f12100b4d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:38.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.467+0000 7f122a9f9700 1 --2- 172.21.15.19:0/3396110905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1224105900 0x7f12241a1d50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:38.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.467+0000 7f122a9f9700 1 --2- 172.21.15.19:0/3396110905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1224102350 0x7f122419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:38.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.467+0000 7f122a9f9700 1 --2- 172.21.15.19:0/3396110905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1224102ca0 0x7f122419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:38.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.467+0000 7f122a9f9700 1 -- 172.21.15.19:0/3396110905 >> 172.21.15.19:0/3396110905 conn(0x7f12240fe020 msgr2=0x7f12241088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:38.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.467+0000 7f122a9f9700 1 -- 172.21.15.19:0/3396110905 shutdown_connections 2024-05-09T02:10:38.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:38.468+0000 7f122a9f9700 1 -- 172.21.15.19:0/3396110905 wait complete. 2024-05-09T02:10:39.231 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T02:10:39.239 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T02:10:39.240 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-09T02:10:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:39 smithi059 ceph-mon[34781]: pgmap v378: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:39.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:39 smithi059 ceph-mon[34781]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:10:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:39.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:39 smithi196 ceph-mon[34718]: pgmap v378: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:39.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:39 smithi196 ceph-mon[34718]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:10:39.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:39 smithi019 ceph-mon[30151]: pgmap v378: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:39.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:39 smithi019 ceph-mon[30151]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:10:41.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.306+0000 7f9853d11700 1 -- 172.21.15.19:0/639330814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f984c104350 msgr2=0x7f984c104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:41.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.306+0000 7f9853d11700 1 --2- 172.21.15.19:0/639330814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f984c104350 0x7f984c104760 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f984800b0a0 tx=0x7f984800b3b0 comp rx=0 tx=0).stop 2024-05-09T02:10:41.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.306+0000 7f9853d11700 1 -- 172.21.15.19:0/639330814 shutdown_connections 2024-05-09T02:10:41.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.306+0000 7f9853d11700 1 --2- 172.21.15.19:0/639330814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f984c105e20 0x7f984c10c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:41.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.306+0000 7f9853d11700 1 --2- 172.21.15.19:0/639330814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f984c104350 0x7f984c104760 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:41.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.306+0000 7f9853d11700 1 --2- 172.21.15.19:0/639330814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f984c105480 0x7f984c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:41.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.306+0000 7f9853d11700 1 -- 172.21.15.19:0/639330814 >> 172.21.15.19:0/639330814 conn(0x7f984c0ff990 msgr2=0x7f984c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:41.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.306+0000 7f9853d11700 1 -- 172.21.15.19:0/639330814 shutdown_connections 2024-05-09T02:10:41.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.306+0000 7f9853d11700 1 -- 172.21.15.19:0/639330814 wait complete. 2024-05-09T02:10:41.311 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:41 smithi019 ceph-mon[30151]: pgmap v379: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:10:41.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.307+0000 7f9853d11700 1 Processor -- start 2024-05-09T02:10:41.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.307+0000 7f9853d11700 1 -- start start 2024-05-09T02:10:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f9853d11700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f984c104350 0x7f984c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f9853d11700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f984c105480 0x7f984c19b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f9853d11700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f984c105e20 0x7f984c1a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f9853d11700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f984c10fe90 con 0x7f984c104350 2024-05-09T02:10:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f9853d11700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f984c10fd10 con 0x7f984c105480 2024-05-09T02:10:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f9853d11700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f984c110010 con 0x7f984c105e20 2024-05-09T02:10:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f9851aad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f984c104350 0x7f984c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f9851aad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f984c104350 0x7f984c19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45838/0 (socket says 172.21.15.19:45838) 2024-05-09T02:10:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f9851aad700 1 -- 172.21.15.19:0/2508370279 learned_addr learned my addr 172.21.15.19:0/2508370279 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f98512ac700 1 --2- 172.21.15.19:0/2508370279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f984c105480 0x7f984c19b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f98522ae700 1 --2- 172.21.15.19:0/2508370279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f984c105e20 0x7f984c1a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f9851aad700 1 -- 172.21.15.19:0/2508370279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f984c105e20 msgr2=0x7f984c1a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f9851aad700 1 --2- 172.21.15.19:0/2508370279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f984c105e20 0x7f984c1a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f9851aad700 1 -- 172.21.15.19:0/2508370279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f984c105480 msgr2=0x7f984c19b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f9851aad700 1 --2- 172.21.15.19:0/2508370279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f984c105480 0x7f984c19b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f9851aad700 1 -- 172.21.15.19:0/2508370279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984c1a2680 con 0x7f984c104350 2024-05-09T02:10:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.308+0000 7f9851aad700 1 --2- 172.21.15.19:0/2508370279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f984c104350 0x7f984c19ab20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f984800b070 tx=0x7f9848004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.309+0000 7f9842ffd700 1 -- 172.21.15.19:0/2508370279 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848021070 con 0x7f984c104350 2024-05-09T02:10:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.309+0000 7f9853d11700 1 -- 172.21.15.19:0/2508370279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984c1a2910 con 0x7f984c104350 2024-05-09T02:10:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.309+0000 7f9842ffd700 1 -- 172.21.15.19:0/2508370279 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f984801d900 con 0x7f984c104350 2024-05-09T02:10:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.309+0000 7f9842ffd700 1 -- 172.21.15.19:0/2508370279 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98480053a0 con 0x7f984c104350 2024-05-09T02:10:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.309+0000 7f9853d11700 1 -- 172.21.15.19:0/2508370279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984c1a2e50 con 0x7f984c104350 2024-05-09T02:10:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.310+0000 7f9853d11700 1 -- 172.21.15.19:0/2508370279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984c0512c0 con 0x7f984c104350 2024-05-09T02:10:41.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.312+0000 7f9842ffd700 1 -- 172.21.15.19:0/2508370279 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9848024070 con 0x7f984c104350 2024-05-09T02:10:41.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.312+0000 7f9842ffd700 1 --2- 172.21.15.19:0/2508370279 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f98380b2880 0x7f98380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:41.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.312+0000 7f98512ac700 1 --2- 172.21.15.19:0/2508370279 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f98380b2880 0x7f98380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:41.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.313+0000 7f9842ffd700 1 -- 172.21.15.19:0/2508370279 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f984800f020 con 0x7f984c104350 2024-05-09T02:10:41.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.313+0000 7f98512ac700 1 --2- 172.21.15.19:0/2508370279 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f98380b2880 0x7f98380b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f983c006fd0 tx=0x7f983c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:41.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.316+0000 7f9842ffd700 1 -- 172.21.15.19:0/2508370279 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f984802b020 con 0x7f984c104350 2024-05-09T02:10:41.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:41 smithi059 ceph-mon[34781]: pgmap v379: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:10:41.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:41 smithi196 ceph-mon[34718]: pgmap v379: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:10:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.575+0000 7f9853d11700 1 -- 172.21.15.19:0/2508370279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f984c02e5f0 con 0x7f984c104350 2024-05-09T02:10:41.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.576+0000 7f9842ffd700 1 -- 172.21.15.19:0/2508370279 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3081 (secure 0 0 0) 0x7f98480abb20 con 0x7f984c104350 2024-05-09T02:10:41.578 INFO:teuthology.orchestra.run.smithi019.stdout:e28 2024-05-09T02:10:41.578 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T02:10:41.578 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T02:10:41.578 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 28 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-05-09T02:07:50.316306+0000 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-05-09T02:08:11.346147+0000 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 300 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-05-09T02:10:41.579 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 0 2024-05-09T02:10:41.580 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T02:10:41.580 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 5 2024-05-09T02:10:41.580 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1,2,3,4 2024-05-09T02:10:41.580 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=24484,1=24473,2=24463,3=24457,4=24455} 2024-05-09T02:10:41.580 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-05-09T02:10:41.580 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-05-09T02:10:41.580 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-05-09T02:10:41.580 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-05-09T02:10:41.580 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-05-09T02:10:41.580 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-05-09T02:10:41.580 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-05-09T02:10:41.580 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-05-09T02:10:41.580 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 0 2024-05-09T02:10:41.580 INFO:teuthology.orchestra.run.smithi019.stdout:qdb_cluster leader: 24484 members: 24455,24457,24473,24484,24463 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.i{0:24484} state up:active seq 8 addr [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.e{0:24431} state up:standby-replay seq 1 addr [v2:172.21.15.59:6834/978263217,v1:172.21.15.59:6835/978263217] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b{1:24473} state up:active seq 9 addr [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.l{1:24427} state up:standby-replay seq 1 addr [v2:172.21.15.196:6832/416853517,v1:172.21.15.196:6833/416853517] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c{2:24463} state up:active seq 13 addr [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.k{2:24407} state up:standby-replay seq 1 addr [v2:172.21.15.59:6832/2040289281,v1:172.21.15.59:6833/2040289281] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.f{3:24457} state up:active seq 14 addr [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a{3:14616} state up:standby-replay seq 1 addr [v2:172.21.15.19:6840/3162604494,v1:172.21.15.19:6841/3162604494] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.h{4:24455} state up:active seq 13 addr [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.g{4:14604} state up:standby-replay seq 1 addr [v2:172.21.15.19:6838/810391856,v1:172.21.15.19:6839/810391856] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.19:6834/4121765116,v1:172.21.15.19:6835/4121765116] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T02:10:41.581 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d{-1:14556} state up:standby seq 1 addr [v2:172.21.15.19:6836/2298435247,v1:172.21.15.19:6837/2298435247] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T02:10:41.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.578+0000 7f9853d11700 1 -- 172.21.15.19:0/2508370279 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f98380b2880 msgr2=0x7f98380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:41.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.578+0000 7f9853d11700 1 --2- 172.21.15.19:0/2508370279 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f98380b2880 0x7f98380b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f983c006fd0 tx=0x7f983c008040 comp rx=0 tx=0).stop 2024-05-09T02:10:41.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.578+0000 7f9853d11700 1 -- 172.21.15.19:0/2508370279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f984c104350 msgr2=0x7f984c19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:41.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.578+0000 7f9853d11700 1 --2- 172.21.15.19:0/2508370279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f984c104350 0x7f984c19ab20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f984800b070 tx=0x7f9848004ef0 comp rx=0 tx=0).stop 2024-05-09T02:10:41.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.578+0000 7f9853d11700 1 -- 172.21.15.19:0/2508370279 shutdown_connections 2024-05-09T02:10:41.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.578+0000 7f9853d11700 1 --2- 172.21.15.19:0/2508370279 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f98380b2880 0x7f98380b4d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:41.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.578+0000 7f9853d11700 1 --2- 172.21.15.19:0/2508370279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f984c105e20 0x7f984c1a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:41.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.578+0000 7f9853d11700 1 --2- 172.21.15.19:0/2508370279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f984c104350 0x7f984c19ab20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:41.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.578+0000 7f9853d11700 1 --2- 172.21.15.19:0/2508370279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f984c105480 0x7f984c19b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:41.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.578+0000 7f9853d11700 1 -- 172.21.15.19:0/2508370279 >> 172.21.15.19:0/2508370279 conn(0x7f984c0ff990 msgr2=0x7f984c1086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:41.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.578+0000 7f9853d11700 1 -- 172.21.15.19:0/2508370279 shutdown_connections 2024-05-09T02:10:41.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:41.578+0000 7f9853d11700 1 -- 172.21.15.19:0/2508370279 wait complete. 2024-05-09T02:10:41.583 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T02:10:42.215 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-09T02:10:42.469 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:42.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:42 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2508370279' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T02:10:42.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:42 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2508370279' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T02:10:42.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:42 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2508370279' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T02:10:43.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:43 smithi196 ceph-mon[34718]: pgmap v380: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:43.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:43 smithi019 ceph-mon[30151]: pgmap v380: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:43.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:43 smithi059 ceph-mon[34781]: pgmap v380: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:44.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.349+0000 7fc56b7a9700 1 -- 172.21.15.19:0/3725803738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5640776e0 msgr2=0x7fc564075bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:44.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.349+0000 7fc56b7a9700 1 --2- 172.21.15.19:0/3725803738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5640776e0 0x7fc564075bb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc560009990 tx=0x7fc560009ca0 comp rx=0 tx=0).stop 2024-05-09T02:10:44.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.349+0000 7fc56b7a9700 1 -- 172.21.15.19:0/3725803738 shutdown_connections 2024-05-09T02:10:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.349+0000 7fc56b7a9700 1 --2- 172.21.15.19:0/3725803738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc56410c210 0x7fc56410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.349+0000 7fc56b7a9700 1 --2- 172.21.15.19:0/3725803738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5640776e0 0x7fc564075bb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.349+0000 7fc56b7a9700 1 --2- 172.21.15.19:0/3725803738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5640760f0 0x7fc564076580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.349+0000 7fc56b7a9700 1 -- 172.21.15.19:0/3725803738 >> 172.21.15.19:0/3725803738 conn(0x7fc5640ff9c0 msgr2=0x7fc564101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.349+0000 7fc56b7a9700 1 -- 172.21.15.19:0/3725803738 shutdown_connections 2024-05-09T02:10:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.349+0000 7fc56b7a9700 1 -- 172.21.15.19:0/3725803738 wait complete. 2024-05-09T02:10:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc56b7a9700 1 Processor -- start 2024-05-09T02:10:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc56b7a9700 1 -- start start 2024-05-09T02:10:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc56b7a9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5640760f0 0x7fc56419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc56b7a9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5640776e0 0x7fc56419d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc56b7a9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc56410c210 0x7fc5641a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc56b7a9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc564111fa0 con 0x7fc56410c210 2024-05-09T02:10:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc56b7a9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc564111e20 con 0x7fc5640776e0 2024-05-09T02:10:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc56b7a9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc564112120 con 0x7fc5640760f0 2024-05-09T02:10:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc569d46700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc56410c210 0x7fc5641a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc569d46700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc56410c210 0x7fc5641a4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45860/0 (socket says 172.21.15.19:45860) 2024-05-09T02:10:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc569d46700 1 -- 172.21.15.19:0/3859595080 learned_addr learned my addr 172.21.15.19:0/3859595080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc569545700 1 --2- 172.21.15.19:0/3859595080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5640760f0 0x7fc56419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc568d44700 1 --2- 172.21.15.19:0/3859595080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5640776e0 0x7fc56419d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc569d46700 1 -- 172.21.15.19:0/3859595080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5640760f0 msgr2=0x7fc56419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc569d46700 1 --2- 172.21.15.19:0/3859595080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5640760f0 0x7fc56419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc569d46700 1 -- 172.21.15.19:0/3859595080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5640776e0 msgr2=0x7fc56419d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc569d46700 1 --2- 172.21.15.19:0/3859595080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5640776e0 0x7fc56419d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.350+0000 7fc569d46700 1 -- 172.21.15.19:0/3859595080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5641a4680 con 0x7fc56410c210 2024-05-09T02:10:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.351+0000 7fc569d46700 1 --2- 172.21.15.19:0/3859595080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc56410c210 0x7fc5641a4090 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc55c007d70 tx=0x7fc55c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.351+0000 7fc55a7fc700 1 -- 172.21.15.19:0/3859595080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c016020 con 0x7fc56410c210 2024-05-09T02:10:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.351+0000 7fc55a7fc700 1 -- 172.21.15.19:0/3859595080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc55c00ad40 con 0x7fc56410c210 2024-05-09T02:10:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.351+0000 7fc56b7a9700 1 -- 172.21.15.19:0/3859595080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5641a4970 con 0x7fc56410c210 2024-05-09T02:10:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.351+0000 7fc55a7fc700 1 -- 172.21.15.19:0/3859595080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c0115e0 con 0x7fc56410c210 2024-05-09T02:10:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.351+0000 7fc56b7a9700 1 -- 172.21.15.19:0/3859595080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5641a4f00 con 0x7fc56410c210 2024-05-09T02:10:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.352+0000 7fc56b7a9700 1 -- 172.21.15.19:0/3859595080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5640512c0 con 0x7fc56410c210 2024-05-09T02:10:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.353+0000 7fc55a7fc700 1 -- 172.21.15.19:0/3859595080 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc55c019020 con 0x7fc56410c210 2024-05-09T02:10:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.354+0000 7fc55a7fc700 1 --2- 172.21.15.19:0/3859595080 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5500b2900 0x7fc5500b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.354+0000 7fc569545700 1 --2- 172.21.15.19:0/3859595080 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5500b2900 0x7fc5500b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.354+0000 7fc55a7fc700 1 -- 172.21.15.19:0/3859595080 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc55c0e25c0 con 0x7fc56410c210 2024-05-09T02:10:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.355+0000 7fc569545700 1 --2- 172.21.15.19:0/3859595080 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5500b2900 0x7fc5500b4e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc56000b5c0 tx=0x7fc56001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.358+0000 7fc55a7fc700 1 -- 172.21.15.19:0/3859595080 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc55c020020 con 0x7fc56410c210 2024-05-09T02:10:44.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.575+0000 7fc56b7a9700 1 -- 172.21.15.19:0/3859595080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fc56402e5f0 con 0x7fc56410c210 2024-05-09T02:10:44.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.576+0000 7fc55a7fc700 1 -- 172.21.15.19:0/3859595080 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v81) ==== 56+0+5319 (secure 0 0 0) 0x7fc55c0aa2c0 con 0x7fc56410c210 2024-05-09T02:10:44.579 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 81 2024-05-09T02:10:44.579 INFO:teuthology.orchestra.run.smithi019.stdout:fsid a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:10:44.579 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-05-09T01:59:27.591343+0000 2024-05-09T02:10:44.579 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-05-09T02:08:25.302927+0000 2024-05-09T02:10:44.579 INFO:teuthology.orchestra.run.smithi019.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-09T02:10:44.579 INFO:teuthology.orchestra.run.smithi019.stdout:crush_version 26 2024-05-09T02:10:44.579 INFO:teuthology.orchestra.run.smithi019.stdout:full_ratio 0.95 2024-05-09T02:10:44.579 INFO:teuthology.orchestra.run.smithi019.stdout:backfillfull_ratio 0.9 2024-05-09T02:10:44.579 INFO:teuthology.orchestra.run.smithi019.stdout:nearfull_ratio 0.85 2024-05-09T02:10:44.580 INFO:teuthology.orchestra.run.smithi019.stdout:require_min_compat_client luminous 2024-05-09T02:10:44.580 INFO:teuthology.orchestra.run.smithi019.stdout:min_compat_client luminous 2024-05-09T02:10:44.580 INFO:teuthology.orchestra.run.smithi019.stdout:require_osd_release squid 2024-05-09T02:10:44.580 INFO:teuthology.orchestra.run.smithi019.stdout:stretch_mode_enabled false 2024-05-09T02:10:44.580 INFO:teuthology.orchestra.run.smithi019.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-09T02:10:44.580 INFO:teuthology.orchestra.run.smithi019.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-09T02:10:44.580 INFO:teuthology.orchestra.run.smithi019.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-09T02:10:44.580 INFO:teuthology.orchestra.run.smithi019.stdout:max_osd 12 2024-05-09T02:10:44.580 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 up in weight 1 up_from 8 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6802/3002373576,v1:172.21.15.19:6803/3002373576] [v2:172.21.15.19:6804/3002373576,v1:172.21.15.19:6805/3002373576] exists,up 996c6a3d-3e3a-451c-8789-36006e0b3dce 2024-05-09T02:10:44.580 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6800/827458353,v1:172.21.15.59:6801/827458353] [v2:172.21.15.59:6802/827458353,v1:172.21.15.59:6803/827458353] exists,up 1ac68d09-1198-4e81-86c7-e37f7c41aa5f 2024-05-09T02:10:44.580 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 up in weight 1 up_from 18 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6800/1659504639,v1:172.21.15.196:6801/1659504639] [v2:172.21.15.196:6802/1659504639,v1:172.21.15.196:6803/1659504639] exists,up 818a0de1-04e6-471d-83df-a68833011b4a 2024-05-09T02:10:44.580 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 up in weight 1 up_from 26 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6810/2707671599,v1:172.21.15.19:6811/2707671599] [v2:172.21.15.19:6812/2707671599,v1:172.21.15.19:6813/2707671599] exists,up 7b204854-86b8-454d-aee0-8bb5775ce828 2024-05-09T02:10:44.580 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 up in weight 1 up_from 32 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6808/223037330,v1:172.21.15.59:6809/223037330] [v2:172.21.15.59:6810/223037330,v1:172.21.15.59:6811/223037330] exists,up 091667f1-e887-4a94-8723-6f85b676fef8 2024-05-09T02:10:44.580 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 up in weight 1 up_from 37 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6808/2516161898,v1:172.21.15.196:6809/2516161898] [v2:172.21.15.196:6810/2516161898,v1:172.21.15.196:6811/2516161898] exists,up 2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec 2024-05-09T02:10:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 up in weight 1 up_from 43 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6818/2150451637,v1:172.21.15.19:6819/2150451637] [v2:172.21.15.19:6820/2150451637,v1:172.21.15.19:6821/2150451637] exists,up 4e352851-1279-475f-ba8b-eaf9ca73df74 2024-05-09T02:10:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 up in weight 1 up_from 48 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6816/967902053,v1:172.21.15.59:6817/967902053] [v2:172.21.15.59:6818/967902053,v1:172.21.15.59:6819/967902053] exists,up 83f53f59-9a60-4d87-9a8a-67e5c6f57004 2024-05-09T02:10:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 up in weight 1 up_from 54 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6816/2946064259,v1:172.21.15.196:6817/2946064259] [v2:172.21.15.196:6818/2946064259,v1:172.21.15.196:6819/2946064259] exists,up a8f1bfb5-1cc8-4a65-8396-239bd781e1e8 2024-05-09T02:10:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 up in weight 1 up_from 60 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6826/1735017886,v1:172.21.15.19:6827/1735017886] [v2:172.21.15.19:6828/1735017886,v1:172.21.15.19:6829/1735017886] exists,up 73c58dc3-fcb0-4440-92b3-bf41a2e58fbd 2024-05-09T02:10:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 up in weight 1 up_from 65 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6824/2932899995,v1:172.21.15.59:6825/2932899995] [v2:172.21.15.59:6826/2932899995,v1:172.21.15.59:6827/2932899995] exists,up 8325282f-7325-4880-ab7d-b972717be9dd 2024-05-09T02:10:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 up in weight 1 up_from 71 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6824/2608023001,v1:172.21.15.196:6825/2608023001] [v2:172.21.15.196:6826/2608023001,v1:172.21.15.196:6827/2608023001] exists,up 8b19228c-b286-43c8-a408-169782f13075 2024-05-09T02:10:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.5 [8,11] 2024-05-09T02:10:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.a [2,11] 2024-05-09T02:10:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.10 [2,4] 2024-05-09T02:10:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.1a [8,1] 2024-05-09T02:10:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.1 [4,9] 2024-05-09T02:10:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.c [3,9] 2024-05-09T02:10:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.1a [5,6] 2024-05-09T02:10:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.21 [4,8] 2024-05-09T02:10:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.2a [3,8] 2024-05-09T02:10:44.582 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.36 [4,9] 2024-05-09T02:10:44.582 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3291838961 expires 2024-05-10T02:00:23.920230+0000 2024-05-09T02:10:44.582 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1024133854 expires 2024-05-10T02:00:23.920230+0000 2024-05-09T02:10:44.582 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1176455101 expires 2024-05-10T02:00:23.920230+0000 2024-05-09T02:10:44.582 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/2101052000 expires 2024-05-10T02:00:23.920230+0000 2024-05-09T02:10:44.582 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/4018987404 expires 2024-05-10T01:59:56.604998+0000 2024-05-09T02:10:44.582 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3079777324 expires 2024-05-10T01:59:56.604998+0000 2024-05-09T02:10:44.582 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/1829553843 expires 2024-05-10T01:59:56.604998+0000 2024-05-09T02:10:44.582 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/1829553843 expires 2024-05-10T01:59:56.604998+0000 2024-05-09T02:10:44.582 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/179703528 expires 2024-05-10T01:59:56.604998+0000 2024-05-09T02:10:44.582 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/2101052000 expires 2024-05-10T02:00:23.920230+0000 2024-05-09T02:10:44.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.579+0000 7fc56b7a9700 1 -- 172.21.15.19:0/3859595080 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5500b2900 msgr2=0x7fc5500b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:44.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.579+0000 7fc56b7a9700 1 --2- 172.21.15.19:0/3859595080 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5500b2900 0x7fc5500b4e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc56000b5c0 tx=0x7fc56001a040 comp rx=0 tx=0).stop 2024-05-09T02:10:44.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.579+0000 7fc56b7a9700 1 -- 172.21.15.19:0/3859595080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc56410c210 msgr2=0x7fc5641a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:44.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.579+0000 7fc56b7a9700 1 --2- 172.21.15.19:0/3859595080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc56410c210 0x7fc5641a4090 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc55c007d70 tx=0x7fc55c00a340 comp rx=0 tx=0).stop 2024-05-09T02:10:44.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.579+0000 7fc56b7a9700 1 -- 172.21.15.19:0/3859595080 shutdown_connections 2024-05-09T02:10:44.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.579+0000 7fc56b7a9700 1 --2- 172.21.15.19:0/3859595080 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fc5500b2900 0x7fc5500b4e10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:44.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.579+0000 7fc56b7a9700 1 --2- 172.21.15.19:0/3859595080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc56410c210 0x7fc5641a4090 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:44.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.579+0000 7fc56b7a9700 1 --2- 172.21.15.19:0/3859595080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5640760f0 0x7fc56419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:44.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.579+0000 7fc56b7a9700 1 --2- 172.21.15.19:0/3859595080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5640776e0 0x7fc56419d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:44.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.579+0000 7fc56b7a9700 1 -- 172.21.15.19:0/3859595080 >> 172.21.15.19:0/3859595080 conn(0x7fc5640ff9c0 msgr2=0x7fc5641081a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:44.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.579+0000 7fc56b7a9700 1 -- 172.21.15.19:0/3859595080 shutdown_connections 2024-05-09T02:10:44.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:44.579+0000 7fc56b7a9700 1 -- 172.21.15.19:0/3859595080 wait complete. 2024-05-09T02:10:45.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:45 smithi019 ceph-mon[30151]: pgmap v381: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:10:45.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:45 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3859595080' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T02:10:45.494 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-09T02:10:45.506 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:10:45.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:45 smithi196 ceph-mon[34718]: pgmap v381: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:10:45.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:45 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3859595080' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T02:10:45.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:45 smithi059 ceph-mon[34781]: pgmap v381: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:10:45.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:45 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3859595080' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T02:10:45.760 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:47.527 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:47 smithi019 ceph-mon[30151]: pgmap v382: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:10:47.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:47 smithi196 ceph-mon[34718]: pgmap v382: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:10:47.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.550+0000 7fabe50c7700 1 -- 172.21.15.19:0/3611737520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe01030d0 msgr2=0x7fabe0111160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:47.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.550+0000 7fabe50c7700 1 --2- 172.21.15.19:0/3611737520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe01030d0 0x7fabe0111160 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fabd800b0a0 tx=0x7fabd800b3b0 comp rx=0 tx=0).stop 2024-05-09T02:10:47.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.551+0000 7fabe50c7700 1 -- 172.21.15.19:0/3611737520 shutdown_connections 2024-05-09T02:10:47.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.551+0000 7fabe50c7700 1 --2- 172.21.15.19:0/3611737520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe01030d0 0x7fabe0111160 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:47.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.551+0000 7fabe50c7700 1 --2- 172.21.15.19:0/3611737520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabe01092f0 0x7fabe0109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:47.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.551+0000 7fabe50c7700 1 --2- 172.21.15.19:0/3611737520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabe0102780 0x7fabe0102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:47.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.551+0000 7fabe50c7700 1 -- 172.21.15.19:0/3611737520 >> 172.21.15.19:0/3611737520 conn(0x7fabe00fe530 msgr2=0x7fabe0100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:47.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.551+0000 7fabe50c7700 1 -- 172.21.15.19:0/3611737520 shutdown_connections 2024-05-09T02:10:47.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.551+0000 7fabe50c7700 1 -- 172.21.15.19:0/3611737520 wait complete. 2024-05-09T02:10:47.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.551+0000 7fabe50c7700 1 Processor -- start 2024-05-09T02:10:47.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.551+0000 7fabe50c7700 1 -- start start 2024-05-09T02:10:47.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabe50c7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabe0102780 0x7fabe019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:47.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabe50c7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabe01030d0 0x7fabe019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:47.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabe50c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe01092f0 0x7fabe019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:47.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabe50c7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0114600 con 0x7fabe01092f0 2024-05-09T02:10:47.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabe50c7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0114480 con 0x7fabe0102780 2024-05-09T02:10:47.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabe50c7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0114780 con 0x7fabe01030d0 2024-05-09T02:10:47.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabdf59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe01092f0 0x7fabe019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:47.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabdf59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe01092f0 0x7fabe019f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39160/0 (socket says 172.21.15.19:39160) 2024-05-09T02:10:47.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabdf59e700 1 -- 172.21.15.19:0/4090990185 learned_addr learned my addr 172.21.15.19:0/4090990185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:47.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabd7fff700 1 --2- 172.21.15.19:0/4090990185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabe01030d0 0x7fabe019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:47.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabded9d700 1 --2- 172.21.15.19:0/4090990185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabe0102780 0x7fabe019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:47.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabdf59e700 1 -- 172.21.15.19:0/4090990185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabe01030d0 msgr2=0x7fabe019b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:47.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabdf59e700 1 --2- 172.21.15.19:0/4090990185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabe01030d0 0x7fabe019b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:47.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabdf59e700 1 -- 172.21.15.19:0/4090990185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabe0102780 msgr2=0x7fabe019aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:47.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabdf59e700 1 --2- 172.21.15.19:0/4090990185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabe0102780 0x7fabe019aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:47.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabdf59e700 1 -- 172.21.15.19:0/4090990185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe019fd80 con 0x7fabe01092f0 2024-05-09T02:10:47.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.552+0000 7fabdf59e700 1 --2- 172.21.15.19:0/4090990185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe01092f0 0x7fabe019f6e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fabd800b830 tx=0x7fabd8004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:47.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.553+0000 7fabdcd99700 1 -- 172.21.15.19:0/4090990185 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd8021070 con 0x7fabe01092f0 2024-05-09T02:10:47.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.553+0000 7fabe50c7700 1 -- 172.21.15.19:0/4090990185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe01a0010 con 0x7fabe01092f0 2024-05-09T02:10:47.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.553+0000 7fabdcd99700 1 -- 172.21.15.19:0/4090990185 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabd8003c00 con 0x7fabe01092f0 2024-05-09T02:10:47.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.553+0000 7fabdcd99700 1 -- 172.21.15.19:0/4090990185 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd8004730 con 0x7fabe01092f0 2024-05-09T02:10:47.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.553+0000 7fabe50c7700 1 -- 172.21.15.19:0/4090990185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe01a7990 con 0x7fabe01092f0 2024-05-09T02:10:47.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.554+0000 7fabe50c7700 1 -- 172.21.15.19:0/4090990185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe0105aa0 con 0x7fabe01092f0 2024-05-09T02:10:47.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.555+0000 7fabdcd99700 1 -- 172.21.15.19:0/4090990185 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fabd8007a00 con 0x7fabe01092f0 2024-05-09T02:10:47.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.556+0000 7fabdcd99700 1 --2- 172.21.15.19:0/4090990185 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fabcc0b2880 0x7fabcc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:47.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.556+0000 7fabded9d700 1 --2- 172.21.15.19:0/4090990185 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fabcc0b2880 0x7fabcc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:47.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.556+0000 7fabdcd99700 1 -- 172.21.15.19:0/4090990185 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fabd800f020 con 0x7fabe01092f0 2024-05-09T02:10:47.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.557+0000 7fabded9d700 1 --2- 172.21.15.19:0/4090990185 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fabcc0b2880 0x7fabcc0b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fabd0005fd0 tx=0x7fabd0005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.559+0000 7fabdcd99700 1 -- 172.21.15.19:0/4090990185 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fabd802a020 con 0x7fabe01092f0 2024-05-09T02:10:47.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:47 smithi059 ceph-mon[34781]: pgmap v382: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:10:47.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.823+0000 7fabe50c7700 1 -- 172.21.15.19:0/4090990185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabe019c210 con 0x7fabe01092f0 2024-05-09T02:10:47.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.824+0000 7fabdcd99700 1 -- 172.21.15.19:0/4090990185 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11917 (secure 0 0 0) 0x7fabd80abaf0 con 0x7fabe01092f0 2024-05-09T02:10:47.826 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:10:47.827 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:08:11.346147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24455,24457,24473,24484,24463]},"id":1}]} 2024-05-09T02:10:47.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.826+0000 7fabe50c7700 1 -- 172.21.15.19:0/4090990185 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fabcc0b2880 msgr2=0x7fabcc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:47.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.826+0000 7fabe50c7700 1 --2- 172.21.15.19:0/4090990185 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fabcc0b2880 0x7fabcc0b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fabd0005fd0 tx=0x7fabd0005f60 comp rx=0 tx=0).stop 2024-05-09T02:10:47.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.826+0000 7fabe50c7700 1 -- 172.21.15.19:0/4090990185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe01092f0 msgr2=0x7fabe019f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:47.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.826+0000 7fabe50c7700 1 --2- 172.21.15.19:0/4090990185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe01092f0 0x7fabe019f6e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fabd800b830 tx=0x7fabd8004d90 comp rx=0 tx=0).stop 2024-05-09T02:10:47.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.826+0000 7fabe50c7700 1 -- 172.21.15.19:0/4090990185 shutdown_connections 2024-05-09T02:10:47.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.826+0000 7fabe50c7700 1 --2- 172.21.15.19:0/4090990185 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fabcc0b2880 0x7fabcc0b4d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:47.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.826+0000 7fabe50c7700 1 --2- 172.21.15.19:0/4090990185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe01092f0 0x7fabe019f6e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:47.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.826+0000 7fabe50c7700 1 --2- 172.21.15.19:0/4090990185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabe0102780 0x7fabe019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:47.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.826+0000 7fabe50c7700 1 --2- 172.21.15.19:0/4090990185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabe01030d0 0x7fabe019b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:47.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.826+0000 7fabe50c7700 1 -- 172.21.15.19:0/4090990185 >> 172.21.15.19:0/4090990185 conn(0x7fabe00fe530 msgr2=0x7fabe0107090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:47.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.826+0000 7fabe50c7700 1 -- 172.21.15.19:0/4090990185 shutdown_connections 2024-05-09T02:10:47.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:47.826+0000 7fabe50c7700 1 -- 172.21.15.19:0/4090990185 wait complete. 2024-05-09T02:10:47.830 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T02:10:48.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:48 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/4090990185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:10:48.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:48 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/4090990185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:10:48.581 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:10:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:48 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/4090990185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:10:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:49.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:49 smithi196 ceph-mon[34718]: pgmap v383: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:49.628 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:49 smithi019 ceph-mon[30151]: pgmap v383: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:49.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:49 smithi059 ceph-mon[34781]: pgmap v383: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.651+0000 7ff84886d700 1 -- 172.21.15.19:0/2055137866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8441034c0 msgr2=0x7ff8441038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.651+0000 7ff84886d700 1 --2- 172.21.15.19:0/2055137866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8441034c0 0x7ff8441038d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff8340099c0 tx=0x7ff834009cd0 comp rx=0 tx=0).stop 2024-05-09T02:10:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.651+0000 7ff84886d700 1 -- 172.21.15.19:0/2055137866 shutdown_connections 2024-05-09T02:10:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.651+0000 7ff84886d700 1 --2- 172.21.15.19:0/2055137866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff844108740 0x7ff8441100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.651+0000 7ff84886d700 1 --2- 172.21.15.19:0/2055137866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8441034c0 0x7ff8441038d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.651+0000 7ff84886d700 1 --2- 172.21.15.19:0/2055137866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff844107df0 0x7ff844108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.651+0000 7ff84886d700 1 -- 172.21.15.19:0/2055137866 >> 172.21.15.19:0/2055137866 conn(0x7ff8440fedf0 msgr2=0x7ff844101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.651+0000 7ff84886d700 1 -- 172.21.15.19:0/2055137866 shutdown_connections 2024-05-09T02:10:50.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.651+0000 7ff84886d700 1 -- 172.21.15.19:0/2055137866 wait complete. 2024-05-09T02:10:50.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.652+0000 7ff84886d700 1 Processor -- start 2024-05-09T02:10:50.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.652+0000 7ff84886d700 1 -- start start 2024-05-09T02:10:50.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.652+0000 7ff84886d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8441034c0 0x7ff84419d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.652+0000 7ff84886d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff844107df0 0x7ff84419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.652+0000 7ff84886d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff844108740 0x7ff8441a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.653+0000 7ff84886d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff844113370 con 0x7ff844107df0 2024-05-09T02:10:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.653+0000 7ff84886d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8441131f0 con 0x7ff844108740 2024-05-09T02:10:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.653+0000 7ff84886d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8441134f0 con 0x7ff8441034c0 2024-05-09T02:10:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.653+0000 7ff841d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff844107df0 0x7ff84419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.653+0000 7ff841d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff844107df0 0x7ff84419d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39180/0 (socket says 172.21.15.19:39180) 2024-05-09T02:10:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.653+0000 7ff841d9b700 1 -- 172.21.15.19:0/1896082850 learned_addr learned my addr 172.21.15.19:0/1896082850 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.653+0000 7ff842d9d700 1 --2- 172.21.15.19:0/1896082850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff844108740 0x7ff8441a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.653+0000 7ff84259c700 1 --2- 172.21.15.19:0/1896082850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8441034c0 0x7ff84419d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.653+0000 7ff841d9b700 1 -- 172.21.15.19:0/1896082850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8441034c0 msgr2=0x7ff84419d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.653+0000 7ff841d9b700 1 --2- 172.21.15.19:0/1896082850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8441034c0 0x7ff84419d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.653+0000 7ff841d9b700 1 -- 172.21.15.19:0/1896082850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff844108740 msgr2=0x7ff8441a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.653+0000 7ff841d9b700 1 --2- 172.21.15.19:0/1896082850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff844108740 0x7ff8441a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:50.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.653+0000 7ff841d9b700 1 -- 172.21.15.19:0/1896082850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8441a1fa0 con 0x7ff844107df0 2024-05-09T02:10:50.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.653+0000 7ff84259c700 1 --2- 172.21.15.19:0/1896082850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8441034c0 0x7ff84419d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:10:50.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.653+0000 7ff841d9b700 1 --2- 172.21.15.19:0/1896082850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff844107df0 0x7ff84419d670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff83800d980 tx=0x7ff83800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:50.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.654+0000 7ff8337fe700 1 -- 172.21.15.19:0/1896082850 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838017070 con 0x7ff844107df0 2024-05-09T02:10:50.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.654+0000 7ff84886d700 1 -- 172.21.15.19:0/1896082850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8441a2290 con 0x7ff844107df0 2024-05-09T02:10:50.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.654+0000 7ff8337fe700 1 -- 172.21.15.19:0/1896082850 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff838009de0 con 0x7ff844107df0 2024-05-09T02:10:50.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.654+0000 7ff8337fe700 1 -- 172.21.15.19:0/1896082850 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83800f3c0 con 0x7ff844107df0 2024-05-09T02:10:50.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.654+0000 7ff84886d700 1 -- 172.21.15.19:0/1896082850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8441a9bb0 con 0x7ff844107df0 2024-05-09T02:10:50.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.655+0000 7ff84886d700 1 -- 172.21.15.19:0/1896082850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8440512c0 con 0x7ff844107df0 2024-05-09T02:10:50.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.657+0000 7ff8337fe700 1 -- 172.21.15.19:0/1896082850 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff838009930 con 0x7ff844107df0 2024-05-09T02:10:50.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.658+0000 7ff8337fe700 1 --2- 172.21.15.19:0/1896082850 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff82c0b2880 0x7ff82c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:50.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.658+0000 7ff84259c700 1 --2- 172.21.15.19:0/1896082850 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff82c0b2880 0x7ff82c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:50.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.658+0000 7ff8337fe700 1 -- 172.21.15.19:0/1896082850 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7ff8380e1dc0 con 0x7ff844107df0 2024-05-09T02:10:50.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.659+0000 7ff84259c700 1 --2- 172.21.15.19:0/1896082850 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff82c0b2880 0x7ff82c0b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff834000c00 tx=0x7ff83401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:50.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.662+0000 7ff8337fe700 1 -- 172.21.15.19:0/1896082850 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff8380a9a60 con 0x7ff844107df0 2024-05-09T02:10:50.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.927+0000 7ff84886d700 1 -- 172.21.15.19:0/1896082850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff84419e430 con 0x7ff844107df0 2024-05-09T02:10:50.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.928+0000 7ff8337fe700 1 -- 172.21.15.19:0/1896082850 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11917 (secure 0 0 0) 0x7ff8380a9360 con 0x7ff844107df0 2024-05-09T02:10:50.930 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:10:50.931 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:08:11.346147+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24455,24457,24473,24484,24463]},"id":1}]} 2024-05-09T02:10:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.930+0000 7ff84886d700 1 -- 172.21.15.19:0/1896082850 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff82c0b2880 msgr2=0x7ff82c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.930+0000 7ff84886d700 1 --2- 172.21.15.19:0/1896082850 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff82c0b2880 0x7ff82c0b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff834000c00 tx=0x7ff83401a040 comp rx=0 tx=0).stop 2024-05-09T02:10:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.930+0000 7ff84886d700 1 -- 172.21.15.19:0/1896082850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff844107df0 msgr2=0x7ff84419d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.930+0000 7ff84886d700 1 --2- 172.21.15.19:0/1896082850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff844107df0 0x7ff84419d670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff83800d980 tx=0x7ff83800dc90 comp rx=0 tx=0).stop 2024-05-09T02:10:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.930+0000 7ff84886d700 1 -- 172.21.15.19:0/1896082850 shutdown_connections 2024-05-09T02:10:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.930+0000 7ff84886d700 1 --2- 172.21.15.19:0/1896082850 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ff82c0b2880 0x7ff82c0b4d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.930+0000 7ff84886d700 1 --2- 172.21.15.19:0/1896082850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff844108740 0x7ff8441a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.930+0000 7ff84886d700 1 --2- 172.21.15.19:0/1896082850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8441034c0 0x7ff84419d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.930+0000 7ff84886d700 1 --2- 172.21.15.19:0/1896082850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff844107df0 0x7ff84419d670 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.930+0000 7ff84886d700 1 -- 172.21.15.19:0/1896082850 >> 172.21.15.19:0/1896082850 conn(0x7ff8440fedf0 msgr2=0x7ff8441065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.930+0000 7ff84886d700 1 -- 172.21.15.19:0/1896082850 shutdown_connections 2024-05-09T02:10:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:50.930+0000 7ff84886d700 1 -- 172.21.15.19:0/1896082850 wait complete. 2024-05-09T02:10:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-05-09T02:10:51.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:51 smithi196 ceph-mon[34718]: pgmap v384: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:10:51.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:51 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1896082850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:10:51.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:51 smithi019 ceph-mon[30151]: pgmap v384: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:10:51.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:51 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1896082850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:10:51.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:51 smithi059 ceph-mon[34781]: pgmap v384: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:10:51.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:51 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1896082850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:10:51.740 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T02:10:52.001 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:53.479 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:53 smithi019 ceph-mon[30151]: pgmap v385: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:53.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:53 smithi196 ceph-mon[34718]: pgmap v385: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:53.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:53 smithi059 ceph-mon[34781]: pgmap v385: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.753+0000 7f34bd7a1700 1 -- 172.21.15.19:0/2327714823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b8108f00 msgr2=0x7f34b8109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.753+0000 7f34bd7a1700 1 --2- 172.21.15.19:0/2327714823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b8108f00 0x7f34b8109310 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0099c0 tx=0x7f34ac009cd0 comp rx=0 tx=0).stop 2024-05-09T02:10:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.753+0000 7f34bd7a1700 1 -- 172.21.15.19:0/2327714823 shutdown_connections 2024-05-09T02:10:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.753+0000 7f34bd7a1700 1 --2- 172.21.15.19:0/2327714823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34b8109850 0x7f34b81111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.753+0000 7f34bd7a1700 1 --2- 172.21.15.19:0/2327714823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b81045d0 0x7f34b81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.753+0000 7f34bd7a1700 1 --2- 172.21.15.19:0/2327714823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b8108f00 0x7f34b8109310 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.753+0000 7f34bd7a1700 1 -- 172.21.15.19:0/2327714823 >> 172.21.15.19:0/2327714823 conn(0x7f34b80ffea0 msgr2=0x7f34b81022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.753+0000 7f34bd7a1700 1 -- 172.21.15.19:0/2327714823 shutdown_connections 2024-05-09T02:10:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.753+0000 7f34bd7a1700 1 -- 172.21.15.19:0/2327714823 wait complete. 2024-05-09T02:10:53.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.754+0000 7f34bd7a1700 1 Processor -- start 2024-05-09T02:10:53.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.754+0000 7f34bd7a1700 1 -- start start 2024-05-09T02:10:53.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.754+0000 7f34bd7a1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b81045d0 0x7f34b8076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:53.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.754+0000 7f34bd7a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b8108f00 0x7f34b8077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:53.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.754+0000 7f34bd7a1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34b8109850 0x7f34b8077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.754+0000 7f34bd7a1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34b8114460 con 0x7f34b8108f00 2024-05-09T02:10:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.754+0000 7f34bd7a1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f34b81142e0 con 0x7f34b8109850 2024-05-09T02:10:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.754+0000 7f34bd7a1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34b81145e0 con 0x7f34b81045d0 2024-05-09T02:10:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.754+0000 7f34b67fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b8108f00 0x7f34b8077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.754+0000 7f34b67fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b8108f00 0x7f34b8077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39198/0 (socket says 172.21.15.19:39198) 2024-05-09T02:10:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.755+0000 7f34b67fc700 1 -- 172.21.15.19:0/2791426153 learned_addr learned my addr 172.21.15.19:0/2791426153 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.755+0000 7f34b6ffd700 1 --2- 172.21.15.19:0/2791426153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b81045d0 0x7f34b8076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.755+0000 7f34b77fe700 1 --2- 172.21.15.19:0/2791426153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34b8109850 0x7f34b8077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.755+0000 7f34b67fc700 1 -- 172.21.15.19:0/2791426153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b81045d0 msgr2=0x7f34b8076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.755+0000 7f34b67fc700 1 --2- 172.21.15.19:0/2791426153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b81045d0 0x7f34b8076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.755+0000 7f34b67fc700 1 -- 172.21.15.19:0/2791426153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34b8109850 msgr2=0x7f34b8077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.755+0000 7f34b67fc700 1 --2- 172.21.15.19:0/2791426153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34b8109850 0x7f34b8077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.755+0000 7f34b67fc700 1 -- 172.21.15.19:0/2791426153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b81a91e0 con 0x7f34b8108f00 2024-05-09T02:10:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.755+0000 7f34b67fc700 1 --2- 172.21.15.19:0/2791426153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b8108f00 0x7f34b8077110 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f34ac006010 tx=0x7f34ac0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.755+0000 7f349ffff700 1 -- 172.21.15.19:0/2791426153 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac027070 con 0x7f34b8108f00 2024-05-09T02:10:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.755+0000 7f349ffff700 1 -- 172.21.15.19:0/2791426153 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34ac003f20 con 0x7f34b8108f00 2024-05-09T02:10:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.755+0000 7f34bd7a1700 1 -- 172.21.15.19:0/2791426153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b81a9470 con 0x7f34b8108f00 2024-05-09T02:10:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.755+0000 7f349ffff700 1 -- 172.21.15.19:0/2791426153 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac022cd0 con 0x7f34b8108f00 2024-05-09T02:10:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.755+0000 7f34bd7a1700 1 -- 172.21.15.19:0/2791426153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b81a98c0 con 0x7f34b8108f00 2024-05-09T02:10:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.756+0000 7f34bd7a1700 1 -- 172.21.15.19:0/2791426153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b80512c0 con 0x7f34b8108f00 2024-05-09T02:10:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.758+0000 7f349ffff700 1 -- 172.21.15.19:0/2791426153 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f34ac02a070 con 0x7f34b8108f00 2024-05-09T02:10:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.759+0000 7f349ffff700 1 --2- 172.21.15.19:0/2791426153 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f34a00b2880 0x7f34a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.759+0000 7f34b6ffd700 1 --2- 172.21.15.19:0/2791426153 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f34a00b2880 0x7f34a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.759+0000 7f349ffff700 1 -- 172.21.15.19:0/2791426153 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f34ac0e34c0 con 0x7f34b8108f00 2024-05-09T02:10:53.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.759+0000 7f34b6ffd700 1 --2- 172.21.15.19:0/2791426153 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f34a00b2880 0x7f34a00b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f34a8005950 tx=0x7f34a800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.762+0000 7f349ffff700 1 -- 172.21.15.19:0/2791426153 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f34ac031080 con 0x7f34b8108f00 2024-05-09T02:10:53.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.978+0000 7f34bd7a1700 1 -- 172.21.15.19:0/2791426153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34b802e5f0 con 0x7f34b8108f00 2024-05-09T02:10:53.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.979+0000 7f349ffff700 1 -- 172.21.15.19:0/2791426153 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22698 (secure 0 0 0) 0x7f34ac013070 con 0x7f34b8108f00 2024-05-09T02:10:53.983 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:10:53.983 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"fsid":"a0611f3e-0da7-11ef-bc97-c7b262605968","created":"2024-05-09T01:59:27.591343+0000","modified":"2024-05-09T02:08:25.302927+0000","last_up_change":"2024-05-09T02:06:45.565065+0000","last_in_change":"2024-05-09T02:06:24.967186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T02:02:50.018959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T02:07:43.283868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T02:07:47.110614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"996c6a3d-3e3a-451c-8789-36006e0b3dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6803","nonce":3002373576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6805","nonce":3002373576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6809","nonce":3002373576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6807","nonce":3002373576}]},"public_addr":"172.21.15.19:6803/3002373576","cluster_addr":"172.21.15.19:6805/3002373576","heartbeat_back_addr":"172.21.15.19:6809/3002373576","heartbeat_front_addr":"172.21.15.19:6807/3002373576","state":["exists","up"]},{"osd":1,"uuid":"1ac68d09-1198-4e81-86c7-e37f7c41aa5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6801","nonce":827458353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6803","nonce":827458353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6807","nonce":827458353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6805","nonce":827458353}]},"public_addr":"172.21.15.59:6801/827458353","cluster_addr":"172.21.15.59:6803/827458353","heartbeat_back_addr":"172.21.15.59:6807/827458353","heartbeat_front_addr":"172.21.15.59:6805/827458353","state":["exists","up"]},{"osd":2,"uuid":"818a0de1-04e6-471d-83df-a68833011b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6801","nonce":1659504639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6803","nonce":1659504639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6807","nonce":1659504639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6805","nonce":1659504639}]},"public_addr":"172.21.15.196:6801/1659504639","cluster_addr":"172.21.15.196:6803/1659504639","heartbeat_back_addr":"172.21.15.196:6807/1659504639","heartbeat_front_addr":"172.21.15.196:6805/1659504639","state":["exists","up"]},{"osd":3,"uuid":"7b204854-86b8-454d-aee0-8bb5775ce828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6811","nonce":2707671599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6813","nonce":2707671599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6817","nonce":2707671599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6815","nonce":2707671599}]},"public_addr":"172.21.15.19:6811/2707671599","cluster_addr":"172.21.15.19:6813/2707671599","heartbeat_back_addr":"172.21.15.19:6817/2707671599","heartbeat_front_addr":"172.21.15.19:6815/2707671599","state":["exists","up"]},{"osd":4,"uuid":"091667f1-e887-4a94-8723-6f85b676fef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6809","nonce":223037330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6811","nonce":223037330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6815","nonce":223037330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6813","nonce":223037330}]},"public_addr":"172.21.15.59:6809/223037330","cluster_addr":"172.21.15.59:6811/223037330","heartbeat_back_addr":"172.21.15.59:6815/223037330","heartbeat_front_addr":"172.21.15.59:6813/223037330","state":["exists","up"]},{"osd":5,"uuid":"2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6809","nonce":2516161898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6811","nonce":2516161898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6815","nonce":2516161898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6813","nonce":2516161898}]},"public_addr":"172.21.15.196:6809/2516161898","cluster_addr":"172.21.15.196:6811/2516161898","heartbeat_back_addr":"172.21.15.196:6815/2516161898","heartbeat_front_addr":"172.21.15.196:6813/2516161898","state":["exists","up"]},{"osd":6,"uuid":"4e352851-1279-475f-ba8b-eaf9ca73df74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6819","nonce":2150451637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6821","nonce":2150451637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6825","nonce":2150451637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6823","nonce":2150451637}]},"public_addr":"172.21.15.19:6819/2150451637","cluster_addr":"172.21.15.19:6821/2150451637","heartbeat_back_addr":"172.21.15.19:6825/2150451637","heartbeat_front_addr":"172.21.15.19:6823/2150451637","state":["exists","up"]},{"osd":7,"uuid":"83f53f59-9a60-4d87-9a8a-67e5c6f57004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6817","nonce":967902053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6819","nonce":967902053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6823","nonce":967902053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6821","nonce":967902053}]},"public_addr":"172.21.15.59:6817/967902053","cluster_addr":"172.21.15.59:6819/967902053","heartbeat_back_addr":"172.21.15.59:6823/967902053","heartbeat_front_addr":"172.21.15.59:6821/967902053","state":["exists","up"]},{"osd":8,"uuid":"a8f1bfb5-1cc8-4a65-8396-239bd781e1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6817","nonce":2946064259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6819","nonce":2946064259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6823","nonce":2946064259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6821","nonce":2946064259}]},"public_addr":"172.21.15.196:6817/2946064259","cluster_addr":"172.21.15.196:6819/2946064259","heartbeat_back_addr":"172.21.15.196:6823/2946064259","heartbeat_front_addr":"172.21.15.196:6821/2946064259","state":["exists","up"]},{"osd":9,"uuid":"73c58dc3-fcb0-4440-92b3-bf41a2e58fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6827","nonce":1735017886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6829","nonce":1735017886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6833","nonce":1735017886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6831","nonce":1735017886}]},"public_addr":"172.21.15.19:6827/1735017886","cluster_addr":"172.21.15.19:6829/1735017886","heartbeat_back_addr":"172.21.15.19:6833/1735017886","heartbeat_front_addr":"172.21.15.19:6831/1735017886","state":["exists","up"]},{"osd":10,"uuid":"8325282f-7325-4880-ab7d-b972717be9dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6825","nonce":2932899995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6827","nonce":2932899995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6831","nonce":2932899995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6829","nonce":2932899995}]},"public_addr":"172.21.15.59:6825/2932899995","cluster_addr":"172.21.15.59:6827/2932899995","heartbeat_back_addr":"172.21.15.59:6831/2932899995","heartbeat_front_addr":"172.21.15.59:6829/2932899995","state":["exists","up"]},{"osd":11,"uuid":"8b19228c-b286-43c8-a408-169782f13075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6825","nonce":2608023001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6827","nonce":2608023001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6831","nonce":2608023001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6829","nonce":2608023001}]},"public_addr":"172.21.15.196:6825/2608023001","cluster_addr":"172.21.15.196:6827/2608023001","heartbeat_back_addr":"172.21.15.196:6831/2608023001","heartbeat_front_addr":"172.21.15.196:6829/2608023001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:01:59.457748+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:23.640666+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:47.768243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:13.016746+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:39.060898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:04.508313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:31.366217+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:57.579758+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:23.177396+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:50.966399+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:16.838666+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:43.959387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3291838961":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1024133854":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1176455101":"2024-05-10T02:00:23.920230+0000","172.21.15.19:6801/2101052000":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/4018987404":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/3079777324":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6801/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/179703528":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/2101052000":"2024-05-10T02:00:23.920230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T02:10:53.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.982+0000 7f34bd7a1700 1 -- 172.21.15.19:0/2791426153 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f34a00b2880 msgr2=0x7f34a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:53.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.982+0000 7f34bd7a1700 1 --2- 172.21.15.19:0/2791426153 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f34a00b2880 0x7f34a00b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f34a8005950 tx=0x7f34a800b410 comp rx=0 tx=0).stop 2024-05-09T02:10:53.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.982+0000 7f34bd7a1700 1 -- 172.21.15.19:0/2791426153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b8108f00 msgr2=0x7f34b8077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:53.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.982+0000 7f34bd7a1700 1 --2- 172.21.15.19:0/2791426153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b8108f00 0x7f34b8077110 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f34ac006010 tx=0x7f34ac0047b0 comp rx=0 tx=0).stop 2024-05-09T02:10:53.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.982+0000 7f34bd7a1700 1 -- 172.21.15.19:0/2791426153 shutdown_connections 2024-05-09T02:10:53.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.982+0000 7f34bd7a1700 1 --2- 172.21.15.19:0/2791426153 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f34a00b2880 0x7f34a00b4d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:53.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.982+0000 7f34bd7a1700 1 --2- 172.21.15.19:0/2791426153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34b8109850 0x7f34b8077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:53.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.982+0000 7f34bd7a1700 1 --2- 172.21.15.19:0/2791426153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b81045d0 0x7f34b8076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:53.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.982+0000 7f34bd7a1700 1 --2- 172.21.15.19:0/2791426153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b8108f00 0x7f34b8077110 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:53.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.982+0000 7f34bd7a1700 1 -- 172.21.15.19:0/2791426153 >> 172.21.15.19:0/2791426153 conn(0x7f34b80ffea0 msgr2=0x7f34b810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:53.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.982+0000 7f34bd7a1700 1 -- 172.21.15.19:0/2791426153 shutdown_connections 2024-05-09T02:10:53.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:53.982+0000 7f34bd7a1700 1 -- 172.21.15.19:0/2791426153 wait complete. 2024-05-09T02:10:54.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:54 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2791426153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:10:54.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:54 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2791426153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:10:54.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:54 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2791426153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:10:54.794 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T02:07:50.316306+0000', 'modified': '2024-05-09T02:08:11.346147+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24484, 'mds_1': 24473, 'mds_2': 24463, 'mds_3': 24457, 'mds_4': 24455}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14604': {'gid': 14604, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6839/810391856', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 810391856}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 810391856}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14616': {'gid': 14616, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6841/3162604494', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 3162604494}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 3162604494}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6833/2040289281', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6832', 'nonce': 2040289281}, {'type': 'v1', 'addr': '172.21.15.59:6833', 'nonce': 2040289281}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/416853517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 416853517}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 416853517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6835/978263217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 978263217}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 978263217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.59:6837/2268419175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 2268419175}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2268419175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.196:6835/442694937', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 442694937}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 442694937}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.196:6837/1033177921', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 1033177921}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 1033177921}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.59:6839/1206568409', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6838', 'nonce': 1206568409}, {'type': 'v1', 'addr': '172.21.15.59:6839', 'nonce': 1206568409}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.196:6839/89436664', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 89436664}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 89436664}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24484, 'qdb_cluster': [24455, 24457, 24473, 24484, 24463]} 2024-05-09T02:10:54.794 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-09T02:10:54.794 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-09T02:10:55.063 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:55.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:55 smithi019 ceph-mon[30151]: pgmap v386: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:10:55.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:10:55.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:10:55.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:10:55.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:10:55.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:55 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:10:55.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:55 smithi196 ceph-mon[34718]: pgmap v386: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:10:55.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:10:55.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:10:55.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:10:55.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:10:55.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:55 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:10:55.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:55 smithi059 ceph-mon[34781]: pgmap v386: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:10:55.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:10:55.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:10:55.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:10:55.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:10:55.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:55 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:10:56.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:56 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:10:56.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:56 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:10:56.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:56 smithi196 ceph-mon[34718]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:10:56.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:56 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:10:56.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:56 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:10:56.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:56 smithi019 ceph-mon[30151]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:10:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:56 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:10:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:56 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:10:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:56 smithi059 ceph-mon[34781]: from='mgr.14152 172.21.15.19:0/633830642' entity='mgr.x' 2024-05-09T02:10:56.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.843+0000 7f6a4660a700 1 -- 172.21.15.19:0/3503657788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a40107df0 msgr2=0x7f6a40108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:56.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.843+0000 7f6a4660a700 1 --2- 172.21.15.19:0/3503657788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a40107df0 0x7f6a40108200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6a280099c0 tx=0x7f6a28009cd0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.843+0000 7f6a4660a700 1 -- 172.21.15.19:0/3503657788 shutdown_connections 2024-05-09T02:10:56.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.843+0000 7f6a4660a700 1 --2- 172.21.15.19:0/3503657788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a40108740 0x7f6a401100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.843+0000 7f6a4660a700 1 --2- 172.21.15.19:0/3503657788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a401034c0 0x7f6a401038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.843+0000 7f6a4660a700 1 --2- 172.21.15.19:0/3503657788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a40107df0 0x7f6a40108200 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.843+0000 7f6a4660a700 1 -- 172.21.15.19:0/3503657788 >> 172.21.15.19:0/3503657788 conn(0x7f6a400fedf0 msgr2=0x7f6a40101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:56.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.843+0000 7f6a4660a700 1 -- 172.21.15.19:0/3503657788 shutdown_connections 2024-05-09T02:10:56.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.843+0000 7f6a4660a700 1 -- 172.21.15.19:0/3503657788 wait complete. 2024-05-09T02:10:56.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.844+0000 7f6a4660a700 1 Processor -- start 2024-05-09T02:10:56.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.844+0000 7f6a4660a700 1 -- start start 2024-05-09T02:10:56.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.844+0000 7f6a4660a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a401034c0 0x7f6a4019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.844+0000 7f6a4660a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a40107df0 0x7f6a4019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.844+0000 7f6a4660a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a40108740 0x7f6a4019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.844+0000 7f6a4660a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a40113350 con 0x7f6a40108740 2024-05-09T02:10:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.844+0000 7f6a4660a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a401131d0 con 0x7f6a401034c0 2024-05-09T02:10:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.844+0000 7f6a4660a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a401134d0 con 0x7f6a40107df0 2024-05-09T02:10:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a44ba7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a40108740 0x7f6a4019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a44ba7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a40108740 0x7f6a4019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44088/0 (socket says 172.21.15.19:44088) 2024-05-09T02:10:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a44ba7700 1 -- 172.21.15.19:0/540450298 learned_addr learned my addr 172.21.15.19:0/540450298 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a37fff700 1 --2- 172.21.15.19:0/540450298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a40107df0 0x7f6a4019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a3ffff700 1 --2- 172.21.15.19:0/540450298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a401034c0 0x7f6a4019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a44ba7700 1 -- 172.21.15.19:0/540450298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a40107df0 msgr2=0x7f6a4019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a44ba7700 1 --2- 172.21.15.19:0/540450298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a40107df0 0x7f6a4019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a44ba7700 1 -- 172.21.15.19:0/540450298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a401034c0 msgr2=0x7f6a4019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a44ba7700 1 --2- 172.21.15.19:0/540450298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a401034c0 0x7f6a4019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a44ba7700 1 -- 172.21.15.19:0/540450298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4019fd40 con 0x7f6a40108740 2024-05-09T02:10:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a44ba7700 1 --2- 172.21.15.19:0/540450298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a40108740 0x7f6a4019f6a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6a38007ee0 tx=0x7f6a3800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a3dffb700 1 -- 172.21.15.19:0/540450298 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800ddd0 con 0x7f6a40108740 2024-05-09T02:10:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a3dffb700 1 -- 172.21.15.19:0/540450298 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a38004020 con 0x7f6a40108740 2024-05-09T02:10:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a4660a700 1 -- 172.21.15.19:0/540450298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a401a0030 con 0x7f6a40108740 2024-05-09T02:10:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a3dffb700 1 -- 172.21.15.19:0/540450298 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38015660 con 0x7f6a40108740 2024-05-09T02:10:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.845+0000 7f6a4660a700 1 -- 172.21.15.19:0/540450298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a401a7950 con 0x7f6a40108740 2024-05-09T02:10:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.848+0000 7f6a3dffb700 1 -- 172.21.15.19:0/540450298 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6a38015800 con 0x7f6a40108740 2024-05-09T02:10:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.849+0000 7f6a3dffb700 1 --2- 172.21.15.19:0/540450298 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6a2c0b2880 0x7f6a2c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.849+0000 7f6a3ffff700 1 --2- 172.21.15.19:0/540450298 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6a2c0b2880 0x7f6a2c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.849+0000 7f6a3dffb700 1 -- 172.21.15.19:0/540450298 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f6a380e1cc0 con 0x7f6a40108740 2024-05-09T02:10:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.849+0000 7f6a55020b80 1 Processor -- start 2024-05-09T02:10:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.849+0000 7f6a3ffff700 1 --2- 172.21.15.19:0/540450298 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6a2c0b2880 0x7f6a2c0b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6a30005fd0 tx=0x7f6a3000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.849+0000 7f6a55020b80 1 -- start start 2024-05-09T02:10:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a55020b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf1a9110 0x5638cf2424a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a55020b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf0bf450 0x5638cf2429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a55020b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf23c650 0x5638cf23ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a55020b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5638cf15efa0 con 0x5638cf0bf450 2024-05-09T02:10:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a55020b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5638cf0fd7a0 con 0x5638cf1a9110 2024-05-09T02:10:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a55020b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5638cf15af60 con 0x5638cf23c650 2024-05-09T02:10:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a37fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf0bf450 0x5638cf2429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a37fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf0bf450 0x5638cf2429e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44094/0 (socket says 172.21.15.19:44094) 2024-05-09T02:10:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a37fff700 1 -- 172.21.15.19:0/278310791 learned_addr learned my addr 172.21.15.19:0/278310791 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a3ffff700 1 --2- 172.21.15.19:0/278310791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf1a9110 0x5638cf2424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a44ba7700 1 --2- 172.21.15.19:0/278310791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf23c650 0x5638cf23ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a37fff700 1 -- 172.21.15.19:0/278310791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf23c650 msgr2=0x5638cf23ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a37fff700 1 --2- 172.21.15.19:0/278310791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf23c650 0x5638cf23ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a37fff700 1 -- 172.21.15.19:0/278310791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf1a9110 msgr2=0x5638cf2424a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a37fff700 1 --2- 172.21.15.19:0/278310791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf1a9110 0x5638cf2424a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a37fff700 1 -- 172.21.15.19:0/278310791 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638cf0f5320 con 0x5638cf0bf450 2024-05-09T02:10:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a37fff700 1 --2- 172.21.15.19:0/278310791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf0bf450 0x5638cf2429e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6a40076b40 tx=0x7f6a280037b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9904d2c733436cac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.850+0000 7f6a367fc700 1 -- 172.21.15.19:0/278310791 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28003c50 con 0x5638cf0bf450 2024-05-09T02:10:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.851+0000 7f6a367fc700 1 -- 172.21.15.19:0/278310791 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a28003df0 con 0x5638cf0bf450 2024-05-09T02:10:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.851+0000 7f6a367fc700 1 -- 172.21.15.19:0/278310791 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2800be00 con 0x5638cf0bf450 2024-05-09T02:10:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.851+0000 7f6a55020b80 1 -- 172.21.15.19:0/278310791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf0bf450 msgr2=0x5638cf2429e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.851+0000 7f6a55020b80 1 --2- 172.21.15.19:0/278310791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf0bf450 0x5638cf2429e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6a40076b40 tx=0x7f6a280037b0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.851+0000 7f6a55020b80 1 -- 172.21.15.19:0/278310791 shutdown_connections 2024-05-09T02:10:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.851+0000 7f6a55020b80 1 --2- 172.21.15.19:0/278310791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf23c650 0x5638cf23ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.851+0000 7f6a55020b80 1 --2- 172.21.15.19:0/278310791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf1a9110 0x5638cf2424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.851+0000 7f6a55020b80 1 --2- 172.21.15.19:0/278310791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf0bf450 0x5638cf2429e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.852+0000 7f6a55020b80 1 -- 172.21.15.19:0/278310791 >> 172.21.15.19:0/278310791 conn(0x5638cf11f650 msgr2=0x5638cf2373b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.852+0000 7f6a55020b80 1 -- 172.21.15.19:0/278310791 shutdown_connections 2024-05-09T02:10:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.852+0000 7f6a55020b80 1 -- 172.21.15.19:0/278310791 wait complete. 2024-05-09T02:10:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.852+0000 7f6a55020b80 1 -- start start 2024-05-09T02:10:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.852+0000 7f6a55020b80 1 Processor -- start 2024-05-09T02:10:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a55020b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf0bf450 0x5638cf246080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a55020b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf1a9110 0x5638cf2465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a55020b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf23c650 0x5638cf246b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a55020b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5638cefdab50 con 0x5638cf0bf450 2024-05-09T02:10:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a55020b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5638cf119100 con 0x5638cf1a9110 2024-05-09T02:10:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a55020b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5638cf23d490 con 0x5638cf23c650 2024-05-09T02:10:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a55020b80 10 client.0 _ll_register_callbacks cb 0x5638cefd0670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:10:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a3ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf0bf450 0x5638cf246080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a3ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf0bf450 0x5638cf246080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44104/0 (socket says 172.21.15.19:44104) 2024-05-09T02:10:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a3ffff700 1 -- 172.21.15.19:0/2369665861 learned_addr learned my addr 172.21.15.19:0/2369665861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a37fff700 1 --2- 172.21.15.19:0/2369665861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf1a9110 0x5638cf2465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a44ba7700 1 --2- 172.21.15.19:0/2369665861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf23c650 0x5638cf246b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a3ffff700 1 -- 172.21.15.19:0/2369665861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf23c650 msgr2=0x5638cf246b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a3ffff700 1 --2- 172.21.15.19:0/2369665861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf23c650 0x5638cf246b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a3ffff700 1 -- 172.21.15.19:0/2369665861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf1a9110 msgr2=0x5638cf2465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a3ffff700 1 --2- 172.21.15.19:0/2369665861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf1a9110 0x5638cf2465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a3ffff700 1 -- 172.21.15.19:0/2369665861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638cefda440 con 0x5638cf0bf450 2024-05-09T02:10:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a37fff700 1 --2- 172.21.15.19:0/2369665861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf1a9110 0x5638cf2465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:10:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a44ba7700 1 --2- 172.21.15.19:0/2369665861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf23c650 0x5638cf246b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:10:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a3ffff700 1 --2- 172.21.15.19:0/2369665861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf0bf450 0x5638cf246080 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6a300106b0 tx=0x7f6a30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a5ddba70693c9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.853+0000 7f6a1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.854+0000 7f6a1f7fe700 1 -- 172.21.15.19:0/2369665861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30004a30 con 0x5638cf0bf450 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.854+0000 7f6a1f7fe700 1 -- 172.21.15.19:0/2369665861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f6a30004bd0 con 0x5638cf0bf450 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.854+0000 7f6a55020b80 1 -- 172.21.15.19:0/2369665861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a28003df0 con 0x5638cf0bf450 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.854+0000 7f6a1f7fe700 1 -- 172.21.15.19:0/2369665861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6a3002b070 con 0x5638cf0bf450 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.854+0000 7f6a55020b80 10 client.15120 fetch_fsmap learned FSMap version 28 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.854+0000 7f6a55020b80 1 -- 172.21.15.19:0/2369665861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638cf253a80 con 0x5638cf0bf450 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.854+0000 7f6a1f7fe700 1 -- 172.21.15.19:0/2369665861 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a28003df0 con 0x5638cf0bf450 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.854+0000 7f6a1f7fe700 1 -- 172.21.15.19:0/2369665861 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6a3001d990 con 0x5638cf0bf450 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.854+0000 7f6a55020b80 10 client.15120 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.854+0000 7f6a55020b80 10 client.15120 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.854+0000 7f6a55020b80 20 client.15120 populate_metadata read hostname 'smithi019' 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.854+0000 7f6a55020b80 1 --2- 172.21.15.19:0/2369665861 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5638cf256170 0x5638cf2575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.854+0000 7f6a55020b80 4 client.15120 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.854+0000 7f6a55020b80 1 -- 172.21.15.19:0/2369665861 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638cf16c980 con 0x5638cf256170 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.855+0000 7f6a37fff700 1 --2- 172.21.15.19:0/2369665861 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5638cf256170 0x5638cf2575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.855+0000 7f6a37fff700 1 --2- 172.21.15.19:0/2369665861 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5638cf256170 0x5638cf2575e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b57343adeef6619c server_cookie=6a12bd62c055db05 in_seq=0 out_seq=0 2024-05-09T02:10:56.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.855+0000 7f6a1f7fe700 10 client.15120 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:10:56.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.856+0000 7f6a1f7fe700 1 -- 172.21.15.19:0/2369665861 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5638cf16c980 con 0x5638cf256170 2024-05-09T02:10:56.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.856+0000 7f6a1f7fe700 10 client.15120 handle_command_reply: tid=0 2024-05-09T02:10:56.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.856+0000 7f6a55020b80 1 client.15120 shutdown 2024-05-09T02:10:56.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.856+0000 7f6a55020b80 1 -- 172.21.15.19:0/2369665861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf0bf450 msgr2=0x5638cf246080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:56.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.856+0000 7f6a55020b80 1 --2- 172.21.15.19:0/2369665861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf0bf450 0x5638cf246080 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6a300106b0 tx=0x7f6a30004750 comp rx=0 tx=0).stop 2024-05-09T02:10:56.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.856+0000 7f6a55020b80 1 -- 172.21.15.19:0/2369665861 shutdown_connections 2024-05-09T02:10:56.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.856+0000 7f6a55020b80 1 --2- 172.21.15.19:0/2369665861 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5638cf256170 0x5638cf2575e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.856+0000 7f6a55020b80 1 --2- 172.21.15.19:0/2369665861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf23c650 0x5638cf246b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.856+0000 7f6a55020b80 1 --2- 172.21.15.19:0/2369665861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf0bf450 0x5638cf246080 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.856+0000 7f6a1f7fe700 0 client.15120 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:10:56.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.856+0000 7f6a55020b80 1 --2- 172.21.15.19:0/2369665861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf1a9110 0x5638cf2465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.856+0000 7f6a55020b80 1 -- 172.21.15.19:0/2369665861 >> 172.21.15.19:0/2369665861 conn(0x5638cf11f650 msgr2=0x5638cf237170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:56.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.856+0000 7f6a55020b80 1 -- 172.21.15.19:0/2369665861 shutdown_connections 2024-05-09T02:10:56.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.856+0000 7f6a55020b80 1 -- 172.21.15.19:0/2369665861 wait complete. 2024-05-09T02:10:56.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.856+0000 7f6a55020b80 20 client.15120 trim_cache size 0 max 16384 2024-05-09T02:10:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.877+0000 7f6a55020b80 1 Processor -- start 2024-05-09T02:10:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a55020b80 1 -- start start 2024-05-09T02:10:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a55020b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf0bf450 0x5638cf2522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a55020b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf1a9110 0x5638cf2404c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a55020b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf23c650 0x5638cf260a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a55020b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5638cf255920 con 0x5638cf1a9110 2024-05-09T02:10:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a55020b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5638cf255ff0 con 0x5638cf23c650 2024-05-09T02:10:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a55020b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5638cf2566c0 con 0x5638cf0bf450 2024-05-09T02:10:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a37fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf1a9110 0x5638cf2404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a37fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf1a9110 0x5638cf2404c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44110/0 (socket says 172.21.15.19:44110) 2024-05-09T02:10:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a37fff700 1 -- 172.21.15.19:0/801128512 learned_addr learned my addr 172.21.15.19:0/801128512 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a3ffff700 1 --2- 172.21.15.19:0/801128512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf0bf450 0x5638cf2522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a37fff700 1 -- 172.21.15.19:0/801128512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf0bf450 msgr2=0x5638cf2522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a37fff700 1 --2- 172.21.15.19:0/801128512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf0bf450 0x5638cf2522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a37fff700 1 -- 172.21.15.19:0/801128512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf23c650 msgr2=0x5638cf260a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a37fff700 1 --2- 172.21.15.19:0/801128512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf23c650 0x5638cf260a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a37fff700 1 -- 172.21.15.19:0/801128512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638cf238cb0 con 0x5638cf1a9110 2024-05-09T02:10:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.878+0000 7f6a37fff700 1 --2- 172.21.15.19:0/801128512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf1a9110 0x5638cf2404c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6a28024040 tx=0x7f6a28009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b268c76b306f3e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.879+0000 7f6a35ffb700 1 -- 172.21.15.19:0/801128512 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2800bbd0 con 0x5638cf1a9110 2024-05-09T02:10:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.879+0000 7f6a35ffb700 1 -- 172.21.15.19:0/801128512 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a28036030 con 0x5638cf1a9110 2024-05-09T02:10:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.879+0000 7f6a35ffb700 1 -- 172.21.15.19:0/801128512 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a280313d0 con 0x5638cf1a9110 2024-05-09T02:10:56.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.879+0000 7f6a55020b80 1 -- 172.21.15.19:0/801128512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf1a9110 msgr2=0x5638cf2404c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:56.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.879+0000 7f6a55020b80 1 --2- 172.21.15.19:0/801128512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf1a9110 0x5638cf2404c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6a28024040 tx=0x7f6a28009f10 comp rx=0 tx=0).stop 2024-05-09T02:10:56.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.879+0000 7f6a55020b80 1 -- 172.21.15.19:0/801128512 shutdown_connections 2024-05-09T02:10:56.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.879+0000 7f6a55020b80 1 --2- 172.21.15.19:0/801128512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf23c650 0x5638cf260a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.879+0000 7f6a55020b80 1 --2- 172.21.15.19:0/801128512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf0bf450 0x5638cf2522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.879+0000 7f6a55020b80 1 --2- 172.21.15.19:0/801128512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf1a9110 0x5638cf2404c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.879+0000 7f6a55020b80 1 -- 172.21.15.19:0/801128512 >> 172.21.15.19:0/801128512 conn(0x5638cf11f650 msgr2=0x5638cf23fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:56.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.880+0000 7f6a55020b80 1 -- 172.21.15.19:0/801128512 shutdown_connections 2024-05-09T02:10:56.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.880+0000 7f6a55020b80 1 -- 172.21.15.19:0/801128512 wait complete. 2024-05-09T02:10:56.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.880+0000 7f6a55020b80 1 -- start start 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.880+0000 7f6a55020b80 1 Processor -- start 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.880+0000 7f6a55020b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf0bf450 0x5638cf245d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.880+0000 7f6a55020b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf1a9110 0x5638cf2462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.880+0000 7f6a55020b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf23c650 0x5638cf2467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.880+0000 7f6a55020b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5638cf257460 con 0x5638cf1a9110 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.880+0000 7f6a55020b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5638cf256d90 con 0x5638cf23c650 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.880+0000 7f6a55020b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5638cf257b30 con 0x5638cf0bf450 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a55020b80 10 client.0 _ll_register_callbacks cb 0x5638cefd0670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a37fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf1a9110 0x5638cf2462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a37fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf1a9110 0x5638cf2462a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44120/0 (socket says 172.21.15.19:44120) 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a37fff700 1 -- 172.21.15.19:0/1971769890 learned_addr learned my addr 172.21.15.19:0/1971769890 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a3ffff700 1 --2- 172.21.15.19:0/1971769890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf0bf450 0x5638cf245d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a44ba7700 1 --2- 172.21.15.19:0/1971769890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf23c650 0x5638cf2467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a37fff700 1 -- 172.21.15.19:0/1971769890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf0bf450 msgr2=0x5638cf245d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a37fff700 1 --2- 172.21.15.19:0/1971769890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf0bf450 0x5638cf245d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a37fff700 1 -- 172.21.15.19:0/1971769890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf23c650 msgr2=0x5638cf2467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a37fff700 1 --2- 172.21.15.19:0/1971769890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf23c650 0x5638cf2467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a37fff700 1 -- 172.21.15.19:0/1971769890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638cf25eff0 con 0x5638cf1a9110 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a37fff700 1 --2- 172.21.15.19:0/1971769890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf1a9110 0x5638cf2462a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6a280098c0 tx=0x7f6a280059e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822a42c04061dc8f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a1f7fe700 1 -- 172.21.15.19:0/1971769890 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28040070 con 0x5638cf1a9110 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a1f7fe700 1 -- 172.21.15.19:0/1971769890 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f6a280050d0 con 0x5638cf1a9110 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.881+0000 7f6a55020b80 1 -- 172.21.15.19:0/1971769890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a28036030 con 0x5638cf1a9110 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.882+0000 7f6a1f7fe700 1 -- 172.21.15.19:0/1971769890 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6a2804c050 con 0x5638cf1a9110 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.882+0000 7f6a55020b80 10 client.15126 fetch_fsmap learned FSMap version 28 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.882+0000 7f6a55020b80 1 -- 172.21.15.19:0/1971769890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638cf2c1930 con 0x5638cf1a9110 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.882+0000 7f6a1f7fe700 1 -- 172.21.15.19:0/1971769890 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a28036030 con 0x5638cf1a9110 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.882+0000 7f6a1f7fe700 1 -- 172.21.15.19:0/1971769890 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6a280315d0 con 0x5638cf1a9110 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.882+0000 7f6a55020b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.882+0000 7f6a55020b80 10 client.15126 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.882+0000 7f6a55020b80 20 client.15126 populate_metadata read hostname 'smithi019' 2024-05-09T02:10:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.882+0000 7f6a55020b80 1 --2- 172.21.15.19:0/1971769890 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5638cf2c3640 0x5638cf2c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.882+0000 7f6a55020b80 4 client.15126 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.882+0000 7f6a55020b80 1 -- 172.21.15.19:0/1971769890 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "status"}) -- 0x5638cf166a20 con 0x5638cf2c3640 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.882+0000 7f6a3ffff700 1 --2- 172.21.15.19:0/1971769890 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5638cf2c3640 0x5638cf2c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a1f7fe700 10 client.15126 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a3ffff700 1 --2- 172.21.15.19:0/1971769890 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5638cf2c3640 0x5638cf2c4ab0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=101d950d4bd35096 server_cookie=147baab03417ff2f in_seq=0 out_seq=0 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a1f7fe700 1 -- 172.21.15.19:0/1971769890 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5638cf166a20 con 0x5638cf2c3640 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a1f7fe700 10 client.15126 handle_command_reply: tid=0 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a55020b80 1 client.15126 shutdown 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a55020b80 1 -- 172.21.15.19:0/1971769890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf1a9110 msgr2=0x5638cf2462a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a55020b80 1 --2- 172.21.15.19:0/1971769890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf1a9110 0x5638cf2462a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6a280098c0 tx=0x7f6a280059e0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a55020b80 1 -- 172.21.15.19:0/1971769890 shutdown_connections 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a55020b80 1 --2- 172.21.15.19:0/1971769890 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5638cf2c3640 0x5638cf2c4ab0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a55020b80 1 --2- 172.21.15.19:0/1971769890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638cf23c650 0x5638cf2467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a55020b80 1 --2- 172.21.15.19:0/1971769890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638cf0bf450 0x5638cf245d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a1f7fe700 0 client.15126 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:10:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a55020b80 1 --2- 172.21.15.19:0/1971769890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5638cf1a9110 0x5638cf2462a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a55020b80 1 -- 172.21.15.19:0/1971769890 >> 172.21.15.19:0/1971769890 conn(0x5638cf11f650 msgr2=0x5638cf240540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:56.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a55020b80 1 -- 172.21.15.19:0/1971769890 shutdown_connections 2024-05-09T02:10:56.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.883+0000 7f6a55020b80 1 -- 172.21.15.19:0/1971769890 wait complete. 2024-05-09T02:10:56.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.884+0000 7f6a55020b80 20 client.15126 trim_cache size 0 max 16384 2024-05-09T02:10:56.890 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:10:56.890 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "a0611f3e-0da7-11ef-bc97-c7b262605968", 2024-05-09T02:10:56.890 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-05-09T02:10:56.890 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24455, 2024-05-09T02:10:56.890 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T02:10:56.890 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T02:10:56.890 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T02:10:56.890 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 169.63128977700001, 2024-05-09T02:10:56.891 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-05-09T02:10:56.891 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 81, 2024-05-09T02:10:56.891 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 79, 2024-05-09T02:10:56.891 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 214.030110998 2024-05-09T02:10:56.891 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:10:56.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.884+0000 7f6a4660a700 1 -- 172.21.15.19:0/540450298 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6a2c0b2880 msgr2=0x7f6a2c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:56.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.884+0000 7f6a4660a700 1 --2- 172.21.15.19:0/540450298 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6a2c0b2880 0x7f6a2c0b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6a30005fd0 tx=0x7f6a3000a560 comp rx=0 tx=0).stop 2024-05-09T02:10:56.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.884+0000 7f6a4660a700 1 -- 172.21.15.19:0/540450298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a40108740 msgr2=0x7f6a4019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:56.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.884+0000 7f6a4660a700 1 --2- 172.21.15.19:0/540450298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a40108740 0x7f6a4019f6a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6a38007ee0 tx=0x7f6a3800d3b0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.884+0000 7f6a4660a700 1 -- 172.21.15.19:0/540450298 shutdown_connections 2024-05-09T02:10:56.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.884+0000 7f6a4660a700 1 --2- 172.21.15.19:0/540450298 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f6a2c0b2880 0x7f6a2c0b4d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.884+0000 7f6a4660a700 1 --2- 172.21.15.19:0/540450298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a40108740 0x7f6a4019f6a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.884+0000 7f6a4660a700 1 --2- 172.21.15.19:0/540450298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a401034c0 0x7f6a4019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.884+0000 7f6a4660a700 1 --2- 172.21.15.19:0/540450298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a40107df0 0x7f6a4019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:56.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.884+0000 7f6a4660a700 1 -- 172.21.15.19:0/540450298 >> 172.21.15.19:0/540450298 conn(0x7f6a400fedf0 msgr2=0x7f6a40075d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:56.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.884+0000 7f6a4660a700 1 -- 172.21.15.19:0/540450298 shutdown_connections 2024-05-09T02:10:56.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:56.884+0000 7f6a4660a700 1 -- 172.21.15.19:0/540450298 wait complete. 2024-05-09T02:10:57.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:57 smithi196 ceph-mon[34718]: pgmap v387: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:10:57.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:57 smithi019 ceph-mon[30151]: pgmap v387: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:10:57.577 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-09T02:10:57.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:57 smithi059 ceph-mon[34781]: pgmap v387: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:10:57.831 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:10:59.537 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:10:59 smithi019 ceph-mon[30151]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:59.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:10:59 smithi196 ceph-mon[34718]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:10:59.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.567+0000 7ffbbd598700 1 -- 172.21.15.19:0/3165976749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbb8108740 msgr2=0x7ffbb81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:59.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.567+0000 7ffbbd598700 1 --2- 172.21.15.19:0/3165976749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbb8108740 0x7ffbb81100a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000b0d0 tx=0x7ffbb000b3e0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.567+0000 7ffbbd598700 1 -- 172.21.15.19:0/3165976749 shutdown_connections 2024-05-09T02:10:59.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.567+0000 7ffbbd598700 1 --2- 172.21.15.19:0/3165976749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbb8108740 0x7ffbb81100a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.567+0000 7ffbbd598700 1 --2- 172.21.15.19:0/3165976749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbb81034c0 0x7ffbb81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.567+0000 7ffbbd598700 1 --2- 172.21.15.19:0/3165976749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbb8107df0 0x7ffbb8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.567+0000 7ffbbd598700 1 -- 172.21.15.19:0/3165976749 >> 172.21.15.19:0/3165976749 conn(0x7ffbb80fedc0 msgr2=0x7ffbb81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:59.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.567+0000 7ffbbd598700 1 -- 172.21.15.19:0/3165976749 shutdown_connections 2024-05-09T02:10:59.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.567+0000 7ffbbd598700 1 -- 172.21.15.19:0/3165976749 wait complete. 2024-05-09T02:10:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.568+0000 7ffbbd598700 1 Processor -- start 2024-05-09T02:10:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.568+0000 7ffbbd598700 1 -- start start 2024-05-09T02:10:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.568+0000 7ffbbd598700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbb81034c0 0x7ffbb819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.568+0000 7ffbbd598700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbb8107df0 0x7ffbb819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.568+0000 7ffbbd598700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbb8108740 0x7ffbb819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.568+0000 7ffbbd598700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb8113370 con 0x7ffbb8108740 2024-05-09T02:10:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.568+0000 7ffbbd598700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81131f0 con 0x7ffbb81034c0 2024-05-09T02:10:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.568+0000 7ffbbd598700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81134f0 con 0x7ffbb8107df0 2024-05-09T02:10:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffbb77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbb8108740 0x7ffbb819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffbb77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbb8108740 0x7ffbb819f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44146/0 (socket says 172.21.15.19:44146) 2024-05-09T02:10:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffbb77fe700 1 -- 172.21.15.19:0/4249694477 learned_addr learned my addr 172.21.15.19:0/4249694477 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffbb6ffd700 1 --2- 172.21.15.19:0/4249694477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbb81034c0 0x7ffbb819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffbb67fc700 1 --2- 172.21.15.19:0/4249694477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbb8107df0 0x7ffbb819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffbb77fe700 1 -- 172.21.15.19:0/4249694477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbb8107df0 msgr2=0x7ffbb819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffbb77fe700 1 --2- 172.21.15.19:0/4249694477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbb8107df0 0x7ffbb819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffbb77fe700 1 -- 172.21.15.19:0/4249694477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbb81034c0 msgr2=0x7ffbb819adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffbb77fe700 1 --2- 172.21.15.19:0/4249694477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbb81034c0 0x7ffbb819adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffbb77fe700 1 -- 172.21.15.19:0/4249694477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb819fbd0 con 0x7ffbb8108740 2024-05-09T02:10:59.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffbb67fc700 1 --2- 172.21.15.19:0/4249694477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbb8107df0 0x7ffbb819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:10:59.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffbb77fe700 1 --2- 172.21.15.19:0/4249694477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbb8108740 0x7ffbb819f5e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00095c0 tx=0x7ffbb0007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:59.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffb9ffff700 1 -- 172.21.15.19:0/4249694477 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb001f070 con 0x7ffbb8108740 2024-05-09T02:10:59.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffbbd598700 1 -- 172.21.15.19:0/4249694477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb819fe60 con 0x7ffbb8108740 2024-05-09T02:10:59.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffb9ffff700 1 -- 172.21.15.19:0/4249694477 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbb0003c00 con 0x7ffbb8108740 2024-05-09T02:10:59.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.569+0000 7ffb9ffff700 1 -- 172.21.15.19:0/4249694477 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb001b6a0 con 0x7ffbb8108740 2024-05-09T02:10:59.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.570+0000 7ffbbd598700 1 -- 172.21.15.19:0/4249694477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb81a0340 con 0x7ffbb8108740 2024-05-09T02:10:59.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.572+0000 7ffb9ffff700 1 -- 172.21.15.19:0/4249694477 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffbb0022070 con 0x7ffbb8108740 2024-05-09T02:10:59.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.573+0000 7ffbcbfaeb80 1 Processor -- start 2024-05-09T02:10:59.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.573+0000 7ffb9ffff700 1 --2- 172.21.15.19:0/4249694477 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffba00b2880 0x7ffba00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.573+0000 7ffbcbfaeb80 1 -- start start 2024-05-09T02:10:59.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.573+0000 7ffbb67fc700 1 --2- 172.21.15.19:0/4249694477 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffba00b2880 0x7ffba00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.573+0000 7ffb9ffff700 1 -- 172.21.15.19:0/4249694477 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7ffbb000f020 con 0x7ffbb8108740 2024-05-09T02:10:59.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbcbfaeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad5f6dcb0 0x55dad60761f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbcbfaeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad6076730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbcbfaeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad60703b0 0x55dad60707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbcbfaeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dad5f93050 con 0x55dad5ef3450 2024-05-09T02:10:59.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbcbfaeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dad5f38950 con 0x55dad60703b0 2024-05-09T02:10:59.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbcbfaeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dad5f8f010 con 0x55dad5f6dcb0 2024-05-09T02:10:59.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbb67fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad6076730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbb67fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad6076730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44160/0 (socket says 172.21.15.19:44160) 2024-05-09T02:10:59.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbb67fc700 1 -- 172.21.15.19:0/1090964881 learned_addr learned my addr 172.21.15.19:0/1090964881 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:59.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbb6ffd700 1 --2- 172.21.15.19:0/1090964881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad5f6dcb0 0x55dad60761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbb67fc700 1 --2- 172.21.15.19:0/4249694477 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffba00b2880 0x7ffba00b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0135f0 tx=0x7ffbac0139b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:59.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbb77fe700 1 --2- 172.21.15.19:0/1090964881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad60703b0 0x55dad60707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbb67fc700 1 -- 172.21.15.19:0/1090964881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad5f6dcb0 msgr2=0x55dad60761f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:59.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbb67fc700 1 --2- 172.21.15.19:0/1090964881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad5f6dcb0 0x55dad60761f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbb67fc700 1 -- 172.21.15.19:0/1090964881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad60703b0 msgr2=0x55dad60707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:59.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbb67fc700 1 --2- 172.21.15.19:0/1090964881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad60703b0 0x55dad60707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbb67fc700 1 -- 172.21.15.19:0/1090964881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dad5f294b0 con 0x55dad5ef3450 2024-05-09T02:10:59.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbb77fe700 1 --2- 172.21.15.19:0/1090964881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad60703b0 0x55dad60707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:10:59.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffbb67fc700 1 --2- 172.21.15.19:0/1090964881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad6076730 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00f5f0 tx=0x7ffbac01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5950f6c7bfb11007 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:59.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.574+0000 7ffb9d7fa700 1 -- 172.21.15.19:0/1090964881 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac00fc90 con 0x55dad5ef3450 2024-05-09T02:10:59.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.575+0000 7ffb9d7fa700 1 -- 172.21.15.19:0/1090964881 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbac00fe30 con 0x55dad5ef3450 2024-05-09T02:10:59.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.575+0000 7ffb9d7fa700 1 -- 172.21.15.19:0/1090964881 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac003e50 con 0x55dad5ef3450 2024-05-09T02:10:59.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.575+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/1090964881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 msgr2=0x55dad6076730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.575+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/1090964881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad6076730 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00f5f0 tx=0x7ffbac01d000 comp rx=0 tx=0).stop 2024-05-09T02:10:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.575+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/1090964881 shutdown_connections 2024-05-09T02:10:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.575+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/1090964881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad60703b0 0x55dad60707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.575+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/1090964881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad5f6dcb0 0x55dad60761f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.575+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/1090964881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad6076730 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.575+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/1090964881 >> 172.21.15.19:0/1090964881 conn(0x55dad5ef6f20 msgr2=0x55dad606b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.575+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/1090964881 shutdown_connections 2024-05-09T02:10:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.576+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/1090964881 wait complete. 2024-05-09T02:10:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.576+0000 7ffbcbfaeb80 1 -- start start 2024-05-09T02:10:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.576+0000 7ffbcbfaeb80 1 Processor -- start 2024-05-09T02:10:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.576+0000 7ffbcbfaeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad6079fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.576+0000 7ffbcbfaeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad5f6dcb0 0x55dad607a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.576+0000 7ffbcbfaeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad60703b0 0x55dad607aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.576+0000 7ffbcbfaeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dad60710b0 con 0x55dad5ef3450 2024-05-09T02:10:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.576+0000 7ffbcbfaeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dad5f2dad0 con 0x55dad5f6dcb0 2024-05-09T02:10:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.576+0000 7ffbcbfaeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dad6086180 con 0x55dad60703b0 2024-05-09T02:10:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.576+0000 7ffbb6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad6079fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.576+0000 7ffbcbfaeb80 10 client.0 _ll_register_callbacks cb 0x55dad5e04670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:10:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.576+0000 7ffbb6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad6079fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44176/0 (socket says 172.21.15.19:44176) 2024-05-09T02:10:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.576+0000 7ffbb6ffd700 1 -- 172.21.15.19:0/1836995883 learned_addr learned my addr 172.21.15.19:0/1836995883 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.577+0000 7ffbb67fc700 1 --2- 172.21.15.19:0/1836995883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad5f6dcb0 0x55dad607a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.577+0000 7ffbb77fe700 1 --2- 172.21.15.19:0/1836995883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad60703b0 0x55dad607aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.577+0000 7ffbb6ffd700 1 -- 172.21.15.19:0/1836995883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad60703b0 msgr2=0x55dad607aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.577+0000 7ffbb6ffd700 1 --2- 172.21.15.19:0/1836995883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad60703b0 0x55dad607aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.577+0000 7ffbb6ffd700 1 -- 172.21.15.19:0/1836995883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad5f6dcb0 msgr2=0x55dad607a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.577+0000 7ffbb6ffd700 1 --2- 172.21.15.19:0/1836995883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad5f6dcb0 0x55dad607a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.577+0000 7ffbb6ffd700 1 -- 172.21.15.19:0/1836995883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dad5e0e440 con 0x55dad5ef3450 2024-05-09T02:10:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.577+0000 7ffbb6ffd700 1 --2- 172.21.15.19:0/1836995883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad6079fa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffba8010a20 tx=0x7ffba8010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04731ab03023226 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.577+0000 7ffb9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.577+0000 7ffb9aff5700 1 -- 172.21.15.19:0/1836995883 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba801e060 con 0x55dad5ef3450 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.577+0000 7ffb9aff5700 1 -- 172.21.15.19:0/1836995883 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7ffba800fa20 con 0x55dad5ef3450 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.577+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/1836995883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbac00fe30 con 0x55dad5ef3450 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.578+0000 7ffb9aff5700 1 -- 172.21.15.19:0/1836995883 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffba800fc20 con 0x55dad5ef3450 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.578+0000 7ffbcbfaeb80 10 client.15138 fetch_fsmap learned FSMap version 28 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.578+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/1836995883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dad6087a50 con 0x55dad5ef3450 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.578+0000 7ffb9aff5700 1 -- 172.21.15.19:0/1836995883 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbac00fe30 con 0x55dad5ef3450 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.578+0000 7ffb9aff5700 1 -- 172.21.15.19:0/1836995883 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ffba8003b60 con 0x55dad5ef3450 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.578+0000 7ffbcbfaeb80 10 client.15138 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.578+0000 7ffbcbfaeb80 10 client.15138 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.578+0000 7ffbcbfaeb80 20 client.15138 populate_metadata read hostname 'smithi019' 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.578+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/1836995883 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55dad608a040 0x55dad608b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.578+0000 7ffbcbfaeb80 4 client.15138 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.578+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/1836995883 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dad5fa0a30 con 0x55dad608a040 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.578+0000 7ffbb67fc700 1 --2- 172.21.15.19:0/1836995883 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55dad608a040 0x55dad608b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.579+0000 7ffbb67fc700 1 --2- 172.21.15.19:0/1836995883 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55dad608a040 0x55dad608b4b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6db3503df7e6a4b1 server_cookie=31ae09c56b4f99d9 in_seq=0 out_seq=0 2024-05-09T02:10:59.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.579+0000 7ffb9aff5700 10 client.15138 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:10:59.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.580+0000 7ffb9aff5700 1 -- 172.21.15.19:0/1836995883 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dad5fa0a30 con 0x55dad608a040 2024-05-09T02:10:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.580+0000 7ffb9aff5700 10 client.15138 handle_command_reply: tid=0 2024-05-09T02:10:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.580+0000 7ffbcbfaeb80 1 client.15138 shutdown 2024-05-09T02:10:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.580+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/1836995883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 msgr2=0x55dad6079fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.580+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/1836995883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad6079fa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffba8010a20 tx=0x7ffba8010d30 comp rx=0 tx=0).stop 2024-05-09T02:10:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.580+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/1836995883 shutdown_connections 2024-05-09T02:10:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.580+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/1836995883 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55dad608a040 0x55dad608b4b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.580+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/1836995883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad60703b0 0x55dad607aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.580+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/1836995883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad6079fa0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.580+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/1836995883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad5f6dcb0 0x55dad607a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.580+0000 7ffb9aff5700 0 client.15138 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:10:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.580+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/1836995883 >> 172.21.15.19:0/1836995883 conn(0x55dad5ef6f20 msgr2=0x55dad6064fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.580+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/1836995883 shutdown_connections 2024-05-09T02:10:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.580+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/1836995883 wait complete. 2024-05-09T02:10:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.580+0000 7ffbcbfaeb80 20 client.15138 trim_cache size 0 max 16384 2024-05-09T02:10:59.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.601+0000 7ffbcbfaeb80 1 Processor -- start 2024-05-09T02:10:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.601+0000 7ffbcbfaeb80 1 -- start start 2024-05-09T02:10:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbcbfaeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad607a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbcbfaeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad5f6dcb0 0x55dad60860f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbcbfaeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad60703b0 0x55dad6078fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbcbfaeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dad60897f0 con 0x55dad5ef3450 2024-05-09T02:10:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbcbfaeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dad6089ec0 con 0x55dad60703b0 2024-05-09T02:10:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbcbfaeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dad608a590 con 0x55dad5f6dcb0 2024-05-09T02:10:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbb6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad607a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbb6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad607a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44178/0 (socket says 172.21.15.19:44178) 2024-05-09T02:10:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbb6ffd700 1 -- 172.21.15.19:0/3149692914 learned_addr learned my addr 172.21.15.19:0/3149692914 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbb67fc700 1 --2- 172.21.15.19:0/3149692914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad5f6dcb0 0x55dad60860f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbb6ffd700 1 -- 172.21.15.19:0/3149692914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad5f6dcb0 msgr2=0x55dad60860f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbb6ffd700 1 --2- 172.21.15.19:0/3149692914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad5f6dcb0 0x55dad60860f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbb6ffd700 1 -- 172.21.15.19:0/3149692914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad60703b0 msgr2=0x55dad6078fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:59.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbb77fe700 1 --2- 172.21.15.19:0/3149692914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad60703b0 0x55dad6078fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbb6ffd700 1 --2- 172.21.15.19:0/3149692914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad60703b0 0x55dad6078fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbb6ffd700 1 -- 172.21.15.19:0/3149692914 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dad6067980 con 0x55dad5ef3450 2024-05-09T02:10:59.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.602+0000 7ffbb6ffd700 1 --2- 172.21.15.19:0/3149692914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad607a860 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ffba800fbb0 tx=0x7ffba800a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec28905385983703 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:59.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.603+0000 7ffb9cff9700 1 -- 172.21.15.19:0/3149692914 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba8028040 con 0x55dad5ef3450 2024-05-09T02:10:59.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.603+0000 7ffb9cff9700 1 -- 172.21.15.19:0/3149692914 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffba800fe80 con 0x55dad5ef3450 2024-05-09T02:10:59.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.603+0000 7ffb9cff9700 1 -- 172.21.15.19:0/3149692914 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba80235a0 con 0x55dad5ef3450 2024-05-09T02:10:59.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.603+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/3149692914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 msgr2=0x55dad607a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:59.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/3149692914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad607a860 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ffba800fbb0 tx=0x7ffba800a3f0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/3149692914 shutdown_connections 2024-05-09T02:10:59.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/3149692914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad60703b0 0x55dad6078fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/3149692914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5ef3450 0x55dad607a860 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/3149692914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad5f6dcb0 0x55dad60860f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/3149692914 >> 172.21.15.19:0/3149692914 conn(0x55dad5ef6f20 msgr2=0x55dad6078890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:59.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/3149692914 shutdown_connections 2024-05-09T02:10:59.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/3149692914 wait complete. 2024-05-09T02:10:59.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 -- start start 2024-05-09T02:10:59.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 Processor -- start 2024-05-09T02:10:59.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad5ef3450 0x55dad607a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5f6dcb0 0x55dad607a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad60703b0 0x55dad6078890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dad608b330 con 0x55dad5f6dcb0 2024-05-09T02:10:59.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dad608ac60 con 0x55dad5ef3450 2024-05-09T02:10:59.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.604+0000 7ffbcbfaeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dad608ba00 con 0x55dad60703b0 2024-05-09T02:10:59.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffbcbfaeb80 10 client.0 _ll_register_callbacks cb 0x55dad5e04670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:10:59.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffbb67fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5f6dcb0 0x55dad607a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffbb67fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5f6dcb0 0x55dad607a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44190/0 (socket says 172.21.15.19:44190) 2024-05-09T02:10:59.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffbb67fc700 1 -- 172.21.15.19:0/3788812379 learned_addr learned my addr 172.21.15.19:0/3788812379 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:10:59.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffbb6ffd700 1 --2- 172.21.15.19:0/3788812379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad5ef3450 0x55dad607a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffbb77fe700 1 --2- 172.21.15.19:0/3788812379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad60703b0 0x55dad6078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffbb67fc700 1 -- 172.21.15.19:0/3788812379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad60703b0 msgr2=0x55dad6078890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffbb67fc700 1 --2- 172.21.15.19:0/3788812379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad60703b0 0x55dad6078890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffbb67fc700 1 -- 172.21.15.19:0/3788812379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad5ef3450 msgr2=0x55dad607a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:10:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffbb67fc700 1 --2- 172.21.15.19:0/3788812379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad5ef3450 0x55dad607a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffbb67fc700 1 -- 172.21.15.19:0/3788812379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dad6078e80 con 0x55dad5f6dcb0 2024-05-09T02:10:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffbb77fe700 1 --2- 172.21.15.19:0/3788812379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad60703b0 0x55dad6078890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:10:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffbb67fc700 1 --2- 172.21.15.19:0/3788812379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5f6dcb0 0x55dad607a670 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00f5c0 tx=0x7ffbac00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf5b8d5869f6dd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:10:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffb9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:10:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffb9aff5700 1 -- 172.21.15.19:0/3788812379 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac035060 con 0x55dad5f6dcb0 2024-05-09T02:10:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffb9aff5700 1 -- 172.21.15.19:0/3788812379 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7ffbac00fdd0 con 0x55dad5f6dcb0 2024-05-09T02:10:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.605+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/3788812379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dad5f9aad0 con 0x55dad5f6dcb0 2024-05-09T02:10:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.606+0000 7ffb9aff5700 1 -- 172.21.15.19:0/3788812379 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffbac003680 con 0x55dad5f6dcb0 2024-05-09T02:10:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.606+0000 7ffbcbfaeb80 10 client.15144 fetch_fsmap learned FSMap version 28 2024-05-09T02:10:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.606+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/3788812379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dad60f9f00 con 0x55dad5f6dcb0 2024-05-09T02:10:59.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.606+0000 7ffb9aff5700 1 -- 172.21.15.19:0/3788812379 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dad5f9aad0 con 0x55dad5f6dcb0 2024-05-09T02:10:59.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.606+0000 7ffb9aff5700 1 -- 172.21.15.19:0/3788812379 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ffbac030ce0 con 0x55dad5f6dcb0 2024-05-09T02:10:59.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.606+0000 7ffbcbfaeb80 10 client.15144 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:10:59.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.606+0000 7ffbcbfaeb80 10 client.15144 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T02:10:59.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.606+0000 7ffbcbfaeb80 20 client.15144 populate_metadata read hostname 'smithi019' 2024-05-09T02:10:59.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.606+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/3788812379 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55dad60fc0b0 0x55dad60fd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:10:59.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.606+0000 7ffbcbfaeb80 4 client.15144 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T02:10:59.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.606+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/3788812379 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "status"}) -- 0x55dad5e0e920 con 0x55dad60fc0b0 2024-05-09T02:10:59.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.606+0000 7ffbb6ffd700 1 --2- 172.21.15.19:0/3788812379 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55dad60fc0b0 0x55dad60fd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:10:59.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffbb6ffd700 1 --2- 172.21.15.19:0/3788812379 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55dad60fc0b0 0x55dad60fd520 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b16e7494cf98b543 server_cookie=5d86f87c9a2d31ab in_seq=0 out_seq=0 2024-05-09T02:10:59.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffb9aff5700 10 client.15144 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:10:59.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffb9aff5700 1 -- 172.21.15.19:0/3788812379 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55dad5e0e920 con 0x55dad60fc0b0 2024-05-09T02:10:59.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffb9aff5700 10 client.15144 handle_command_reply: tid=0 2024-05-09T02:10:59.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffbcbfaeb80 1 client.15144 shutdown 2024-05-09T02:10:59.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/3788812379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5f6dcb0 msgr2=0x55dad607a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:59.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/3788812379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5f6dcb0 0x55dad607a670 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00f5c0 tx=0x7ffbac00bf90 comp rx=0 tx=0).stop 2024-05-09T02:10:59.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/3788812379 shutdown_connections 2024-05-09T02:10:59.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/3788812379 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55dad60fc0b0 0x55dad60fd520 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/3788812379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dad60703b0 0x55dad6078890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/3788812379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dad5ef3450 0x55dad607a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffb9aff5700 0 client.15144 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:10:59.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffbcbfaeb80 1 --2- 172.21.15.19:0/3788812379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dad5f6dcb0 0x55dad607a670 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/3788812379 >> 172.21.15.19:0/3788812379 conn(0x55dad5ef6f20 msgr2=0x55dad6073bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:59.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/3788812379 shutdown_connections 2024-05-09T02:10:59.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffbcbfaeb80 1 -- 172.21.15.19:0/3788812379 wait complete. 2024-05-09T02:10:59.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.607+0000 7ffbcbfaeb80 20 client.15144 trim_cache size 0 max 16384 2024-05-09T02:10:59.615 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:10:59.615 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "a0611f3e-0da7-11ef-bc97-c7b262605968", 2024-05-09T02:10:59.615 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-05-09T02:10:59.615 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24457, 2024-05-09T02:10:59.615 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T02:10:59.615 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T02:10:59.615 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T02:10:59.615 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 173.363960518, 2024-05-09T02:10:59.615 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-05-09T02:10:59.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 81, 2024-05-09T02:10:59.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 79, 2024-05-09T02:10:59.615 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 222.79757399600001 2024-05-09T02:10:59.615 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:10:59.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.608+0000 7ffbbd598700 1 -- 172.21.15.19:0/4249694477 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffba00b2880 msgr2=0x7ffba00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:59.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.608+0000 7ffbbd598700 1 --2- 172.21.15.19:0/4249694477 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffba00b2880 0x7ffba00b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0135f0 tx=0x7ffbac0139b0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.608+0000 7ffbbd598700 1 -- 172.21.15.19:0/4249694477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbb8108740 msgr2=0x7ffbb819f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:10:59.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.608+0000 7ffbbd598700 1 --2- 172.21.15.19:0/4249694477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbb8108740 0x7ffbb819f5e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00095c0 tx=0x7ffbb0007a20 comp rx=0 tx=0).stop 2024-05-09T02:10:59.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.608+0000 7ffbbd598700 1 -- 172.21.15.19:0/4249694477 shutdown_connections 2024-05-09T02:10:59.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.608+0000 7ffbbd598700 1 --2- 172.21.15.19:0/4249694477 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7ffba00b2880 0x7ffba00b4d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.608+0000 7ffbbd598700 1 --2- 172.21.15.19:0/4249694477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbb8108740 0x7ffbb819f5e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.608+0000 7ffbbd598700 1 --2- 172.21.15.19:0/4249694477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbb81034c0 0x7ffbb819adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.608+0000 7ffbbd598700 1 --2- 172.21.15.19:0/4249694477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbb8107df0 0x7ffbb819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:10:59.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.608+0000 7ffbbd598700 1 -- 172.21.15.19:0/4249694477 >> 172.21.15.19:0/4249694477 conn(0x7ffbb80fedc0 msgr2=0x7ffbb806b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:10:59.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.608+0000 7ffbbd598700 1 -- 172.21.15.19:0/4249694477 shutdown_connections 2024-05-09T02:10:59.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:10:59.608+0000 7ffbbd598700 1 -- 172.21.15.19:0/4249694477 wait complete. 2024-05-09T02:10:59.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:10:59 smithi059 ceph-mon[34781]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:11:00.352 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-09T02:11:00.605 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:11:01.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:01 smithi019 ceph-mon[30151]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:11:01.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:01 smithi196 ceph-mon[34718]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:11:01.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:01 smithi059 ceph-mon[34781]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T02:11:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.384+0000 7f6339524700 1 -- 172.21.15.19:0/1334003849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6334108780 msgr2=0x7f63341100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.384+0000 7f6339524700 1 --2- 172.21.15.19:0/1334003849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6334108780 0x7f63341100e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f632c00b0d0 tx=0x7f632c00b3e0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.384+0000 7f6339524700 1 -- 172.21.15.19:0/1334003849 shutdown_connections 2024-05-09T02:11:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.384+0000 7f6339524700 1 --2- 172.21.15.19:0/1334003849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6334108780 0x7f63341100e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.384+0000 7f6339524700 1 --2- 172.21.15.19:0/1334003849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6334103550 0x7f6334103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.384+0000 7f6339524700 1 --2- 172.21.15.19:0/1334003849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6334107e30 0x7f6334108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.384+0000 7f6339524700 1 -- 172.21.15.19:0/1334003849 >> 172.21.15.19:0/1334003849 conn(0x7f6334078e00 msgr2=0x7f6334079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.384+0000 7f6339524700 1 -- 172.21.15.19:0/1334003849 shutdown_connections 2024-05-09T02:11:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.385+0000 7f6339524700 1 -- 172.21.15.19:0/1334003849 wait complete. 2024-05-09T02:11:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.385+0000 7f6339524700 1 Processor -- start 2024-05-09T02:11:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.385+0000 7f6339524700 1 -- start start 2024-05-09T02:11:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f6339524700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6334103550 0x7f633419d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f6339524700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6334107e30 0x7f633419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f6339524700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6334108780 0x7f63341a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f6339524700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6334113420 con 0x7f6334103550 2024-05-09T02:11:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f6339524700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63341132a0 con 0x7f6334108780 2024-05-09T02:11:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f6339524700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63341135a0 con 0x7f6334107e30 2024-05-09T02:11:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f6332ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6334103550 0x7f633419d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f6332ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6334103550 0x7f633419d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44202/0 (socket says 172.21.15.19:44202) 2024-05-09T02:11:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f6332ffd700 1 -- 172.21.15.19:0/1860243726 learned_addr learned my addr 172.21.15.19:0/1860243726 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f63337fe700 1 --2- 172.21.15.19:0/1860243726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6334108780 0x7f63341a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f63327fc700 1 --2- 172.21.15.19:0/1860243726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6334107e30 0x7f633419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f6332ffd700 1 -- 172.21.15.19:0/1860243726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6334107e30 msgr2=0x7f633419d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f6332ffd700 1 --2- 172.21.15.19:0/1860243726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6334107e30 0x7f633419d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f6332ffd700 1 -- 172.21.15.19:0/1860243726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6334108780 msgr2=0x7f63341a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f6332ffd700 1 --2- 172.21.15.19:0/1860243726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6334108780 0x7f63341a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.386+0000 7f6332ffd700 1 -- 172.21.15.19:0/1860243726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63341a1f30 con 0x7f6334103550 2024-05-09T02:11:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.387+0000 7f6332ffd700 1 --2- 172.21.15.19:0/1860243726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6334103550 0x7f633419d110 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f632400c9b0 tx=0x7f632400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.387+0000 7f631bfff700 1 -- 172.21.15.19:0/1860243726 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324007b00 con 0x7f6334103550 2024-05-09T02:11:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.387+0000 7f6339524700 1 -- 172.21.15.19:0/1860243726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63341a2220 con 0x7f6334103550 2024-05-09T02:11:02.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.387+0000 7f631bfff700 1 -- 172.21.15.19:0/1860243726 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6324007ca0 con 0x7f6334103550 2024-05-09T02:11:02.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.387+0000 7f631bfff700 1 -- 172.21.15.19:0/1860243726 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324018660 con 0x7f6334103550 2024-05-09T02:11:02.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.387+0000 7f6339524700 1 -- 172.21.15.19:0/1860243726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63341a2760 con 0x7f6334103550 2024-05-09T02:11:02.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.390+0000 7f631bfff700 1 -- 172.21.15.19:0/1860243726 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6324004830 con 0x7f6334103550 2024-05-09T02:11:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.391+0000 7f631bfff700 1 --2- 172.21.15.19:0/1860243726 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f631c0b2880 0x7f631c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.391+0000 7f6347f3ab80 1 Processor -- start 2024-05-09T02:11:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.391+0000 7f63327fc700 1 --2- 172.21.15.19:0/1860243726 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f631c0b2880 0x7f631c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.391+0000 7f631bfff700 1 -- 172.21.15.19:0/1860243726 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f63240e2200 con 0x7f6334103550 2024-05-09T02:11:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.391+0000 7f6347f3ab80 1 -- start start 2024-05-09T02:11:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f6347f3ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a0991110 0x5588a0a2a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f6347f3ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a08a7450 0x5588a0a2a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f6347f3ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0a24650 0x5588a0a24a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f6347f3ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5588a0946fa0 con 0x5588a0a24650 2024-05-09T02:11:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f6347f3ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5588a08e57a0 con 0x5588a0991110 2024-05-09T02:11:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f6347f3ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5588a0942f60 con 0x5588a08a7450 2024-05-09T02:11:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f63327fc700 1 --2- 172.21.15.19:0/1860243726 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f631c0b2880 0x7f631c0b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6328006fd0 tx=0x7f6328008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f63337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0a24650 0x5588a0a24a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f63337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0a24650 0x5588a0a24a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44212/0 (socket says 172.21.15.19:44212) 2024-05-09T02:11:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f63337fe700 1 -- 172.21.15.19:0/2772476261 learned_addr learned my addr 172.21.15.19:0/2772476261 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f6332ffd700 1 --2- 172.21.15.19:0/2772476261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a0991110 0x5588a0a2a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f63327fc700 1 --2- 172.21.15.19:0/2772476261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a08a7450 0x5588a0a2a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f63337fe700 1 -- 172.21.15.19:0/2772476261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a08a7450 msgr2=0x5588a0a2a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f63337fe700 1 --2- 172.21.15.19:0/2772476261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a08a7450 0x5588a0a2a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f63337fe700 1 -- 172.21.15.19:0/2772476261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a0991110 msgr2=0x5588a0a2a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f63337fe700 1 --2- 172.21.15.19:0/2772476261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a0991110 0x5588a0a2a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.392+0000 7f63337fe700 1 -- 172.21.15.19:0/2772476261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a08dd320 con 0x5588a0a24650 2024-05-09T02:11:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.393+0000 7f63337fe700 1 --2- 172.21.15.19:0/2772476261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0a24650 0x5588a0a24a80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f632c005b80 tx=0x7f632c0090d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2063ceb4efa0c232 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.393+0000 7f63197fa700 1 -- 172.21.15.19:0/2772476261 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c003a00 con 0x5588a0a24650 2024-05-09T02:11:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.393+0000 7f63197fa700 1 -- 172.21.15.19:0/2772476261 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f632c003ba0 con 0x5588a0a24650 2024-05-09T02:11:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.393+0000 7f63197fa700 1 -- 172.21.15.19:0/2772476261 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c0052b0 con 0x5588a0a24650 2024-05-09T02:11:02.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.394+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2772476261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0a24650 msgr2=0x5588a0a24a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:02.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.394+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/2772476261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0a24650 0x5588a0a24a80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f632c005b80 tx=0x7f632c0090d0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.394+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2772476261 shutdown_connections 2024-05-09T02:11:02.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.394+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/2772476261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0a24650 0x5588a0a24a80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.394+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/2772476261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a0991110 0x5588a0a2a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.394+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/2772476261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a08a7450 0x5588a0a2a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.394+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2772476261 >> 172.21.15.19:0/2772476261 conn(0x5588a0907650 msgr2=0x5588a0a1f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:02.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.394+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2772476261 shutdown_connections 2024-05-09T02:11:02.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.394+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2772476261 wait complete. 2024-05-09T02:11:02.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.394+0000 7f6347f3ab80 1 -- start start 2024-05-09T02:11:02.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.394+0000 7f6347f3ab80 1 Processor -- start 2024-05-09T02:11:02.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f6347f3ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a08a7450 0x5588a0a2e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f6347f3ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0991110 0x5588a0a2e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f6347f3ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a0a24650 0x5588a0a2ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f6347f3ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5588a07c2b50 con 0x5588a0991110 2024-05-09T02:11:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f6347f3ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5588a0901100 con 0x5588a08a7450 2024-05-09T02:11:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f6347f3ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5588a0a31180 con 0x5588a0a24650 2024-05-09T02:11:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f6347f3ab80 10 client.0 _ll_register_callbacks cb 0x5588a07b8670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:11:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f63327fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0991110 0x5588a0a2e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f63327fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0991110 0x5588a0a2e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44228/0 (socket says 172.21.15.19:44228) 2024-05-09T02:11:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f63327fc700 1 -- 172.21.15.19:0/2441487483 learned_addr learned my addr 172.21.15.19:0/2441487483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f63337fe700 1 --2- 172.21.15.19:0/2441487483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a0a24650 0x5588a0a2ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f6332ffd700 1 --2- 172.21.15.19:0/2441487483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a08a7450 0x5588a0a2e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f63327fc700 1 -- 172.21.15.19:0/2441487483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a0a24650 msgr2=0x5588a0a2ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f63327fc700 1 --2- 172.21.15.19:0/2441487483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a0a24650 0x5588a0a2ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f63327fc700 1 -- 172.21.15.19:0/2441487483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a08a7450 msgr2=0x5588a0a2e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f63327fc700 1 --2- 172.21.15.19:0/2441487483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a08a7450 0x5588a0a2e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f63327fc700 1 -- 172.21.15.19:0/2441487483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a07c2440 con 0x5588a0991110 2024-05-09T02:11:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f63327fc700 1 --2- 172.21.15.19:0/2441487483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0991110 0x5588a0a2e690 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f632800ddd0 tx=0x7f6328004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4304824b2dfb5605 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f6316ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:11:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.395+0000 7f6316ff5700 1 -- 172.21.15.19:0/2441487483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6328024020 con 0x5588a0991110 2024-05-09T02:11:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.396+0000 7f6316ff5700 1 -- 172.21.15.19:0/2441487483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f632800f040 con 0x5588a0991110 2024-05-09T02:11:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.396+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2441487483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f632c003ba0 con 0x5588a0991110 2024-05-09T02:11:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.396+0000 7f6316ff5700 1 -- 172.21.15.19:0/2441487483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f632802f050 con 0x5588a0991110 2024-05-09T02:11:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.396+0000 7f6347f3ab80 10 client.15156 fetch_fsmap learned FSMap version 28 2024-05-09T02:11:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.396+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2441487483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588a0a3bb40 con 0x5588a0991110 2024-05-09T02:11:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.396+0000 7f6316ff5700 1 -- 172.21.15.19:0/2441487483 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63280208a0 con 0x5588a0991110 2024-05-09T02:11:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.396+0000 7f6316ff5700 1 -- 172.21.15.19:0/2441487483 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6328013070 con 0x5588a0991110 2024-05-09T02:11:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.396+0000 7f6347f3ab80 10 client.15156 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:11:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.396+0000 7f6347f3ab80 10 client.15156 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T02:11:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.396+0000 7f6347f3ab80 20 client.15156 populate_metadata read hostname 'smithi019' 2024-05-09T02:11:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.396+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/2441487483 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5588a0a3e1c0 0x5588a0a3f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.396+0000 7f6347f3ab80 4 client.15156 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:11:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.396+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2441487483 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f632c003ba0 con 0x5588a0a3e1c0 2024-05-09T02:11:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.397+0000 7f6332ffd700 1 --2- 172.21.15.19:0/2441487483 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5588a0a3e1c0 0x5588a0a3f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.397+0000 7f6332ffd700 1 --2- 172.21.15.19:0/2441487483 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5588a0a3e1c0 0x5588a0a3f630 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78c9357de08ab490 server_cookie=e7bf4ccee19de0b0 in_seq=0 out_seq=0 2024-05-09T02:11:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.397+0000 7f6316ff5700 10 client.15156 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:11:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.398+0000 7f6316ff5700 1 -- 172.21.15.19:0/2441487483 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f632c003ba0 con 0x5588a0a3e1c0 2024-05-09T02:11:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.398+0000 7f6316ff5700 10 client.15156 handle_command_reply: tid=0 2024-05-09T02:11:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.398+0000 7f6347f3ab80 1 client.15156 shutdown 2024-05-09T02:11:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.398+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2441487483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0991110 msgr2=0x5588a0a2e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.398+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/2441487483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0991110 0x5588a0a2e690 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f632800ddd0 tx=0x7f6328004860 comp rx=0 tx=0).stop 2024-05-09T02:11:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.398+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2441487483 shutdown_connections 2024-05-09T02:11:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.398+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/2441487483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a0a24650 0x5588a0a2ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.398+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/2441487483 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5588a0a3e1c0 0x5588a0a3f630 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.398+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/2441487483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a08a7450 0x5588a0a2e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.398+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/2441487483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0991110 0x5588a0a2e690 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.398+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2441487483 >> 172.21.15.19:0/2441487483 conn(0x5588a0907650 msgr2=0x5588a0a1f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.398+0000 7f6316ff5700 0 client.15156 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:11:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.398+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2441487483 shutdown_connections 2024-05-09T02:11:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.398+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2441487483 wait complete. 2024-05-09T02:11:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.398+0000 7f6347f3ab80 20 client.15156 trim_cache size 0 max 16384 2024-05-09T02:11:02.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f6347f3ab80 1 Processor -- start 2024-05-09T02:11:02.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f6347f3ab80 1 -- start start 2024-05-09T02:11:02.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f6347f3ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a08a7450 0x5588a0a2ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f6347f3ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a0991110 0x5588a0a2d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f6347f3ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0a24650 0x5588a0a27db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f6347f3ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5588a0a3d970 con 0x5588a0a24650 2024-05-09T02:11:02.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f6347f3ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5588a0a3e040 con 0x5588a0991110 2024-05-09T02:11:02.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f6347f3ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5588a0a3e710 con 0x5588a08a7450 2024-05-09T02:11:02.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f63337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0a24650 0x5588a0a27db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f63337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0a24650 0x5588a0a27db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44232/0 (socket says 172.21.15.19:44232) 2024-05-09T02:11:02.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f63337fe700 1 -- 172.21.15.19:0/2014594049 learned_addr learned my addr 172.21.15.19:0/2014594049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:02.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f63327fc700 1 --2- 172.21.15.19:0/2014594049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a0991110 0x5588a0a2d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f6332ffd700 1 --2- 172.21.15.19:0/2014594049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a08a7450 0x5588a0a2ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f63337fe700 1 -- 172.21.15.19:0/2014594049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a08a7450 msgr2=0x5588a0a2ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:02.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f63337fe700 1 --2- 172.21.15.19:0/2014594049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a08a7450 0x5588a0a2ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f63337fe700 1 -- 172.21.15.19:0/2014594049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a0991110 msgr2=0x5588a0a2d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:02.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f63337fe700 1 --2- 172.21.15.19:0/2014594049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a0991110 0x5588a0a2d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.420+0000 7f63337fe700 1 -- 172.21.15.19:0/2014594049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a0a24cb0 con 0x5588a0a24650 2024-05-09T02:11:02.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.421+0000 7f63337fe700 1 --2- 172.21.15.19:0/2014594049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0a24650 0x5588a0a27db0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f632c00b0d0 tx=0x7f632c009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85589e09e47edf91 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:02.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.421+0000 7f6318ff9700 1 -- 172.21.15.19:0/2014594049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c02b070 con 0x5588a0a24650 2024-05-09T02:11:02.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.421+0000 7f6318ff9700 1 -- 172.21.15.19:0/2014594049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f632c003c00 con 0x5588a0a24650 2024-05-09T02:11:02.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.421+0000 7f6318ff9700 1 -- 172.21.15.19:0/2014594049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c005420 con 0x5588a0a24650 2024-05-09T02:11:02.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.422+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2014594049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0a24650 msgr2=0x5588a0a27db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:02.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.422+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/2014594049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0a24650 0x5588a0a27db0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f632c00b0d0 tx=0x7f632c009480 comp rx=0 tx=0).stop 2024-05-09T02:11:02.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.422+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2014594049 shutdown_connections 2024-05-09T02:11:02.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.422+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/2014594049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a0a24650 0x5588a0a27db0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.422+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/2014594049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a08a7450 0x5588a0a2ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.422+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/2014594049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a0991110 0x5588a0a2d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.422+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2014594049 >> 172.21.15.19:0/2014594049 conn(0x5588a0907650 msgr2=0x5588a0a2ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:02.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.422+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2014594049 shutdown_connections 2024-05-09T02:11:02.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.422+0000 7f6347f3ab80 1 -- 172.21.15.19:0/2014594049 wait complete. 2024-05-09T02:11:02.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.422+0000 7f6347f3ab80 1 -- start start 2024-05-09T02:11:02.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.422+0000 7f6347f3ab80 1 Processor -- start 2024-05-09T02:11:02.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6347f3ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a08a7450 0x5588a0a2b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6347f3ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a0991110 0x5588a0a2b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6347f3ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a0a24650 0x5588a0a2bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6347f3ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5588a0a3f4b0 con 0x5588a08a7450 2024-05-09T02:11:02.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6347f3ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5588a0a3ede0 con 0x5588a0a24650 2024-05-09T02:11:02.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6347f3ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5588a0a3fb80 con 0x5588a0991110 2024-05-09T02:11:02.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6332ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a08a7450 0x5588a0a2b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6347f3ab80 10 client.0 _ll_register_callbacks cb 0x5588a0a29e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:11:02.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6332ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a08a7450 0x5588a0a2b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44234/0 (socket says 172.21.15.19:44234) 2024-05-09T02:11:02.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6332ffd700 1 -- 172.21.15.19:0/284083450 learned_addr learned my addr 172.21.15.19:0/284083450 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:02.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f63327fc700 1 --2- 172.21.15.19:0/284083450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a0991110 0x5588a0a2b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f63337fe700 1 --2- 172.21.15.19:0/284083450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a0a24650 0x5588a0a2bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6332ffd700 1 -- 172.21.15.19:0/284083450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a0991110 msgr2=0x5588a0a2b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:02.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6332ffd700 1 --2- 172.21.15.19:0/284083450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a0991110 0x5588a0a2b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6332ffd700 1 -- 172.21.15.19:0/284083450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a0a24650 msgr2=0x5588a0a2bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:02.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6332ffd700 1 --2- 172.21.15.19:0/284083450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a0a24650 0x5588a0a2bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6332ffd700 1 -- 172.21.15.19:0/284083450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a0a470c0 con 0x5588a08a7450 2024-05-09T02:11:02.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f63327fc700 1 --2- 172.21.15.19:0/284083450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a0991110 0x5588a0a2b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:11:02.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6332ffd700 1 --2- 172.21.15.19:0/284083450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a08a7450 0x5588a0a2b150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f63240a59d0 tx=0x7f63240cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e36b4f93a6c5d44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:02.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6316ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:11:02.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6316ff5700 1 -- 172.21.15.19:0/284083450 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63240a9460 con 0x5588a08a7450 2024-05-09T02:11:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6316ff5700 1 -- 172.21.15.19:0/284083450 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f63240a5be0 con 0x5588a08a7450 2024-05-09T02:11:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.423+0000 7f6347f3ab80 1 -- 172.21.15.19:0/284083450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f632c003c00 con 0x5588a08a7450 2024-05-09T02:11:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.424+0000 7f6316ff5700 1 -- 172.21.15.19:0/284083450 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f63240b2660 con 0x5588a08a7450 2024-05-09T02:11:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.424+0000 7f6347f3ab80 10 client.15162 fetch_fsmap learned FSMap version 28 2024-05-09T02:11:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.424+0000 7f6347f3ab80 1 -- 172.21.15.19:0/284083450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588a0aa9a10 con 0x5588a08a7450 2024-05-09T02:11:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.424+0000 7f6316ff5700 1 -- 172.21.15.19:0/284083450 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f632c003c00 con 0x5588a08a7450 2024-05-09T02:11:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.424+0000 7f6316ff5700 1 -- 172.21.15.19:0/284083450 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f63240aaba0 con 0x5588a08a7450 2024-05-09T02:11:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.424+0000 7f6347f3ab80 10 client.15162 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:11:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.424+0000 7f6347f3ab80 10 client.15162 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T02:11:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.424+0000 7f6347f3ab80 20 client.15162 populate_metadata read hostname 'smithi019' 2024-05-09T02:11:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.424+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/284083450 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5588a0aabeb0 0x5588a0aad320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.424+0000 7f6347f3ab80 4 client.15162 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T02:11:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.424+0000 7f6347f3ab80 1 -- 172.21.15.19:0/284083450 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "status"}) -- 0x5588a0954980 con 0x5588a0aabeb0 2024-05-09T02:11:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.424+0000 7f63327fc700 1 --2- 172.21.15.19:0/284083450 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5588a0aabeb0 0x5588a0aad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:02.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6316ff5700 10 client.15162 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:11:02.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f63327fc700 1 --2- 172.21.15.19:0/284083450 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5588a0aabeb0 0x5588a0aad320 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e0cbbec93dab51d server_cookie=83ca6c2eade163ab in_seq=0 out_seq=0 2024-05-09T02:11:02.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6316ff5700 1 -- 172.21.15.19:0/284083450 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5588a0954980 con 0x5588a0aabeb0 2024-05-09T02:11:02.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6316ff5700 10 client.15162 handle_command_reply: tid=0 2024-05-09T02:11:02.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6347f3ab80 1 client.15162 shutdown 2024-05-09T02:11:02.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6347f3ab80 1 -- 172.21.15.19:0/284083450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a08a7450 msgr2=0x5588a0a2b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:02.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/284083450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a08a7450 0x5588a0a2b150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f63240a59d0 tx=0x7f63240cc040 comp rx=0 tx=0).stop 2024-05-09T02:11:02.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6347f3ab80 1 -- 172.21.15.19:0/284083450 shutdown_connections 2024-05-09T02:11:02.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/284083450 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5588a0aabeb0 0x5588a0aad320 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/284083450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5588a0a24650 0x5588a0a2bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/284083450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5588a08a7450 0x5588a0a2b150 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6347f3ab80 1 --2- 172.21.15.19:0/284083450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588a0991110 0x5588a0a2b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6347f3ab80 1 -- 172.21.15.19:0/284083450 >> 172.21.15.19:0/284083450 conn(0x5588a0907650 msgr2=0x5588a0a2d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:02.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6316ff5700 0 client.15162 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:11:02.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6347f3ab80 1 -- 172.21.15.19:0/284083450 shutdown_connections 2024-05-09T02:11:02.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6347f3ab80 1 -- 172.21.15.19:0/284083450 wait complete. 2024-05-09T02:11:02.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.425+0000 7f6347f3ab80 20 client.15162 trim_cache size 0 max 16384 2024-05-09T02:11:02.434 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:11:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "a0611f3e-0da7-11ef-bc97-c7b262605968", 2024-05-09T02:11:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-05-09T02:11:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24463, 2024-05-09T02:11:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T02:11:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T02:11:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T02:11:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 177.19497093000001, 2024-05-09T02:11:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-05-09T02:11:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 81, 2024-05-09T02:11:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 79, 2024-05-09T02:11:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 222.64332354000001 2024-05-09T02:11:02.435 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:11:02.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.426+0000 7f6339524700 1 -- 172.21.15.19:0/1860243726 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f631c0b2880 msgr2=0x7f631c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:02.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.426+0000 7f6339524700 1 --2- 172.21.15.19:0/1860243726 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f631c0b2880 0x7f631c0b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6328006fd0 tx=0x7f6328008040 comp rx=0 tx=0).stop 2024-05-09T02:11:02.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.426+0000 7f6339524700 1 -- 172.21.15.19:0/1860243726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6334103550 msgr2=0x7f633419d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:02.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.426+0000 7f6339524700 1 --2- 172.21.15.19:0/1860243726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6334103550 0x7f633419d110 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f632400c9b0 tx=0x7f632400ccc0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.426+0000 7f6339524700 1 -- 172.21.15.19:0/1860243726 shutdown_connections 2024-05-09T02:11:02.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.426+0000 7f6339524700 1 --2- 172.21.15.19:0/1860243726 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f631c0b2880 0x7f631c0b4d90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.426+0000 7f6339524700 1 --2- 172.21.15.19:0/1860243726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6334108780 0x7f63341a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.426+0000 7f6339524700 1 --2- 172.21.15.19:0/1860243726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6334103550 0x7f633419d110 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.426+0000 7f6339524700 1 --2- 172.21.15.19:0/1860243726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6334107e30 0x7f633419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:02.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.426+0000 7f6339524700 1 -- 172.21.15.19:0/1860243726 >> 172.21.15.19:0/1860243726 conn(0x7f6334078e00 msgr2=0x7f633410e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:02.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.426+0000 7f6339524700 1 -- 172.21.15.19:0/1860243726 shutdown_connections 2024-05-09T02:11:02.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:02.426+0000 7f6339524700 1 -- 172.21.15.19:0/1860243726 wait complete. 2024-05-09T02:11:03.280 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T02:11:03.532 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:11:03.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:03 smithi196 ceph-mon[34718]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:11:03.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:03 smithi019 ceph-mon[30151]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:11:03.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:03 smithi059 ceph-mon[34781]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:11:05.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.238+0000 7fba7c8a0700 1 -- 172.21.15.19:0/272597626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781034c0 msgr2=0x7fba781038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:05.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.238+0000 7fba7c8a0700 1 --2- 172.21.15.19:0/272597626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781034c0 0x7fba781038d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fba600099c0 tx=0x7fba60009cd0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.238+0000 7fba7c8a0700 1 -- 172.21.15.19:0/272597626 shutdown_connections 2024-05-09T02:11:05.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.238+0000 7fba7c8a0700 1 --2- 172.21.15.19:0/272597626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba78108740 0x7fba781100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.238+0000 7fba7c8a0700 1 --2- 172.21.15.19:0/272597626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba781034c0 0x7fba781038d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.238+0000 7fba7c8a0700 1 --2- 172.21.15.19:0/272597626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba78107df0 0x7fba78108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.238+0000 7fba7c8a0700 1 -- 172.21.15.19:0/272597626 >> 172.21.15.19:0/272597626 conn(0x7fba780fedf0 msgr2=0x7fba78101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.238+0000 7fba7c8a0700 1 -- 172.21.15.19:0/272597626 shutdown_connections 2024-05-09T02:11:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.238+0000 7fba7c8a0700 1 -- 172.21.15.19:0/272597626 wait complete. 2024-05-09T02:11:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.239+0000 7fba7c8a0700 1 Processor -- start 2024-05-09T02:11:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.239+0000 7fba7c8a0700 1 -- start start 2024-05-09T02:11:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.239+0000 7fba7c8a0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba781034c0 0x7fba78199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.239+0000 7fba7c8a0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba78107df0 0x7fba7819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.239+0000 7fba7c8a0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba78108740 0x7fba7819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.239+0000 7fba7c8a0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fba78113370 con 0x7fba78107df0 2024-05-09T02:11:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.239+0000 7fba7c8a0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fba781131f0 con 0x7fba78108740 2024-05-09T02:11:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.239+0000 7fba7c8a0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba781134f0 con 0x7fba781034c0 2024-05-09T02:11:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.240+0000 7fba75d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba78107df0 0x7fba7819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.240+0000 7fba75d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba78107df0 0x7fba7819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44246/0 (socket says 172.21.15.19:44246) 2024-05-09T02:11:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.240+0000 7fba75d9b700 1 -- 172.21.15.19:0/436286272 learned_addr learned my addr 172.21.15.19:0/436286272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.240+0000 7fba7659c700 1 --2- 172.21.15.19:0/436286272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba781034c0 0x7fba78199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.240+0000 7fba76d9d700 1 --2- 172.21.15.19:0/436286272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba78108740 0x7fba7819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.240+0000 7fba75d9b700 1 -- 172.21.15.19:0/436286272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba781034c0 msgr2=0x7fba78199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.240+0000 7fba75d9b700 1 --2- 172.21.15.19:0/436286272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba781034c0 0x7fba78199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.240+0000 7fba75d9b700 1 -- 172.21.15.19:0/436286272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba78108740 msgr2=0x7fba7819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.240+0000 7fba75d9b700 1 --2- 172.21.15.19:0/436286272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba78108740 0x7fba7819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.240+0000 7fba75d9b700 1 -- 172.21.15.19:0/436286272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7819eb80 con 0x7fba78107df0 2024-05-09T02:11:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.240+0000 7fba7659c700 1 --2- 172.21.15.19:0/436286272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba781034c0 0x7fba78199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.240+0000 7fba75d9b700 1 --2- 172.21.15.19:0/436286272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba78107df0 0x7fba7819a300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fba680098f0 tx=0x7fba68009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.240+0000 7fba6f7fe700 1 -- 172.21.15.19:0/436286272 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6800d020 con 0x7fba78107df0 2024-05-09T02:11:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.240+0000 7fba6f7fe700 1 -- 172.21.15.19:0/436286272 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba6800d1c0 con 0x7fba78107df0 2024-05-09T02:11:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.240+0000 7fba6f7fe700 1 -- 172.21.15.19:0/436286272 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6800c920 con 0x7fba78107df0 2024-05-09T02:11:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.241+0000 7fba7c8a0700 1 -- 172.21.15.19:0/436286272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7819ee70 con 0x7fba78107df0 2024-05-09T02:11:05.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.241+0000 7fba7c8a0700 1 -- 172.21.15.19:0/436286272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7819f3b0 con 0x7fba78107df0 2024-05-09T02:11:05.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.244+0000 7fba6f7fe700 1 -- 172.21.15.19:0/436286272 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fba6800cac0 con 0x7fba78107df0 2024-05-09T02:11:05.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.244+0000 7fba8b2b6b80 1 Processor -- start 2024-05-09T02:11:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.245+0000 7fba8b2b6b80 1 -- start start 2024-05-09T02:11:05.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.245+0000 7fba8b2b6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd708bd110 0x55dd70954d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.245+0000 7fba8b2b6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd709552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.245+0000 7fba8b2b6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd709593b0 0x55dd709597c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.245+0000 7fba8b2b6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dd70872fa0 con 0x55dd709593b0 2024-05-09T02:11:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.245+0000 7fba8b2b6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dd708117a0 con 0x55dd707d3450 2024-05-09T02:11:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.245+0000 7fba8b2b6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dd7086ef60 con 0x55dd708bd110 2024-05-09T02:11:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.245+0000 7fba6f7fe700 1 --2- 172.21.15.19:0/436286272 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fba640b2880 0x7fba640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.245+0000 7fba75d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd709593b0 0x55dd709597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.245+0000 7fba75d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd709593b0 0x55dd709597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44250/0 (socket says 172.21.15.19:44250) 2024-05-09T02:11:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.245+0000 7fba75d9b700 1 -- 172.21.15.19:0/3136139508 learned_addr learned my addr 172.21.15.19:0/3136139508 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.245+0000 7fba76d9d700 1 --2- 172.21.15.19:0/436286272 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fba640b2880 0x7fba640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.245+0000 7fba76d9d700 1 --2- 172.21.15.19:0/3136139508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd708bd110 0x55dd70954d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.245+0000 7fba7659c700 1 --2- 172.21.15.19:0/3136139508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd709552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.246+0000 7fba75d9b700 1 -- 172.21.15.19:0/3136139508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd708bd110 msgr2=0x55dd70954d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:05.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.246+0000 7fba75d9b700 1 --2- 172.21.15.19:0/3136139508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd708bd110 0x55dd70954d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.246+0000 7fba75d9b700 1 -- 172.21.15.19:0/3136139508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 msgr2=0x55dd709552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:05.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.246+0000 7fba75d9b700 1 --2- 172.21.15.19:0/3136139508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd709552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.246+0000 7fba75d9b700 1 -- 172.21.15.19:0/3136139508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd70809320 con 0x55dd709593b0 2024-05-09T02:11:05.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.246+0000 7fba6f7fe700 1 -- 172.21.15.19:0/436286272 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fba680e3550 con 0x7fba78107df0 2024-05-09T02:11:05.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.246+0000 7fba76d9d700 1 --2- 172.21.15.19:0/436286272 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fba640b2880 0x7fba640b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fba70006130 tx=0x7fba70007980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:05.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.246+0000 7fba75d9b700 1 --2- 172.21.15.19:0/3136139508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd709593b0 0x55dd709597c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fba680aa850 tx=0x7fba68070090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288f02bb67bd699a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:05.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.246+0000 7fba6cff9700 1 -- 172.21.15.19:0/3136139508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68026030 con 0x55dd709593b0 2024-05-09T02:11:05.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.246+0000 7fba6cff9700 1 -- 172.21.15.19:0/3136139508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba680de070 con 0x55dd709593b0 2024-05-09T02:11:05.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.246+0000 7fba6cff9700 1 -- 172.21.15.19:0/3136139508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba680ab250 con 0x55dd709593b0 2024-05-09T02:11:05.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.247+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/3136139508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd709593b0 msgr2=0x55dd709597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:05.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.247+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/3136139508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd709593b0 0x55dd709597c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fba680aa850 tx=0x7fba68070090 comp rx=0 tx=0).stop 2024-05-09T02:11:05.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.247+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/3136139508 shutdown_connections 2024-05-09T02:11:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.247+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/3136139508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd709593b0 0x55dd709597c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.247+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/3136139508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd708bd110 0x55dd70954d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.247+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/3136139508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd709552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.247+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/3136139508 >> 172.21.15.19:0/3136139508 conn(0x55dd70833650 msgr2=0x55dd7094b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.247+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/3136139508 shutdown_connections 2024-05-09T02:11:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.247+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/3136139508 wait complete. 2024-05-09T02:11:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.248+0000 7fba8b2b6b80 1 -- start start 2024-05-09T02:11:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.248+0000 7fba8b2b6b80 1 Processor -- start 2024-05-09T02:11:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.248+0000 7fba8b2b6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd7095e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.248+0000 7fba8b2b6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd708bd110 0x55dd7095eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.248+0000 7fba8b2b6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd709593b0 0x55dd7095f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.248+0000 7fba8b2b6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dd706eeb50 con 0x55dd709593b0 2024-05-09T02:11:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.248+0000 7fba8b2b6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dd7082d100 con 0x55dd707d3450 2024-05-09T02:11:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.248+0000 7fba8b2b6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dd7095a100 con 0x55dd708bd110 2024-05-09T02:11:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.248+0000 7fba8b2b6b80 10 client.0 _ll_register_callbacks cb 0x55dd706e4670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:11:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.248+0000 7fba76d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd709593b0 0x55dd7095f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.248+0000 7fba76d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd709593b0 0x55dd7095f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44262/0 (socket says 172.21.15.19:44262) 2024-05-09T02:11:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.248+0000 7fba76d9d700 1 -- 172.21.15.19:0/791329902 learned_addr learned my addr 172.21.15.19:0/791329902 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.248+0000 7fba75d9b700 1 --2- 172.21.15.19:0/791329902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd708bd110 0x55dd7095eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.249+0000 7fba7659c700 1 --2- 172.21.15.19:0/791329902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd7095e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.249+0000 7fba76d9d700 1 -- 172.21.15.19:0/791329902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd708bd110 msgr2=0x55dd7095eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.249+0000 7fba76d9d700 1 --2- 172.21.15.19:0/791329902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd708bd110 0x55dd7095eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.249+0000 7fba76d9d700 1 -- 172.21.15.19:0/791329902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 msgr2=0x55dd7095e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.249+0000 7fba76d9d700 1 --2- 172.21.15.19:0/791329902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd7095e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.249+0000 7fba76d9d700 1 -- 172.21.15.19:0/791329902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd706ee440 con 0x55dd709593b0 2024-05-09T02:11:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.249+0000 7fba75d9b700 1 --2- 172.21.15.19:0/791329902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd708bd110 0x55dd7095eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.249+0000 7fba76d9d700 1 --2- 172.21.15.19:0/791329902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd709593b0 0x55dd7095f0e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fba70014010 tx=0x7fba7001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410c0e8378ea842f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.249+0000 7fba5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:11:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.249+0000 7fba5dffb700 1 -- 172.21.15.19:0/791329902 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba70013660 con 0x55dd709593b0 2024-05-09T02:11:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.249+0000 7fba5dffb700 1 -- 172.21.15.19:0/791329902 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fba70019dd0 con 0x55dd709593b0 2024-05-09T02:11:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.249+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/791329902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba680de070 con 0x55dd709593b0 2024-05-09T02:11:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.249+0000 7fba5dffb700 1 -- 172.21.15.19:0/791329902 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba70004bb0 con 0x55dd709593b0 2024-05-09T02:11:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.249+0000 7fba8b2b6b80 10 client.15174 fetch_fsmap learned FSMap version 28 2024-05-09T02:11:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.250+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/791329902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dd7096be70 con 0x55dd709593b0 2024-05-09T02:11:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.250+0000 7fba5dffb700 1 -- 172.21.15.19:0/791329902 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba680de070 con 0x55dd709593b0 2024-05-09T02:11:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.250+0000 7fba5dffb700 1 -- 172.21.15.19:0/791329902 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fba70016040 con 0x55dd709593b0 2024-05-09T02:11:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.250+0000 7fba8b2b6b80 10 client.15174 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:11:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.250+0000 7fba8b2b6b80 10 client.15174 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T02:11:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.250+0000 7fba8b2b6b80 20 client.15174 populate_metadata read hostname 'smithi019' 2024-05-09T02:11:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.250+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/791329902 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dd7096e4e0 0x55dd7096f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.250+0000 7fba8b2b6b80 4 client.15174 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:11:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.250+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/791329902 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd70880980 con 0x55dd7096e4e0 2024-05-09T02:11:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.250+0000 7fba7659c700 1 --2- 172.21.15.19:0/791329902 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dd7096e4e0 0x55dd7096f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.251+0000 7fba5dffb700 10 client.15174 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:11:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.251+0000 7fba7659c700 1 --2- 172.21.15.19:0/791329902 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dd7096e4e0 0x55dd7096f950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1633d65ef1aeece3 server_cookie=aec91cf5eff70196 in_seq=0 out_seq=0 2024-05-09T02:11:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.252+0000 7fba5dffb700 1 -- 172.21.15.19:0/791329902 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dd70880980 con 0x55dd7096e4e0 2024-05-09T02:11:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.252+0000 7fba5dffb700 10 client.15174 handle_command_reply: tid=0 2024-05-09T02:11:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.252+0000 7fba8b2b6b80 1 client.15174 shutdown 2024-05-09T02:11:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.252+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/791329902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd709593b0 msgr2=0x55dd7095f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.252+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/791329902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd709593b0 0x55dd7095f0e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fba70014010 tx=0x7fba7001a000 comp rx=0 tx=0).stop 2024-05-09T02:11:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.252+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/791329902 shutdown_connections 2024-05-09T02:11:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.252+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/791329902 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dd7096e4e0 0x55dd7096f950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.252+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/791329902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd709593b0 0x55dd7095f0e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.252+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/791329902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd7095e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.252+0000 7fba5dffb700 0 client.15174 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:11:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.252+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/791329902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd708bd110 0x55dd7095eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.252+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/791329902 >> 172.21.15.19:0/791329902 conn(0x55dd70833650 msgr2=0x55dd7094b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.252+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/791329902 shutdown_connections 2024-05-09T02:11:05.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.252+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/791329902 wait complete. 2024-05-09T02:11:05.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.252+0000 7fba8b2b6b80 20 client.15174 trim_cache size 0 max 16384 2024-05-09T02:11:05.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.288+0000 7fba8b2b6b80 1 Processor -- start 2024-05-09T02:11:05.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.288+0000 7fba8b2b6b80 1 -- start start 2024-05-09T02:11:05.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba8b2b6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd7096a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba8b2b6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd708bd110 0x55dd7094fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba8b2b6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd709593b0 0x55dd70957280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba8b2b6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dd7096dc90 con 0x55dd708bd110 2024-05-09T02:11:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba8b2b6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dd7096e360 con 0x55dd707d3450 2024-05-09T02:11:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba8b2b6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dd7096ea30 con 0x55dd709593b0 2024-05-09T02:11:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba75d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd708bd110 0x55dd7094fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba75d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd708bd110 0x55dd7094fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44272/0 (socket says 172.21.15.19:44272) 2024-05-09T02:11:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba75d9b700 1 -- 172.21.15.19:0/2162990303 learned_addr learned my addr 172.21.15.19:0/2162990303 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba7659c700 1 --2- 172.21.15.19:0/2162990303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd7096a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba76d9d700 1 --2- 172.21.15.19:0/2162990303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd709593b0 0x55dd70957280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba75d9b700 1 -- 172.21.15.19:0/2162990303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd709593b0 msgr2=0x55dd70957280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba75d9b700 1 --2- 172.21.15.19:0/2162990303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd709593b0 0x55dd70957280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba75d9b700 1 -- 172.21.15.19:0/2162990303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 msgr2=0x55dd7096a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba75d9b700 1 --2- 172.21.15.19:0/2162990303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd7096a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba75d9b700 1 -- 172.21.15.19:0/2162990303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd70955980 con 0x55dd708bd110 2024-05-09T02:11:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.289+0000 7fba7659c700 1 --2- 172.21.15.19:0/2162990303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd7096a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:11:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.290+0000 7fba75d9b700 1 --2- 172.21.15.19:0/2162990303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd708bd110 0x55dd7094fd50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fba6800a270 tx=0x7fba68005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a668f72050cf8722 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.290+0000 7fba5ffff700 1 -- 172.21.15.19:0/2162990303 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba680aa660 con 0x55dd708bd110 2024-05-09T02:11:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.290+0000 7fba5ffff700 1 -- 172.21.15.19:0/2162990303 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba680ab250 con 0x55dd708bd110 2024-05-09T02:11:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.290+0000 7fba5ffff700 1 -- 172.21.15.19:0/2162990303 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6800cdb0 con 0x55dd708bd110 2024-05-09T02:11:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.291+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/2162990303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd708bd110 msgr2=0x55dd7094fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.291+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/2162990303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd708bd110 0x55dd7094fd50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fba6800a270 tx=0x7fba68005c80 comp rx=0 tx=0).stop 2024-05-09T02:11:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.291+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/2162990303 shutdown_connections 2024-05-09T02:11:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.291+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/2162990303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd709593b0 0x55dd70957280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.291+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/2162990303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd7096a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.291+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/2162990303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd708bd110 0x55dd7094fd50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.291+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/2162990303 >> 172.21.15.19:0/2162990303 conn(0x55dd70833650 msgr2=0x55dd7094f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.291+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/2162990303 shutdown_connections 2024-05-09T02:11:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.291+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/2162990303 wait complete. 2024-05-09T02:11:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.291+0000 7fba8b2b6b80 1 -- start start 2024-05-09T02:11:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.291+0000 7fba8b2b6b80 1 Processor -- start 2024-05-09T02:11:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba8b2b6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd709751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba8b2b6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd708bd110 0x55dd70975720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba8b2b6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd709593b0 0x55dd709d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba8b2b6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dd7096f7d0 con 0x55dd708bd110 2024-05-09T02:11:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba8b2b6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dd7096f100 con 0x55dd707d3450 2024-05-09T02:11:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba8b2b6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dd7096fea0 con 0x55dd709593b0 2024-05-09T02:11:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba75d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd708bd110 0x55dd70975720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba8b2b6b80 10 client.0 _ll_register_callbacks cb 0x55dd706e4670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:11:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba75d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd708bd110 0x55dd70975720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44282/0 (socket says 172.21.15.19:44282) 2024-05-09T02:11:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba75d9b700 1 -- 172.21.15.19:0/1313328636 learned_addr learned my addr 172.21.15.19:0/1313328636 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba76d9d700 1 --2- 172.21.15.19:0/1313328636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd709593b0 0x55dd709d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba7659c700 1 --2- 172.21.15.19:0/1313328636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd709751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba75d9b700 1 -- 172.21.15.19:0/1313328636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd709593b0 msgr2=0x55dd709d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba75d9b700 1 --2- 172.21.15.19:0/1313328636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd709593b0 0x55dd709d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba75d9b700 1 -- 172.21.15.19:0/1313328636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 msgr2=0x55dd709751e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba75d9b700 1 --2- 172.21.15.19:0/1313328636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd709751e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba75d9b700 1 -- 172.21.15.19:0/1313328636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd709d2f00 con 0x55dd708bd110 2024-05-09T02:11:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba76d9d700 1 --2- 172.21.15.19:0/1313328636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd709593b0 0x55dd709d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba7659c700 1 --2- 172.21.15.19:0/1313328636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd709751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:11:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba75d9b700 1 --2- 172.21.15.19:0/1313328636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd708bd110 0x55dd70975720 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fba680aaae0 tx=0x7fba6800c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd9b8a1b66d961 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.292+0000 7fba5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:11:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.293+0000 7fba5dffb700 1 -- 172.21.15.19:0/1313328636 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba680aa660 con 0x55dd708bd110 2024-05-09T02:11:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.293+0000 7fba5dffb700 1 -- 172.21.15.19:0/1313328636 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fba680cbe00 con 0x55dd708bd110 2024-05-09T02:11:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.293+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/1313328636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba680ab250 con 0x55dd708bd110 2024-05-09T02:11:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.293+0000 7fba5dffb700 1 -- 172.21.15.19:0/1313328636 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba680a62e0 con 0x55dd708bd110 2024-05-09T02:11:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.293+0000 7fba8b2b6b80 10 client.15180 fetch_fsmap learned FSMap version 28 2024-05-09T02:11:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.293+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/1313328636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dd709da1c0 con 0x55dd708bd110 2024-05-09T02:11:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.293+0000 7fba5dffb700 1 -- 172.21.15.19:0/1313328636 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba680ab250 con 0x55dd708bd110 2024-05-09T02:11:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.293+0000 7fba5dffb700 1 -- 172.21.15.19:0/1313328636 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fba680b3520 con 0x55dd708bd110 2024-05-09T02:11:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.293+0000 7fba8b2b6b80 10 client.15180 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:11:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.293+0000 7fba8b2b6b80 10 client.15180 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T02:11:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.293+0000 7fba8b2b6b80 20 client.15180 populate_metadata read hostname 'smithi019' 2024-05-09T02:11:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.293+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/1313328636 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dd709dbf20 0x55dd709dd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.293+0000 7fba8b2b6b80 4 client.15180 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T02:11:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.293+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/1313328636 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "status"}) -- 0x55dd7087aa20 con 0x55dd709dbf20 2024-05-09T02:11:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.294+0000 7fba7659c700 1 --2- 172.21.15.19:0/1313328636 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dd709dbf20 0x55dd709dd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.294+0000 7fba7659c700 1 --2- 172.21.15.19:0/1313328636 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dd709dbf20 0x55dd709dd390 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45041dd08e739d29 server_cookie=2b92d340ba8f17a in_seq=0 out_seq=0 2024-05-09T02:11:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.294+0000 7fba5dffb700 10 client.15180 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:11:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.294+0000 7fba5dffb700 1 -- 172.21.15.19:0/1313328636 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55dd7087aa20 con 0x55dd709dbf20 2024-05-09T02:11:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.294+0000 7fba5dffb700 10 client.15180 handle_command_reply: tid=0 2024-05-09T02:11:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.294+0000 7fba8b2b6b80 1 client.15180 shutdown 2024-05-09T02:11:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.294+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/1313328636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd708bd110 msgr2=0x55dd70975720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.294+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/1313328636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd708bd110 0x55dd70975720 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fba680aaae0 tx=0x7fba6800c3d0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/1313328636 shutdown_connections 2024-05-09T02:11:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/1313328636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd709593b0 0x55dd709d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/1313328636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd707d3450 0x55dd709751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/1313328636 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dd709dbf20 0x55dd709dd390 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba8b2b6b80 1 --2- 172.21.15.19:0/1313328636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd708bd110 0x55dd70975720 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba5dffb700 0 client.15180 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:11:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/1313328636 >> 172.21.15.19:0/1313328636 conn(0x55dd70833650 msgr2=0x55dd7094a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/1313328636 shutdown_connections 2024-05-09T02:11:05.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba8b2b6b80 1 -- 172.21.15.19:0/1313328636 wait complete. 2024-05-09T02:11:05.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba8b2b6b80 20 client.15180 trim_cache size 0 max 16384 2024-05-09T02:11:05.302 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:11:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "a0611f3e-0da7-11ef-bc97-c7b262605968", 2024-05-09T02:11:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-05-09T02:11:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24473, 2024-05-09T02:11:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T02:11:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T02:11:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T02:11:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 181.06971098299999, 2024-05-09T02:11:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-05-09T02:11:05.303 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 81, 2024-05-09T02:11:05.303 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 79, 2024-05-09T02:11:05.303 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 209.24390009199999 2024-05-09T02:11:05.303 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:11:05.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba7c8a0700 1 -- 172.21.15.19:0/436286272 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fba640b2880 msgr2=0x7fba640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba7c8a0700 1 --2- 172.21.15.19:0/436286272 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fba640b2880 0x7fba640b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fba70006130 tx=0x7fba70007980 comp rx=0 tx=0).stop 2024-05-09T02:11:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba7c8a0700 1 -- 172.21.15.19:0/436286272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba78107df0 msgr2=0x7fba7819a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba7c8a0700 1 --2- 172.21.15.19:0/436286272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba78107df0 0x7fba7819a300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fba680098f0 tx=0x7fba68009c00 comp rx=0 tx=0).stop 2024-05-09T02:11:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba7c8a0700 1 -- 172.21.15.19:0/436286272 shutdown_connections 2024-05-09T02:11:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba7c8a0700 1 --2- 172.21.15.19:0/436286272 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fba640b2880 0x7fba640b4d90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba7c8a0700 1 --2- 172.21.15.19:0/436286272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba78108740 0x7fba7819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba7c8a0700 1 --2- 172.21.15.19:0/436286272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba781034c0 0x7fba78199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba7c8a0700 1 --2- 172.21.15.19:0/436286272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba78107df0 0x7fba7819a300 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.295+0000 7fba7c8a0700 1 -- 172.21.15.19:0/436286272 >> 172.21.15.19:0/436286272 conn(0x7fba780fedf0 msgr2=0x7fba78101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.296+0000 7fba7c8a0700 1 -- 172.21.15.19:0/436286272 shutdown_connections 2024-05-09T02:11:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:05.296+0000 7fba7c8a0700 1 -- 172.21.15.19:0/436286272 wait complete. 2024-05-09T02:11:05.518 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:05 smithi059 ceph-mon[34781]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:11:05.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:05 smithi196 ceph-mon[34718]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:11:05.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:05 smithi019 ceph-mon[30151]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T02:11:06.059 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T02:11:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:11:07.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:07 smithi196 ceph-mon[34718]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:11:07.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:07 smithi019 ceph-mon[30151]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:11:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:07 smithi059 ceph-mon[34781]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T02:11:08.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.088+0000 7faf86007700 1 -- 172.21.15.19:0/1786390004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf80108f00 msgr2=0x7faf80109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:08.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.088+0000 7faf86007700 1 --2- 172.21.15.19:0/1786390004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf80108f00 0x7faf80109310 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7faf680099c0 tx=0x7faf68009cd0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.088+0000 7faf86007700 1 -- 172.21.15.19:0/1786390004 shutdown_connections 2024-05-09T02:11:08.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.088+0000 7faf86007700 1 --2- 172.21.15.19:0/1786390004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf80109850 0x7faf801111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.088+0000 7faf86007700 1 --2- 172.21.15.19:0/1786390004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf80102ae0 0x7faf80102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.088+0000 7faf86007700 1 --2- 172.21.15.19:0/1786390004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf80108f00 0x7faf80109310 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.088+0000 7faf86007700 1 -- 172.21.15.19:0/1786390004 >> 172.21.15.19:0/1786390004 conn(0x7faf800fe540 msgr2=0x7faf80100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:08.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.088+0000 7faf86007700 1 -- 172.21.15.19:0/1786390004 shutdown_connections 2024-05-09T02:11:08.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.088+0000 7faf86007700 1 -- 172.21.15.19:0/1786390004 wait complete. 2024-05-09T02:11:08.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.089+0000 7faf86007700 1 Processor -- start 2024-05-09T02:11:08.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.089+0000 7faf86007700 1 -- start start 2024-05-09T02:11:08.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.089+0000 7faf86007700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf80102ae0 0x7faf80103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.089+0000 7faf86007700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf80108f00 0x7faf80103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.089+0000 7faf86007700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf80109850 0x7faf801044b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.089+0000 7faf86007700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf801144b0 con 0x7faf80108f00 2024-05-09T02:11:08.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.089+0000 7faf86007700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faf80114330 con 0x7faf80102ae0 2024-05-09T02:11:08.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.089+0000 7faf86007700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faf80114630 con 0x7faf80109850 2024-05-09T02:11:08.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.089+0000 7faf7effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf80108f00 0x7faf80103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.089+0000 7faf7effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf80108f00 0x7faf80103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32970/0 (socket says 172.21.15.19:32970) 2024-05-09T02:11:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.089+0000 7faf7effd700 1 -- 172.21.15.19:0/2586272250 learned_addr learned my addr 172.21.15.19:0/2586272250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.089+0000 7faf7ffff700 1 --2- 172.21.15.19:0/2586272250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf80109850 0x7faf801044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.089+0000 7faf7f7fe700 1 --2- 172.21.15.19:0/2586272250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf80102ae0 0x7faf80103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.090+0000 7faf7effd700 1 -- 172.21.15.19:0/2586272250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf80109850 msgr2=0x7faf801044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.090+0000 7faf7effd700 1 --2- 172.21.15.19:0/2586272250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf80109850 0x7faf801044b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.090+0000 7faf7effd700 1 -- 172.21.15.19:0/2586272250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf80102ae0 msgr2=0x7faf80103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.090+0000 7faf7effd700 1 --2- 172.21.15.19:0/2586272250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf80102ae0 0x7faf80103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.090+0000 7faf7effd700 1 -- 172.21.15.19:0/2586272250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf801a9260 con 0x7faf80108f00 2024-05-09T02:11:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.090+0000 7faf7effd700 1 --2- 172.21.15.19:0/2586272250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf80108f00 0x7faf80103f70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faf68006010 tx=0x7faf680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.090+0000 7faf7cff9700 1 -- 172.21.15.19:0/2586272250 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf68027070 con 0x7faf80108f00 2024-05-09T02:11:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.090+0000 7faf7cff9700 1 -- 172.21.15.19:0/2586272250 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf68003f20 con 0x7faf80108f00 2024-05-09T02:11:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.090+0000 7faf7cff9700 1 -- 172.21.15.19:0/2586272250 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf68022c60 con 0x7faf80108f00 2024-05-09T02:11:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.090+0000 7faf86007700 1 -- 172.21.15.19:0/2586272250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf801a94f0 con 0x7faf80108f00 2024-05-09T02:11:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.090+0000 7faf86007700 1 -- 172.21.15.19:0/2586272250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf801a9940 con 0x7faf80108f00 2024-05-09T02:11:08.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.093+0000 7faf7cff9700 1 -- 172.21.15.19:0/2586272250 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7faf6802a070 con 0x7faf80108f00 2024-05-09T02:11:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.094+0000 7faf7cff9700 1 --2- 172.21.15.19:0/2586272250 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7faf6c0b2880 0x7faf6c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.094+0000 7faf94a1db80 1 Processor -- start 2024-05-09T02:11:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.094+0000 7faf7f7fe700 1 --2- 172.21.15.19:0/2586272250 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7faf6c0b2880 0x7faf6c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.094+0000 7faf94a1db80 1 -- start start 2024-05-09T02:11:08.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.094+0000 7faf7cff9700 1 -- 172.21.15.19:0/2586272250 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7faf68022e00 con 0x7faf80108f00 2024-05-09T02:11:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.094+0000 7faf94a1db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da3ffcb0 0x5567da5081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.094+0000 7faf94a1db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da385450 0x5567da508730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.094+0000 7faf94a1db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da5023b0 0x5567da5027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.094+0000 7faf94a1db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567da425050 con 0x5567da3ffcb0 2024-05-09T02:11:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.094+0000 7faf94a1db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5567da3ca950 con 0x5567da5023b0 2024-05-09T02:11:08.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.094+0000 7faf94a1db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5567da421010 con 0x5567da385450 2024-05-09T02:11:08.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.095+0000 7faf7f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da3ffcb0 0x5567da5081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.095+0000 7faf7f7fe700 1 --2- 172.21.15.19:0/2586272250 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7faf6c0b2880 0x7faf6c0b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faf70005950 tx=0x7faf7000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:08.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.095+0000 7faf7f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da3ffcb0 0x5567da5081f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32986/0 (socket says 172.21.15.19:32986) 2024-05-09T02:11:08.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.095+0000 7faf7f7fe700 1 -- 172.21.15.19:0/3441627815 learned_addr learned my addr 172.21.15.19:0/3441627815 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:08.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.095+0000 7faf7ffff700 1 --2- 172.21.15.19:0/3441627815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da5023b0 0x5567da5027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.095+0000 7faf7effd700 1 --2- 172.21.15.19:0/3441627815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da385450 0x5567da508730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.095+0000 7faf7f7fe700 1 -- 172.21.15.19:0/3441627815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da385450 msgr2=0x5567da508730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:08.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.095+0000 7faf7f7fe700 1 --2- 172.21.15.19:0/3441627815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da385450 0x5567da508730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.095+0000 7faf7f7fe700 1 -- 172.21.15.19:0/3441627815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da5023b0 msgr2=0x5567da5027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:08.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.095+0000 7faf7f7fe700 1 --2- 172.21.15.19:0/3441627815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da5023b0 0x5567da5027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.095+0000 7faf7f7fe700 1 -- 172.21.15.19:0/3441627815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567da3bb4b0 con 0x5567da3ffcb0 2024-05-09T02:11:08.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.095+0000 7faf7f7fe700 1 --2- 172.21.15.19:0/3441627815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da3ffcb0 0x5567da5081f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faf70010fa0 tx=0x7faf70013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6416ccf319794627 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:08.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.095+0000 7faf75ffb700 1 -- 172.21.15.19:0/3441627815 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7000be60 con 0x5567da3ffcb0 2024-05-09T02:11:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.095+0000 7faf75ffb700 1 -- 172.21.15.19:0/3441627815 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf70011db0 con 0x5567da3ffcb0 2024-05-09T02:11:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.095+0000 7faf75ffb700 1 -- 172.21.15.19:0/3441627815 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700266f0 con 0x5567da3ffcb0 2024-05-09T02:11:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.096+0000 7faf94a1db80 1 -- 172.21.15.19:0/3441627815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da3ffcb0 msgr2=0x5567da5081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.096+0000 7faf94a1db80 1 --2- 172.21.15.19:0/3441627815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da3ffcb0 0x5567da5081f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faf70010fa0 tx=0x7faf70013000 comp rx=0 tx=0).stop 2024-05-09T02:11:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.096+0000 7faf94a1db80 1 -- 172.21.15.19:0/3441627815 shutdown_connections 2024-05-09T02:11:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.096+0000 7faf94a1db80 1 --2- 172.21.15.19:0/3441627815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da5023b0 0x5567da5027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.096+0000 7faf94a1db80 1 --2- 172.21.15.19:0/3441627815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da3ffcb0 0x5567da5081f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.096+0000 7faf94a1db80 1 --2- 172.21.15.19:0/3441627815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da385450 0x5567da508730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.096+0000 7faf94a1db80 1 -- 172.21.15.19:0/3441627815 >> 172.21.15.19:0/3441627815 conn(0x5567da388f20 msgr2=0x5567da4fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.096+0000 7faf94a1db80 1 -- 172.21.15.19:0/3441627815 shutdown_connections 2024-05-09T02:11:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.096+0000 7faf94a1db80 1 -- 172.21.15.19:0/3441627815 wait complete. 2024-05-09T02:11:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.097+0000 7faf94a1db80 1 -- start start 2024-05-09T02:11:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.097+0000 7faf94a1db80 1 Processor -- start 2024-05-09T02:11:08.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.097+0000 7faf94a1db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da385450 0x5567da50bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.097+0000 7faf94a1db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da3ffcb0 0x5567da50c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.097+0000 7faf94a1db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da5023b0 0x5567da50ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.097+0000 7faf94a1db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567da503110 con 0x5567da385450 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.097+0000 7faf94a1db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5567da3bfad0 con 0x5567da3ffcb0 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.097+0000 7faf94a1db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5567da50efd0 con 0x5567da5023b0 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.097+0000 7faf94a1db80 10 client.0 _ll_register_callbacks cb 0x5567da296670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.097+0000 7faf7f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da385450 0x5567da50bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.098+0000 7faf7f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da385450 0x5567da50bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33000/0 (socket says 172.21.15.19:33000) 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.098+0000 7faf7f7fe700 1 -- 172.21.15.19:0/1539326298 learned_addr learned my addr 172.21.15.19:0/1539326298 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.098+0000 7faf7effd700 1 --2- 172.21.15.19:0/1539326298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da3ffcb0 0x5567da50c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.098+0000 7faf7ffff700 1 --2- 172.21.15.19:0/1539326298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da5023b0 0x5567da50ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.098+0000 7faf7f7fe700 1 -- 172.21.15.19:0/1539326298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da5023b0 msgr2=0x5567da50ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.098+0000 7faf7f7fe700 1 --2- 172.21.15.19:0/1539326298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da5023b0 0x5567da50ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.098+0000 7faf7f7fe700 1 -- 172.21.15.19:0/1539326298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da3ffcb0 msgr2=0x5567da50c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.098+0000 7faf7f7fe700 1 --2- 172.21.15.19:0/1539326298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da3ffcb0 0x5567da50c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.098+0000 7faf7f7fe700 1 -- 172.21.15.19:0/1539326298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567da2a0440 con 0x5567da385450 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.098+0000 7faf7ffff700 1 --2- 172.21.15.19:0/1539326298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da5023b0 0x5567da50ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:11:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.098+0000 7faf7f7fe700 1 --2- 172.21.15.19:0/1539326298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da385450 0x5567da50bfb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c010 tx=0x7faf70005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0487c5d6c9ed93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.098+0000 7faf5effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:11:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.098+0000 7faf5effd700 1 -- 172.21.15.19:0/1539326298 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf70011bb0 con 0x5567da385450 2024-05-09T02:11:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.098+0000 7faf5effd700 1 -- 172.21.15.19:0/1539326298 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7faf70004750 con 0x5567da385450 2024-05-09T02:11:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.098+0000 7faf94a1db80 1 -- 172.21.15.19:0/1539326298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf70011db0 con 0x5567da385450 2024-05-09T02:11:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.099+0000 7faf5effd700 1 -- 172.21.15.19:0/1539326298 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf70004950 con 0x5567da385450 2024-05-09T02:11:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.099+0000 7faf94a1db80 10 client.15192 fetch_fsmap learned FSMap version 28 2024-05-09T02:11:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.099+0000 7faf94a1db80 1 -- 172.21.15.19:0/1539326298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5567da5199d0 con 0x5567da385450 2024-05-09T02:11:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.099+0000 7faf5effd700 1 -- 172.21.15.19:0/1539326298 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf70011db0 con 0x5567da385450 2024-05-09T02:11:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.099+0000 7faf5effd700 1 -- 172.21.15.19:0/1539326298 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7faf7003c420 con 0x5567da385450 2024-05-09T02:11:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.099+0000 7faf94a1db80 10 client.15192 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:11:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.099+0000 7faf94a1db80 10 client.15192 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T02:11:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.099+0000 7faf94a1db80 20 client.15192 populate_metadata read hostname 'smithi019' 2024-05-09T02:11:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.099+0000 7faf94a1db80 1 --2- 172.21.15.19:0/1539326298 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5567da51bd20 0x5567da51d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.099+0000 7faf94a1db80 4 client.15192 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.099+0000 7faf94a1db80 1 -- 172.21.15.19:0/1539326298 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567da432a30 con 0x5567da51bd20 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.099+0000 7faf7effd700 1 --2- 172.21.15.19:0/1539326298 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5567da51bd20 0x5567da51d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.100+0000 7faf5effd700 10 client.15192 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.100+0000 7faf7effd700 1 --2- 172.21.15.19:0/1539326298 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5567da51bd20 0x5567da51d190 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=467617a6cc3e5880 server_cookie=70e5266a64e75655 in_seq=0 out_seq=0 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.101+0000 7faf5effd700 1 -- 172.21.15.19:0/1539326298 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5567da432a30 con 0x5567da51bd20 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.101+0000 7faf5effd700 10 client.15192 handle_command_reply: tid=0 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.101+0000 7faf94a1db80 1 client.15192 shutdown 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.101+0000 7faf94a1db80 1 -- 172.21.15.19:0/1539326298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da385450 msgr2=0x5567da50bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.101+0000 7faf94a1db80 1 --2- 172.21.15.19:0/1539326298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da385450 0x5567da50bfb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c010 tx=0x7faf70005320 comp rx=0 tx=0).stop 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.101+0000 7faf94a1db80 1 -- 172.21.15.19:0/1539326298 shutdown_connections 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.101+0000 7faf94a1db80 1 --2- 172.21.15.19:0/1539326298 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5567da51bd20 0x5567da51d190 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.101+0000 7faf94a1db80 1 --2- 172.21.15.19:0/1539326298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da5023b0 0x5567da50ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.101+0000 7faf94a1db80 1 --2- 172.21.15.19:0/1539326298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da385450 0x5567da50bfb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.101+0000 7faf94a1db80 1 --2- 172.21.15.19:0/1539326298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da3ffcb0 0x5567da50c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.101+0000 7faf5effd700 0 client.15192 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:11:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.101+0000 7faf94a1db80 1 -- 172.21.15.19:0/1539326298 >> 172.21.15.19:0/1539326298 conn(0x5567da388f20 msgr2=0x5567da4f6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.101+0000 7faf94a1db80 1 -- 172.21.15.19:0/1539326298 shutdown_connections 2024-05-09T02:11:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.101+0000 7faf94a1db80 1 -- 172.21.15.19:0/1539326298 wait complete. 2024-05-09T02:11:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.101+0000 7faf94a1db80 20 client.15192 trim_cache size 0 max 16384 2024-05-09T02:11:08.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.122+0000 7faf94a1db80 1 Processor -- start 2024-05-09T02:11:08.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.122+0000 7faf94a1db80 1 -- start start 2024-05-09T02:11:08.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf94a1db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da385450 0x5567da5070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf94a1db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da3ffcb0 0x5567da5075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf94a1db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da5023b0 0x5567da5265f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf94a1db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567da51b4d0 con 0x5567da3ffcb0 2024-05-09T02:11:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf94a1db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5567da51bba0 con 0x5567da5023b0 2024-05-09T02:11:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf94a1db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5567da51c270 con 0x5567da385450 2024-05-09T02:11:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf7effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da3ffcb0 0x5567da5075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf7effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da3ffcb0 0x5567da5075e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33008/0 (socket says 172.21.15.19:33008) 2024-05-09T02:11:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf7effd700 1 -- 172.21.15.19:0/4018438160 learned_addr learned my addr 172.21.15.19:0/4018438160 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf7f7fe700 1 --2- 172.21.15.19:0/4018438160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da385450 0x5567da5070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf7ffff700 1 --2- 172.21.15.19:0/4018438160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da5023b0 0x5567da5265f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf7effd700 1 -- 172.21.15.19:0/4018438160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da385450 msgr2=0x5567da5070a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf7effd700 1 --2- 172.21.15.19:0/4018438160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da385450 0x5567da5070a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf7effd700 1 -- 172.21.15.19:0/4018438160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da5023b0 msgr2=0x5567da5265f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf7effd700 1 --2- 172.21.15.19:0/4018438160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da5023b0 0x5567da5265f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf7effd700 1 -- 172.21.15.19:0/4018438160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567da50ed70 con 0x5567da3ffcb0 2024-05-09T02:11:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf7ffff700 1 --2- 172.21.15.19:0/4018438160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da5023b0 0x5567da5265f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:11:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.123+0000 7faf7effd700 1 --2- 172.21.15.19:0/4018438160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da3ffcb0 0x5567da5075e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faf680e0b80 tx=0x7faf680e0a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c893c4239abbf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:08.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.124+0000 7faf757fa700 1 -- 172.21.15.19:0/4018438160 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf680c3070 con 0x5567da3ffcb0 2024-05-09T02:11:08.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.124+0000 7faf757fa700 1 -- 172.21.15.19:0/4018438160 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf680aa6e0 con 0x5567da3ffcb0 2024-05-09T02:11:08.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.124+0000 7faf757fa700 1 -- 172.21.15.19:0/4018438160 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf680920b0 con 0x5567da3ffcb0 2024-05-09T02:11:08.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.124+0000 7faf94a1db80 1 -- 172.21.15.19:0/4018438160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da3ffcb0 msgr2=0x5567da5075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.124+0000 7faf94a1db80 1 --2- 172.21.15.19:0/4018438160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da3ffcb0 0x5567da5075e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faf680e0b80 tx=0x7faf680e0a00 comp rx=0 tx=0).stop 2024-05-09T02:11:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.124+0000 7faf94a1db80 1 -- 172.21.15.19:0/4018438160 shutdown_connections 2024-05-09T02:11:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.124+0000 7faf94a1db80 1 --2- 172.21.15.19:0/4018438160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da5023b0 0x5567da5265f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.124+0000 7faf94a1db80 1 --2- 172.21.15.19:0/4018438160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da385450 0x5567da5070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.124+0000 7faf94a1db80 1 --2- 172.21.15.19:0/4018438160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da3ffcb0 0x5567da5075e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.124+0000 7faf94a1db80 1 -- 172.21.15.19:0/4018438160 >> 172.21.15.19:0/4018438160 conn(0x5567da388f20 msgr2=0x5567da51ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.124+0000 7faf94a1db80 1 -- 172.21.15.19:0/4018438160 shutdown_connections 2024-05-09T02:11:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.125+0000 7faf94a1db80 1 -- 172.21.15.19:0/4018438160 wait complete. 2024-05-09T02:11:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.125+0000 7faf94a1db80 1 -- start start 2024-05-09T02:11:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.125+0000 7faf94a1db80 1 Processor -- start 2024-05-09T02:11:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.125+0000 7faf94a1db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da385450 0x5567da520d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.125+0000 7faf94a1db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da3ffcb0 0x5567da5212d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.125+0000 7faf94a1db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da5023b0 0x5567da525be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.125+0000 7faf94a1db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567da51d010 con 0x5567da5023b0 2024-05-09T02:11:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.125+0000 7faf94a1db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5567da51c940 con 0x5567da385450 2024-05-09T02:11:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.125+0000 7faf94a1db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5567da51d6e0 con 0x5567da3ffcb0 2024-05-09T02:11:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.125+0000 7faf94a1db80 10 client.0 _ll_register_callbacks cb 0x5567da296670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:11:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf7ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da5023b0 0x5567da525be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf7ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da5023b0 0x5567da525be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33024/0 (socket says 172.21.15.19:33024) 2024-05-09T02:11:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf7ffff700 1 -- 172.21.15.19:0/731863548 learned_addr learned my addr 172.21.15.19:0/731863548 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf7effd700 1 --2- 172.21.15.19:0/731863548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da3ffcb0 0x5567da5212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf7f7fe700 1 --2- 172.21.15.19:0/731863548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da385450 0x5567da520d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf7ffff700 1 -- 172.21.15.19:0/731863548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da3ffcb0 msgr2=0x5567da5212d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf7ffff700 1 --2- 172.21.15.19:0/731863548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da3ffcb0 0x5567da5212d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf7ffff700 1 -- 172.21.15.19:0/731863548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da385450 msgr2=0x5567da520d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf7ffff700 1 --2- 172.21.15.19:0/731863548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da385450 0x5567da520d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf7ffff700 1 -- 172.21.15.19:0/731863548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567da4f9980 con 0x5567da5023b0 2024-05-09T02:11:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf7f7fe700 1 --2- 172.21.15.19:0/731863548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da385450 0x5567da520d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:11:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf7ffff700 1 --2- 172.21.15.19:0/731863548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da5023b0 0x5567da525be0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faf78004270 tx=0x7faf78004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc728cf117402765 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf5effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:11:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf5effd700 1 -- 172.21.15.19:0/731863548 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7801b070 con 0x5567da5023b0 2024-05-09T02:11:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf5effd700 1 -- 172.21.15.19:0/731863548 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7faf78007a30 con 0x5567da5023b0 2024-05-09T02:11:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf94a1db80 1 -- 172.21.15.19:0/731863548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf680aa6e0 con 0x5567da5023b0 2024-05-09T02:11:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf5effd700 1 -- 172.21.15.19:0/731863548 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf78027050 con 0x5567da5023b0 2024-05-09T02:11:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf94a1db80 10 client.15198 fetch_fsmap learned FSMap version 28 2024-05-09T02:11:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.126+0000 7faf94a1db80 1 -- 172.21.15.19:0/731863548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5567da587190 con 0x5567da5023b0 2024-05-09T02:11:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.127+0000 7faf5effd700 1 -- 172.21.15.19:0/731863548 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf680aa6e0 con 0x5567da5023b0 2024-05-09T02:11:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.127+0000 7faf5effd700 1 -- 172.21.15.19:0/731863548 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7faf7801f3b0 con 0x5567da5023b0 2024-05-09T02:11:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.127+0000 7faf94a1db80 10 client.15198 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T02:11:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.127+0000 7faf94a1db80 10 client.15198 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T02:11:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.127+0000 7faf94a1db80 20 client.15198 populate_metadata read hostname 'smithi019' 2024-05-09T02:11:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.127+0000 7faf94a1db80 1 --2- 172.21.15.19:0/731863548 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5567da589590 0x5567da58aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.127+0000 7faf94a1db80 4 client.15198 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T02:11:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.127+0000 7faf94a1db80 1 -- 172.21.15.19:0/731863548 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "status"}) -- 0x5567da42cad0 con 0x5567da589590 2024-05-09T02:11:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.127+0000 7faf7f7fe700 1 --2- 172.21.15.19:0/731863548 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5567da589590 0x5567da58aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf5effd700 10 client.15198 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:11:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf7f7fe700 1 --2- 172.21.15.19:0/731863548 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5567da589590 0x5567da58aa00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb9eb0acd04b1b2f server_cookie=d547ab40d35d2f0d in_seq=0 out_seq=0 2024-05-09T02:11:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf5effd700 1 -- 172.21.15.19:0/731863548 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5567da42cad0 con 0x5567da589590 2024-05-09T02:11:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf5effd700 10 client.15198 handle_command_reply: tid=0 2024-05-09T02:11:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf94a1db80 1 client.15198 shutdown 2024-05-09T02:11:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf94a1db80 1 -- 172.21.15.19:0/731863548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da5023b0 msgr2=0x5567da525be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf94a1db80 1 --2- 172.21.15.19:0/731863548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da5023b0 0x5567da525be0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faf78004270 tx=0x7faf78004580 comp rx=0 tx=0).stop 2024-05-09T02:11:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf94a1db80 1 -- 172.21.15.19:0/731863548 shutdown_connections 2024-05-09T02:11:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf94a1db80 1 --2- 172.21.15.19:0/731863548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567da5023b0 0x5567da525be0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf94a1db80 1 --2- 172.21.15.19:0/731863548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567da385450 0x5567da520d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf94a1db80 1 --2- 172.21.15.19:0/731863548 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5567da589590 0x5567da58aa00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf94a1db80 1 --2- 172.21.15.19:0/731863548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567da3ffcb0 0x5567da5212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf94a1db80 1 -- 172.21.15.19:0/731863548 >> 172.21.15.19:0/731863548 conn(0x5567da388f20 msgr2=0x5567da4f6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf5effd700 0 client.15198 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:11:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf94a1db80 1 -- 172.21.15.19:0/731863548 shutdown_connections 2024-05-09T02:11:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf94a1db80 1 -- 172.21.15.19:0/731863548 wait complete. 2024-05-09T02:11:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.128+0000 7faf94a1db80 20 client.15198 trim_cache size 0 max 16384 2024-05-09T02:11:08.136 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:11:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "a0611f3e-0da7-11ef-bc97-c7b262605968", 2024-05-09T02:11:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-05-09T02:11:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24484, 2024-05-09T02:11:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-09T02:11:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-09T02:11:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-09T02:11:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 196.93551654199999, 2024-05-09T02:11:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-05-09T02:11:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 81, 2024-05-09T02:11:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 79, 2024-05-09T02:11:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 222.41522296100001 2024-05-09T02:11:08.137 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:11:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.129+0000 7faf86007700 1 -- 172.21.15.19:0/2586272250 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7faf6c0b2880 msgr2=0x7faf6c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.129+0000 7faf86007700 1 --2- 172.21.15.19:0/2586272250 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7faf6c0b2880 0x7faf6c0b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faf70005950 tx=0x7faf7000b410 comp rx=0 tx=0).stop 2024-05-09T02:11:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.129+0000 7faf86007700 1 -- 172.21.15.19:0/2586272250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf80108f00 msgr2=0x7faf80103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.129+0000 7faf86007700 1 --2- 172.21.15.19:0/2586272250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf80108f00 0x7faf80103f70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faf68006010 tx=0x7faf680047b0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.129+0000 7faf86007700 1 -- 172.21.15.19:0/2586272250 shutdown_connections 2024-05-09T02:11:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.129+0000 7faf86007700 1 --2- 172.21.15.19:0/2586272250 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7faf6c0b2880 0x7faf6c0b4d90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.129+0000 7faf86007700 1 --2- 172.21.15.19:0/2586272250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf80109850 0x7faf801044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.129+0000 7faf86007700 1 --2- 172.21.15.19:0/2586272250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf80102ae0 0x7faf80103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.129+0000 7faf86007700 1 --2- 172.21.15.19:0/2586272250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf80108f00 0x7faf80103f70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.129+0000 7faf86007700 1 -- 172.21.15.19:0/2586272250 >> 172.21.15.19:0/2586272250 conn(0x7faf800fe540 msgr2=0x7faf80107110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.129+0000 7faf86007700 1 -- 172.21.15.19:0/2586272250 shutdown_connections 2024-05-09T02:11:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.129+0000 7faf86007700 1 -- 172.21.15.19:0/2586272250 wait complete. 2024-05-09T02:11:08.839 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T02:11:08.848 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T02:11:08.848 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T02:11:08.849 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T02:11:08.849 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-09T02:11:08.850 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T02:11:08.850 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T02:11:08.850 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-09T02:11:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.968+0000 7fbb1027e640 1 -- 172.21.15.19:0/1128171774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb08104eb0 msgr2=0x7fbb081052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.968+0000 7fbb1027e640 1 --2- 172.21.15.19:0/1128171774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb08104eb0 0x7fbb081052c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009900 tx=0x7fbafc02a410 comp rx=0 tx=0).stop 2024-05-09T02:11:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.968+0000 7fbb1027e640 1 -- 172.21.15.19:0/1128171774 shutdown_connections 2024-05-09T02:11:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.968+0000 7fbb1027e640 1 --2- 172.21.15.19:0/1128171774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb08109bb0 0x7fbb0810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.968+0000 7fbb1027e640 1 --2- 172.21.15.19:0/1128171774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb08104eb0 0x7fbb081052c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.968+0000 7fbb1027e640 1 --2- 172.21.15.19:0/1128171774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb08103c70 0x7fbb08104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.968+0000 7fbb1027e640 1 -- 172.21.15.19:0/1128171774 >> 172.21.15.19:0/1128171774 conn(0x7fbb080fee40 msgr2=0x7fbb08101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:08.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.968+0000 7fbb1027e640 1 -- 172.21.15.19:0/1128171774 shutdown_connections 2024-05-09T02:11:08.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.968+0000 7fbb1027e640 1 -- 172.21.15.19:0/1128171774 wait complete. 2024-05-09T02:11:08.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.969+0000 7fbb1027e640 1 Processor -- start 2024-05-09T02:11:08.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.969+0000 7fbb1027e640 1 -- start start 2024-05-09T02:11:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.969+0000 7fbb1027e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb08103c70 0x7fbb0819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.969+0000 7fbb1027e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb08104eb0 0x7fbb0819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.969+0000 7fbb1027e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb08109bb0 0x7fbb081a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.969+0000 7fbb1027e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0810f7a0 con 0x7fbb08104eb0 2024-05-09T02:11:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.969+0000 7fbb1027e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0810f620 con 0x7fbb08103c70 2024-05-09T02:11:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.969+0000 7fbb1027e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0810f920 con 0x7fbb08109bb0 2024-05-09T02:11:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.969+0000 7fbb0d819640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb08104eb0 0x7fbb0819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.969+0000 7fbb0d819640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb08104eb0 0x7fbb0819a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33046/0 (socket says 172.21.15.19:33046) 2024-05-09T02:11:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.969+0000 7fbb0d819640 1 -- 172.21.15.19:0/351677717 learned_addr learned my addr 172.21.15.19:0/351677717 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.969+0000 7fbb0e01a640 1 --2- 172.21.15.19:0/351677717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb08103c70 0x7fbb0819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.970+0000 7fbb0e81b640 1 --2- 172.21.15.19:0/351677717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb08109bb0 0x7fbb081a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.970+0000 7fbb0d819640 1 -- 172.21.15.19:0/351677717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb08109bb0 msgr2=0x7fbb081a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.970+0000 7fbb0d819640 1 --2- 172.21.15.19:0/351677717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb08109bb0 0x7fbb081a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.970+0000 7fbb0d819640 1 -- 172.21.15.19:0/351677717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb08103c70 msgr2=0x7fbb0819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.970+0000 7fbb0d819640 1 --2- 172.21.15.19:0/351677717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb08103c70 0x7fbb0819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.970+0000 7fbb0d819640 1 -- 172.21.15.19:0/351677717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb081a1ec0 con 0x7fbb08104eb0 2024-05-09T02:11:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.970+0000 7fbb0e81b640 1 --2- 172.21.15.19:0/351677717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb08109bb0 0x7fbb081a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:11:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.970+0000 7fbb0d819640 1 --2- 172.21.15.19:0/351677717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb08104eb0 0x7fbb0819a8c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbafc006010 tx=0x7fbafc004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.970+0000 7fbb077fe640 1 -- 172.21.15.19:0/351677717 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc03f030 con 0x7fbb08104eb0 2024-05-09T02:11:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.970+0000 7fbb1027e640 1 -- 172.21.15.19:0/351677717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb081a2150 con 0x7fbb08104eb0 2024-05-09T02:11:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.970+0000 7fbb077fe640 1 -- 172.21.15.19:0/351677717 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbafc003cb0 con 0x7fbb08104eb0 2024-05-09T02:11:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.970+0000 7fbb077fe640 1 -- 172.21.15.19:0/351677717 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc035070 con 0x7fbb08104eb0 2024-05-09T02:11:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.970+0000 7fbb1027e640 1 -- 172.21.15.19:0/351677717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb081a2640 con 0x7fbb08104eb0 2024-05-09T02:11:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.971+0000 7fbb1027e640 1 -- 172.21.15.19:0/351677717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb081089c0 con 0x7fbb08104eb0 2024-05-09T02:11:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.975+0000 7fbb077fe640 1 -- 172.21.15.19:0/351677717 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbafc042030 con 0x7fbb08104eb0 2024-05-09T02:11:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.975+0000 7fbb077fe640 1 --2- 172.21.15.19:0/351677717 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fbaf00b27e0 0x7fbaf00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.975+0000 7fbb0e01a640 1 --2- 172.21.15.19:0/351677717 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fbaf00b27e0 0x7fbaf00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.975+0000 7fbb077fe640 1 -- 172.21.15.19:0/351677717 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fbafc102f20 con 0x7fbb08104eb0 2024-05-09T02:11:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.975+0000 7fbb0e01a640 1 --2- 172.21.15.19:0/351677717 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fbaf00b27e0 0x7fbaf00b4cf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbb08104870 tx=0x7fbb0000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:08.978+0000 7fbb077fe640 1 -- 172.21.15.19:0/351677717 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbafc0cac20 con 0x7fbb08104eb0 2024-05-09T02:11:09.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:09.222+0000 7fbb1027e640 1 -- 172.21.15.19:0/351677717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fbb080638d0 con 0x7fbb08104eb0 2024-05-09T02:11:09.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:09.269+0000 7fbb077fe640 1 -- 172.21.15.19:0/351677717 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fbafc003e60 con 0x7fbb08104eb0 2024-05-09T02:11:09.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:09.272+0000 7fbb1027e640 1 -- 172.21.15.19:0/351677717 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fbaf00b27e0 msgr2=0x7fbaf00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:09.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:09.272+0000 7fbb1027e640 1 --2- 172.21.15.19:0/351677717 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fbaf00b27e0 0x7fbaf00b4cf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbb08104870 tx=0x7fbb0000b390 comp rx=0 tx=0).stop 2024-05-09T02:11:09.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:09.272+0000 7fbb1027e640 1 -- 172.21.15.19:0/351677717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb08104eb0 msgr2=0x7fbb0819a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:09.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:09.272+0000 7fbb1027e640 1 --2- 172.21.15.19:0/351677717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb08104eb0 0x7fbb0819a8c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbafc006010 tx=0x7fbafc004340 comp rx=0 tx=0).stop 2024-05-09T02:11:09.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:09.272+0000 7fbb1027e640 1 -- 172.21.15.19:0/351677717 shutdown_connections 2024-05-09T02:11:09.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:09.272+0000 7fbb1027e640 1 --2- 172.21.15.19:0/351677717 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fbaf00b27e0 0x7fbaf00b4cf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:09.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:09.272+0000 7fbb1027e640 1 --2- 172.21.15.19:0/351677717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb08109bb0 0x7fbb081a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:09.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:09.272+0000 7fbb1027e640 1 --2- 172.21.15.19:0/351677717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb08104eb0 0x7fbb0819a8c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:09.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:09.272+0000 7fbb1027e640 1 --2- 172.21.15.19:0/351677717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb08103c70 0x7fbb0819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:09.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:09.272+0000 7fbb1027e640 1 -- 172.21.15.19:0/351677717 >> 172.21.15.19:0/351677717 conn(0x7fbb080fee40 msgr2=0x7fbb08105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:09.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:09.272+0000 7fbb1027e640 1 -- 172.21.15.19:0/351677717 shutdown_connections 2024-05-09T02:11:09.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:09.272+0000 7fbb1027e640 1 -- 172.21.15.19:0/351677717 wait complete. 2024-05-09T02:11:09.291 INFO:teuthology.run_tasks:Running task kclient... 2024-05-09T02:11:09.317 INFO:tasks.kclient:Mounting kernel clients... 2024-05-09T02:11:09.317 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-09T02:11:09.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T02:11:09.317 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-05-09T02:11:09.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T02:11:09.335 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-05-09T02:11:09.403 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-05-09T02:11:09.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T02:11:09.405 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-05-09T02:11:09.405 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T02:11:09.405 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-05-09T02:11:09.405 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-05-09T02:11:09.405 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-09T02:11:09.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:09 smithi196 ceph-mon[34718]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:11:09.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:09 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/351677717' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-09T02:11:09.545 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:09 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: ignoring --setuser ceph since I am not root 2024-05-09T02:11:09.545 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:09 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: ignoring --setgroup ceph since I am not root 2024-05-09T02:11:09.545 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:09 smithi196 ceph-mgr[36574]: -- 172.21.15.196:0/2229772992 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fa4b9385a0 con 0x55fa506c2480 2024-05-09T02:11:09.545 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:09 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:09.477+0000 7fe3b6b21200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T02:11:09.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:09 smithi019 ceph-mon[30151]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:11:09.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:09 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/351677717' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-09T02:11:09.571 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:09 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: ignoring --setuser ceph since I am not root 2024-05-09T02:11:09.571 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:09 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: ignoring --setgroup ceph since I am not root 2024-05-09T02:11:09.571 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:09 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:09.493+0000 7f9c8b9aa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T02:11:09.680 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:09 smithi059 ceph-mon[34781]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T02:11:09.680 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:09 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/351677717' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-09T02:11:09.680 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:09 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: ignoring --setuser ceph since I am not root 2024-05-09T02:11:09.680 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:09 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: ignoring --setgroup ceph since I am not root 2024-05-09T02:11:09.680 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:09 smithi059 ceph-mgr[36174]: -- 172.21.15.59:0/1841871917 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a89e0a5a0 con 0x561a8eb94480 2024-05-09T02:11:09.680 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:09 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:09.483+0000 7fe4a67a8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T02:11:09.680 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:09 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:09.677+0000 7fe4a67a8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T02:11:09.703 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:11:09.966 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:09 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:09.785+0000 7fe4a67a8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T02:11:09.967 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:09 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:09.700+0000 7f9c8b9aa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T02:11:09.968 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:09 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:09.815+0000 7f9c8b9aa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T02:11:10.044 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:09 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:09.665+0000 7fe3b6b21200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T02:11:10.044 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:09 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:09.769+0000 7fe3b6b21200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T02:11:10.246 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:10 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:10.151+0000 7f9c8b9aa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T02:11:10.246 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:10 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:10.243+0000 7f9c8b9aa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T02:11:10.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:10 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/351677717' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-09T02:11:10.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:10 smithi059 ceph-mon[34781]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-09T02:11:10.467 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:10 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:10.108+0000 7fe4a67a8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T02:11:10.467 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:10 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:10.198+0000 7fe4a67a8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T02:11:10.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:10 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/351677717' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-09T02:11:10.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:10 smithi196 ceph-mon[34718]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-09T02:11:10.545 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:10 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:10.081+0000 7fe3b6b21200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T02:11:10.545 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:10 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:10.167+0000 7fe3b6b21200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T02:11:10.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:10 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/351677717' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-09T02:11:10.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:10 smithi019 ceph-mon[30151]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-09T02:11:11.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.478+0000 7fd4bfc62700 1 -- 172.21.15.19:0/796817991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b8108740 msgr2=0x7fd4b81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:11.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.478+0000 7fd4bfc62700 1 --2- 172.21.15.19:0/796817991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b8108740 0x7fd4b81100a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000b0d0 tx=0x7fd4b000b3e0 comp rx=0 tx=0).stop 2024-05-09T02:11:11.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.478+0000 7fd4bfc62700 1 -- 172.21.15.19:0/796817991 shutdown_connections 2024-05-09T02:11:11.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.478+0000 7fd4bfc62700 1 --2- 172.21.15.19:0/796817991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b8108740 0x7fd4b81100a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:11.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.478+0000 7fd4bfc62700 1 --2- 172.21.15.19:0/796817991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4b81034c0 0x7fd4b81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:11.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.478+0000 7fd4bfc62700 1 --2- 172.21.15.19:0/796817991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8107df0 0x7fd4b8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:11.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.478+0000 7fd4bfc62700 1 -- 172.21.15.19:0/796817991 >> 172.21.15.19:0/796817991 conn(0x7fd4b80fedc0 msgr2=0x7fd4b81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:11.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.478+0000 7fd4bfc62700 1 -- 172.21.15.19:0/796817991 shutdown_connections 2024-05-09T02:11:11.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.478+0000 7fd4bfc62700 1 -- 172.21.15.19:0/796817991 wait complete. 2024-05-09T02:11:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.479+0000 7fd4bfc62700 1 Processor -- start 2024-05-09T02:11:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.479+0000 7fd4bfc62700 1 -- start start 2024-05-09T02:11:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.479+0000 7fd4bfc62700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4b81034c0 0x7fd4b81a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.479+0000 7fd4bfc62700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8107df0 0x7fd4b81a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.479+0000 7fd4bfc62700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b8108740 0x7fd4b81a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.479+0000 7fd4bfc62700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b8113370 con 0x7fd4b8108740 2024-05-09T02:11:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.479+0000 7fd4bfc62700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81131f0 con 0x7fd4b81034c0 2024-05-09T02:11:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.479+0000 7fd4bfc62700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81134f0 con 0x7fd4b8107df0 2024-05-09T02:11:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4be1ff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b8108740 0x7fd4b81a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4bd9fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4b81034c0 0x7fd4b81a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4be1ff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b8108740 0x7fd4b81a5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33078/0 (socket says 172.21.15.19:33078) 2024-05-09T02:11:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4bd9fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4b81034c0 0x7fd4b81a13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:36604/0 (socket says 172.21.15.19:36604) 2024-05-09T02:11:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4be1ff700 1 -- 172.21.15.19:0/519432447 learned_addr learned my addr 172.21.15.19:0/519432447 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4bd1fd700 1 --2- 172.21.15.19:0/519432447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8107df0 0x7fd4b81a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4be1ff700 1 -- 172.21.15.19:0/519432447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8107df0 msgr2=0x7fd4b81a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4be1ff700 1 --2- 172.21.15.19:0/519432447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8107df0 0x7fd4b81a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4be1ff700 1 -- 172.21.15.19:0/519432447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4b81034c0 msgr2=0x7fd4b81a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4be1ff700 1 --2- 172.21.15.19:0/519432447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4b81034c0 0x7fd4b81a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4be1ff700 1 -- 172.21.15.19:0/519432447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b81a62b0 con 0x7fd4b8108740 2024-05-09T02:11:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4bd9fe700 1 --2- 172.21.15.19:0/519432447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4b81034c0 0x7fd4b81a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4bd1fd700 1 --2- 172.21.15.19:0/519432447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8107df0 0x7fd4b81a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4be1ff700 1 --2- 172.21.15.19:0/519432447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b8108740 0x7fd4b81a5c10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0007a00 tx=0x7fd4b0007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4aeffd700 1 -- 172.21.15.19:0/519432447 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b001f070 con 0x7fd4b8108740 2024-05-09T02:11:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4bfc62700 1 -- 172.21.15.19:0/519432447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b81a6540 con 0x7fd4b8108740 2024-05-09T02:11:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4aeffd700 1 -- 172.21.15.19:0/519432447 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b0004c50 con 0x7fd4b8108740 2024-05-09T02:11:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4aeffd700 1 -- 172.21.15.19:0/519432447 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b001b690 con 0x7fd4b8108740 2024-05-09T02:11:11.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.480+0000 7fd4bfc62700 1 -- 172.21.15.19:0/519432447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b81a6a20 con 0x7fd4b8108740 2024-05-09T02:11:11.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.481+0000 7fd4bfc62700 1 -- 172.21.15.19:0/519432447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b80512c0 con 0x7fd4b8108740 2024-05-09T02:11:11.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.484+0000 7fd4aeffd700 1 -- 172.21.15.19:0/519432447 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7fd4b0022070 con 0x7fd4b8108740 2024-05-09T02:11:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.485+0000 7fd4aeffd700 1 --2- 172.21.15.19:0/519432447 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd4a40b28d0 0x7fd4a40b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.485+0000 7fd4bd9fe700 1 -- 172.21.15.19:0/519432447 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd4a40b28d0 msgr2=0x7fd4a40b4de0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/972543870 2024-05-09T02:11:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.485+0000 7fd4aeffd700 1 -- 172.21.15.19:0/519432447 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fd4b000f020 con 0x7fd4b8108740 2024-05-09T02:11:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.485+0000 7fd4bd9fe700 1 --2- 172.21.15.19:0/519432447 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd4a40b28d0 0x7fd4a40b4de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T02:11:11.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.488+0000 7fd4aeffd700 1 -- 172.21.15.19:0/519432447 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd4b0029020 con 0x7fd4b8108740 2024-05-09T02:11:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.686+0000 7fd4bd9fe700 1 -- 172.21.15.19:0/519432447 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd4a40b28d0 msgr2=0x7fd4a40b4de0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/972543870 2024-05-09T02:11:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.686+0000 7fd4bd9fe700 1 --2- 172.21.15.19:0/519432447 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd4a40b28d0 0x7fd4a40b4de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T02:11:11.704 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:11 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:11.356+0000 7fe3b6b21200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T02:11:11.704 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:11 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:11.529+0000 7fe3b6b21200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T02:11:11.716 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:11 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:11.446+0000 7fe4a67a8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T02:11:11.716 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:11 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:11.626+0000 7fe4a67a8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T02:11:11.731 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:11 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:11.545+0000 7f9c8b9aa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T02:11:11.731 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:11 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:11.730+0000 7f9c8b9aa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T02:11:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.752+0000 7fd4bfc62700 1 -- 172.21.15.19:0/519432447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd4b81adec0 con 0x7fd4b8108740 2024-05-09T02:11:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.752+0000 7fd4aeffd700 1 -- 172.21.15.19:0/519432447 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7fd4b00ab4a0 con 0x7fd4b8108740 2024-05-09T02:11:11.754 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T02:11:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.754+0000 7fd4bfc62700 1 -- 172.21.15.19:0/519432447 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd4a40b28d0 msgr2=0x7fd4a40b4de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:11:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.755+0000 7fd4bfc62700 1 --2- 172.21.15.19:0/519432447 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd4a40b28d0 0x7fd4a40b4de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:11.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.755+0000 7fd4bfc62700 1 -- 172.21.15.19:0/519432447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b8108740 msgr2=0x7fd4b81a5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.755+0000 7fd4bfc62700 1 --2- 172.21.15.19:0/519432447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b8108740 0x7fd4b81a5c10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0007a00 tx=0x7fd4b0007a30 comp rx=0 tx=0).stop 2024-05-09T02:11:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.755+0000 7fd4bfc62700 1 -- 172.21.15.19:0/519432447 shutdown_connections 2024-05-09T02:11:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.755+0000 7fd4bfc62700 1 --2- 172.21.15.19:0/519432447 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fd4a40b28d0 0x7fd4a40b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.755+0000 7fd4bfc62700 1 --2- 172.21.15.19:0/519432447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b8108740 0x7fd4b81a5c10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.755+0000 7fd4bfc62700 1 --2- 172.21.15.19:0/519432447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4b81034c0 0x7fd4b81a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.755+0000 7fd4bfc62700 1 --2- 172.21.15.19:0/519432447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8107df0 0x7fd4b81a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.755+0000 7fd4bfc62700 1 -- 172.21.15.19:0/519432447 >> 172.21.15.19:0/519432447 conn(0x7fd4b80fedc0 msgr2=0x7fd4b8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.755+0000 7fd4bfc62700 1 -- 172.21.15.19:0/519432447 shutdown_connections 2024-05-09T02:11:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:11.755+0000 7fd4bfc62700 1 -- 172.21.15.19:0/519432447 wait complete. 2024-05-09T02:11:11.998 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:11 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/519432447' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T02:11:11.998 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:11 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:11.702+0000 7fe3b6b21200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T02:11:12.067 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:11 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/519432447' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T02:11:12.067 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:11 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:11.914+0000 7f9c8b9aa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T02:11:12.109 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:11 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/519432447' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T02:11:12.109 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:11 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:11.804+0000 7fe4a67a8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T02:11:12.294 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:11 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:11.995+0000 7fe3b6b21200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T02:11:12.295 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:12 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:12.089+0000 7fe3b6b21200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T02:11:12.321 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:12 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:12.227+0000 7f9c8b9aa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T02:11:12.407 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T02:11:12.407 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T02:11:12.407 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-05-09T02:11:12.407 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T02:11:12.407 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-05-09T02:11:12.407 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T02:11:12.407 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T02:11:12.407 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T02:11:12.407 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-05-09T02:11:12.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T02:11:12.408 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-05-09T02:11:12.418 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:12 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:12.107+0000 7fe4a67a8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T02:11:12.418 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:12 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:12.204+0000 7fe4a67a8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T02:11:12.425 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T02:11:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T02:11:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T02:11:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-09T02:11:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-05-09T02:11:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-09T02:11:12.426 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T02:11:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-05-09T02:11:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-05-09T02:11:12.426 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T02:11:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-05-09T02:11:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-05-09T02:11:12.426 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T02:11:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-05-09T02:11:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-05-09T02:11:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T02:11:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42011sec preferred_lft 42011sec 2024-05-09T02:11:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-05-09T02:11:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-09T02:11:12.427 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T02:11:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-05-09T02:11:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-05-09T02:11:12.427 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T02:11:12.427 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T02:11:12.427 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-09T02:11:12.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-05-09T02:11:12.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-05-09T02:11:12.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-05-09T02:11:12.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T02:11:12.428 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-09T02:11:12.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T02:11:12.593 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:12 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:12.327+0000 7f9c8b9aa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T02:11:12.594 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:12 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:12.549+0000 7f9c8b9aa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T02:11:12.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T02:11:12.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T02:11:12.639 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T02:11:12.666 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-05-09T02:11:12.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T02:11:12.668 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-05-09T02:11:12.716 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:12 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:12.416+0000 7fe4a67a8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T02:11:12.716 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:12 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:12.618+0000 7fe4a67a8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T02:11:12.734 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-05-09T02:11:12.734 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T02:11:12.734 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T02:11:12.734 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T02:11:12.734 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T02:11:12.735 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T02:11:12.735 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-09T02:11:12.735 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T02:11:12.735 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T02:11:12.736 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T02:11:12.736 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-09T02:11:12.794 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:12 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:12.296+0000 7fe3b6b21200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T02:11:12.794 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:12 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:12.493+0000 7fe3b6b21200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T02:11:12.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T02:11:12.963 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:12 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:12.771+0000 7f9c8b9aa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T02:11:13.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T02:11:13.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T02:11:13.071 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-05-09T02:11:13.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T02:11:13.089 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-05-09T02:11:13.145 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T02:11:13.145 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-09T02:11:13.145 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--tmp-cephfs 2024-05-09T02:11:13.145 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-05-09T02:11:13.145 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-09T02:11:13.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T02:11:13.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T02:11:13.271 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-05-09T02:11:13.271 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T02:11:13.271 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-09T02:11:13.271 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-05-09T02:11:13.271 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T02:11:13.271 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-05-09T02:11:13.271 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-05-09T02:11:13.272 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-05-09T02:11:13.272 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-09T02:11:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T02:11:13.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T02:11:13.406 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T02:11:13.406 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-09T02:11:13.406 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-05-09T02:11:13.407 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T02:11:13.407 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-09T02:11:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T02:11:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T02:11:13.490 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T02:11:13.491 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /tmp/cephfs 2024-05-09T02:11:13.507 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/tmp/cephfs' 2024-05-09T02:11:13.507 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T02:11:13.507 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /tmp/cephfs 2024-05-09T02:11:13.544 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:13 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:13.097+0000 7fe3b6b21200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T02:11:13.544 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:13 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:13.323+0000 7fe3b6b21200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T02:11:13.563 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-09T02:11:13.563 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-09T02:11:13.563 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/3263a7e6-90e1-4c3c-b293-25c3c473ace0 2024-05-09T02:11:13.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T02:11:13.563 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/3263a7e6-90e1-4c3c-b293-25c3c473ace0 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2024-05-09T02:11:13.571 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:13 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:13.438+0000 7f9c8b9aa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T02:11:13.640 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-05-09T02:11:13.640 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-05-09T02:11:13.716 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:13 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:13.252+0000 7fe4a67a8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T02:11:13.716 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:13 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:13.485+0000 7fe4a67a8200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T02:11:13.763 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-05-09T02:11:13.763 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-05-09T02:11:13.763 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.59:3300,172.21.15.196:3300" 2024-05-09T02:11:13.763 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@a0611f3e-0da7-11ef-bc97-c7b262605968.cephfs=/volumes/qa/sv_1/3263a7e6-90e1-4c3c-b293-25c3c473ace0 2024-05-09T02:11:13.763 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.19:3300/172.21.15.59:3300/172.21.15.196:3300" will pass to kernel 2024-05-09T02:11:13.766 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T02:11:13.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T02:11:13.766 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /tmp/cephfs 2024-05-09T02:11:13.794 DEBUG:teuthology.orchestra.run.smithi019:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T02:11:13.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T02:11:13.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T02:11:13.894 INFO:teuthology.orchestra.run.smithi019.stdout:client15231 2024-05-09T02:11:13.894 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T02:11:13.895 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T02:11:13.912 DEBUG:teuthology.orchestra.run.smithi019:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T02:11:14.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T02:11:14.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T02:11:14.012 INFO:teuthology.orchestra.run.smithi019.stdout:client15231 2024-05-09T02:11:14.013 DEBUG:teuthology.orchestra.run.smithi019:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T02:11:14.044 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:13 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:13.754+0000 7fe3b6b21200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T02:11:14.044 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:13 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:13.852+0000 7fe3b6b21200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T02:11:14.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T02:11:14.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T02:11:14.071 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:13 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:13.687+0000 7f9c8b9aa200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T02:11:14.072 INFO:teuthology.orchestra.run.smithi019.stdout:client15231 2024-05-09T02:11:14.073 DEBUG:teuthology.orchestra.run.smithi019:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968.client15231/status') 2024-05-09T02:11:14.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T02:11:14.135 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.15231 (3)192.168.0.1:0/706009415 2024-05-09T02:11:14.135 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-05-09T02:11:14.135 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-05-09T02:11:14.135 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-05-09T02:11:14.135 INFO:teuthology.orchestra.run.smithi019.stderr:69 bytes copied, 2.6656e-05 s, 2.6 MB/s 2024-05-09T02:11:14.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T02:11:14.143 INFO:teuthology.run_tasks:Running task exec... 2024-05-09T02:11:14.152 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T02:11:14.152 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T02:11:14.152 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-09T02:11:14.216 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:13 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:13.931+0000 7fe4a67a8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T02:11:14.216 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:14 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:14.034+0000 7fe4a67a8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T02:11:14.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.268+0000 7f2344b8d640 1 -- 172.21.15.19:0/726185372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2340108a90 msgr2=0x7f234010aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:14.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.268+0000 7f2344b8d640 1 --2- 172.21.15.19:0/726185372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2340108a90 0x7f234010aeb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2338009ca0 tx=0x7f233802a8c0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.268+0000 7f2344b8d640 1 -- 172.21.15.19:0/726185372 shutdown_connections 2024-05-09T02:11:14.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.268+0000 7f2344b8d640 1 --2- 172.21.15.19:0/726185372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2340108a90 0x7f234010aeb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.268+0000 7f2344b8d640 1 --2- 172.21.15.19:0/726185372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2340103d90 0x7f23401041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.268+0000 7f2344b8d640 1 --2- 172.21.15.19:0/726185372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2340102b50 0x7f2340103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.268+0000 7f2344b8d640 1 -- 172.21.15.19:0/726185372 >> 172.21.15.19:0/726185372 conn(0x7f23400fdd40 msgr2=0x7f23401001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:14.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.268+0000 7f2344b8d640 1 -- 172.21.15.19:0/726185372 shutdown_connections 2024-05-09T02:11:14.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.268+0000 7f2344b8d640 1 -- 172.21.15.19:0/726185372 wait complete. 2024-05-09T02:11:14.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.269+0000 7f2344b8d640 1 Processor -- start 2024-05-09T02:11:14.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.269+0000 7f2344b8d640 1 -- start start 2024-05-09T02:11:14.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.269+0000 7f2344b8d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2340102b50 0x7f234006af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:14.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.269+0000 7f2344b8d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2340103d90 0x7f234019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:14.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.269+0000 7f2344b8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2340108a90 0x7f23401a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:14.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.269+0000 7f2344b8d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f234010e660 con 0x7f2340103d90 2024-05-09T02:11:14.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.269+0000 7f2344b8d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f234010e4e0 con 0x7f2340102b50 2024-05-09T02:11:14.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.269+0000 7f2344b8d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f234010e7e0 con 0x7f2340108a90 2024-05-09T02:11:14.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f2337fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2340103d90 0x7f234019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:14.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f233e59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2340102b50 0x7f234006af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:14.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f2337fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2340103d90 0x7f234019f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33082/0 (socket says 172.21.15.19:33082) 2024-05-09T02:11:14.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f2337fff640 1 -- 172.21.15.19:0/3113310403 learned_addr learned my addr 172.21.15.19:0/3113310403 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:14.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f233ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2340108a90 0x7f23401a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:14.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f233e59c640 1 -- 172.21.15.19:0/3113310403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2340108a90 msgr2=0x7f23401a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:14.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f233e59c640 1 --2- 172.21.15.19:0/3113310403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2340108a90 0x7f23401a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f233e59c640 1 -- 172.21.15.19:0/3113310403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2340103d90 msgr2=0x7f234019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f233e59c640 1 --2- 172.21.15.19:0/3113310403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2340103d90 0x7f234019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f233e59c640 1 -- 172.21.15.19:0/3113310403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23401a1e00 con 0x7f2340102b50 2024-05-09T02:11:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f2337fff640 1 --2- 172.21.15.19:0/3113310403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2340103d90 0x7f234019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f233ed9d640 1 --2- 172.21.15.19:0/3113310403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2340108a90 0x7f23401a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f233e59c640 1 --2- 172.21.15.19:0/3113310403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2340102b50 0x7f234006af60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2340102420 tx=0x7f233000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f23377fe640 1 -- 172.21.15.19:0/3113310403 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233000cc40 con 0x7f2340102b50 2024-05-09T02:11:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f23377fe640 1 -- 172.21.15.19:0/3113310403 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f233000cde0 con 0x7f2340102b50 2024-05-09T02:11:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.270+0000 7f2344b8d640 1 -- 172.21.15.19:0/3113310403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23401a20f0 con 0x7f2340102b50 2024-05-09T02:11:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.271+0000 7f23377fe640 1 -- 172.21.15.19:0/3113310403 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330010430 con 0x7f2340102b50 2024-05-09T02:11:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.271+0000 7f2344b8d640 1 -- 172.21.15.19:0/3113310403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23401a2640 con 0x7f2340102b50 2024-05-09T02:11:14.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.272+0000 7f2344b8d640 1 -- 172.21.15.19:0/3113310403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23401078a0 con 0x7f2340102b50 2024-05-09T02:11:14.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.273+0000 7f23377fe640 1 -- 172.21.15.19:0/3113310403 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f2330004020 con 0x7f2340102b50 2024-05-09T02:11:14.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.274+0000 7f23377fe640 1 --2- 172.21.15.19:0/3113310403 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f232c0b2830 0x7f232c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.274+0000 7f2337fff640 1 -- 172.21.15.19:0/3113310403 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f232c0b2830 msgr2=0x7f232c0b4d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/972543870 2024-05-09T02:11:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.274+0000 7f23377fe640 1 -- 172.21.15.19:0/3113310403 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f23300e2700 con 0x7f2340102b50 2024-05-09T02:11:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.274+0000 7f2337fff640 1 --2- 172.21.15.19:0/3113310403 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f232c0b2830 0x7f232c0b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T02:11:14.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.279+0000 7f23377fe640 1 -- 172.21.15.19:0/3113310403 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f23300aa400 con 0x7f2340102b50 2024-05-09T02:11:14.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.475+0000 7f2337fff640 1 -- 172.21.15.19:0/3113310403 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f232c0b2830 msgr2=0x7f232c0b4d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/972543870 2024-05-09T02:11:14.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.475+0000 7f2337fff640 1 --2- 172.21.15.19:0/3113310403 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f232c0b2830 0x7f232c0b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T02:11:14.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.516+0000 7f2344b8d640 1 -- 172.21.15.19:0/3113310403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f234002dff0 con 0x7f2340102b50 2024-05-09T02:11:14.544 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:14 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:14.229+0000 7fe3b6b21200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T02:11:14.571 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:14 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:14.156+0000 7f9c8b9aa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T02:11:14.571 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:14 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:14.266+0000 7f9c8b9aa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T02:11:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.572+0000 7f23377fe640 1 -- 172.21.15.19:0/3113310403 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7f23300a9d00 con 0x7f2340102b50 2024-05-09T02:11:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.574+0000 7f2344b8d640 1 -- 172.21.15.19:0/3113310403 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f232c0b2830 msgr2=0x7f232c0b4d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:11:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.574+0000 7f2344b8d640 1 --2- 172.21.15.19:0/3113310403 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f232c0b2830 0x7f232c0b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.574+0000 7f2344b8d640 1 -- 172.21.15.19:0/3113310403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2340102b50 msgr2=0x7f234006af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.574+0000 7f2344b8d640 1 --2- 172.21.15.19:0/3113310403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2340102b50 0x7f234006af60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2340102420 tx=0x7f233000ee50 comp rx=0 tx=0).stop 2024-05-09T02:11:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.574+0000 7f2344b8d640 1 -- 172.21.15.19:0/3113310403 shutdown_connections 2024-05-09T02:11:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.574+0000 7f2344b8d640 1 --2- 172.21.15.19:0/3113310403 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f232c0b2830 0x7f232c0b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.574+0000 7f2344b8d640 1 --2- 172.21.15.19:0/3113310403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2340108a90 0x7f23401a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.574+0000 7f2344b8d640 1 --2- 172.21.15.19:0/3113310403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2340103d90 0x7f234019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.574+0000 7f2344b8d640 1 --2- 172.21.15.19:0/3113310403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2340102b50 0x7f234006af60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.574+0000 7f2344b8d640 1 -- 172.21.15.19:0/3113310403 >> 172.21.15.19:0/3113310403 conn(0x7f23400fdd40 msgr2=0x7f2340104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.574+0000 7f2344b8d640 1 -- 172.21.15.19:0/3113310403 shutdown_connections 2024-05-09T02:11:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.574+0000 7f2344b8d640 1 -- 172.21.15.19:0/3113310403 wait complete. 2024-05-09T02:11:14.587 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-09T02:11:14.598 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-09T02:11:14.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T02:11:14.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:14 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3113310403' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T02:11:14.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:14 smithi059 ceph-mon[34781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T02:11:14.716 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:14 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:14.420+0000 7fe4a67a8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T02:11:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.766+0000 7f5605096640 1 -- 172.21.15.19:0/3908117344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5600102af0 msgr2=0x7f56001037f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.766+0000 7f5605096640 1 --2- 172.21.15.19:0/3908117344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5600102af0 0x7f56001037f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009900 tx=0x7f55f002a410 comp rx=0 tx=0).stop 2024-05-09T02:11:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.766+0000 7f5605096640 1 -- 172.21.15.19:0/3908117344 shutdown_connections 2024-05-09T02:11:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.766+0000 7f5605096640 1 --2- 172.21.15.19:0/3908117344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5600108a30 0x7f560010ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.766+0000 7f5605096640 1 --2- 172.21.15.19:0/3908117344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5600103d30 0x7f5600104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.766+0000 7f5605096640 1 --2- 172.21.15.19:0/3908117344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5600102af0 0x7f56001037f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.766+0000 7f5605096640 1 -- 172.21.15.19:0/3908117344 >> 172.21.15.19:0/3908117344 conn(0x7f56000fdd00 msgr2=0x7f5600100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.766+0000 7f5605096640 1 -- 172.21.15.19:0/3908117344 shutdown_connections 2024-05-09T02:11:14.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.766+0000 7f5605096640 1 -- 172.21.15.19:0/3908117344 wait complete. 2024-05-09T02:11:14.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.767+0000 7f5605096640 1 Processor -- start 2024-05-09T02:11:14.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.767+0000 7f5605096640 1 -- start start 2024-05-09T02:11:14.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.767+0000 7f5605096640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5600102af0 0x7f560006aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:14.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.767+0000 7f5605096640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5600103d30 0x7f560019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:14.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.767+0000 7f5605096640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5600108a30 0x7f56001a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:14.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.767+0000 7f5605096640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f560010e5d0 con 0x7f5600108a30 2024-05-09T02:11:14.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.767+0000 7f5605096640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f560010e450 con 0x7f5600103d30 2024-05-09T02:11:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.767+0000 7f5605096640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f560010e750 con 0x7f5600102af0 2024-05-09T02:11:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.768+0000 7f55ff59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5600108a30 0x7f56001a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.768+0000 7f55ff59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5600108a30 0x7f56001a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33098/0 (socket says 172.21.15.19:33098) 2024-05-09T02:11:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.768+0000 7f55ff59e640 1 -- 172.21.15.19:0/3483866041 learned_addr learned my addr 172.21.15.19:0/3483866041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.768+0000 7f55fe59c640 1 --2- 172.21.15.19:0/3483866041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5600103d30 0x7f560019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.768+0000 7f55fed9d640 1 --2- 172.21.15.19:0/3483866041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5600102af0 0x7f560006aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.768+0000 7f55fe59c640 1 -- 172.21.15.19:0/3483866041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5600102af0 msgr2=0x7f560006aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.768+0000 7f55fe59c640 1 --2- 172.21.15.19:0/3483866041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5600102af0 0x7f560006aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.768+0000 7f55fe59c640 1 -- 172.21.15.19:0/3483866041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5600108a30 msgr2=0x7f56001a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.768+0000 7f55fe59c640 1 --2- 172.21.15.19:0/3483866041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5600108a30 0x7f56001a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.768+0000 7f55fe59c640 1 -- 172.21.15.19:0/3483866041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56001a1e80 con 0x7f5600103d30 2024-05-09T02:11:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.768+0000 7f55ff59e640 1 --2- 172.21.15.19:0/3483866041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5600108a30 0x7f56001a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.768+0000 7f55fe59c640 1 --2- 172.21.15.19:0/3483866041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5600103d30 0x7f560019f0f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f55f400e9f0 tx=0x7f55f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.768+0000 7f55e7fff640 1 -- 172.21.15.19:0/3483866041 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f400ccc0 con 0x7f5600103d30 2024-05-09T02:11:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.769+0000 7f5605096640 1 -- 172.21.15.19:0/3483866041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56001a2170 con 0x7f5600103d30 2024-05-09T02:11:14.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.769+0000 7f55e7fff640 1 -- 172.21.15.19:0/3483866041 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f40108e0 con 0x7f5600103d30 2024-05-09T02:11:14.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.769+0000 7f55e7fff640 1 -- 172.21.15.19:0/3483866041 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4018880 con 0x7f5600103d30 2024-05-09T02:11:14.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.769+0000 7f5605096640 1 -- 172.21.15.19:0/3483866041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56001a2710 con 0x7f5600103d30 2024-05-09T02:11:14.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.770+0000 7f5605096640 1 -- 172.21.15.19:0/3483866041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5600107840 con 0x7f5600103d30 2024-05-09T02:11:14.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.771+0000 7f55e7fff640 1 -- 172.21.15.19:0/3483866041 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f55f4010430 con 0x7f5600103d30 2024-05-09T02:11:14.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.772+0000 7f55e7fff640 1 --2- 172.21.15.19:0/3483866041 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f55e80b2840 0x7f55e80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:14.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.772+0000 7f55fed9d640 1 -- 172.21.15.19:0/3483866041 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f55e80b2840 msgr2=0x7f55e80b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/972543870 2024-05-09T02:11:14.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.772+0000 7f55e7fff640 1 -- 172.21.15.19:0/3483866041 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f55f40e25d0 con 0x7f5600103d30 2024-05-09T02:11:14.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.772+0000 7f55fed9d640 1 --2- 172.21.15.19:0/3483866041 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f55e80b2840 0x7f55e80b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T02:11:14.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.777+0000 7f55e7fff640 1 -- 172.21.15.19:0/3483866041 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f55f40a9c80 con 0x7f5600103d30 2024-05-09T02:11:14.932 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:14 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3113310403' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T02:11:14.932 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:14 smithi196 ceph-mon[34718]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T02:11:14.932 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:14 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:14.590+0000 7fe3b6b21200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T02:11:14.932 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:14 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:14.684+0000 7fe3b6b21200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T02:11:14.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.973+0000 7f55fed9d640 1 -- 172.21.15.19:0/3483866041 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f55e80b2840 msgr2=0x7f55e80b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/972543870 2024-05-09T02:11:14.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.973+0000 7f55fed9d640 1 --2- 172.21.15.19:0/3483866041 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f55e80b2840 0x7f55e80b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T02:11:14.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.974+0000 7f5605096640 1 -- 172.21.15.19:0/3483866041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f560002dc30 con 0x7f5600103d30 2024-05-09T02:11:14.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.975+0000 7f55e7fff640 1 -- 172.21.15.19:0/3483866041 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22698 (secure 0 0 0) 0x7f55f40adf80 con 0x7f5600103d30 2024-05-09T02:11:14.977 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:14.977 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"fsid":"a0611f3e-0da7-11ef-bc97-c7b262605968","created":"2024-05-09T01:59:27.591343+0000","modified":"2024-05-09T02:08:25.302927+0000","last_up_change":"2024-05-09T02:06:45.565065+0000","last_in_change":"2024-05-09T02:06:24.967186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T02:02:50.018959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T02:07:43.283868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T02:07:47.110614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"996c6a3d-3e3a-451c-8789-36006e0b3dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6803","nonce":3002373576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6805","nonce":3002373576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6809","nonce":3002373576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6807","nonce":3002373576}]},"public_addr":"172.21.15.19:6803/3002373576","cluster_addr":"172.21.15.19:6805/3002373576","heartbeat_back_addr":"172.21.15.19:6809/3002373576","heartbeat_front_addr":"172.21.15.19:6807/3002373576","state":["exists","up"]},{"osd":1,"uuid":"1ac68d09-1198-4e81-86c7-e37f7c41aa5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6801","nonce":827458353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6803","nonce":827458353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6807","nonce":827458353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6805","nonce":827458353}]},"public_addr":"172.21.15.59:6801/827458353","cluster_addr":"172.21.15.59:6803/827458353","heartbeat_back_addr":"172.21.15.59:6807/827458353","heartbeat_front_addr":"172.21.15.59:6805/827458353","state":["exists","up"]},{"osd":2,"uuid":"818a0de1-04e6-471d-83df-a68833011b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6801","nonce":1659504639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6803","nonce":1659504639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6807","nonce":1659504639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6805","nonce":1659504639}]},"public_addr":"172.21.15.196:6801/1659504639","cluster_addr":"172.21.15.196:6803/1659504639","heartbeat_back_addr":"172.21.15.196:6807/1659504639","heartbeat_front_addr":"172.21.15.196:6805/1659504639","state":["exists","up"]},{"osd":3,"uuid":"7b204854-86b8-454d-aee0-8bb5775ce828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6811","nonce":2707671599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6813","nonce":2707671599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6817","nonce":2707671599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6815","nonce":2707671599}]},"public_addr":"172.21.15.19:6811/2707671599","cluster_addr":"172.21.15.19:6813/2707671599","heartbeat_back_addr":"172.21.15.19:6817/2707671599","heartbeat_front_addr":"172.21.15.19:6815/2707671599","state":["exists","up"]},{"osd":4,"uuid":"091667f1-e887-4a94-8723-6f85b676fef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6809","nonce":223037330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6811","nonce":223037330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6815","nonce":223037330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6813","nonce":223037330}]},"public_addr":"172.21.15.59:6809/223037330","cluster_addr":"172.21.15.59:6811/223037330","heartbeat_back_addr":"172.21.15.59:6815/223037330","heartbeat_front_addr":"172.21.15.59:6813/223037330","state":["exists","up"]},{"osd":5,"uuid":"2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6809","nonce":2516161898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6811","nonce":2516161898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6815","nonce":2516161898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6813","nonce":2516161898}]},"public_addr":"172.21.15.196:6809/2516161898","cluster_addr":"172.21.15.196:6811/2516161898","heartbeat_back_addr":"172.21.15.196:6815/2516161898","heartbeat_front_addr":"172.21.15.196:6813/2516161898","state":["exists","up"]},{"osd":6,"uuid":"4e352851-1279-475f-ba8b-eaf9ca73df74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6819","nonce":2150451637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6821","nonce":2150451637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6825","nonce":2150451637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6823","nonce":2150451637}]},"public_addr":"172.21.15.19:6819/2150451637","cluster_addr":"172.21.15.19:6821/2150451637","heartbeat_back_addr":"172.21.15.19:6825/2150451637","heartbeat_front_addr":"172.21.15.19:6823/2150451637","state":["exists","up"]},{"osd":7,"uuid":"83f53f59-9a60-4d87-9a8a-67e5c6f57004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6817","nonce":967902053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6819","nonce":967902053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6823","nonce":967902053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6821","nonce":967902053}]},"public_addr":"172.21.15.59:6817/967902053","cluster_addr":"172.21.15.59:6819/967902053","heartbeat_back_addr":"172.21.15.59:6823/967902053","heartbeat_front_addr":"172.21.15.59:6821/967902053","state":["exists","up"]},{"osd":8,"uuid":"a8f1bfb5-1cc8-4a65-8396-239bd781e1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6817","nonce":2946064259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6819","nonce":2946064259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6823","nonce":2946064259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6821","nonce":2946064259}]},"public_addr":"172.21.15.196:6817/2946064259","cluster_addr":"172.21.15.196:6819/2946064259","heartbeat_back_addr":"172.21.15.196:6823/2946064259","heartbeat_front_addr":"172.21.15.196:6821/2946064259","state":["exists","up"]},{"osd":9,"uuid":"73c58dc3-fcb0-4440-92b3-bf41a2e58fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6827","nonce":1735017886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6829","nonce":1735017886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6833","nonce":1735017886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6831","nonce":1735017886}]},"public_addr":"172.21.15.19:6827/1735017886","cluster_addr":"172.21.15.19:6829/1735017886","heartbeat_back_addr":"172.21.15.19:6833/1735017886","heartbeat_front_addr":"172.21.15.19:6831/1735017886","state":["exists","up"]},{"osd":10,"uuid":"8325282f-7325-4880-ab7d-b972717be9dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6825","nonce":2932899995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6827","nonce":2932899995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6831","nonce":2932899995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6829","nonce":2932899995}]},"public_addr":"172.21.15.59:6825/2932899995","cluster_addr":"172.21.15.59:6827/2932899995","heartbeat_back_addr":"172.21.15.59:6831/2932899995","heartbeat_front_addr":"172.21.15.59:6829/2932899995","state":["exists","up"]},{"osd":11,"uuid":"8b19228c-b286-43c8-a408-169782f13075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6825","nonce":2608023001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6827","nonce":2608023001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6831","nonce":2608023001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6829","nonce":2608023001}]},"public_addr":"172.21.15.196:6825/2608023001","cluster_addr":"172.21.15.196:6827/2608023001","heartbeat_back_addr":"172.21.15.196:6831/2608023001","heartbeat_front_addr":"172.21.15.196:6829/2608023001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:01:59.457748+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:23.640666+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:47.768243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:13.016746+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:39.060898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:04.508313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:31.366217+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:57.579758+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:23.177396+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:50.966399+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:16.838666+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:43.959387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3291838961":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1024133854":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1176455101":"2024-05-10T02:00:23.920230+0000","172.21.15.19:6801/2101052000":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/4018987404":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/3079777324":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6801/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/179703528":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/2101052000":"2024-05-10T02:00:23.920230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T02:11:14.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.978+0000 7f5605096640 1 -- 172.21.15.19:0/3483866041 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f55e80b2840 msgr2=0x7f55e80b4d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:11:14.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.978+0000 7f5605096640 1 --2- 172.21.15.19:0/3483866041 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f55e80b2840 0x7f55e80b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.978+0000 7f5605096640 1 -- 172.21.15.19:0/3483866041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5600103d30 msgr2=0x7f560019f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:14.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.978+0000 7f5605096640 1 --2- 172.21.15.19:0/3483866041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5600103d30 0x7f560019f0f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f55f400e9f0 tx=0x7f55f400eeb0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.978+0000 7f5605096640 1 -- 172.21.15.19:0/3483866041 shutdown_connections 2024-05-09T02:11:14.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.978+0000 7f5605096640 1 --2- 172.21.15.19:0/3483866041 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f55e80b2840 0x7f55e80b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.978+0000 7f5605096640 1 --2- 172.21.15.19:0/3483866041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5600108a30 0x7f56001a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.978+0000 7f5605096640 1 --2- 172.21.15.19:0/3483866041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5600103d30 0x7f560019f0f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.978+0000 7f5605096640 1 --2- 172.21.15.19:0/3483866041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5600102af0 0x7f560006aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:14.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.978+0000 7f5605096640 1 -- 172.21.15.19:0/3483866041 >> 172.21.15.19:0/3483866041 conn(0x7f56000fdd00 msgr2=0x7f5600104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:14.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.978+0000 7f5605096640 1 -- 172.21.15.19:0/3483866041 shutdown_connections 2024-05-09T02:11:14.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:14.978+0000 7f5605096640 1 -- 172.21.15.19:0/3483866041 wait complete. 2024-05-09T02:11:14.989 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:14 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:14.663+0000 7f9c8b9aa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T02:11:14.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:14 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3113310403' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T02:11:14.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:14 smithi019 ceph-mon[30151]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T02:11:14.991 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T02:02:50.018959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T02:07:43.283868+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T02:07:47.110614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T02:11:14.991 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T02:11:15.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.109+0000 7f3b8759e640 1 -- 172.21.15.19:0/436312567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b88104c80 msgr2=0x7f3b88105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:15.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.109+0000 7f3b8759e640 1 --2- 172.21.15.19:0/436312567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b88104c80 0x7f3b88105090 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009900 tx=0x7f3b7002a410 comp rx=0 tx=0).stop 2024-05-09T02:11:15.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.109+0000 7f3b8759e640 1 -- 172.21.15.19:0/436312567 shutdown_connections 2024-05-09T02:11:15.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.109+0000 7f3b8759e640 1 --2- 172.21.15.19:0/436312567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b88109980 0x7f3b8810bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.109+0000 7f3b8759e640 1 --2- 172.21.15.19:0/436312567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b88104c80 0x7f3b88105090 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.109+0000 7f3b8759e640 1 --2- 172.21.15.19:0/436312567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b88103a40 0x7f3b88104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.109+0000 7f3b8759e640 1 -- 172.21.15.19:0/436312567 >> 172.21.15.19:0/436312567 conn(0x7f3b880fec50 msgr2=0x7f3b881010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:15.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.109+0000 7f3b8759e640 1 -- 172.21.15.19:0/436312567 shutdown_connections 2024-05-09T02:11:15.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.109+0000 7f3b8759e640 1 -- 172.21.15.19:0/436312567 wait complete. 2024-05-09T02:11:15.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.110+0000 7f3b8759e640 1 Processor -- start 2024-05-09T02:11:15.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.110+0000 7f3b8759e640 1 -- start start 2024-05-09T02:11:15.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.110+0000 7f3b8759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b88103a40 0x7f3b88076980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:15.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.110+0000 7f3b8759e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b88104c80 0x7f3b88075020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:15.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.110+0000 7f3b8759e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b88109980 0x7f3b88075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:15.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.110+0000 7f3b8759e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8810f560 con 0x7f3b88103a40 2024-05-09T02:11:15.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.110+0000 7f3b8759e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8810f3e0 con 0x7f3b88104c80 2024-05-09T02:11:15.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.110+0000 7f3b8759e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8810f6e0 con 0x7f3b88109980 2024-05-09T02:11:15.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.110+0000 7f3b8659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b88103a40 0x7f3b88076980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.111+0000 7f3b8659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b88103a40 0x7f3b88076980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33118/0 (socket says 172.21.15.19:33118) 2024-05-09T02:11:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.111+0000 7f3b8659c640 1 -- 172.21.15.19:0/4220512158 learned_addr learned my addr 172.21.15.19:0/4220512158 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.111+0000 7f3b86d9d640 1 --2- 172.21.15.19:0/4220512158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b88109980 0x7f3b88075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.111+0000 7f3b7ffff640 1 --2- 172.21.15.19:0/4220512158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b88104c80 0x7f3b88075020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.111+0000 7f3b8659c640 1 -- 172.21.15.19:0/4220512158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b88109980 msgr2=0x7f3b88075560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.111+0000 7f3b8659c640 1 --2- 172.21.15.19:0/4220512158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b88109980 0x7f3b88075560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.111+0000 7f3b8659c640 1 -- 172.21.15.19:0/4220512158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b88104c80 msgr2=0x7f3b88075020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.111+0000 7f3b8659c640 1 --2- 172.21.15.19:0/4220512158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b88104c80 0x7f3b88075020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.111+0000 7f3b8659c640 1 -- 172.21.15.19:0/4220512158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b88075c90 con 0x7f3b88103a40 2024-05-09T02:11:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.111+0000 7f3b8659c640 1 --2- 172.21.15.19:0/4220512158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b88103a40 0x7f3b88076980 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800e9f0 tx=0x7f3b7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.111+0000 7f3b7f7fe640 1 -- 172.21.15.19:0/4220512158 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7800ccc0 con 0x7f3b88103a40 2024-05-09T02:11:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.111+0000 7f3b7f7fe640 1 -- 172.21.15.19:0/4220512158 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b780035f0 con 0x7f3b88103a40 2024-05-09T02:11:15.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.111+0000 7f3b8759e640 1 -- 172.21.15.19:0/4220512158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b881a4260 con 0x7f3b88103a40 2024-05-09T02:11:15.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.111+0000 7f3b7f7fe640 1 -- 172.21.15.19:0/4220512158 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7800ce60 con 0x7f3b88103a40 2024-05-09T02:11:15.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.111+0000 7f3b8759e640 1 -- 172.21.15.19:0/4220512158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b881a46d0 con 0x7f3b88103a40 2024-05-09T02:11:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.112+0000 7f3b8759e640 1 -- 172.21.15.19:0/4220512158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b88108790 con 0x7f3b88103a40 2024-05-09T02:11:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.114+0000 7f3b7f7fe640 1 -- 172.21.15.19:0/4220512158 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f3b78003aa0 con 0x7f3b88103a40 2024-05-09T02:11:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.115+0000 7f3b7f7fe640 1 --2- 172.21.15.19:0/4220512158 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3b740b2830 0x7f3b740b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.115+0000 7f3b7ffff640 1 -- 172.21.15.19:0/4220512158 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3b740b2830 msgr2=0x7f3b740b4d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/972543870 2024-05-09T02:11:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.115+0000 7f3b7f7fe640 1 -- 172.21.15.19:0/4220512158 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f3b780e2700 con 0x7f3b88103a40 2024-05-09T02:11:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.115+0000 7f3b7ffff640 1 --2- 172.21.15.19:0/4220512158 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3b740b2830 0x7f3b740b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T02:11:15.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.123+0000 7f3b7f7fe640 1 -- 172.21.15.19:0/4220512158 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3b780aa3a0 con 0x7f3b88103a40 2024-05-09T02:11:15.152 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:14 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:14.797+0000 7fe4a67a8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T02:11:15.153 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:14 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:14.894+0000 7fe4a67a8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T02:11:15.294 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:14 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:14.930+0000 7fe3b6b21200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T02:11:15.294 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:15 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:15.064+0000 7fe3b6b21200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T02:11:15.295 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:15 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:15.162+0000 7fe3b6b21200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T02:11:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.316+0000 7f3b7ffff640 1 -- 172.21.15.19:0/4220512158 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3b740b2830 msgr2=0x7f3b740b4d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/972543870 2024-05-09T02:11:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.316+0000 7f3b7ffff640 1 --2- 172.21.15.19:0/4220512158 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3b740b2830 0x7f3b740b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T02:11:15.321 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:15.058+0000 7f9c8b9aa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T02:11:15.321 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:15.167+0000 7f9c8b9aa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T02:11:15.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.321+0000 7f3b8759e640 1 -- 172.21.15.19:0/4220512158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3b880638d0 con 0x7f3b88103a40 2024-05-09T02:11:15.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.321+0000 7f3b7f7fe640 1 -- 172.21.15.19:0/4220512158 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f3b780b2420 con 0x7f3b88103a40 2024-05-09T02:11:15.323 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-05-09T02:11:15.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.324+0000 7f3b8759e640 1 -- 172.21.15.19:0/4220512158 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3b740b2830 msgr2=0x7f3b740b4d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:11:15.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.324+0000 7f3b8759e640 1 --2- 172.21.15.19:0/4220512158 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3b740b2830 0x7f3b740b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.324+0000 7f3b8759e640 1 -- 172.21.15.19:0/4220512158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b88103a40 msgr2=0x7f3b88076980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:15.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.324+0000 7f3b8759e640 1 --2- 172.21.15.19:0/4220512158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b88103a40 0x7f3b88076980 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800e9f0 tx=0x7f3b7800eeb0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.324+0000 7f3b8759e640 1 -- 172.21.15.19:0/4220512158 shutdown_connections 2024-05-09T02:11:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.324+0000 7f3b8759e640 1 --2- 172.21.15.19:0/4220512158 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f3b740b2830 0x7f3b740b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.324+0000 7f3b8759e640 1 --2- 172.21.15.19:0/4220512158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b88109980 0x7f3b88075560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.324+0000 7f3b8759e640 1 --2- 172.21.15.19:0/4220512158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b88104c80 0x7f3b88075020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.324+0000 7f3b8759e640 1 --2- 172.21.15.19:0/4220512158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b88103a40 0x7f3b88076980 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.324+0000 7f3b8759e640 1 -- 172.21.15.19:0/4220512158 >> 172.21.15.19:0/4220512158 conn(0x7f3b880fec50 msgr2=0x7f3b88105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.324+0000 7f3b8759e640 1 -- 172.21.15.19:0/4220512158 shutdown_connections 2024-05-09T02:11:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.324+0000 7f3b8759e640 1 -- 172.21.15.19:0/4220512158 wait complete. 2024-05-09T02:11:15.337 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T02:11:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.455+0000 7f7d4788b640 1 -- 172.21.15.19:0/2662452051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d401022b0 msgr2=0x7f7d40102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.455+0000 7f7d4788b640 1 --2- 172.21.15.19:0/2662452051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d401022b0 0x7f7d40102fb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c009900 tx=0x7f7d3c02a410 comp rx=0 tx=0).stop 2024-05-09T02:11:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.455+0000 7f7d4788b640 1 -- 172.21.15.19:0/2662452051 shutdown_connections 2024-05-09T02:11:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.455+0000 7f7d4788b640 1 --2- 172.21.15.19:0/2662452051 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d401081f0 0x7f7d4010a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.455+0000 7f7d4788b640 1 --2- 172.21.15.19:0/2662452051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d401034f0 0x7f7d40103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.455+0000 7f7d4788b640 1 --2- 172.21.15.19:0/2662452051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d401022b0 0x7f7d40102fb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.455+0000 7f7d4788b640 1 -- 172.21.15.19:0/2662452051 >> 172.21.15.19:0/2662452051 conn(0x7f7d400fd480 msgr2=0x7f7d400ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.455+0000 7f7d4788b640 1 -- 172.21.15.19:0/2662452051 shutdown_connections 2024-05-09T02:11:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.455+0000 7f7d4788b640 1 -- 172.21.15.19:0/2662452051 wait complete. 2024-05-09T02:11:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.455+0000 7f7d4788b640 1 Processor -- start 2024-05-09T02:11:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.455+0000 7f7d4788b640 1 -- start start 2024-05-09T02:11:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d4788b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d401022b0 0x7f7d4019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d4788b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d401034f0 0x7f7d4019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d4788b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d401081f0 0x7f7d401a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d4788b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4010f750 con 0x7f7d401034f0 2024-05-09T02:11:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d4788b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4010f5d0 con 0x7f7d401081f0 2024-05-09T02:11:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d4788b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4010f8d0 con 0x7f7d401022b0 2024-05-09T02:11:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d44e26640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d401034f0 0x7f7d4019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d44e26640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d401034f0 0x7f7d4019a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33136/0 (socket says 172.21.15.19:33136) 2024-05-09T02:11:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d44e26640 1 -- 172.21.15.19:0/3870325497 learned_addr learned my addr 172.21.15.19:0/3870325497 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d45627640 1 --2- 172.21.15.19:0/3870325497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d401022b0 0x7f7d4019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d45e28640 1 --2- 172.21.15.19:0/3870325497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d401081f0 0x7f7d401a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d44e26640 1 -- 172.21.15.19:0/3870325497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d401022b0 msgr2=0x7f7d4019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d44e26640 1 --2- 172.21.15.19:0/3870325497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d401022b0 0x7f7d4019a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d44e26640 1 -- 172.21.15.19:0/3870325497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d401081f0 msgr2=0x7f7d401a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d44e26640 1 --2- 172.21.15.19:0/3870325497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d401081f0 0x7f7d401a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d44e26640 1 -- 172.21.15.19:0/3870325497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d401a1ea0 con 0x7f7d401034f0 2024-05-09T02:11:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.456+0000 7f7d44e26640 1 --2- 172.21.15.19:0/3870325497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d401034f0 0x7f7d4019a910 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000e9f0 tx=0x7f7d3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:15.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.457+0000 7f7d367fc640 1 -- 172.21.15.19:0/3870325497 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000cd80 con 0x7f7d401034f0 2024-05-09T02:11:15.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.457+0000 7f7d4788b640 1 -- 172.21.15.19:0/3870325497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d401a2190 con 0x7f7d401034f0 2024-05-09T02:11:15.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.457+0000 7f7d367fc640 1 -- 172.21.15.19:0/3870325497 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d300054d0 con 0x7f7d401034f0 2024-05-09T02:11:15.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.457+0000 7f7d367fc640 1 -- 172.21.15.19:0/3870325497 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30010690 con 0x7f7d401034f0 2024-05-09T02:11:15.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.457+0000 7f7d4788b640 1 -- 172.21.15.19:0/3870325497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d401a26e0 con 0x7f7d401034f0 2024-05-09T02:11:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.458+0000 7f7d4788b640 1 -- 172.21.15.19:0/3870325497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d40107000 con 0x7f7d401034f0 2024-05-09T02:11:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.459+0000 7f7d367fc640 1 -- 172.21.15.19:0/3870325497 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f7d30004e50 con 0x7f7d401034f0 2024-05-09T02:11:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.460+0000 7f7d367fc640 1 --2- 172.21.15.19:0/3870325497 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7d2c0b2840 0x7f7d2c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.460+0000 7f7d45627640 1 -- 172.21.15.19:0/3870325497 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7d2c0b2840 msgr2=0x7f7d2c0b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/972543870 2024-05-09T02:11:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.460+0000 7f7d367fc640 1 -- 172.21.15.19:0/3870325497 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f7d300e28c0 con 0x7f7d401034f0 2024-05-09T02:11:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.460+0000 7f7d45627640 1 --2- 172.21.15.19:0/3870325497 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7d2c0b2840 0x7f7d2c0b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T02:11:15.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.464+0000 7f7d367fc640 1 -- 172.21.15.19:0/3870325497 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d30054df0 con 0x7f7d401034f0 2024-05-09T02:11:15.466 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:15 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:15.150+0000 7fe4a67a8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T02:11:15.466 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:15 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:15.290+0000 7fe4a67a8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T02:11:15.466 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:15 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:15.390+0000 7fe4a67a8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T02:11:15.591 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:15 smithi019 ceph-mon[30151]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T02:11:15.591 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:15 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:11:15.591 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:15 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3483866041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:11:15.591 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:15 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/4220512158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T02:11:15.591 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:15.437+0000 7f9c8b9aa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T02:11:15.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.657+0000 7f7d4788b640 1 -- 172.21.15.19:0/3870325497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d4002dc30 con 0x7f7d401034f0 2024-05-09T02:11:15.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.658+0000 7f7d367fc640 1 -- 172.21.15.19:0/3870325497 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) ==== 104+0+11 (secure 0 0 0) 0x7f7d300aa5c0 con 0x7f7d401034f0 2024-05-09T02:11:15.659 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-05-09T02:11:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.660+0000 7f7d4788b640 1 -- 172.21.15.19:0/3870325497 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7d2c0b2840 msgr2=0x7f7d2c0b4d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:11:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.660+0000 7f7d4788b640 1 --2- 172.21.15.19:0/3870325497 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7d2c0b2840 0x7f7d2c0b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.660+0000 7f7d4788b640 1 -- 172.21.15.19:0/3870325497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d401034f0 msgr2=0x7f7d4019a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.660+0000 7f7d4788b640 1 --2- 172.21.15.19:0/3870325497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d401034f0 0x7f7d4019a910 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000e9f0 tx=0x7f7d3000edf0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.660+0000 7f7d4788b640 1 -- 172.21.15.19:0/3870325497 shutdown_connections 2024-05-09T02:11:15.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.660+0000 7f7d4788b640 1 --2- 172.21.15.19:0/3870325497 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7f7d2c0b2840 0x7f7d2c0b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.660+0000 7f7d4788b640 1 --2- 172.21.15.19:0/3870325497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d401081f0 0x7f7d401a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.660+0000 7f7d4788b640 1 --2- 172.21.15.19:0/3870325497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d401034f0 0x7f7d4019a910 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.660+0000 7f7d4788b640 1 --2- 172.21.15.19:0/3870325497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d401022b0 0x7f7d4019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.660+0000 7f7d4788b640 1 -- 172.21.15.19:0/3870325497 >> 172.21.15.19:0/3870325497 conn(0x7f7d400fd480 msgr2=0x7f7d401041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:15.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.660+0000 7f7d4788b640 1 -- 172.21.15.19:0/3870325497 shutdown_connections 2024-05-09T02:11:15.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.660+0000 7f7d4788b640 1 -- 172.21.15.19:0/3870325497 wait complete. 2024-05-09T02:11:15.673 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T02:11:15.794 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:15 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:15.495+0000 7fe3b6b21200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T02:11:15.795 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:15 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:15.592+0000 7fe3b6b21200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T02:11:15.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:15 smithi196 ceph-mon[34718]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T02:11:15.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:15 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:11:15.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:15 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3483866041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:11:15.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:15 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/4220512158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T02:11:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.795+0000 7fcf6c972640 1 -- 172.21.15.19:0/703729346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf68103d60 msgr2=0x7fcf68104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:15.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.795+0000 7fcf6c972640 1 --2- 172.21.15.19:0/703729346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf68103d60 0x7fcf68104170 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009900 tx=0x7fcf5002a410 comp rx=0 tx=0).stop 2024-05-09T02:11:15.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.795+0000 7fcf6c972640 1 -- 172.21.15.19:0/703729346 shutdown_connections 2024-05-09T02:11:15.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.795+0000 7fcf6c972640 1 --2- 172.21.15.19:0/703729346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf68108a60 0x7fcf6810ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.795+0000 7fcf6c972640 1 --2- 172.21.15.19:0/703729346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf68103d60 0x7fcf68104170 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.795+0000 7fcf6c972640 1 --2- 172.21.15.19:0/703729346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf68102b20 0x7fcf68103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.795+0000 7fcf6c972640 1 -- 172.21.15.19:0/703729346 >> 172.21.15.19:0/703729346 conn(0x7fcf680fdd10 msgr2=0x7fcf681001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:15.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.795+0000 7fcf6c972640 1 -- 172.21.15.19:0/703729346 shutdown_connections 2024-05-09T02:11:15.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.795+0000 7fcf6c972640 1 -- 172.21.15.19:0/703729346 wait complete. 2024-05-09T02:11:15.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.796+0000 7fcf6c972640 1 Processor -- start 2024-05-09T02:11:15.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.796+0000 7fcf6c972640 1 -- start start 2024-05-09T02:11:15.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.796+0000 7fcf6c972640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf68102b20 0x7fcf681992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:15.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.796+0000 7fcf6c972640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf68103d60 0x7fcf68199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:15.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.796+0000 7fcf6c972640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf68108a60 0x7fcf681a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:15.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.796+0000 7fcf6c972640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6810e690 con 0x7fcf68108a60 2024-05-09T02:11:15.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.796+0000 7fcf6c972640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6810e510 con 0x7fcf68103d60 2024-05-09T02:11:15.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.796+0000 7fcf6c972640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6810e810 con 0x7fcf68102b20 2024-05-09T02:11:15.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.796+0000 7fcf66d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf68108a60 0x7fcf681a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:15.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.796+0000 7fcf5ffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf68103d60 0x7fcf68199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:15.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.796+0000 7fcf66d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf68108a60 0x7fcf681a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33162/0 (socket says 172.21.15.19:33162) 2024-05-09T02:11:15.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.796+0000 7fcf66d9d640 1 -- 172.21.15.19:0/1544413594 learned_addr learned my addr 172.21.15.19:0/1544413594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:15.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.796+0000 7fcf6659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf68102b20 0x7fcf681992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:15.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.797+0000 7fcf66d9d640 1 -- 172.21.15.19:0/1544413594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf68102b20 msgr2=0x7fcf681992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:15.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.797+0000 7fcf66d9d640 1 --2- 172.21.15.19:0/1544413594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf68102b20 0x7fcf681992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.797+0000 7fcf66d9d640 1 -- 172.21.15.19:0/1544413594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf68103d60 msgr2=0x7fcf68199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:15.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.797+0000 7fcf66d9d640 1 --2- 172.21.15.19:0/1544413594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf68103d60 0x7fcf68199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:15.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.797+0000 7fcf66d9d640 1 -- 172.21.15.19:0/1544413594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf681a0e30 con 0x7fcf68108a60 2024-05-09T02:11:15.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.797+0000 7fcf5ffff640 1 --2- 172.21.15.19:0/1544413594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf68103d60 0x7fcf68199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:15.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.797+0000 7fcf6659c640 1 --2- 172.21.15.19:0/1544413594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf68102b20 0x7fcf681992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.797+0000 7fcf66d9d640 1 --2- 172.21.15.19:0/1544413594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf68108a60 0x7fcf681a07c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000eb10 tx=0x7fcf6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.797+0000 7fcf5f7fe640 1 -- 172.21.15.19:0/1544413594 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60004b20 con 0x7fcf68108a60 2024-05-09T02:11:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.797+0000 7fcf5f7fe640 1 -- 172.21.15.19:0/1544413594 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf60004cc0 con 0x7fcf68108a60 2024-05-09T02:11:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.797+0000 7fcf6c972640 1 -- 172.21.15.19:0/1544413594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf681a1120 con 0x7fcf68108a60 2024-05-09T02:11:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.797+0000 7fcf5f7fe640 1 -- 172.21.15.19:0/1544413594 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60010670 con 0x7fcf68108a60 2024-05-09T02:11:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.797+0000 7fcf6c972640 1 -- 172.21.15.19:0/1544413594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf681a15f0 con 0x7fcf68108a60 2024-05-09T02:11:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.798+0000 7fcf6c972640 1 -- 172.21.15.19:0/1544413594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf68107870 con 0x7fcf68108a60 2024-05-09T02:11:15.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.801+0000 7fcf5f7fe640 1 -- 172.21.15.19:0/1544413594 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7fcf60005000 con 0x7fcf68108a60 2024-05-09T02:11:15.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.801+0000 7fcf5f7fe640 1 --2- 172.21.15.19:0/1544413594 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fcf540b2840 0x7fcf540b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:15.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.802+0000 7fcf6659c640 1 -- 172.21.15.19:0/1544413594 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fcf540b2840 msgr2=0x7fcf540b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/972543870 2024-05-09T02:11:15.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.802+0000 7fcf5f7fe640 1 -- 172.21.15.19:0/1544413594 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fcf600e2a60 con 0x7fcf68108a60 2024-05-09T02:11:15.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.802+0000 7fcf6659c640 1 --2- 172.21.15.19:0/1544413594 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fcf540b2840 0x7fcf540b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T02:11:15.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:15.804+0000 7fcf5f7fe640 1 -- 172.21.15.19:0/1544413594 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf60054df0 con 0x7fcf68108a60 2024-05-09T02:11:15.832 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:15 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:15.733+0000 7fe4a67a8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T02:11:15.833 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:15 smithi059 ceph-mon[34781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T02:11:15.833 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:15 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:11:15.833 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:15 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3483866041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:11:15.833 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:15 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/4220512158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T02:11:16.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.000+0000 7fcf6c972640 1 -- 172.21.15.19:0/1544413594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcf680638d0 con 0x7fcf68108a60 2024-05-09T02:11:16.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.001+0000 7fcf5f7fe640 1 -- 172.21.15.19:0/1544413594 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) ==== 100+0+11 (secure 0 0 0) 0x7fcf600aa760 con 0x7fcf68108a60 2024-05-09T02:11:16.003 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-05-09T02:11:16.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.003+0000 7fcf6659c640 1 -- 172.21.15.19:0/1544413594 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fcf540b2840 msgr2=0x7fcf540b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/972543870 2024-05-09T02:11:16.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.003+0000 7fcf6659c640 1 --2- 172.21.15.19:0/1544413594 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fcf540b2840 0x7fcf540b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T02:11:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.003+0000 7fcf6c972640 1 -- 172.21.15.19:0/1544413594 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fcf540b2840 msgr2=0x7fcf540b4d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:11:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.003+0000 7fcf6c972640 1 --2- 172.21.15.19:0/1544413594 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fcf540b2840 0x7fcf540b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.003+0000 7fcf6c972640 1 -- 172.21.15.19:0/1544413594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf68108a60 msgr2=0x7fcf681a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.003+0000 7fcf6c972640 1 --2- 172.21.15.19:0/1544413594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf68108a60 0x7fcf681a07c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000eb10 tx=0x7fcf6000c2d0 comp rx=0 tx=0).stop 2024-05-09T02:11:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.003+0000 7fcf6c972640 1 -- 172.21.15.19:0/1544413594 shutdown_connections 2024-05-09T02:11:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.003+0000 7fcf6c972640 1 --2- 172.21.15.19:0/1544413594 >> [v2:172.21.15.19:6800/972543870,v1:172.21.15.19:6801/972543870] conn(0x7fcf540b2840 0x7fcf540b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.003+0000 7fcf6c972640 1 --2- 172.21.15.19:0/1544413594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf68108a60 0x7fcf681a07c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:16.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.003+0000 7fcf6c972640 1 --2- 172.21.15.19:0/1544413594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf68103d60 0x7fcf68199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:16.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.003+0000 7fcf6c972640 1 --2- 172.21.15.19:0/1544413594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf68102b20 0x7fcf681992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:16.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.003+0000 7fcf6c972640 1 -- 172.21.15.19:0/1544413594 >> 172.21.15.19:0/1544413594 conn(0x7fcf680fdd10 msgr2=0x7fcf68104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:16.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.003+0000 7fcf6c972640 1 -- 172.21.15.19:0/1544413594 shutdown_connections 2024-05-09T02:11:16.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:16.003+0000 7fcf6c972640 1 -- 172.21.15.19:0/1544413594 wait complete. 2024-05-09T02:11:16.015 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:15.589+0000 7f9c8b9aa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T02:11:16.015 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:15.697+0000 7f9c8b9aa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T02:11:16.017 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-09T02:11:16.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:11:16.216 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:15 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:15.830+0000 7fe4a67a8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T02:11:16.288 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:11:16.321 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:16.050+0000 7f9c8b9aa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T02:11:16.321 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:16.154+0000 7f9c8b9aa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T02:11:16.544 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:16 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:16.194+0000 7fe3b6b21200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T02:11:16.545 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:11:16 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z[36570]: 2024-05-09T02:11:16.414+0000 7fe3b6b21200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T02:11:16.846 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:16 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3870325497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T02:11:16.846 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:16 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1544413594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T02:11:16.846 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:16 smithi019 ceph-mon[30151]: Standby manager daemon z restarted 2024-05-09T02:11:16.846 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:16 smithi019 ceph-mon[30151]: Standby manager daemon z started 2024-05-09T02:11:16.846 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:16 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.196:0/4285178671' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T02:11:16.846 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:16 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.196:0/4285178671' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T02:11:16.846 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:16 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.196:0/4285178671' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T02:11:16.846 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:16 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.196:0/4285178671' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T02:11:16.966 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:16 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:16.464+0000 7fe4a67a8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T02:11:16.966 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:11:16 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y[36170]: 2024-05-09T02:11:16.694+0000 7fe4a67a8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T02:11:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:16 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3870325497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T02:11:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:16 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1544413594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T02:11:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:16 smithi059 ceph-mon[34781]: Standby manager daemon z restarted 2024-05-09T02:11:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:16 smithi059 ceph-mon[34781]: Standby manager daemon z started 2024-05-09T02:11:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:16 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.196:0/4285178671' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T02:11:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:16 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.196:0/4285178671' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T02:11:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:16 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.196:0/4285178671' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T02:11:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:16 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.196:0/4285178671' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T02:11:17.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:16 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3870325497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T02:11:17.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:16 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1544413594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T02:11:17.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:16 smithi196 ceph-mon[34718]: Standby manager daemon z restarted 2024-05-09T02:11:17.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:16 smithi196 ceph-mon[34718]: Standby manager daemon z started 2024-05-09T02:11:17.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:16 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.196:0/4285178671' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T02:11:17.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:16 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.196:0/4285178671' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T02:11:17.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:16 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.196:0/4285178671' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T02:11:17.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:16 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.196:0/4285178671' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T02:11:17.264 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:16.844+0000 7f9c8b9aa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T02:11:17.264 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:17.083+0000 7f9c8b9aa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T02:11:17.572 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:17.387+0000 7f9be9f74700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:11:17.572 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:17.387+0000 7f9be9f74700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:11:17.573 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:17.387+0000 7f9be9f74700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:11:17.573 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:17.387+0000 7f9be9f74700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:11:17.573 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:17.387+0000 7f9be9f74700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:11:17.573 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:17.395+0000 7f9becf7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:11:17.573 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:17.395+0000 7f9becf7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:11:17.573 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:17.395+0000 7f9becf7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:11:17.573 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:17.395+0000 7f9becf7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:11:17.573 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:11:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:11:17.395+0000 7f9becf7a700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:11:17.855 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-05-09T02:11:17.856 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: Standby manager daemon y restarted 2024-05-09T02:11:17.856 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: Standby manager daemon y started 2024-05-09T02:11:17.856 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.59:0/3630413064' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T02:11:17.856 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.59:0/3630413064' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T02:11:17.856 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.59:0/3630413064' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T02:11:17.856 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.? 172.21.15.59:0/3630413064' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T02:11:17.856 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: Active manager daemon x restarted 2024-05-09T02:11:17.856 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: Activating manager daemon x 2024-05-09T02:11:17.856 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T02:11:17.856 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: mgrmap e21: x(active, starting, since 0.0174036s), standbys: y, z 2024-05-09T02:11:17.856 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:11:17.856 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:11:17.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:11:17.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T02:11:17.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T02:11:17.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T02:11:17.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T02:11:17.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T02:11:17.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T02:11:17.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T02:11:17.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T02:11:17.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T02:11:17.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T02:11:17.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T02:11:17.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T02:11:17.857 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T02:11:17.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T02:11:17.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T02:11:17.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:11:17.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:11:17.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:11:17.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:11:17.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:11:17.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:11:17.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:11:17.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:11:17.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:11:17.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:11:17.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:11:17.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:11:17.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T02:11:17.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T02:11:17.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T02:11:17.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: Manager daemon x is now available 2024-05-09T02:11:17.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:11:17.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T02:11:17.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:17 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T02:11:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-05-09T02:11:17.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: Standby manager daemon y restarted 2024-05-09T02:11:17.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: Standby manager daemon y started 2024-05-09T02:11:17.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.59:0/3630413064' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T02:11:17.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.59:0/3630413064' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T02:11:17.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.59:0/3630413064' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T02:11:17.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.? 172.21.15.59:0/3630413064' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T02:11:17.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: Active manager daemon x restarted 2024-05-09T02:11:17.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: Activating manager daemon x 2024-05-09T02:11:17.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T02:11:17.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: mgrmap e21: x(active, starting, since 0.0174036s), standbys: y, z 2024-05-09T02:11:17.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:11:17.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:11:17.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:11:17.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T02:11:17.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T02:11:17.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T02:11:17.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T02:11:17.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T02:11:17.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T02:11:17.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T02:11:17.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T02:11:17.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T02:11:17.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T02:11:17.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T02:11:17.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T02:11:17.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T02:11:17.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T02:11:17.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T02:11:17.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: Manager daemon x is now available 2024-05-09T02:11:17.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:11:17.970 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T02:11:17.970 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:17 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T02:11:18.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-05-09T02:11:18.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: Standby manager daemon y restarted 2024-05-09T02:11:18.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: Standby manager daemon y started 2024-05-09T02:11:18.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.59:0/3630413064' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T02:11:18.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.59:0/3630413064' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T02:11:18.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.59:0/3630413064' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T02:11:18.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.? 172.21.15.59:0/3630413064' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T02:11:18.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: Active manager daemon x restarted 2024-05-09T02:11:18.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: Activating manager daemon x 2024-05-09T02:11:18.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T02:11:18.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: mgrmap e21: x(active, starting, since 0.0174036s), standbys: y, z 2024-05-09T02:11:18.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T02:11:18.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T02:11:18.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T02:11:18.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T02:11:18.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T02:11:18.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T02:11:18.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T02:11:18.047 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T02:11:18.047 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T02:11:18.047 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T02:11:18.047 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T02:11:18.047 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T02:11:18.047 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T02:11:18.047 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T02:11:18.047 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T02:11:18.047 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T02:11:18.047 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T02:11:18.047 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T02:11:18.047 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T02:11:18.047 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T02:11:18.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T02:11:18.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T02:11:18.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T02:11:18.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T02:11:18.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T02:11:18.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T02:11:18.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T02:11:18.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T02:11:18.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T02:11:18.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T02:11:18.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T02:11:18.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T02:11:18.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T02:11:18.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: Manager daemon x is now available 2024-05-09T02:11:18.048 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:11:18.049 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T02:11:18.049 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:17 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T02:11:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.122+0000 7f67b62d5700 1 -- 172.21.15.19:0/1104856211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b0110070 msgr2=0x7f67b01179d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.122+0000 7f67b62d5700 1 --2- 172.21.15.19:0/1104856211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b0110070 0x7f67b01179d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009fc0 tx=0x7f67a800b330 comp rx=0 tx=0).stop 2024-05-09T02:11:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.123+0000 7f67b62d5700 1 -- 172.21.15.19:0/1104856211 shutdown_connections 2024-05-09T02:11:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.123+0000 7f67b62d5700 1 --2- 172.21.15.19:0/1104856211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b0110070 0x7f67b01179d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.123+0000 7f67b62d5700 1 --2- 172.21.15.19:0/1104856211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b010adf0 0x7f67b010b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.123+0000 7f67b62d5700 1 --2- 172.21.15.19:0/1104856211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b010f720 0x7f67b010fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.123+0000 7f67b62d5700 1 -- 172.21.15.19:0/1104856211 >> 172.21.15.19:0/1104856211 conn(0x7f67b006f6b0 msgr2=0x7f67b006faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:18.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.125+0000 7f67b62d5700 1 -- 172.21.15.19:0/1104856211 shutdown_connections 2024-05-09T02:11:18.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.125+0000 7f67b62d5700 1 -- 172.21.15.19:0/1104856211 wait complete. 2024-05-09T02:11:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.126+0000 7f67b62d5700 1 Processor -- start 2024-05-09T02:11:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.126+0000 7f67b62d5700 1 -- start start 2024-05-09T02:11:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.126+0000 7f67b62d5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b010adf0 0x7f67b01a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.126+0000 7f67b62d5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b010f720 0x7f67b01a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.126+0000 7f67b62d5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b0110070 0x7f67b01a7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.126+0000 7f67b62d5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f67b011a890 con 0x7f67b010f720 2024-05-09T02:11:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.126+0000 7f67b62d5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67b011a710 con 0x7f67b0110070 2024-05-09T02:11:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.126+0000 7f67b62d5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f67b011aa10 con 0x7f67b010adf0 2024-05-09T02:11:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.127+0000 7f67b52d3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b010adf0 0x7f67b01a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.127+0000 7f67b52d3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b010adf0 0x7f67b01a3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:53598/0 (socket says 172.21.15.19:53598) 2024-05-09T02:11:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.127+0000 7f67b52d3700 1 -- 172.21.15.19:0/3727570757 learned_addr learned my addr 172.21.15.19:0/3727570757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.127+0000 7f67b4ad2700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b010f720 0x7f67b01a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.127+0000 7f67b5ad4700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b0110070 0x7f67b01a7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.127+0000 7f67b52d3700 1 -- 172.21.15.19:0/3727570757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b0110070 msgr2=0x7f67b01a7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.127+0000 7f67b52d3700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b0110070 0x7f67b01a7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.127+0000 7f67b52d3700 1 -- 172.21.15.19:0/3727570757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b010f720 msgr2=0x7f67b01a39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.127+0000 7f67b52d3700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b010f720 0x7f67b01a39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.127+0000 7f67b52d3700 1 -- 172.21.15.19:0/3727570757 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b01a82d0 con 0x7f67b010adf0 2024-05-09T02:11:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.127+0000 7f67b4ad2700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b010f720 0x7f67b01a39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.127+0000 7f67b5ad4700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b0110070 0x7f67b01a7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.127+0000 7f67b52d3700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b010adf0 0x7f67b01a3400 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f67a400e9a0 tx=0x7f67a400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.128+0000 7f67a27fc700 1 -- 172.21.15.19:0/3727570757 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4004a30 con 0x7f67b010adf0 2024-05-09T02:11:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.128+0000 7f67a27fc700 1 -- 172.21.15.19:0/3727570757 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a400ccc0 con 0x7f67b010adf0 2024-05-09T02:11:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.128+0000 7f67a27fc700 1 -- 172.21.15.19:0/3727570757 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4005680 con 0x7f67b010adf0 2024-05-09T02:11:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.128+0000 7f67b62d5700 1 -- 172.21.15.19:0/3727570757 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b01a85c0 con 0x7f67b010adf0 2024-05-09T02:11:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.128+0000 7f67b62d5700 1 -- 172.21.15.19:0/3727570757 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b01afee0 con 0x7f67b010adf0 2024-05-09T02:11:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.129+0000 7f67b62d5700 1 -- 172.21.15.19:0/3727570757 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b00512c0 con 0x7f67b010adf0 2024-05-09T02:11:18.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.131+0000 7f67a27fc700 1 -- 172.21.15.19:0/3727570757 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f67a400c810 con 0x7f67b010adf0 2024-05-09T02:11:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.132+0000 7f67a27fc700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f679c0b2670 0x7f679c0b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.133+0000 7f67b4ad2700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f679c0b2670 0x7f679c0b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.133+0000 7f67a27fc700 1 -- 172.21.15.19:0/3727570757 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f67a40e20b0 con 0x7f67b010adf0 2024-05-09T02:11:18.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.133+0000 7f67b4ad2700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f679c0b2670 0x7f679c0b4b80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f67b010cba0 tx=0x7f67ac009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:18.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.138+0000 7f67a27fc700 1 -- 172.21.15.19:0/3727570757 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f67a40a9fa0 con 0x7f67b010adf0 2024-05-09T02:11:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.441+0000 7f67b62d5700 1 -- 172.21.15.19:0/3727570757 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67b002e5f0 con 0x7f67b010adf0 2024-05-09T02:11:18.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.442+0000 7f67a27fc700 1 -- 172.21.15.19:0/3727570757 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11917 (secure 0 0 0) 0x7f67a40a98a0 con 0x7f67b010adf0 2024-05-09T02:11:18.443 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:18.443 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:11:14.517298+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24463,24484,24473,24455,24457]},"id":1}]} 2024-05-09T02:11:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.444+0000 7f67b62d5700 1 -- 172.21.15.19:0/3727570757 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f679c0b2670 msgr2=0x7f679c0b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.444+0000 7f67b62d5700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f679c0b2670 0x7f679c0b4b80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f67b010cba0 tx=0x7f67ac009260 comp rx=0 tx=0).stop 2024-05-09T02:11:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.444+0000 7f67b62d5700 1 -- 172.21.15.19:0/3727570757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b010adf0 msgr2=0x7f67b01a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.444+0000 7f67b62d5700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b010adf0 0x7f67b01a3400 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f67a400e9a0 tx=0x7f67a400ecb0 comp rx=0 tx=0).stop 2024-05-09T02:11:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.444+0000 7f67b62d5700 1 -- 172.21.15.19:0/3727570757 shutdown_connections 2024-05-09T02:11:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.444+0000 7f67b62d5700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b0110070 0x7f67b01a7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.444+0000 7f67b62d5700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67b010adf0 0x7f67b01a3400 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.444+0000 7f67b62d5700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f679c0b2670 0x7f679c0b4b80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:18.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.444+0000 7f67b62d5700 1 --2- 172.21.15.19:0/3727570757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67b010f720 0x7f67b01a39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:18.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.444+0000 7f67b62d5700 1 -- 172.21.15.19:0/3727570757 >> 172.21.15.19:0/3727570757 conn(0x7f67b006f6b0 msgr2=0x7f67b010d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:18.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.444+0000 7f67b62d5700 1 -- 172.21.15.19:0/3727570757 shutdown_connections 2024-05-09T02:11:18.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:18.444+0000 7f67b62d5700 1 -- 172.21.15.19:0/3727570757 wait complete. 2024-05-09T02:11:18.447 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-05-09T02:11:19.126 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:19 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:19.126 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:19 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:19.126 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:19 smithi019 ceph-mon[30151]: mgrmap e22: x(active, since 1.022s), standbys: y, z 2024-05-09T02:11:19.126 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:19 smithi019 ceph-mon[30151]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:11:19.126 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:19 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3727570757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:11:19.154 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-09T02:11:19.154 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-09T02:11:19.154 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T02:11:19.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.273+0000 7f86574f9640 1 -- 172.21.15.19:0/2871106858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8650104e70 msgr2=0x7f8650105280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:19.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.273+0000 7f86574f9640 1 --2- 172.21.15.19:0/2871106858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8650104e70 0x7f8650105280 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8640009900 tx=0x7f864002a410 comp rx=0 tx=0).stop 2024-05-09T02:11:19.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.273+0000 7f86574f9640 1 -- 172.21.15.19:0/2871106858 shutdown_connections 2024-05-09T02:11:19.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.273+0000 7f86574f9640 1 --2- 172.21.15.19:0/2871106858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8650109b70 0x7f865010bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.273+0000 7f86574f9640 1 --2- 172.21.15.19:0/2871106858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8650104e70 0x7f8650105280 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.273+0000 7f86574f9640 1 --2- 172.21.15.19:0/2871106858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8650103c30 0x7f8650104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.274+0000 7f86574f9640 1 -- 172.21.15.19:0/2871106858 >> 172.21.15.19:0/2871106858 conn(0x7f86500fee40 msgr2=0x7f86501012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.274+0000 7f86574f9640 1 -- 172.21.15.19:0/2871106858 shutdown_connections 2024-05-09T02:11:19.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.274+0000 7f86574f9640 1 -- 172.21.15.19:0/2871106858 wait complete. 2024-05-09T02:11:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.274+0000 7f86574f9640 1 Processor -- start 2024-05-09T02:11:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.274+0000 7f86574f9640 1 -- start start 2024-05-09T02:11:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.274+0000 7f86574f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8650103c30 0x7f865019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.274+0000 7f86574f9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8650104e70 0x7f865019a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.274+0000 7f86574f9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8650109b70 0x7f86501a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.274+0000 7f86574f9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f865010f730 con 0x7f8650103c30 2024-05-09T02:11:19.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.274+0000 7f86574f9640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f865010f5b0 con 0x7f8650109b70 2024-05-09T02:11:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f86574f9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f865010f8b0 con 0x7f8650104e70 2024-05-09T02:11:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f8655295640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8650103c30 0x7f865019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f8655295640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8650103c30 0x7f865019a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44486/0 (socket says 172.21.15.19:44486) 2024-05-09T02:11:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f8655295640 1 -- 172.21.15.19:0/3421052904 learned_addr learned my addr 172.21.15.19:0/3421052904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f8655a96640 1 --2- 172.21.15.19:0/3421052904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8650109b70 0x7f86501a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f8654a94640 1 --2- 172.21.15.19:0/3421052904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8650104e70 0x7f865019a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f8655295640 1 -- 172.21.15.19:0/3421052904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8650104e70 msgr2=0x7f865019a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f8655295640 1 --2- 172.21.15.19:0/3421052904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8650104e70 0x7f865019a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f8655295640 1 -- 172.21.15.19:0/3421052904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8650109b70 msgr2=0x7f86501a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f8655295640 1 --2- 172.21.15.19:0/3421052904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8650109b70 0x7f86501a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f8655295640 1 -- 172.21.15.19:0/3421052904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86501a1e70 con 0x7f8650103c30 2024-05-09T02:11:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f8655295640 1 --2- 172.21.15.19:0/3421052904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8650103c30 0x7f865019a330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f864c00e980 tx=0x7f864c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f86467fc640 1 -- 172.21.15.19:0/3421052904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c00cd80 con 0x7f8650103c30 2024-05-09T02:11:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f86574f9640 1 -- 172.21.15.19:0/3421052904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86501a2160 con 0x7f8650103c30 2024-05-09T02:11:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f86467fc640 1 -- 172.21.15.19:0/3421052904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f864c0108e0 con 0x7f8650103c30 2024-05-09T02:11:19.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f86467fc640 1 -- 172.21.15.19:0/3421052904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c018890 con 0x7f8650103c30 2024-05-09T02:11:19.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.275+0000 7f86574f9640 1 -- 172.21.15.19:0/3421052904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86501a26b0 con 0x7f8650103c30 2024-05-09T02:11:19.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.276+0000 7f86574f9640 1 -- 172.21.15.19:0/3421052904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8650108980 con 0x7f8650103c30 2024-05-09T02:11:19.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.279+0000 7f86467fc640 1 -- 172.21.15.19:0/3421052904 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f864c01f020 con 0x7f8650103c30 2024-05-09T02:11:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.279+0000 7f86467fc640 1 --2- 172.21.15.19:0/3421052904 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f863c0b25e0 0x7f863c0b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.280+0000 7f8654a94640 1 --2- 172.21.15.19:0/3421052904 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f863c0b25e0 0x7f863c0b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.280+0000 7f86467fc640 1 -- 172.21.15.19:0/3421052904 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f864c0e2700 con 0x7f8650103c30 2024-05-09T02:11:19.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.280+0000 7f8654a94640 1 --2- 172.21.15.19:0/3421052904 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f863c0b25e0 0x7f863c0b4af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8640000c00 tx=0x7f8640031400 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:19.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.283+0000 7f86467fc640 1 -- 172.21.15.19:0/3421052904 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f864c0aa5b0 con 0x7f8650103c30 2024-05-09T02:11:19.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:19 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:19.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:19 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:19.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:19 smithi196 ceph-mon[34718]: mgrmap e22: x(active, since 1.022s), standbys: y, z 2024-05-09T02:11:19.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:19 smithi196 ceph-mon[34718]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:11:19.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:19 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3727570757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:11:19.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:19 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:19.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:19 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:19.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:19 smithi059 ceph-mon[34781]: mgrmap e22: x(active, since 1.022s), standbys: y, z 2024-05-09T02:11:19.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:19 smithi059 ceph-mon[34781]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:11:19.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:19 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3727570757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:11:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.487+0000 7f86574f9640 1 -- 172.21.15.19:0/3421052904 --> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8650063110 con 0x7f863c0b25e0 2024-05-09T02:11:19.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.495+0000 7f86467fc640 1 -- 172.21.15.19:0/3421052904 <== mgr.15222 v2:172.21.15.19:6800/2638922161 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258055 (secure 0 0 0) 0x7f8650063110 con 0x7f863c0b25e0 2024-05-09T02:11:19.500 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:19.502 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-09T02:11:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.499+0000 7f86574f9640 1 -- 172.21.15.19:0/3421052904 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f863c0b25e0 msgr2=0x7f863c0b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.499+0000 7f86574f9640 1 --2- 172.21.15.19:0/3421052904 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f863c0b25e0 0x7f863c0b4af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8640000c00 tx=0x7f8640031400 comp rx=0 tx=0).stop 2024-05-09T02:11:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.499+0000 7f86574f9640 1 -- 172.21.15.19:0/3421052904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8650103c30 msgr2=0x7f865019a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.499+0000 7f86574f9640 1 --2- 172.21.15.19:0/3421052904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8650103c30 0x7f865019a330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f864c00e980 tx=0x7f864c00ee40 comp rx=0 tx=0).stop 2024-05-09T02:11:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.499+0000 7f86574f9640 1 -- 172.21.15.19:0/3421052904 shutdown_connections 2024-05-09T02:11:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.499+0000 7f86574f9640 1 --2- 172.21.15.19:0/3421052904 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f863c0b25e0 0x7f863c0b4af0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.499+0000 7f86574f9640 1 --2- 172.21.15.19:0/3421052904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8650109b70 0x7f86501a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.499+0000 7f86574f9640 1 --2- 172.21.15.19:0/3421052904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8650104e70 0x7f865019a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.499+0000 7f86574f9640 1 --2- 172.21.15.19:0/3421052904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8650103c30 0x7f865019a330 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.499+0000 7f86574f9640 1 -- 172.21.15.19:0/3421052904 >> 172.21.15.19:0/3421052904 conn(0x7f86500fee40 msgr2=0x7f8650105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.499+0000 7f86574f9640 1 -- 172.21.15.19:0/3421052904 shutdown_connections 2024-05-09T02:11:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.499+0000 7f86574f9640 1 -- 172.21.15.19:0/3421052904 wait complete. 2024-05-09T02:11:19.512 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-05-09T02:11:19.110260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192474,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2120,"num_read_kb":1205,"num_write":499,"num_write_kb":3617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2952064,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":895,"ondisk_log_size":895,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376960,"kb_used_data":11140,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564864,"statfs":{"total":1151940427776,"available":1151554420736,"internally_reserved":0,"allocated":11407360,"data_stored":8663488,"data_compressed":272094,"data_compressed_allocated":360448,"data_compressed_original":3465216,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.983889"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093030+0000","last_change":"2024-05-09T02:07:48.203957+0000","last_active":"2024-05-09T02:11:17.093030+0000","last_peered":"2024-05-09T02:11:17.093030+0000","last_clean":"2024-05-09T02:11:17.093030+0000","last_became_active":"2024-05-09T02:07:48.203354+0000","last_became_peered":"2024-05-09T02:07:48.203354+0000","last_unstale":"2024-05-09T02:11:17.093030+0000","last_undegraded":"2024-05-09T02:11:17.093030+0000","last_fullsized":"2024-05-09T02:11:17.093030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:06.169693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282389+0000","last_change":"2024-05-09T02:07:48.192885+0000","last_active":"2024-05-09T02:11:17.282389+0000","last_peered":"2024-05-09T02:11:17.282389+0000","last_clean":"2024-05-09T02:11:17.282389+0000","last_became_active":"2024-05-09T02:07:48.192476+0000","last_became_peered":"2024-05-09T02:07:48.192476+0000","last_unstale":"2024-05-09T02:11:17.282389+0000","last_undegraded":"2024-05-09T02:11:17.282389+0000","last_fullsized":"2024-05-09T02:11:17.282389+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:33.710309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282172+0000","last_change":"2024-05-09T02:07:48.198684+0000","last_active":"2024-05-09T02:11:17.282172+0000","last_peered":"2024-05-09T02:11:17.282172+0000","last_clean":"2024-05-09T02:11:17.282172+0000","last_became_active":"2024-05-09T02:07:48.198344+0000","last_became_peered":"2024-05-09T02:07:48.198344+0000","last_unstale":"2024-05-09T02:11:17.282172+0000","last_undegraded":"2024-05-09T02:11:17.282172+0000","last_fullsized":"2024-05-09T02:11:17.282172+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:19.990063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282720+0000","last_change":"2024-05-09T02:07:48.189209+0000","last_active":"2024-05-09T02:11:17.282720+0000","last_peered":"2024-05-09T02:11:17.282720+0000","last_clean":"2024-05-09T02:11:17.282720+0000","last_became_active":"2024-05-09T02:07:48.188744+0000","last_became_peered":"2024-05-09T02:07:48.188744+0000","last_unstale":"2024-05-09T02:11:17.282720+0000","last_undegraded":"2024-05-09T02:11:17.282720+0000","last_fullsized":"2024-05-09T02:11:17.282720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:03:25.216849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200644+0000","last_change":"2024-05-09T02:07:48.187897+0000","last_active":"2024-05-09T02:11:17.200644+0000","last_peered":"2024-05-09T02:11:17.200644+0000","last_clean":"2024-05-09T02:11:17.200644+0000","last_became_active":"2024-05-09T02:07:48.187476+0000","last_became_peered":"2024-05-09T02:07:48.187476+0000","last_unstale":"2024-05-09T02:11:17.200644+0000","last_undegraded":"2024-05-09T02:11:17.200644+0000","last_fullsized":"2024-05-09T02:11:17.200644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:39.366520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.096096+0000","last_change":"2024-05-09T02:07:48.195436+0000","last_active":"2024-05-09T02:11:17.096096+0000","last_peered":"2024-05-09T02:11:17.096096+0000","last_clean":"2024-05-09T02:11:17.096096+0000","last_became_active":"2024-05-09T02:07:48.194982+0000","last_became_peered":"2024-05-09T02:07:48.194982+0000","last_unstale":"2024-05-09T02:11:17.096096+0000","last_undegraded":"2024-05-09T02:11:17.096096+0000","last_fullsized":"2024-05-09T02:11:17.096096+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:19.746701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395057+0000","last_change":"2024-05-09T02:07:48.198458+0000","last_active":"2024-05-09T02:11:17.395057+0000","last_peered":"2024-05-09T02:11:17.395057+0000","last_clean":"2024-05-09T02:11:17.395057+0000","last_became_active":"2024-05-09T02:07:48.198089+0000","last_became_peered":"2024-05-09T02:07:48.198089+0000","last_unstale":"2024-05-09T02:11:17.395057+0000","last_undegraded":"2024-05-09T02:11:17.395057+0000","last_fullsized":"2024-05-09T02:11:17.395057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:37:44.999438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200698+0000","last_change":"2024-05-09T02:07:48.200294+0000","last_active":"2024-05-09T02:11:17.200698+0000","last_peered":"2024-05-09T02:11:17.200698+0000","last_clean":"2024-05-09T02:11:17.200698+0000","last_became_active":"2024-05-09T02:07:48.199949+0000","last_became_peered":"2024-05-09T02:07:48.199949+0000","last_unstale":"2024-05-09T02:11:17.200698+0000","last_undegraded":"2024-05-09T02:11:17.200698+0000","last_fullsized":"2024-05-09T02:11:17.200698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:39.263849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.283155+0000","last_change":"2024-05-09T02:07:48.196282+0000","last_active":"2024-05-09T02:11:17.283155+0000","last_peered":"2024-05-09T02:11:17.283155+0000","last_clean":"2024-05-09T02:11:17.283155+0000","last_became_active":"2024-05-09T02:07:48.195887+0000","last_became_peered":"2024-05-09T02:07:48.195887+0000","last_unstale":"2024-05-09T02:11:17.283155+0000","last_undegraded":"2024-05-09T02:11:17.283155+0000","last_fullsized":"2024-05-09T02:11:17.283155+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:26.081867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.096575+0000","last_change":"2024-05-09T02:08:25.320636+0000","last_active":"2024-05-09T02:11:17.096575+0000","last_peered":"2024-05-09T02:11:17.096575+0000","last_clean":"2024-05-09T02:11:17.096575+0000","last_became_active":"2024-05-09T02:08:25.320155+0000","last_became_peered":"2024-05-09T02:08:25.320155+0000","last_unstale":"2024-05-09T02:11:17.096575+0000","last_undegraded":"2024-05-09T02:11:17.096575+0000","last_fullsized":"2024-05-09T02:11:17.096575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:55.728773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395383+0000","last_change":"2024-05-09T02:07:48.197649+0000","last_active":"2024-05-09T02:11:17.395383+0000","last_peered":"2024-05-09T02:11:17.395383+0000","last_clean":"2024-05-09T02:11:17.395383+0000","last_became_active":"2024-05-09T02:07:48.196884+0000","last_became_peered":"2024-05-09T02:07:48.196884+0000","last_unstale":"2024-05-09T02:11:17.395383+0000","last_undegraded":"2024-05-09T02:11:17.395383+0000","last_fullsized":"2024-05-09T02:11:17.395383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:23.851216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282732+0000","last_change":"2024-05-09T02:07:48.193568+0000","last_active":"2024-05-09T02:11:17.282732+0000","last_peered":"2024-05-09T02:11:17.282732+0000","last_clean":"2024-05-09T02:11:17.282732+0000","last_became_active":"2024-05-09T02:07:48.193117+0000","last_became_peered":"2024-05-09T02:07:48.193117+0000","last_unstale":"2024-05-09T02:11:17.282732+0000","last_undegraded":"2024-05-09T02:11:17.282732+0000","last_fullsized":"2024-05-09T02:11:17.282732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:34.371246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093762+0000","last_change":"2024-05-09T02:07:48.196314+0000","last_active":"2024-05-09T02:11:17.093762+0000","last_peered":"2024-05-09T02:11:17.093762+0000","last_clean":"2024-05-09T02:11:17.093762+0000","last_became_active":"2024-05-09T02:07:48.195988+0000","last_became_peered":"2024-05-09T02:07:48.195988+0000","last_unstale":"2024-05-09T02:11:17.093762+0000","last_undegraded":"2024-05-09T02:11:17.093762+0000","last_fullsized":"2024-05-09T02:11:17.093762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:08.556166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281979+0000","last_change":"2024-05-09T02:07:48.192974+0000","last_active":"2024-05-09T02:11:17.281979+0000","last_peered":"2024-05-09T02:11:17.281979+0000","last_clean":"2024-05-09T02:11:17.281979+0000","last_became_active":"2024-05-09T02:07:48.192599+0000","last_became_peered":"2024-05-09T02:07:48.192599+0000","last_unstale":"2024-05-09T02:11:17.281979+0000","last_undegraded":"2024-05-09T02:11:17.281979+0000","last_fullsized":"2024-05-09T02:11:17.281979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:37.192219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281774+0000","last_change":"2024-05-09T02:07:48.193208+0000","last_active":"2024-05-09T02:11:17.281774+0000","last_peered":"2024-05-09T02:11:17.281774+0000","last_clean":"2024-05-09T02:11:17.281774+0000","last_became_active":"2024-05-09T02:07:48.192654+0000","last_became_peered":"2024-05-09T02:07:48.192654+0000","last_unstale":"2024-05-09T02:11:17.281774+0000","last_undegraded":"2024-05-09T02:11:17.281774+0000","last_fullsized":"2024-05-09T02:11:17.281774+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:55:42.870372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281908+0000","last_change":"2024-05-09T02:07:48.187568+0000","last_active":"2024-05-09T02:11:17.281908+0000","last_peered":"2024-05-09T02:11:17.281908+0000","last_clean":"2024-05-09T02:11:17.281908+0000","last_became_active":"2024-05-09T02:07:48.187159+0000","last_became_peered":"2024-05-09T02:07:48.187159+0000","last_unstale":"2024-05-09T02:11:17.281908+0000","last_undegraded":"2024-05-09T02:11:17.281908+0000","last_fullsized":"2024-05-09T02:11:17.281908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:04.994094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093126+0000","last_change":"2024-05-09T02:07:48.204406+0000","last_active":"2024-05-09T02:11:17.093126+0000","last_peered":"2024-05-09T02:11:17.093126+0000","last_clean":"2024-05-09T02:11:17.093126+0000","last_became_active":"2024-05-09T02:07:48.203592+0000","last_became_peered":"2024-05-09T02:07:48.203592+0000","last_unstale":"2024-05-09T02:11:17.093126+0000","last_undegraded":"2024-05-09T02:11:17.093126+0000","last_fullsized":"2024-05-09T02:11:17.093126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:21.655807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.198463+0000","last_change":"2024-05-09T02:07:48.195326+0000","last_active":"2024-05-09T02:11:17.198463+0000","last_peered":"2024-05-09T02:11:17.198463+0000","last_clean":"2024-05-09T02:11:17.198463+0000","last_became_active":"2024-05-09T02:07:48.194969+0000","last_became_peered":"2024-05-09T02:07:48.194969+0000","last_unstale":"2024-05-09T02:11:17.198463+0000","last_undegraded":"2024-05-09T02:11:17.198463+0000","last_fullsized":"2024-05-09T02:11:17.198463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:00.269753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281961+0000","last_change":"2024-05-09T02:07:48.190706+0000","last_active":"2024-05-09T02:11:17.281961+0000","last_peered":"2024-05-09T02:11:17.281961+0000","last_clean":"2024-05-09T02:11:17.281961+0000","last_became_active":"2024-05-09T02:07:48.190171+0000","last_became_peered":"2024-05-09T02:07:48.190171+0000","last_unstale":"2024-05-09T02:11:17.281961+0000","last_undegraded":"2024-05-09T02:11:17.281961+0000","last_fullsized":"2024-05-09T02:11:17.281961+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:19:31.333618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":234,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.394913+0000","last_change":"2024-05-09T02:07:45.158224+0000","last_active":"2024-05-09T02:11:17.394913+0000","last_peered":"2024-05-09T02:11:17.394913+0000","last_clean":"2024-05-09T02:11:17.394913+0000","last_became_active":"2024-05-09T02:07:45.157722+0000","last_became_peered":"2024-05-09T02:07:45.157722+0000","last_unstale":"2024-05-09T02:11:17.394913+0000","last_undegraded":"2024-05-09T02:11:17.394913+0000","last_fullsized":"2024-05-09T02:11:17.394913+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:31.750699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.201211+0000","last_change":"2024-05-09T02:07:48.197978+0000","last_active":"2024-05-09T02:11:17.201211+0000","last_peered":"2024-05-09T02:11:17.201211+0000","last_clean":"2024-05-09T02:11:17.201211+0000","last_became_active":"2024-05-09T02:07:48.197077+0000","last_became_peered":"2024-05-09T02:07:48.197077+0000","last_unstale":"2024-05-09T02:11:17.201211+0000","last_undegraded":"2024-05-09T02:11:17.201211+0000","last_fullsized":"2024-05-09T02:11:17.201211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:25.897904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.201481+0000","last_change":"2024-05-09T02:07:45.185758+0000","last_active":"2024-05-09T02:11:17.201481+0000","last_peered":"2024-05-09T02:11:17.201481+0000","last_clean":"2024-05-09T02:11:17.201481+0000","last_became_active":"2024-05-09T02:07:45.185411+0000","last_became_peered":"2024-05-09T02:07:45.185411+0000","last_unstale":"2024-05-09T02:11:17.201481+0000","last_undegraded":"2024-05-09T02:11:17.201481+0000","last_fullsized":"2024-05-09T02:11:17.201481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:04.410523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282579+0000","last_change":"2024-05-09T02:07:48.199794+0000","last_active":"2024-05-09T02:11:17.282579+0000","last_peered":"2024-05-09T02:11:17.282579+0000","last_clean":"2024-05-09T02:11:17.282579+0000","last_became_active":"2024-05-09T02:07:48.199453+0000","last_became_peered":"2024-05-09T02:07:48.199453+0000","last_unstale":"2024-05-09T02:11:17.282579+0000","last_undegraded":"2024-05-09T02:11:17.282579+0000","last_fullsized":"2024-05-09T02:11:17.282579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:09.735830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200991+0000","last_change":"2024-05-09T02:07:45.162221+0000","last_active":"2024-05-09T02:11:17.200991+0000","last_peered":"2024-05-09T02:11:17.200991+0000","last_clean":"2024-05-09T02:11:17.200991+0000","last_became_active":"2024-05-09T02:07:45.161748+0000","last_became_peered":"2024-05-09T02:07:45.161748+0000","last_unstale":"2024-05-09T02:11:17.200991+0000","last_undegraded":"2024-05-09T02:11:17.200991+0000","last_fullsized":"2024-05-09T02:11:17.200991+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:38.837411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282244+0000","last_change":"2024-05-09T02:07:48.188741+0000","last_active":"2024-05-09T02:11:17.282244+0000","last_peered":"2024-05-09T02:11:17.282244+0000","last_clean":"2024-05-09T02:11:17.282244+0000","last_became_active":"2024-05-09T02:07:48.188329+0000","last_became_peered":"2024-05-09T02:07:48.188329+0000","last_unstale":"2024-05-09T02:11:17.282244+0000","last_undegraded":"2024-05-09T02:11:17.282244+0000","last_fullsized":"2024-05-09T02:11:17.282244+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:46.773336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200769+0000","last_change":"2024-05-09T02:07:45.163573+0000","last_active":"2024-05-09T02:11:17.200769+0000","last_peered":"2024-05-09T02:11:17.200769+0000","last_clean":"2024-05-09T02:11:17.200769+0000","last_became_active":"2024-05-09T02:07:45.163162+0000","last_became_peered":"2024-05-09T02:07:45.163162+0000","last_unstale":"2024-05-09T02:11:17.200769+0000","last_undegraded":"2024-05-09T02:11:17.200769+0000","last_fullsized":"2024-05-09T02:11:17.200769+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:12.356586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209130+0000","last_change":"2024-05-09T02:07:48.203928+0000","last_active":"2024-05-09T02:11:17.209130+0000","last_peered":"2024-05-09T02:11:17.209130+0000","last_clean":"2024-05-09T02:11:17.209130+0000","last_became_active":"2024-05-09T02:07:48.203224+0000","last_became_peered":"2024-05-09T02:07:48.203224+0000","last_unstale":"2024-05-09T02:11:17.209130+0000","last_undegraded":"2024-05-09T02:11:17.209130+0000","last_fullsized":"2024-05-09T02:11:17.209130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:00.568050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":229,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.535090+0000","last_change":"2024-05-09T02:07:45.160161+0000","last_active":"2024-05-09T02:11:17.535090+0000","last_peered":"2024-05-09T02:11:17.535090+0000","last_clean":"2024-05-09T02:11:17.535090+0000","last_became_active":"2024-05-09T02:07:45.159118+0000","last_became_peered":"2024-05-09T02:07:45.159118+0000","last_unstale":"2024-05-09T02:11:17.535090+0000","last_undegraded":"2024-05-09T02:11:17.535090+0000","last_fullsized":"2024-05-09T02:11:17.535090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:04.510877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":193,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200570+0000","last_change":"2024-05-09T02:08:25.319860+0000","last_active":"2024-05-09T02:11:17.200570+0000","last_peered":"2024-05-09T02:11:17.200570+0000","last_clean":"2024-05-09T02:11:17.200570+0000","last_became_active":"2024-05-09T02:08:25.319370+0000","last_became_peered":"2024-05-09T02:08:25.319370+0000","last_unstale":"2024-05-09T02:11:17.200570+0000","last_undegraded":"2024-05-09T02:11:17.200570+0000","last_fullsized":"2024-05-09T02:11:17.200570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:38.504057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.096002+0000","last_change":"2024-05-09T02:08:25.327069+0000","last_active":"2024-05-09T02:11:17.096002+0000","last_peered":"2024-05-09T02:11:17.096002+0000","last_clean":"2024-05-09T02:11:17.096002+0000","last_became_active":"2024-05-09T02:08:25.316629+0000","last_became_peered":"2024-05-09T02:08:25.316629+0000","last_unstale":"2024-05-09T02:11:17.096002+0000","last_undegraded":"2024-05-09T02:11:17.096002+0000","last_fullsized":"2024-05-09T02:11:17.096002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:51.028256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209046+0000","last_change":"2024-05-09T02:07:48.201056+0000","last_active":"2024-05-09T02:11:17.209046+0000","last_peered":"2024-05-09T02:11:17.209046+0000","last_clean":"2024-05-09T02:11:17.209046+0000","last_became_active":"2024-05-09T02:07:48.200671+0000","last_became_peered":"2024-05-09T02:07:48.200671+0000","last_unstale":"2024-05-09T02:11:17.209046+0000","last_undegraded":"2024-05-09T02:11:17.209046+0000","last_fullsized":"2024-05-09T02:11:17.209046+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:37:16.138170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093596+0000","last_change":"2024-05-09T02:07:45.160417+0000","last_active":"2024-05-09T02:11:17.093596+0000","last_peered":"2024-05-09T02:11:17.093596+0000","last_clean":"2024-05-09T02:11:17.093596+0000","last_became_active":"2024-05-09T02:07:45.160076+0000","last_became_peered":"2024-05-09T02:07:45.160076+0000","last_unstale":"2024-05-09T02:11:17.093596+0000","last_undegraded":"2024-05-09T02:11:17.093596+0000","last_fullsized":"2024-05-09T02:11:17.093596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:53.653494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200346+0000","last_change":"2024-05-09T02:07:48.203267+0000","last_active":"2024-05-09T02:11:17.200346+0000","last_peered":"2024-05-09T02:11:17.200346+0000","last_clean":"2024-05-09T02:11:17.200346+0000","last_became_active":"2024-05-09T02:07:48.202747+0000","last_became_peered":"2024-05-09T02:07:48.202747+0000","last_unstale":"2024-05-09T02:11:17.200346+0000","last_undegraded":"2024-05-09T02:11:17.200346+0000","last_fullsized":"2024-05-09T02:11:17.200346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.954400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.208818+0000","last_change":"2024-05-09T02:07:45.164713+0000","last_active":"2024-05-09T02:11:17.208818+0000","last_peered":"2024-05-09T02:11:17.208818+0000","last_clean":"2024-05-09T02:11:17.208818+0000","last_became_active":"2024-05-09T02:07:45.164324+0000","last_became_peered":"2024-05-09T02:07:45.164324+0000","last_unstale":"2024-05-09T02:11:17.208818+0000","last_undegraded":"2024-05-09T02:11:17.208818+0000","last_fullsized":"2024-05-09T02:11:17.208818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:02.481419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.198016+0000","last_change":"2024-05-09T02:07:48.190305+0000","last_active":"2024-05-09T02:11:17.198016+0000","last_peered":"2024-05-09T02:11:17.198016+0000","last_clean":"2024-05-09T02:11:17.198016+0000","last_became_active":"2024-05-09T02:07:48.189971+0000","last_became_peered":"2024-05-09T02:07:48.189971+0000","last_unstale":"2024-05-09T02:11:17.198016+0000","last_undegraded":"2024-05-09T02:11:17.198016+0000","last_fullsized":"2024-05-09T02:11:17.198016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:23.565092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":414,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.534552+0000","last_change":"2024-05-09T02:07:45.164273+0000","last_active":"2024-05-09T02:11:17.534552+0000","last_peered":"2024-05-09T02:11:17.534552+0000","last_clean":"2024-05-09T02:11:17.534552+0000","last_became_active":"2024-05-09T02:07:45.163183+0000","last_became_peered":"2024-05-09T02:07:45.163183+0000","last_unstale":"2024-05-09T02:11:17.534552+0000","last_undegraded":"2024-05-09T02:11:17.534552+0000","last_fullsized":"2024-05-09T02:11:17.534552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:11.395177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":380,"num_read_kb":380,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.283024+0000","last_change":"2024-05-09T02:07:48.202995+0000","last_active":"2024-05-09T02:11:17.283024+0000","last_peered":"2024-05-09T02:11:17.283024+0000","last_clean":"2024-05-09T02:11:17.283024+0000","last_became_active":"2024-05-09T02:07:48.200915+0000","last_became_peered":"2024-05-09T02:07:48.200915+0000","last_unstale":"2024-05-09T02:11:17.283024+0000","last_undegraded":"2024-05-09T02:11:17.283024+0000","last_fullsized":"2024-05-09T02:11:17.283024+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:25.555725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395316+0000","last_change":"2024-05-09T02:07:45.166221+0000","last_active":"2024-05-09T02:11:17.395316+0000","last_peered":"2024-05-09T02:11:17.395316+0000","last_clean":"2024-05-09T02:11:17.395316+0000","last_became_active":"2024-05-09T02:07:45.165813+0000","last_became_peered":"2024-05-09T02:07:45.165813+0000","last_unstale":"2024-05-09T02:11:17.395316+0000","last_undegraded":"2024-05-09T02:11:17.395316+0000","last_fullsized":"2024-05-09T02:11:17.395316+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:01.480828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282689+0000","last_change":"2024-05-09T02:07:48.201475+0000","last_active":"2024-05-09T02:11:17.282689+0000","last_peered":"2024-05-09T02:11:17.282689+0000","last_clean":"2024-05-09T02:11:17.282689+0000","last_became_active":"2024-05-09T02:07:48.201129+0000","last_became_peered":"2024-05-09T02:07:48.201129+0000","last_unstale":"2024-05-09T02:11:17.282689+0000","last_undegraded":"2024-05-09T02:11:17.282689+0000","last_fullsized":"2024-05-09T02:11:17.282689+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:51.726752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209334+0000","last_change":"2024-05-09T02:07:45.185045+0000","last_active":"2024-05-09T02:11:17.209334+0000","last_peered":"2024-05-09T02:11:17.209334+0000","last_clean":"2024-05-09T02:11:17.209334+0000","last_became_active":"2024-05-09T02:07:45.184592+0000","last_became_peered":"2024-05-09T02:07:45.184592+0000","last_unstale":"2024-05-09T02:11:17.209334+0000","last_undegraded":"2024-05-09T02:11:17.209334+0000","last_fullsized":"2024-05-09T02:11:17.209334+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:56.220022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282290+0000","last_change":"2024-05-09T02:07:48.199873+0000","last_active":"2024-05-09T02:11:17.282290+0000","last_peered":"2024-05-09T02:11:17.282290+0000","last_clean":"2024-05-09T02:11:17.282290+0000","last_became_active":"2024-05-09T02:07:48.199502+0000","last_became_peered":"2024-05-09T02:07:48.199502+0000","last_unstale":"2024-05-09T02:11:17.282290+0000","last_undegraded":"2024-05-09T02:11:17.282290+0000","last_fullsized":"2024-05-09T02:11:17.282290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:15.406904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'39","reported_seq":461,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.535242+0000","last_change":"2024-05-09T02:07:45.166167+0000","last_active":"2024-05-09T02:11:17.535242+0000","last_peered":"2024-05-09T02:11:17.535242+0000","last_clean":"2024-05-09T02:11:17.535242+0000","last_became_active":"2024-05-09T02:07:45.165800+0000","last_became_peered":"2024-05-09T02:07:45.165800+0000","last_unstale":"2024-05-09T02:11:17.535242+0000","last_undegraded":"2024-05-09T02:11:17.535242+0000","last_fullsized":"2024-05-09T02:11:17.535242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:03.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68147,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":393,"num_read_kb":263,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200785+0000","last_change":"2024-05-09T02:07:48.187809+0000","last_active":"2024-05-09T02:11:17.200785+0000","last_peered":"2024-05-09T02:11:17.200785+0000","last_clean":"2024-05-09T02:11:17.200785+0000","last_became_active":"2024-05-09T02:07:48.186939+0000","last_became_peered":"2024-05-09T02:07:48.186939+0000","last_unstale":"2024-05-09T02:11:17.200785+0000","last_undegraded":"2024-05-09T02:11:17.200785+0000","last_fullsized":"2024-05-09T02:11:17.200785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:13:15.600343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281818+0000","last_change":"2024-05-09T02:07:45.163756+0000","last_active":"2024-05-09T02:11:17.281818+0000","last_peered":"2024-05-09T02:11:17.281818+0000","last_clean":"2024-05-09T02:11:17.281818+0000","last_became_active":"2024-05-09T02:07:45.163388+0000","last_became_peered":"2024-05-09T02:07:45.163388+0000","last_unstale":"2024-05-09T02:11:17.281818+0000","last_undegraded":"2024-05-09T02:11:17.281818+0000","last_fullsized":"2024-05-09T02:11:17.281818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:41.937148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093618+0000","last_change":"2024-05-09T02:07:48.195218+0000","last_active":"2024-05-09T02:11:17.093618+0000","last_peered":"2024-05-09T02:11:17.093618+0000","last_clean":"2024-05-09T02:11:17.093618+0000","last_became_active":"2024-05-09T02:07:48.194895+0000","last_became_peered":"2024-05-09T02:07:48.194895+0000","last_unstale":"2024-05-09T02:11:17.093618+0000","last_undegraded":"2024-05-09T02:11:17.093618+0000","last_fullsized":"2024-05-09T02:11:17.093618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:49.086594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209218+0000","last_change":"2024-05-09T02:07:45.158039+0000","last_active":"2024-05-09T02:11:17.209218+0000","last_peered":"2024-05-09T02:11:17.209218+0000","last_clean":"2024-05-09T02:11:17.209218+0000","last_became_active":"2024-05-09T02:07:45.157255+0000","last_became_peered":"2024-05-09T02:07:45.157255+0000","last_unstale":"2024-05-09T02:11:17.209218+0000","last_undegraded":"2024-05-09T02:11:17.209218+0000","last_fullsized":"2024-05-09T02:11:17.209218+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:56.513999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395281+0000","last_change":"2024-05-09T02:07:48.200150+0000","last_active":"2024-05-09T02:11:17.395281+0000","last_peered":"2024-05-09T02:11:17.395281+0000","last_clean":"2024-05-09T02:11:17.395281+0000","last_became_active":"2024-05-09T02:07:48.199336+0000","last_became_peered":"2024-05-09T02:07:48.199336+0000","last_unstale":"2024-05-09T02:11:17.395281+0000","last_undegraded":"2024-05-09T02:11:17.395281+0000","last_fullsized":"2024-05-09T02:11:17.395281+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:45.699119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'8","reported_seq":232,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.534669+0000","last_change":"2024-05-09T02:07:45.165476+0000","last_active":"2024-05-09T02:11:17.534669+0000","last_peered":"2024-05-09T02:11:17.534669+0000","last_clean":"2024-05-09T02:11:17.534669+0000","last_became_active":"2024-05-09T02:07:45.165107+0000","last_became_peered":"2024-05-09T02:07:45.165107+0000","last_unstale":"2024-05-09T02:11:17.534669+0000","last_undegraded":"2024-05-09T02:11:17.534669+0000","last_fullsized":"2024-05-09T02:11:17.534669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:15:33.518982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395017+0000","last_change":"2024-05-09T02:07:48.193487+0000","last_active":"2024-05-09T02:11:17.395017+0000","last_peered":"2024-05-09T02:11:17.395017+0000","last_clean":"2024-05-09T02:11:17.395017+0000","last_became_active":"2024-05-09T02:07:48.193108+0000","last_became_peered":"2024-05-09T02:07:48.193108+0000","last_unstale":"2024-05-09T02:11:17.395017+0000","last_undegraded":"2024-05-09T02:11:17.395017+0000","last_fullsized":"2024-05-09T02:11:17.395017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:11.895058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":71,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200474+0000","last_change":"2024-05-09T02:08:25.335525+0000","last_active":"2024-05-09T02:11:17.200474+0000","last_peered":"2024-05-09T02:11:17.200474+0000","last_clean":"2024-05-09T02:11:17.200474+0000","last_became_active":"2024-05-09T02:08:25.319746+0000","last_became_peered":"2024-05-09T02:08:25.319746+0000","last_unstale":"2024-05-09T02:11:17.200474+0000","last_undegraded":"2024-05-09T02:11:17.200474+0000","last_fullsized":"2024-05-09T02:11:17.200474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:41.774066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209033+0000","last_change":"2024-05-09T02:07:48.196125+0000","last_active":"2024-05-09T02:11:17.209033+0000","last_peered":"2024-05-09T02:11:17.209033+0000","last_clean":"2024-05-09T02:11:17.209033+0000","last_became_active":"2024-05-09T02:07:48.195737+0000","last_became_peered":"2024-05-09T02:07:48.195737+0000","last_unstale":"2024-05-09T02:11:17.209033+0000","last_undegraded":"2024-05-09T02:11:17.209033+0000","last_fullsized":"2024-05-09T02:11:17.209033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:43.264720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200887+0000","last_change":"2024-05-09T02:07:45.163791+0000","last_active":"2024-05-09T02:11:17.200887+0000","last_peered":"2024-05-09T02:11:17.200887+0000","last_clean":"2024-05-09T02:11:17.200887+0000","last_became_active":"2024-05-09T02:07:45.163129+0000","last_became_peered":"2024-05-09T02:07:45.163129+0000","last_unstale":"2024-05-09T02:11:17.200887+0000","last_undegraded":"2024-05-09T02:11:17.200887+0000","last_fullsized":"2024-05-09T02:11:17.200887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:48.562850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092838+0000","last_change":"2024-05-09T02:07:48.197782+0000","last_active":"2024-05-09T02:11:17.092838+0000","last_peered":"2024-05-09T02:11:17.092838+0000","last_clean":"2024-05-09T02:11:17.092838+0000","last_became_active":"2024-05-09T02:07:48.197353+0000","last_became_peered":"2024-05-09T02:07:48.197353+0000","last_unstale":"2024-05-09T02:11:17.092838+0000","last_undegraded":"2024-05-09T02:11:17.092838+0000","last_fullsized":"2024-05-09T02:11:17.092838+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:17.053571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.198207+0000","last_change":"2024-05-09T02:07:45.187028+0000","last_active":"2024-05-09T02:11:17.198207+0000","last_peered":"2024-05-09T02:11:17.198207+0000","last_clean":"2024-05-09T02:11:17.198207+0000","last_became_active":"2024-05-09T02:07:45.186639+0000","last_became_peered":"2024-05-09T02:07:45.186639+0000","last_unstale":"2024-05-09T02:11:17.198207+0000","last_undegraded":"2024-05-09T02:11:17.198207+0000","last_fullsized":"2024-05-09T02:11:17.198207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:35.834374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.096134+0000","last_change":"2024-05-09T02:07:48.207150+0000","last_active":"2024-05-09T02:11:17.096134+0000","last_peered":"2024-05-09T02:11:17.096134+0000","last_clean":"2024-05-09T02:11:17.096134+0000","last_became_active":"2024-05-09T02:07:48.206797+0000","last_became_peered":"2024-05-09T02:07:48.206797+0000","last_unstale":"2024-05-09T02:11:17.096134+0000","last_undegraded":"2024-05-09T02:11:17.096134+0000","last_fullsized":"2024-05-09T02:11:17.096134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:37.328133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":228,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.534534+0000","last_change":"2024-05-09T02:07:45.165422+0000","last_active":"2024-05-09T02:11:17.534534+0000","last_peered":"2024-05-09T02:11:17.534534+0000","last_clean":"2024-05-09T02:11:17.534534+0000","last_became_active":"2024-05-09T02:07:45.164998+0000","last_became_peered":"2024-05-09T02:07:45.164998+0000","last_unstale":"2024-05-09T02:11:17.534534+0000","last_undegraded":"2024-05-09T02:11:17.534534+0000","last_fullsized":"2024-05-09T02:11:17.534534+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:49.124323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.096098+0000","last_change":"2024-05-09T02:07:48.206538+0000","last_active":"2024-05-09T02:11:17.096098+0000","last_peered":"2024-05-09T02:11:17.096098+0000","last_clean":"2024-05-09T02:11:17.096098+0000","last_became_active":"2024-05-09T02:07:48.206061+0000","last_became_peered":"2024-05-09T02:07:48.206061+0000","last_unstale":"2024-05-09T02:11:17.096098+0000","last_undegraded":"2024-05-09T02:11:17.096098+0000","last_fullsized":"2024-05-09T02:11:17.096098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:24.075091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209082+0000","last_change":"2024-05-09T02:07:48.202235+0000","last_active":"2024-05-09T02:11:17.209082+0000","last_peered":"2024-05-09T02:11:17.209082+0000","last_clean":"2024-05-09T02:11:17.209082+0000","last_became_active":"2024-05-09T02:07:48.200891+0000","last_became_peered":"2024-05-09T02:07:48.200891+0000","last_unstale":"2024-05-09T02:11:17.209082+0000","last_undegraded":"2024-05-09T02:11:17.209082+0000","last_fullsized":"2024-05-09T02:11:17.209082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:39.061267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'7","reported_seq":227,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.401985+0000","last_change":"2024-05-09T02:07:45.184582+0000","last_active":"2024-05-09T02:11:17.401985+0000","last_peered":"2024-05-09T02:11:17.401985+0000","last_clean":"2024-05-09T02:11:17.401985+0000","last_became_active":"2024-05-09T02:07:45.184187+0000","last_became_peered":"2024-05-09T02:07:45.184187+0000","last_unstale":"2024-05-09T02:11:17.401985+0000","last_undegraded":"2024-05-09T02:11:17.401985+0000","last_fullsized":"2024-05-09T02:11:17.401985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:11.343797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200901+0000","last_change":"2024-05-09T02:07:48.199247+0000","last_active":"2024-05-09T02:11:17.200901+0000","last_peered":"2024-05-09T02:11:17.200901+0000","last_clean":"2024-05-09T02:11:17.200901+0000","last_became_active":"2024-05-09T02:07:48.198121+0000","last_became_peered":"2024-05-09T02:07:48.198121+0000","last_unstale":"2024-05-09T02:11:17.200901+0000","last_undegraded":"2024-05-09T02:11:17.200901+0000","last_fullsized":"2024-05-09T02:11:17.200901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:45:12.885616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282741+0000","last_change":"2024-05-09T02:07:45.163750+0000","last_active":"2024-05-09T02:11:17.282741+0000","last_peered":"2024-05-09T02:11:17.282741+0000","last_clean":"2024-05-09T02:11:17.282741+0000","last_became_active":"2024-05-09T02:07:45.163039+0000","last_became_peered":"2024-05-09T02:07:45.163039+0000","last_unstale":"2024-05-09T02:11:17.282741+0000","last_undegraded":"2024-05-09T02:11:17.282741+0000","last_fullsized":"2024-05-09T02:11:17.282741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:06.794514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":103,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395125+0000","last_change":"2024-05-09T02:06:20.334734+0000","last_active":"2024-05-09T02:11:17.395125+0000","last_peered":"2024-05-09T02:11:17.395125+0000","last_clean":"2024-05-09T02:11:17.395125+0000","last_became_active":"2024-05-09T02:06:20.329198+0000","last_became_peered":"2024-05-09T02:06:20.329198+0000","last_unstale":"2024-05-09T02:11:17.395125+0000","last_undegraded":"2024-05-09T02:11:17.395125+0000","last_fullsized":"2024-05-09T02:11:17.395125+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:02:50.632016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:02:50.632016+0000","last_clean_scrub_stamp":"2024-05-09T02:02:50.632016+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:08:15.495982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093432+0000","last_change":"2024-05-09T02:08:25.321187+0000","last_active":"2024-05-09T02:11:17.093432+0000","last_peered":"2024-05-09T02:11:17.093432+0000","last_clean":"2024-05-09T02:11:17.093432+0000","last_became_active":"2024-05-09T02:08:25.320693+0000","last_became_peered":"2024-05-09T02:08:25.320693+0000","last_unstale":"2024-05-09T02:11:17.093432+0000","last_undegraded":"2024-05-09T02:11:17.093432+0000","last_fullsized":"2024-05-09T02:11:17.093432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:00.176016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395267+0000","last_change":"2024-05-09T02:07:45.158053+0000","last_active":"2024-05-09T02:11:17.395267+0000","last_peered":"2024-05-09T02:11:17.395267+0000","last_clean":"2024-05-09T02:11:17.395267+0000","last_became_active":"2024-05-09T02:07:45.157619+0000","last_became_peered":"2024-05-09T02:07:45.157619+0000","last_unstale":"2024-05-09T02:11:17.395267+0000","last_undegraded":"2024-05-09T02:11:17.395267+0000","last_fullsized":"2024-05-09T02:11:17.395267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:01.618475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092687+0000","last_change":"2024-05-09T02:08:25.341377+0000","last_active":"2024-05-09T02:11:17.092687+0000","last_peered":"2024-05-09T02:11:17.092687+0000","last_clean":"2024-05-09T02:11:17.092687+0000","last_became_active":"2024-05-09T02:08:25.318683+0000","last_became_peered":"2024-05-09T02:08:25.318683+0000","last_unstale":"2024-05-09T02:11:17.092687+0000","last_undegraded":"2024-05-09T02:11:17.092687+0000","last_fullsized":"2024-05-09T02:11:17.092687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:12.456338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093765+0000","last_change":"2024-05-09T02:07:48.186892+0000","last_active":"2024-05-09T02:11:17.093765+0000","last_peered":"2024-05-09T02:11:17.093765+0000","last_clean":"2024-05-09T02:11:17.093765+0000","last_became_active":"2024-05-09T02:07:48.186543+0000","last_became_peered":"2024-05-09T02:07:48.186543+0000","last_unstale":"2024-05-09T02:11:17.093765+0000","last_undegraded":"2024-05-09T02:11:17.093765+0000","last_fullsized":"2024-05-09T02:11:17.093765+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:01.210504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'51","reported_seq":103,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.336046+0000","last_change":"2024-05-09T02:07:45.158007+0000","last_active":"2024-05-09T02:11:17.336046+0000","last_peered":"2024-05-09T02:11:17.336046+0000","last_clean":"2024-05-09T02:11:17.336046+0000","last_became_active":"2024-05-09T02:07:45.157612+0000","last_became_peered":"2024-05-09T02:07:45.157612+0000","last_unstale":"2024-05-09T02:11:17.336046+0000","last_undegraded":"2024-05-09T02:11:17.336046+0000","last_fullsized":"2024-05-09T02:11:17.336046+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:01.526756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.201332+0000","last_change":"2024-05-09T02:07:48.186065+0000","last_active":"2024-05-09T02:11:17.201332+0000","last_peered":"2024-05-09T02:11:17.201332+0000","last_clean":"2024-05-09T02:11:17.201332+0000","last_became_active":"2024-05-09T02:07:48.185746+0000","last_became_peered":"2024-05-09T02:07:48.185746+0000","last_unstale":"2024-05-09T02:11:17.201332+0000","last_undegraded":"2024-05-09T02:11:17.201332+0000","last_fullsized":"2024-05-09T02:11:17.201332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:09.570261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282918+0000","last_change":"2024-05-09T02:07:46.005439+0000","last_active":"2024-05-09T02:11:17.282918+0000","last_peered":"2024-05-09T02:11:17.282918+0000","last_clean":"2024-05-09T02:11:17.282918+0000","last_became_active":"2024-05-09T02:07:46.005087+0000","last_became_peered":"2024-05-09T02:07:46.005087+0000","last_unstale":"2024-05-09T02:11:17.282918+0000","last_undegraded":"2024-05-09T02:11:17.282918+0000","last_fullsized":"2024-05-09T02:11:17.282918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:08.543310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282547+0000","last_change":"2024-05-09T02:07:48.194062+0000","last_active":"2024-05-09T02:11:17.282547+0000","last_peered":"2024-05-09T02:11:17.282547+0000","last_clean":"2024-05-09T02:11:17.282547+0000","last_became_active":"2024-05-09T02:07:48.193712+0000","last_became_peered":"2024-05-09T02:07:48.193712+0000","last_unstale":"2024-05-09T02:11:17.282547+0000","last_undegraded":"2024-05-09T02:11:17.282547+0000","last_fullsized":"2024-05-09T02:11:17.282547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:48.394898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281926+0000","last_change":"2024-05-09T02:07:45.184821+0000","last_active":"2024-05-09T02:11:17.281926+0000","last_peered":"2024-05-09T02:11:17.281926+0000","last_clean":"2024-05-09T02:11:17.281926+0000","last_became_active":"2024-05-09T02:07:45.184437+0000","last_became_peered":"2024-05-09T02:07:45.184437+0000","last_unstale":"2024-05-09T02:11:17.281926+0000","last_undegraded":"2024-05-09T02:11:17.281926+0000","last_fullsized":"2024-05-09T02:11:17.281926+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:30.864171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200507+0000","last_change":"2024-05-09T02:07:48.187635+0000","last_active":"2024-05-09T02:11:17.200507+0000","last_peered":"2024-05-09T02:11:17.200507+0000","last_clean":"2024-05-09T02:11:17.200507+0000","last_became_active":"2024-05-09T02:07:48.187071+0000","last_became_peered":"2024-05-09T02:07:48.187071+0000","last_unstale":"2024-05-09T02:11:17.200507+0000","last_undegraded":"2024-05-09T02:11:17.200507+0000","last_fullsized":"2024-05-09T02:11:17.200507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:10.716715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200715+0000","last_change":"2024-05-09T02:07:45.162166+0000","last_active":"2024-05-09T02:11:17.200715+0000","last_peered":"2024-05-09T02:11:17.200715+0000","last_clean":"2024-05-09T02:11:17.200715+0000","last_became_active":"2024-05-09T02:07:45.161787+0000","last_became_peered":"2024-05-09T02:07:45.161787+0000","last_unstale":"2024-05-09T02:11:17.200715+0000","last_undegraded":"2024-05-09T02:11:17.200715+0000","last_fullsized":"2024-05-09T02:11:17.200715+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:17.925214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092598+0000","last_change":"2024-05-09T02:07:48.202980+0000","last_active":"2024-05-09T02:11:17.092598+0000","last_peered":"2024-05-09T02:11:17.092598+0000","last_clean":"2024-05-09T02:11:17.092598+0000","last_became_active":"2024-05-09T02:07:48.202609+0000","last_became_peered":"2024-05-09T02:07:48.202609+0000","last_unstale":"2024-05-09T02:11:17.092598+0000","last_undegraded":"2024-05-09T02:11:17.092598+0000","last_fullsized":"2024-05-09T02:11:17.092598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:59.993451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092625+0000","last_change":"2024-05-09T02:08:25.326142+0000","last_active":"2024-05-09T02:11:17.092625+0000","last_peered":"2024-05-09T02:11:17.092625+0000","last_clean":"2024-05-09T02:11:17.092625+0000","last_became_active":"2024-05-09T02:08:25.317666+0000","last_became_peered":"2024-05-09T02:08:25.317666+0000","last_unstale":"2024-05-09T02:11:17.092625+0000","last_undegraded":"2024-05-09T02:11:17.092625+0000","last_fullsized":"2024-05-09T02:11:17.092625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:26.178211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200928+0000","last_change":"2024-05-09T02:07:48.202219+0000","last_active":"2024-05-09T02:11:17.200928+0000","last_peered":"2024-05-09T02:11:17.200928+0000","last_clean":"2024-05-09T02:11:17.200928+0000","last_became_active":"2024-05-09T02:07:48.201819+0000","last_became_peered":"2024-05-09T02:07:48.201819+0000","last_unstale":"2024-05-09T02:11:17.200928+0000","last_undegraded":"2024-05-09T02:11:17.200928+0000","last_fullsized":"2024-05-09T02:11:17.200928+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:51.573359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200684+0000","last_change":"2024-05-09T02:07:45.162228+0000","last_active":"2024-05-09T02:11:17.200684+0000","last_peered":"2024-05-09T02:11:17.200684+0000","last_clean":"2024-05-09T02:11:17.200684+0000","last_became_active":"2024-05-09T02:07:45.161847+0000","last_became_peered":"2024-05-09T02:07:45.161847+0000","last_unstale":"2024-05-09T02:11:17.200684+0000","last_undegraded":"2024-05-09T02:11:17.200684+0000","last_fullsized":"2024-05-09T02:11:17.200684+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:47.459709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395519+0000","last_change":"2024-05-09T02:07:48.201401+0000","last_active":"2024-05-09T02:11:17.395519+0000","last_peered":"2024-05-09T02:11:17.395519+0000","last_clean":"2024-05-09T02:11:17.395519+0000","last_became_active":"2024-05-09T02:07:48.201040+0000","last_became_peered":"2024-05-09T02:07:48.201040+0000","last_unstale":"2024-05-09T02:11:17.395519+0000","last_undegraded":"2024-05-09T02:11:17.395519+0000","last_fullsized":"2024-05-09T02:11:17.395519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:58.038690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":226,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282029+0000","last_change":"2024-05-09T02:07:46.005273+0000","last_active":"2024-05-09T02:11:17.282029+0000","last_peered":"2024-05-09T02:11:17.282029+0000","last_clean":"2024-05-09T02:11:17.282029+0000","last_became_active":"2024-05-09T02:07:46.004732+0000","last_became_peered":"2024-05-09T02:07:46.004732+0000","last_unstale":"2024-05-09T02:11:17.282029+0000","last_undegraded":"2024-05-09T02:11:17.282029+0000","last_fullsized":"2024-05-09T02:11:17.282029+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:55.576955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395468+0000","last_change":"2024-05-09T02:07:48.197032+0000","last_active":"2024-05-09T02:11:17.395468+0000","last_peered":"2024-05-09T02:11:17.395468+0000","last_clean":"2024-05-09T02:11:17.395468+0000","last_became_active":"2024-05-09T02:07:48.196553+0000","last_became_peered":"2024-05-09T02:07:48.196553+0000","last_unstale":"2024-05-09T02:11:17.395468+0000","last_undegraded":"2024-05-09T02:11:17.395468+0000","last_fullsized":"2024-05-09T02:11:17.395468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:50:25.081024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'41","reported_seq":78,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.333602+0000","last_change":"2024-05-09T02:07:45.166456+0000","last_active":"2024-05-09T02:11:17.333602+0000","last_peered":"2024-05-09T02:11:17.333602+0000","last_clean":"2024-05-09T02:11:17.333602+0000","last_became_active":"2024-05-09T02:07:45.166132+0000","last_became_peered":"2024-05-09T02:07:45.166132+0000","last_unstale":"2024-05-09T02:11:17.333602+0000","last_undegraded":"2024-05-09T02:11:17.333602+0000","last_fullsized":"2024-05-09T02:11:17.333602+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:15.701802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395265+0000","last_change":"2024-05-09T02:08:25.323303+0000","last_active":"2024-05-09T02:11:17.395265+0000","last_peered":"2024-05-09T02:11:17.395265+0000","last_clean":"2024-05-09T02:11:17.395265+0000","last_became_active":"2024-05-09T02:08:25.321752+0000","last_became_peered":"2024-05-09T02:08:25.321752+0000","last_unstale":"2024-05-09T02:11:17.395265+0000","last_undegraded":"2024-05-09T02:11:17.395265+0000","last_fullsized":"2024-05-09T02:11:17.395265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:17.791737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093809+0000","last_change":"2024-05-09T02:07:45.155480+0000","last_active":"2024-05-09T02:11:17.093809+0000","last_peered":"2024-05-09T02:11:17.093809+0000","last_clean":"2024-05-09T02:11:17.093809+0000","last_became_active":"2024-05-09T02:07:45.155085+0000","last_became_peered":"2024-05-09T02:07:45.155085+0000","last_unstale":"2024-05-09T02:11:17.093809+0000","last_undegraded":"2024-05-09T02:11:17.093809+0000","last_fullsized":"2024-05-09T02:11:17.093809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:05.547857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395468+0000","last_change":"2024-05-09T02:07:48.194480+0000","last_active":"2024-05-09T02:11:17.395468+0000","last_peered":"2024-05-09T02:11:17.395468+0000","last_clean":"2024-05-09T02:11:17.395468+0000","last_became_active":"2024-05-09T02:07:48.194136+0000","last_became_peered":"2024-05-09T02:07:48.194136+0000","last_unstale":"2024-05-09T02:11:17.395468+0000","last_undegraded":"2024-05-09T02:11:17.395468+0000","last_fullsized":"2024-05-09T02:11:17.395468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:09.177762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092843+0000","last_change":"2024-05-09T02:07:48.195152+0000","last_active":"2024-05-09T02:11:17.092843+0000","last_peered":"2024-05-09T02:11:17.092843+0000","last_clean":"2024-05-09T02:11:17.092843+0000","last_became_active":"2024-05-09T02:07:48.194779+0000","last_became_peered":"2024-05-09T02:07:48.194779+0000","last_unstale":"2024-05-09T02:11:17.092843+0000","last_undegraded":"2024-05-09T02:11:17.092843+0000","last_fullsized":"2024-05-09T02:11:17.092843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:39.345452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281768+0000","last_change":"2024-05-09T02:08:25.322235+0000","last_active":"2024-05-09T02:11:17.281768+0000","last_peered":"2024-05-09T02:11:17.281768+0000","last_clean":"2024-05-09T02:11:17.281768+0000","last_became_active":"2024-05-09T02:08:25.321674+0000","last_became_peered":"2024-05-09T02:08:25.321674+0000","last_unstale":"2024-05-09T02:11:17.281768+0000","last_undegraded":"2024-05-09T02:11:17.281768+0000","last_fullsized":"2024-05-09T02:11:17.281768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:52.671669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281697+0000","last_change":"2024-05-09T02:07:48.195710+0000","last_active":"2024-05-09T02:11:17.281697+0000","last_peered":"2024-05-09T02:11:17.281697+0000","last_clean":"2024-05-09T02:11:17.281697+0000","last_became_active":"2024-05-09T02:07:48.195332+0000","last_became_peered":"2024-05-09T02:07:48.195332+0000","last_unstale":"2024-05-09T02:11:17.281697+0000","last_undegraded":"2024-05-09T02:11:17.281697+0000","last_fullsized":"2024-05-09T02:11:17.281697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:12:17.904166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282255+0000","last_change":"2024-05-09T02:07:48.200791+0000","last_active":"2024-05-09T02:11:17.282255+0000","last_peered":"2024-05-09T02:11:17.282255+0000","last_clean":"2024-05-09T02:11:17.282255+0000","last_became_active":"2024-05-09T02:07:48.199916+0000","last_became_peered":"2024-05-09T02:07:48.199916+0000","last_unstale":"2024-05-09T02:11:17.282255+0000","last_undegraded":"2024-05-09T02:11:17.282255+0000","last_fullsized":"2024-05-09T02:11:17.282255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:32.692375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093770+0000","last_change":"2024-05-09T02:07:48.191662+0000","last_active":"2024-05-09T02:11:17.093770+0000","last_peered":"2024-05-09T02:11:17.093770+0000","last_clean":"2024-05-09T02:11:17.093770+0000","last_became_active":"2024-05-09T02:07:48.191336+0000","last_became_peered":"2024-05-09T02:07:48.191336+0000","last_unstale":"2024-05-09T02:11:17.093770+0000","last_undegraded":"2024-05-09T02:11:17.093770+0000","last_fullsized":"2024-05-09T02:11:17.093770+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:00.733611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092797+0000","last_change":"2024-05-09T02:07:48.195182+0000","last_active":"2024-05-09T02:11:17.092797+0000","last_peered":"2024-05-09T02:11:17.092797+0000","last_clean":"2024-05-09T02:11:17.092797+0000","last_became_active":"2024-05-09T02:07:48.194828+0000","last_became_peered":"2024-05-09T02:07:48.194828+0000","last_unstale":"2024-05-09T02:11:17.092797+0000","last_undegraded":"2024-05-09T02:11:17.092797+0000","last_fullsized":"2024-05-09T02:11:17.092797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:33.427409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282054+0000","last_change":"2024-05-09T02:07:48.199688+0000","last_active":"2024-05-09T02:11:17.282054+0000","last_peered":"2024-05-09T02:11:17.282054+0000","last_clean":"2024-05-09T02:11:17.282054+0000","last_became_active":"2024-05-09T02:07:48.199326+0000","last_became_peered":"2024-05-09T02:07:48.199326+0000","last_unstale":"2024-05-09T02:11:17.282054+0000","last_undegraded":"2024-05-09T02:11:17.282054+0000","last_fullsized":"2024-05-09T02:11:17.282054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:30.662738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.201606+0000","last_change":"2024-05-09T02:07:48.190727+0000","last_active":"2024-05-09T02:11:17.201606+0000","last_peered":"2024-05-09T02:11:17.201606+0000","last_clean":"2024-05-09T02:11:17.201606+0000","last_became_active":"2024-05-09T02:07:48.190397+0000","last_became_peered":"2024-05-09T02:07:48.190397+0000","last_unstale":"2024-05-09T02:11:17.201606+0000","last_undegraded":"2024-05-09T02:11:17.201606+0000","last_fullsized":"2024-05-09T02:11:17.201606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:42.010669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.208911+0000","last_change":"2024-05-09T02:07:48.200444+0000","last_active":"2024-05-09T02:11:17.208911+0000","last_peered":"2024-05-09T02:11:17.208911+0000","last_clean":"2024-05-09T02:11:17.208911+0000","last_became_active":"2024-05-09T02:07:48.200031+0000","last_became_peered":"2024-05-09T02:07:48.200031+0000","last_unstale":"2024-05-09T02:11:17.208911+0000","last_undegraded":"2024-05-09T02:11:17.208911+0000","last_fullsized":"2024-05-09T02:11:17.208911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:30.269792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200630+0000","last_change":"2024-05-09T02:07:48.203717+0000","last_active":"2024-05-09T02:11:17.200630+0000","last_peered":"2024-05-09T02:11:17.200630+0000","last_clean":"2024-05-09T02:11:17.200630+0000","last_became_active":"2024-05-09T02:07:48.203347+0000","last_became_peered":"2024-05-09T02:07:48.203347+0000","last_unstale":"2024-05-09T02:11:17.200630+0000","last_undegraded":"2024-05-09T02:11:17.200630+0000","last_fullsized":"2024-05-09T02:11:17.200630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:17.391504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282202+0000","last_change":"2024-05-09T02:08:25.323029+0000","last_active":"2024-05-09T02:11:17.282202+0000","last_peered":"2024-05-09T02:11:17.282202+0000","last_clean":"2024-05-09T02:11:17.282202+0000","last_became_active":"2024-05-09T02:08:25.322524+0000","last_became_peered":"2024-05-09T02:08:25.322524+0000","last_unstale":"2024-05-09T02:11:17.282202+0000","last_undegraded":"2024-05-09T02:11:17.282202+0000","last_fullsized":"2024-05-09T02:11:17.282202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:25.017511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093749+0000","last_change":"2024-05-09T02:07:48.187802+0000","last_active":"2024-05-09T02:11:17.093749+0000","last_peered":"2024-05-09T02:11:17.093749+0000","last_clean":"2024-05-09T02:11:17.093749+0000","last_became_active":"2024-05-09T02:07:48.187478+0000","last_became_peered":"2024-05-09T02:07:48.187478+0000","last_unstale":"2024-05-09T02:11:17.093749+0000","last_undegraded":"2024-05-09T02:11:17.093749+0000","last_fullsized":"2024-05-09T02:11:17.093749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:12.981386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092634+0000","last_change":"2024-05-09T02:07:48.192117+0000","last_active":"2024-05-09T02:11:17.092634+0000","last_peered":"2024-05-09T02:11:17.092634+0000","last_clean":"2024-05-09T02:11:17.092634+0000","last_became_active":"2024-05-09T02:07:48.191726+0000","last_became_peered":"2024-05-09T02:07:48.191726+0000","last_unstale":"2024-05-09T02:11:17.092634+0000","last_undegraded":"2024-05-09T02:11:17.092634+0000","last_fullsized":"2024-05-09T02:11:17.092634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:07.735582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107049,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601872,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2039,"num_read_kb":1136,"num_write":374,"num_write_kb":1465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1805616,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678073,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":438649,"data_compressed":8666,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874300,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":1097854,"data_compressed":44349,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037828,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":946176,"data_stored":831800,"data_compressed":27812,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234056,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":831949,"data_compressed":27812,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430285,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":966656,"data_stored":1033459,"data_compressed":44349,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593811,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":438966,"data_compressed":8666,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":497868,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":921600,"data_stored":439523,"data_compressed":8666,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":497976,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149795,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":962708,"data_compressed":33968,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953565,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":429704,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790041,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":1163032,"data_compressed":50505,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135202,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202406,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T02:11:19.516 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T02:11:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.668+0000 7fa958dc7640 1 -- 172.21.15.19:0/2384424467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954104e50 msgr2=0x7fa954105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.668+0000 7fa958dc7640 1 --2- 172.21.15.19:0/2384424467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954104e50 0x7fa954105260 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa948009900 tx=0x7fa94802a410 comp rx=0 tx=0).stop 2024-05-09T02:11:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.668+0000 7fa958dc7640 1 -- 172.21.15.19:0/2384424467 shutdown_connections 2024-05-09T02:11:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.668+0000 7fa958dc7640 1 --2- 172.21.15.19:0/2384424467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa954109b50 0x7fa95410bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.668+0000 7fa958dc7640 1 --2- 172.21.15.19:0/2384424467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954104e50 0x7fa954105260 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.668+0000 7fa958dc7640 1 --2- 172.21.15.19:0/2384424467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa954103c10 0x7fa954104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.668+0000 7fa958dc7640 1 -- 172.21.15.19:0/2384424467 >> 172.21.15.19:0/2384424467 conn(0x7fa9540fee00 msgr2=0x7fa9541012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:19.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.668+0000 7fa958dc7640 1 -- 172.21.15.19:0/2384424467 shutdown_connections 2024-05-09T02:11:19.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.668+0000 7fa958dc7640 1 -- 172.21.15.19:0/2384424467 wait complete. 2024-05-09T02:11:19.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.669+0000 7fa958dc7640 1 Processor -- start 2024-05-09T02:11:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.669+0000 7fa958dc7640 1 -- start start 2024-05-09T02:11:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa958dc7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa954103c10 0x7fa95419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa958dc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954104e50 0x7fa95419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa958dc7640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa954109b50 0x7fa9541a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa958dc7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa95410f740 con 0x7fa954104e50 2024-05-09T02:11:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa958dc7640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa95410f5c0 con 0x7fa954109b50 2024-05-09T02:11:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa958dc7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa95410f8c0 con 0x7fa954103c10 2024-05-09T02:11:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa951d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954104e50 0x7fa95419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa951d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954104e50 0x7fa95419a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44506/0 (socket says 172.21.15.19:44506) 2024-05-09T02:11:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa951d9b640 1 -- 172.21.15.19:0/174380304 learned_addr learned my addr 172.21.15.19:0/174380304 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa95259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa954103c10 0x7fa95419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa952d9d640 1 --2- 172.21.15.19:0/174380304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa954109b50 0x7fa9541a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa95259c640 1 -- 172.21.15.19:0/174380304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa954109b50 msgr2=0x7fa9541a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa95259c640 1 --2- 172.21.15.19:0/174380304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa954109b50 0x7fa9541a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa95259c640 1 -- 172.21.15.19:0/174380304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954104e50 msgr2=0x7fa95419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa95259c640 1 --2- 172.21.15.19:0/174380304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954104e50 0x7fa95419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa95259c640 1 -- 172.21.15.19:0/174380304 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9541a1e70 con 0x7fa954103c10 2024-05-09T02:11:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa951d9b640 1 --2- 172.21.15.19:0/174380304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954104e50 0x7fa95419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.670+0000 7fa95259c640 1 --2- 172.21.15.19:0/174380304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa954103c10 0x7fa95419a3a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa9541034e0 tx=0x7fa94400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.671+0000 7fa9437fe640 1 -- 172.21.15.19:0/174380304 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94400cc40 con 0x7fa954103c10 2024-05-09T02:11:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.671+0000 7fa958dc7640 1 -- 172.21.15.19:0/174380304 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9541a2160 con 0x7fa954103c10 2024-05-09T02:11:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.671+0000 7fa9437fe640 1 -- 172.21.15.19:0/174380304 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa94400cde0 con 0x7fa954103c10 2024-05-09T02:11:19.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.671+0000 7fa9437fe640 1 -- 172.21.15.19:0/174380304 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944010430 con 0x7fa954103c10 2024-05-09T02:11:19.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.671+0000 7fa958dc7640 1 -- 172.21.15.19:0/174380304 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9541a2700 con 0x7fa954103c10 2024-05-09T02:11:19.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.672+0000 7fa958dc7640 1 -- 172.21.15.19:0/174380304 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa95406b0d0 con 0x7fa954103c10 2024-05-09T02:11:19.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.674+0000 7fa9437fe640 1 -- 172.21.15.19:0/174380304 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fa944003eb0 con 0x7fa954103c10 2024-05-09T02:11:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.674+0000 7fa9437fe640 1 --2- 172.21.15.19:0/174380304 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa93c0b25e0 0x7fa93c0b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.674+0000 7fa9437fe640 1 -- 172.21.15.19:0/174380304 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fa9440e23c0 con 0x7fa954103c10 2024-05-09T02:11:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.674+0000 7fa951d9b640 1 --2- 172.21.15.19:0/174380304 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa93c0b25e0 0x7fa93c0b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.675+0000 7fa951d9b640 1 --2- 172.21.15.19:0/174380304 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa93c0b25e0 0x7fa93c0b4af0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa948031440 tx=0x7fa9480314a0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.677+0000 7fa9437fe640 1 -- 172.21.15.19:0/174380304 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9440aa340 con 0x7fa954103c10 2024-05-09T02:11:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.891+0000 7fa958dc7640 1 -- 172.21.15.19:0/174380304 --> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa954063110 con 0x7fa93c0b25e0 2024-05-09T02:11:19.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.898+0000 7fa9437fe640 1 -- 172.21.15.19:0/174380304 <== mgr.15222 v2:172.21.15.19:6800/2638922161 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258055 (secure 0 0 0) 0x7fa954063110 con 0x7fa93c0b25e0 2024-05-09T02:11:19.902 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-09T02:11:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.902+0000 7fa958dc7640 1 -- 172.21.15.19:0/174380304 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa93c0b25e0 msgr2=0x7fa93c0b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.902+0000 7fa958dc7640 1 --2- 172.21.15.19:0/174380304 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa93c0b25e0 0x7fa93c0b4af0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa948031440 tx=0x7fa9480314a0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.902+0000 7fa958dc7640 1 -- 172.21.15.19:0/174380304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa954103c10 msgr2=0x7fa95419a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.902+0000 7fa958dc7640 1 --2- 172.21.15.19:0/174380304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa954103c10 0x7fa95419a3a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa9541034e0 tx=0x7fa94400ee50 comp rx=0 tx=0).stop 2024-05-09T02:11:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.902+0000 7fa958dc7640 1 -- 172.21.15.19:0/174380304 shutdown_connections 2024-05-09T02:11:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.902+0000 7fa958dc7640 1 --2- 172.21.15.19:0/174380304 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa93c0b25e0 0x7fa93c0b4af0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.902+0000 7fa958dc7640 1 --2- 172.21.15.19:0/174380304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa954109b50 0x7fa9541a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.902+0000 7fa958dc7640 1 --2- 172.21.15.19:0/174380304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa954104e50 0x7fa95419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.902+0000 7fa958dc7640 1 --2- 172.21.15.19:0/174380304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa954103c10 0x7fa95419a3a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.902+0000 7fa958dc7640 1 -- 172.21.15.19:0/174380304 >> 172.21.15.19:0/174380304 conn(0x7fa9540fee00 msgr2=0x7fa954105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.902+0000 7fa958dc7640 1 -- 172.21.15.19:0/174380304 shutdown_connections 2024-05-09T02:11:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:19.902+0000 7fa958dc7640 1 -- 172.21.15.19:0/174380304 wait complete. 2024-05-09T02:11:19.916 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-05-09T02:11:19.110260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192474,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2120,"num_read_kb":1205,"num_write":499,"num_write_kb":3617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2952064,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":895,"ondisk_log_size":895,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376960,"kb_used_data":11140,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564864,"statfs":{"total":1151940427776,"available":1151554420736,"internally_reserved":0,"allocated":11407360,"data_stored":8663488,"data_compressed":272094,"data_compressed_allocated":360448,"data_compressed_original":3465216,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.983889"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093030+0000","last_change":"2024-05-09T02:07:48.203957+0000","last_active":"2024-05-09T02:11:17.093030+0000","last_peered":"2024-05-09T02:11:17.093030+0000","last_clean":"2024-05-09T02:11:17.093030+0000","last_became_active":"2024-05-09T02:07:48.203354+0000","last_became_peered":"2024-05-09T02:07:48.203354+0000","last_unstale":"2024-05-09T02:11:17.093030+0000","last_undegraded":"2024-05-09T02:11:17.093030+0000","last_fullsized":"2024-05-09T02:11:17.093030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:06.169693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282389+0000","last_change":"2024-05-09T02:07:48.192885+0000","last_active":"2024-05-09T02:11:17.282389+0000","last_peered":"2024-05-09T02:11:17.282389+0000","last_clean":"2024-05-09T02:11:17.282389+0000","last_became_active":"2024-05-09T02:07:48.192476+0000","last_became_peered":"2024-05-09T02:07:48.192476+0000","last_unstale":"2024-05-09T02:11:17.282389+0000","last_undegraded":"2024-05-09T02:11:17.282389+0000","last_fullsized":"2024-05-09T02:11:17.282389+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:33.710309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282172+0000","last_change":"2024-05-09T02:07:48.198684+0000","last_active":"2024-05-09T02:11:17.282172+0000","last_peered":"2024-05-09T02:11:17.282172+0000","last_clean":"2024-05-09T02:11:17.282172+0000","last_became_active":"2024-05-09T02:07:48.198344+0000","last_became_peered":"2024-05-09T02:07:48.198344+0000","last_unstale":"2024-05-09T02:11:17.282172+0000","last_undegraded":"2024-05-09T02:11:17.282172+0000","last_fullsized":"2024-05-09T02:11:17.282172+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:19.990063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282720+0000","last_change":"2024-05-09T02:07:48.189209+0000","last_active":"2024-05-09T02:11:17.282720+0000","last_peered":"2024-05-09T02:11:17.282720+0000","last_clean":"2024-05-09T02:11:17.282720+0000","last_became_active":"2024-05-09T02:07:48.188744+0000","last_became_peered":"2024-05-09T02:07:48.188744+0000","last_unstale":"2024-05-09T02:11:17.282720+0000","last_undegraded":"2024-05-09T02:11:17.282720+0000","last_fullsized":"2024-05-09T02:11:17.282720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:03:25.216849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200644+0000","last_change":"2024-05-09T02:07:48.187897+0000","last_active":"2024-05-09T02:11:17.200644+0000","last_peered":"2024-05-09T02:11:17.200644+0000","last_clean":"2024-05-09T02:11:17.200644+0000","last_became_active":"2024-05-09T02:07:48.187476+0000","last_became_peered":"2024-05-09T02:07:48.187476+0000","last_unstale":"2024-05-09T02:11:17.200644+0000","last_undegraded":"2024-05-09T02:11:17.200644+0000","last_fullsized":"2024-05-09T02:11:17.200644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:39.366520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.096096+0000","last_change":"2024-05-09T02:07:48.195436+0000","last_active":"2024-05-09T02:11:17.096096+0000","last_peered":"2024-05-09T02:11:17.096096+0000","last_clean":"2024-05-09T02:11:17.096096+0000","last_became_active":"2024-05-09T02:07:48.194982+0000","last_became_peered":"2024-05-09T02:07:48.194982+0000","last_unstale":"2024-05-09T02:11:17.096096+0000","last_undegraded":"2024-05-09T02:11:17.096096+0000","last_fullsized":"2024-05-09T02:11:17.096096+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:19.746701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395057+0000","last_change":"2024-05-09T02:07:48.198458+0000","last_active":"2024-05-09T02:11:17.395057+0000","last_peered":"2024-05-09T02:11:17.395057+0000","last_clean":"2024-05-09T02:11:17.395057+0000","last_became_active":"2024-05-09T02:07:48.198089+0000","last_became_peered":"2024-05-09T02:07:48.198089+0000","last_unstale":"2024-05-09T02:11:17.395057+0000","last_undegraded":"2024-05-09T02:11:17.395057+0000","last_fullsized":"2024-05-09T02:11:17.395057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:37:44.999438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200698+0000","last_change":"2024-05-09T02:07:48.200294+0000","last_active":"2024-05-09T02:11:17.200698+0000","last_peered":"2024-05-09T02:11:17.200698+0000","last_clean":"2024-05-09T02:11:17.200698+0000","last_became_active":"2024-05-09T02:07:48.199949+0000","last_became_peered":"2024-05-09T02:07:48.199949+0000","last_unstale":"2024-05-09T02:11:17.200698+0000","last_undegraded":"2024-05-09T02:11:17.200698+0000","last_fullsized":"2024-05-09T02:11:17.200698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:39.263849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.283155+0000","last_change":"2024-05-09T02:07:48.196282+0000","last_active":"2024-05-09T02:11:17.283155+0000","last_peered":"2024-05-09T02:11:17.283155+0000","last_clean":"2024-05-09T02:11:17.283155+0000","last_became_active":"2024-05-09T02:07:48.195887+0000","last_became_peered":"2024-05-09T02:07:48.195887+0000","last_unstale":"2024-05-09T02:11:17.283155+0000","last_undegraded":"2024-05-09T02:11:17.283155+0000","last_fullsized":"2024-05-09T02:11:17.283155+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:26.081867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.096575+0000","last_change":"2024-05-09T02:08:25.320636+0000","last_active":"2024-05-09T02:11:17.096575+0000","last_peered":"2024-05-09T02:11:17.096575+0000","last_clean":"2024-05-09T02:11:17.096575+0000","last_became_active":"2024-05-09T02:08:25.320155+0000","last_became_peered":"2024-05-09T02:08:25.320155+0000","last_unstale":"2024-05-09T02:11:17.096575+0000","last_undegraded":"2024-05-09T02:11:17.096575+0000","last_fullsized":"2024-05-09T02:11:17.096575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:55.728773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395383+0000","last_change":"2024-05-09T02:07:48.197649+0000","last_active":"2024-05-09T02:11:17.395383+0000","last_peered":"2024-05-09T02:11:17.395383+0000","last_clean":"2024-05-09T02:11:17.395383+0000","last_became_active":"2024-05-09T02:07:48.196884+0000","last_became_peered":"2024-05-09T02:07:48.196884+0000","last_unstale":"2024-05-09T02:11:17.395383+0000","last_undegraded":"2024-05-09T02:11:17.395383+0000","last_fullsized":"2024-05-09T02:11:17.395383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:23.851216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282732+0000","last_change":"2024-05-09T02:07:48.193568+0000","last_active":"2024-05-09T02:11:17.282732+0000","last_peered":"2024-05-09T02:11:17.282732+0000","last_clean":"2024-05-09T02:11:17.282732+0000","last_became_active":"2024-05-09T02:07:48.193117+0000","last_became_peered":"2024-05-09T02:07:48.193117+0000","last_unstale":"2024-05-09T02:11:17.282732+0000","last_undegraded":"2024-05-09T02:11:17.282732+0000","last_fullsized":"2024-05-09T02:11:17.282732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:34.371246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093762+0000","last_change":"2024-05-09T02:07:48.196314+0000","last_active":"2024-05-09T02:11:17.093762+0000","last_peered":"2024-05-09T02:11:17.093762+0000","last_clean":"2024-05-09T02:11:17.093762+0000","last_became_active":"2024-05-09T02:07:48.195988+0000","last_became_peered":"2024-05-09T02:07:48.195988+0000","last_unstale":"2024-05-09T02:11:17.093762+0000","last_undegraded":"2024-05-09T02:11:17.093762+0000","last_fullsized":"2024-05-09T02:11:17.093762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:08.556166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281979+0000","last_change":"2024-05-09T02:07:48.192974+0000","last_active":"2024-05-09T02:11:17.281979+0000","last_peered":"2024-05-09T02:11:17.281979+0000","last_clean":"2024-05-09T02:11:17.281979+0000","last_became_active":"2024-05-09T02:07:48.192599+0000","last_became_peered":"2024-05-09T02:07:48.192599+0000","last_unstale":"2024-05-09T02:11:17.281979+0000","last_undegraded":"2024-05-09T02:11:17.281979+0000","last_fullsized":"2024-05-09T02:11:17.281979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:37.192219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281774+0000","last_change":"2024-05-09T02:07:48.193208+0000","last_active":"2024-05-09T02:11:17.281774+0000","last_peered":"2024-05-09T02:11:17.281774+0000","last_clean":"2024-05-09T02:11:17.281774+0000","last_became_active":"2024-05-09T02:07:48.192654+0000","last_became_peered":"2024-05-09T02:07:48.192654+0000","last_unstale":"2024-05-09T02:11:17.281774+0000","last_undegraded":"2024-05-09T02:11:17.281774+0000","last_fullsized":"2024-05-09T02:11:17.281774+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:55:42.870372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281908+0000","last_change":"2024-05-09T02:07:48.187568+0000","last_active":"2024-05-09T02:11:17.281908+0000","last_peered":"2024-05-09T02:11:17.281908+0000","last_clean":"2024-05-09T02:11:17.281908+0000","last_became_active":"2024-05-09T02:07:48.187159+0000","last_became_peered":"2024-05-09T02:07:48.187159+0000","last_unstale":"2024-05-09T02:11:17.281908+0000","last_undegraded":"2024-05-09T02:11:17.281908+0000","last_fullsized":"2024-05-09T02:11:17.281908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:04.994094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093126+0000","last_change":"2024-05-09T02:07:48.204406+0000","last_active":"2024-05-09T02:11:17.093126+0000","last_peered":"2024-05-09T02:11:17.093126+0000","last_clean":"2024-05-09T02:11:17.093126+0000","last_became_active":"2024-05-09T02:07:48.203592+0000","last_became_peered":"2024-05-09T02:07:48.203592+0000","last_unstale":"2024-05-09T02:11:17.093126+0000","last_undegraded":"2024-05-09T02:11:17.093126+0000","last_fullsized":"2024-05-09T02:11:17.093126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:21.655807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.198463+0000","last_change":"2024-05-09T02:07:48.195326+0000","last_active":"2024-05-09T02:11:17.198463+0000","last_peered":"2024-05-09T02:11:17.198463+0000","last_clean":"2024-05-09T02:11:17.198463+0000","last_became_active":"2024-05-09T02:07:48.194969+0000","last_became_peered":"2024-05-09T02:07:48.194969+0000","last_unstale":"2024-05-09T02:11:17.198463+0000","last_undegraded":"2024-05-09T02:11:17.198463+0000","last_fullsized":"2024-05-09T02:11:17.198463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:00.269753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281961+0000","last_change":"2024-05-09T02:07:48.190706+0000","last_active":"2024-05-09T02:11:17.281961+0000","last_peered":"2024-05-09T02:11:17.281961+0000","last_clean":"2024-05-09T02:11:17.281961+0000","last_became_active":"2024-05-09T02:07:48.190171+0000","last_became_peered":"2024-05-09T02:07:48.190171+0000","last_unstale":"2024-05-09T02:11:17.281961+0000","last_undegraded":"2024-05-09T02:11:17.281961+0000","last_fullsized":"2024-05-09T02:11:17.281961+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:19:31.333618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":234,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.394913+0000","last_change":"2024-05-09T02:07:45.158224+0000","last_active":"2024-05-09T02:11:17.394913+0000","last_peered":"2024-05-09T02:11:17.394913+0000","last_clean":"2024-05-09T02:11:17.394913+0000","last_became_active":"2024-05-09T02:07:45.157722+0000","last_became_peered":"2024-05-09T02:07:45.157722+0000","last_unstale":"2024-05-09T02:11:17.394913+0000","last_undegraded":"2024-05-09T02:11:17.394913+0000","last_fullsized":"2024-05-09T02:11:17.394913+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:31.750699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.201211+0000","last_change":"2024-05-09T02:07:48.197978+0000","last_active":"2024-05-09T02:11:17.201211+0000","last_peered":"2024-05-09T02:11:17.201211+0000","last_clean":"2024-05-09T02:11:17.201211+0000","last_became_active":"2024-05-09T02:07:48.197077+0000","last_became_peered":"2024-05-09T02:07:48.197077+0000","last_unstale":"2024-05-09T02:11:17.201211+0000","last_undegraded":"2024-05-09T02:11:17.201211+0000","last_fullsized":"2024-05-09T02:11:17.201211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:25.897904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.201481+0000","last_change":"2024-05-09T02:07:45.185758+0000","last_active":"2024-05-09T02:11:17.201481+0000","last_peered":"2024-05-09T02:11:17.201481+0000","last_clean":"2024-05-09T02:11:17.201481+0000","last_became_active":"2024-05-09T02:07:45.185411+0000","last_became_peered":"2024-05-09T02:07:45.185411+0000","last_unstale":"2024-05-09T02:11:17.201481+0000","last_undegraded":"2024-05-09T02:11:17.201481+0000","last_fullsized":"2024-05-09T02:11:17.201481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:04.410523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282579+0000","last_change":"2024-05-09T02:07:48.199794+0000","last_active":"2024-05-09T02:11:17.282579+0000","last_peered":"2024-05-09T02:11:17.282579+0000","last_clean":"2024-05-09T02:11:17.282579+0000","last_became_active":"2024-05-09T02:07:48.199453+0000","last_became_peered":"2024-05-09T02:07:48.199453+0000","last_unstale":"2024-05-09T02:11:17.282579+0000","last_undegraded":"2024-05-09T02:11:17.282579+0000","last_fullsized":"2024-05-09T02:11:17.282579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:09.735830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200991+0000","last_change":"2024-05-09T02:07:45.162221+0000","last_active":"2024-05-09T02:11:17.200991+0000","last_peered":"2024-05-09T02:11:17.200991+0000","last_clean":"2024-05-09T02:11:17.200991+0000","last_became_active":"2024-05-09T02:07:45.161748+0000","last_became_peered":"2024-05-09T02:07:45.161748+0000","last_unstale":"2024-05-09T02:11:17.200991+0000","last_undegraded":"2024-05-09T02:11:17.200991+0000","last_fullsized":"2024-05-09T02:11:17.200991+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:38.837411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282244+0000","last_change":"2024-05-09T02:07:48.188741+0000","last_active":"2024-05-09T02:11:17.282244+0000","last_peered":"2024-05-09T02:11:17.282244+0000","last_clean":"2024-05-09T02:11:17.282244+0000","last_became_active":"2024-05-09T02:07:48.188329+0000","last_became_peered":"2024-05-09T02:07:48.188329+0000","last_unstale":"2024-05-09T02:11:17.282244+0000","last_undegraded":"2024-05-09T02:11:17.282244+0000","last_fullsized":"2024-05-09T02:11:17.282244+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:46.773336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200769+0000","last_change":"2024-05-09T02:07:45.163573+0000","last_active":"2024-05-09T02:11:17.200769+0000","last_peered":"2024-05-09T02:11:17.200769+0000","last_clean":"2024-05-09T02:11:17.200769+0000","last_became_active":"2024-05-09T02:07:45.163162+0000","last_became_peered":"2024-05-09T02:07:45.163162+0000","last_unstale":"2024-05-09T02:11:17.200769+0000","last_undegraded":"2024-05-09T02:11:17.200769+0000","last_fullsized":"2024-05-09T02:11:17.200769+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:12.356586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209130+0000","last_change":"2024-05-09T02:07:48.203928+0000","last_active":"2024-05-09T02:11:17.209130+0000","last_peered":"2024-05-09T02:11:17.209130+0000","last_clean":"2024-05-09T02:11:17.209130+0000","last_became_active":"2024-05-09T02:07:48.203224+0000","last_became_peered":"2024-05-09T02:07:48.203224+0000","last_unstale":"2024-05-09T02:11:17.209130+0000","last_undegraded":"2024-05-09T02:11:17.209130+0000","last_fullsized":"2024-05-09T02:11:17.209130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:00.568050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":229,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.535090+0000","last_change":"2024-05-09T02:07:45.160161+0000","last_active":"2024-05-09T02:11:17.535090+0000","last_peered":"2024-05-09T02:11:17.535090+0000","last_clean":"2024-05-09T02:11:17.535090+0000","last_became_active":"2024-05-09T02:07:45.159118+0000","last_became_peered":"2024-05-09T02:07:45.159118+0000","last_unstale":"2024-05-09T02:11:17.535090+0000","last_undegraded":"2024-05-09T02:11:17.535090+0000","last_fullsized":"2024-05-09T02:11:17.535090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:04.510877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":193,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200570+0000","last_change":"2024-05-09T02:08:25.319860+0000","last_active":"2024-05-09T02:11:17.200570+0000","last_peered":"2024-05-09T02:11:17.200570+0000","last_clean":"2024-05-09T02:11:17.200570+0000","last_became_active":"2024-05-09T02:08:25.319370+0000","last_became_peered":"2024-05-09T02:08:25.319370+0000","last_unstale":"2024-05-09T02:11:17.200570+0000","last_undegraded":"2024-05-09T02:11:17.200570+0000","last_fullsized":"2024-05-09T02:11:17.200570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:38.504057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.096002+0000","last_change":"2024-05-09T02:08:25.327069+0000","last_active":"2024-05-09T02:11:17.096002+0000","last_peered":"2024-05-09T02:11:17.096002+0000","last_clean":"2024-05-09T02:11:17.096002+0000","last_became_active":"2024-05-09T02:08:25.316629+0000","last_became_peered":"2024-05-09T02:08:25.316629+0000","last_unstale":"2024-05-09T02:11:17.096002+0000","last_undegraded":"2024-05-09T02:11:17.096002+0000","last_fullsized":"2024-05-09T02:11:17.096002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:51.028256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209046+0000","last_change":"2024-05-09T02:07:48.201056+0000","last_active":"2024-05-09T02:11:17.209046+0000","last_peered":"2024-05-09T02:11:17.209046+0000","last_clean":"2024-05-09T02:11:17.209046+0000","last_became_active":"2024-05-09T02:07:48.200671+0000","last_became_peered":"2024-05-09T02:07:48.200671+0000","last_unstale":"2024-05-09T02:11:17.209046+0000","last_undegraded":"2024-05-09T02:11:17.209046+0000","last_fullsized":"2024-05-09T02:11:17.209046+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:37:16.138170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093596+0000","last_change":"2024-05-09T02:07:45.160417+0000","last_active":"2024-05-09T02:11:17.093596+0000","last_peered":"2024-05-09T02:11:17.093596+0000","last_clean":"2024-05-09T02:11:17.093596+0000","last_became_active":"2024-05-09T02:07:45.160076+0000","last_became_peered":"2024-05-09T02:07:45.160076+0000","last_unstale":"2024-05-09T02:11:17.093596+0000","last_undegraded":"2024-05-09T02:11:17.093596+0000","last_fullsized":"2024-05-09T02:11:17.093596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:53.653494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200346+0000","last_change":"2024-05-09T02:07:48.203267+0000","last_active":"2024-05-09T02:11:17.200346+0000","last_peered":"2024-05-09T02:11:17.200346+0000","last_clean":"2024-05-09T02:11:17.200346+0000","last_became_active":"2024-05-09T02:07:48.202747+0000","last_became_peered":"2024-05-09T02:07:48.202747+0000","last_unstale":"2024-05-09T02:11:17.200346+0000","last_undegraded":"2024-05-09T02:11:17.200346+0000","last_fullsized":"2024-05-09T02:11:17.200346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.954400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.208818+0000","last_change":"2024-05-09T02:07:45.164713+0000","last_active":"2024-05-09T02:11:17.208818+0000","last_peered":"2024-05-09T02:11:17.208818+0000","last_clean":"2024-05-09T02:11:17.208818+0000","last_became_active":"2024-05-09T02:07:45.164324+0000","last_became_peered":"2024-05-09T02:07:45.164324+0000","last_unstale":"2024-05-09T02:11:17.208818+0000","last_undegraded":"2024-05-09T02:11:17.208818+0000","last_fullsized":"2024-05-09T02:11:17.208818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:02.481419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.198016+0000","last_change":"2024-05-09T02:07:48.190305+0000","last_active":"2024-05-09T02:11:17.198016+0000","last_peered":"2024-05-09T02:11:17.198016+0000","last_clean":"2024-05-09T02:11:17.198016+0000","last_became_active":"2024-05-09T02:07:48.189971+0000","last_became_peered":"2024-05-09T02:07:48.189971+0000","last_unstale":"2024-05-09T02:11:17.198016+0000","last_undegraded":"2024-05-09T02:11:17.198016+0000","last_fullsized":"2024-05-09T02:11:17.198016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:23.565092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":414,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.534552+0000","last_change":"2024-05-09T02:07:45.164273+0000","last_active":"2024-05-09T02:11:17.534552+0000","last_peered":"2024-05-09T02:11:17.534552+0000","last_clean":"2024-05-09T02:11:17.534552+0000","last_became_active":"2024-05-09T02:07:45.163183+0000","last_became_peered":"2024-05-09T02:07:45.163183+0000","last_unstale":"2024-05-09T02:11:17.534552+0000","last_undegraded":"2024-05-09T02:11:17.534552+0000","last_fullsized":"2024-05-09T02:11:17.534552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:11.395177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":380,"num_read_kb":380,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.283024+0000","last_change":"2024-05-09T02:07:48.202995+0000","last_active":"2024-05-09T02:11:17.283024+0000","last_peered":"2024-05-09T02:11:17.283024+0000","last_clean":"2024-05-09T02:11:17.283024+0000","last_became_active":"2024-05-09T02:07:48.200915+0000","last_became_peered":"2024-05-09T02:07:48.200915+0000","last_unstale":"2024-05-09T02:11:17.283024+0000","last_undegraded":"2024-05-09T02:11:17.283024+0000","last_fullsized":"2024-05-09T02:11:17.283024+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:25.555725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395316+0000","last_change":"2024-05-09T02:07:45.166221+0000","last_active":"2024-05-09T02:11:17.395316+0000","last_peered":"2024-05-09T02:11:17.395316+0000","last_clean":"2024-05-09T02:11:17.395316+0000","last_became_active":"2024-05-09T02:07:45.165813+0000","last_became_peered":"2024-05-09T02:07:45.165813+0000","last_unstale":"2024-05-09T02:11:17.395316+0000","last_undegraded":"2024-05-09T02:11:17.395316+0000","last_fullsized":"2024-05-09T02:11:17.395316+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:01.480828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282689+0000","last_change":"2024-05-09T02:07:48.201475+0000","last_active":"2024-05-09T02:11:17.282689+0000","last_peered":"2024-05-09T02:11:17.282689+0000","last_clean":"2024-05-09T02:11:17.282689+0000","last_became_active":"2024-05-09T02:07:48.201129+0000","last_became_peered":"2024-05-09T02:07:48.201129+0000","last_unstale":"2024-05-09T02:11:17.282689+0000","last_undegraded":"2024-05-09T02:11:17.282689+0000","last_fullsized":"2024-05-09T02:11:17.282689+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:51.726752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209334+0000","last_change":"2024-05-09T02:07:45.185045+0000","last_active":"2024-05-09T02:11:17.209334+0000","last_peered":"2024-05-09T02:11:17.209334+0000","last_clean":"2024-05-09T02:11:17.209334+0000","last_became_active":"2024-05-09T02:07:45.184592+0000","last_became_peered":"2024-05-09T02:07:45.184592+0000","last_unstale":"2024-05-09T02:11:17.209334+0000","last_undegraded":"2024-05-09T02:11:17.209334+0000","last_fullsized":"2024-05-09T02:11:17.209334+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:56.220022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282290+0000","last_change":"2024-05-09T02:07:48.199873+0000","last_active":"2024-05-09T02:11:17.282290+0000","last_peered":"2024-05-09T02:11:17.282290+0000","last_clean":"2024-05-09T02:11:17.282290+0000","last_became_active":"2024-05-09T02:07:48.199502+0000","last_became_peered":"2024-05-09T02:07:48.199502+0000","last_unstale":"2024-05-09T02:11:17.282290+0000","last_undegraded":"2024-05-09T02:11:17.282290+0000","last_fullsized":"2024-05-09T02:11:17.282290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:15.406904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'39","reported_seq":461,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.535242+0000","last_change":"2024-05-09T02:07:45.166167+0000","last_active":"2024-05-09T02:11:17.535242+0000","last_peered":"2024-05-09T02:11:17.535242+0000","last_clean":"2024-05-09T02:11:17.535242+0000","last_became_active":"2024-05-09T02:07:45.165800+0000","last_became_peered":"2024-05-09T02:07:45.165800+0000","last_unstale":"2024-05-09T02:11:17.535242+0000","last_undegraded":"2024-05-09T02:11:17.535242+0000","last_fullsized":"2024-05-09T02:11:17.535242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:03.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68147,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":393,"num_read_kb":263,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200785+0000","last_change":"2024-05-09T02:07:48.187809+0000","last_active":"2024-05-09T02:11:17.200785+0000","last_peered":"2024-05-09T02:11:17.200785+0000","last_clean":"2024-05-09T02:11:17.200785+0000","last_became_active":"2024-05-09T02:07:48.186939+0000","last_became_peered":"2024-05-09T02:07:48.186939+0000","last_unstale":"2024-05-09T02:11:17.200785+0000","last_undegraded":"2024-05-09T02:11:17.200785+0000","last_fullsized":"2024-05-09T02:11:17.200785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:13:15.600343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281818+0000","last_change":"2024-05-09T02:07:45.163756+0000","last_active":"2024-05-09T02:11:17.281818+0000","last_peered":"2024-05-09T02:11:17.281818+0000","last_clean":"2024-05-09T02:11:17.281818+0000","last_became_active":"2024-05-09T02:07:45.163388+0000","last_became_peered":"2024-05-09T02:07:45.163388+0000","last_unstale":"2024-05-09T02:11:17.281818+0000","last_undegraded":"2024-05-09T02:11:17.281818+0000","last_fullsized":"2024-05-09T02:11:17.281818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:41.937148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093618+0000","last_change":"2024-05-09T02:07:48.195218+0000","last_active":"2024-05-09T02:11:17.093618+0000","last_peered":"2024-05-09T02:11:17.093618+0000","last_clean":"2024-05-09T02:11:17.093618+0000","last_became_active":"2024-05-09T02:07:48.194895+0000","last_became_peered":"2024-05-09T02:07:48.194895+0000","last_unstale":"2024-05-09T02:11:17.093618+0000","last_undegraded":"2024-05-09T02:11:17.093618+0000","last_fullsized":"2024-05-09T02:11:17.093618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:49.086594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209218+0000","last_change":"2024-05-09T02:07:45.158039+0000","last_active":"2024-05-09T02:11:17.209218+0000","last_peered":"2024-05-09T02:11:17.209218+0000","last_clean":"2024-05-09T02:11:17.209218+0000","last_became_active":"2024-05-09T02:07:45.157255+0000","last_became_peered":"2024-05-09T02:07:45.157255+0000","last_unstale":"2024-05-09T02:11:17.209218+0000","last_undegraded":"2024-05-09T02:11:17.209218+0000","last_fullsized":"2024-05-09T02:11:17.209218+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:56.513999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395281+0000","last_change":"2024-05-09T02:07:48.200150+0000","last_active":"2024-05-09T02:11:17.395281+0000","last_peered":"2024-05-09T02:11:17.395281+0000","last_clean":"2024-05-09T02:11:17.395281+0000","last_became_active":"2024-05-09T02:07:48.199336+0000","last_became_peered":"2024-05-09T02:07:48.199336+0000","last_unstale":"2024-05-09T02:11:17.395281+0000","last_undegraded":"2024-05-09T02:11:17.395281+0000","last_fullsized":"2024-05-09T02:11:17.395281+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:45.699119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'8","reported_seq":232,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.534669+0000","last_change":"2024-05-09T02:07:45.165476+0000","last_active":"2024-05-09T02:11:17.534669+0000","last_peered":"2024-05-09T02:11:17.534669+0000","last_clean":"2024-05-09T02:11:17.534669+0000","last_became_active":"2024-05-09T02:07:45.165107+0000","last_became_peered":"2024-05-09T02:07:45.165107+0000","last_unstale":"2024-05-09T02:11:17.534669+0000","last_undegraded":"2024-05-09T02:11:17.534669+0000","last_fullsized":"2024-05-09T02:11:17.534669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:15:33.518982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395017+0000","last_change":"2024-05-09T02:07:48.193487+0000","last_active":"2024-05-09T02:11:17.395017+0000","last_peered":"2024-05-09T02:11:17.395017+0000","last_clean":"2024-05-09T02:11:17.395017+0000","last_became_active":"2024-05-09T02:07:48.193108+0000","last_became_peered":"2024-05-09T02:07:48.193108+0000","last_unstale":"2024-05-09T02:11:17.395017+0000","last_undegraded":"2024-05-09T02:11:17.395017+0000","last_fullsized":"2024-05-09T02:11:17.395017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:11.895058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":71,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200474+0000","last_change":"2024-05-09T02:08:25.335525+0000","last_active":"2024-05-09T02:11:17.200474+0000","last_peered":"2024-05-09T02:11:17.200474+0000","last_clean":"2024-05-09T02:11:17.200474+0000","last_became_active":"2024-05-09T02:08:25.319746+0000","last_became_peered":"2024-05-09T02:08:25.319746+0000","last_unstale":"2024-05-09T02:11:17.200474+0000","last_undegraded":"2024-05-09T02:11:17.200474+0000","last_fullsized":"2024-05-09T02:11:17.200474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:41.774066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209033+0000","last_change":"2024-05-09T02:07:48.196125+0000","last_active":"2024-05-09T02:11:17.209033+0000","last_peered":"2024-05-09T02:11:17.209033+0000","last_clean":"2024-05-09T02:11:17.209033+0000","last_became_active":"2024-05-09T02:07:48.195737+0000","last_became_peered":"2024-05-09T02:07:48.195737+0000","last_unstale":"2024-05-09T02:11:17.209033+0000","last_undegraded":"2024-05-09T02:11:17.209033+0000","last_fullsized":"2024-05-09T02:11:17.209033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:43.264720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200887+0000","last_change":"2024-05-09T02:07:45.163791+0000","last_active":"2024-05-09T02:11:17.200887+0000","last_peered":"2024-05-09T02:11:17.200887+0000","last_clean":"2024-05-09T02:11:17.200887+0000","last_became_active":"2024-05-09T02:07:45.163129+0000","last_became_peered":"2024-05-09T02:07:45.163129+0000","last_unstale":"2024-05-09T02:11:17.200887+0000","last_undegraded":"2024-05-09T02:11:17.200887+0000","last_fullsized":"2024-05-09T02:11:17.200887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:48.562850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092838+0000","last_change":"2024-05-09T02:07:48.197782+0000","last_active":"2024-05-09T02:11:17.092838+0000","last_peered":"2024-05-09T02:11:17.092838+0000","last_clean":"2024-05-09T02:11:17.092838+0000","last_became_active":"2024-05-09T02:07:48.197353+0000","last_became_peered":"2024-05-09T02:07:48.197353+0000","last_unstale":"2024-05-09T02:11:17.092838+0000","last_undegraded":"2024-05-09T02:11:17.092838+0000","last_fullsized":"2024-05-09T02:11:17.092838+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:17.053571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.198207+0000","last_change":"2024-05-09T02:07:45.187028+0000","last_active":"2024-05-09T02:11:17.198207+0000","last_peered":"2024-05-09T02:11:17.198207+0000","last_clean":"2024-05-09T02:11:17.198207+0000","last_became_active":"2024-05-09T02:07:45.186639+0000","last_became_peered":"2024-05-09T02:07:45.186639+0000","last_unstale":"2024-05-09T02:11:17.198207+0000","last_undegraded":"2024-05-09T02:11:17.198207+0000","last_fullsized":"2024-05-09T02:11:17.198207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:35.834374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.096134+0000","last_change":"2024-05-09T02:07:48.207150+0000","last_active":"2024-05-09T02:11:17.096134+0000","last_peered":"2024-05-09T02:11:17.096134+0000","last_clean":"2024-05-09T02:11:17.096134+0000","last_became_active":"2024-05-09T02:07:48.206797+0000","last_became_peered":"2024-05-09T02:07:48.206797+0000","last_unstale":"2024-05-09T02:11:17.096134+0000","last_undegraded":"2024-05-09T02:11:17.096134+0000","last_fullsized":"2024-05-09T02:11:17.096134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:37.328133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":228,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.534534+0000","last_change":"2024-05-09T02:07:45.165422+0000","last_active":"2024-05-09T02:11:17.534534+0000","last_peered":"2024-05-09T02:11:17.534534+0000","last_clean":"2024-05-09T02:11:17.534534+0000","last_became_active":"2024-05-09T02:07:45.164998+0000","last_became_peered":"2024-05-09T02:07:45.164998+0000","last_unstale":"2024-05-09T02:11:17.534534+0000","last_undegraded":"2024-05-09T02:11:17.534534+0000","last_fullsized":"2024-05-09T02:11:17.534534+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:49.124323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.096098+0000","last_change":"2024-05-09T02:07:48.206538+0000","last_active":"2024-05-09T02:11:17.096098+0000","last_peered":"2024-05-09T02:11:17.096098+0000","last_clean":"2024-05-09T02:11:17.096098+0000","last_became_active":"2024-05-09T02:07:48.206061+0000","last_became_peered":"2024-05-09T02:07:48.206061+0000","last_unstale":"2024-05-09T02:11:17.096098+0000","last_undegraded":"2024-05-09T02:11:17.096098+0000","last_fullsized":"2024-05-09T02:11:17.096098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:24.075091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209082+0000","last_change":"2024-05-09T02:07:48.202235+0000","last_active":"2024-05-09T02:11:17.209082+0000","last_peered":"2024-05-09T02:11:17.209082+0000","last_clean":"2024-05-09T02:11:17.209082+0000","last_became_active":"2024-05-09T02:07:48.200891+0000","last_became_peered":"2024-05-09T02:07:48.200891+0000","last_unstale":"2024-05-09T02:11:17.209082+0000","last_undegraded":"2024-05-09T02:11:17.209082+0000","last_fullsized":"2024-05-09T02:11:17.209082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:39.061267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'7","reported_seq":227,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.401985+0000","last_change":"2024-05-09T02:07:45.184582+0000","last_active":"2024-05-09T02:11:17.401985+0000","last_peered":"2024-05-09T02:11:17.401985+0000","last_clean":"2024-05-09T02:11:17.401985+0000","last_became_active":"2024-05-09T02:07:45.184187+0000","last_became_peered":"2024-05-09T02:07:45.184187+0000","last_unstale":"2024-05-09T02:11:17.401985+0000","last_undegraded":"2024-05-09T02:11:17.401985+0000","last_fullsized":"2024-05-09T02:11:17.401985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:11.343797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200901+0000","last_change":"2024-05-09T02:07:48.199247+0000","last_active":"2024-05-09T02:11:17.200901+0000","last_peered":"2024-05-09T02:11:17.200901+0000","last_clean":"2024-05-09T02:11:17.200901+0000","last_became_active":"2024-05-09T02:07:48.198121+0000","last_became_peered":"2024-05-09T02:07:48.198121+0000","last_unstale":"2024-05-09T02:11:17.200901+0000","last_undegraded":"2024-05-09T02:11:17.200901+0000","last_fullsized":"2024-05-09T02:11:17.200901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:45:12.885616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282741+0000","last_change":"2024-05-09T02:07:45.163750+0000","last_active":"2024-05-09T02:11:17.282741+0000","last_peered":"2024-05-09T02:11:17.282741+0000","last_clean":"2024-05-09T02:11:17.282741+0000","last_became_active":"2024-05-09T02:07:45.163039+0000","last_became_peered":"2024-05-09T02:07:45.163039+0000","last_unstale":"2024-05-09T02:11:17.282741+0000","last_undegraded":"2024-05-09T02:11:17.282741+0000","last_fullsized":"2024-05-09T02:11:17.282741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:06.794514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":103,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395125+0000","last_change":"2024-05-09T02:06:20.334734+0000","last_active":"2024-05-09T02:11:17.395125+0000","last_peered":"2024-05-09T02:11:17.395125+0000","last_clean":"2024-05-09T02:11:17.395125+0000","last_became_active":"2024-05-09T02:06:20.329198+0000","last_became_peered":"2024-05-09T02:06:20.329198+0000","last_unstale":"2024-05-09T02:11:17.395125+0000","last_undegraded":"2024-05-09T02:11:17.395125+0000","last_fullsized":"2024-05-09T02:11:17.395125+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:02:50.632016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:02:50.632016+0000","last_clean_scrub_stamp":"2024-05-09T02:02:50.632016+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:08:15.495982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093432+0000","last_change":"2024-05-09T02:08:25.321187+0000","last_active":"2024-05-09T02:11:17.093432+0000","last_peered":"2024-05-09T02:11:17.093432+0000","last_clean":"2024-05-09T02:11:17.093432+0000","last_became_active":"2024-05-09T02:08:25.320693+0000","last_became_peered":"2024-05-09T02:08:25.320693+0000","last_unstale":"2024-05-09T02:11:17.093432+0000","last_undegraded":"2024-05-09T02:11:17.093432+0000","last_fullsized":"2024-05-09T02:11:17.093432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:00.176016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395267+0000","last_change":"2024-05-09T02:07:45.158053+0000","last_active":"2024-05-09T02:11:17.395267+0000","last_peered":"2024-05-09T02:11:17.395267+0000","last_clean":"2024-05-09T02:11:17.395267+0000","last_became_active":"2024-05-09T02:07:45.157619+0000","last_became_peered":"2024-05-09T02:07:45.157619+0000","last_unstale":"2024-05-09T02:11:17.395267+0000","last_undegraded":"2024-05-09T02:11:17.395267+0000","last_fullsized":"2024-05-09T02:11:17.395267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:01.618475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092687+0000","last_change":"2024-05-09T02:08:25.341377+0000","last_active":"2024-05-09T02:11:17.092687+0000","last_peered":"2024-05-09T02:11:17.092687+0000","last_clean":"2024-05-09T02:11:17.092687+0000","last_became_active":"2024-05-09T02:08:25.318683+0000","last_became_peered":"2024-05-09T02:08:25.318683+0000","last_unstale":"2024-05-09T02:11:17.092687+0000","last_undegraded":"2024-05-09T02:11:17.092687+0000","last_fullsized":"2024-05-09T02:11:17.092687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:12.456338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093765+0000","last_change":"2024-05-09T02:07:48.186892+0000","last_active":"2024-05-09T02:11:17.093765+0000","last_peered":"2024-05-09T02:11:17.093765+0000","last_clean":"2024-05-09T02:11:17.093765+0000","last_became_active":"2024-05-09T02:07:48.186543+0000","last_became_peered":"2024-05-09T02:07:48.186543+0000","last_unstale":"2024-05-09T02:11:17.093765+0000","last_undegraded":"2024-05-09T02:11:17.093765+0000","last_fullsized":"2024-05-09T02:11:17.093765+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:01.210504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'51","reported_seq":103,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.336046+0000","last_change":"2024-05-09T02:07:45.158007+0000","last_active":"2024-05-09T02:11:17.336046+0000","last_peered":"2024-05-09T02:11:17.336046+0000","last_clean":"2024-05-09T02:11:17.336046+0000","last_became_active":"2024-05-09T02:07:45.157612+0000","last_became_peered":"2024-05-09T02:07:45.157612+0000","last_unstale":"2024-05-09T02:11:17.336046+0000","last_undegraded":"2024-05-09T02:11:17.336046+0000","last_fullsized":"2024-05-09T02:11:17.336046+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:01.526756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.201332+0000","last_change":"2024-05-09T02:07:48.186065+0000","last_active":"2024-05-09T02:11:17.201332+0000","last_peered":"2024-05-09T02:11:17.201332+0000","last_clean":"2024-05-09T02:11:17.201332+0000","last_became_active":"2024-05-09T02:07:48.185746+0000","last_became_peered":"2024-05-09T02:07:48.185746+0000","last_unstale":"2024-05-09T02:11:17.201332+0000","last_undegraded":"2024-05-09T02:11:17.201332+0000","last_fullsized":"2024-05-09T02:11:17.201332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:09.570261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282918+0000","last_change":"2024-05-09T02:07:46.005439+0000","last_active":"2024-05-09T02:11:17.282918+0000","last_peered":"2024-05-09T02:11:17.282918+0000","last_clean":"2024-05-09T02:11:17.282918+0000","last_became_active":"2024-05-09T02:07:46.005087+0000","last_became_peered":"2024-05-09T02:07:46.005087+0000","last_unstale":"2024-05-09T02:11:17.282918+0000","last_undegraded":"2024-05-09T02:11:17.282918+0000","last_fullsized":"2024-05-09T02:11:17.282918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:08.543310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282547+0000","last_change":"2024-05-09T02:07:48.194062+0000","last_active":"2024-05-09T02:11:17.282547+0000","last_peered":"2024-05-09T02:11:17.282547+0000","last_clean":"2024-05-09T02:11:17.282547+0000","last_became_active":"2024-05-09T02:07:48.193712+0000","last_became_peered":"2024-05-09T02:07:48.193712+0000","last_unstale":"2024-05-09T02:11:17.282547+0000","last_undegraded":"2024-05-09T02:11:17.282547+0000","last_fullsized":"2024-05-09T02:11:17.282547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:48.394898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281926+0000","last_change":"2024-05-09T02:07:45.184821+0000","last_active":"2024-05-09T02:11:17.281926+0000","last_peered":"2024-05-09T02:11:17.281926+0000","last_clean":"2024-05-09T02:11:17.281926+0000","last_became_active":"2024-05-09T02:07:45.184437+0000","last_became_peered":"2024-05-09T02:07:45.184437+0000","last_unstale":"2024-05-09T02:11:17.281926+0000","last_undegraded":"2024-05-09T02:11:17.281926+0000","last_fullsized":"2024-05-09T02:11:17.281926+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:30.864171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200507+0000","last_change":"2024-05-09T02:07:48.187635+0000","last_active":"2024-05-09T02:11:17.200507+0000","last_peered":"2024-05-09T02:11:17.200507+0000","last_clean":"2024-05-09T02:11:17.200507+0000","last_became_active":"2024-05-09T02:07:48.187071+0000","last_became_peered":"2024-05-09T02:07:48.187071+0000","last_unstale":"2024-05-09T02:11:17.200507+0000","last_undegraded":"2024-05-09T02:11:17.200507+0000","last_fullsized":"2024-05-09T02:11:17.200507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:10.716715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200715+0000","last_change":"2024-05-09T02:07:45.162166+0000","last_active":"2024-05-09T02:11:17.200715+0000","last_peered":"2024-05-09T02:11:17.200715+0000","last_clean":"2024-05-09T02:11:17.200715+0000","last_became_active":"2024-05-09T02:07:45.161787+0000","last_became_peered":"2024-05-09T02:07:45.161787+0000","last_unstale":"2024-05-09T02:11:17.200715+0000","last_undegraded":"2024-05-09T02:11:17.200715+0000","last_fullsized":"2024-05-09T02:11:17.200715+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:17.925214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092598+0000","last_change":"2024-05-09T02:07:48.202980+0000","last_active":"2024-05-09T02:11:17.092598+0000","last_peered":"2024-05-09T02:11:17.092598+0000","last_clean":"2024-05-09T02:11:17.092598+0000","last_became_active":"2024-05-09T02:07:48.202609+0000","last_became_peered":"2024-05-09T02:07:48.202609+0000","last_unstale":"2024-05-09T02:11:17.092598+0000","last_undegraded":"2024-05-09T02:11:17.092598+0000","last_fullsized":"2024-05-09T02:11:17.092598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:59.993451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092625+0000","last_change":"2024-05-09T02:08:25.326142+0000","last_active":"2024-05-09T02:11:17.092625+0000","last_peered":"2024-05-09T02:11:17.092625+0000","last_clean":"2024-05-09T02:11:17.092625+0000","last_became_active":"2024-05-09T02:08:25.317666+0000","last_became_peered":"2024-05-09T02:08:25.317666+0000","last_unstale":"2024-05-09T02:11:17.092625+0000","last_undegraded":"2024-05-09T02:11:17.092625+0000","last_fullsized":"2024-05-09T02:11:17.092625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:26.178211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200928+0000","last_change":"2024-05-09T02:07:48.202219+0000","last_active":"2024-05-09T02:11:17.200928+0000","last_peered":"2024-05-09T02:11:17.200928+0000","last_clean":"2024-05-09T02:11:17.200928+0000","last_became_active":"2024-05-09T02:07:48.201819+0000","last_became_peered":"2024-05-09T02:07:48.201819+0000","last_unstale":"2024-05-09T02:11:17.200928+0000","last_undegraded":"2024-05-09T02:11:17.200928+0000","last_fullsized":"2024-05-09T02:11:17.200928+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:51.573359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200684+0000","last_change":"2024-05-09T02:07:45.162228+0000","last_active":"2024-05-09T02:11:17.200684+0000","last_peered":"2024-05-09T02:11:17.200684+0000","last_clean":"2024-05-09T02:11:17.200684+0000","last_became_active":"2024-05-09T02:07:45.161847+0000","last_became_peered":"2024-05-09T02:07:45.161847+0000","last_unstale":"2024-05-09T02:11:17.200684+0000","last_undegraded":"2024-05-09T02:11:17.200684+0000","last_fullsized":"2024-05-09T02:11:17.200684+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:47.459709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395519+0000","last_change":"2024-05-09T02:07:48.201401+0000","last_active":"2024-05-09T02:11:17.395519+0000","last_peered":"2024-05-09T02:11:17.395519+0000","last_clean":"2024-05-09T02:11:17.395519+0000","last_became_active":"2024-05-09T02:07:48.201040+0000","last_became_peered":"2024-05-09T02:07:48.201040+0000","last_unstale":"2024-05-09T02:11:17.395519+0000","last_undegraded":"2024-05-09T02:11:17.395519+0000","last_fullsized":"2024-05-09T02:11:17.395519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:58.038690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":226,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282029+0000","last_change":"2024-05-09T02:07:46.005273+0000","last_active":"2024-05-09T02:11:17.282029+0000","last_peered":"2024-05-09T02:11:17.282029+0000","last_clean":"2024-05-09T02:11:17.282029+0000","last_became_active":"2024-05-09T02:07:46.004732+0000","last_became_peered":"2024-05-09T02:07:46.004732+0000","last_unstale":"2024-05-09T02:11:17.282029+0000","last_undegraded":"2024-05-09T02:11:17.282029+0000","last_fullsized":"2024-05-09T02:11:17.282029+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:55.576955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395468+0000","last_change":"2024-05-09T02:07:48.197032+0000","last_active":"2024-05-09T02:11:17.395468+0000","last_peered":"2024-05-09T02:11:17.395468+0000","last_clean":"2024-05-09T02:11:17.395468+0000","last_became_active":"2024-05-09T02:07:48.196553+0000","last_became_peered":"2024-05-09T02:07:48.196553+0000","last_unstale":"2024-05-09T02:11:17.395468+0000","last_undegraded":"2024-05-09T02:11:17.395468+0000","last_fullsized":"2024-05-09T02:11:17.395468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:50:25.081024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'41","reported_seq":78,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.333602+0000","last_change":"2024-05-09T02:07:45.166456+0000","last_active":"2024-05-09T02:11:17.333602+0000","last_peered":"2024-05-09T02:11:17.333602+0000","last_clean":"2024-05-09T02:11:17.333602+0000","last_became_active":"2024-05-09T02:07:45.166132+0000","last_became_peered":"2024-05-09T02:07:45.166132+0000","last_unstale":"2024-05-09T02:11:17.333602+0000","last_undegraded":"2024-05-09T02:11:17.333602+0000","last_fullsized":"2024-05-09T02:11:17.333602+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:15.701802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395265+0000","last_change":"2024-05-09T02:08:25.323303+0000","last_active":"2024-05-09T02:11:17.395265+0000","last_peered":"2024-05-09T02:11:17.395265+0000","last_clean":"2024-05-09T02:11:17.395265+0000","last_became_active":"2024-05-09T02:08:25.321752+0000","last_became_peered":"2024-05-09T02:08:25.321752+0000","last_unstale":"2024-05-09T02:11:17.395265+0000","last_undegraded":"2024-05-09T02:11:17.395265+0000","last_fullsized":"2024-05-09T02:11:17.395265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:17.791737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093809+0000","last_change":"2024-05-09T02:07:45.155480+0000","last_active":"2024-05-09T02:11:17.093809+0000","last_peered":"2024-05-09T02:11:17.093809+0000","last_clean":"2024-05-09T02:11:17.093809+0000","last_became_active":"2024-05-09T02:07:45.155085+0000","last_became_peered":"2024-05-09T02:07:45.155085+0000","last_unstale":"2024-05-09T02:11:17.093809+0000","last_undegraded":"2024-05-09T02:11:17.093809+0000","last_fullsized":"2024-05-09T02:11:17.093809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:05.547857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395468+0000","last_change":"2024-05-09T02:07:48.194480+0000","last_active":"2024-05-09T02:11:17.395468+0000","last_peered":"2024-05-09T02:11:17.395468+0000","last_clean":"2024-05-09T02:11:17.395468+0000","last_became_active":"2024-05-09T02:07:48.194136+0000","last_became_peered":"2024-05-09T02:07:48.194136+0000","last_unstale":"2024-05-09T02:11:17.395468+0000","last_undegraded":"2024-05-09T02:11:17.395468+0000","last_fullsized":"2024-05-09T02:11:17.395468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:09.177762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092843+0000","last_change":"2024-05-09T02:07:48.195152+0000","last_active":"2024-05-09T02:11:17.092843+0000","last_peered":"2024-05-09T02:11:17.092843+0000","last_clean":"2024-05-09T02:11:17.092843+0000","last_became_active":"2024-05-09T02:07:48.194779+0000","last_became_peered":"2024-05-09T02:07:48.194779+0000","last_unstale":"2024-05-09T02:11:17.092843+0000","last_undegraded":"2024-05-09T02:11:17.092843+0000","last_fullsized":"2024-05-09T02:11:17.092843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:39.345452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281768+0000","last_change":"2024-05-09T02:08:25.322235+0000","last_active":"2024-05-09T02:11:17.281768+0000","last_peered":"2024-05-09T02:11:17.281768+0000","last_clean":"2024-05-09T02:11:17.281768+0000","last_became_active":"2024-05-09T02:08:25.321674+0000","last_became_peered":"2024-05-09T02:08:25.321674+0000","last_unstale":"2024-05-09T02:11:17.281768+0000","last_undegraded":"2024-05-09T02:11:17.281768+0000","last_fullsized":"2024-05-09T02:11:17.281768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:52.671669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281697+0000","last_change":"2024-05-09T02:07:48.195710+0000","last_active":"2024-05-09T02:11:17.281697+0000","last_peered":"2024-05-09T02:11:17.281697+0000","last_clean":"2024-05-09T02:11:17.281697+0000","last_became_active":"2024-05-09T02:07:48.195332+0000","last_became_peered":"2024-05-09T02:07:48.195332+0000","last_unstale":"2024-05-09T02:11:17.281697+0000","last_undegraded":"2024-05-09T02:11:17.281697+0000","last_fullsized":"2024-05-09T02:11:17.281697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:12:17.904166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282255+0000","last_change":"2024-05-09T02:07:48.200791+0000","last_active":"2024-05-09T02:11:17.282255+0000","last_peered":"2024-05-09T02:11:17.282255+0000","last_clean":"2024-05-09T02:11:17.282255+0000","last_became_active":"2024-05-09T02:07:48.199916+0000","last_became_peered":"2024-05-09T02:07:48.199916+0000","last_unstale":"2024-05-09T02:11:17.282255+0000","last_undegraded":"2024-05-09T02:11:17.282255+0000","last_fullsized":"2024-05-09T02:11:17.282255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:32.692375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093770+0000","last_change":"2024-05-09T02:07:48.191662+0000","last_active":"2024-05-09T02:11:17.093770+0000","last_peered":"2024-05-09T02:11:17.093770+0000","last_clean":"2024-05-09T02:11:17.093770+0000","last_became_active":"2024-05-09T02:07:48.191336+0000","last_became_peered":"2024-05-09T02:07:48.191336+0000","last_unstale":"2024-05-09T02:11:17.093770+0000","last_undegraded":"2024-05-09T02:11:17.093770+0000","last_fullsized":"2024-05-09T02:11:17.093770+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:00.733611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092797+0000","last_change":"2024-05-09T02:07:48.195182+0000","last_active":"2024-05-09T02:11:17.092797+0000","last_peered":"2024-05-09T02:11:17.092797+0000","last_clean":"2024-05-09T02:11:17.092797+0000","last_became_active":"2024-05-09T02:07:48.194828+0000","last_became_peered":"2024-05-09T02:07:48.194828+0000","last_unstale":"2024-05-09T02:11:17.092797+0000","last_undegraded":"2024-05-09T02:11:17.092797+0000","last_fullsized":"2024-05-09T02:11:17.092797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:33.427409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282054+0000","last_change":"2024-05-09T02:07:48.199688+0000","last_active":"2024-05-09T02:11:17.282054+0000","last_peered":"2024-05-09T02:11:17.282054+0000","last_clean":"2024-05-09T02:11:17.282054+0000","last_became_active":"2024-05-09T02:07:48.199326+0000","last_became_peered":"2024-05-09T02:07:48.199326+0000","last_unstale":"2024-05-09T02:11:17.282054+0000","last_undegraded":"2024-05-09T02:11:17.282054+0000","last_fullsized":"2024-05-09T02:11:17.282054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:30.662738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.201606+0000","last_change":"2024-05-09T02:07:48.190727+0000","last_active":"2024-05-09T02:11:17.201606+0000","last_peered":"2024-05-09T02:11:17.201606+0000","last_clean":"2024-05-09T02:11:17.201606+0000","last_became_active":"2024-05-09T02:07:48.190397+0000","last_became_peered":"2024-05-09T02:07:48.190397+0000","last_unstale":"2024-05-09T02:11:17.201606+0000","last_undegraded":"2024-05-09T02:11:17.201606+0000","last_fullsized":"2024-05-09T02:11:17.201606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:42.010669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.208911+0000","last_change":"2024-05-09T02:07:48.200444+0000","last_active":"2024-05-09T02:11:17.208911+0000","last_peered":"2024-05-09T02:11:17.208911+0000","last_clean":"2024-05-09T02:11:17.208911+0000","last_became_active":"2024-05-09T02:07:48.200031+0000","last_became_peered":"2024-05-09T02:07:48.200031+0000","last_unstale":"2024-05-09T02:11:17.208911+0000","last_undegraded":"2024-05-09T02:11:17.208911+0000","last_fullsized":"2024-05-09T02:11:17.208911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:30.269792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200630+0000","last_change":"2024-05-09T02:07:48.203717+0000","last_active":"2024-05-09T02:11:17.200630+0000","last_peered":"2024-05-09T02:11:17.200630+0000","last_clean":"2024-05-09T02:11:17.200630+0000","last_became_active":"2024-05-09T02:07:48.203347+0000","last_became_peered":"2024-05-09T02:07:48.203347+0000","last_unstale":"2024-05-09T02:11:17.200630+0000","last_undegraded":"2024-05-09T02:11:17.200630+0000","last_fullsized":"2024-05-09T02:11:17.200630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:17.391504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282202+0000","last_change":"2024-05-09T02:08:25.323029+0000","last_active":"2024-05-09T02:11:17.282202+0000","last_peered":"2024-05-09T02:11:17.282202+0000","last_clean":"2024-05-09T02:11:17.282202+0000","last_became_active":"2024-05-09T02:08:25.322524+0000","last_became_peered":"2024-05-09T02:08:25.322524+0000","last_unstale":"2024-05-09T02:11:17.282202+0000","last_undegraded":"2024-05-09T02:11:17.282202+0000","last_fullsized":"2024-05-09T02:11:17.282202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:25.017511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093749+0000","last_change":"2024-05-09T02:07:48.187802+0000","last_active":"2024-05-09T02:11:17.093749+0000","last_peered":"2024-05-09T02:11:17.093749+0000","last_clean":"2024-05-09T02:11:17.093749+0000","last_became_active":"2024-05-09T02:07:48.187478+0000","last_became_peered":"2024-05-09T02:07:48.187478+0000","last_unstale":"2024-05-09T02:11:17.093749+0000","last_undegraded":"2024-05-09T02:11:17.093749+0000","last_fullsized":"2024-05-09T02:11:17.093749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:12.981386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092634+0000","last_change":"2024-05-09T02:07:48.192117+0000","last_active":"2024-05-09T02:11:17.092634+0000","last_peered":"2024-05-09T02:11:17.092634+0000","last_clean":"2024-05-09T02:11:17.092634+0000","last_became_active":"2024-05-09T02:07:48.191726+0000","last_became_peered":"2024-05-09T02:07:48.191726+0000","last_unstale":"2024-05-09T02:11:17.092634+0000","last_undegraded":"2024-05-09T02:11:17.092634+0000","last_fullsized":"2024-05-09T02:11:17.092634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:07.735582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107049,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601872,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2039,"num_read_kb":1136,"num_write":374,"num_write_kb":1465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1805616,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678073,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":438649,"data_compressed":8666,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874300,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":1097854,"data_compressed":44349,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037828,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":946176,"data_stored":831800,"data_compressed":27812,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234056,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":831949,"data_compressed":27812,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430285,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":966656,"data_stored":1033459,"data_compressed":44349,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593811,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":438966,"data_compressed":8666,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":497868,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":921600,"data_stored":439523,"data_compressed":8666,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":497976,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149795,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":962708,"data_compressed":33968,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953565,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":429704,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790041,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":1163032,"data_compressed":50505,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135202,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202406,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T02:11:19.920 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-09T02:11:19.920 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T02:11:20.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.084+0000 7fa88dfa8640 1 -- 172.21.15.19:0/3328530660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888102b10 msgr2=0x7fa888103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:20.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.084+0000 7fa88dfa8640 1 --2- 172.21.15.19:0/3328530660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888102b10 0x7fa888103810 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa878009900 tx=0x7fa87802a410 comp rx=0 tx=0).stop 2024-05-09T02:11:20.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.084+0000 7fa88dfa8640 1 -- 172.21.15.19:0/3328530660 shutdown_connections 2024-05-09T02:11:20.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.084+0000 7fa88dfa8640 1 --2- 172.21.15.19:0/3328530660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa888108a50 0x7fa88810ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:20.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.084+0000 7fa88dfa8640 1 --2- 172.21.15.19:0/3328530660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa888103d50 0x7fa888104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:20.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.084+0000 7fa88dfa8640 1 --2- 172.21.15.19:0/3328530660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888102b10 0x7fa888103810 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:20.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.084+0000 7fa88dfa8640 1 -- 172.21.15.19:0/3328530660 >> 172.21.15.19:0/3328530660 conn(0x7fa8880fdd00 msgr2=0x7fa8881001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:20.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.084+0000 7fa88dfa8640 1 -- 172.21.15.19:0/3328530660 shutdown_connections 2024-05-09T02:11:20.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.084+0000 7fa88dfa8640 1 -- 172.21.15.19:0/3328530660 wait complete. 2024-05-09T02:11:20.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.084+0000 7fa88dfa8640 1 Processor -- start 2024-05-09T02:11:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa88dfa8640 1 -- start start 2024-05-09T02:11:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa88dfa8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa888102b10 0x7fa8881992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa88dfa8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888103d50 0x7fa888199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa88dfa8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa888108a50 0x7fa8881a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa88dfa8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa88810e610 con 0x7fa888103d50 2024-05-09T02:11:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa88dfa8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa88810e490 con 0x7fa888102b10 2024-05-09T02:11:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa88dfa8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa88810e790 con 0x7fa888108a50 2024-05-09T02:11:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa886ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888103d50 0x7fa888199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa886ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888103d50 0x7fa888199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44518/0 (socket says 172.21.15.19:44518) 2024-05-09T02:11:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa886ffd640 1 -- 172.21.15.19:0/588702489 learned_addr learned my addr 172.21.15.19:0/588702489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa8877fe640 1 --2- 172.21.15.19:0/588702489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa888102b10 0x7fa8881992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa887fff640 1 --2- 172.21.15.19:0/588702489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa888108a50 0x7fa8881a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa886ffd640 1 -- 172.21.15.19:0/588702489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa888108a50 msgr2=0x7fa8881a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa886ffd640 1 --2- 172.21.15.19:0/588702489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa888108a50 0x7fa8881a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa886ffd640 1 -- 172.21.15.19:0/588702489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa888102b10 msgr2=0x7fa8881992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa886ffd640 1 --2- 172.21.15.19:0/588702489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa888102b10 0x7fa8881992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.085+0000 7fa886ffd640 1 -- 172.21.15.19:0/588702489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8881a0e10 con 0x7fa888103d50 2024-05-09T02:11:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.086+0000 7fa886ffd640 1 --2- 172.21.15.19:0/588702489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888103d50 0x7fa888199810 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00e9f0 tx=0x7fa87c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.086+0000 7fa884ff9640 1 -- 172.21.15.19:0/588702489 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c00cd80 con 0x7fa888103d50 2024-05-09T02:11:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.086+0000 7fa884ff9640 1 -- 172.21.15.19:0/588702489 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa87c0054d0 con 0x7fa888103d50 2024-05-09T02:11:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.086+0000 7fa88dfa8640 1 -- 172.21.15.19:0/588702489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8881a1100 con 0x7fa888103d50 2024-05-09T02:11:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.086+0000 7fa884ff9640 1 -- 172.21.15.19:0/588702489 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c010690 con 0x7fa888103d50 2024-05-09T02:11:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.086+0000 7fa88dfa8640 1 -- 172.21.15.19:0/588702489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8881a1600 con 0x7fa888103d50 2024-05-09T02:11:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.087+0000 7fa88dfa8640 1 -- 172.21.15.19:0/588702489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa888107860 con 0x7fa888103d50 2024-05-09T02:11:20.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.088+0000 7fa884ff9640 1 -- 172.21.15.19:0/588702489 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 23) ==== 149772+0+0 (secure 0 0 0) 0x7fa87c004e90 con 0x7fa888103d50 2024-05-09T02:11:20.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.089+0000 7fa884ff9640 1 --2- 172.21.15.19:0/588702489 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8700b2960 0x7fa8700b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:20.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.089+0000 7fa8877fe640 1 --2- 172.21.15.19:0/588702489 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8700b2960 0x7fa8700b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:20.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.089+0000 7fa884ff9640 1 -- 172.21.15.19:0/588702489 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fa87c0e2a90 con 0x7fa888103d50 2024-05-09T02:11:20.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.090+0000 7fa8877fe640 1 --2- 172.21.15.19:0/588702489 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8700b2960 0x7fa8700b4e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa878005ab0 tx=0x7fa878031400 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:20.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.094+0000 7fa884ff9640 1 -- 172.21.15.19:0/588702489 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa87c0aa410 con 0x7fa888103d50 2024-05-09T02:11:20.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.288+0000 7fa88dfa8640 1 -- 172.21.15.19:0/588702489 --> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa888063110 con 0x7fa8700b2960 2024-05-09T02:11:20.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.303+0000 7fa884ff9640 1 -- 172.21.15.19:0/588702489 <== mgr.15222 v2:172.21.15.19:6800/2638922161 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258055 (secure 0 0 0) 0x7fa888063110 con 0x7fa8700b2960 2024-05-09T02:11:20.308 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:20.309 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-09T02:11:20.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.307+0000 7fa88dfa8640 1 -- 172.21.15.19:0/588702489 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8700b2960 msgr2=0x7fa8700b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:20.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.307+0000 7fa88dfa8640 1 --2- 172.21.15.19:0/588702489 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8700b2960 0x7fa8700b4e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa878005ab0 tx=0x7fa878031400 comp rx=0 tx=0).stop 2024-05-09T02:11:20.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.307+0000 7fa88dfa8640 1 -- 172.21.15.19:0/588702489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888103d50 msgr2=0x7fa888199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:20.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.307+0000 7fa88dfa8640 1 --2- 172.21.15.19:0/588702489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888103d50 0x7fa888199810 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00e9f0 tx=0x7fa87c00edf0 comp rx=0 tx=0).stop 2024-05-09T02:11:20.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.307+0000 7fa88dfa8640 1 -- 172.21.15.19:0/588702489 shutdown_connections 2024-05-09T02:11:20.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.307+0000 7fa88dfa8640 1 --2- 172.21.15.19:0/588702489 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8700b2960 0x7fa8700b4e70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:20.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.307+0000 7fa88dfa8640 1 --2- 172.21.15.19:0/588702489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa888108a50 0x7fa8881a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:20.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.307+0000 7fa88dfa8640 1 --2- 172.21.15.19:0/588702489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888103d50 0x7fa888199810 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:20.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.307+0000 7fa88dfa8640 1 --2- 172.21.15.19:0/588702489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa888102b10 0x7fa8881992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:20.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.307+0000 7fa88dfa8640 1 -- 172.21.15.19:0/588702489 >> 172.21.15.19:0/588702489 conn(0x7fa8880fdd00 msgr2=0x7fa888104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:20.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.307+0000 7fa88dfa8640 1 -- 172.21.15.19:0/588702489 shutdown_connections 2024-05-09T02:11:20.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:20.307+0000 7fa88dfa8640 1 -- 172.21.15.19:0/588702489 wait complete. 2024-05-09T02:11:20.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:20 smithi019 ceph-mon[30151]: [09/May/2024:02:11:18] ENGINE Bus STARTING 2024-05-09T02:11:20.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:20 smithi019 ceph-mon[30151]: [09/May/2024:02:11:18] ENGINE Serving on https://172.21.15.19:7150 2024-05-09T02:11:20.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:20 smithi019 ceph-mon[30151]: [09/May/2024:02:11:19] ENGINE Serving on http://172.21.15.19:8765 2024-05-09T02:11:20.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:20 smithi019 ceph-mon[30151]: [09/May/2024:02:11:19] ENGINE Bus STARTED 2024-05-09T02:11:20.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:20 smithi019 ceph-mon[30151]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:11:20.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:20 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:20.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:20 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:20.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:20 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:20.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:20 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:20.320 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:20 smithi019 ceph-mon[30151]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-09T02:11:20.321 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-05-09T02:11:19.110260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192474,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2120,"num_read_kb":1205,"num_write":499,"num_write_kb":3617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2952064,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":895,"ondisk_log_size":895,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376960,"kb_used_data":11140,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564864,"statfs":{"total":1151940427776,"available":1151554420736,"internally_reserved":0,"allocated":11407360,"data_stored":8663488,"data_compressed":272094,"data_compressed_allocated":360448,"data_compressed_original":3465216,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.983889"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093030+0000","last_change":"2024-05-09T02:07:48.203957+0000","last_active":"2024-05-09T02:11:17.093030+0000","last_peered":"2024-05-09T02:11:17.093030+0000","last_clean":"2024-05-09T02:11:17.093030+0000","last_became_active":"2024-05-09T02:07:48.203354+0000","last_became_peered":"2024-05-09T02:07:48.203354+0000","last_unstale":"2024-05-09T02:11:17.093030+0000","last_undegraded":"2024-05-09T02:11:17.093030+0000","last_fullsized":"2024-05-09T02:11:17.093030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:06.169693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282389+0000","last_change":"2024-05-09T02:07:48.192885+0000","last_active":"2024-05-09T02:11:17.282389+0000","last_peered":"2024-05-09T02:11:17.282389+0000","last_clean":"2024-05-09T02:11:17.282389+0000","last_became_active":"2024-05-09T02:07:48.192476+0000","last_became_peered":"2024-05-09T02:07:48.192476+0000","last_unstale":"2024-05-09T02:11:17.282389+0000","last_undegraded":"2024-05-09T02:11:17.282389+0000","last_fullsized":"2024-05-09T02:11:17.282389+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:33.710309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282172+0000","last_change":"2024-05-09T02:07:48.198684+0000","last_active":"2024-05-09T02:11:17.282172+0000","last_peered":"2024-05-09T02:11:17.282172+0000","last_clean":"2024-05-09T02:11:17.282172+0000","last_became_active":"2024-05-09T02:07:48.198344+0000","last_became_peered":"2024-05-09T02:07:48.198344+0000","last_unstale":"2024-05-09T02:11:17.282172+0000","last_undegraded":"2024-05-09T02:11:17.282172+0000","last_fullsized":"2024-05-09T02:11:17.282172+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:19.990063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282720+0000","last_change":"2024-05-09T02:07:48.189209+0000","last_active":"2024-05-09T02:11:17.282720+0000","last_peered":"2024-05-09T02:11:17.282720+0000","last_clean":"2024-05-09T02:11:17.282720+0000","last_became_active":"2024-05-09T02:07:48.188744+0000","last_became_peered":"2024-05-09T02:07:48.188744+0000","last_unstale":"2024-05-09T02:11:17.282720+0000","last_undegraded":"2024-05-09T02:11:17.282720+0000","last_fullsized":"2024-05-09T02:11:17.282720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:03:25.216849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200644+0000","last_change":"2024-05-09T02:07:48.187897+0000","last_active":"2024-05-09T02:11:17.200644+0000","last_peered":"2024-05-09T02:11:17.200644+0000","last_clean":"2024-05-09T02:11:17.200644+0000","last_became_active":"2024-05-09T02:07:48.187476+0000","last_became_peered":"2024-05-09T02:07:48.187476+0000","last_unstale":"2024-05-09T02:11:17.200644+0000","last_undegraded":"2024-05-09T02:11:17.200644+0000","last_fullsized":"2024-05-09T02:11:17.200644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:39.366520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.096096+0000","last_change":"2024-05-09T02:07:48.195436+0000","last_active":"2024-05-09T02:11:17.096096+0000","last_peered":"2024-05-09T02:11:17.096096+0000","last_clean":"2024-05-09T02:11:17.096096+0000","last_became_active":"2024-05-09T02:07:48.194982+0000","last_became_peered":"2024-05-09T02:07:48.194982+0000","last_unstale":"2024-05-09T02:11:17.096096+0000","last_undegraded":"2024-05-09T02:11:17.096096+0000","last_fullsized":"2024-05-09T02:11:17.096096+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:19.746701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395057+0000","last_change":"2024-05-09T02:07:48.198458+0000","last_active":"2024-05-09T02:11:17.395057+0000","last_peered":"2024-05-09T02:11:17.395057+0000","last_clean":"2024-05-09T02:11:17.395057+0000","last_became_active":"2024-05-09T02:07:48.198089+0000","last_became_peered":"2024-05-09T02:07:48.198089+0000","last_unstale":"2024-05-09T02:11:17.395057+0000","last_undegraded":"2024-05-09T02:11:17.395057+0000","last_fullsized":"2024-05-09T02:11:17.395057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:37:44.999438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200698+0000","last_change":"2024-05-09T02:07:48.200294+0000","last_active":"2024-05-09T02:11:17.200698+0000","last_peered":"2024-05-09T02:11:17.200698+0000","last_clean":"2024-05-09T02:11:17.200698+0000","last_became_active":"2024-05-09T02:07:48.199949+0000","last_became_peered":"2024-05-09T02:07:48.199949+0000","last_unstale":"2024-05-09T02:11:17.200698+0000","last_undegraded":"2024-05-09T02:11:17.200698+0000","last_fullsized":"2024-05-09T02:11:17.200698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:39.263849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.283155+0000","last_change":"2024-05-09T02:07:48.196282+0000","last_active":"2024-05-09T02:11:17.283155+0000","last_peered":"2024-05-09T02:11:17.283155+0000","last_clean":"2024-05-09T02:11:17.283155+0000","last_became_active":"2024-05-09T02:07:48.195887+0000","last_became_peered":"2024-05-09T02:07:48.195887+0000","last_unstale":"2024-05-09T02:11:17.283155+0000","last_undegraded":"2024-05-09T02:11:17.283155+0000","last_fullsized":"2024-05-09T02:11:17.283155+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:24:26.081867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.096575+0000","last_change":"2024-05-09T02:08:25.320636+0000","last_active":"2024-05-09T02:11:17.096575+0000","last_peered":"2024-05-09T02:11:17.096575+0000","last_clean":"2024-05-09T02:11:17.096575+0000","last_became_active":"2024-05-09T02:08:25.320155+0000","last_became_peered":"2024-05-09T02:08:25.320155+0000","last_unstale":"2024-05-09T02:11:17.096575+0000","last_undegraded":"2024-05-09T02:11:17.096575+0000","last_fullsized":"2024-05-09T02:11:17.096575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:55.728773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395383+0000","last_change":"2024-05-09T02:07:48.197649+0000","last_active":"2024-05-09T02:11:17.395383+0000","last_peered":"2024-05-09T02:11:17.395383+0000","last_clean":"2024-05-09T02:11:17.395383+0000","last_became_active":"2024-05-09T02:07:48.196884+0000","last_became_peered":"2024-05-09T02:07:48.196884+0000","last_unstale":"2024-05-09T02:11:17.395383+0000","last_undegraded":"2024-05-09T02:11:17.395383+0000","last_fullsized":"2024-05-09T02:11:17.395383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:44:23.851216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282732+0000","last_change":"2024-05-09T02:07:48.193568+0000","last_active":"2024-05-09T02:11:17.282732+0000","last_peered":"2024-05-09T02:11:17.282732+0000","last_clean":"2024-05-09T02:11:17.282732+0000","last_became_active":"2024-05-09T02:07:48.193117+0000","last_became_peered":"2024-05-09T02:07:48.193117+0000","last_unstale":"2024-05-09T02:11:17.282732+0000","last_undegraded":"2024-05-09T02:11:17.282732+0000","last_fullsized":"2024-05-09T02:11:17.282732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:34.371246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093762+0000","last_change":"2024-05-09T02:07:48.196314+0000","last_active":"2024-05-09T02:11:17.093762+0000","last_peered":"2024-05-09T02:11:17.093762+0000","last_clean":"2024-05-09T02:11:17.093762+0000","last_became_active":"2024-05-09T02:07:48.195988+0000","last_became_peered":"2024-05-09T02:07:48.195988+0000","last_unstale":"2024-05-09T02:11:17.093762+0000","last_undegraded":"2024-05-09T02:11:17.093762+0000","last_fullsized":"2024-05-09T02:11:17.093762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:08.556166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281979+0000","last_change":"2024-05-09T02:07:48.192974+0000","last_active":"2024-05-09T02:11:17.281979+0000","last_peered":"2024-05-09T02:11:17.281979+0000","last_clean":"2024-05-09T02:11:17.281979+0000","last_became_active":"2024-05-09T02:07:48.192599+0000","last_became_peered":"2024-05-09T02:07:48.192599+0000","last_unstale":"2024-05-09T02:11:17.281979+0000","last_undegraded":"2024-05-09T02:11:17.281979+0000","last_fullsized":"2024-05-09T02:11:17.281979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:37.192219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281774+0000","last_change":"2024-05-09T02:07:48.193208+0000","last_active":"2024-05-09T02:11:17.281774+0000","last_peered":"2024-05-09T02:11:17.281774+0000","last_clean":"2024-05-09T02:11:17.281774+0000","last_became_active":"2024-05-09T02:07:48.192654+0000","last_became_peered":"2024-05-09T02:07:48.192654+0000","last_unstale":"2024-05-09T02:11:17.281774+0000","last_undegraded":"2024-05-09T02:11:17.281774+0000","last_fullsized":"2024-05-09T02:11:17.281774+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:55:42.870372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281908+0000","last_change":"2024-05-09T02:07:48.187568+0000","last_active":"2024-05-09T02:11:17.281908+0000","last_peered":"2024-05-09T02:11:17.281908+0000","last_clean":"2024-05-09T02:11:17.281908+0000","last_became_active":"2024-05-09T02:07:48.187159+0000","last_became_peered":"2024-05-09T02:07:48.187159+0000","last_unstale":"2024-05-09T02:11:17.281908+0000","last_undegraded":"2024-05-09T02:11:17.281908+0000","last_fullsized":"2024-05-09T02:11:17.281908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:04.994094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093126+0000","last_change":"2024-05-09T02:07:48.204406+0000","last_active":"2024-05-09T02:11:17.093126+0000","last_peered":"2024-05-09T02:11:17.093126+0000","last_clean":"2024-05-09T02:11:17.093126+0000","last_became_active":"2024-05-09T02:07:48.203592+0000","last_became_peered":"2024-05-09T02:07:48.203592+0000","last_unstale":"2024-05-09T02:11:17.093126+0000","last_undegraded":"2024-05-09T02:11:17.093126+0000","last_fullsized":"2024-05-09T02:11:17.093126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:40:21.655807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.198463+0000","last_change":"2024-05-09T02:07:48.195326+0000","last_active":"2024-05-09T02:11:17.198463+0000","last_peered":"2024-05-09T02:11:17.198463+0000","last_clean":"2024-05-09T02:11:17.198463+0000","last_became_active":"2024-05-09T02:07:48.194969+0000","last_became_peered":"2024-05-09T02:07:48.194969+0000","last_unstale":"2024-05-09T02:11:17.198463+0000","last_undegraded":"2024-05-09T02:11:17.198463+0000","last_fullsized":"2024-05-09T02:11:17.198463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:00.269753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281961+0000","last_change":"2024-05-09T02:07:48.190706+0000","last_active":"2024-05-09T02:11:17.281961+0000","last_peered":"2024-05-09T02:11:17.281961+0000","last_clean":"2024-05-09T02:11:17.281961+0000","last_became_active":"2024-05-09T02:07:48.190171+0000","last_became_peered":"2024-05-09T02:07:48.190171+0000","last_unstale":"2024-05-09T02:11:17.281961+0000","last_undegraded":"2024-05-09T02:11:17.281961+0000","last_fullsized":"2024-05-09T02:11:17.281961+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:19:31.333618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":234,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.394913+0000","last_change":"2024-05-09T02:07:45.158224+0000","last_active":"2024-05-09T02:11:17.394913+0000","last_peered":"2024-05-09T02:11:17.394913+0000","last_clean":"2024-05-09T02:11:17.394913+0000","last_became_active":"2024-05-09T02:07:45.157722+0000","last_became_peered":"2024-05-09T02:07:45.157722+0000","last_unstale":"2024-05-09T02:11:17.394913+0000","last_undegraded":"2024-05-09T02:11:17.394913+0000","last_fullsized":"2024-05-09T02:11:17.394913+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:31.750699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.201211+0000","last_change":"2024-05-09T02:07:48.197978+0000","last_active":"2024-05-09T02:11:17.201211+0000","last_peered":"2024-05-09T02:11:17.201211+0000","last_clean":"2024-05-09T02:11:17.201211+0000","last_became_active":"2024-05-09T02:07:48.197077+0000","last_became_peered":"2024-05-09T02:07:48.197077+0000","last_unstale":"2024-05-09T02:11:17.201211+0000","last_undegraded":"2024-05-09T02:11:17.201211+0000","last_fullsized":"2024-05-09T02:11:17.201211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:25.897904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.201481+0000","last_change":"2024-05-09T02:07:45.185758+0000","last_active":"2024-05-09T02:11:17.201481+0000","last_peered":"2024-05-09T02:11:17.201481+0000","last_clean":"2024-05-09T02:11:17.201481+0000","last_became_active":"2024-05-09T02:07:45.185411+0000","last_became_peered":"2024-05-09T02:07:45.185411+0000","last_unstale":"2024-05-09T02:11:17.201481+0000","last_undegraded":"2024-05-09T02:11:17.201481+0000","last_fullsized":"2024-05-09T02:11:17.201481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:04.410523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282579+0000","last_change":"2024-05-09T02:07:48.199794+0000","last_active":"2024-05-09T02:11:17.282579+0000","last_peered":"2024-05-09T02:11:17.282579+0000","last_clean":"2024-05-09T02:11:17.282579+0000","last_became_active":"2024-05-09T02:07:48.199453+0000","last_became_peered":"2024-05-09T02:07:48.199453+0000","last_unstale":"2024-05-09T02:11:17.282579+0000","last_undegraded":"2024-05-09T02:11:17.282579+0000","last_fullsized":"2024-05-09T02:11:17.282579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:09.735830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200991+0000","last_change":"2024-05-09T02:07:45.162221+0000","last_active":"2024-05-09T02:11:17.200991+0000","last_peered":"2024-05-09T02:11:17.200991+0000","last_clean":"2024-05-09T02:11:17.200991+0000","last_became_active":"2024-05-09T02:07:45.161748+0000","last_became_peered":"2024-05-09T02:07:45.161748+0000","last_unstale":"2024-05-09T02:11:17.200991+0000","last_undegraded":"2024-05-09T02:11:17.200991+0000","last_fullsized":"2024-05-09T02:11:17.200991+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:38.837411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282244+0000","last_change":"2024-05-09T02:07:48.188741+0000","last_active":"2024-05-09T02:11:17.282244+0000","last_peered":"2024-05-09T02:11:17.282244+0000","last_clean":"2024-05-09T02:11:17.282244+0000","last_became_active":"2024-05-09T02:07:48.188329+0000","last_became_peered":"2024-05-09T02:07:48.188329+0000","last_unstale":"2024-05-09T02:11:17.282244+0000","last_undegraded":"2024-05-09T02:11:17.282244+0000","last_fullsized":"2024-05-09T02:11:17.282244+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:46.773336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200769+0000","last_change":"2024-05-09T02:07:45.163573+0000","last_active":"2024-05-09T02:11:17.200769+0000","last_peered":"2024-05-09T02:11:17.200769+0000","last_clean":"2024-05-09T02:11:17.200769+0000","last_became_active":"2024-05-09T02:07:45.163162+0000","last_became_peered":"2024-05-09T02:07:45.163162+0000","last_unstale":"2024-05-09T02:11:17.200769+0000","last_undegraded":"2024-05-09T02:11:17.200769+0000","last_fullsized":"2024-05-09T02:11:17.200769+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:12.356586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209130+0000","last_change":"2024-05-09T02:07:48.203928+0000","last_active":"2024-05-09T02:11:17.209130+0000","last_peered":"2024-05-09T02:11:17.209130+0000","last_clean":"2024-05-09T02:11:17.209130+0000","last_became_active":"2024-05-09T02:07:48.203224+0000","last_became_peered":"2024-05-09T02:07:48.203224+0000","last_unstale":"2024-05-09T02:11:17.209130+0000","last_undegraded":"2024-05-09T02:11:17.209130+0000","last_fullsized":"2024-05-09T02:11:17.209130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:00.568050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":229,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.535090+0000","last_change":"2024-05-09T02:07:45.160161+0000","last_active":"2024-05-09T02:11:17.535090+0000","last_peered":"2024-05-09T02:11:17.535090+0000","last_clean":"2024-05-09T02:11:17.535090+0000","last_became_active":"2024-05-09T02:07:45.159118+0000","last_became_peered":"2024-05-09T02:07:45.159118+0000","last_unstale":"2024-05-09T02:11:17.535090+0000","last_undegraded":"2024-05-09T02:11:17.535090+0000","last_fullsized":"2024-05-09T02:11:17.535090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:04.510877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":193,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200570+0000","last_change":"2024-05-09T02:08:25.319860+0000","last_active":"2024-05-09T02:11:17.200570+0000","last_peered":"2024-05-09T02:11:17.200570+0000","last_clean":"2024-05-09T02:11:17.200570+0000","last_became_active":"2024-05-09T02:08:25.319370+0000","last_became_peered":"2024-05-09T02:08:25.319370+0000","last_unstale":"2024-05-09T02:11:17.200570+0000","last_undegraded":"2024-05-09T02:11:17.200570+0000","last_fullsized":"2024-05-09T02:11:17.200570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:38.504057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.096002+0000","last_change":"2024-05-09T02:08:25.327069+0000","last_active":"2024-05-09T02:11:17.096002+0000","last_peered":"2024-05-09T02:11:17.096002+0000","last_clean":"2024-05-09T02:11:17.096002+0000","last_became_active":"2024-05-09T02:08:25.316629+0000","last_became_peered":"2024-05-09T02:08:25.316629+0000","last_unstale":"2024-05-09T02:11:17.096002+0000","last_undegraded":"2024-05-09T02:11:17.096002+0000","last_fullsized":"2024-05-09T02:11:17.096002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:51.028256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209046+0000","last_change":"2024-05-09T02:07:48.201056+0000","last_active":"2024-05-09T02:11:17.209046+0000","last_peered":"2024-05-09T02:11:17.209046+0000","last_clean":"2024-05-09T02:11:17.209046+0000","last_became_active":"2024-05-09T02:07:48.200671+0000","last_became_peered":"2024-05-09T02:07:48.200671+0000","last_unstale":"2024-05-09T02:11:17.209046+0000","last_undegraded":"2024-05-09T02:11:17.209046+0000","last_fullsized":"2024-05-09T02:11:17.209046+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:37:16.138170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093596+0000","last_change":"2024-05-09T02:07:45.160417+0000","last_active":"2024-05-09T02:11:17.093596+0000","last_peered":"2024-05-09T02:11:17.093596+0000","last_clean":"2024-05-09T02:11:17.093596+0000","last_became_active":"2024-05-09T02:07:45.160076+0000","last_became_peered":"2024-05-09T02:07:45.160076+0000","last_unstale":"2024-05-09T02:11:17.093596+0000","last_undegraded":"2024-05-09T02:11:17.093596+0000","last_fullsized":"2024-05-09T02:11:17.093596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:53.653494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200346+0000","last_change":"2024-05-09T02:07:48.203267+0000","last_active":"2024-05-09T02:11:17.200346+0000","last_peered":"2024-05-09T02:11:17.200346+0000","last_clean":"2024-05-09T02:11:17.200346+0000","last_became_active":"2024-05-09T02:07:48.202747+0000","last_became_peered":"2024-05-09T02:07:48.202747+0000","last_unstale":"2024-05-09T02:11:17.200346+0000","last_undegraded":"2024-05-09T02:11:17.200346+0000","last_fullsized":"2024-05-09T02:11:17.200346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:43:59.954400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.208818+0000","last_change":"2024-05-09T02:07:45.164713+0000","last_active":"2024-05-09T02:11:17.208818+0000","last_peered":"2024-05-09T02:11:17.208818+0000","last_clean":"2024-05-09T02:11:17.208818+0000","last_became_active":"2024-05-09T02:07:45.164324+0000","last_became_peered":"2024-05-09T02:07:45.164324+0000","last_unstale":"2024-05-09T02:11:17.208818+0000","last_undegraded":"2024-05-09T02:11:17.208818+0000","last_fullsized":"2024-05-09T02:11:17.208818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:02.481419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.198016+0000","last_change":"2024-05-09T02:07:48.190305+0000","last_active":"2024-05-09T02:11:17.198016+0000","last_peered":"2024-05-09T02:11:17.198016+0000","last_clean":"2024-05-09T02:11:17.198016+0000","last_became_active":"2024-05-09T02:07:48.189971+0000","last_became_peered":"2024-05-09T02:07:48.189971+0000","last_unstale":"2024-05-09T02:11:17.198016+0000","last_undegraded":"2024-05-09T02:11:17.198016+0000","last_fullsized":"2024-05-09T02:11:17.198016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:23.565092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":414,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.534552+0000","last_change":"2024-05-09T02:07:45.164273+0000","last_active":"2024-05-09T02:11:17.534552+0000","last_peered":"2024-05-09T02:11:17.534552+0000","last_clean":"2024-05-09T02:11:17.534552+0000","last_became_active":"2024-05-09T02:07:45.163183+0000","last_became_peered":"2024-05-09T02:07:45.163183+0000","last_unstale":"2024-05-09T02:11:17.534552+0000","last_undegraded":"2024-05-09T02:11:17.534552+0000","last_fullsized":"2024-05-09T02:11:17.534552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:11.395177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":380,"num_read_kb":380,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.283024+0000","last_change":"2024-05-09T02:07:48.202995+0000","last_active":"2024-05-09T02:11:17.283024+0000","last_peered":"2024-05-09T02:11:17.283024+0000","last_clean":"2024-05-09T02:11:17.283024+0000","last_became_active":"2024-05-09T02:07:48.200915+0000","last_became_peered":"2024-05-09T02:07:48.200915+0000","last_unstale":"2024-05-09T02:11:17.283024+0000","last_undegraded":"2024-05-09T02:11:17.283024+0000","last_fullsized":"2024-05-09T02:11:17.283024+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:25.555725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395316+0000","last_change":"2024-05-09T02:07:45.166221+0000","last_active":"2024-05-09T02:11:17.395316+0000","last_peered":"2024-05-09T02:11:17.395316+0000","last_clean":"2024-05-09T02:11:17.395316+0000","last_became_active":"2024-05-09T02:07:45.165813+0000","last_became_peered":"2024-05-09T02:07:45.165813+0000","last_unstale":"2024-05-09T02:11:17.395316+0000","last_undegraded":"2024-05-09T02:11:17.395316+0000","last_fullsized":"2024-05-09T02:11:17.395316+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:01.480828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282689+0000","last_change":"2024-05-09T02:07:48.201475+0000","last_active":"2024-05-09T02:11:17.282689+0000","last_peered":"2024-05-09T02:11:17.282689+0000","last_clean":"2024-05-09T02:11:17.282689+0000","last_became_active":"2024-05-09T02:07:48.201129+0000","last_became_peered":"2024-05-09T02:07:48.201129+0000","last_unstale":"2024-05-09T02:11:17.282689+0000","last_undegraded":"2024-05-09T02:11:17.282689+0000","last_fullsized":"2024-05-09T02:11:17.282689+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:51.726752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209334+0000","last_change":"2024-05-09T02:07:45.185045+0000","last_active":"2024-05-09T02:11:17.209334+0000","last_peered":"2024-05-09T02:11:17.209334+0000","last_clean":"2024-05-09T02:11:17.209334+0000","last_became_active":"2024-05-09T02:07:45.184592+0000","last_became_peered":"2024-05-09T02:07:45.184592+0000","last_unstale":"2024-05-09T02:11:17.209334+0000","last_undegraded":"2024-05-09T02:11:17.209334+0000","last_fullsized":"2024-05-09T02:11:17.209334+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:56.220022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282290+0000","last_change":"2024-05-09T02:07:48.199873+0000","last_active":"2024-05-09T02:11:17.282290+0000","last_peered":"2024-05-09T02:11:17.282290+0000","last_clean":"2024-05-09T02:11:17.282290+0000","last_became_active":"2024-05-09T02:07:48.199502+0000","last_became_peered":"2024-05-09T02:07:48.199502+0000","last_unstale":"2024-05-09T02:11:17.282290+0000","last_undegraded":"2024-05-09T02:11:17.282290+0000","last_fullsized":"2024-05-09T02:11:17.282290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:15.406904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'39","reported_seq":461,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.535242+0000","last_change":"2024-05-09T02:07:45.166167+0000","last_active":"2024-05-09T02:11:17.535242+0000","last_peered":"2024-05-09T02:11:17.535242+0000","last_clean":"2024-05-09T02:11:17.535242+0000","last_became_active":"2024-05-09T02:07:45.165800+0000","last_became_peered":"2024-05-09T02:07:45.165800+0000","last_unstale":"2024-05-09T02:11:17.535242+0000","last_undegraded":"2024-05-09T02:11:17.535242+0000","last_fullsized":"2024-05-09T02:11:17.535242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:03.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68147,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":393,"num_read_kb":263,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200785+0000","last_change":"2024-05-09T02:07:48.187809+0000","last_active":"2024-05-09T02:11:17.200785+0000","last_peered":"2024-05-09T02:11:17.200785+0000","last_clean":"2024-05-09T02:11:17.200785+0000","last_became_active":"2024-05-09T02:07:48.186939+0000","last_became_peered":"2024-05-09T02:07:48.186939+0000","last_unstale":"2024-05-09T02:11:17.200785+0000","last_undegraded":"2024-05-09T02:11:17.200785+0000","last_fullsized":"2024-05-09T02:11:17.200785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:13:15.600343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281818+0000","last_change":"2024-05-09T02:07:45.163756+0000","last_active":"2024-05-09T02:11:17.281818+0000","last_peered":"2024-05-09T02:11:17.281818+0000","last_clean":"2024-05-09T02:11:17.281818+0000","last_became_active":"2024-05-09T02:07:45.163388+0000","last_became_peered":"2024-05-09T02:07:45.163388+0000","last_unstale":"2024-05-09T02:11:17.281818+0000","last_undegraded":"2024-05-09T02:11:17.281818+0000","last_fullsized":"2024-05-09T02:11:17.281818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:41.937148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093618+0000","last_change":"2024-05-09T02:07:48.195218+0000","last_active":"2024-05-09T02:11:17.093618+0000","last_peered":"2024-05-09T02:11:17.093618+0000","last_clean":"2024-05-09T02:11:17.093618+0000","last_became_active":"2024-05-09T02:07:48.194895+0000","last_became_peered":"2024-05-09T02:07:48.194895+0000","last_unstale":"2024-05-09T02:11:17.093618+0000","last_undegraded":"2024-05-09T02:11:17.093618+0000","last_fullsized":"2024-05-09T02:11:17.093618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:49.086594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209218+0000","last_change":"2024-05-09T02:07:45.158039+0000","last_active":"2024-05-09T02:11:17.209218+0000","last_peered":"2024-05-09T02:11:17.209218+0000","last_clean":"2024-05-09T02:11:17.209218+0000","last_became_active":"2024-05-09T02:07:45.157255+0000","last_became_peered":"2024-05-09T02:07:45.157255+0000","last_unstale":"2024-05-09T02:11:17.209218+0000","last_undegraded":"2024-05-09T02:11:17.209218+0000","last_fullsized":"2024-05-09T02:11:17.209218+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:56.513999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395281+0000","last_change":"2024-05-09T02:07:48.200150+0000","last_active":"2024-05-09T02:11:17.395281+0000","last_peered":"2024-05-09T02:11:17.395281+0000","last_clean":"2024-05-09T02:11:17.395281+0000","last_became_active":"2024-05-09T02:07:48.199336+0000","last_became_peered":"2024-05-09T02:07:48.199336+0000","last_unstale":"2024-05-09T02:11:17.395281+0000","last_undegraded":"2024-05-09T02:11:17.395281+0000","last_fullsized":"2024-05-09T02:11:17.395281+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:45.699119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'8","reported_seq":232,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.534669+0000","last_change":"2024-05-09T02:07:45.165476+0000","last_active":"2024-05-09T02:11:17.534669+0000","last_peered":"2024-05-09T02:11:17.534669+0000","last_clean":"2024-05-09T02:11:17.534669+0000","last_became_active":"2024-05-09T02:07:45.165107+0000","last_became_peered":"2024-05-09T02:07:45.165107+0000","last_unstale":"2024-05-09T02:11:17.534669+0000","last_undegraded":"2024-05-09T02:11:17.534669+0000","last_fullsized":"2024-05-09T02:11:17.534669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:15:33.518982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395017+0000","last_change":"2024-05-09T02:07:48.193487+0000","last_active":"2024-05-09T02:11:17.395017+0000","last_peered":"2024-05-09T02:11:17.395017+0000","last_clean":"2024-05-09T02:11:17.395017+0000","last_became_active":"2024-05-09T02:07:48.193108+0000","last_became_peered":"2024-05-09T02:07:48.193108+0000","last_unstale":"2024-05-09T02:11:17.395017+0000","last_undegraded":"2024-05-09T02:11:17.395017+0000","last_fullsized":"2024-05-09T02:11:17.395017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:11.895058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":71,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200474+0000","last_change":"2024-05-09T02:08:25.335525+0000","last_active":"2024-05-09T02:11:17.200474+0000","last_peered":"2024-05-09T02:11:17.200474+0000","last_clean":"2024-05-09T02:11:17.200474+0000","last_became_active":"2024-05-09T02:08:25.319746+0000","last_became_peered":"2024-05-09T02:08:25.319746+0000","last_unstale":"2024-05-09T02:11:17.200474+0000","last_undegraded":"2024-05-09T02:11:17.200474+0000","last_fullsized":"2024-05-09T02:11:17.200474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:41.774066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209033+0000","last_change":"2024-05-09T02:07:48.196125+0000","last_active":"2024-05-09T02:11:17.209033+0000","last_peered":"2024-05-09T02:11:17.209033+0000","last_clean":"2024-05-09T02:11:17.209033+0000","last_became_active":"2024-05-09T02:07:48.195737+0000","last_became_peered":"2024-05-09T02:07:48.195737+0000","last_unstale":"2024-05-09T02:11:17.209033+0000","last_undegraded":"2024-05-09T02:11:17.209033+0000","last_fullsized":"2024-05-09T02:11:17.209033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:43.264720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200887+0000","last_change":"2024-05-09T02:07:45.163791+0000","last_active":"2024-05-09T02:11:17.200887+0000","last_peered":"2024-05-09T02:11:17.200887+0000","last_clean":"2024-05-09T02:11:17.200887+0000","last_became_active":"2024-05-09T02:07:45.163129+0000","last_became_peered":"2024-05-09T02:07:45.163129+0000","last_unstale":"2024-05-09T02:11:17.200887+0000","last_undegraded":"2024-05-09T02:11:17.200887+0000","last_fullsized":"2024-05-09T02:11:17.200887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:48.562850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092838+0000","last_change":"2024-05-09T02:07:48.197782+0000","last_active":"2024-05-09T02:11:17.092838+0000","last_peered":"2024-05-09T02:11:17.092838+0000","last_clean":"2024-05-09T02:11:17.092838+0000","last_became_active":"2024-05-09T02:07:48.197353+0000","last_became_peered":"2024-05-09T02:07:48.197353+0000","last_unstale":"2024-05-09T02:11:17.092838+0000","last_undegraded":"2024-05-09T02:11:17.092838+0000","last_fullsized":"2024-05-09T02:11:17.092838+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:17.053571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.198207+0000","last_change":"2024-05-09T02:07:45.187028+0000","last_active":"2024-05-09T02:11:17.198207+0000","last_peered":"2024-05-09T02:11:17.198207+0000","last_clean":"2024-05-09T02:11:17.198207+0000","last_became_active":"2024-05-09T02:07:45.186639+0000","last_became_peered":"2024-05-09T02:07:45.186639+0000","last_unstale":"2024-05-09T02:11:17.198207+0000","last_undegraded":"2024-05-09T02:11:17.198207+0000","last_fullsized":"2024-05-09T02:11:17.198207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:35.834374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.096134+0000","last_change":"2024-05-09T02:07:48.207150+0000","last_active":"2024-05-09T02:11:17.096134+0000","last_peered":"2024-05-09T02:11:17.096134+0000","last_clean":"2024-05-09T02:11:17.096134+0000","last_became_active":"2024-05-09T02:07:48.206797+0000","last_became_peered":"2024-05-09T02:07:48.206797+0000","last_unstale":"2024-05-09T02:11:17.096134+0000","last_undegraded":"2024-05-09T02:11:17.096134+0000","last_fullsized":"2024-05-09T02:11:17.096134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:37.328133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":228,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.534534+0000","last_change":"2024-05-09T02:07:45.165422+0000","last_active":"2024-05-09T02:11:17.534534+0000","last_peered":"2024-05-09T02:11:17.534534+0000","last_clean":"2024-05-09T02:11:17.534534+0000","last_became_active":"2024-05-09T02:07:45.164998+0000","last_became_peered":"2024-05-09T02:07:45.164998+0000","last_unstale":"2024-05-09T02:11:17.534534+0000","last_undegraded":"2024-05-09T02:11:17.534534+0000","last_fullsized":"2024-05-09T02:11:17.534534+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:49.124323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.096098+0000","last_change":"2024-05-09T02:07:48.206538+0000","last_active":"2024-05-09T02:11:17.096098+0000","last_peered":"2024-05-09T02:11:17.096098+0000","last_clean":"2024-05-09T02:11:17.096098+0000","last_became_active":"2024-05-09T02:07:48.206061+0000","last_became_peered":"2024-05-09T02:07:48.206061+0000","last_unstale":"2024-05-09T02:11:17.096098+0000","last_undegraded":"2024-05-09T02:11:17.096098+0000","last_fullsized":"2024-05-09T02:11:17.096098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:24.075091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.209082+0000","last_change":"2024-05-09T02:07:48.202235+0000","last_active":"2024-05-09T02:11:17.209082+0000","last_peered":"2024-05-09T02:11:17.209082+0000","last_clean":"2024-05-09T02:11:17.209082+0000","last_became_active":"2024-05-09T02:07:48.200891+0000","last_became_peered":"2024-05-09T02:07:48.200891+0000","last_unstale":"2024-05-09T02:11:17.209082+0000","last_undegraded":"2024-05-09T02:11:17.209082+0000","last_fullsized":"2024-05-09T02:11:17.209082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:39.061267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'7","reported_seq":227,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.401985+0000","last_change":"2024-05-09T02:07:45.184582+0000","last_active":"2024-05-09T02:11:17.401985+0000","last_peered":"2024-05-09T02:11:17.401985+0000","last_clean":"2024-05-09T02:11:17.401985+0000","last_became_active":"2024-05-09T02:07:45.184187+0000","last_became_peered":"2024-05-09T02:07:45.184187+0000","last_unstale":"2024-05-09T02:11:17.401985+0000","last_undegraded":"2024-05-09T02:11:17.401985+0000","last_fullsized":"2024-05-09T02:11:17.401985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:11.343797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200901+0000","last_change":"2024-05-09T02:07:48.199247+0000","last_active":"2024-05-09T02:11:17.200901+0000","last_peered":"2024-05-09T02:11:17.200901+0000","last_clean":"2024-05-09T02:11:17.200901+0000","last_became_active":"2024-05-09T02:07:48.198121+0000","last_became_peered":"2024-05-09T02:07:48.198121+0000","last_unstale":"2024-05-09T02:11:17.200901+0000","last_undegraded":"2024-05-09T02:11:17.200901+0000","last_fullsized":"2024-05-09T02:11:17.200901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:45:12.885616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282741+0000","last_change":"2024-05-09T02:07:45.163750+0000","last_active":"2024-05-09T02:11:17.282741+0000","last_peered":"2024-05-09T02:11:17.282741+0000","last_clean":"2024-05-09T02:11:17.282741+0000","last_became_active":"2024-05-09T02:07:45.163039+0000","last_became_peered":"2024-05-09T02:07:45.163039+0000","last_unstale":"2024-05-09T02:11:17.282741+0000","last_undegraded":"2024-05-09T02:11:17.282741+0000","last_fullsized":"2024-05-09T02:11:17.282741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:06.794514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":103,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395125+0000","last_change":"2024-05-09T02:06:20.334734+0000","last_active":"2024-05-09T02:11:17.395125+0000","last_peered":"2024-05-09T02:11:17.395125+0000","last_clean":"2024-05-09T02:11:17.395125+0000","last_became_active":"2024-05-09T02:06:20.329198+0000","last_became_peered":"2024-05-09T02:06:20.329198+0000","last_unstale":"2024-05-09T02:11:17.395125+0000","last_undegraded":"2024-05-09T02:11:17.395125+0000","last_fullsized":"2024-05-09T02:11:17.395125+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:02:50.632016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:02:50.632016+0000","last_clean_scrub_stamp":"2024-05-09T02:02:50.632016+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:08:15.495982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093432+0000","last_change":"2024-05-09T02:08:25.321187+0000","last_active":"2024-05-09T02:11:17.093432+0000","last_peered":"2024-05-09T02:11:17.093432+0000","last_clean":"2024-05-09T02:11:17.093432+0000","last_became_active":"2024-05-09T02:08:25.320693+0000","last_became_peered":"2024-05-09T02:08:25.320693+0000","last_unstale":"2024-05-09T02:11:17.093432+0000","last_undegraded":"2024-05-09T02:11:17.093432+0000","last_fullsized":"2024-05-09T02:11:17.093432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:00.176016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395267+0000","last_change":"2024-05-09T02:07:45.158053+0000","last_active":"2024-05-09T02:11:17.395267+0000","last_peered":"2024-05-09T02:11:17.395267+0000","last_clean":"2024-05-09T02:11:17.395267+0000","last_became_active":"2024-05-09T02:07:45.157619+0000","last_became_peered":"2024-05-09T02:07:45.157619+0000","last_unstale":"2024-05-09T02:11:17.395267+0000","last_undegraded":"2024-05-09T02:11:17.395267+0000","last_fullsized":"2024-05-09T02:11:17.395267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:01.618475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092687+0000","last_change":"2024-05-09T02:08:25.341377+0000","last_active":"2024-05-09T02:11:17.092687+0000","last_peered":"2024-05-09T02:11:17.092687+0000","last_clean":"2024-05-09T02:11:17.092687+0000","last_became_active":"2024-05-09T02:08:25.318683+0000","last_became_peered":"2024-05-09T02:08:25.318683+0000","last_unstale":"2024-05-09T02:11:17.092687+0000","last_undegraded":"2024-05-09T02:11:17.092687+0000","last_fullsized":"2024-05-09T02:11:17.092687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:12.456338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093765+0000","last_change":"2024-05-09T02:07:48.186892+0000","last_active":"2024-05-09T02:11:17.093765+0000","last_peered":"2024-05-09T02:11:17.093765+0000","last_clean":"2024-05-09T02:11:17.093765+0000","last_became_active":"2024-05-09T02:07:48.186543+0000","last_became_peered":"2024-05-09T02:07:48.186543+0000","last_unstale":"2024-05-09T02:11:17.093765+0000","last_undegraded":"2024-05-09T02:11:17.093765+0000","last_fullsized":"2024-05-09T02:11:17.093765+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:01.210504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'51","reported_seq":103,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.336046+0000","last_change":"2024-05-09T02:07:45.158007+0000","last_active":"2024-05-09T02:11:17.336046+0000","last_peered":"2024-05-09T02:11:17.336046+0000","last_clean":"2024-05-09T02:11:17.336046+0000","last_became_active":"2024-05-09T02:07:45.157612+0000","last_became_peered":"2024-05-09T02:07:45.157612+0000","last_unstale":"2024-05-09T02:11:17.336046+0000","last_undegraded":"2024-05-09T02:11:17.336046+0000","last_fullsized":"2024-05-09T02:11:17.336046+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:21:01.526756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.201332+0000","last_change":"2024-05-09T02:07:48.186065+0000","last_active":"2024-05-09T02:11:17.201332+0000","last_peered":"2024-05-09T02:11:17.201332+0000","last_clean":"2024-05-09T02:11:17.201332+0000","last_became_active":"2024-05-09T02:07:48.185746+0000","last_became_peered":"2024-05-09T02:07:48.185746+0000","last_unstale":"2024-05-09T02:11:17.201332+0000","last_undegraded":"2024-05-09T02:11:17.201332+0000","last_fullsized":"2024-05-09T02:11:17.201332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:09.570261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282918+0000","last_change":"2024-05-09T02:07:46.005439+0000","last_active":"2024-05-09T02:11:17.282918+0000","last_peered":"2024-05-09T02:11:17.282918+0000","last_clean":"2024-05-09T02:11:17.282918+0000","last_became_active":"2024-05-09T02:07:46.005087+0000","last_became_peered":"2024-05-09T02:07:46.005087+0000","last_unstale":"2024-05-09T02:11:17.282918+0000","last_undegraded":"2024-05-09T02:11:17.282918+0000","last_fullsized":"2024-05-09T02:11:17.282918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:08.543310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282547+0000","last_change":"2024-05-09T02:07:48.194062+0000","last_active":"2024-05-09T02:11:17.282547+0000","last_peered":"2024-05-09T02:11:17.282547+0000","last_clean":"2024-05-09T02:11:17.282547+0000","last_became_active":"2024-05-09T02:07:48.193712+0000","last_became_peered":"2024-05-09T02:07:48.193712+0000","last_unstale":"2024-05-09T02:11:17.282547+0000","last_undegraded":"2024-05-09T02:11:17.282547+0000","last_fullsized":"2024-05-09T02:11:17.282547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:48.394898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281926+0000","last_change":"2024-05-09T02:07:45.184821+0000","last_active":"2024-05-09T02:11:17.281926+0000","last_peered":"2024-05-09T02:11:17.281926+0000","last_clean":"2024-05-09T02:11:17.281926+0000","last_became_active":"2024-05-09T02:07:45.184437+0000","last_became_peered":"2024-05-09T02:07:45.184437+0000","last_unstale":"2024-05-09T02:11:17.281926+0000","last_undegraded":"2024-05-09T02:11:17.281926+0000","last_fullsized":"2024-05-09T02:11:17.281926+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:30.864171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200507+0000","last_change":"2024-05-09T02:07:48.187635+0000","last_active":"2024-05-09T02:11:17.200507+0000","last_peered":"2024-05-09T02:11:17.200507+0000","last_clean":"2024-05-09T02:11:17.200507+0000","last_became_active":"2024-05-09T02:07:48.187071+0000","last_became_peered":"2024-05-09T02:07:48.187071+0000","last_unstale":"2024-05-09T02:11:17.200507+0000","last_undegraded":"2024-05-09T02:11:17.200507+0000","last_fullsized":"2024-05-09T02:11:17.200507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:10.716715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200715+0000","last_change":"2024-05-09T02:07:45.162166+0000","last_active":"2024-05-09T02:11:17.200715+0000","last_peered":"2024-05-09T02:11:17.200715+0000","last_clean":"2024-05-09T02:11:17.200715+0000","last_became_active":"2024-05-09T02:07:45.161787+0000","last_became_peered":"2024-05-09T02:07:45.161787+0000","last_unstale":"2024-05-09T02:11:17.200715+0000","last_undegraded":"2024-05-09T02:11:17.200715+0000","last_fullsized":"2024-05-09T02:11:17.200715+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:17.925214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092598+0000","last_change":"2024-05-09T02:07:48.202980+0000","last_active":"2024-05-09T02:11:17.092598+0000","last_peered":"2024-05-09T02:11:17.092598+0000","last_clean":"2024-05-09T02:11:17.092598+0000","last_became_active":"2024-05-09T02:07:48.202609+0000","last_became_peered":"2024-05-09T02:07:48.202609+0000","last_unstale":"2024-05-09T02:11:17.092598+0000","last_undegraded":"2024-05-09T02:11:17.092598+0000","last_fullsized":"2024-05-09T02:11:17.092598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:59.993451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092625+0000","last_change":"2024-05-09T02:08:25.326142+0000","last_active":"2024-05-09T02:11:17.092625+0000","last_peered":"2024-05-09T02:11:17.092625+0000","last_clean":"2024-05-09T02:11:17.092625+0000","last_became_active":"2024-05-09T02:08:25.317666+0000","last_became_peered":"2024-05-09T02:08:25.317666+0000","last_unstale":"2024-05-09T02:11:17.092625+0000","last_undegraded":"2024-05-09T02:11:17.092625+0000","last_fullsized":"2024-05-09T02:11:17.092625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:26.178211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200928+0000","last_change":"2024-05-09T02:07:48.202219+0000","last_active":"2024-05-09T02:11:17.200928+0000","last_peered":"2024-05-09T02:11:17.200928+0000","last_clean":"2024-05-09T02:11:17.200928+0000","last_became_active":"2024-05-09T02:07:48.201819+0000","last_became_peered":"2024-05-09T02:07:48.201819+0000","last_unstale":"2024-05-09T02:11:17.200928+0000","last_undegraded":"2024-05-09T02:11:17.200928+0000","last_fullsized":"2024-05-09T02:11:17.200928+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:51.573359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200684+0000","last_change":"2024-05-09T02:07:45.162228+0000","last_active":"2024-05-09T02:11:17.200684+0000","last_peered":"2024-05-09T02:11:17.200684+0000","last_clean":"2024-05-09T02:11:17.200684+0000","last_became_active":"2024-05-09T02:07:45.161847+0000","last_became_peered":"2024-05-09T02:07:45.161847+0000","last_unstale":"2024-05-09T02:11:17.200684+0000","last_undegraded":"2024-05-09T02:11:17.200684+0000","last_fullsized":"2024-05-09T02:11:17.200684+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:47.459709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395519+0000","last_change":"2024-05-09T02:07:48.201401+0000","last_active":"2024-05-09T02:11:17.395519+0000","last_peered":"2024-05-09T02:11:17.395519+0000","last_clean":"2024-05-09T02:11:17.395519+0000","last_became_active":"2024-05-09T02:07:48.201040+0000","last_became_peered":"2024-05-09T02:07:48.201040+0000","last_unstale":"2024-05-09T02:11:17.395519+0000","last_undegraded":"2024-05-09T02:11:17.395519+0000","last_fullsized":"2024-05-09T02:11:17.395519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:58.038690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":226,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282029+0000","last_change":"2024-05-09T02:07:46.005273+0000","last_active":"2024-05-09T02:11:17.282029+0000","last_peered":"2024-05-09T02:11:17.282029+0000","last_clean":"2024-05-09T02:11:17.282029+0000","last_became_active":"2024-05-09T02:07:46.004732+0000","last_became_peered":"2024-05-09T02:07:46.004732+0000","last_unstale":"2024-05-09T02:11:17.282029+0000","last_undegraded":"2024-05-09T02:11:17.282029+0000","last_fullsized":"2024-05-09T02:11:17.282029+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:55.576955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395468+0000","last_change":"2024-05-09T02:07:48.197032+0000","last_active":"2024-05-09T02:11:17.395468+0000","last_peered":"2024-05-09T02:11:17.395468+0000","last_clean":"2024-05-09T02:11:17.395468+0000","last_became_active":"2024-05-09T02:07:48.196553+0000","last_became_peered":"2024-05-09T02:07:48.196553+0000","last_unstale":"2024-05-09T02:11:17.395468+0000","last_undegraded":"2024-05-09T02:11:17.395468+0000","last_fullsized":"2024-05-09T02:11:17.395468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:50:25.081024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'41","reported_seq":78,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.333602+0000","last_change":"2024-05-09T02:07:45.166456+0000","last_active":"2024-05-09T02:11:17.333602+0000","last_peered":"2024-05-09T02:11:17.333602+0000","last_clean":"2024-05-09T02:11:17.333602+0000","last_became_active":"2024-05-09T02:07:45.166132+0000","last_became_peered":"2024-05-09T02:07:45.166132+0000","last_unstale":"2024-05-09T02:11:17.333602+0000","last_undegraded":"2024-05-09T02:11:17.333602+0000","last_fullsized":"2024-05-09T02:11:17.333602+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:15.701802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395265+0000","last_change":"2024-05-09T02:08:25.323303+0000","last_active":"2024-05-09T02:11:17.395265+0000","last_peered":"2024-05-09T02:11:17.395265+0000","last_clean":"2024-05-09T02:11:17.395265+0000","last_became_active":"2024-05-09T02:08:25.321752+0000","last_became_peered":"2024-05-09T02:08:25.321752+0000","last_unstale":"2024-05-09T02:11:17.395265+0000","last_undegraded":"2024-05-09T02:11:17.395265+0000","last_fullsized":"2024-05-09T02:11:17.395265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:52:17.791737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093809+0000","last_change":"2024-05-09T02:07:45.155480+0000","last_active":"2024-05-09T02:11:17.093809+0000","last_peered":"2024-05-09T02:11:17.093809+0000","last_clean":"2024-05-09T02:11:17.093809+0000","last_became_active":"2024-05-09T02:07:45.155085+0000","last_became_peered":"2024-05-09T02:07:45.155085+0000","last_unstale":"2024-05-09T02:11:17.093809+0000","last_undegraded":"2024-05-09T02:11:17.093809+0000","last_fullsized":"2024-05-09T02:11:17.093809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:44.132259+0000","last_clean_scrub_stamp":"2024-05-09T02:07:44.132259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:05.547857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.395468+0000","last_change":"2024-05-09T02:07:48.194480+0000","last_active":"2024-05-09T02:11:17.395468+0000","last_peered":"2024-05-09T02:11:17.395468+0000","last_clean":"2024-05-09T02:11:17.395468+0000","last_became_active":"2024-05-09T02:07:48.194136+0000","last_became_peered":"2024-05-09T02:07:48.194136+0000","last_unstale":"2024-05-09T02:11:17.395468+0000","last_undegraded":"2024-05-09T02:11:17.395468+0000","last_fullsized":"2024-05-09T02:11:17.395468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:56:09.177762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092843+0000","last_change":"2024-05-09T02:07:48.195152+0000","last_active":"2024-05-09T02:11:17.092843+0000","last_peered":"2024-05-09T02:11:17.092843+0000","last_clean":"2024-05-09T02:11:17.092843+0000","last_became_active":"2024-05-09T02:07:48.194779+0000","last_became_peered":"2024-05-09T02:07:48.194779+0000","last_unstale":"2024-05-09T02:11:17.092843+0000","last_undegraded":"2024-05-09T02:11:17.092843+0000","last_fullsized":"2024-05-09T02:11:17.092843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:39.345452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281768+0000","last_change":"2024-05-09T02:08:25.322235+0000","last_active":"2024-05-09T02:11:17.281768+0000","last_peered":"2024-05-09T02:11:17.281768+0000","last_clean":"2024-05-09T02:11:17.281768+0000","last_became_active":"2024-05-09T02:08:25.321674+0000","last_became_peered":"2024-05-09T02:08:25.321674+0000","last_unstale":"2024-05-09T02:11:17.281768+0000","last_undegraded":"2024-05-09T02:11:17.281768+0000","last_fullsized":"2024-05-09T02:11:17.281768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:51:52.671669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.281697+0000","last_change":"2024-05-09T02:07:48.195710+0000","last_active":"2024-05-09T02:11:17.281697+0000","last_peered":"2024-05-09T02:11:17.281697+0000","last_clean":"2024-05-09T02:11:17.281697+0000","last_became_active":"2024-05-09T02:07:48.195332+0000","last_became_peered":"2024-05-09T02:07:48.195332+0000","last_unstale":"2024-05-09T02:11:17.281697+0000","last_undegraded":"2024-05-09T02:11:17.281697+0000","last_fullsized":"2024-05-09T02:11:17.281697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:12:17.904166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282255+0000","last_change":"2024-05-09T02:07:48.200791+0000","last_active":"2024-05-09T02:11:17.282255+0000","last_peered":"2024-05-09T02:11:17.282255+0000","last_clean":"2024-05-09T02:11:17.282255+0000","last_became_active":"2024-05-09T02:07:48.199916+0000","last_became_peered":"2024-05-09T02:07:48.199916+0000","last_unstale":"2024-05-09T02:11:17.282255+0000","last_undegraded":"2024-05-09T02:11:17.282255+0000","last_fullsized":"2024-05-09T02:11:17.282255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:32.692375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093770+0000","last_change":"2024-05-09T02:07:48.191662+0000","last_active":"2024-05-09T02:11:17.093770+0000","last_peered":"2024-05-09T02:11:17.093770+0000","last_clean":"2024-05-09T02:11:17.093770+0000","last_became_active":"2024-05-09T02:07:48.191336+0000","last_became_peered":"2024-05-09T02:07:48.191336+0000","last_unstale":"2024-05-09T02:11:17.093770+0000","last_undegraded":"2024-05-09T02:11:17.093770+0000","last_fullsized":"2024-05-09T02:11:17.093770+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:00.733611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092797+0000","last_change":"2024-05-09T02:07:48.195182+0000","last_active":"2024-05-09T02:11:17.092797+0000","last_peered":"2024-05-09T02:11:17.092797+0000","last_clean":"2024-05-09T02:11:17.092797+0000","last_became_active":"2024-05-09T02:07:48.194828+0000","last_became_peered":"2024-05-09T02:07:48.194828+0000","last_unstale":"2024-05-09T02:11:17.092797+0000","last_undegraded":"2024-05-09T02:11:17.092797+0000","last_fullsized":"2024-05-09T02:11:17.092797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:33.427409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282054+0000","last_change":"2024-05-09T02:07:48.199688+0000","last_active":"2024-05-09T02:11:17.282054+0000","last_peered":"2024-05-09T02:11:17.282054+0000","last_clean":"2024-05-09T02:11:17.282054+0000","last_became_active":"2024-05-09T02:07:48.199326+0000","last_became_peered":"2024-05-09T02:07:48.199326+0000","last_unstale":"2024-05-09T02:11:17.282054+0000","last_undegraded":"2024-05-09T02:11:17.282054+0000","last_fullsized":"2024-05-09T02:11:17.282054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:44:30.662738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.201606+0000","last_change":"2024-05-09T02:07:48.190727+0000","last_active":"2024-05-09T02:11:17.201606+0000","last_peered":"2024-05-09T02:11:17.201606+0000","last_clean":"2024-05-09T02:11:17.201606+0000","last_became_active":"2024-05-09T02:07:48.190397+0000","last_became_peered":"2024-05-09T02:07:48.190397+0000","last_unstale":"2024-05-09T02:11:17.201606+0000","last_undegraded":"2024-05-09T02:11:17.201606+0000","last_fullsized":"2024-05-09T02:11:17.201606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:42.010669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.208911+0000","last_change":"2024-05-09T02:07:48.200444+0000","last_active":"2024-05-09T02:11:17.208911+0000","last_peered":"2024-05-09T02:11:17.208911+0000","last_clean":"2024-05-09T02:11:17.208911+0000","last_became_active":"2024-05-09T02:07:48.200031+0000","last_became_peered":"2024-05-09T02:07:48.200031+0000","last_unstale":"2024-05-09T02:11:17.208911+0000","last_undegraded":"2024-05-09T02:11:17.208911+0000","last_fullsized":"2024-05-09T02:11:17.208911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:30.269792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.200630+0000","last_change":"2024-05-09T02:07:48.203717+0000","last_active":"2024-05-09T02:11:17.200630+0000","last_peered":"2024-05-09T02:11:17.200630+0000","last_clean":"2024-05-09T02:11:17.200630+0000","last_became_active":"2024-05-09T02:07:48.203347+0000","last_became_peered":"2024-05-09T02:07:48.203347+0000","last_unstale":"2024-05-09T02:11:17.200630+0000","last_undegraded":"2024-05-09T02:11:17.200630+0000","last_fullsized":"2024-05-09T02:11:17.200630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:36:17.391504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.282202+0000","last_change":"2024-05-09T02:08:25.323029+0000","last_active":"2024-05-09T02:11:17.282202+0000","last_peered":"2024-05-09T02:11:17.282202+0000","last_clean":"2024-05-09T02:11:17.282202+0000","last_became_active":"2024-05-09T02:08:25.322524+0000","last_became_peered":"2024-05-09T02:08:25.322524+0000","last_unstale":"2024-05-09T02:11:17.282202+0000","last_undegraded":"2024-05-09T02:11:17.282202+0000","last_fullsized":"2024-05-09T02:11:17.282202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:25.017511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.093749+0000","last_change":"2024-05-09T02:07:48.187802+0000","last_active":"2024-05-09T02:11:17.093749+0000","last_peered":"2024-05-09T02:11:17.093749+0000","last_clean":"2024-05-09T02:11:17.093749+0000","last_became_active":"2024-05-09T02:07:48.187478+0000","last_became_peered":"2024-05-09T02:07:48.187478+0000","last_unstale":"2024-05-09T02:11:17.093749+0000","last_undegraded":"2024-05-09T02:11:17.093749+0000","last_fullsized":"2024-05-09T02:11:17.093749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:12.981386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T02:11:17.092634+0000","last_change":"2024-05-09T02:07:48.192117+0000","last_active":"2024-05-09T02:11:17.092634+0000","last_peered":"2024-05-09T02:11:17.092634+0000","last_clean":"2024-05-09T02:11:17.092634+0000","last_became_active":"2024-05-09T02:07:48.191726+0000","last_became_peered":"2024-05-09T02:07:48.191726+0000","last_unstale":"2024-05-09T02:11:17.092634+0000","last_undegraded":"2024-05-09T02:11:17.092634+0000","last_fullsized":"2024-05-09T02:11:17.092634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T02:07:47.154696+0000","last_clean_scrub_stamp":"2024-05-09T02:07:47.154696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:07.735582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107049,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601872,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2039,"num_read_kb":1136,"num_write":374,"num_write_kb":1465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1805616,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678073,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":438649,"data_compressed":8666,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874300,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":1097854,"data_compressed":44349,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037828,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":946176,"data_stored":831800,"data_compressed":27812,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234056,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":831949,"data_compressed":27812,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430285,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":966656,"data_stored":1033459,"data_compressed":44349,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593811,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":438966,"data_compressed":8666,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":497868,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":921600,"data_stored":439523,"data_compressed":8666,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":497976,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149795,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":962708,"data_compressed":33968,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953565,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":429704,"data_compressed":5767,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790041,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":1163032,"data_compressed":50505,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35683,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135202,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202406,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T02:11:20.325 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:11:20.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:20 smithi059 ceph-mon[34781]: [09/May/2024:02:11:18] ENGINE Bus STARTING 2024-05-09T02:11:20.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:20 smithi059 ceph-mon[34781]: [09/May/2024:02:11:18] ENGINE Serving on https://172.21.15.19:7150 2024-05-09T02:11:20.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:20 smithi059 ceph-mon[34781]: [09/May/2024:02:11:19] ENGINE Serving on http://172.21.15.19:8765 2024-05-09T02:11:20.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:20 smithi059 ceph-mon[34781]: [09/May/2024:02:11:19] ENGINE Bus STARTED 2024-05-09T02:11:20.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:20 smithi059 ceph-mon[34781]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:11:20.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:20 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:20.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:20 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:20.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:20 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:20.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:20 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:20.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:20 smithi059 ceph-mon[34781]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-09T02:11:20.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:20 smithi196 ceph-mon[34718]: [09/May/2024:02:11:18] ENGINE Bus STARTING 2024-05-09T02:11:20.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:20 smithi196 ceph-mon[34718]: [09/May/2024:02:11:18] ENGINE Serving on https://172.21.15.19:7150 2024-05-09T02:11:20.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:20 smithi196 ceph-mon[34718]: [09/May/2024:02:11:19] ENGINE Serving on http://172.21.15.19:8765 2024-05-09T02:11:20.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:20 smithi196 ceph-mon[34718]: [09/May/2024:02:11:19] ENGINE Bus STARTED 2024-05-09T02:11:20.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:20 smithi196 ceph-mon[34718]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:11:20.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:20 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:20.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:20 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:20.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:20 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:20.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:20 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:20.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:20 smithi196 ceph-mon[34718]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-09T02:11:20.619 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:11:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:11:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:11:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:11:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:21 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:11:21.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:11:21.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:11:21.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:21 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:11:21.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:11:21.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T02:11:21.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:21.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:21.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:21 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:22.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:22 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi059 to 177.2M 2024-05-09T02:11:22.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:22 smithi019 ceph-mon[30151]: Unable to set osd_memory_target on smithi059 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-05-09T02:11:22.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:22 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-05-09T02:11:22.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:22 smithi019 ceph-mon[30151]: Unable to set osd_memory_target on smithi196 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-05-09T02:11:22.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:22 smithi019 ceph-mon[30151]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:11:22.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:22 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:22.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:22 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:22.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:22 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:22.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:22 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:22.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:22 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:22.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:22 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:22.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:22 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:11:22.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:22 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:11:22.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:22 smithi019 ceph-mon[30151]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-09T02:11:22.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.399+0000 7ff76c84f700 1 -- 172.21.15.19:0/3033339225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff768108f20 msgr2=0x7ff768109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:22.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.399+0000 7ff76c84f700 1 --2- 172.21.15.19:0/3033339225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff768108f20 0x7ff768109330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009920 tx=0x7ff75c009c30 comp rx=0 tx=0).stop 2024-05-09T02:11:22.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.399+0000 7ff76c84f700 1 -- 172.21.15.19:0/3033339225 shutdown_connections 2024-05-09T02:11:22.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.399+0000 7ff76c84f700 1 --2- 172.21.15.19:0/3033339225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff768109870 0x7ff7681111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:22.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.399+0000 7ff76c84f700 1 --2- 172.21.15.19:0/3033339225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7681045f0 0x7ff768104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:22.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.399+0000 7ff76c84f700 1 --2- 172.21.15.19:0/3033339225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff768108f20 0x7ff768109330 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:22.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.399+0000 7ff76c84f700 1 -- 172.21.15.19:0/3033339225 >> 172.21.15.19:0/3033339225 conn(0x7ff768100040 msgr2=0x7ff768102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:22.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.399+0000 7ff76c84f700 1 -- 172.21.15.19:0/3033339225 shutdown_connections 2024-05-09T02:11:22.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.399+0000 7ff76c84f700 1 -- 172.21.15.19:0/3033339225 wait complete. 2024-05-09T02:11:22.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.400+0000 7ff76c84f700 1 Processor -- start 2024-05-09T02:11:22.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.400+0000 7ff76c84f700 1 -- start start 2024-05-09T02:11:22.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff76c84f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7681045f0 0x7ff76819af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:22.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff76c84f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff768108f20 0x7ff76819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:22.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff76c84f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff768109870 0x7ff76819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:22.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff76c84f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff768114480 con 0x7ff768108f20 2024-05-09T02:11:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff76c84f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff768114300 con 0x7ff768109870 2024-05-09T02:11:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff76c84f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff768114600 con 0x7ff7681045f0 2024-05-09T02:11:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff765d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff768108f20 0x7ff76819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff765d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff768108f20 0x7ff76819b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44530/0 (socket says 172.21.15.19:44530) 2024-05-09T02:11:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff76659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7681045f0 0x7ff76819af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff765d9b700 1 -- 172.21.15.19:0/4234163262 learned_addr learned my addr 172.21.15.19:0/4234163262 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff766d9d700 1 --2- 172.21.15.19:0/4234163262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff768109870 0x7ff76819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff76659c700 1 -- 172.21.15.19:0/4234163262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff768109870 msgr2=0x7ff76819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff76659c700 1 --2- 172.21.15.19:0/4234163262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff768109870 0x7ff76819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff76659c700 1 -- 172.21.15.19:0/4234163262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff768108f20 msgr2=0x7ff76819b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff76659c700 1 --2- 172.21.15.19:0/4234163262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff768108f20 0x7ff76819b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff76659c700 1 -- 172.21.15.19:0/4234163262 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76819fe20 con 0x7ff7681045f0 2024-05-09T02:11:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff766d9d700 1 --2- 172.21.15.19:0/4234163262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff768109870 0x7ff76819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff765d9b700 1 --2- 172.21.15.19:0/4234163262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff768108f20 0x7ff76819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.401+0000 7ff76659c700 1 --2- 172.21.15.19:0/4234163262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7681045f0 0x7ff76819af50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff75800eb90 tx=0x7ff75800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.402+0000 7ff7577fe700 1 -- 172.21.15.19:0/4234163262 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75800cc90 con 0x7ff7681045f0 2024-05-09T02:11:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.402+0000 7ff7577fe700 1 -- 172.21.15.19:0/4234163262 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff75800ce30 con 0x7ff7681045f0 2024-05-09T02:11:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.402+0000 7ff7577fe700 1 -- 172.21.15.19:0/4234163262 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758010430 con 0x7ff7681045f0 2024-05-09T02:11:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.402+0000 7ff76c84f700 1 -- 172.21.15.19:0/4234163262 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7681a0110 con 0x7ff7681045f0 2024-05-09T02:11:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.402+0000 7ff76c84f700 1 -- 172.21.15.19:0/4234163262 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7681a7a30 con 0x7ff7681045f0 2024-05-09T02:11:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.403+0000 7ff76c84f700 1 -- 172.21.15.19:0/4234163262 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7680512c0 con 0x7ff7681045f0 2024-05-09T02:11:22.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.404+0000 7ff7577fe700 1 -- 172.21.15.19:0/4234163262 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7ff7580469c0 con 0x7ff7681045f0 2024-05-09T02:11:22.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.405+0000 7ff7577fe700 1 --2- 172.21.15.19:0/4234163262 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff7500b2a80 0x7ff7500b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:22.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.406+0000 7ff765d9b700 1 --2- 172.21.15.19:0/4234163262 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff7500b2a80 0x7ff7500b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:22.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.406+0000 7ff7577fe700 1 -- 172.21.15.19:0/4234163262 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7ff7580e2af0 con 0x7ff7681045f0 2024-05-09T02:11:22.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.406+0000 7ff765d9b700 1 --2- 172.21.15.19:0/4234163262 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff7500b2a80 0x7ff7500b4f90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff75c005f40 tx=0x7ff75c019040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:22.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.409+0000 7ff7577fe700 1 -- 172.21.15.19:0/4234163262 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff7580aa360 con 0x7ff7681045f0 2024-05-09T02:11:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:22 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi059 to 177.2M 2024-05-09T02:11:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:22 smithi059 ceph-mon[34781]: Unable to set osd_memory_target on smithi059 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-05-09T02:11:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:22 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-05-09T02:11:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:22 smithi059 ceph-mon[34781]: Unable to set osd_memory_target on smithi196 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-05-09T02:11:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:22 smithi059 ceph-mon[34781]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:11:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:22 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:22.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:22 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:22.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:22 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:22.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:22 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:22.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:22 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:22.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:22 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:22.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:22 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:11:22.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:22 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:11:22.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:22 smithi059 ceph-mon[34781]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-09T02:11:22.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:22 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi059 to 177.2M 2024-05-09T02:11:22.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:22 smithi196 ceph-mon[34718]: Unable to set osd_memory_target on smithi059 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-05-09T02:11:22.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:22 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-05-09T02:11:22.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:22 smithi196 ceph-mon[34718]: Unable to set osd_memory_target on smithi196 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-05-09T02:11:22.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:22 smithi196 ceph-mon[34718]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:11:22.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:22 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:22.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:22 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:22.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:22 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:22.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:22 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:22.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:22 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:22.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:22 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:11:22.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:22 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:11:22.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:22 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:11:22.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:22 smithi196 ceph-mon[34718]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-09T02:11:22.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.689+0000 7ff76c84f700 1 -- 172.21.15.19:0/4234163262 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff76819c2b0 con 0x7ff7681045f0 2024-05-09T02:11:22.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.690+0000 7ff7577fe700 1 -- 172.21.15.19:0/4234163262 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11917 (secure 0 0 0) 0x7ff7580a9c60 con 0x7ff7681045f0 2024-05-09T02:11:22.692 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:22.693 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:11:14.517298+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24463,24484,24473,24455,24457]},"id":1}]} 2024-05-09T02:11:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.692+0000 7ff76c84f700 1 -- 172.21.15.19:0/4234163262 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff7500b2a80 msgr2=0x7ff7500b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.692+0000 7ff76c84f700 1 --2- 172.21.15.19:0/4234163262 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff7500b2a80 0x7ff7500b4f90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff75c005f40 tx=0x7ff75c019040 comp rx=0 tx=0).stop 2024-05-09T02:11:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.692+0000 7ff76c84f700 1 -- 172.21.15.19:0/4234163262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7681045f0 msgr2=0x7ff76819af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.692+0000 7ff76c84f700 1 --2- 172.21.15.19:0/4234163262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7681045f0 0x7ff76819af50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff75800eb90 tx=0x7ff75800eea0 comp rx=0 tx=0).stop 2024-05-09T02:11:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.692+0000 7ff76c84f700 1 -- 172.21.15.19:0/4234163262 shutdown_connections 2024-05-09T02:11:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.692+0000 7ff76c84f700 1 --2- 172.21.15.19:0/4234163262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff768109870 0x7ff76819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.692+0000 7ff76c84f700 1 --2- 172.21.15.19:0/4234163262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7681045f0 0x7ff76819af50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.692+0000 7ff76c84f700 1 --2- 172.21.15.19:0/4234163262 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff7500b2a80 0x7ff7500b4f90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.692+0000 7ff76c84f700 1 --2- 172.21.15.19:0/4234163262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff768108f20 0x7ff76819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.692+0000 7ff76c84f700 1 -- 172.21.15.19:0/4234163262 >> 172.21.15.19:0/4234163262 conn(0x7ff768100040 msgr2=0x7ff768102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.692+0000 7ff76c84f700 1 -- 172.21.15.19:0/4234163262 shutdown_connections 2024-05-09T02:11:22.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:22.692+0000 7ff76c84f700 1 -- 172.21.15.19:0/4234163262 wait complete. 2024-05-09T02:11:22.696 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-05-09T02:11:23.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-09T02:11:23.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: Unable to set osd_memory_target on smithi019 to 185876992: error parsing value: Value '185876992' is below minimum 939524096 2024-05-09T02:11:23.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-09T02:11:23.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-09T02:11:23.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-09T02:11:23.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: Updating smithi019:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:11:23.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: Updating smithi059:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:11:23.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: Updating smithi196:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:11:23.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:11:23.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:11:23.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:11:23.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: Updating smithi059:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T02:11:23.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: Updating smithi196:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T02:11:23.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: Updating smithi019:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T02:11:23.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:11:23.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:11:23.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:11:23.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:23 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/4234163262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:11:23.385 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T02:11:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-09T02:11:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: Unable to set osd_memory_target on smithi019 to 185876992: error parsing value: Value '185876992' is below minimum 939524096 2024-05-09T02:11:23.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: Updating smithi019:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: Updating smithi059:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: Updating smithi196:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: Updating smithi059:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: Updating smithi196:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: Updating smithi019:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.468 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.468 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.468 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:11:23.468 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:11:23.468 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:11:23.468 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.468 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:23 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/4234163262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:11:23.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-09T02:11:23.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: Unable to set osd_memory_target on smithi019 to 185876992: error parsing value: Value '185876992' is below minimum 939524096 2024-05-09T02:11:23.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-09T02:11:23.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-09T02:11:23.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-09T02:11:23.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: Updating smithi019:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:11:23.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: Updating smithi059:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:11:23.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: Updating smithi196:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T02:11:23.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:11:23.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:11:23.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-05-09T02:11:23.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: Updating smithi059:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T02:11:23.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: Updating smithi196:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T02:11:23.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: Updating smithi019:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T02:11:23.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:11:23.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:11:23.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:11:23.546 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:23.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:23 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/4234163262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:11:23.657 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:11:24.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:24 smithi019 ceph-mon[30151]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:11:24.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:24 smithi059 ceph-mon[34781]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:11:24.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:24 smithi196 ceph-mon[34718]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:11:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.398+0000 7efc430e3700 1 -- 172.21.15.19:0/4075410622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3c107df0 msgr2=0x7efc3c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.398+0000 7efc430e3700 1 --2- 172.21.15.19:0/4075410622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3c107df0 0x7efc3c108200 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efc240099c0 tx=0x7efc24009cd0 comp rx=0 tx=0).stop 2024-05-09T02:11:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.398+0000 7efc430e3700 1 -- 172.21.15.19:0/4075410622 shutdown_connections 2024-05-09T02:11:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.398+0000 7efc430e3700 1 --2- 172.21.15.19:0/4075410622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c108740 0x7efc3c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.398+0000 7efc430e3700 1 --2- 172.21.15.19:0/4075410622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc3c1034c0 0x7efc3c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.398+0000 7efc430e3700 1 --2- 172.21.15.19:0/4075410622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3c107df0 0x7efc3c108200 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.398+0000 7efc430e3700 1 -- 172.21.15.19:0/4075410622 >> 172.21.15.19:0/4075410622 conn(0x7efc3c0fedc0 msgr2=0x7efc3c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.398+0000 7efc430e3700 1 -- 172.21.15.19:0/4075410622 shutdown_connections 2024-05-09T02:11:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.398+0000 7efc430e3700 1 -- 172.21.15.19:0/4075410622 wait complete. 2024-05-09T02:11:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.398+0000 7efc430e3700 1 Processor -- start 2024-05-09T02:11:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.398+0000 7efc430e3700 1 -- start start 2024-05-09T02:11:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc430e3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc3c1034c0 0x7efc3c0ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc430e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3c107df0 0x7efc3c0ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc430e3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c108740 0x7efc3c0ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc430e3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c113350 con 0x7efc3c107df0 2024-05-09T02:11:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc430e3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c1131d0 con 0x7efc3c108740 2024-05-09T02:11:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc430e3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c1134d0 con 0x7efc3c1034c0 2024-05-09T02:11:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3c107df0 0x7efc3c0ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3c107df0 0x7efc3c0ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44544/0 (socket says 172.21.15.19:44544) 2024-05-09T02:11:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc3bfff700 1 -- 172.21.15.19:0/2314723698 learned_addr learned my addr 172.21.15.19:0/2314723698 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc41680700 1 --2- 172.21.15.19:0/2314723698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c108740 0x7efc3c0ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc40e7f700 1 --2- 172.21.15.19:0/2314723698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc3c1034c0 0x7efc3c0ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc3bfff700 1 -- 172.21.15.19:0/2314723698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc3c1034c0 msgr2=0x7efc3c0ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc3bfff700 1 --2- 172.21.15.19:0/2314723698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc3c1034c0 0x7efc3c0ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc3bfff700 1 -- 172.21.15.19:0/2314723698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c108740 msgr2=0x7efc3c0ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc3bfff700 1 --2- 172.21.15.19:0/2314723698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c108740 0x7efc3c0ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.399+0000 7efc3bfff700 1 -- 172.21.15.19:0/2314723698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c1a9120 con 0x7efc3c107df0 2024-05-09T02:11:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.400+0000 7efc3bfff700 1 --2- 172.21.15.19:0/2314723698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3c107df0 0x7efc3c0ff7e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efc240098e0 tx=0x7efc240057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.400+0000 7efc39ffb700 1 -- 172.21.15.19:0/2314723698 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc24027070 con 0x7efc3c107df0 2024-05-09T02:11:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.400+0000 7efc430e3700 1 -- 172.21.15.19:0/2314723698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3c1a93b0 con 0x7efc3c107df0 2024-05-09T02:11:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.400+0000 7efc39ffb700 1 -- 172.21.15.19:0/2314723698 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc24003f20 con 0x7efc3c107df0 2024-05-09T02:11:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.400+0000 7efc39ffb700 1 -- 172.21.15.19:0/2314723698 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc24022d70 con 0x7efc3c107df0 2024-05-09T02:11:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.400+0000 7efc430e3700 1 -- 172.21.15.19:0/2314723698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3c1a9800 con 0x7efc3c107df0 2024-05-09T02:11:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.401+0000 7efc430e3700 1 -- 172.21.15.19:0/2314723698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc3c0512c0 con 0x7efc3c107df0 2024-05-09T02:11:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.403+0000 7efc39ffb700 1 -- 172.21.15.19:0/2314723698 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7efc2402a070 con 0x7efc3c107df0 2024-05-09T02:11:25.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.404+0000 7efc39ffb700 1 --2- 172.21.15.19:0/2314723698 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc280b2a80 0x7efc280b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:25.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.404+0000 7efc40e7f700 1 --2- 172.21.15.19:0/2314723698 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc280b2a80 0x7efc280b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:25.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.404+0000 7efc39ffb700 1 -- 172.21.15.19:0/2314723698 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7efc240e3e30 con 0x7efc3c107df0 2024-05-09T02:11:25.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.404+0000 7efc40e7f700 1 --2- 172.21.15.19:0/2314723698 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc280b2a80 0x7efc280b4f90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00ac50 tx=0x7efc2c00a380 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.408+0000 7efc39ffb700 1 -- 172.21.15.19:0/2314723698 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc24031080 con 0x7efc3c107df0 2024-05-09T02:11:25.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.622+0000 7efc430e3700 1 -- 172.21.15.19:0/2314723698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc3c02e7b0 con 0x7efc3c107df0 2024-05-09T02:11:25.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.624+0000 7efc39ffb700 1 -- 172.21.15.19:0/2314723698 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+23072 (secure 0 0 0) 0x7efc24013070 con 0x7efc3c107df0 2024-05-09T02:11:25.627 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:25.627 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":82,"fsid":"a0611f3e-0da7-11ef-bc97-c7b262605968","created":"2024-05-09T01:59:27.591343+0000","modified":"2024-05-09T02:11:17.084882+0000","last_up_change":"2024-05-09T02:06:45.565065+0000","last_in_change":"2024-05-09T02:06:24.967186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T02:02:50.018959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T02:07:43.283868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T02:07:47.110614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"996c6a3d-3e3a-451c-8789-36006e0b3dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6803","nonce":3002373576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6805","nonce":3002373576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6809","nonce":3002373576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6807","nonce":3002373576}]},"public_addr":"172.21.15.19:6803/3002373576","cluster_addr":"172.21.15.19:6805/3002373576","heartbeat_back_addr":"172.21.15.19:6809/3002373576","heartbeat_front_addr":"172.21.15.19:6807/3002373576","state":["exists","up"]},{"osd":1,"uuid":"1ac68d09-1198-4e81-86c7-e37f7c41aa5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6801","nonce":827458353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6803","nonce":827458353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6807","nonce":827458353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6805","nonce":827458353}]},"public_addr":"172.21.15.59:6801/827458353","cluster_addr":"172.21.15.59:6803/827458353","heartbeat_back_addr":"172.21.15.59:6807/827458353","heartbeat_front_addr":"172.21.15.59:6805/827458353","state":["exists","up"]},{"osd":2,"uuid":"818a0de1-04e6-471d-83df-a68833011b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6801","nonce":1659504639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6803","nonce":1659504639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6807","nonce":1659504639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6805","nonce":1659504639}]},"public_addr":"172.21.15.196:6801/1659504639","cluster_addr":"172.21.15.196:6803/1659504639","heartbeat_back_addr":"172.21.15.196:6807/1659504639","heartbeat_front_addr":"172.21.15.196:6805/1659504639","state":["exists","up"]},{"osd":3,"uuid":"7b204854-86b8-454d-aee0-8bb5775ce828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6811","nonce":2707671599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6813","nonce":2707671599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6817","nonce":2707671599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6815","nonce":2707671599}]},"public_addr":"172.21.15.19:6811/2707671599","cluster_addr":"172.21.15.19:6813/2707671599","heartbeat_back_addr":"172.21.15.19:6817/2707671599","heartbeat_front_addr":"172.21.15.19:6815/2707671599","state":["exists","up"]},{"osd":4,"uuid":"091667f1-e887-4a94-8723-6f85b676fef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6809","nonce":223037330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6811","nonce":223037330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6815","nonce":223037330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6813","nonce":223037330}]},"public_addr":"172.21.15.59:6809/223037330","cluster_addr":"172.21.15.59:6811/223037330","heartbeat_back_addr":"172.21.15.59:6815/223037330","heartbeat_front_addr":"172.21.15.59:6813/223037330","state":["exists","up"]},{"osd":5,"uuid":"2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6809","nonce":2516161898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6811","nonce":2516161898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6815","nonce":2516161898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6813","nonce":2516161898}]},"public_addr":"172.21.15.196:6809/2516161898","cluster_addr":"172.21.15.196:6811/2516161898","heartbeat_back_addr":"172.21.15.196:6815/2516161898","heartbeat_front_addr":"172.21.15.196:6813/2516161898","state":["exists","up"]},{"osd":6,"uuid":"4e352851-1279-475f-ba8b-eaf9ca73df74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6819","nonce":2150451637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6821","nonce":2150451637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6825","nonce":2150451637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6823","nonce":2150451637}]},"public_addr":"172.21.15.19:6819/2150451637","cluster_addr":"172.21.15.19:6821/2150451637","heartbeat_back_addr":"172.21.15.19:6825/2150451637","heartbeat_front_addr":"172.21.15.19:6823/2150451637","state":["exists","up"]},{"osd":7,"uuid":"83f53f59-9a60-4d87-9a8a-67e5c6f57004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6817","nonce":967902053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6819","nonce":967902053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6823","nonce":967902053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6821","nonce":967902053}]},"public_addr":"172.21.15.59:6817/967902053","cluster_addr":"172.21.15.59:6819/967902053","heartbeat_back_addr":"172.21.15.59:6823/967902053","heartbeat_front_addr":"172.21.15.59:6821/967902053","state":["exists","up"]},{"osd":8,"uuid":"a8f1bfb5-1cc8-4a65-8396-239bd781e1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6817","nonce":2946064259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6819","nonce":2946064259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6823","nonce":2946064259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6821","nonce":2946064259}]},"public_addr":"172.21.15.196:6817/2946064259","cluster_addr":"172.21.15.196:6819/2946064259","heartbeat_back_addr":"172.21.15.196:6823/2946064259","heartbeat_front_addr":"172.21.15.196:6821/2946064259","state":["exists","up"]},{"osd":9,"uuid":"73c58dc3-fcb0-4440-92b3-bf41a2e58fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6827","nonce":1735017886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6829","nonce":1735017886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6833","nonce":1735017886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6831","nonce":1735017886}]},"public_addr":"172.21.15.19:6827/1735017886","cluster_addr":"172.21.15.19:6829/1735017886","heartbeat_back_addr":"172.21.15.19:6833/1735017886","heartbeat_front_addr":"172.21.15.19:6831/1735017886","state":["exists","up"]},{"osd":10,"uuid":"8325282f-7325-4880-ab7d-b972717be9dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6825","nonce":2932899995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6827","nonce":2932899995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6831","nonce":2932899995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6829","nonce":2932899995}]},"public_addr":"172.21.15.59:6825/2932899995","cluster_addr":"172.21.15.59:6827/2932899995","heartbeat_back_addr":"172.21.15.59:6831/2932899995","heartbeat_front_addr":"172.21.15.59:6829/2932899995","state":["exists","up"]},{"osd":11,"uuid":"8b19228c-b286-43c8-a408-169782f13075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6825","nonce":2608023001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6827","nonce":2608023001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6831","nonce":2608023001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6829","nonce":2608023001}]},"public_addr":"172.21.15.196:6825/2608023001","cluster_addr":"172.21.15.196:6827/2608023001","heartbeat_back_addr":"172.21.15.196:6831/2608023001","heartbeat_front_addr":"172.21.15.196:6829/2608023001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:01:59.457748+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:23.640666+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:47.768243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:13.016746+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:39.060898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:04.508313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:31.366217+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:57.579758+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:23.177396+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:50.966399+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:16.838666+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:43.959387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3660764754":"2024-05-10T02:11:17.084815+0000","172.21.15.19:0/3573608809":"2024-05-10T02:11:17.084815+0000","172.21.15.19:0/3291838961":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1024133854":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1176455101":"2024-05-10T02:00:23.920230+0000","172.21.15.19:6801/2101052000":"2024-05-10T02:00:23.920230+0000","172.21.15.19:6801/972543870":"2024-05-10T02:11:17.084815+0000","172.21.15.19:0/4018987404":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/3079777324":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/972543870":"2024-05-10T02:11:17.084815+0000","172.21.15.19:6800/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/557070897":"2024-05-10T02:11:17.084815+0000","172.21.15.19:0/758334732":"2024-05-10T02:11:17.084815+0000","172.21.15.19:6801/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/179703528":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/2101052000":"2024-05-10T02:00:23.920230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T02:11:25.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.626+0000 7efc430e3700 1 -- 172.21.15.19:0/2314723698 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc280b2a80 msgr2=0x7efc280b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:25.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.626+0000 7efc430e3700 1 --2- 172.21.15.19:0/2314723698 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc280b2a80 0x7efc280b4f90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00ac50 tx=0x7efc2c00a380 comp rx=0 tx=0).stop 2024-05-09T02:11:25.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.626+0000 7efc430e3700 1 -- 172.21.15.19:0/2314723698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3c107df0 msgr2=0x7efc3c0ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:25.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.626+0000 7efc430e3700 1 --2- 172.21.15.19:0/2314723698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3c107df0 0x7efc3c0ff7e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efc240098e0 tx=0x7efc240057c0 comp rx=0 tx=0).stop 2024-05-09T02:11:25.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.626+0000 7efc430e3700 1 -- 172.21.15.19:0/2314723698 shutdown_connections 2024-05-09T02:11:25.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.626+0000 7efc430e3700 1 --2- 172.21.15.19:0/2314723698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c108740 0x7efc3c0ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:25.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.626+0000 7efc430e3700 1 --2- 172.21.15.19:0/2314723698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc3c1034c0 0x7efc3c0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:25.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.626+0000 7efc430e3700 1 --2- 172.21.15.19:0/2314723698 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc280b2a80 0x7efc280b4f90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.627+0000 7efc430e3700 1 --2- 172.21.15.19:0/2314723698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3c107df0 0x7efc3c0ff7e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.627+0000 7efc430e3700 1 -- 172.21.15.19:0/2314723698 >> 172.21.15.19:0/2314723698 conn(0x7efc3c0fedc0 msgr2=0x7efc3c06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.627+0000 7efc430e3700 1 -- 172.21.15.19:0/2314723698 shutdown_connections 2024-05-09T02:11:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:25.627+0000 7efc430e3700 1 -- 172.21.15.19:0/2314723698 wait complete. 2024-05-09T02:11:26.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:26 smithi019 ceph-mon[30151]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 293 B/s wr, 18 op/s 2024-05-09T02:11:26.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:26 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2314723698' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:11:26.330 DEBUG:tasks.fwd_scrub:Yielding 2024-05-09T02:11:26.331 INFO:teuthology.run_tasks:Running task exec... 2024-05-09T02:11:26.341 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T02:11:26.342 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T02:11:26.342 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-09T02:11:26.345 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-09T02:11:26.345 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T02:11:26.345 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T02:11:26.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:26 smithi059 ceph-mon[34781]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 293 B/s wr, 18 op/s 2024-05-09T02:11:26.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:26 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2314723698' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:11:26.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.470+0000 7fcd8e376640 1 -- 172.21.15.19:0/3437207987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd88108bb0 msgr2=0x7fcd88102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.470+0000 7fcd8e376640 1 --2- 172.21.15.19:0/3437207987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd88108bb0 0x7fcd88102750 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcd78009900 tx=0x7fcd7802a410 comp rx=0 tx=0).stop 2024-05-09T02:11:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.470+0000 7fcd8e376640 1 -- 172.21.15.19:0/3437207987 shutdown_connections 2024-05-09T02:11:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.470+0000 7fcd8e376640 1 --2- 172.21.15.19:0/3437207987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd88103300 0x7fcd8810f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.470+0000 7fcd8e376640 1 --2- 172.21.15.19:0/3437207987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd88109050 0x7fcd88102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.470+0000 7fcd8e376640 1 --2- 172.21.15.19:0/3437207987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd88108bb0 0x7fcd88102750 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.470+0000 7fcd8e376640 1 -- 172.21.15.19:0/3437207987 >> 172.21.15.19:0/3437207987 conn(0x7fcd88078280 msgr2=0x7fcd880786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.470+0000 7fcd8e376640 1 -- 172.21.15.19:0/3437207987 shutdown_connections 2024-05-09T02:11:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.470+0000 7fcd8e376640 1 -- 172.21.15.19:0/3437207987 wait complete. 2024-05-09T02:11:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.471+0000 7fcd8e376640 1 Processor -- start 2024-05-09T02:11:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.471+0000 7fcd8e376640 1 -- start start 2024-05-09T02:11:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd8e376640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd88103300 0x7fcd8819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd8e376640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd88108bb0 0x7fcd8819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd8e376640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd88109050 0x7fcd8819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd8e376640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd88112da0 con 0x7fcd88109050 2024-05-09T02:11:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd8e376640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcd88112c20 con 0x7fcd88108bb0 2024-05-09T02:11:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd8e376640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd88112f20 con 0x7fcd88103300 2024-05-09T02:11:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd8c913640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd88109050 0x7fcd8819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd8c913640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd88109050 0x7fcd8819c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44572/0 (socket says 172.21.15.19:44572) 2024-05-09T02:11:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd8c913640 1 -- 172.21.15.19:0/576533012 learned_addr learned my addr 172.21.15.19:0/576533012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd877fe640 1 --2- 172.21.15.19:0/576533012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd88108bb0 0x7fcd8819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd87fff640 1 --2- 172.21.15.19:0/576533012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd88103300 0x7fcd8819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:26.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd8c913640 1 -- 172.21.15.19:0/576533012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd88103300 msgr2=0x7fcd8819b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:26.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd8c913640 1 --2- 172.21.15.19:0/576533012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd88103300 0x7fcd8819b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:26.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd8c913640 1 -- 172.21.15.19:0/576533012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd88108bb0 msgr2=0x7fcd8819bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:26.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd8c913640 1 --2- 172.21.15.19:0/576533012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd88108bb0 0x7fcd8819bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:26.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd8c913640 1 -- 172.21.15.19:0/576533012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd881a6a80 con 0x7fcd88109050 2024-05-09T02:11:26.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.472+0000 7fcd8c913640 1 --2- 172.21.15.19:0/576533012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd88109050 0x7fcd8819c1e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000eaa0 tx=0x7fcd8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:26.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.473+0000 7fcd857fa640 1 -- 172.21.15.19:0/576533012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8000cdb0 con 0x7fcd88109050 2024-05-09T02:11:26.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.473+0000 7fcd857fa640 1 -- 172.21.15.19:0/576533012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd80005020 con 0x7fcd88109050 2024-05-09T02:11:26.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.473+0000 7fcd857fa640 1 -- 172.21.15.19:0/576533012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd800116d0 con 0x7fcd88109050 2024-05-09T02:11:26.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.473+0000 7fcd8e376640 1 -- 172.21.15.19:0/576533012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd881a6d70 con 0x7fcd88109050 2024-05-09T02:11:26.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.473+0000 7fcd8e376640 1 -- 172.21.15.19:0/576533012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd881a71b0 con 0x7fcd88109050 2024-05-09T02:11:26.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.474+0000 7fcd8e376640 1 -- 172.21.15.19:0/576533012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd88050870 con 0x7fcd88109050 2024-05-09T02:11:26.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.476+0000 7fcd857fa640 1 -- 172.21.15.19:0/576533012 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fcd800051d0 con 0x7fcd88109050 2024-05-09T02:11:26.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.476+0000 7fcd857fa640 1 --2- 172.21.15.19:0/576533012 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcd740b29e0 0x7fcd740b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:26.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.476+0000 7fcd857fa640 1 -- 172.21.15.19:0/576533012 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fcd800e2d90 con 0x7fcd88109050 2024-05-09T02:11:26.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.476+0000 7fcd87fff640 1 --2- 172.21.15.19:0/576533012 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcd740b29e0 0x7fcd740b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:26.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.477+0000 7fcd87fff640 1 --2- 172.21.15.19:0/576533012 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcd740b29e0 0x7fcd740b4ef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcd78000c00 tx=0x7fcd78031400 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:26.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.480+0000 7fcd857fa640 1 -- 172.21.15.19:0/576533012 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcd80016030 con 0x7fcd88109050 2024-05-09T02:11:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:26 smithi196 ceph-mon[34718]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 293 B/s wr, 18 op/s 2024-05-09T02:11:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:26 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2314723698' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:11:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:11:26.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.690+0000 7fcd8e376640 1 -- 172.21.15.19:0/576533012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fcd8806b970 con 0x7fcd88109050 2024-05-09T02:11:26.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.695+0000 7fcd857fa640 1 -- 172.21.15.19:0/576533012 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v32)=0 v32) ==== 129+0+0 (secure 0 0 0) 0x7fcd800aa5a0 con 0x7fcd88109050 2024-05-09T02:11:26.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.697+0000 7fcd8e376640 1 -- 172.21.15.19:0/576533012 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcd740b29e0 msgr2=0x7fcd740b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:26.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.697+0000 7fcd8e376640 1 --2- 172.21.15.19:0/576533012 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcd740b29e0 0x7fcd740b4ef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcd78000c00 tx=0x7fcd78031400 comp rx=0 tx=0).stop 2024-05-09T02:11:26.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.697+0000 7fcd8e376640 1 -- 172.21.15.19:0/576533012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd88109050 msgr2=0x7fcd8819c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:26.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.697+0000 7fcd8e376640 1 --2- 172.21.15.19:0/576533012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd88109050 0x7fcd8819c1e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000eaa0 tx=0x7fcd8000ef60 comp rx=0 tx=0).stop 2024-05-09T02:11:26.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.697+0000 7fcd8e376640 1 -- 172.21.15.19:0/576533012 shutdown_connections 2024-05-09T02:11:26.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.697+0000 7fcd8e376640 1 --2- 172.21.15.19:0/576533012 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcd740b29e0 0x7fcd740b4ef0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:26.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.697+0000 7fcd8e376640 1 --2- 172.21.15.19:0/576533012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd88109050 0x7fcd8819c1e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:26.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.697+0000 7fcd8e376640 1 --2- 172.21.15.19:0/576533012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd88108bb0 0x7fcd8819bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:26.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.697+0000 7fcd8e376640 1 --2- 172.21.15.19:0/576533012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd88103300 0x7fcd8819b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:26.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.697+0000 7fcd8e376640 1 -- 172.21.15.19:0/576533012 >> 172.21.15.19:0/576533012 conn(0x7fcd88078280 msgr2=0x7fcd88104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:26.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.698+0000 7fcd8e376640 1 -- 172.21.15.19:0/576533012 shutdown_connections 2024-05-09T02:11:26.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.698+0000 7fcd8e376640 1 -- 172.21.15.19:0/576533012 wait complete. 2024-05-09T02:11:26.712 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-09T02:11:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.829+0000 7f5e35594640 1 -- 172.21.15.19:0/2162537358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e30109030 msgr2=0x7f5e30102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.829+0000 7f5e35594640 1 --2- 172.21.15.19:0/2162537358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e30109030 0x7f5e30102c70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009900 tx=0x7f5e1802a410 comp rx=0 tx=0).stop 2024-05-09T02:11:26.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.830+0000 7f5e35594640 1 -- 172.21.15.19:0/2162537358 shutdown_connections 2024-05-09T02:11:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.830+0000 7f5e35594640 1 --2- 172.21.15.19:0/2162537358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e301032e0 0x7f5e3010f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.830+0000 7f5e35594640 1 --2- 172.21.15.19:0/2162537358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e30109030 0x7f5e30102c70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.830+0000 7f5e35594640 1 --2- 172.21.15.19:0/2162537358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e30108b90 0x7f5e30102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.830+0000 7f5e35594640 1 -- 172.21.15.19:0/2162537358 >> 172.21.15.19:0/2162537358 conn(0x7f5e300fe390 msgr2=0x7f5e301007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.830+0000 7f5e35594640 1 -- 172.21.15.19:0/2162537358 shutdown_connections 2024-05-09T02:11:26.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.830+0000 7f5e35594640 1 -- 172.21.15.19:0/2162537358 wait complete. 2024-05-09T02:11:26.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.830+0000 7f5e35594640 1 Processor -- start 2024-05-09T02:11:26.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.830+0000 7f5e35594640 1 -- start start 2024-05-09T02:11:26.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e35594640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e301032e0 0x7f5e301a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:26.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e35594640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e30108b90 0x7f5e301a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:26.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e35594640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e30109030 0x7f5e3019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:26.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e35594640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e30113e90 con 0x7f5e30109030 2024-05-09T02:11:26.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e35594640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5e30113d10 con 0x7f5e301032e0 2024-05-09T02:11:26.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e35594640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5e30114010 con 0x7f5e30108b90 2024-05-09T02:11:26.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e2f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e30109030 0x7f5e3019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:26.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e2f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e30109030 0x7f5e3019a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58142/0 (socket says 172.21.15.19:58142) 2024-05-09T02:11:26.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e2f7fe640 1 -- 172.21.15.19:0/3709319472 learned_addr learned my addr 172.21.15.19:0/3709319472 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:26.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e2effd640 1 --2- 172.21.15.19:0/3709319472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e301032e0 0x7f5e301a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:26.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e27fff640 1 --2- 172.21.15.19:0/3709319472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e30108b90 0x7f5e301a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:26.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e2f7fe640 1 -- 172.21.15.19:0/3709319472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e30108b90 msgr2=0x7f5e301a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:26.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e2f7fe640 1 --2- 172.21.15.19:0/3709319472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e30108b90 0x7f5e301a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:26.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e2f7fe640 1 -- 172.21.15.19:0/3709319472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e301032e0 msgr2=0x7f5e301a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:26.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e2f7fe640 1 --2- 172.21.15.19:0/3709319472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e301032e0 0x7f5e301a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:26.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e2f7fe640 1 -- 172.21.15.19:0/3709319472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3019ae90 con 0x7f5e30109030 2024-05-09T02:11:26.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e2f7fe640 1 --2- 172.21.15.19:0/3709319472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e30109030 0x7f5e3019a7b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800eaa0 tx=0x7f5e2800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:26.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e2cff9640 1 -- 172.21.15.19:0/3709319472 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2800cdb0 con 0x7f5e30109030 2024-05-09T02:11:26.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.831+0000 7f5e2cff9640 1 -- 172.21.15.19:0/3709319472 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e28005020 con 0x7f5e30109030 2024-05-09T02:11:26.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.832+0000 7f5e35594640 1 -- 172.21.15.19:0/3709319472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3019b180 con 0x7f5e30109030 2024-05-09T02:11:26.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.832+0000 7f5e2cff9640 1 -- 172.21.15.19:0/3709319472 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e280116d0 con 0x7f5e30109030 2024-05-09T02:11:26.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.832+0000 7f5e35594640 1 -- 172.21.15.19:0/3709319472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e301a73d0 con 0x7f5e30109030 2024-05-09T02:11:26.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.833+0000 7f5e35594640 1 -- 172.21.15.19:0/3709319472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e30103e70 con 0x7f5e30109030 2024-05-09T02:11:26.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.834+0000 7f5e2cff9640 1 -- 172.21.15.19:0/3709319472 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f5e280051d0 con 0x7f5e30109030 2024-05-09T02:11:26.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.835+0000 7f5e2cff9640 1 --2- 172.21.15.19:0/3709319472 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5e1c0b29e0 0x7f5e1c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:26.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.835+0000 7f5e2cff9640 1 -- 172.21.15.19:0/3709319472 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f5e280e2d80 con 0x7f5e30109030 2024-05-09T02:11:26.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.835+0000 7f5e2effd640 1 --2- 172.21.15.19:0/3709319472 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5e1c0b29e0 0x7f5e1c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:26.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.835+0000 7f5e2effd640 1 --2- 172.21.15.19:0/3709319472 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5e1c0b29e0 0x7f5e1c0b4ef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5e3019bae0 tx=0x7f5e20005f90 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:26.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:26.838+0000 7f5e2cff9640 1 -- 172.21.15.19:0/3709319472 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5e28016030 con 0x7f5e30109030 2024-05-09T02:11:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.041+0000 7f5e35594640 1 -- 172.21.15.19:0/3709319472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f5e301a75a0 con 0x7f5e30109030 2024-05-09T02:11:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.046+0000 7f5e2cff9640 1 -- 172.21.15.19:0/3709319472 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v33)=0 v33) ==== 138+0+0 (secure 0 0 0) 0x7f5e280aa590 con 0x7f5e30109030 2024-05-09T02:11:27.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.048+0000 7f5e35594640 1 -- 172.21.15.19:0/3709319472 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5e1c0b29e0 msgr2=0x7f5e1c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:27.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.048+0000 7f5e35594640 1 --2- 172.21.15.19:0/3709319472 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5e1c0b29e0 0x7f5e1c0b4ef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5e3019bae0 tx=0x7f5e20005f90 comp rx=0 tx=0).stop 2024-05-09T02:11:27.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.048+0000 7f5e35594640 1 -- 172.21.15.19:0/3709319472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e30109030 msgr2=0x7f5e3019a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:27.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.048+0000 7f5e35594640 1 --2- 172.21.15.19:0/3709319472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e30109030 0x7f5e3019a7b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800eaa0 tx=0x7f5e2800ef60 comp rx=0 tx=0).stop 2024-05-09T02:11:27.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.048+0000 7f5e35594640 1 -- 172.21.15.19:0/3709319472 shutdown_connections 2024-05-09T02:11:27.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.048+0000 7f5e35594640 1 --2- 172.21.15.19:0/3709319472 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5e1c0b29e0 0x7f5e1c0b4ef0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.048+0000 7f5e35594640 1 --2- 172.21.15.19:0/3709319472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e30109030 0x7f5e3019a7b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.048+0000 7f5e35594640 1 --2- 172.21.15.19:0/3709319472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e30108b90 0x7f5e301a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.048+0000 7f5e35594640 1 --2- 172.21.15.19:0/3709319472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e301032e0 0x7f5e301a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.048+0000 7f5e35594640 1 -- 172.21.15.19:0/3709319472 >> 172.21.15.19:0/3709319472 conn(0x7f5e300fe390 msgr2=0x7f5e3010e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.048+0000 7f5e35594640 1 -- 172.21.15.19:0/3709319472 shutdown_connections 2024-05-09T02:11:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.048+0000 7f5e35594640 1 -- 172.21.15.19:0/3709319472 wait complete. 2024-05-09T02:11:27.063 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-09T02:11:27.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.180+0000 7f073e6a7640 1 -- 172.21.15.19:0/2124294258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0738109040 msgr2=0x7f0738102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:27.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.180+0000 7f073e6a7640 1 --2- 172.21.15.19:0/2124294258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0738109040 0x7f0738102c80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0720009900 tx=0x7f072002a410 comp rx=0 tx=0).stop 2024-05-09T02:11:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.180+0000 7f073e6a7640 1 -- 172.21.15.19:0/2124294258 shutdown_connections 2024-05-09T02:11:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.180+0000 7f073e6a7640 1 --2- 172.21.15.19:0/2124294258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07381032f0 0x7f073810f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.180+0000 7f073e6a7640 1 --2- 172.21.15.19:0/2124294258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0738109040 0x7f0738102c80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.180+0000 7f073e6a7640 1 --2- 172.21.15.19:0/2124294258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0738108ba0 0x7f0738102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.180+0000 7f073e6a7640 1 -- 172.21.15.19:0/2124294258 >> 172.21.15.19:0/2124294258 conn(0x7f07380782e0 msgr2=0x7f0738078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.180+0000 7f073e6a7640 1 -- 172.21.15.19:0/2124294258 shutdown_connections 2024-05-09T02:11:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.180+0000 7f073e6a7640 1 -- 172.21.15.19:0/2124294258 wait complete. 2024-05-09T02:11:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.181+0000 7f073e6a7640 1 Processor -- start 2024-05-09T02:11:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.181+0000 7f073e6a7640 1 -- start start 2024-05-09T02:11:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.181+0000 7f073e6a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07381032f0 0x7f073819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.181+0000 7f073e6a7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0738108ba0 0x7f073819fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.181+0000 7f073e6a7640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0738109040 0x7f0738199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:27.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.181+0000 7f073e6a7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0738112d90 con 0x7f07381032f0 2024-05-09T02:11:27.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.181+0000 7f073e6a7640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0738112c10 con 0x7f0738109040 2024-05-09T02:11:27.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.181+0000 7f073e6a7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0738112f10 con 0x7f0738108ba0 2024-05-09T02:11:27.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.181+0000 7f0737fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07381032f0 0x7f073819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:27.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.181+0000 7f0737fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07381032f0 0x7f073819f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58170/0 (socket says 172.21.15.19:58170) 2024-05-09T02:11:27.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.181+0000 7f0737fff640 1 -- 172.21.15.19:0/1545151232 learned_addr learned my addr 172.21.15.19:0/1545151232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:27.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.181+0000 7f073cc44640 1 --2- 172.21.15.19:0/1545151232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0738109040 0x7f0738199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:27.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.181+0000 7f072ffff640 1 --2- 172.21.15.19:0/1545151232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0738108ba0 0x7f073819fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:27.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.182+0000 7f0737fff640 1 -- 172.21.15.19:0/1545151232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0738108ba0 msgr2=0x7f073819fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:27.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.182+0000 7f0737fff640 1 --2- 172.21.15.19:0/1545151232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0738108ba0 0x7f073819fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.182+0000 7f0737fff640 1 -- 172.21.15.19:0/1545151232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0738109040 msgr2=0x7f0738199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:27.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.182+0000 7f0737fff640 1 --2- 172.21.15.19:0/1545151232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0738109040 0x7f0738199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.182+0000 7f0737fff640 1 -- 172.21.15.19:0/1545151232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0738199da0 con 0x7f07381032f0 2024-05-09T02:11:27.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.182+0000 7f0737fff640 1 --2- 172.21.15.19:0/1545151232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07381032f0 0x7f073819f4f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f072800e9c0 tx=0x7f072800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:27.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.182+0000 7f0735ffb640 1 -- 172.21.15.19:0/1545151232 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072800cd80 con 0x7f07381032f0 2024-05-09T02:11:27.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.182+0000 7f0735ffb640 1 -- 172.21.15.19:0/1545151232 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07280054d0 con 0x7f07381032f0 2024-05-09T02:11:27.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.182+0000 7f073e6a7640 1 -- 172.21.15.19:0/1545151232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f073819a030 con 0x7f07381032f0 2024-05-09T02:11:27.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.182+0000 7f0735ffb640 1 -- 172.21.15.19:0/1545151232 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728010430 con 0x7f07381032f0 2024-05-09T02:11:27.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.182+0000 7f073e6a7640 1 -- 172.21.15.19:0/1545151232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07381a6290 con 0x7f07381032f0 2024-05-09T02:11:27.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.183+0000 7f073e6a7640 1 -- 172.21.15.19:0/1545151232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0738103e80 con 0x7f07381032f0 2024-05-09T02:11:27.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.185+0000 7f0735ffb640 1 -- 172.21.15.19:0/1545151232 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f0728004fb0 con 0x7f07381032f0 2024-05-09T02:11:27.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.185+0000 7f0735ffb640 1 --2- 172.21.15.19:0/1545151232 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f07240b29e0 0x7f07240b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:27.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.186+0000 7f072ffff640 1 --2- 172.21.15.19:0/1545151232 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f07240b29e0 0x7f07240b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:27.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.186+0000 7f0735ffb640 1 -- 172.21.15.19:0/1545151232 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f07280e2e00 con 0x7f07381032f0 2024-05-09T02:11:27.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.186+0000 7f072ffff640 1 --2- 172.21.15.19:0/1545151232 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f07240b29e0 0x7f07240b4ef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0720009230 tx=0x7f0720031400 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:27.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.189+0000 7f0735ffb640 1 -- 172.21.15.19:0/1545151232 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0728054df0 con 0x7f07381032f0 2024-05-09T02:11:27.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.390+0000 7f073e6a7640 1 -- 172.21.15.19:0/1545151232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f07381a6460 con 0x7f07381032f0 2024-05-09T02:11:27.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.394+0000 7f0735ffb640 1 -- 172.21.15.19:0/1545151232 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v34)=0 v34) ==== 133+0+0 (secure 0 0 0) 0x7f07280aa670 con 0x7f07381032f0 2024-05-09T02:11:27.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.397+0000 7f073e6a7640 1 -- 172.21.15.19:0/1545151232 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f07240b29e0 msgr2=0x7f07240b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:27.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.397+0000 7f073e6a7640 1 --2- 172.21.15.19:0/1545151232 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f07240b29e0 0x7f07240b4ef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0720009230 tx=0x7f0720031400 comp rx=0 tx=0).stop 2024-05-09T02:11:27.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.397+0000 7f073e6a7640 1 -- 172.21.15.19:0/1545151232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07381032f0 msgr2=0x7f073819f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:27.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.397+0000 7f073e6a7640 1 --2- 172.21.15.19:0/1545151232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07381032f0 0x7f073819f4f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f072800e9c0 tx=0x7f072800ee80 comp rx=0 tx=0).stop 2024-05-09T02:11:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.397+0000 7f073e6a7640 1 -- 172.21.15.19:0/1545151232 shutdown_connections 2024-05-09T02:11:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.397+0000 7f073e6a7640 1 --2- 172.21.15.19:0/1545151232 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f07240b29e0 0x7f07240b4ef0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.397+0000 7f073e6a7640 1 --2- 172.21.15.19:0/1545151232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0738109040 0x7f0738199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.397+0000 7f073e6a7640 1 --2- 172.21.15.19:0/1545151232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0738108ba0 0x7f073819fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.397+0000 7f073e6a7640 1 --2- 172.21.15.19:0/1545151232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07381032f0 0x7f073819f4f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.397+0000 7f073e6a7640 1 -- 172.21.15.19:0/1545151232 >> 172.21.15.19:0/1545151232 conn(0x7f07380782e0 msgr2=0x7f07381012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.397+0000 7f073e6a7640 1 -- 172.21.15.19:0/1545151232 shutdown_connections 2024-05-09T02:11:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.397+0000 7f073e6a7640 1 -- 172.21.15.19:0/1545151232 wait complete. 2024-05-09T02:11:27.413 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-09T02:11:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.531+0000 7fb996496640 1 -- 172.21.15.19:0/3162567301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb99010b860 msgr2=0x7fb99010dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.531+0000 7fb996496640 1 --2- 172.21.15.19:0/3162567301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb99010b860 0x7fb99010dc80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb984009900 tx=0x7fb98402a510 comp rx=0 tx=0).stop 2024-05-09T02:11:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.531+0000 7fb996496640 1 -- 172.21.15.19:0/3162567301 shutdown_connections 2024-05-09T02:11:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.531+0000 7fb996496640 1 --2- 172.21.15.19:0/3162567301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb99010e210 0x7fb990110700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.531+0000 7fb996496640 1 --2- 172.21.15.19:0/3162567301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb99010b860 0x7fb99010dc80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.531+0000 7fb996496640 1 --2- 172.21.15.19:0/3162567301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb990104e00 0x7fb990105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.531+0000 7fb996496640 1 -- 172.21.15.19:0/3162567301 >> 172.21.15.19:0/3162567301 conn(0x7fb9900782e0 msgr2=0x7fb990078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.531+0000 7fb996496640 1 -- 172.21.15.19:0/3162567301 shutdown_connections 2024-05-09T02:11:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.531+0000 7fb996496640 1 -- 172.21.15.19:0/3162567301 wait complete. 2024-05-09T02:11:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.532+0000 7fb996496640 1 Processor -- start 2024-05-09T02:11:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.532+0000 7fb996496640 1 -- start start 2024-05-09T02:11:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.532+0000 7fb996496640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb990104e00 0x7fb9901026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.532+0000 7fb996496640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb99010b860 0x7fb990102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.532+0000 7fb996496640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99010e210 0x7fb990103140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.532+0000 7fb996496640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb990113ec0 con 0x7fb99010b860 2024-05-09T02:11:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.532+0000 7fb996496640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb990113d40 con 0x7fb99010e210 2024-05-09T02:11:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.532+0000 7fb996496640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb990114040 con 0x7fb990104e00 2024-05-09T02:11:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.532+0000 7fb98f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb99010b860 0x7fb990102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.532+0000 7fb98f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb99010b860 0x7fb990102c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58184/0 (socket says 172.21.15.19:58184) 2024-05-09T02:11:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.532+0000 7fb98f7fe640 1 -- 172.21.15.19:0/1888772885 learned_addr learned my addr 172.21.15.19:0/1888772885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.532+0000 7fb994a33640 1 --2- 172.21.15.19:0/1888772885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99010e210 0x7fb990103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.533+0000 7fb98ffff640 1 --2- 172.21.15.19:0/1888772885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb990104e00 0x7fb9901026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.533+0000 7fb98f7fe640 1 -- 172.21.15.19:0/1888772885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb990104e00 msgr2=0x7fb9901026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:27.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.533+0000 7fb98f7fe640 1 --2- 172.21.15.19:0/1888772885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb990104e00 0x7fb9901026c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.533+0000 7fb98f7fe640 1 -- 172.21.15.19:0/1888772885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99010e210 msgr2=0x7fb990103140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:27.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.533+0000 7fb98f7fe640 1 --2- 172.21.15.19:0/1888772885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99010e210 0x7fb990103140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.533+0000 7fb98f7fe640 1 -- 172.21.15.19:0/1888772885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9901a8be0 con 0x7fb99010b860 2024-05-09T02:11:27.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.533+0000 7fb98f7fe640 1 --2- 172.21.15.19:0/1888772885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb99010b860 0x7fb990102c00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb984005ab0 tx=0x7fb984003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:27.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.533+0000 7fb98d7fa640 1 -- 172.21.15.19:0/1888772885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98403f070 con 0x7fb99010b860 2024-05-09T02:11:27.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.533+0000 7fb98d7fa640 1 -- 172.21.15.19:0/1888772885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb984031d60 con 0x7fb99010b860 2024-05-09T02:11:27.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.533+0000 7fb996496640 1 -- 172.21.15.19:0/1888772885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9901a8e70 con 0x7fb99010b860 2024-05-09T02:11:27.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.533+0000 7fb98d7fa640 1 -- 172.21.15.19:0/1888772885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984035030 con 0x7fb99010b860 2024-05-09T02:11:27.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.533+0000 7fb996496640 1 -- 172.21.15.19:0/1888772885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9901a9270 con 0x7fb99010b860 2024-05-09T02:11:27.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.534+0000 7fb996496640 1 -- 172.21.15.19:0/1888772885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb99006a240 con 0x7fb99010b860 2024-05-09T02:11:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.536+0000 7fb98d7fa640 1 -- 172.21.15.19:0/1888772885 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fb984042070 con 0x7fb99010b860 2024-05-09T02:11:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.537+0000 7fb98d7fa640 1 --2- 172.21.15.19:0/1888772885 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb9780b29e0 0x7fb9780b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.537+0000 7fb98ffff640 1 --2- 172.21.15.19:0/1888772885 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb9780b29e0 0x7fb9780b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.537+0000 7fb98d7fa640 1 -- 172.21.15.19:0/1888772885 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fb984103610 con 0x7fb99010b860 2024-05-09T02:11:27.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.537+0000 7fb98ffff640 1 --2- 172.21.15.19:0/1888772885 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb9780b29e0 0x7fb9780b4ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb980005d00 tx=0x7fb98000b410 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:27.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.540+0000 7fb98d7fa640 1 -- 172.21.15.19:0/1888772885 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb9840ca7d0 con 0x7fb99010b860 2024-05-09T02:11:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.768+0000 7fb996496640 1 -- 172.21.15.19:0/1888772885 --> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fb9901019d0 con 0x7fb9780b29e0 2024-05-09T02:11:27.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.843+0000 7fb98d7fa640 1 -- 172.21.15.19:0/1888772885 <== mgr.15222 v2:172.21.15.19:6800/2638922161 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb9901019d0 con 0x7fb9780b29e0 2024-05-09T02:11:27.845 INFO:teuthology.orchestra.run.smithi019.stdout:Schedule set for path / 2024-05-09T02:11:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.845+0000 7fb996496640 1 -- 172.21.15.19:0/1888772885 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb9780b29e0 msgr2=0x7fb9780b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.845+0000 7fb996496640 1 --2- 172.21.15.19:0/1888772885 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb9780b29e0 0x7fb9780b4ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb980005d00 tx=0x7fb98000b410 comp rx=0 tx=0).stop 2024-05-09T02:11:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.845+0000 7fb996496640 1 -- 172.21.15.19:0/1888772885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb99010b860 msgr2=0x7fb990102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.845+0000 7fb996496640 1 --2- 172.21.15.19:0/1888772885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb99010b860 0x7fb990102c00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb984005ab0 tx=0x7fb984003cd0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.845+0000 7fb996496640 1 -- 172.21.15.19:0/1888772885 shutdown_connections 2024-05-09T02:11:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.845+0000 7fb996496640 1 --2- 172.21.15.19:0/1888772885 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb9780b29e0 0x7fb9780b4ef0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.845+0000 7fb996496640 1 --2- 172.21.15.19:0/1888772885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99010e210 0x7fb990103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.845+0000 7fb996496640 1 --2- 172.21.15.19:0/1888772885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb99010b860 0x7fb990102c00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.845+0000 7fb996496640 1 --2- 172.21.15.19:0/1888772885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb990104e00 0x7fb9901026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.845+0000 7fb996496640 1 -- 172.21.15.19:0/1888772885 >> 172.21.15.19:0/1888772885 conn(0x7fb9900782e0 msgr2=0x7fb9901012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.845+0000 7fb996496640 1 -- 172.21.15.19:0/1888772885 shutdown_connections 2024-05-09T02:11:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.845+0000 7fb996496640 1 -- 172.21.15.19:0/1888772885 wait complete. 2024-05-09T02:11:27.860 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-09T02:11:27.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/576533012' entity='client.admin' 2024-05-09T02:11:27.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:11:27.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:11:27.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:11:27.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:27.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3709319472' entity='client.admin' 2024-05-09T02:11:27.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:11:27.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:11:27.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:11:27.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:27.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 227 B/s wr, 14 op/s 2024-05-09T02:11:27.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1545151232' entity='client.admin' 2024-05-09T02:11:27.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:11:27.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:11:27.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:11:27.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:27 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:27.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/576533012' entity='client.admin' 2024-05-09T02:11:27.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:11:27.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:11:27.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:11:27.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:27.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3709319472' entity='client.admin' 2024-05-09T02:11:27.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:11:27.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:11:27.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:11:27.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:27.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 227 B/s wr, 14 op/s 2024-05-09T02:11:27.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1545151232' entity='client.admin' 2024-05-09T02:11:27.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:11:27.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:11:27.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:11:27.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:27 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:27.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.976+0000 7fcac2092640 1 -- 172.21.15.19:0/3389968354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc10b880 msgr2=0x7fcabc10dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:27.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.976+0000 7fcac2092640 1 --2- 172.21.15.19:0/3389968354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc10b880 0x7fcabc10dca0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009900 tx=0x7fcaa402a480 comp rx=0 tx=0).stop 2024-05-09T02:11:27.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.976+0000 7fcac2092640 1 -- 172.21.15.19:0/3389968354 shutdown_connections 2024-05-09T02:11:27.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.976+0000 7fcac2092640 1 --2- 172.21.15.19:0/3389968354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcabc10e230 0x7fcabc110700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.976+0000 7fcac2092640 1 --2- 172.21.15.19:0/3389968354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc10b880 0x7fcabc10dca0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.976+0000 7fcac2092640 1 --2- 172.21.15.19:0/3389968354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcabc104e20 0x7fcabc105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.976+0000 7fcac2092640 1 -- 172.21.15.19:0/3389968354 >> 172.21.15.19:0/3389968354 conn(0x7fcabc0782e0 msgr2=0x7fcabc078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:27.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.976+0000 7fcac2092640 1 -- 172.21.15.19:0/3389968354 shutdown_connections 2024-05-09T02:11:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.976+0000 7fcac2092640 1 -- 172.21.15.19:0/3389968354 wait complete. 2024-05-09T02:11:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.977+0000 7fcac2092640 1 Processor -- start 2024-05-09T02:11:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.977+0000 7fcac2092640 1 -- start start 2024-05-09T02:11:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.977+0000 7fcac2092640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcabc104e20 0x7fcabc1026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.977+0000 7fcac2092640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcabc10b880 0x7fcabc102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.977+0000 7fcac2092640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc10e230 0x7fcabc103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.977+0000 7fcac2092640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc113e70 con 0x7fcabc10e230 2024-05-09T02:11:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.977+0000 7fcac2092640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc113cf0 con 0x7fcabc10b880 2024-05-09T02:11:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.977+0000 7fcac2092640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc113ff0 con 0x7fcabc104e20 2024-05-09T02:11:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.977+0000 7fcabbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc10e230 0x7fcabc103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.977+0000 7fcabbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc10e230 0x7fcabc103160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58204/0 (socket says 172.21.15.19:58204) 2024-05-09T02:11:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.977+0000 7fcabb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcabc104e20 0x7fcabc1026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.977+0000 7fcabbfff640 1 -- 172.21.15.19:0/45218763 learned_addr learned my addr 172.21.15.19:0/45218763 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:27.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.977+0000 7fcab3fff640 1 --2- 172.21.15.19:0/45218763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcabc10b880 0x7fcabc102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:27.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.978+0000 7fcabbfff640 1 -- 172.21.15.19:0/45218763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcabc104e20 msgr2=0x7fcabc1026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:27.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.978+0000 7fcabbfff640 1 --2- 172.21.15.19:0/45218763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcabc104e20 0x7fcabc1026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.978+0000 7fcabbfff640 1 -- 172.21.15.19:0/45218763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcabc10b880 msgr2=0x7fcabc102c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:27.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.978+0000 7fcabbfff640 1 --2- 172.21.15.19:0/45218763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcabc10b880 0x7fcabc102c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:27.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.978+0000 7fcabbfff640 1 -- 172.21.15.19:0/45218763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc1a8c10 con 0x7fcabc10e230 2024-05-09T02:11:27.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.978+0000 7fcabbfff640 1 --2- 172.21.15.19:0/45218763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc10e230 0x7fcabc103160 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcab400eb10 tx=0x7fcab400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:27.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.978+0000 7fcab97fa640 1 -- 172.21.15.19:0/45218763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4004b20 con 0x7fcabc10e230 2024-05-09T02:11:27.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.978+0000 7fcab97fa640 1 -- 172.21.15.19:0/45218763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcab4004cc0 con 0x7fcabc10e230 2024-05-09T02:11:27.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.978+0000 7fcac2092640 1 -- 172.21.15.19:0/45218763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc1a8f00 con 0x7fcabc10e230 2024-05-09T02:11:27.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.978+0000 7fcab97fa640 1 -- 172.21.15.19:0/45218763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4010670 con 0x7fcabc10e230 2024-05-09T02:11:27.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.978+0000 7fcac2092640 1 -- 172.21.15.19:0/45218763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcabc1a9310 con 0x7fcabc10e230 2024-05-09T02:11:27.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.979+0000 7fcac2092640 1 -- 172.21.15.19:0/45218763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcabc06a240 con 0x7fcabc10e230 2024-05-09T02:11:27.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.981+0000 7fcab97fa640 1 -- 172.21.15.19:0/45218763 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fcab4005000 con 0x7fcabc10e230 2024-05-09T02:11:27.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.981+0000 7fcab97fa640 1 --2- 172.21.15.19:0/45218763 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcaa80b29e0 0x7fcaa80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:27.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.982+0000 7fcabb7fe640 1 --2- 172.21.15.19:0/45218763 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcaa80b29e0 0x7fcaa80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:27.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.982+0000 7fcab97fa640 1 -- 172.21.15.19:0/45218763 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fcab40e2f70 con 0x7fcabc10e230 2024-05-09T02:11:27.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.982+0000 7fcabb7fe640 1 --2- 172.21.15.19:0/45218763 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcaa80b29e0 0x7fcaa80b4ef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00bbe0 tx=0x7fcaac00b390 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:27.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:27.985+0000 7fcab97fa640 1 -- 172.21.15.19:0/45218763 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcab4054df0 con 0x7fcabc10e230 2024-05-09T02:11:28.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/576533012' entity='client.admin' 2024-05-09T02:11:28.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:11:28.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:11:28.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:11:28.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:28.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3709319472' entity='client.admin' 2024-05-09T02:11:28.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:11:28.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:11:28.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:11:28.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:28.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 227 B/s wr, 14 op/s 2024-05-09T02:11:28.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1545151232' entity='client.admin' 2024-05-09T02:11:28.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:11:28.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:11:28.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:11:28.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:27 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:11:28.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.206+0000 7fcac2092640 1 -- 172.21.15.19:0/45218763 --> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fcabc1019d0 con 0x7fcaa80b29e0 2024-05-09T02:11:28.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.275+0000 7fcab97fa640 1 -- 172.21.15.19:0/45218763 <== mgr.15222 v2:172.21.15.19:6800/2638922161 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fcabc1019d0 con 0x7fcaa80b29e0 2024-05-09T02:11:28.278 INFO:teuthology.orchestra.run.smithi019.stdout:Retention added to path / 2024-05-09T02:11:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.277+0000 7fcac2092640 1 -- 172.21.15.19:0/45218763 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcaa80b29e0 msgr2=0x7fcaa80b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.277+0000 7fcac2092640 1 --2- 172.21.15.19:0/45218763 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcaa80b29e0 0x7fcaa80b4ef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00bbe0 tx=0x7fcaac00b390 comp rx=0 tx=0).stop 2024-05-09T02:11:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.277+0000 7fcac2092640 1 -- 172.21.15.19:0/45218763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc10e230 msgr2=0x7fcabc103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.277+0000 7fcac2092640 1 --2- 172.21.15.19:0/45218763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc10e230 0x7fcabc103160 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcab400eb10 tx=0x7fcab400c2d0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.277+0000 7fcac2092640 1 -- 172.21.15.19:0/45218763 shutdown_connections 2024-05-09T02:11:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.277+0000 7fcac2092640 1 --2- 172.21.15.19:0/45218763 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcaa80b29e0 0x7fcaa80b4ef0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.277+0000 7fcac2092640 1 --2- 172.21.15.19:0/45218763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc10e230 0x7fcabc103160 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.277+0000 7fcac2092640 1 --2- 172.21.15.19:0/45218763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcabc10b880 0x7fcabc102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.277+0000 7fcac2092640 1 --2- 172.21.15.19:0/45218763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcabc104e20 0x7fcabc1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.277+0000 7fcac2092640 1 -- 172.21.15.19:0/45218763 >> 172.21.15.19:0/45218763 conn(0x7fcabc0782e0 msgr2=0x7fcabc1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:28.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.277+0000 7fcac2092640 1 -- 172.21.15.19:0/45218763 shutdown_connections 2024-05-09T02:11:28.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.277+0000 7fcac2092640 1 -- 172.21.15.19:0/45218763 wait complete. 2024-05-09T02:11:28.291 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-09T02:11:28.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.353+0000 7f54735c4700 1 -- 172.21.15.19:0/1398174998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f546c108f00 msgr2=0x7f546c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.353+0000 7f54735c4700 1 --2- 172.21.15.19:0/1398174998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f546c108f00 0x7f546c109310 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f545c0099c0 tx=0x7f545c009cd0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.353+0000 7f54735c4700 1 -- 172.21.15.19:0/1398174998 shutdown_connections 2024-05-09T02:11:28.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.353+0000 7f54735c4700 1 --2- 172.21.15.19:0/1398174998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c109850 0x7f546c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.353+0000 7f54735c4700 1 --2- 172.21.15.19:0/1398174998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f546c1045d0 0x7f546c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.353+0000 7f54735c4700 1 --2- 172.21.15.19:0/1398174998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f546c108f00 0x7f546c109310 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.353+0000 7f54735c4700 1 -- 172.21.15.19:0/1398174998 >> 172.21.15.19:0/1398174998 conn(0x7f546c0ffea0 msgr2=0x7f546c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:28.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.353+0000 7f54735c4700 1 -- 172.21.15.19:0/1398174998 shutdown_connections 2024-05-09T02:11:28.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.353+0000 7f54735c4700 1 -- 172.21.15.19:0/1398174998 wait complete. 2024-05-09T02:11:28.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.354+0000 7f54735c4700 1 Processor -- start 2024-05-09T02:11:28.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.354+0000 7f54735c4700 1 -- start start 2024-05-09T02:11:28.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.354+0000 7f54735c4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f546c1045d0 0x7f546c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.354+0000 7f54735c4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c108f00 0x7f546c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.354+0000 7f54735c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f546c109850 0x7f546c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.354+0000 7f54735c4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f546c114460 con 0x7f546c109850 2024-05-09T02:11:28.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.354+0000 7f54735c4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1142e0 con 0x7f546c1045d0 2024-05-09T02:11:28.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.354+0000 7f54735c4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1145e0 con 0x7f546c108f00 2024-05-09T02:11:28.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.354+0000 7f5471b61700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f546c109850 0x7f546c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.354+0000 7f5471b61700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f546c109850 0x7f546c19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58218/0 (socket says 172.21.15.19:58218) 2024-05-09T02:11:28.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.354+0000 7f5471b61700 1 -- 172.21.15.19:0/2824366210 learned_addr learned my addr 172.21.15.19:0/2824366210 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:28.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.354+0000 7f5470b5f700 1 --2- 172.21.15.19:0/2824366210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c108f00 0x7f546c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.354+0000 7f5471360700 1 --2- 172.21.15.19:0/2824366210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f546c1045d0 0x7f546c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.355+0000 7f5470b5f700 1 -- 172.21.15.19:0/2824366210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f546c1045d0 msgr2=0x7f546c19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.355+0000 7f5470b5f700 1 --2- 172.21.15.19:0/2824366210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f546c1045d0 0x7f546c19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.355+0000 7f5470b5f700 1 -- 172.21.15.19:0/2824366210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f546c109850 msgr2=0x7f546c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.355+0000 7f5470b5f700 1 --2- 172.21.15.19:0/2824366210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f546c109850 0x7f546c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.355+0000 7f5470b5f700 1 -- 172.21.15.19:0/2824366210 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c19fce0 con 0x7f546c108f00 2024-05-09T02:11:28.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.355+0000 7f5471b61700 1 --2- 172.21.15.19:0/2824366210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f546c109850 0x7f546c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:28.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.355+0000 7f5470b5f700 1 --2- 172.21.15.19:0/2824366210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c108f00 0x7f546c19b3b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f545c0098e0 tx=0x7f545c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:28.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.355+0000 7f54627fc700 1 -- 172.21.15.19:0/2824366210 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c026070 con 0x7f546c108f00 2024-05-09T02:11:28.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.355+0000 7f54735c4700 1 -- 172.21.15.19:0/2824366210 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546c19ff70 con 0x7f546c108f00 2024-05-09T02:11:28.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.355+0000 7f54627fc700 1 -- 172.21.15.19:0/2824366210 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f545c003c70 con 0x7f546c108f00 2024-05-09T02:11:28.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.355+0000 7f54627fc700 1 -- 172.21.15.19:0/2824366210 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c00f9c0 con 0x7f546c108f00 2024-05-09T02:11:28.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.355+0000 7f54735c4700 1 -- 172.21.15.19:0/2824366210 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f546c1a78f0 con 0x7f546c108f00 2024-05-09T02:11:28.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.358+0000 7f54627fc700 1 -- 172.21.15.19:0/2824366210 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f545c029070 con 0x7f546c108f00 2024-05-09T02:11:28.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.359+0000 7f54627fc700 1 --2- 172.21.15.19:0/2824366210 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f54580b2a80 0x7f54580b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.359+0000 7f5481fdab80 1 Processor -- start 2024-05-09T02:11:28.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.359+0000 7f5481fdab80 1 -- start start 2024-05-09T02:11:28.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.359+0000 7f54627fc700 1 -- 172.21.15.19:0/2824366210 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f545c0e2f60 con 0x7f546c108f00 2024-05-09T02:11:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.359+0000 7f5471360700 1 --2- 172.21.15.19:0/2824366210 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f54580b2a80 0x7f54580b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5481fdab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aae5d80 0x56257abee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5481fdab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5481fdab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257abe8480 0x56257abe8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5481fdab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56257ab0b120 con 0x56257aa6b520 2024-05-09T02:11:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5481fdab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56257aab0a20 con 0x56257aae5d80 2024-05-09T02:11:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5481fdab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56257ab070e0 con 0x56257abe8480 2024-05-09T02:11:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5470b5f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5471360700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aae5d80 0x56257abee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5470b5f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abee800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58224/0 (socket says 172.21.15.19:58224) 2024-05-09T02:11:28.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5470b5f700 1 -- 172.21.15.19:0/852415562 learned_addr learned my addr 172.21.15.19:0/852415562 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5471b61700 1 --2- 172.21.15.19:0/852415562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257abe8480 0x56257abe8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5471360700 1 --2- 172.21.15.19:0/2824366210 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f54580b2a80 0x7f54580b4f90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5468009680 tx=0x7f5468011720 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5471360700 1 -- 172.21.15.19:0/852415562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257abe8480 msgr2=0x56257abe8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5471360700 1 --2- 172.21.15.19:0/852415562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257abe8480 0x56257abe8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5471360700 1 -- 172.21.15.19:0/852415562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 msgr2=0x56257abee800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5471360700 1 --2- 172.21.15.19:0/852415562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abee800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5471360700 1 -- 172.21.15.19:0/852415562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56257aaa1580 con 0x56257aae5d80 2024-05-09T02:11:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.360+0000 7f5470b5f700 1 --2- 172.21.15.19:0/852415562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.361+0000 7f5471360700 1 --2- 172.21.15.19:0/852415562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aae5d80 0x56257abee2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5468011b20 tx=0x7f546801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5679b7b94834676c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.361+0000 7f54577fe700 1 -- 172.21.15.19:0/852415562 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468017040 con 0x56257aae5d80 2024-05-09T02:11:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.361+0000 7f54577fe700 1 -- 172.21.15.19:0/852415562 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5468019050 con 0x56257aae5d80 2024-05-09T02:11:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.361+0000 7f54577fe700 1 -- 172.21.15.19:0/852415562 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546800e610 con 0x56257aae5d80 2024-05-09T02:11:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.362+0000 7f5481fdab80 1 -- 172.21.15.19:0/852415562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aae5d80 msgr2=0x56257abee2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.362+0000 7f5481fdab80 1 --2- 172.21.15.19:0/852415562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aae5d80 0x56257abee2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5468011b20 tx=0x7f546801c000 comp rx=0 tx=0).stop 2024-05-09T02:11:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.362+0000 7f5481fdab80 1 -- 172.21.15.19:0/852415562 shutdown_connections 2024-05-09T02:11:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.362+0000 7f5481fdab80 1 --2- 172.21.15.19:0/852415562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257abe8480 0x56257abe8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.362+0000 7f5481fdab80 1 --2- 172.21.15.19:0/852415562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aae5d80 0x56257abee2c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.362+0000 7f5481fdab80 1 --2- 172.21.15.19:0/852415562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.362+0000 7f5481fdab80 1 -- 172.21.15.19:0/852415562 >> 172.21.15.19:0/852415562 conn(0x56257aa6eff0 msgr2=0x56257abe32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.362+0000 7f5481fdab80 1 -- 172.21.15.19:0/852415562 shutdown_connections 2024-05-09T02:11:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.362+0000 7f5481fdab80 1 -- 172.21.15.19:0/852415562 wait complete. 2024-05-09T02:11:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.362+0000 7f5481fdab80 1 -- start start 2024-05-09T02:11:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.363+0000 7f5481fdab80 1 Processor -- start 2024-05-09T02:11:28.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.363+0000 7f5481fdab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6b520 0x56257abf1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.363+0000 7f5481fdab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aae5d80 0x56257abf2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.363+0000 7f5481fdab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257abe8480 0x56257abf2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.363+0000 7f5481fdab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56257abe91e0 con 0x56257aae5d80 2024-05-09T02:11:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.363+0000 7f5481fdab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56257aaa5ba0 con 0x56257aa6b520 2024-05-09T02:11:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.363+0000 7f5481fdab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56257abf50a0 con 0x56257abe8480 2024-05-09T02:11:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.363+0000 7f5481fdab80 10 client.0 _ll_register_callbacks cb 0x56257a97c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:11:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.363+0000 7f5470b5f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aae5d80 0x56257abf2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.363+0000 7f5470b5f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aae5d80 0x56257abf2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58238/0 (socket says 172.21.15.19:58238) 2024-05-09T02:11:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.363+0000 7f5470b5f700 1 -- 172.21.15.19:0/3320441652 learned_addr learned my addr 172.21.15.19:0/3320441652 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.363+0000 7f5471b61700 1 --2- 172.21.15.19:0/3320441652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257abe8480 0x56257abf2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.363+0000 7f5471360700 1 --2- 172.21.15.19:0/3320441652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6b520 0x56257abf1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.364+0000 7f5470b5f700 1 -- 172.21.15.19:0/3320441652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257abe8480 msgr2=0x56257abf2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.364+0000 7f5470b5f700 1 --2- 172.21.15.19:0/3320441652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257abe8480 0x56257abf2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.364+0000 7f5470b5f700 1 -- 172.21.15.19:0/3320441652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6b520 msgr2=0x56257abf1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:28.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.364+0000 7f5470b5f700 1 --2- 172.21.15.19:0/3320441652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6b520 0x56257abf1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.364+0000 7f5470b5f700 1 -- 172.21.15.19:0/3320441652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56257a986510 con 0x56257aae5d80 2024-05-09T02:11:28.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.364+0000 7f5470b5f700 1 --2- 172.21.15.19:0/3320441652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aae5d80 0x56257abf2530 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f545c0047d0 tx=0x7f545c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7336e24063dafa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:28.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.364+0000 7f5454ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:11:28.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.364+0000 7f5454ff9700 1 -- 172.21.15.19:0/3320441652 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c0aa800 con 0x56257aae5d80 2024-05-09T02:11:28.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.364+0000 7f5454ff9700 1 -- 172.21.15.19:0/3320441652 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f545c0b2c50 con 0x56257aae5d80 2024-05-09T02:11:28.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.364+0000 7f5481fdab80 1 -- 172.21.15.19:0/3320441652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5468019050 con 0x56257aae5d80 2024-05-09T02:11:28.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.364+0000 7f5454ff9700 1 -- 172.21.15.19:0/3320441652 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f545c0aa100 con 0x56257aae5d80 2024-05-09T02:11:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.364+0000 7f5481fdab80 10 client.15381 fetch_fsmap learned FSMap version 29 2024-05-09T02:11:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.364+0000 7f5481fdab80 1 -- 172.21.15.19:0/3320441652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56257abff9c0 con 0x56257aae5d80 2024-05-09T02:11:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.365+0000 7f5454ff9700 1 -- 172.21.15.19:0/3320441652 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5468019050 con 0x56257aae5d80 2024-05-09T02:11:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.365+0000 7f5454ff9700 1 -- 172.21.15.19:0/3320441652 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f545c00fb60 con 0x56257aae5d80 2024-05-09T02:11:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.365+0000 7f5481fdab80 10 client.15381 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T02:11:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.365+0000 7f5481fdab80 10 client.15381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:11:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.365+0000 7f5481fdab80 20 client.15381 populate_metadata read hostname 'smithi019' 2024-05-09T02:11:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.365+0000 7f5481fdab80 1 --2- 172.21.15.19:0/3320441652 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56257ac01f30 0x56257ac043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.365+0000 7f5481fdab80 4 client.15381 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:11:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.365+0000 7f5481fdab80 1 -- 172.21.15.19:0/3320441652 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56257ab18b00 con 0x56257ac01f30 2024-05-09T02:11:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.365+0000 7f5471360700 1 --2- 172.21.15.19:0/3320441652 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56257ac01f30 0x56257ac043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.366+0000 7f5454ff9700 10 client.15381 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:11:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.366+0000 7f5471360700 1 --2- 172.21.15.19:0/3320441652 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56257ac01f30 0x56257ac043a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e1772d912a1d6b server_cookie=822e840aef05db9c in_seq=0 out_seq=0 2024-05-09T02:11:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.367+0000 7f5454ff9700 1 -- 172.21.15.19:0/3320441652 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56257ab18b00 con 0x56257ac01f30 2024-05-09T02:11:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.367+0000 7f5454ff9700 10 client.15381 handle_command_reply: tid=0 2024-05-09T02:11:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.367+0000 7f5481fdab80 1 client.15381 shutdown 2024-05-09T02:11:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.367+0000 7f5481fdab80 1 -- 172.21.15.19:0/3320441652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aae5d80 msgr2=0x56257abf2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.367+0000 7f5481fdab80 1 --2- 172.21.15.19:0/3320441652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aae5d80 0x56257abf2530 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f545c0047d0 tx=0x7f545c009310 comp rx=0 tx=0).stop 2024-05-09T02:11:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.367+0000 7f5481fdab80 1 -- 172.21.15.19:0/3320441652 shutdown_connections 2024-05-09T02:11:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.367+0000 7f5481fdab80 1 --2- 172.21.15.19:0/3320441652 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56257ac01f30 0x56257ac043a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.367+0000 7f5481fdab80 1 --2- 172.21.15.19:0/3320441652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257abe8480 0x56257abf2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.367+0000 7f5454ff9700 0 client.15381 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:11:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.367+0000 7f5481fdab80 1 --2- 172.21.15.19:0/3320441652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6b520 0x56257abf1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.367+0000 7f5481fdab80 1 --2- 172.21.15.19:0/3320441652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aae5d80 0x56257abf2530 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.367+0000 7f5481fdab80 1 -- 172.21.15.19:0/3320441652 >> 172.21.15.19:0/3320441652 conn(0x56257aa6eff0 msgr2=0x56257abdd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.367+0000 7f5481fdab80 1 -- 172.21.15.19:0/3320441652 shutdown_connections 2024-05-09T02:11:28.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.367+0000 7f5481fdab80 1 -- 172.21.15.19:0/3320441652 wait complete. 2024-05-09T02:11:28.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.367+0000 7f5481fdab80 20 client.15381 trim_cache size 0 max 16384 2024-05-09T02:11:28.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.389+0000 7f5481fdab80 1 Processor -- start 2024-05-09T02:11:28.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.389+0000 7f5481fdab80 1 -- start start 2024-05-09T02:11:28.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.389+0000 7f5481fdab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6eff0 0x56257abddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.389+0000 7f5481fdab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abec390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.389+0000 7f5481fdab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257aae5d80 0x56257abfddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.389+0000 7f5481fdab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1aed30 con 0x56257aa6b520 2024-05-09T02:11:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.389+0000 7f5481fdab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1b0870 con 0x56257aa6eff0 2024-05-09T02:11:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.389+0000 7f5481fdab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1af400 con 0x56257aae5d80 2024-05-09T02:11:28.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.390+0000 7f5470b5f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abec390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.390+0000 7f5470b5f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abec390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58250/0 (socket says 172.21.15.19:58250) 2024-05-09T02:11:28.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.390+0000 7f5471b61700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257aae5d80 0x56257abfddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.390+0000 7f5470b5f700 1 -- 172.21.15.19:0/48216224 learned_addr learned my addr 172.21.15.19:0/48216224 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:28.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.390+0000 7f5471360700 1 --2- 172.21.15.19:0/48216224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6eff0 0x56257abddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.390+0000 7f5471b61700 1 -- 172.21.15.19:0/48216224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6eff0 msgr2=0x56257abddaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:28.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.390+0000 7f5471b61700 1 --2- 172.21.15.19:0/48216224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6eff0 0x56257abddaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.390+0000 7f5471b61700 1 -- 172.21.15.19:0/48216224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 msgr2=0x56257abec390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:28.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.390+0000 7f5471b61700 1 --2- 172.21.15.19:0/48216224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abec390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.390+0000 7f5471b61700 1 -- 172.21.15.19:0/48216224 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56257abdfa50 con 0x56257aae5d80 2024-05-09T02:11:28.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.390+0000 7f5470b5f700 1 --2- 172.21.15.19:0/48216224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abec390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:28.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.390+0000 7f5471b61700 1 --2- 172.21.15.19:0/48216224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257aae5d80 0x56257abfddb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5464007900 tx=0x7f546400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13d45cb8e6d5ce78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:28.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.390+0000 7f5456ffd700 1 -- 172.21.15.19:0/48216224 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464004db0 con 0x56257aae5d80 2024-05-09T02:11:28.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.390+0000 7f5456ffd700 1 -- 172.21.15.19:0/48216224 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f546400e670 con 0x56257aae5d80 2024-05-09T02:11:28.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.391+0000 7f5456ffd700 1 -- 172.21.15.19:0/48216224 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464004280 con 0x56257aae5d80 2024-05-09T02:11:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.391+0000 7f5481fdab80 1 -- 172.21.15.19:0/48216224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257aae5d80 msgr2=0x56257abfddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.391+0000 7f5481fdab80 1 --2- 172.21.15.19:0/48216224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257aae5d80 0x56257abfddb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5464007900 tx=0x7f546400b040 comp rx=0 tx=0).stop 2024-05-09T02:11:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.391+0000 7f5481fdab80 1 -- 172.21.15.19:0/48216224 shutdown_connections 2024-05-09T02:11:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.391+0000 7f5481fdab80 1 --2- 172.21.15.19:0/48216224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257aae5d80 0x56257abfddb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.391+0000 7f5481fdab80 1 --2- 172.21.15.19:0/48216224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6eff0 0x56257abddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.391+0000 7f5481fdab80 1 --2- 172.21.15.19:0/48216224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abec390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.391+0000 7f5481fdab80 1 -- 172.21.15.19:0/48216224 >> 172.21.15.19:0/48216224 conn(0x56257abe35a0 msgr2=0x56257abebc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:28.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.391+0000 7f5481fdab80 1 -- 172.21.15.19:0/48216224 shutdown_connections 2024-05-09T02:11:28.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.391+0000 7f5481fdab80 1 -- 172.21.15.19:0/48216224 wait complete. 2024-05-09T02:11:28.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.392+0000 7f5481fdab80 1 -- start start 2024-05-09T02:11:28.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.392+0000 7f5481fdab80 1 Processor -- start 2024-05-09T02:11:28.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.392+0000 7f5481fdab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abfa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.392+0000 7f5481fdab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6eff0 0x56257abf6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.392+0000 7f5481fdab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257aae5d80 0x56257abf74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5481fdab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1b01a0 con 0x56257aa6b520 2024-05-09T02:11:28.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5481fdab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1afad0 con 0x56257aa6eff0 2024-05-09T02:11:28.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5481fdab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1accf0 con 0x56257aae5d80 2024-05-09T02:11:28.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5471360700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abfa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5481fdab80 10 client.0 _ll_register_callbacks cb 0x56257a97c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:11:28.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5471360700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abfa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58260/0 (socket says 172.21.15.19:58260) 2024-05-09T02:11:28.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5471360700 1 -- 172.21.15.19:0/3682811139 learned_addr learned my addr 172.21.15.19:0/3682811139 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:28.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5470b5f700 1 --2- 172.21.15.19:0/3682811139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6eff0 0x56257abf6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5471360700 1 -- 172.21.15.19:0/3682811139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257aae5d80 msgr2=0x56257abf74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:28.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5471b61700 1 --2- 172.21.15.19:0/3682811139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257aae5d80 0x56257abf74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5471360700 1 --2- 172.21.15.19:0/3682811139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257aae5d80 0x56257abf74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5471360700 1 -- 172.21.15.19:0/3682811139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6eff0 msgr2=0x56257abf6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:28.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5471360700 1 --2- 172.21.15.19:0/3682811139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6eff0 0x56257abf6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5471360700 1 -- 172.21.15.19:0/3682811139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56257abeeb70 con 0x56257aa6b520 2024-05-09T02:11:28.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5470b5f700 1 --2- 172.21.15.19:0/3682811139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6eff0 0x56257abf6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:11:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5471b61700 1 --2- 172.21.15.19:0/3682811139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257aae5d80 0x56257abf74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:11:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5471360700 1 --2- 172.21.15.19:0/3682811139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abfa6b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5468011f30 tx=0x7f5468009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5ac2d499f32489 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5454ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:11:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5454ff9700 1 -- 172.21.15.19:0/3682811139 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468017040 con 0x56257aa6b520 2024-05-09T02:11:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5454ff9700 1 -- 172.21.15.19:0/3682811139 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f546801dbb0 con 0x56257aa6b520 2024-05-09T02:11:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.393+0000 7f5481fdab80 1 -- 172.21.15.19:0/3682811139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f546400e670 con 0x56257aa6b520 2024-05-09T02:11:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.394+0000 7f5454ff9700 1 -- 172.21.15.19:0/3682811139 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f546801ddb0 con 0x56257aa6b520 2024-05-09T02:11:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.394+0000 7f5481fdab80 10 client.15387 fetch_fsmap learned FSMap version 29 2024-05-09T02:11:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.394+0000 7f5481fdab80 1 -- 172.21.15.19:0/3682811139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56257ac6d470 con 0x56257aa6b520 2024-05-09T02:11:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.394+0000 7f5454ff9700 1 -- 172.21.15.19:0/3682811139 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f546400e670 con 0x56257aa6b520 2024-05-09T02:11:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.394+0000 7f5454ff9700 1 -- 172.21.15.19:0/3682811139 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f54680046f0 con 0x56257aa6b520 2024-05-09T02:11:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.394+0000 7f5481fdab80 10 client.15387 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T02:11:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.394+0000 7f5481fdab80 10 client.15387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:11:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.394+0000 7f5481fdab80 20 client.15387 populate_metadata read hostname 'smithi019' 2024-05-09T02:11:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.394+0000 7f5481fdab80 1 --2- 172.21.15.19:0/3682811139 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56257abe8480 0x56257ac6f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.394+0000 7f5481fdab80 4 client.15387 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T02:11:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.394+0000 7f5481fdab80 1 -- 172.21.15.19:0/3682811139 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56257ab12ba0 con 0x56257abe8480 2024-05-09T02:11:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.395+0000 7f5470b5f700 1 --2- 172.21.15.19:0/3682811139 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56257abe8480 0x56257ac6f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.395+0000 7f5454ff9700 10 client.15387 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:11:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.395+0000 7f5470b5f700 1 --2- 172.21.15.19:0/3682811139 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56257abe8480 0x56257ac6f9f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36ab5e80fee60faa server_cookie=e287092e14beebcb in_seq=0 out_seq=0 2024-05-09T02:11:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.396+0000 7f5454ff9700 1 -- 172.21.15.19:0/3682811139 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56257ab12ba0 con 0x56257abe8480 2024-05-09T02:11:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.396+0000 7f5454ff9700 10 client.15387 handle_command_reply: tid=0 2024-05-09T02:11:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.396+0000 7f5481fdab80 1 client.15387 shutdown 2024-05-09T02:11:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.396+0000 7f5481fdab80 1 -- 172.21.15.19:0/3682811139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 msgr2=0x56257abfa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.396+0000 7f5481fdab80 1 --2- 172.21.15.19:0/3682811139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abfa6b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5468011f30 tx=0x7f5468009260 comp rx=0 tx=0).stop 2024-05-09T02:11:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.397+0000 7f5481fdab80 1 -- 172.21.15.19:0/3682811139 shutdown_connections 2024-05-09T02:11:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.397+0000 7f5481fdab80 1 --2- 172.21.15.19:0/3682811139 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56257abe8480 0x56257ac6f9f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.397+0000 7f5481fdab80 1 --2- 172.21.15.19:0/3682811139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56257aae5d80 0x56257abf74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.397+0000 7f5481fdab80 1 --2- 172.21.15.19:0/3682811139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56257aa6b520 0x56257abfa6b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.397+0000 7f5481fdab80 1 --2- 172.21.15.19:0/3682811139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56257aa6eff0 0x56257abf6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.397+0000 7f5454ff9700 0 client.15387 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:11:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.397+0000 7f5481fdab80 1 -- 172.21.15.19:0/3682811139 >> 172.21.15.19:0/3682811139 conn(0x56257abe35a0 msgr2=0x56257abdd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.397+0000 7f5481fdab80 1 -- 172.21.15.19:0/3682811139 shutdown_connections 2024-05-09T02:11:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.397+0000 7f5481fdab80 1 -- 172.21.15.19:0/3682811139 wait complete. 2024-05-09T02:11:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.397+0000 7f5481fdab80 20 client.15387 trim_cache size 0 max 16384 2024-05-09T02:11:28.413 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:11:28.413 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T02:11:28.413 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "c7502900-31f3-4cce-9d82-727e7fa3c05d", 2024-05-09T02:11:28.413 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T02:11:28.413 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:11:28.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.398+0000 7f54735c4700 1 -- 172.21.15.19:0/2824366210 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f54580b2a80 msgr2=0x7f54580b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.398+0000 7f54735c4700 1 --2- 172.21.15.19:0/2824366210 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f54580b2a80 0x7f54580b4f90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5468009680 tx=0x7f5468011720 comp rx=0 tx=0).stop 2024-05-09T02:11:28.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.398+0000 7f54735c4700 1 -- 172.21.15.19:0/2824366210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c108f00 msgr2=0x7f546c19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.398+0000 7f54735c4700 1 --2- 172.21.15.19:0/2824366210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c108f00 0x7f546c19b3b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f545c0098e0 tx=0x7f545c004270 comp rx=0 tx=0).stop 2024-05-09T02:11:28.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.398+0000 7f54735c4700 1 -- 172.21.15.19:0/2824366210 shutdown_connections 2024-05-09T02:11:28.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.398+0000 7f54735c4700 1 --2- 172.21.15.19:0/2824366210 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f54580b2a80 0x7f54580b4f90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.398+0000 7f54735c4700 1 --2- 172.21.15.19:0/2824366210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f546c109850 0x7f546c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.398+0000 7f54735c4700 1 --2- 172.21.15.19:0/2824366210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f546c1045d0 0x7f546c19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.398+0000 7f54735c4700 1 --2- 172.21.15.19:0/2824366210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c108f00 0x7f546c19b3b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.398+0000 7f54735c4700 1 -- 172.21.15.19:0/2824366210 >> 172.21.15.19:0/2824366210 conn(0x7f546c0ffea0 msgr2=0x7f546c1086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:28.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.398+0000 7f54735c4700 1 -- 172.21.15.19:0/2824366210 shutdown_connections 2024-05-09T02:11:28.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.398+0000 7f54735c4700 1 -- 172.21.15.19:0/2824366210 wait complete. 2024-05-09T02:11:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.414+0000 7f45eb573640 1 -- 172.21.15.19:0/971395914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e4104e10 msgr2=0x7f45e4105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.414+0000 7f45eb573640 1 --2- 172.21.15.19:0/971395914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e4104e10 0x7f45e4105220 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009900 tx=0x7f45e002a510 comp rx=0 tx=0).stop 2024-05-09T02:11:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.414+0000 7f45eb573640 1 -- 172.21.15.19:0/971395914 shutdown_connections 2024-05-09T02:11:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.414+0000 7f45eb573640 1 --2- 172.21.15.19:0/971395914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45e410e220 0x7f45e4110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.414+0000 7f45eb573640 1 --2- 172.21.15.19:0/971395914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45e410b870 0x7f45e410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.414+0000 7f45eb573640 1 --2- 172.21.15.19:0/971395914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e4104e10 0x7f45e4105220 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.414+0000 7f45eb573640 1 -- 172.21.15.19:0/971395914 >> 172.21.15.19:0/971395914 conn(0x7f45e40792a0 msgr2=0x7f45e40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:28.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.414+0000 7f45eb573640 1 -- 172.21.15.19:0/971395914 shutdown_connections 2024-05-09T02:11:28.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.414+0000 7f45eb573640 1 -- 172.21.15.19:0/971395914 wait complete. 2024-05-09T02:11:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.415+0000 7f45eb573640 1 Processor -- start 2024-05-09T02:11:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.415+0000 7f45eb573640 1 -- start start 2024-05-09T02:11:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45eb573640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45e4104e10 0x7f45e41a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45eb573640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e410b870 0x7f45e41a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45eb573640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45e410e220 0x7f45e419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45eb573640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f45e4113e80 con 0x7f45e410b870 2024-05-09T02:11:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45eb573640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45e4113d00 con 0x7f45e4104e10 2024-05-09T02:11:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45eb573640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45e4114000 con 0x7f45e410e220 2024-05-09T02:11:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45e8b0e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e410b870 0x7f45e41a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45e8b0e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e410b870 0x7f45e41a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58276/0 (socket says 172.21.15.19:58276) 2024-05-09T02:11:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45e8b0e640 1 -- 172.21.15.19:0/44786992 learned_addr learned my addr 172.21.15.19:0/44786992 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45e930f640 1 --2- 172.21.15.19:0/44786992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45e4104e10 0x7f45e41a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45e9b10640 1 --2- 172.21.15.19:0/44786992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45e410e220 0x7f45e419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45e8b0e640 1 -- 172.21.15.19:0/44786992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45e410e220 msgr2=0x7f45e419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45e8b0e640 1 --2- 172.21.15.19:0/44786992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45e410e220 0x7f45e419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45e8b0e640 1 -- 172.21.15.19:0/44786992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45e4104e10 msgr2=0x7f45e41a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45e8b0e640 1 --2- 172.21.15.19:0/44786992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45e4104e10 0x7f45e41a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.416+0000 7f45e8b0e640 1 -- 172.21.15.19:0/44786992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e419ae10 con 0x7f45e410b870 2024-05-09T02:11:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.417+0000 7f45e8b0e640 1 --2- 172.21.15.19:0/44786992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e410b870 0x7f45e41a0aa0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f45d400e9f0 tx=0x7f45d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.417+0000 7f45da7fc640 1 -- 172.21.15.19:0/44786992 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d400cd80 con 0x7f45e410b870 2024-05-09T02:11:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.417+0000 7f45da7fc640 1 -- 172.21.15.19:0/44786992 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45d40054d0 con 0x7f45e410b870 2024-05-09T02:11:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.417+0000 7f45eb573640 1 -- 172.21.15.19:0/44786992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e419b0a0 con 0x7f45e410b870 2024-05-09T02:11:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.417+0000 7f45da7fc640 1 -- 172.21.15.19:0/44786992 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4010690 con 0x7f45e410b870 2024-05-09T02:11:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.417+0000 7f45eb573640 1 -- 172.21.15.19:0/44786992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e41a7300 con 0x7f45e410b870 2024-05-09T02:11:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.418+0000 7f45eb573640 1 -- 172.21.15.19:0/44786992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e4075780 con 0x7f45e410b870 2024-05-09T02:11:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.420+0000 7f45da7fc640 1 -- 172.21.15.19:0/44786992 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f45d4004e90 con 0x7f45e410b870 2024-05-09T02:11:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.420+0000 7f45da7fc640 1 --2- 172.21.15.19:0/44786992 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45d00b2a60 0x7f45d00b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.421+0000 7f45da7fc640 1 -- 172.21.15.19:0/44786992 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f45d40e2be0 con 0x7f45e410b870 2024-05-09T02:11:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.421+0000 7f45e930f640 1 --2- 172.21.15.19:0/44786992 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45d00b2a60 0x7f45d00b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.421+0000 7f45e930f640 1 --2- 172.21.15.19:0/44786992 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45d00b2a60 0x7f45d00b4f70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f45e0031440 tx=0x7f45e0009560 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.427+0000 7f45da7fc640 1 -- 172.21.15.19:0/44786992 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45d40aa450 con 0x7f45e410b870 2024-05-09T02:11:28.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.647+0000 7f45eb573640 1 -- 172.21.15.19:0/44786992 --> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f45e4101940 con 0x7f45d00b2a60 2024-05-09T02:11:28.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.656+0000 7f45da7fc640 1 -- 172.21.15.19:0/44786992 <== mgr.15222 v2:172.21.15.19:6800/2638922161 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f45e4101940 con 0x7f45d00b2a60 2024-05-09T02:11:28.658 INFO:teuthology.orchestra.run.smithi019.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-09T00:00:00", "created": "2024-05-09T02:11:27", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-09T02:11:28.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.658+0000 7f45eb573640 1 -- 172.21.15.19:0/44786992 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45d00b2a60 msgr2=0x7f45d00b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.658+0000 7f45eb573640 1 --2- 172.21.15.19:0/44786992 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45d00b2a60 0x7f45d00b4f70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f45e0031440 tx=0x7f45e0009560 comp rx=0 tx=0).stop 2024-05-09T02:11:28.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.658+0000 7f45eb573640 1 -- 172.21.15.19:0/44786992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e410b870 msgr2=0x7f45e41a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.658+0000 7f45eb573640 1 --2- 172.21.15.19:0/44786992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e410b870 0x7f45e41a0aa0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f45d400e9f0 tx=0x7f45d400edf0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.658+0000 7f45eb573640 1 -- 172.21.15.19:0/44786992 shutdown_connections 2024-05-09T02:11:28.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.658+0000 7f45eb573640 1 --2- 172.21.15.19:0/44786992 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45d00b2a60 0x7f45d00b4f70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.658+0000 7f45eb573640 1 --2- 172.21.15.19:0/44786992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45e410e220 0x7f45e419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.658+0000 7f45eb573640 1 --2- 172.21.15.19:0/44786992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e410b870 0x7f45e41a0aa0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.658+0000 7f45eb573640 1 --2- 172.21.15.19:0/44786992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45e4104e10 0x7f45e41a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.658+0000 7f45eb573640 1 -- 172.21.15.19:0/44786992 >> 172.21.15.19:0/44786992 conn(0x7f45e40792a0 msgr2=0x7f45e4101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:28.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.658+0000 7f45eb573640 1 -- 172.21.15.19:0/44786992 shutdown_connections 2024-05-09T02:11:28.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.658+0000 7f45eb573640 1 -- 172.21.15.19:0/44786992 wait complete. 2024-05-09T02:11:28.672 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-09T02:11:28.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.788+0000 7f7f21a27640 1 -- 172.21.15.19:0/3119140139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f1c10b810 msgr2=0x7f7f1c10dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.788+0000 7f7f21a27640 1 --2- 172.21.15.19:0/3119140139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f1c10b810 0x7f7f1c10dc30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009900 tx=0x7f7f0402a410 comp rx=0 tx=0).stop 2024-05-09T02:11:28.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.788+0000 7f7f21a27640 1 -- 172.21.15.19:0/3119140139 shutdown_connections 2024-05-09T02:11:28.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.788+0000 7f7f21a27640 1 --2- 172.21.15.19:0/3119140139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c10e200 0x7f7f1c1106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.788+0000 7f7f21a27640 1 --2- 172.21.15.19:0/3119140139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f1c10b810 0x7f7f1c10dc30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.788+0000 7f7f21a27640 1 --2- 172.21.15.19:0/3119140139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c104db0 0x7f7f1c1051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.788+0000 7f7f21a27640 1 -- 172.21.15.19:0/3119140139 >> 172.21.15.19:0/3119140139 conn(0x7f7f1c078280 msgr2=0x7f7f1c0786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:28.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.788+0000 7f7f21a27640 1 -- 172.21.15.19:0/3119140139 shutdown_connections 2024-05-09T02:11:28.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.788+0000 7f7f21a27640 1 -- 172.21.15.19:0/3119140139 wait complete. 2024-05-09T02:11:28.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.788+0000 7f7f21a27640 1 Processor -- start 2024-05-09T02:11:28.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f21a27640 1 -- start start 2024-05-09T02:11:28.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f21a27640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f1c104db0 0x7f7f1c1a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f21a27640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c10b810 0x7f7f1c1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f21a27640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c10e200 0x7f7f1c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f21a27640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c113e60 con 0x7f7f1c104db0 2024-05-09T02:11:28.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f21a27640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c113ce0 con 0x7f7f1c10e200 2024-05-09T02:11:28.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f21a27640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c113fe0 con 0x7f7f1c10b810 2024-05-09T02:11:28.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f1affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f1c104db0 0x7f7f1c1a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f1affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f1c104db0 0x7f7f1c1a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58294/0 (socket says 172.21.15.19:58294) 2024-05-09T02:11:28.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f1affd640 1 -- 172.21.15.19:0/4060687642 learned_addr learned my addr 172.21.15.19:0/4060687642 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:28.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f1a7fc640 1 --2- 172.21.15.19:0/4060687642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c10b810 0x7f7f1c1a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f1b7fe640 1 --2- 172.21.15.19:0/4060687642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c10e200 0x7f7f1c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f1affd640 1 -- 172.21.15.19:0/4060687642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c10b810 msgr2=0x7f7f1c1a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f1affd640 1 --2- 172.21.15.19:0/4060687642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c10b810 0x7f7f1c1a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f1affd640 1 -- 172.21.15.19:0/4060687642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c10e200 msgr2=0x7f7f1c19a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:28.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f1affd640 1 --2- 172.21.15.19:0/4060687642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c10e200 0x7f7f1c19a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:28.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f1affd640 1 -- 172.21.15.19:0/4060687642 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1c19adb0 con 0x7f7f1c104db0 2024-05-09T02:11:28.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.789+0000 7f7f1a7fc640 1 --2- 172.21.15.19:0/4060687642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c10b810 0x7f7f1c1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:11:28.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.790+0000 7f7f1affd640 1 --2- 172.21.15.19:0/4060687642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f1c104db0 0x7f7f1c1a0500 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400e9f0 tx=0x7f7f1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:28.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.790+0000 7f7f20a25640 1 -- 172.21.15.19:0/4060687642 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400cd80 con 0x7f7f1c104db0 2024-05-09T02:11:28.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.790+0000 7f7f20a25640 1 -- 172.21.15.19:0/4060687642 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f140108e0 con 0x7f7f1c104db0 2024-05-09T02:11:28.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.790+0000 7f7f21a27640 1 -- 172.21.15.19:0/4060687642 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1c19b040 con 0x7f7f1c104db0 2024-05-09T02:11:28.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.790+0000 7f7f20a25640 1 -- 172.21.15.19:0/4060687642 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14018980 con 0x7f7f1c104db0 2024-05-09T02:11:28.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.790+0000 7f7f21a27640 1 -- 172.21.15.19:0/4060687642 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1c1a72a0 con 0x7f7f1c104db0 2024-05-09T02:11:28.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.791+0000 7f7f21a27640 1 -- 172.21.15.19:0/4060687642 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f1c06bc00 con 0x7f7f1c104db0 2024-05-09T02:11:28.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.792+0000 7f7f20a25640 1 -- 172.21.15.19:0/4060687642 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f7f14010430 con 0x7f7f1c104db0 2024-05-09T02:11:28.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.794+0000 7f7f20a25640 1 --2- 172.21.15.19:0/4060687642 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7f080b29e0 0x7f7f080b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:28.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.794+0000 7f7f1a7fc640 1 --2- 172.21.15.19:0/4060687642 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7f080b29e0 0x7f7f080b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:28.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.794+0000 7f7f20a25640 1 -- 172.21.15.19:0/4060687642 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f7f140e3110 con 0x7f7f1c104db0 2024-05-09T02:11:28.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.794+0000 7f7f1a7fc640 1 --2- 172.21.15.19:0/4060687642 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7f080b29e0 0x7f7f080b4ef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7f04031440 tx=0x7f7f04009680 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:28.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:28.798+0000 7f7f20a25640 1 -- 172.21.15.19:0/4060687642 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f140aa980 con 0x7f7f1c104db0 2024-05-09T02:11:29.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:29.012+0000 7f7f21a27640 1 -- 172.21.15.19:0/4060687642 --> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7f1c101a80 con 0x7f7f080b29e0 2024-05-09T02:11:29.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:29.019+0000 7f7f20a25640 1 -- 172.21.15.19:0/4060687642 <== mgr.15222 v2:172.21.15.19:6800/2638922161 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7f1c101a80 con 0x7f7f080b29e0 2024-05-09T02:11:29.022 INFO:teuthology.orchestra.run.smithi019.stdout:/ 1m 6m3h 2024-05-09T02:11:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:29.021+0000 7f7f21a27640 1 -- 172.21.15.19:0/4060687642 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7f080b29e0 msgr2=0x7f7f080b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:29.021+0000 7f7f21a27640 1 --2- 172.21.15.19:0/4060687642 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7f080b29e0 0x7f7f080b4ef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7f04031440 tx=0x7f7f04009680 comp rx=0 tx=0).stop 2024-05-09T02:11:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:29.021+0000 7f7f21a27640 1 -- 172.21.15.19:0/4060687642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f1c104db0 msgr2=0x7f7f1c1a0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:29.021+0000 7f7f21a27640 1 --2- 172.21.15.19:0/4060687642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f1c104db0 0x7f7f1c1a0500 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400e9f0 tx=0x7f7f1400eeb0 comp rx=0 tx=0).stop 2024-05-09T02:11:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:29.021+0000 7f7f21a27640 1 -- 172.21.15.19:0/4060687642 shutdown_connections 2024-05-09T02:11:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:29.021+0000 7f7f21a27640 1 --2- 172.21.15.19:0/4060687642 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7f080b29e0 0x7f7f080b4ef0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:29.021+0000 7f7f21a27640 1 --2- 172.21.15.19:0/4060687642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c10e200 0x7f7f1c19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:29.021+0000 7f7f21a27640 1 --2- 172.21.15.19:0/4060687642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c10b810 0x7f7f1c1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:29.021+0000 7f7f21a27640 1 --2- 172.21.15.19:0/4060687642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f1c104db0 0x7f7f1c1a0500 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:29.021+0000 7f7f21a27640 1 -- 172.21.15.19:0/4060687642 >> 172.21.15.19:0/4060687642 conn(0x7f7f1c078280 msgr2=0x7f7f1c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:29.022+0000 7f7f21a27640 1 -- 172.21.15.19:0/4060687642 shutdown_connections 2024-05-09T02:11:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:29.022+0000 7f7f21a27640 1 -- 172.21.15.19:0/4060687642 wait complete. 2024-05-09T02:11:29.035 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-09T02:11:29.062 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-09T02:11:29.074 INFO:teuthology.run_tasks:Running task exec... 2024-05-09T02:11:29.114 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T02:11:29.114 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T02:11:29.115 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-05-09T02:11:29.118 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:11:29.195 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:29.210 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem Size Used Avail Use% Mounted on 2024-05-09T02:11:29.210 INFO:teuthology.orchestra.run.smithi019.stdout:0@a0611f3e-0da7-11ef-bc97-c7b262605968.cephfs=/volumes/qa/sv_1/3263a7e6-90e1-4c3c-b293-25c3c473ace0 340G 0 340G 0% /tmp/cephfs 2024-05-09T02:11:29.214 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-05-09T02:11:29.287 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:29 smithi019 ceph-mon[30151]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:11:29.287 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:29 smithi019 ceph-mon[30151]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:11:29.288 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:29 smithi019 ceph-mon[30151]: scrub queued for path: / 2024-05-09T02:11:29.288 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:29 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:11:29.288 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:29 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:11:29.288 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:29 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:11:29.308 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-05-09T02:11:29.394 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-05-09T02:11:29.420 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:11:29.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:29 smithi059 ceph-mon[34781]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:11:29.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:29 smithi059 ceph-mon[34781]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:11:29.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:29 smithi059 ceph-mon[34781]: scrub queued for path: / 2024-05-09T02:11:29.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:29 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:11:29.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:29 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:11:29.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:29 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:11:29.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:29 smithi196 ceph-mon[34718]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:11:29.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:29 smithi196 ceph-mon[34718]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:11:29.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:29 smithi196 ceph-mon[34718]: scrub queued for path: / 2024-05-09T02:11:29.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:29 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:11:29.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:29 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:11:29.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:29 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:11:30.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:30 smithi059 ceph-mon[34781]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:11:30.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:30 smithi059 ceph-mon[34781]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:11:30.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:30 smithi059 ceph-mon[34781]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 60 KiB/s wr, 32 op/s 2024-05-09T02:11:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:30 smithi196 ceph-mon[34718]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:11:30.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:30 smithi196 ceph-mon[34718]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:11:30.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:30 smithi196 ceph-mon[34718]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 60 KiB/s wr, 32 op/s 2024-05-09T02:11:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:30 smithi019 ceph-mon[30151]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:11:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:30 smithi019 ceph-mon[30151]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:11:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:30 smithi019 ceph-mon[30151]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 60 KiB/s wr, 32 op/s 2024-05-09T02:11:31.326 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-05-09T02:11:31.404 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-05-09T02:11:31.406 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-05-09T02:11:31.485 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-09T02:11:31.562 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/: 2024-05-09T02:11:31.562 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:31.562 INFO:teuthology.orchestra.run.smithi019.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 . 2024-05-09T02:11:31.562 INFO:teuthology.orchestra.run.smithi019.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 .. 2024-05-09T02:11:31.562 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 data 2024-05-09T02:11:31.562 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:31.562 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data: 2024-05-09T02:11:31.562 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:31.563 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:11:31.563 INFO:teuthology.orchestra.run.smithi019.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 .. 2024-05-09T02:11:31.564 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-05-09T02:11:31.650 INFO:teuthology.orchestra.run.smithi019.stdout:[Service] 2024-05-09T02:11:31.650 INFO:teuthology.orchestra.run.smithi019.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-05-09T02:11:31.650 INFO:teuthology.orchestra.run.smithi019.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-05-09T02:11:31.653 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-05-09T02:11:31.753 INFO:teuthology.orchestra.run.smithi019.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-05-09T02:11:32.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.287+0000 7fae69ac5700 1 -- 172.21.15.19:0/2767256700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae64105dd0 msgr2=0x7fae6410e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:32.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.287+0000 7fae69ac5700 1 --2- 172.21.15.19:0/2767256700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae64105dd0 0x7fae6410e320 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fae58009990 tx=0x7fae58009ca0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.287+0000 7fae69ac5700 1 -- 172.21.15.19:0/2767256700 shutdown_connections 2024-05-09T02:11:32.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.287+0000 7fae69ac5700 1 --2- 172.21.15.19:0/2767256700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae6410e8f0 0x7fae64110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.287+0000 7fae69ac5700 1 --2- 172.21.15.19:0/2767256700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae64105480 0x7fae64105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.287+0000 7fae69ac5700 1 --2- 172.21.15.19:0/2767256700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae64105dd0 0x7fae6410e320 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.287+0000 7fae69ac5700 1 -- 172.21.15.19:0/2767256700 >> 172.21.15.19:0/2767256700 conn(0x7fae640fef30 msgr2=0x7fae64101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:32.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.287+0000 7fae69ac5700 1 -- 172.21.15.19:0/2767256700 shutdown_connections 2024-05-09T02:11:32.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.287+0000 7fae69ac5700 1 -- 172.21.15.19:0/2767256700 wait complete. 2024-05-09T02:11:32.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.288+0000 7fae69ac5700 1 Processor -- start 2024-05-09T02:11:32.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.288+0000 7fae69ac5700 1 -- start start 2024-05-09T02:11:32.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.288+0000 7fae69ac5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae64105480 0x7fae6419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.288+0000 7fae69ac5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae64105dd0 0x7fae6419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.288+0000 7fae69ac5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae6410e8f0 0x7fae6419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.288+0000 7fae69ac5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae64114470 con 0x7fae64105dd0 2024-05-09T02:11:32.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.288+0000 7fae69ac5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae641142f0 con 0x7fae64105480 2024-05-09T02:11:32.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.288+0000 7fae69ac5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae641145f0 con 0x7fae6410e8f0 2024-05-09T02:11:32.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.288+0000 7fae62ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae64105dd0 0x7fae6419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.289+0000 7fae62ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae64105dd0 0x7fae6419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58304/0 (socket says 172.21.15.19:58304) 2024-05-09T02:11:32.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.289+0000 7fae62ffd700 1 -- 172.21.15.19:0/141271294 learned_addr learned my addr 172.21.15.19:0/141271294 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:32.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.289+0000 7fae63fff700 1 --2- 172.21.15.19:0/141271294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae6410e8f0 0x7fae6419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.289+0000 7fae637fe700 1 --2- 172.21.15.19:0/141271294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae64105480 0x7fae6419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.289+0000 7fae62ffd700 1 -- 172.21.15.19:0/141271294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae6410e8f0 msgr2=0x7fae6419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:32.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.289+0000 7fae62ffd700 1 --2- 172.21.15.19:0/141271294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae6410e8f0 0x7fae6419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.289+0000 7fae62ffd700 1 -- 172.21.15.19:0/141271294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae64105480 msgr2=0x7fae6419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:32.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.289+0000 7fae62ffd700 1 --2- 172.21.15.19:0/141271294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae64105480 0x7fae6419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.289+0000 7fae62ffd700 1 -- 172.21.15.19:0/141271294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6419fcd0 con 0x7fae64105dd0 2024-05-09T02:11:32.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.289+0000 7fae62ffd700 1 --2- 172.21.15.19:0/141271294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae64105dd0 0x7fae6419b3a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fae58000c00 tx=0x7fae58009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:32.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.289+0000 7fae60ff9700 1 -- 172.21.15.19:0/141271294 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58027070 con 0x7fae64105dd0 2024-05-09T02:11:32.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.289+0000 7fae60ff9700 1 -- 172.21.15.19:0/141271294 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae58022470 con 0x7fae64105dd0 2024-05-09T02:11:32.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.289+0000 7fae69ac5700 1 -- 172.21.15.19:0/141271294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6419ff60 con 0x7fae64105dd0 2024-05-09T02:11:32.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.289+0000 7fae60ff9700 1 -- 172.21.15.19:0/141271294 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58017650 con 0x7fae64105dd0 2024-05-09T02:11:32.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.290+0000 7fae69ac5700 1 -- 172.21.15.19:0/141271294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae641a0440 con 0x7fae64105dd0 2024-05-09T02:11:32.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.292+0000 7fae60ff9700 1 -- 172.21.15.19:0/141271294 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fae5802a070 con 0x7fae64105dd0 2024-05-09T02:11:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.293+0000 7fae60ff9700 1 --2- 172.21.15.19:0/141271294 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae4c0b2a80 0x7fae4c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.293+0000 7fae60ff9700 1 -- 172.21.15.19:0/141271294 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fae580e3590 con 0x7fae64105dd0 2024-05-09T02:11:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.293+0000 7fae637fe700 1 --2- 172.21.15.19:0/141271294 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae4c0b2a80 0x7fae4c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.293+0000 7fae637fe700 1 --2- 172.21.15.19:0/141271294 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae4c0b2a80 0x7fae4c0b4f90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fae54004630 tx=0x7fae5400b3c0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae784dbb80 1 Processor -- start 2024-05-09T02:11:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae784dbb80 1 -- start start 2024-05-09T02:11:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae784dbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ad2c160 0x55880adc54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae784dbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ac424a0 0x55880adc5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae784dbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 0x55880adbfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae784dbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55880ace1ff0 con 0x55880ad2c160 2024-05-09T02:11:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae784dbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55880ac807f0 con 0x55880adbf6a0 2024-05-09T02:11:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae784dbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55880acddfb0 con 0x55880ac424a0 2024-05-09T02:11:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae637fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ad2c160 0x55880adc54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae62ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ac424a0 0x55880adc5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae637fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ad2c160 0x55880adc54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58320/0 (socket says 172.21.15.19:58320) 2024-05-09T02:11:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae637fe700 1 -- 172.21.15.19:0/1418876020 learned_addr learned my addr 172.21.15.19:0/1418876020 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae62ffd700 1 -- 172.21.15.19:0/1418876020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 msgr2=0x55880adbfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae62ffd700 1 --2- 172.21.15.19:0/1418876020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 0x55880adbfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae62ffd700 1 -- 172.21.15.19:0/1418876020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ad2c160 msgr2=0x55880adc54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae62ffd700 1 --2- 172.21.15.19:0/1418876020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ad2c160 0x55880adc54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.294+0000 7fae62ffd700 1 -- 172.21.15.19:0/1418876020 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55880ac78370 con 0x55880ac424a0 2024-05-09T02:11:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.295+0000 7fae62ffd700 1 --2- 172.21.15.19:0/1418876020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ac424a0 0x55880adc5a30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fae580e0c20 tx=0x7fae58009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfaa25fc78a20247 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.295+0000 7fae49ffb700 1 -- 172.21.15.19:0/1418876020 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae580aa2a0 con 0x55880ac424a0 2024-05-09T02:11:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.295+0000 7fae49ffb700 1 -- 172.21.15.19:0/1418876020 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae580aae90 con 0x55880ac424a0 2024-05-09T02:11:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.295+0000 7fae49ffb700 1 -- 172.21.15.19:0/1418876020 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae580e10f0 con 0x55880ac424a0 2024-05-09T02:11:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.296+0000 7fae784dbb80 1 -- 172.21.15.19:0/1418876020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ac424a0 msgr2=0x55880adc5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.296+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1418876020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ac424a0 0x55880adc5a30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fae580e0c20 tx=0x7fae58009ca0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.296+0000 7fae784dbb80 1 -- 172.21.15.19:0/1418876020 shutdown_connections 2024-05-09T02:11:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.296+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1418876020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 0x55880adbfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.296+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1418876020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ad2c160 0x55880adc54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.296+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1418876020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ac424a0 0x55880adc5a30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.296+0000 7fae784dbb80 1 -- 172.21.15.19:0/1418876020 >> 172.21.15.19:0/1418876020 conn(0x55880aca26a0 msgr2=0x55880adba400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.296+0000 7fae784dbb80 1 -- 172.21.15.19:0/1418876020 shutdown_connections 2024-05-09T02:11:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.296+0000 7fae784dbb80 1 -- 172.21.15.19:0/1418876020 wait complete. 2024-05-09T02:11:32.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.296+0000 7fae784dbb80 1 -- start start 2024-05-09T02:11:32.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.296+0000 7fae784dbb80 1 Processor -- start 2024-05-09T02:11:32.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae784dbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ac424a0 0x55880adced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae784dbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ad2c160 0x55880adcf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae784dbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 0x55880adc9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae784dbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55880ab5dba0 con 0x55880ac424a0 2024-05-09T02:11:32.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae784dbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55880ac9c150 con 0x55880adbf6a0 2024-05-09T02:11:32.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae784dbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55880adc0360 con 0x55880ad2c160 2024-05-09T02:11:32.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae784dbb80 10 client.0 _ll_register_callbacks cb 0x55880ab536c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:11:32.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae637fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ac424a0 0x55880adced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae62ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ad2c160 0x55880adcf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae637fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ac424a0 0x55880adced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58324/0 (socket says 172.21.15.19:58324) 2024-05-09T02:11:32.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae637fe700 1 -- 172.21.15.19:0/1483595931 learned_addr learned my addr 172.21.15.19:0/1483595931 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:32.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae62ffd700 1 -- 172.21.15.19:0/1483595931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 msgr2=0x55880adc9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:32.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae63fff700 1 --2- 172.21.15.19:0/1483595931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 0x55880adc9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae62ffd700 1 --2- 172.21.15.19:0/1483595931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 0x55880adc9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae62ffd700 1 -- 172.21.15.19:0/1483595931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ac424a0 msgr2=0x55880adced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:32.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae62ffd700 1 --2- 172.21.15.19:0/1483595931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ac424a0 0x55880adced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae62ffd700 1 -- 172.21.15.19:0/1483595931 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55880ab5d490 con 0x55880ad2c160 2024-05-09T02:11:32.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae637fe700 1 --2- 172.21.15.19:0/1483595931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ac424a0 0x55880adced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:32.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.297+0000 7fae63fff700 1 --2- 172.21.15.19:0/1483595931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 0x55880adc9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:11:32.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae62ffd700 1 --2- 172.21.15.19:0/1483595931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ad2c160 0x55880adcf2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fae580aa440 tx=0x7fae58092340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d449945d043e261 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae42ffd700 1 -- 172.21.15.19:0/1483595931 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae580aa2a0 con 0x55880ad2c160 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae42ffd700 1 -- 172.21.15.19:0/1483595931 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fae580aac00 con 0x55880ad2c160 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae784dbb80 1 -- 172.21.15.19:0/1483595931 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae580aae90 con 0x55880ad2c160 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae42ffd700 1 -- 172.21.15.19:0/1483595931 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fae580aeb40 con 0x55880ad2c160 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae784dbb80 10 client.25123 fetch_fsmap learned FSMap version 29 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae784dbb80 1 -- 172.21.15.19:0/1483595931 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55880add69a0 con 0x55880ad2c160 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae42ffd700 1 -- 172.21.15.19:0/1483595931 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae580aae90 con 0x55880ad2c160 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae42ffd700 1 -- 172.21.15.19:0/1483595931 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fae580a99f0 con 0x55880ad2c160 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae784dbb80 10 client.25123 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae784dbb80 10 client.25123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae784dbb80 20 client.25123 populate_metadata read hostname 'smithi019' 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1483595931 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55880add8ca0 0x55880addb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae784dbb80 4 client.25123 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.298+0000 7fae784dbb80 1 -- 172.21.15.19:0/1483595931 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55880acef9d0 con 0x55880add8ca0 2024-05-09T02:11:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.299+0000 7fae637fe700 1 --2- 172.21.15.19:0/1483595931 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55880add8ca0 0x55880addb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.299+0000 7fae42ffd700 10 client.25123 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:11:32.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.299+0000 7fae637fe700 1 --2- 172.21.15.19:0/1483595931 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55880add8ca0 0x55880addb0c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5f312223a8e516c server_cookie=8da03c7f6f49182c in_seq=0 out_seq=0 2024-05-09T02:11:32.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.300+0000 7fae42ffd700 1 -- 172.21.15.19:0/1483595931 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55880acef9d0 con 0x55880add8ca0 2024-05-09T02:11:32.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.300+0000 7fae42ffd700 10 client.25123 handle_command_reply: tid=0 2024-05-09T02:11:32.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.300+0000 7fae784dbb80 1 client.25123 shutdown 2024-05-09T02:11:32.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.301+0000 7fae784dbb80 1 -- 172.21.15.19:0/1483595931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ad2c160 msgr2=0x55880adcf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:32.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.301+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1483595931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ad2c160 0x55880adcf2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fae580aa440 tx=0x7fae58092340 comp rx=0 tx=0).stop 2024-05-09T02:11:32.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.301+0000 7fae784dbb80 1 -- 172.21.15.19:0/1483595931 shutdown_connections 2024-05-09T02:11:32.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.301+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1483595931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 0x55880adc9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.301+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1483595931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ac424a0 0x55880adced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.301+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1483595931 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55880add8ca0 0x55880addb0c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.301+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1483595931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ad2c160 0x55880adcf2c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.301+0000 7fae784dbb80 1 -- 172.21.15.19:0/1483595931 >> 172.21.15.19:0/1483595931 conn(0x55880aca26a0 msgr2=0x55880adba1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:32.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.301+0000 7fae42ffd700 0 client.25123 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:11:32.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.301+0000 7fae784dbb80 1 -- 172.21.15.19:0/1483595931 shutdown_connections 2024-05-09T02:11:32.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.301+0000 7fae784dbb80 1 -- 172.21.15.19:0/1483595931 wait complete. 2024-05-09T02:11:32.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.301+0000 7fae784dbb80 20 client.25123 trim_cache size 0 max 16384 2024-05-09T02:11:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.321+0000 7fae784dbb80 1 Processor -- start 2024-05-09T02:11:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae784dbb80 1 -- start start 2024-05-09T02:11:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae784dbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ac424a0 0x55880adc8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae784dbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ad2c160 0x55880adc4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae784dbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 0x55880adc2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae784dbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae641aeda0 con 0x55880ac424a0 2024-05-09T02:11:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae784dbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae641b08e0 con 0x55880adbf6a0 2024-05-09T02:11:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae784dbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae641af470 con 0x55880ad2c160 2024-05-09T02:11:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae637fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ac424a0 0x55880adc8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae637fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ac424a0 0x55880adc8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58340/0 (socket says 172.21.15.19:58340) 2024-05-09T02:11:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae637fe700 1 -- 172.21.15.19:0/3543449790 learned_addr learned my addr 172.21.15.19:0/3543449790 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae62ffd700 1 --2- 172.21.15.19:0/3543449790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ad2c160 0x55880adc4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae63fff700 1 --2- 172.21.15.19:0/3543449790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 0x55880adc2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae62ffd700 1 -- 172.21.15.19:0/3543449790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 msgr2=0x55880adc2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae62ffd700 1 --2- 172.21.15.19:0/3543449790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 0x55880adc2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae62ffd700 1 -- 172.21.15.19:0/3543449790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ac424a0 msgr2=0x55880adc8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:32.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae62ffd700 1 --2- 172.21.15.19:0/3543449790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ac424a0 0x55880adc8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae62ffd700 1 -- 172.21.15.19:0/3543449790 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55880adc7710 con 0x55880ad2c160 2024-05-09T02:11:32.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae637fe700 1 --2- 172.21.15.19:0/3543449790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ac424a0 0x55880adc8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:32.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.322+0000 7fae63fff700 1 --2- 172.21.15.19:0/3543449790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 0x55880adc2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:11:32.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.323+0000 7fae62ffd700 1 --2- 172.21.15.19:0/3543449790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ad2c160 0x55880adc4980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae580e12e0 tx=0x7fae58009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d38e4be9f7fe1922 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:32.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.328+0000 7fae497fa700 1 -- 172.21.15.19:0/3543449790 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae580aae90 con 0x55880ad2c160 2024-05-09T02:11:32.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.328+0000 7fae497fa700 1 -- 172.21.15.19:0/3543449790 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae580aa790 con 0x55880ad2c160 2024-05-09T02:11:32.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.328+0000 7fae497fa700 1 -- 172.21.15.19:0/3543449790 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae580a6380 con 0x55880ad2c160 2024-05-09T02:11:32.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.329+0000 7fae784dbb80 1 -- 172.21.15.19:0/3543449790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ad2c160 msgr2=0x55880adc4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:32.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.329+0000 7fae784dbb80 1 --2- 172.21.15.19:0/3543449790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ad2c160 0x55880adc4980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae580e12e0 tx=0x7fae58009310 comp rx=0 tx=0).stop 2024-05-09T02:11:32.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.329+0000 7fae784dbb80 1 -- 172.21.15.19:0/3543449790 shutdown_connections 2024-05-09T02:11:32.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.329+0000 7fae784dbb80 1 --2- 172.21.15.19:0/3543449790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880adbf6a0 0x55880adc2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.329+0000 7fae784dbb80 1 --2- 172.21.15.19:0/3543449790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ac424a0 0x55880adc8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.329+0000 7fae784dbb80 1 --2- 172.21.15.19:0/3543449790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880ad2c160 0x55880adc4980 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.329+0000 7fae784dbb80 1 -- 172.21.15.19:0/3543449790 >> 172.21.15.19:0/3543449790 conn(0x55880aca26a0 msgr2=0x55880adc4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:32.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.329+0000 7fae784dbb80 1 -- 172.21.15.19:0/3543449790 shutdown_connections 2024-05-09T02:11:32.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.329+0000 7fae784dbb80 1 -- 172.21.15.19:0/3543449790 wait complete. 2024-05-09T02:11:32.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.330+0000 7fae784dbb80 1 -- start start 2024-05-09T02:11:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.330+0000 7fae784dbb80 1 Processor -- start 2024-05-09T02:11:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.330+0000 7fae784dbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880ac424a0 0x55880add4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.330+0000 7fae784dbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ad2c160 0x55880add54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.330+0000 7fae784dbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880adbf6a0 0x55880ae3ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.330+0000 7fae784dbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae641b0210 con 0x55880ad2c160 2024-05-09T02:11:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.330+0000 7fae784dbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae641afb40 con 0x55880ac424a0 2024-05-09T02:11:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae784dbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae641acd40 con 0x55880adbf6a0 2024-05-09T02:11:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae62ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ad2c160 0x55880add54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae62ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ad2c160 0x55880add54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58350/0 (socket says 172.21.15.19:58350) 2024-05-09T02:11:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae62ffd700 1 -- 172.21.15.19:0/1977387900 learned_addr learned my addr 172.21.15.19:0/1977387900 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:11:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae63fff700 1 --2- 172.21.15.19:0/1977387900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880adbf6a0 0x55880ae3ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae784dbb80 10 client.0 _ll_register_callbacks cb 0x55880ab536c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:11:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae637fe700 1 --2- 172.21.15.19:0/1977387900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880ac424a0 0x55880add4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae62ffd700 1 -- 172.21.15.19:0/1977387900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880adbf6a0 msgr2=0x55880ae3ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae62ffd700 1 --2- 172.21.15.19:0/1977387900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880adbf6a0 0x55880ae3ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae62ffd700 1 -- 172.21.15.19:0/1977387900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880ac424a0 msgr2=0x55880add4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:11:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae62ffd700 1 --2- 172.21.15.19:0/1977387900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880ac424a0 0x55880add4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae62ffd700 1 -- 172.21.15.19:0/1977387900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55880ae3f250 con 0x55880ad2c160 2024-05-09T02:11:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae62ffd700 1 --2- 172.21.15.19:0/1977387900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ad2c160 0x55880add54f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fae580a9e90 tx=0x7fae580aa160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc14d946495c760 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:11:32.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:11:32.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae42ffd700 1 -- 172.21.15.19:0/1977387900 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae580aae90 con 0x55880ad2c160 2024-05-09T02:11:32.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae42ffd700 1 -- 172.21.15.19:0/1977387900 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fae580b30d0 con 0x55880ad2c160 2024-05-09T02:11:32.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.331+0000 7fae784dbb80 1 -- 172.21.15.19:0/1977387900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae580aa790 con 0x55880ad2c160 2024-05-09T02:11:32.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.332+0000 7fae42ffd700 1 -- 172.21.15.19:0/1977387900 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fae580b3560 con 0x55880ad2c160 2024-05-09T02:11:32.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.332+0000 7fae784dbb80 10 client.15417 fetch_fsmap learned FSMap version 29 2024-05-09T02:11:32.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.332+0000 7fae784dbb80 1 -- 172.21.15.19:0/1977387900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55880ae3f250 con 0x55880ad2c160 2024-05-09T02:11:32.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.332+0000 7fae42ffd700 1 -- 172.21.15.19:0/1977387900 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae58084e50 con 0x55880ad2c160 2024-05-09T02:11:32.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.332+0000 7fae42ffd700 1 -- 172.21.15.19:0/1977387900 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fae580abac0 con 0x55880ad2c160 2024-05-09T02:11:32.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.332+0000 7fae784dbb80 10 client.15417 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T02:11:32.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.332+0000 7fae784dbb80 10 client.15417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:11:32.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.332+0000 7fae784dbb80 20 client.15417 populate_metadata read hostname 'smithi019' 2024-05-09T02:11:32.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.332+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1977387900 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55880ae47a00 0x55880ae49e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:11:32.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.332+0000 7fae784dbb80 4 client.15417 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:11:32.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.332+0000 7fae784dbb80 1 -- 172.21.15.19:0/1977387900 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55880ace9a70 con 0x55880ae47a00 2024-05-09T02:11:32.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.332+0000 7fae637fe700 1 --2- 172.21.15.19:0/1977387900 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55880ae47a00 0x55880ae49e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:11:32.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.333+0000 7fae42ffd700 10 client.15417 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:11:32.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.333+0000 7fae637fe700 1 --2- 172.21.15.19:0/1977387900 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55880ae47a00 0x55880ae49e20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc0c0d37264258b0 server_cookie=ce999e68d47ae9c3 in_seq=0 out_seq=0 2024-05-09T02:11:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.333+0000 7fae42ffd700 1 -- 172.21.15.19:0/1977387900 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55880ace9a70 con 0x55880ae47a00 2024-05-09T02:11:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.333+0000 7fae42ffd700 10 client.15417 handle_command_reply: tid=0 2024-05-09T02:11:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.333+0000 7fae784dbb80 1 client.15417 shutdown 2024-05-09T02:11:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.333+0000 7fae784dbb80 1 -- 172.21.15.19:0/1977387900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ad2c160 msgr2=0x55880add54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.333+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1977387900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ad2c160 0x55880add54f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fae580a9e90 tx=0x7fae580aa160 comp rx=0 tx=0).stop 2024-05-09T02:11:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.334+0000 7fae784dbb80 1 -- 172.21.15.19:0/1977387900 shutdown_connections 2024-05-09T02:11:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.334+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1977387900 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55880ae47a00 0x55880ae49e20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.334+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1977387900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55880adbf6a0 0x55880ae3ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.334+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1977387900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55880ac424a0 0x55880add4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.334+0000 7fae42ffd700 0 client.15417 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:11:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.334+0000 7fae784dbb80 1 --2- 172.21.15.19:0/1977387900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55880ad2c160 0x55880add54f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.334+0000 7fae784dbb80 1 -- 172.21.15.19:0/1977387900 >> 172.21.15.19:0/1977387900 conn(0x55880aca26a0 msgr2=0x55880adc2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.334+0000 7fae784dbb80 1 -- 172.21.15.19:0/1977387900 shutdown_connections 2024-05-09T02:11:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.334+0000 7fae784dbb80 1 -- 172.21.15.19:0/1977387900 wait complete. 2024-05-09T02:11:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.334+0000 7fae784dbb80 20 client.15417 trim_cache size 0 max 16384 2024-05-09T02:11:32.340 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:11:32.340 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T02:11:32.340 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T02:11:32.341 INFO:teuthology.orchestra.run.smithi019.stdout: "c7502900-31f3-4cce-9d82-727e7fa3c05d": { 2024-05-09T02:11:32.341 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T02:11:32.341 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "c7502900-31f3-4cce-9d82-727e7fa3c05d", 2024-05-09T02:11:32.341 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T02:11:32.341 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:11:32.341 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:11:32.341 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:11:32.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.334+0000 7fae69ac5700 1 -- 172.21.15.19:0/141271294 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae4c0b2a80 msgr2=0x7fae4c0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:32.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.334+0000 7fae69ac5700 1 --2- 172.21.15.19:0/141271294 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae4c0b2a80 0x7fae4c0b4f90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fae54004630 tx=0x7fae5400b3c0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.334+0000 7fae69ac5700 1 -- 172.21.15.19:0/141271294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae64105dd0 msgr2=0x7fae6419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:11:32.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.334+0000 7fae69ac5700 1 --2- 172.21.15.19:0/141271294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae64105dd0 0x7fae6419b3a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fae58000c00 tx=0x7fae58009580 comp rx=0 tx=0).stop 2024-05-09T02:11:32.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.335+0000 7fae69ac5700 1 -- 172.21.15.19:0/141271294 shutdown_connections 2024-05-09T02:11:32.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.335+0000 7fae69ac5700 1 --2- 172.21.15.19:0/141271294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae6410e8f0 0x7fae6419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.335+0000 7fae69ac5700 1 --2- 172.21.15.19:0/141271294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae64105480 0x7fae6419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.335+0000 7fae69ac5700 1 --2- 172.21.15.19:0/141271294 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae4c0b2a80 0x7fae4c0b4f90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.335+0000 7fae69ac5700 1 --2- 172.21.15.19:0/141271294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae64105dd0 0x7fae6419b3a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:11:32.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.335+0000 7fae69ac5700 1 -- 172.21.15.19:0/141271294 >> 172.21.15.19:0/141271294 conn(0x7fae640fef30 msgr2=0x7fae640ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:11:32.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.335+0000 7fae69ac5700 1 -- 172.21.15.19:0/141271294 shutdown_connections 2024-05-09T02:11:32.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:11:32.335+0000 7fae69ac5700 1 -- 172.21.15.19:0/141271294 wait complete. 2024-05-09T02:11:32.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:32 smithi019 ceph-mon[30151]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-05-09T02:11:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:32 smithi059 ceph-mon[34781]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-05-09T02:11:32.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:32 smithi196 ceph-mon[34718]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-05-09T02:11:33.070 INFO:tasks.cephfs.filesystem:scrub status for tag:c7502900-31f3-4cce-9d82-727e7fa3c05d - {'path': '/', 'tag': 'c7502900-31f3-4cce-9d82-727e7fa3c05d', 'options': 'recursive,force'} 2024-05-09T02:11:34.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:34 smithi019 ceph-mon[30151]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-05-09T02:11:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:34 smithi059 ceph-mon[34781]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-05-09T02:11:34.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:34 smithi196 ceph-mon[34718]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-05-09T02:11:36.366 INFO:teuthology.orchestra.run.smithi019.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-05-09T02:11:36.370 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-09T02:11:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/: 2024-05-09T02:11:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 . 2024-05-09T02:11:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 .. 2024-05-09T02:11:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 data 2024-05-09T02:11:36.414 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data: 2024-05-09T02:11:36.414 INFO:teuthology.orchestra.run.smithi019.stdout:total 35 2024-05-09T02:11:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 . 2024-05-09T02:11:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 .. 2024-05-09T02:11:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 base 2024-05-09T02:11:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 May 9 02:11 global 2024-05-09T02:11:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 log 2024-05-09T02:11:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_commit_ts 2024-05-09T02:11:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_dynshmem 2024-05-09T02:11:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 9 02:11 pg_hba.conf 2024-05-09T02:11:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 9 02:11 pg_ident.conf 2024-05-09T02:11:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 pg_logical 2024-05-09T02:11:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 9 02:11 pg_multixact 2024-05-09T02:11:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_notify 2024-05-09T02:11:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_replslot 2024-05-09T02:11:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_serial 2024-05-09T02:11:36.416 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_snapshots 2024-05-09T02:11:36.416 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_stat 2024-05-09T02:11:36.416 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_stat_tmp 2024-05-09T02:11:36.416 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 pg_subtrans 2024-05-09T02:11:36.416 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_tblspc 2024-05-09T02:11:36.416 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_twophase 2024-05-09T02:11:36.416 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 PG_VERSION 2024-05-09T02:11:36.416 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 9 02:11 pg_wal 2024-05-09T02:11:36.416 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 pg_xact 2024-05-09T02:11:36.416 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 9 02:11 postgresql.auto.conf 2024-05-09T02:11:36.416 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 9 02:11 postgresql.conf 2024-05-09T02:11:36.416 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.416 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/base: 2024-05-09T02:11:36.416 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.417 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 . 2024-05-09T02:11:36.417 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.417 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 9 02:11 1 2024-05-09T02:11:36.417 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 9 02:11 13493 2024-05-09T02:11:36.417 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 9 02:11 13494 2024-05-09T02:11:36.417 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.417 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-09T02:11:36.417 INFO:teuthology.orchestra.run.smithi019.stdout:total 7777 2024-05-09T02:11:36.417 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 9 02:11 . 2024-05-09T02:11:36.417 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 .. 2024-05-09T02:11:36.417 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 112 2024-05-09T02:11:36.417 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 113 2024-05-09T02:11:36.417 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 02:11 1247 2024-05-09T02:11:36.417 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1247_fsm 2024-05-09T02:11:36.418 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1247_vm 2024-05-09T02:11:36.418 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 9 02:11 1249 2024-05-09T02:11:36.418 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1249_fsm 2024-05-09T02:11:36.418 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1249_vm 2024-05-09T02:11:36.418 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 9 02:11 1255 2024-05-09T02:11:36.418 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1255_fsm 2024-05-09T02:11:36.418 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1255_vm 2024-05-09T02:11:36.418 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 9 02:11 1259 2024-05-09T02:11:36.418 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1259_fsm 2024-05-09T02:11:36.418 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1259_vm 2024-05-09T02:11:36.418 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 9 02:11 13344 2024-05-09T02:11:36.418 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13344_fsm 2024-05-09T02:11:36.418 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13344_vm 2024-05-09T02:11:36.419 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13346 2024-05-09T02:11:36.419 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13348 2024-05-09T02:11:36.419 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13349 2024-05-09T02:11:36.419 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13349_fsm 2024-05-09T02:11:36.419 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13349_vm 2024-05-09T02:11:36.419 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13351 2024-05-09T02:11:36.419 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13353 2024-05-09T02:11:36.419 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13354 2024-05-09T02:11:36.419 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13354_fsm 2024-05-09T02:11:36.419 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13354_vm 2024-05-09T02:11:36.419 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13356 2024-05-09T02:11:36.419 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13358 2024-05-09T02:11:36.420 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13359 2024-05-09T02:11:36.420 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13359_fsm 2024-05-09T02:11:36.420 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13359_vm 2024-05-09T02:11:36.420 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13361 2024-05-09T02:11:36.420 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13363 2024-05-09T02:11:36.420 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 1417 2024-05-09T02:11:36.420 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 1418 2024-05-09T02:11:36.420 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 174 2024-05-09T02:11:36.420 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 175 2024-05-09T02:11:36.420 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2187 2024-05-09T02:11:36.420 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2224 2024-05-09T02:11:36.420 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2328 2024-05-09T02:11:36.421 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2336 2024-05-09T02:11:36.421 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2337 2024-05-09T02:11:36.421 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2579 2024-05-09T02:11:36.421 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2600 2024-05-09T02:11:36.421 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2600_fsm 2024-05-09T02:11:36.421 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2600_vm 2024-05-09T02:11:36.421 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2601 2024-05-09T02:11:36.421 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2601_fsm 2024-05-09T02:11:36.421 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2601_vm 2024-05-09T02:11:36.421 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2602 2024-05-09T02:11:36.421 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2602_fsm 2024-05-09T02:11:36.421 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2602_vm 2024-05-09T02:11:36.421 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2603 2024-05-09T02:11:36.422 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2603_fsm 2024-05-09T02:11:36.422 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2603_vm 2024-05-09T02:11:36.422 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2604 2024-05-09T02:11:36.422 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2605 2024-05-09T02:11:36.422 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2605_fsm 2024-05-09T02:11:36.422 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2605_vm 2024-05-09T02:11:36.422 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2606 2024-05-09T02:11:36.422 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2606_fsm 2024-05-09T02:11:36.422 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2606_vm 2024-05-09T02:11:36.422 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2607 2024-05-09T02:11:36.422 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2607_fsm 2024-05-09T02:11:36.422 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2607_vm 2024-05-09T02:11:36.422 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 9 02:11 2608 2024-05-09T02:11:36.422 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2608_fsm 2024-05-09T02:11:36.423 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2608_vm 2024-05-09T02:11:36.423 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 9 02:11 2609 2024-05-09T02:11:36.423 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2609_fsm 2024-05-09T02:11:36.423 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2609_vm 2024-05-09T02:11:36.423 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2610 2024-05-09T02:11:36.423 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2610_fsm 2024-05-09T02:11:36.423 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2610_vm 2024-05-09T02:11:36.423 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2611 2024-05-09T02:11:36.423 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2612 2024-05-09T02:11:36.423 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2612_fsm 2024-05-09T02:11:36.423 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2612_vm 2024-05-09T02:11:36.423 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2613 2024-05-09T02:11:36.423 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2615 2024-05-09T02:11:36.424 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2615_fsm 2024-05-09T02:11:36.424 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2615_vm 2024-05-09T02:11:36.424 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2616 2024-05-09T02:11:36.424 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2616_fsm 2024-05-09T02:11:36.424 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2616_vm 2024-05-09T02:11:36.424 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 9 02:11 2617 2024-05-09T02:11:36.424 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2617_fsm 2024-05-09T02:11:36.424 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2617_vm 2024-05-09T02:11:36.424 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 9 02:11 2618 2024-05-09T02:11:36.424 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2618_fsm 2024-05-09T02:11:36.424 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2618_vm 2024-05-09T02:11:36.424 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 9 02:11 2619 2024-05-09T02:11:36.424 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2619_fsm 2024-05-09T02:11:36.425 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2619_vm 2024-05-09T02:11:36.425 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2620 2024-05-09T02:11:36.425 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2650 2024-05-09T02:11:36.425 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2651 2024-05-09T02:11:36.425 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2652 2024-05-09T02:11:36.425 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2653 2024-05-09T02:11:36.425 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2654 2024-05-09T02:11:36.425 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2655 2024-05-09T02:11:36.425 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2656 2024-05-09T02:11:36.425 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2657 2024-05-09T02:11:36.426 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 9 02:11 2658 2024-05-09T02:11:36.426 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 02:11 2659 2024-05-09T02:11:36.426 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2660 2024-05-09T02:11:36.426 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2661 2024-05-09T02:11:36.426 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2662 2024-05-09T02:11:36.426 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2663 2024-05-09T02:11:36.426 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2664 2024-05-09T02:11:36.426 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2665 2024-05-09T02:11:36.426 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2666 2024-05-09T02:11:36.426 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2667 2024-05-09T02:11:36.426 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2668 2024-05-09T02:11:36.426 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2669 2024-05-09T02:11:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2670 2024-05-09T02:11:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 9 02:11 2673 2024-05-09T02:11:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 9 02:11 2674 2024-05-09T02:11:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 9 02:11 2675 2024-05-09T02:11:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2678 2024-05-09T02:11:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2679 2024-05-09T02:11:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2680 2024-05-09T02:11:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2681 2024-05-09T02:11:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2682 2024-05-09T02:11:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2683 2024-05-09T02:11:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2684 2024-05-09T02:11:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2685 2024-05-09T02:11:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2686 2024-05-09T02:11:36.427 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2687 2024-05-09T02:11:36.428 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2688 2024-05-09T02:11:36.428 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2689 2024-05-09T02:11:36.428 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 9 02:11 2690 2024-05-09T02:11:36.428 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 9 02:11 2691 2024-05-09T02:11:36.428 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2692 2024-05-09T02:11:36.428 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2693 2024-05-09T02:11:36.428 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2696 2024-05-09T02:11:36.428 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2699 2024-05-09T02:11:36.428 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2701 2024-05-09T02:11:36.428 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2702 2024-05-09T02:11:36.428 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2703 2024-05-09T02:11:36.428 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2704 2024-05-09T02:11:36.428 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2753 2024-05-09T02:11:36.428 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2753_fsm 2024-05-09T02:11:36.428 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2753_vm 2024-05-09T02:11:36.429 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2754 2024-05-09T02:11:36.429 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2755 2024-05-09T02:11:36.429 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2756 2024-05-09T02:11:36.429 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2757 2024-05-09T02:11:36.429 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2830 2024-05-09T02:11:36.429 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2831 2024-05-09T02:11:36.429 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2832 2024-05-09T02:11:36.429 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2833 2024-05-09T02:11:36.429 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2834 2024-05-09T02:11:36.429 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2835 2024-05-09T02:11:36.429 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2836 2024-05-09T02:11:36.429 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2837 2024-05-09T02:11:36.429 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 9 02:11 2838 2024-05-09T02:11:36.429 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2838_fsm 2024-05-09T02:11:36.430 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2838_vm 2024-05-09T02:11:36.430 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2839 2024-05-09T02:11:36.430 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2840 2024-05-09T02:11:36.430 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2840_fsm 2024-05-09T02:11:36.430 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2840_vm 2024-05-09T02:11:36.430 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2841 2024-05-09T02:11:36.430 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2995 2024-05-09T02:11:36.430 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2996 2024-05-09T02:11:36.430 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3079 2024-05-09T02:11:36.430 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3079_fsm 2024-05-09T02:11:36.430 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3079_vm 2024-05-09T02:11:36.430 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3080 2024-05-09T02:11:36.430 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3081 2024-05-09T02:11:36.430 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 3085 2024-05-09T02:11:36.431 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3118 2024-05-09T02:11:36.431 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3119 2024-05-09T02:11:36.431 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 9 02:11 3164 2024-05-09T02:11:36.431 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3256 2024-05-09T02:11:36.431 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3257 2024-05-09T02:11:36.431 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3258 2024-05-09T02:11:36.431 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3350 2024-05-09T02:11:36.431 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3351 2024-05-09T02:11:36.431 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3379 2024-05-09T02:11:36.431 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3380 2024-05-09T02:11:36.431 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3381 2024-05-09T02:11:36.431 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3394 2024-05-09T02:11:36.431 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3394_fsm 2024-05-09T02:11:36.431 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3394_vm 2024-05-09T02:11:36.431 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3395 2024-05-09T02:11:36.432 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3429 2024-05-09T02:11:36.432 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3430 2024-05-09T02:11:36.432 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3431 2024-05-09T02:11:36.433 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3433 2024-05-09T02:11:36.433 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3439 2024-05-09T02:11:36.433 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3440 2024-05-09T02:11:36.433 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 3455 2024-05-09T02:11:36.433 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 9 02:11 3456 2024-05-09T02:11:36.433 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3456_fsm 2024-05-09T02:11:36.434 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3456_vm 2024-05-09T02:11:36.434 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3466 2024-05-09T02:11:36.434 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3467 2024-05-09T02:11:36.434 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3468 2024-05-09T02:11:36.434 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3501 2024-05-09T02:11:36.434 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3502 2024-05-09T02:11:36.435 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3503 2024-05-09T02:11:36.435 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3534 2024-05-09T02:11:36.435 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3541 2024-05-09T02:11:36.436 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3541_fsm 2024-05-09T02:11:36.436 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3541_vm 2024-05-09T02:11:36.436 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3542 2024-05-09T02:11:36.436 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3574 2024-05-09T02:11:36.436 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3575 2024-05-09T02:11:36.436 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3576 2024-05-09T02:11:36.437 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3596 2024-05-09T02:11:36.437 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3597 2024-05-09T02:11:36.437 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3598 2024-05-09T02:11:36.437 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3599 2024-05-09T02:11:36.437 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3600 2024-05-09T02:11:36.437 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3600_fsm 2024-05-09T02:11:36.438 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3600_vm 2024-05-09T02:11:36.438 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3601 2024-05-09T02:11:36.438 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3601_fsm 2024-05-09T02:11:36.438 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3601_vm 2024-05-09T02:11:36.438 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3602 2024-05-09T02:11:36.438 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3602_fsm 2024-05-09T02:11:36.439 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3602_vm 2024-05-09T02:11:36.439 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3603 2024-05-09T02:11:36.439 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3603_fsm 2024-05-09T02:11:36.439 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3603_vm 2024-05-09T02:11:36.439 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3604 2024-05-09T02:11:36.439 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3605 2024-05-09T02:11:36.440 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3606 2024-05-09T02:11:36.440 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3607 2024-05-09T02:11:36.440 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3608 2024-05-09T02:11:36.440 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 3609 2024-05-09T02:11:36.441 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3712 2024-05-09T02:11:36.441 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3764 2024-05-09T02:11:36.441 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3764_fsm 2024-05-09T02:11:36.441 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3764_vm 2024-05-09T02:11:36.441 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3766 2024-05-09T02:11:36.441 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3767 2024-05-09T02:11:36.442 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3997 2024-05-09T02:11:36.442 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4143 2024-05-09T02:11:36.442 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4144 2024-05-09T02:11:36.442 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4145 2024-05-09T02:11:36.442 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4146 2024-05-09T02:11:36.442 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4147 2024-05-09T02:11:36.442 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4148 2024-05-09T02:11:36.443 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4149 2024-05-09T02:11:36.443 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4150 2024-05-09T02:11:36.443 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4151 2024-05-09T02:11:36.443 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4152 2024-05-09T02:11:36.443 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4153 2024-05-09T02:11:36.443 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4154 2024-05-09T02:11:36.444 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4155 2024-05-09T02:11:36.444 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4156 2024-05-09T02:11:36.444 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4157 2024-05-09T02:11:36.444 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4158 2024-05-09T02:11:36.445 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4159 2024-05-09T02:11:36.445 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4160 2024-05-09T02:11:36.445 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4161 2024-05-09T02:11:36.445 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4162 2024-05-09T02:11:36.445 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4163 2024-05-09T02:11:36.445 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4164 2024-05-09T02:11:36.446 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4165 2024-05-09T02:11:36.450 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4166 2024-05-09T02:11:36.450 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4167 2024-05-09T02:11:36.450 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4168 2024-05-09T02:11:36.450 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4169 2024-05-09T02:11:36.450 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4170 2024-05-09T02:11:36.450 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4171 2024-05-09T02:11:36.450 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4172 2024-05-09T02:11:36.450 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4173 2024-05-09T02:11:36.450 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4174 2024-05-09T02:11:36.450 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 5002 2024-05-09T02:11:36.450 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 548 2024-05-09T02:11:36.450 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 549 2024-05-09T02:11:36.451 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6102 2024-05-09T02:11:36.451 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6104 2024-05-09T02:11:36.451 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6106 2024-05-09T02:11:36.451 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6110 2024-05-09T02:11:36.451 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6111 2024-05-09T02:11:36.451 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6112 2024-05-09T02:11:36.451 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6113 2024-05-09T02:11:36.452 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6117 2024-05-09T02:11:36.452 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 826 2024-05-09T02:11:36.452 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 827 2024-05-09T02:11:36.452 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 828 2024-05-09T02:11:36.452 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 02:11 pg_filenode.map 2024-05-09T02:11:36.452 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 PG_VERSION 2024-05-09T02:11:36.453 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.453 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-09T02:11:36.453 INFO:teuthology.orchestra.run.smithi019.stdout:total 7777 2024-05-09T02:11:36.453 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 9 02:11 . 2024-05-09T02:11:36.453 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 .. 2024-05-09T02:11:36.453 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 112 2024-05-09T02:11:36.453 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 113 2024-05-09T02:11:36.453 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 02:11 1247 2024-05-09T02:11:36.453 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1247_fsm 2024-05-09T02:11:36.453 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1247_vm 2024-05-09T02:11:36.453 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 9 02:11 1249 2024-05-09T02:11:36.454 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1249_fsm 2024-05-09T02:11:36.454 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1249_vm 2024-05-09T02:11:36.454 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 9 02:11 1255 2024-05-09T02:11:36.454 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1255_fsm 2024-05-09T02:11:36.454 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1255_vm 2024-05-09T02:11:36.454 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 9 02:11 1259 2024-05-09T02:11:36.454 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1259_fsm 2024-05-09T02:11:36.454 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1259_vm 2024-05-09T02:11:36.454 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 9 02:11 13344 2024-05-09T02:11:36.455 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13344_fsm 2024-05-09T02:11:36.455 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13344_vm 2024-05-09T02:11:36.455 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13346 2024-05-09T02:11:36.455 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13348 2024-05-09T02:11:36.455 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13349 2024-05-09T02:11:36.455 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13349_fsm 2024-05-09T02:11:36.455 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13349_vm 2024-05-09T02:11:36.456 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13351 2024-05-09T02:11:36.456 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13353 2024-05-09T02:11:36.456 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13354 2024-05-09T02:11:36.456 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13354_fsm 2024-05-09T02:11:36.456 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13354_vm 2024-05-09T02:11:36.456 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13356 2024-05-09T02:11:36.456 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13358 2024-05-09T02:11:36.456 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13359 2024-05-09T02:11:36.456 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13359_fsm 2024-05-09T02:11:36.456 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13359_vm 2024-05-09T02:11:36.456 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13361 2024-05-09T02:11:36.456 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13363 2024-05-09T02:11:36.457 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 1417 2024-05-09T02:11:36.457 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 1418 2024-05-09T02:11:36.457 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 174 2024-05-09T02:11:36.457 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 175 2024-05-09T02:11:36.457 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2187 2024-05-09T02:11:36.457 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2224 2024-05-09T02:11:36.457 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2328 2024-05-09T02:11:36.457 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2336 2024-05-09T02:11:36.457 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2337 2024-05-09T02:11:36.457 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2579 2024-05-09T02:11:36.457 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2600 2024-05-09T02:11:36.457 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2600_fsm 2024-05-09T02:11:36.457 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2600_vm 2024-05-09T02:11:36.457 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2601 2024-05-09T02:11:36.457 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2601_fsm 2024-05-09T02:11:36.458 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2601_vm 2024-05-09T02:11:36.458 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2602 2024-05-09T02:11:36.458 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2602_fsm 2024-05-09T02:11:36.458 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2602_vm 2024-05-09T02:11:36.458 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2603 2024-05-09T02:11:36.458 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2603_fsm 2024-05-09T02:11:36.458 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2603_vm 2024-05-09T02:11:36.458 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2604 2024-05-09T02:11:36.458 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2605 2024-05-09T02:11:36.458 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2605_fsm 2024-05-09T02:11:36.458 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2605_vm 2024-05-09T02:11:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2606 2024-05-09T02:11:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2606_fsm 2024-05-09T02:11:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2606_vm 2024-05-09T02:11:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2607 2024-05-09T02:11:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2607_fsm 2024-05-09T02:11:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2607_vm 2024-05-09T02:11:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 9 02:11 2608 2024-05-09T02:11:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2608_fsm 2024-05-09T02:11:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2608_vm 2024-05-09T02:11:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 9 02:11 2609 2024-05-09T02:11:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2609_fsm 2024-05-09T02:11:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2609_vm 2024-05-09T02:11:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2610 2024-05-09T02:11:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2610_fsm 2024-05-09T02:11:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2610_vm 2024-05-09T02:11:36.460 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2611 2024-05-09T02:11:36.460 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2612 2024-05-09T02:11:36.460 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2612_fsm 2024-05-09T02:11:36.460 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2612_vm 2024-05-09T02:11:36.460 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2613 2024-05-09T02:11:36.460 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2615 2024-05-09T02:11:36.460 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2615_fsm 2024-05-09T02:11:36.460 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2615_vm 2024-05-09T02:11:36.460 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2616 2024-05-09T02:11:36.460 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2616_fsm 2024-05-09T02:11:36.460 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2616_vm 2024-05-09T02:11:36.460 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 9 02:11 2617 2024-05-09T02:11:36.460 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2617_fsm 2024-05-09T02:11:36.460 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2617_vm 2024-05-09T02:11:36.461 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 9 02:11 2618 2024-05-09T02:11:36.461 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2618_fsm 2024-05-09T02:11:36.461 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2618_vm 2024-05-09T02:11:36.461 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 9 02:11 2619 2024-05-09T02:11:36.461 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2619_fsm 2024-05-09T02:11:36.461 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2619_vm 2024-05-09T02:11:36.461 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2620 2024-05-09T02:11:36.461 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2650 2024-05-09T02:11:36.461 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2651 2024-05-09T02:11:36.461 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2652 2024-05-09T02:11:36.461 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2653 2024-05-09T02:11:36.461 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2654 2024-05-09T02:11:36.461 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2655 2024-05-09T02:11:36.461 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2656 2024-05-09T02:11:36.461 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2657 2024-05-09T02:11:36.462 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 9 02:11 2658 2024-05-09T02:11:36.462 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 02:11 2659 2024-05-09T02:11:36.462 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2660 2024-05-09T02:11:36.462 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2661 2024-05-09T02:11:36.462 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2662 2024-05-09T02:11:36.462 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2663 2024-05-09T02:11:36.462 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2664 2024-05-09T02:11:36.462 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2665 2024-05-09T02:11:36.462 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2666 2024-05-09T02:11:36.462 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2667 2024-05-09T02:11:36.462 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2668 2024-05-09T02:11:36.462 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2669 2024-05-09T02:11:36.462 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2670 2024-05-09T02:11:36.462 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 9 02:11 2673 2024-05-09T02:11:36.463 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 9 02:11 2674 2024-05-09T02:11:36.463 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 9 02:11 2675 2024-05-09T02:11:36.463 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2678 2024-05-09T02:11:36.463 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2679 2024-05-09T02:11:36.463 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2680 2024-05-09T02:11:36.463 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2681 2024-05-09T02:11:36.463 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2682 2024-05-09T02:11:36.463 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2683 2024-05-09T02:11:36.463 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2684 2024-05-09T02:11:36.463 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2685 2024-05-09T02:11:36.463 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2686 2024-05-09T02:11:36.463 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2687 2024-05-09T02:11:36.463 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2688 2024-05-09T02:11:36.463 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2689 2024-05-09T02:11:36.464 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 9 02:11 2690 2024-05-09T02:11:36.464 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 9 02:11 2691 2024-05-09T02:11:36.464 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2692 2024-05-09T02:11:36.464 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2693 2024-05-09T02:11:36.464 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2696 2024-05-09T02:11:36.464 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2699 2024-05-09T02:11:36.464 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2701 2024-05-09T02:11:36.464 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2702 2024-05-09T02:11:36.464 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2703 2024-05-09T02:11:36.464 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2704 2024-05-09T02:11:36.464 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2753 2024-05-09T02:11:36.464 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2753_fsm 2024-05-09T02:11:36.464 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2753_vm 2024-05-09T02:11:36.464 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2754 2024-05-09T02:11:36.464 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2755 2024-05-09T02:11:36.465 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2756 2024-05-09T02:11:36.465 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2757 2024-05-09T02:11:36.465 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2830 2024-05-09T02:11:36.465 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2831 2024-05-09T02:11:36.465 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2832 2024-05-09T02:11:36.465 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2833 2024-05-09T02:11:36.465 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2834 2024-05-09T02:11:36.465 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2835 2024-05-09T02:11:36.465 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2836 2024-05-09T02:11:36.465 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2837 2024-05-09T02:11:36.465 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 9 02:11 2838 2024-05-09T02:11:36.465 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2838_fsm 2024-05-09T02:11:36.465 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2838_vm 2024-05-09T02:11:36.465 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2839 2024-05-09T02:11:36.465 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2840 2024-05-09T02:11:36.466 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2840_fsm 2024-05-09T02:11:36.466 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2840_vm 2024-05-09T02:11:36.466 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2841 2024-05-09T02:11:36.466 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2995 2024-05-09T02:11:36.466 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2996 2024-05-09T02:11:36.466 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3079 2024-05-09T02:11:36.466 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3079_fsm 2024-05-09T02:11:36.466 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3079_vm 2024-05-09T02:11:36.466 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3080 2024-05-09T02:11:36.466 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3081 2024-05-09T02:11:36.466 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 3085 2024-05-09T02:11:36.466 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3118 2024-05-09T02:11:36.466 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3119 2024-05-09T02:11:36.466 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 9 02:11 3164 2024-05-09T02:11:36.467 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3256 2024-05-09T02:11:36.467 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3257 2024-05-09T02:11:36.467 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3258 2024-05-09T02:11:36.467 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3350 2024-05-09T02:11:36.467 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3351 2024-05-09T02:11:36.467 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3379 2024-05-09T02:11:36.467 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3380 2024-05-09T02:11:36.467 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3381 2024-05-09T02:11:36.467 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3394 2024-05-09T02:11:36.467 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3394_fsm 2024-05-09T02:11:36.467 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3394_vm 2024-05-09T02:11:36.468 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3395 2024-05-09T02:11:36.468 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3429 2024-05-09T02:11:36.468 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3430 2024-05-09T02:11:36.468 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3431 2024-05-09T02:11:36.468 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3433 2024-05-09T02:11:36.468 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3439 2024-05-09T02:11:36.468 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3440 2024-05-09T02:11:36.468 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 3455 2024-05-09T02:11:36.468 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 9 02:11 3456 2024-05-09T02:11:36.468 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3456_fsm 2024-05-09T02:11:36.468 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3456_vm 2024-05-09T02:11:36.468 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3466 2024-05-09T02:11:36.468 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3467 2024-05-09T02:11:36.468 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3468 2024-05-09T02:11:36.468 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3501 2024-05-09T02:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3502 2024-05-09T02:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3503 2024-05-09T02:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3534 2024-05-09T02:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3541 2024-05-09T02:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3541_fsm 2024-05-09T02:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3541_vm 2024-05-09T02:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3542 2024-05-09T02:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3574 2024-05-09T02:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3575 2024-05-09T02:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3576 2024-05-09T02:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3596 2024-05-09T02:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3597 2024-05-09T02:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3598 2024-05-09T02:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3599 2024-05-09T02:11:36.469 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3600 2024-05-09T02:11:36.470 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3600_fsm 2024-05-09T02:11:36.470 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3600_vm 2024-05-09T02:11:36.470 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3601 2024-05-09T02:11:36.470 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3601_fsm 2024-05-09T02:11:36.470 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3601_vm 2024-05-09T02:11:36.470 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3602 2024-05-09T02:11:36.470 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3602_fsm 2024-05-09T02:11:36.470 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3602_vm 2024-05-09T02:11:36.470 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3603 2024-05-09T02:11:36.470 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3603_fsm 2024-05-09T02:11:36.470 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3603_vm 2024-05-09T02:11:36.470 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3604 2024-05-09T02:11:36.470 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3605 2024-05-09T02:11:36.470 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3606 2024-05-09T02:11:36.471 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3607 2024-05-09T02:11:36.471 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3608 2024-05-09T02:11:36.471 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 3609 2024-05-09T02:11:36.471 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3712 2024-05-09T02:11:36.471 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3764 2024-05-09T02:11:36.471 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3764_fsm 2024-05-09T02:11:36.471 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3764_vm 2024-05-09T02:11:36.471 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3766 2024-05-09T02:11:36.471 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3767 2024-05-09T02:11:36.471 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3997 2024-05-09T02:11:36.471 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4143 2024-05-09T02:11:36.471 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4144 2024-05-09T02:11:36.471 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4145 2024-05-09T02:11:36.471 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4146 2024-05-09T02:11:36.471 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4147 2024-05-09T02:11:36.472 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4148 2024-05-09T02:11:36.472 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4149 2024-05-09T02:11:36.472 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4150 2024-05-09T02:11:36.472 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4151 2024-05-09T02:11:36.472 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4152 2024-05-09T02:11:36.472 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4153 2024-05-09T02:11:36.472 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4154 2024-05-09T02:11:36.472 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4155 2024-05-09T02:11:36.472 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4156 2024-05-09T02:11:36.472 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4157 2024-05-09T02:11:36.472 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4158 2024-05-09T02:11:36.472 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4159 2024-05-09T02:11:36.472 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4160 2024-05-09T02:11:36.472 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4161 2024-05-09T02:11:36.472 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4162 2024-05-09T02:11:36.473 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4163 2024-05-09T02:11:36.473 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4164 2024-05-09T02:11:36.473 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4165 2024-05-09T02:11:36.473 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4166 2024-05-09T02:11:36.473 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4167 2024-05-09T02:11:36.473 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4168 2024-05-09T02:11:36.473 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4169 2024-05-09T02:11:36.473 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4170 2024-05-09T02:11:36.473 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4171 2024-05-09T02:11:36.473 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4172 2024-05-09T02:11:36.473 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4173 2024-05-09T02:11:36.473 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4174 2024-05-09T02:11:36.473 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 5002 2024-05-09T02:11:36.474 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 548 2024-05-09T02:11:36.474 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 549 2024-05-09T02:11:36.474 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6102 2024-05-09T02:11:36.474 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6104 2024-05-09T02:11:36.474 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6106 2024-05-09T02:11:36.474 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6110 2024-05-09T02:11:36.474 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6111 2024-05-09T02:11:36.474 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6112 2024-05-09T02:11:36.474 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6113 2024-05-09T02:11:36.474 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6117 2024-05-09T02:11:36.474 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 826 2024-05-09T02:11:36.474 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 827 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 828 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 02:11 pg_filenode.map 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 PG_VERSION 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout:total 7777 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 9 02:11 . 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 .. 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 112 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 113 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 02:11 1247 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1247_fsm 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1247_vm 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 9 02:11 1249 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1249_fsm 2024-05-09T02:11:36.475 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1249_vm 2024-05-09T02:11:36.476 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 9 02:11 1255 2024-05-09T02:11:36.476 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1255_fsm 2024-05-09T02:11:36.476 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1255_vm 2024-05-09T02:11:36.476 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 9 02:11 1259 2024-05-09T02:11:36.476 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1259_fsm 2024-05-09T02:11:36.476 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1259_vm 2024-05-09T02:11:36.476 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 9 02:11 13344 2024-05-09T02:11:36.476 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13344_fsm 2024-05-09T02:11:36.476 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13344_vm 2024-05-09T02:11:36.476 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13346 2024-05-09T02:11:36.476 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13348 2024-05-09T02:11:36.476 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13349 2024-05-09T02:11:36.476 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13349_fsm 2024-05-09T02:11:36.476 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13349_vm 2024-05-09T02:11:36.476 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13351 2024-05-09T02:11:36.477 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13353 2024-05-09T02:11:36.477 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13354 2024-05-09T02:11:36.477 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13354_fsm 2024-05-09T02:11:36.477 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13354_vm 2024-05-09T02:11:36.477 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13356 2024-05-09T02:11:36.477 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13358 2024-05-09T02:11:36.477 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13359 2024-05-09T02:11:36.477 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13359_fsm 2024-05-09T02:11:36.477 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13359_vm 2024-05-09T02:11:36.477 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13361 2024-05-09T02:11:36.477 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13363 2024-05-09T02:11:36.477 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 1417 2024-05-09T02:11:36.477 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 1418 2024-05-09T02:11:36.477 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 174 2024-05-09T02:11:36.477 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 175 2024-05-09T02:11:36.478 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2187 2024-05-09T02:11:36.478 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2224 2024-05-09T02:11:36.478 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2328 2024-05-09T02:11:36.478 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2336 2024-05-09T02:11:36.478 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2337 2024-05-09T02:11:36.478 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2579 2024-05-09T02:11:36.478 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2600 2024-05-09T02:11:36.478 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2600_fsm 2024-05-09T02:11:36.478 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2600_vm 2024-05-09T02:11:36.478 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2601 2024-05-09T02:11:36.478 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2601_fsm 2024-05-09T02:11:36.478 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2601_vm 2024-05-09T02:11:36.478 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2602 2024-05-09T02:11:36.478 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2602_fsm 2024-05-09T02:11:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2602_vm 2024-05-09T02:11:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2603 2024-05-09T02:11:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2603_fsm 2024-05-09T02:11:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2603_vm 2024-05-09T02:11:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2604 2024-05-09T02:11:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2605 2024-05-09T02:11:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2605_fsm 2024-05-09T02:11:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2605_vm 2024-05-09T02:11:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2606 2024-05-09T02:11:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2606_fsm 2024-05-09T02:11:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2606_vm 2024-05-09T02:11:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2607 2024-05-09T02:11:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2607_fsm 2024-05-09T02:11:36.479 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2607_vm 2024-05-09T02:11:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 9 02:11 2608 2024-05-09T02:11:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2608_fsm 2024-05-09T02:11:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2608_vm 2024-05-09T02:11:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 9 02:11 2609 2024-05-09T02:11:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2609_fsm 2024-05-09T02:11:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2609_vm 2024-05-09T02:11:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2610 2024-05-09T02:11:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2610_fsm 2024-05-09T02:11:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2610_vm 2024-05-09T02:11:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2611 2024-05-09T02:11:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2612 2024-05-09T02:11:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2612_fsm 2024-05-09T02:11:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2612_vm 2024-05-09T02:11:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2613 2024-05-09T02:11:36.480 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2615 2024-05-09T02:11:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2615_fsm 2024-05-09T02:11:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2615_vm 2024-05-09T02:11:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2616 2024-05-09T02:11:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2616_fsm 2024-05-09T02:11:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2616_vm 2024-05-09T02:11:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 9 02:11 2617 2024-05-09T02:11:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2617_fsm 2024-05-09T02:11:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2617_vm 2024-05-09T02:11:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 9 02:11 2618 2024-05-09T02:11:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2618_fsm 2024-05-09T02:11:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2618_vm 2024-05-09T02:11:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 9 02:11 2619 2024-05-09T02:11:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2619_fsm 2024-05-09T02:11:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2619_vm 2024-05-09T02:11:36.481 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2620 2024-05-09T02:11:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2650 2024-05-09T02:11:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2651 2024-05-09T02:11:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2652 2024-05-09T02:11:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2653 2024-05-09T02:11:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2654 2024-05-09T02:11:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2655 2024-05-09T02:11:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2656 2024-05-09T02:11:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2657 2024-05-09T02:11:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 9 02:11 2658 2024-05-09T02:11:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 02:11 2659 2024-05-09T02:11:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2660 2024-05-09T02:11:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2661 2024-05-09T02:11:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2662 2024-05-09T02:11:36.482 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2663 2024-05-09T02:11:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2664 2024-05-09T02:11:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2665 2024-05-09T02:11:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2666 2024-05-09T02:11:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2667 2024-05-09T02:11:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2668 2024-05-09T02:11:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2669 2024-05-09T02:11:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2670 2024-05-09T02:11:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 9 02:11 2673 2024-05-09T02:11:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 9 02:11 2674 2024-05-09T02:11:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 9 02:11 2675 2024-05-09T02:11:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2678 2024-05-09T02:11:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2679 2024-05-09T02:11:36.483 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2680 2024-05-09T02:11:36.484 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2681 2024-05-09T02:11:36.484 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2682 2024-05-09T02:11:36.484 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2683 2024-05-09T02:11:36.484 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2684 2024-05-09T02:11:36.484 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2685 2024-05-09T02:11:36.484 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2686 2024-05-09T02:11:36.484 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2687 2024-05-09T02:11:36.484 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2688 2024-05-09T02:11:36.484 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2689 2024-05-09T02:11:36.484 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 9 02:11 2690 2024-05-09T02:11:36.484 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 9 02:11 2691 2024-05-09T02:11:36.484 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2692 2024-05-09T02:11:36.484 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2693 2024-05-09T02:11:36.485 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2696 2024-05-09T02:11:36.485 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2699 2024-05-09T02:11:36.485 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2701 2024-05-09T02:11:36.485 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2702 2024-05-09T02:11:36.485 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2703 2024-05-09T02:11:36.485 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2704 2024-05-09T02:11:36.485 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2753 2024-05-09T02:11:36.485 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2753_fsm 2024-05-09T02:11:36.485 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2753_vm 2024-05-09T02:11:36.485 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2754 2024-05-09T02:11:36.485 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2755 2024-05-09T02:11:36.485 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2756 2024-05-09T02:11:36.485 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2757 2024-05-09T02:11:36.485 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2830 2024-05-09T02:11:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2831 2024-05-09T02:11:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2832 2024-05-09T02:11:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2833 2024-05-09T02:11:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2834 2024-05-09T02:11:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2835 2024-05-09T02:11:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2836 2024-05-09T02:11:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2837 2024-05-09T02:11:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 9 02:11 2838 2024-05-09T02:11:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2838_fsm 2024-05-09T02:11:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2838_vm 2024-05-09T02:11:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2839 2024-05-09T02:11:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2840 2024-05-09T02:11:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2840_fsm 2024-05-09T02:11:36.486 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2840_vm 2024-05-09T02:11:36.487 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2841 2024-05-09T02:11:36.487 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2995 2024-05-09T02:11:36.487 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2996 2024-05-09T02:11:36.487 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3079 2024-05-09T02:11:36.487 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3079_fsm 2024-05-09T02:11:36.487 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3079_vm 2024-05-09T02:11:36.487 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3080 2024-05-09T02:11:36.487 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3081 2024-05-09T02:11:36.487 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 3085 2024-05-09T02:11:36.487 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3118 2024-05-09T02:11:36.487 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3119 2024-05-09T02:11:36.487 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 9 02:11 3164 2024-05-09T02:11:36.487 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3256 2024-05-09T02:11:36.487 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3257 2024-05-09T02:11:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3258 2024-05-09T02:11:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3350 2024-05-09T02:11:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3351 2024-05-09T02:11:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3379 2024-05-09T02:11:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3380 2024-05-09T02:11:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3381 2024-05-09T02:11:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3394 2024-05-09T02:11:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3394_fsm 2024-05-09T02:11:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3394_vm 2024-05-09T02:11:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3395 2024-05-09T02:11:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3429 2024-05-09T02:11:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3430 2024-05-09T02:11:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3431 2024-05-09T02:11:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3433 2024-05-09T02:11:36.488 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3439 2024-05-09T02:11:36.489 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3440 2024-05-09T02:11:36.489 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 3455 2024-05-09T02:11:36.489 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 9 02:11 3456 2024-05-09T02:11:36.489 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3456_fsm 2024-05-09T02:11:36.489 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3456_vm 2024-05-09T02:11:36.489 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3466 2024-05-09T02:11:36.489 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3467 2024-05-09T02:11:36.489 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3468 2024-05-09T02:11:36.489 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3501 2024-05-09T02:11:36.489 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3502 2024-05-09T02:11:36.489 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3503 2024-05-09T02:11:36.489 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3534 2024-05-09T02:11:36.489 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3541 2024-05-09T02:11:36.489 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3541_fsm 2024-05-09T02:11:36.490 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3541_vm 2024-05-09T02:11:36.490 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3542 2024-05-09T02:11:36.490 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3574 2024-05-09T02:11:36.490 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3575 2024-05-09T02:11:36.490 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3576 2024-05-09T02:11:36.490 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3596 2024-05-09T02:11:36.490 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3597 2024-05-09T02:11:36.490 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3598 2024-05-09T02:11:36.490 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3599 2024-05-09T02:11:36.491 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3600 2024-05-09T02:11:36.491 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3600_fsm 2024-05-09T02:11:36.491 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3600_vm 2024-05-09T02:11:36.491 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3601 2024-05-09T02:11:36.491 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3601_fsm 2024-05-09T02:11:36.491 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3601_vm 2024-05-09T02:11:36.491 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3602 2024-05-09T02:11:36.491 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3602_fsm 2024-05-09T02:11:36.491 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3602_vm 2024-05-09T02:11:36.491 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3603 2024-05-09T02:11:36.491 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3603_fsm 2024-05-09T02:11:36.491 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3603_vm 2024-05-09T02:11:36.491 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3604 2024-05-09T02:11:36.492 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3605 2024-05-09T02:11:36.492 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3606 2024-05-09T02:11:36.492 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3607 2024-05-09T02:11:36.492 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3608 2024-05-09T02:11:36.492 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 3609 2024-05-09T02:11:36.492 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3712 2024-05-09T02:11:36.492 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3764 2024-05-09T02:11:36.492 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3764_fsm 2024-05-09T02:11:36.492 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3764_vm 2024-05-09T02:11:36.492 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3766 2024-05-09T02:11:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3767 2024-05-09T02:11:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3997 2024-05-09T02:11:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4143 2024-05-09T02:11:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4144 2024-05-09T02:11:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4145 2024-05-09T02:11:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4146 2024-05-09T02:11:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4147 2024-05-09T02:11:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4148 2024-05-09T02:11:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4149 2024-05-09T02:11:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4150 2024-05-09T02:11:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4151 2024-05-09T02:11:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4152 2024-05-09T02:11:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4153 2024-05-09T02:11:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4154 2024-05-09T02:11:36.493 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4155 2024-05-09T02:11:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4156 2024-05-09T02:11:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4157 2024-05-09T02:11:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4158 2024-05-09T02:11:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4159 2024-05-09T02:11:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4160 2024-05-09T02:11:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4161 2024-05-09T02:11:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4162 2024-05-09T02:11:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4163 2024-05-09T02:11:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4164 2024-05-09T02:11:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4165 2024-05-09T02:11:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4166 2024-05-09T02:11:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4167 2024-05-09T02:11:36.494 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4168 2024-05-09T02:11:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4169 2024-05-09T02:11:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4170 2024-05-09T02:11:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4171 2024-05-09T02:11:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4172 2024-05-09T02:11:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4173 2024-05-09T02:11:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4174 2024-05-09T02:11:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 5002 2024-05-09T02:11:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 548 2024-05-09T02:11:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 549 2024-05-09T02:11:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6102 2024-05-09T02:11:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6104 2024-05-09T02:11:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6106 2024-05-09T02:11:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6110 2024-05-09T02:11:36.495 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6111 2024-05-09T02:11:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6112 2024-05-09T02:11:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6113 2024-05-09T02:11:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6117 2024-05-09T02:11:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 826 2024-05-09T02:11:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 827 2024-05-09T02:11:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 828 2024-05-09T02:11:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 02:11 pg_filenode.map 2024-05-09T02:11:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 PG_VERSION 2024-05-09T02:11:36.496 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/global: 2024-05-09T02:11:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:total 537 2024-05-09T02:11:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 May 9 02:11 . 2024-05-09T02:11:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.496 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1213 2024-05-09T02:11:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1213_fsm 2024-05-09T02:11:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1213_vm 2024-05-09T02:11:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1214 2024-05-09T02:11:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1214_fsm 2024-05-09T02:11:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1214_vm 2024-05-09T02:11:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 1232 2024-05-09T02:11:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 1233 2024-05-09T02:11:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1260 2024-05-09T02:11:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1260_fsm 2024-05-09T02:11:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1260_vm 2024-05-09T02:11:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1261 2024-05-09T02:11:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1261_fsm 2024-05-09T02:11:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1261_vm 2024-05-09T02:11:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1262 2024-05-09T02:11:36.497 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1262_fsm 2024-05-09T02:11:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1262_vm 2024-05-09T02:11:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2396 2024-05-09T02:11:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2396_fsm 2024-05-09T02:11:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2396_vm 2024-05-09T02:11:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2397 2024-05-09T02:11:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2671 2024-05-09T02:11:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2672 2024-05-09T02:11:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2676 2024-05-09T02:11:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2677 2024-05-09T02:11:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2694 2024-05-09T02:11:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2695 2024-05-09T02:11:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2697 2024-05-09T02:11:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2698 2024-05-09T02:11:36.498 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2846 2024-05-09T02:11:36.499 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2847 2024-05-09T02:11:36.499 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2964 2024-05-09T02:11:36.499 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2965 2024-05-09T02:11:36.499 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2966 2024-05-09T02:11:36.499 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2967 2024-05-09T02:11:36.499 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3592 2024-05-09T02:11:36.499 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3593 2024-05-09T02:11:36.499 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4060 2024-05-09T02:11:36.499 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4061 2024-05-09T02:11:36.499 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4175 2024-05-09T02:11:36.499 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4176 2024-05-09T02:11:36.499 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4177 2024-05-09T02:11:36.499 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4178 2024-05-09T02:11:36.499 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4181 2024-05-09T02:11:36.499 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4182 2024-05-09T02:11:36.500 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4183 2024-05-09T02:11:36.500 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4184 2024-05-09T02:11:36.500 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4185 2024-05-09T02:11:36.500 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4186 2024-05-09T02:11:36.500 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6000 2024-05-09T02:11:36.500 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6001 2024-05-09T02:11:36.500 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6002 2024-05-09T02:11:36.500 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6100 2024-05-09T02:11:36.500 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6114 2024-05-09T02:11:36.500 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6115 2024-05-09T02:11:36.500 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 pg_control 2024-05-09T02:11:36.500 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 02:11 pg_filenode.map 2024-05-09T02:11:36.500 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.500 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/log: 2024-05-09T02:11:36.501 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.501 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:11:36.501 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.501 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.501 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-09T02:11:36.501 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.501 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:11:36.501 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.501 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.501 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-09T02:11:36.501 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.501 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:11:36.501 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.501 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.501 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-09T02:11:36.502 INFO:teuthology.orchestra.run.smithi019.stdout:total 1 2024-05-09T02:11:36.502 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 . 2024-05-09T02:11:36.502 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.502 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 mappings 2024-05-09T02:11:36.502 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 9 02:11 replorigin_checkpoint 2024-05-09T02:11:36.502 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 snapshots 2024-05-09T02:11:36.502 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.502 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-09T02:11:36.502 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.502 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:11:36.502 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 .. 2024-05-09T02:11:36.502 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.502 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-09T02:11:36.502 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.502 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:11:36.503 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 .. 2024-05-09T02:11:36.503 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.503 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-09T02:11:36.503 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.503 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 9 02:11 . 2024-05-09T02:11:36.503 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.503 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 members 2024-05-09T02:11:36.503 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 offsets 2024-05-09T02:11:36.503 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.504 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-09T02:11:36.504 INFO:teuthology.orchestra.run.smithi019.stdout:total 8 2024-05-09T02:11:36.504 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 . 2024-05-09T02:11:36.504 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 9 02:11 .. 2024-05-09T02:11:36.504 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 0000 2024-05-09T02:11:36.504 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.504 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-09T02:11:36.504 INFO:teuthology.orchestra.run.smithi019.stdout:total 8 2024-05-09T02:11:36.504 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 . 2024-05-09T02:11:36.504 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 9 02:11 .. 2024-05-09T02:11:36.504 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 0000 2024-05-09T02:11:36.504 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.504 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-09T02:11:36.505 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.506 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:11:36.506 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.506 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.506 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-09T02:11:36.506 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.506 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:11:36.506 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.506 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.506 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-09T02:11:36.506 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.506 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:11:36.506 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.506 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.507 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-09T02:11:36.507 INFO:teuthology.orchestra.run.smithi019.stdout:total 8 2024-05-09T02:11:36.507 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 . 2024-05-09T02:11:36.507 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.507 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 0000 2024-05-09T02:11:36.507 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.507 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-09T02:11:36.507 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.507 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:11:36.507 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.507 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.507 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-09T02:11:36.507 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.507 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:11:36.507 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.508 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.508 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-09T02:11:36.508 INFO:teuthology.orchestra.run.smithi019.stdout:total 16384 2024-05-09T02:11:36.508 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 9 02:11 . 2024-05-09T02:11:36.508 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.508 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:11 000000010000000000000001 2024-05-09T02:11:36.508 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 archive_status 2024-05-09T02:11:36.508 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.508 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-09T02:11:36.508 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:11:36.508 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:11:36.508 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 9 02:11 .. 2024-05-09T02:11:36.508 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:11:36.508 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-09T02:11:36.508 INFO:teuthology.orchestra.run.smithi019.stdout:total 8 2024-05-09T02:11:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 . 2024-05-09T02:11:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 9 02:11 .. 2024-05-09T02:11:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 0000 2024-05-09T02:11:36.509 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-05-09T02:11:36.592 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:36 smithi019 ceph-mon[30151]: pgmap v12: 97 pgs: 97 active+clean; 2.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 143 KiB/s wr, 54 op/s 2024-05-09T02:11:36.596 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:36 smithi196 ceph-mon[34718]: pgmap v12: 97 pgs: 97 active+clean; 2.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 143 KiB/s wr, 54 op/s 2024-05-09T02:11:36.639 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-05-09T02:11:36.685 INFO:teuthology.orchestra.run.smithi019.stderr:dropping old tables... 2024-05-09T02:11:36.686 INFO:teuthology.orchestra.run.smithi019.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-05-09T02:11:36.686 INFO:teuthology.orchestra.run.smithi019.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-05-09T02:11:36.686 INFO:teuthology.orchestra.run.smithi019.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-05-09T02:11:36.686 INFO:teuthology.orchestra.run.smithi019.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-05-09T02:11:36.686 INFO:teuthology.orchestra.run.smithi019.stderr:creating tables... 2024-05-09T02:11:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:generating data (client-side)... 2024-05-09T02:11:36.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:36 smithi059 ceph-mon[34781]: pgmap v12: 97 pgs: 97 active+clean; 2.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 143 KiB/s wr, 54 op/s 2024-05-09T02:11:36.776 INFO:teuthology.orchestra.run.smithi019.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.89 s) 2024-05-09T02:11:36.925 INFO:teuthology.orchestra.run.smithi019.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.21 s, remaining 3.16 s) 2024-05-09T02:11:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.36 s, remaining 3.51 s) 2024-05-09T02:11:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.44 s, remaining 3.05 s) 2024-05-09T02:11:37.288 INFO:teuthology.orchestra.run.smithi019.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.57 s, remaining 3.10 s) 2024-05-09T02:11:37.379 INFO:teuthology.orchestra.run.smithi019.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.66 s, remaining 2.88 s) 2024-05-09T02:11:37.551 INFO:teuthology.orchestra.run.smithi019.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.84 s, remaining 2.99 s) 2024-05-09T02:11:37.693 INFO:teuthology.orchestra.run.smithi019.stderr:800000 of 3200000 tuples (25%) done (elapsed 0.98 s, remaining 2.94 s) 2024-05-09T02:11:37.799 INFO:teuthology.orchestra.run.smithi019.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.08 s, remaining 2.77 s) 2024-05-09T02:11:37.926 INFO:teuthology.orchestra.run.smithi019.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.21 s, remaining 2.67 s) 2024-05-09T02:11:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.36 s, remaining 2.59 s) 2024-05-09T02:11:38.159 INFO:teuthology.orchestra.run.smithi019.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.44 s, remaining 2.41 s) 2024-05-09T02:11:38.292 INFO:teuthology.orchestra.run.smithi019.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.58 s, remaining 2.31 s) 2024-05-09T02:11:38.467 INFO:teuthology.orchestra.run.smithi019.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.75 s, remaining 2.25 s) 2024-05-09T02:11:38.527 INFO:teuthology.orchestra.run.smithi019.stderr:1500000 of 3200000 tuples (46%) done (elapsed 1.81 s, remaining 2.05 s) 2024-05-09T02:11:38.689 INFO:teuthology.orchestra.run.smithi019.stderr:1600000 of 3200000 tuples (50%) done (elapsed 1.97 s, remaining 1.97 s) 2024-05-09T02:11:38.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:38 smithi059 ceph-mon[34781]: pgmap v13: 97 pgs: 97 active+clean; 2.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 143 KiB/s wr, 43 op/s 2024-05-09T02:11:38.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:38 smithi196 ceph-mon[34718]: pgmap v13: 97 pgs: 97 active+clean; 2.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 143 KiB/s wr, 43 op/s 2024-05-09T02:11:38.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:38 smithi019 ceph-mon[30151]: pgmap v13: 97 pgs: 97 active+clean; 2.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 143 KiB/s wr, 43 op/s 2024-05-09T02:11:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.12 s, remaining 1.87 s) 2024-05-09T02:11:38.918 INFO:teuthology.orchestra.run.smithi019.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.20 s, remaining 1.71 s) 2024-05-09T02:11:39.059 INFO:teuthology.orchestra.run.smithi019.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.34 s, remaining 1.60 s) 2024-05-09T02:11:39.164 INFO:teuthology.orchestra.run.smithi019.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.45 s, remaining 1.47 s) 2024-05-09T02:11:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.59 s, remaining 1.35 s) 2024-05-09T02:11:39.462 INFO:teuthology.orchestra.run.smithi019.stderr:2200000 of 3200000 tuples (68%) done (elapsed 2.75 s, remaining 1.25 s) 2024-05-09T02:11:39.564 INFO:teuthology.orchestra.run.smithi019.stderr:2300000 of 3200000 tuples (71%) done (elapsed 2.85 s, remaining 1.11 s) 2024-05-09T02:11:39.702 INFO:teuthology.orchestra.run.smithi019.stderr:2400000 of 3200000 tuples (75%) done (elapsed 2.99 s, remaining 1.00 s) 2024-05-09T02:11:39.879 INFO:teuthology.orchestra.run.smithi019.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.16 s, remaining 0.89 s) 2024-05-09T02:11:39.986 INFO:teuthology.orchestra.run.smithi019.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.27 s, remaining 0.76 s) 2024-05-09T02:11:40.128 INFO:teuthology.orchestra.run.smithi019.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.41 s, remaining 0.63 s) 2024-05-09T02:11:40.231 INFO:teuthology.orchestra.run.smithi019.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.52 s, remaining 0.50 s) 2024-05-09T02:11:40.380 INFO:teuthology.orchestra.run.smithi019.stderr:2900000 of 3200000 tuples (90%) done (elapsed 3.67 s, remaining 0.38 s) 2024-05-09T02:11:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:3000000 of 3200000 tuples (93%) done (elapsed 3.80 s, remaining 0.25 s) 2024-05-09T02:11:40.629 INFO:teuthology.orchestra.run.smithi019.stderr:3100000 of 3200000 tuples (96%) done (elapsed 3.91 s, remaining 0.13 s) 2024-05-09T02:11:40.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:40 smithi059 ceph-mon[34781]: pgmap v14: 97 pgs: 97 active+clean; 157 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 22 MiB/s wr, 162 op/s 2024-05-09T02:11:40.738 INFO:teuthology.orchestra.run.smithi019.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.02 s, remaining 0.00 s) 2024-05-09T02:11:40.781 INFO:teuthology.orchestra.run.smithi019.stderr:vacuuming... 2024-05-09T02:11:40.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:40 smithi196 ceph-mon[34718]: pgmap v14: 97 pgs: 97 active+clean; 157 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 22 MiB/s wr, 162 op/s 2024-05-09T02:11:40.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:40 smithi019 ceph-mon[30151]: pgmap v14: 97 pgs: 97 active+clean; 157 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 22 MiB/s wr, 162 op/s 2024-05-09T02:11:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:creating primary keys... 2024-05-09T02:11:41.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:41 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:11:41.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:41 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:11:41.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:41 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:11:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:done in 5.91 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 4.08 s, vacuum 0.62 s, primary keys 1.20 s). 2024-05-09T02:11:42.597 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:42 smithi019 ceph-mon[30151]: pgmap v15: 97 pgs: 97 active+clean; 157 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 22 MiB/s wr, 143 op/s 2024-05-09T02:11:42.600 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-05-09T02:11:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:starting vacuum...end. 2024-05-09T02:11:42.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:42 smithi059 ceph-mon[34781]: pgmap v15: 97 pgs: 97 active+clean; 157 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 22 MiB/s wr, 143 op/s 2024-05-09T02:11:42.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:42 smithi196 ceph-mon[34718]: pgmap v15: 97 pgs: 97 active+clean; 157 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 22 MiB/s wr, 143 op/s 2024-05-09T02:11:44.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:44 smithi019 ceph-mon[30151]: pgmap v16: 97 pgs: 97 active+clean; 313 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 46 MiB/s wr, 155 op/s 2024-05-09T02:11:44.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:44 smithi059 ceph-mon[34781]: pgmap v16: 97 pgs: 97 active+clean; 313 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 46 MiB/s wr, 155 op/s 2024-05-09T02:11:44.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:44 smithi196 ceph-mon[34718]: pgmap v16: 97 pgs: 97 active+clean; 313 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 46 MiB/s wr, 155 op/s 2024-05-09T02:11:46.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:46 smithi059 ceph-mon[34781]: pgmap v17: 97 pgs: 97 active+clean; 498 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 74 MiB/s wr, 187 op/s 2024-05-09T02:11:46.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:46 smithi196 ceph-mon[34718]: pgmap v17: 97 pgs: 97 active+clean; 498 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 74 MiB/s wr, 187 op/s 2024-05-09T02:11:46.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:46 smithi019 ceph-mon[30151]: pgmap v17: 97 pgs: 97 active+clean; 498 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 74 MiB/s wr, 187 op/s 2024-05-09T02:11:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 5.0 s, 3467.0 tps, lat 28.324 ms stddev 27.067 2024-05-09T02:11:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:48 smithi059 ceph-mon[34781]: pgmap v18: 97 pgs: 97 active+clean; 498 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 74 MiB/s wr, 163 op/s 2024-05-09T02:11:48.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:48 smithi196 ceph-mon[34718]: pgmap v18: 97 pgs: 97 active+clean; 498 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 74 MiB/s wr, 163 op/s 2024-05-09T02:11:48.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:48 smithi019 ceph-mon[30151]: pgmap v18: 97 pgs: 97 active+clean; 498 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 74 MiB/s wr, 163 op/s 2024-05-09T02:11:50.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:50 smithi059 ceph-mon[34781]: pgmap v19: 97 pgs: 97 active+clean; 867 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 78 MiB/s wr, 372 op/s 2024-05-09T02:11:50.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:50 smithi196 ceph-mon[34718]: pgmap v19: 97 pgs: 97 active+clean; 867 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 78 MiB/s wr, 372 op/s 2024-05-09T02:11:50.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:50 smithi019 ceph-mon[30151]: pgmap v19: 97 pgs: 97 active+clean; 867 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 78 MiB/s wr, 372 op/s 2024-05-09T02:11:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 10.0 s, 3749.6 tps, lat 26.667 ms stddev 26.882 2024-05-09T02:11:52.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:52 smithi059 ceph-mon[34781]: pgmap v20: 97 pgs: 97 active+clean; 867 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 MiB/s wr, 254 op/s 2024-05-09T02:11:52.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:52 smithi196 ceph-mon[34718]: pgmap v20: 97 pgs: 97 active+clean; 867 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 MiB/s wr, 254 op/s 2024-05-09T02:11:52.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:52 smithi019 ceph-mon[30151]: pgmap v20: 97 pgs: 97 active+clean; 867 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 MiB/s wr, 254 op/s 2024-05-09T02:11:54.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:54 smithi059 ceph-mon[34781]: pgmap v21: 97 pgs: 97 active+clean; 891 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 MiB/s wr, 377 op/s 2024-05-09T02:11:54.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:54 smithi196 ceph-mon[34718]: pgmap v21: 97 pgs: 97 active+clean; 891 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 MiB/s wr, 377 op/s 2024-05-09T02:11:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:54 smithi019 ceph-mon[30151]: pgmap v21: 97 pgs: 97 active+clean; 891 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 MiB/s wr, 377 op/s 2024-05-09T02:11:56.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:56 smithi019 ceph-mon[30151]: pgmap v22: 97 pgs: 97 active+clean; 928 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 37 MiB/s wr, 569 op/s 2024-05-09T02:11:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:56 smithi059 ceph-mon[34781]: pgmap v22: 97 pgs: 97 active+clean; 928 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 37 MiB/s wr, 569 op/s 2024-05-09T02:11:56.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:56 smithi196 ceph-mon[34718]: pgmap v22: 97 pgs: 97 active+clean; 928 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 37 MiB/s wr, 569 op/s 2024-05-09T02:11:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 15.0 s, 3744.6 tps, lat 26.690 ms stddev 26.108 2024-05-09T02:11:57.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:57 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:11:57.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:57 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:11:57.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:57 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:11:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:58 smithi059 ceph-mon[34781]: pgmap v23: 97 pgs: 97 active+clean; 928 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 MiB/s wr, 536 op/s 2024-05-09T02:11:58.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:58 smithi196 ceph-mon[34718]: pgmap v23: 97 pgs: 97 active+clean; 928 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 MiB/s wr, 536 op/s 2024-05-09T02:11:58.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:58 smithi019 ceph-mon[30151]: pgmap v23: 97 pgs: 97 active+clean; 928 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 MiB/s wr, 536 op/s 2024-05-09T02:11:59.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:11:59 smithi059 ceph-mon[34781]: pgmap v24: 97 pgs: 97 active+clean; 935 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 855 op/s 2024-05-09T02:11:59.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:11:59 smithi196 ceph-mon[34718]: pgmap v24: 97 pgs: 97 active+clean; 935 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 855 op/s 2024-05-09T02:11:59.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:11:59 smithi019 ceph-mon[30151]: pgmap v24: 97 pgs: 97 active+clean; 935 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 855 op/s 2024-05-09T02:12:00.571 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:12:00 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:12:00.288+0000 7f9bc6a6e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:12:00.571 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:12:00 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:12:00.288+0000 7f9bc6a6e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:12:00.571 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:12:00 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:12:00.288+0000 7f9bc6a6e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:12:00.571 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:12:00 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:12:00.288+0000 7f9bc6a6e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:12:00.571 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:12:00 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:12:00.288+0000 7f9bc6a6e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T02:12:02.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:01 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:12:02.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:01 smithi019 ceph-mon[30151]: pgmap v25: 97 pgs: 97 active+clean; 935 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 646 op/s 2024-05-09T02:12:02.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:01 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:12:02.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:01 smithi059 ceph-mon[34781]: pgmap v25: 97 pgs: 97 active+clean; 935 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 646 op/s 2024-05-09T02:12:02.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:01 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:12:02.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:01 smithi196 ceph-mon[34718]: pgmap v25: 97 pgs: 97 active+clean; 935 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 646 op/s 2024-05-09T02:12:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 20.0 s, 2016.0 tps, lat 49.512 ms stddev 124.539 2024-05-09T02:12:03.071 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:12:03.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:02 smithi019 ceph-mon[30151]: mgrmap e25: x(active, since 44s), standbys: y, z 2024-05-09T02:12:03.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:02 smithi059 ceph-mon[34781]: mgrmap e25: x(active, since 44s), standbys: y, z 2024-05-09T02:12:03.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:02 smithi196 ceph-mon[34718]: mgrmap e25: x(active, since 44s), standbys: y, z 2024-05-09T02:12:03.443 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:12:03.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:03 smithi019 ceph-mon[30151]: pgmap v26: 97 pgs: 97 active+clean; 944 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 20 MiB/s wr, 897 op/s 2024-05-09T02:12:04.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:03 smithi059 ceph-mon[34781]: pgmap v26: 97 pgs: 97 active+clean; 944 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 20 MiB/s wr, 897 op/s 2024-05-09T02:12:04.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:03 smithi196 ceph-mon[34718]: pgmap v26: 97 pgs: 97 active+clean; 944 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 20 MiB/s wr, 897 op/s 2024-05-09T02:12:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.417+0000 7f79db337700 1 -- 172.21.15.19:0/2893674989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d4107df0 msgr2=0x7f79d4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.417+0000 7f79db337700 1 --2- 172.21.15.19:0/2893674989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d4107df0 0x7f79d4108200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f79c40099c0 tx=0x7f79c4009cd0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.417+0000 7f79db337700 1 -- 172.21.15.19:0/2893674989 shutdown_connections 2024-05-09T02:12:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.417+0000 7f79db337700 1 --2- 172.21.15.19:0/2893674989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79d4108740 0x7f79d41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.417+0000 7f79db337700 1 --2- 172.21.15.19:0/2893674989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79d41034c0 0x7f79d41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.417+0000 7f79db337700 1 --2- 172.21.15.19:0/2893674989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d4107df0 0x7f79d4108200 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.417+0000 7f79db337700 1 -- 172.21.15.19:0/2893674989 >> 172.21.15.19:0/2893674989 conn(0x7f79d40fedc0 msgr2=0x7f79d41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.417+0000 7f79db337700 1 -- 172.21.15.19:0/2893674989 shutdown_connections 2024-05-09T02:12:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.417+0000 7f79db337700 1 -- 172.21.15.19:0/2893674989 wait complete. 2024-05-09T02:12:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.418+0000 7f79db337700 1 Processor -- start 2024-05-09T02:12:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.419+0000 7f79db337700 1 -- start start 2024-05-09T02:12:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.419+0000 7f79db337700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d41034c0 0x7f79d4110d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.419+0000 7f79db337700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79d4107df0 0x7f79d4111250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.419+0000 7f79db337700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79d4108740 0x7f79d4111790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.419+0000 7f79db337700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79d41166e0 con 0x7f79d41034c0 2024-05-09T02:12:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.419+0000 7f79db337700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f79d4116560 con 0x7f79d4108740 2024-05-09T02:12:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.419+0000 7f79db337700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79d4116860 con 0x7f79d4107df0 2024-05-09T02:12:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.419+0000 7f79d90d3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d41034c0 0x7f79d4110d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.419+0000 7f79d90d3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d41034c0 0x7f79d4110d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41956/0 (socket says 172.21.15.19:41956) 2024-05-09T02:12:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.419+0000 7f79d90d3700 1 -- 172.21.15.19:0/2414283292 learned_addr learned my addr 172.21.15.19:0/2414283292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.419+0000 7f79d98d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79d4108740 0x7f79d4111790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.419+0000 7f79d88d2700 1 --2- 172.21.15.19:0/2414283292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79d4107df0 0x7f79d4111250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.420+0000 7f79d98d4700 1 -- 172.21.15.19:0/2414283292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79d4107df0 msgr2=0x7f79d4111250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.420+0000 7f79d98d4700 1 --2- 172.21.15.19:0/2414283292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79d4107df0 0x7f79d4111250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.420+0000 7f79d98d4700 1 -- 172.21.15.19:0/2414283292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d41034c0 msgr2=0x7f79d4110d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.420+0000 7f79d98d4700 1 --2- 172.21.15.19:0/2414283292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d41034c0 0x7f79d4110d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.420+0000 7f79d98d4700 1 -- 172.21.15.19:0/2414283292 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d4074400 con 0x7f79d4108740 2024-05-09T02:12:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.420+0000 7f79d88d2700 1 --2- 172.21.15.19:0/2414283292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79d4107df0 0x7f79d4111250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:12:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.421+0000 7f79d98d4700 1 --2- 172.21.15.19:0/2414283292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79d4108740 0x7f79d4111790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00efc0 tx=0x7f79cc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.421+0000 7f79ca7fc700 1 -- 172.21.15.19:0/2414283292 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc0092e0 con 0x7f79d4108740 2024-05-09T02:12:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.421+0000 7f79ca7fc700 1 -- 172.21.15.19:0/2414283292 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79cc004bb0 con 0x7f79d4108740 2024-05-09T02:12:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.421+0000 7f79db337700 1 -- 172.21.15.19:0/2414283292 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d4074690 con 0x7f79d4108740 2024-05-09T02:12:05.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.421+0000 7f79ca7fc700 1 -- 172.21.15.19:0/2414283292 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc004020 con 0x7f79d4108740 2024-05-09T02:12:05.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.421+0000 7f79db337700 1 -- 172.21.15.19:0/2414283292 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d4074ba0 con 0x7f79d4108740 2024-05-09T02:12:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.426+0000 7f79ca7fc700 1 -- 172.21.15.19:0/2414283292 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f79cc01b030 con 0x7f79d4108740 2024-05-09T02:12:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.427+0000 7f79e9d4db80 1 Processor -- start 2024-05-09T02:12:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.428+0000 7f79e9d4db80 1 -- start start 2024-05-09T02:12:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.428+0000 7f79ca7fc700 1 --2- 172.21.15.19:0/2414283292 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79c00b2ca0 0x7f79c00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.428+0000 7f79ca7fc700 1 -- 172.21.15.19:0/2414283292 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f79cc0e2100 con 0x7f79d4108740 2024-05-09T02:12:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.428+0000 7f79e9d4db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a954369d00 0x55a954472240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.428+0000 7f79e9d4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9542ef4a0 0x55a954472780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.428+0000 7f79e9d4db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a95446c400 0x55a95446c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.428+0000 7f79e9d4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a95438f0a0 con 0x55a9542ef4a0 2024-05-09T02:12:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.428+0000 7f79e9d4db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a9543349a0 con 0x55a954369d00 2024-05-09T02:12:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.428+0000 7f79e9d4db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a95438b060 con 0x55a95446c400 2024-05-09T02:12:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.428+0000 7f79d88d2700 1 --2- 172.21.15.19:0/2414283292 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79c00b2ca0 0x7f79c00b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.428+0000 7f79d90d3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a954369d00 0x55a954472240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.428+0000 7f79d88d2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9542ef4a0 0x55a954472780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.428+0000 7f79d90d3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a954369d00 0x55a954472240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:35726/0 (socket says 172.21.15.19:35726) 2024-05-09T02:12:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.428+0000 7f79d90d3700 1 -- 172.21.15.19:0/3230461833 learned_addr learned my addr 172.21.15.19:0/3230461833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.428+0000 7f79d98d4700 1 --2- 172.21.15.19:0/3230461833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a95446c400 0x55a95446c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.429+0000 7f79d90d3700 1 -- 172.21.15.19:0/3230461833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a95446c400 msgr2=0x55a95446c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:05.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.429+0000 7f79d90d3700 1 --2- 172.21.15.19:0/3230461833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a95446c400 0x55a95446c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.429+0000 7f79d90d3700 1 -- 172.21.15.19:0/3230461833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9542ef4a0 msgr2=0x55a954472780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:05.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.429+0000 7f79d90d3700 1 --2- 172.21.15.19:0/3230461833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9542ef4a0 0x55a954472780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.429+0000 7f79d90d3700 1 -- 172.21.15.19:0/3230461833 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a954325500 con 0x55a954369d00 2024-05-09T02:12:05.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.429+0000 7f79d90d3700 1 --2- 172.21.15.19:0/3230461833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a954369d00 0x55a954472240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f79d0007da0 tx=0x7f79d00106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14e0756a64f7c75a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:05.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.429+0000 7f79d88d2700 1 --2- 172.21.15.19:0/2414283292 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79c00b2ca0 0x7f79c00b51b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79c4000c00 tx=0x7f79c40058e0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:05.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.429+0000 7f79bf7fe700 1 -- 172.21.15.19:0/3230461833 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0010e10 con 0x55a954369d00 2024-05-09T02:12:05.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.429+0000 7f79bf7fe700 1 -- 172.21.15.19:0/3230461833 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79d00118f0 con 0x55a954369d00 2024-05-09T02:12:05.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.429+0000 7f79bf7fe700 1 -- 172.21.15.19:0/3230461833 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d00197e0 con 0x55a954369d00 2024-05-09T02:12:05.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.431+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3230461833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a954369d00 msgr2=0x55a954472240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:05.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.431+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3230461833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a954369d00 0x55a954472240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f79d0007da0 tx=0x7f79d00106b0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.431+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3230461833 shutdown_connections 2024-05-09T02:12:05.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.431+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3230461833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a95446c400 0x55a95446c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.431+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3230461833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a954369d00 0x55a954472240 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.431+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3230461833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9542ef4a0 0x55a954472780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.431+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3230461833 >> 172.21.15.19:0/3230461833 conn(0x55a9542f2f70 msgr2=0x55a954467220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:05.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.431+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3230461833 shutdown_connections 2024-05-09T02:12:05.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.431+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3230461833 wait complete. 2024-05-09T02:12:05.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.432+0000 7f79e9d4db80 1 -- start start 2024-05-09T02:12:05.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.432+0000 7f79e9d4db80 1 Processor -- start 2024-05-09T02:12:05.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.433+0000 7f79e9d4db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9542ef4a0 0x55a954475fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.433+0000 7f79e9d4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 0x55a9544764f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.433+0000 7f79e9d4db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a95446c400 0x55a954476a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.433+0000 7f79e9d4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a95446d160 con 0x55a954369d00 2024-05-09T02:12:05.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.433+0000 7f79d88d2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 0x55a9544764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.433+0000 7f79e9d4db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a954329b20 con 0x55a95446c400 2024-05-09T02:12:05.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.433+0000 7f79e9d4db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a954479020 con 0x55a9542ef4a0 2024-05-09T02:12:05.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.433+0000 7f79d88d2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 0x55a9544764f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41978/0 (socket says 172.21.15.19:41978) 2024-05-09T02:12:05.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.433+0000 7f79d88d2700 1 -- 172.21.15.19:0/3926980498 learned_addr learned my addr 172.21.15.19:0/3926980498 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:05.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.433+0000 7f79e9d4db80 10 client.0 _ll_register_callbacks cb 0x55a9542006c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:05.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.433+0000 7f79d98d4700 1 --2- 172.21.15.19:0/3926980498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a95446c400 0x55a954476a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.434+0000 7f79d88d2700 1 -- 172.21.15.19:0/3926980498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9542ef4a0 msgr2=0x55a954475fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:05.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.434+0000 7f79d90d3700 1 --2- 172.21.15.19:0/3926980498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9542ef4a0 0x55a954475fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.434+0000 7f79d88d2700 1 --2- 172.21.15.19:0/3926980498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9542ef4a0 0x55a954475fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.434+0000 7f79d88d2700 1 -- 172.21.15.19:0/3926980498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a95446c400 msgr2=0x55a954476a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:05.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.434+0000 7f79d88d2700 1 --2- 172.21.15.19:0/3926980498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a95446c400 0x55a954476a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.434+0000 7f79d88d2700 1 -- 172.21.15.19:0/3926980498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a95420a490 con 0x55a954369d00 2024-05-09T02:12:05.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.434+0000 7f79d98d4700 1 --2- 172.21.15.19:0/3926980498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a95446c400 0x55a954476a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:12:05.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.434+0000 7f79d90d3700 1 --2- 172.21.15.19:0/3926980498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9542ef4a0 0x55a954475fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:12:05.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.434+0000 7f79d88d2700 1 --2- 172.21.15.19:0/3926980498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 0x55a9544764f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f79c4006010 tx=0x7f79c4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3c84f9558acf59 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:05.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.434+0000 7f79bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:12:05.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.434+0000 7f79bcff9700 1 -- 172.21.15.19:0/3926980498 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4021040 con 0x55a954369d00 2024-05-09T02:12:05.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.434+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3926980498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79d00118f0 con 0x55a954369d00 2024-05-09T02:12:05.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.434+0000 7f79bcff9700 1 -- 172.21.15.19:0/3926980498 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f79c400f870 con 0x55a954369d00 2024-05-09T02:12:05.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.435+0000 7f79bcff9700 1 -- 172.21.15.19:0/3926980498 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f79c400bcf0 con 0x55a954369d00 2024-05-09T02:12:05.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.435+0000 7f79e9d4db80 10 client.15435 fetch_fsmap learned FSMap version 31 2024-05-09T02:12:05.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.435+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3926980498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a9544837d0 con 0x55a954369d00 2024-05-09T02:12:05.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.436+0000 7f79bcff9700 1 -- 172.21.15.19:0/3926980498 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79d00118f0 con 0x55a954369d00 2024-05-09T02:12:05.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.437+0000 7f79bcff9700 1 -- 172.21.15.19:0/3926980498 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f79c4027040 con 0x55a954369d00 2024-05-09T02:12:05.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.437+0000 7f79e9d4db80 10 client.15435 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T02:12:05.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.437+0000 7f79e9d4db80 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:12:05.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.437+0000 7f79e9d4db80 20 client.15435 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:05.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.437+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3926980498 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55a954485e00 0x55a9544882b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.437+0000 7f79e9d4db80 4 client.15435 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:12:05.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.437+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3926980498 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a95439ca80 con 0x55a954485e00 2024-05-09T02:12:05.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.438+0000 7f79d90d3700 1 --2- 172.21.15.19:0/3926980498 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55a954485e00 0x55a9544882b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.438+0000 7f79bcff9700 10 client.15435 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:05.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.438+0000 7f79d90d3700 1 --2- 172.21.15.19:0/3926980498 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55a954485e00 0x55a9544882b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a06ab8d77b7224e2 server_cookie=9b3449481e9fb334 in_seq=0 out_seq=0 2024-05-09T02:12:05.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.440+0000 7f79bcff9700 1 -- 172.21.15.19:0/3926980498 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a95439ca80 con 0x55a954485e00 2024-05-09T02:12:05.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.440+0000 7f79bcff9700 10 client.15435 handle_command_reply: tid=0 2024-05-09T02:12:05.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.440+0000 7f79e9d4db80 1 client.15435 shutdown 2024-05-09T02:12:05.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.440+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3926980498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 msgr2=0x55a9544764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:05.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.440+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3926980498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 0x55a9544764f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f79c4006010 tx=0x7f79c4009cd0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.440+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3926980498 shutdown_connections 2024-05-09T02:12:05.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.440+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3926980498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a95446c400 0x55a954476a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.440+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3926980498 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55a954485e00 0x55a9544882b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.440+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3926980498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9542ef4a0 0x55a954475fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.440+0000 7f79bcff9700 0 client.15435 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:05.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.440+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3926980498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 0x55a9544764f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.440+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3926980498 >> 172.21.15.19:0/3926980498 conn(0x55a9542f2f70 msgr2=0x55a954460ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:05.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.441+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3926980498 shutdown_connections 2024-05-09T02:12:05.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.441+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3926980498 wait complete. 2024-05-09T02:12:05.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.441+0000 7f79e9d4db80 20 client.15435 trim_cache size 0 max 16384 2024-05-09T02:12:05.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.472+0000 7f79e9d4db80 1 Processor -- start 2024-05-09T02:12:05.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.472+0000 7f79e9d4db80 1 -- start start 2024-05-09T02:12:05.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.472+0000 7f79e9d4db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a9542ef4a0 0x55a954471480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.472+0000 7f79e9d4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 0x55a954461a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.472+0000 7f79e9d4db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a95446c400 0x55a954474f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.473+0000 7f79d90d3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a9542ef4a0 0x55a954471480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.473+0000 7f79d90d3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a9542ef4a0 0x55a954471480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:35734/0 (socket says 172.21.15.19:35734) 2024-05-09T02:12:05.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.473+0000 7f79d88d2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 0x55a954461a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.473+0000 7f79e9d4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79d41b3480 con 0x55a954369d00 2024-05-09T02:12:05.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.473+0000 7f79e9d4db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f79d41b5020 con 0x55a9542ef4a0 2024-05-09T02:12:05.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.473+0000 7f79d98d4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a95446c400 0x55a954474f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.473+0000 7f79d88d2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 0x55a954461a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41988/0 (socket says 172.21.15.19:41988) 2024-05-09T02:12:05.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.473+0000 7f79d88d2700 1 -- 172.21.15.19:0/2433621881 learned_addr learned my addr 172.21.15.19:0/2433621881 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:05.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.473+0000 7f79e9d4db80 1 -- 172.21.15.19:0/2433621881 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79d41b3b50 con 0x55a95446c400 2024-05-09T02:12:05.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.474+0000 7f79d98d4700 1 -- 172.21.15.19:0/2433621881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a9542ef4a0 msgr2=0x55a954471480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:05.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.474+0000 7f79d98d4700 1 --2- 172.21.15.19:0/2433621881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a9542ef4a0 0x55a954471480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.474+0000 7f79d98d4700 1 -- 172.21.15.19:0/2433621881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 msgr2=0x55a954461a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:05.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.474+0000 7f79d98d4700 1 --2- 172.21.15.19:0/2433621881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 0x55a954461a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.474+0000 7f79d98d4700 1 -- 172.21.15.19:0/2433621881 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a954468ea0 con 0x55a95446c400 2024-05-09T02:12:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.474+0000 7f79d98d4700 1 --2- 172.21.15.19:0/2433621881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a95446c400 0x55a954474f10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0df6d0 tx=0x7f79cc0e1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46783e5a66aba48b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.475+0000 7f79beffd700 1 -- 172.21.15.19:0/2433621881 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc0c2070 con 0x55a95446c400 2024-05-09T02:12:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.475+0000 7f79beffd700 1 -- 172.21.15.19:0/2433621881 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79cc0a96f0 con 0x55a95446c400 2024-05-09T02:12:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.475+0000 7f79beffd700 1 -- 172.21.15.19:0/2433621881 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc0dfc10 con 0x55a95446c400 2024-05-09T02:12:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.475+0000 7f79e9d4db80 1 -- 172.21.15.19:0/2433621881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a95446c400 msgr2=0x55a954474f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.475+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/2433621881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a95446c400 0x55a954474f10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0df6d0 tx=0x7f79cc0e1040 comp rx=0 tx=0).stop 2024-05-09T02:12:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.476+0000 7f79e9d4db80 1 -- 172.21.15.19:0/2433621881 shutdown_connections 2024-05-09T02:12:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.476+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/2433621881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a95446c400 0x55a954474f10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.476+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/2433621881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a9542ef4a0 0x55a954471480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.476+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/2433621881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 0x55a954461a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.476+0000 7f79e9d4db80 1 -- 172.21.15.19:0/2433621881 >> 172.21.15.19:0/2433621881 conn(0x55a9542f2f70 msgr2=0x55a954475f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.476+0000 7f79e9d4db80 1 -- 172.21.15.19:0/2433621881 shutdown_connections 2024-05-09T02:12:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.476+0000 7f79e9d4db80 1 -- 172.21.15.19:0/2433621881 wait complete. 2024-05-09T02:12:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.476+0000 7f79e9d4db80 1 -- start start 2024-05-09T02:12:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.476+0000 7f79e9d4db80 1 Processor -- start 2024-05-09T02:12:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.477+0000 7f79e9d4db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9542ef4a0 0x55a954472e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.477+0000 7f79e9d4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 0x55a9544733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.477+0000 7f79e9d4db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a95446c400 0x55a9544738e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.477+0000 7f79e9d4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79d41b48f0 con 0x55a954369d00 2024-05-09T02:12:05.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.477+0000 7f79e9d4db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f79d41b4220 con 0x55a95446c400 2024-05-09T02:12:05.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.477+0000 7f79e9d4db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79d41b1420 con 0x55a9542ef4a0 2024-05-09T02:12:05.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.477+0000 7f79e9d4db80 10 client.0 _ll_register_callbacks cb 0x55a9542006c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:05.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.477+0000 7f79d98d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a95446c400 0x55a9544738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.477+0000 7f79d98d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a95446c400 0x55a9544738e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:35750/0 (socket says 172.21.15.19:35750) 2024-05-09T02:12:05.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.477+0000 7f79d98d4700 1 -- 172.21.15.19:0/3123876333 learned_addr learned my addr 172.21.15.19:0/3123876333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:05.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.478+0000 7f79d88d2700 1 --2- 172.21.15.19:0/3123876333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 0x55a9544733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.478+0000 7f79d90d3700 1 --2- 172.21.15.19:0/3123876333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9542ef4a0 0x55a954472e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.478+0000 7f79d98d4700 1 -- 172.21.15.19:0/3123876333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9542ef4a0 msgr2=0x55a954472e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:05.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.478+0000 7f79d98d4700 1 --2- 172.21.15.19:0/3123876333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9542ef4a0 0x55a954472e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.478+0000 7f79d98d4700 1 -- 172.21.15.19:0/3123876333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 msgr2=0x55a9544733a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:05.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.478+0000 7f79d98d4700 1 --2- 172.21.15.19:0/3123876333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 0x55a9544733a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.478+0000 7f79d98d4700 1 -- 172.21.15.19:0/3123876333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9544ef620 con 0x55a95446c400 2024-05-09T02:12:05.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.478+0000 7f79d90d3700 1 --2- 172.21.15.19:0/3123876333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9542ef4a0 0x55a954472e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:12:05.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.478+0000 7f79d98d4700 1 --2- 172.21.15.19:0/3123876333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a95446c400 0x55a9544738e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0df7e0 tx=0x7f79cc0dfbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96a490f941c3fab0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:05.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.478+0000 7f79bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:12:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.478+0000 7f79bcff9700 1 -- 172.21.15.19:0/3123876333 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc0c2040 con 0x55a95446c400 2024-05-09T02:12:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.478+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3123876333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79cc0a96f0 con 0x55a95446c400 2024-05-09T02:12:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.478+0000 7f79bcff9700 1 -- 172.21.15.19:0/3123876333 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f79cc0a8b60 con 0x55a95446c400 2024-05-09T02:12:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.479+0000 7f79bcff9700 1 -- 172.21.15.19:0/3123876333 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f79cc0adaa0 con 0x55a95446c400 2024-05-09T02:12:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.479+0000 7f79e9d4db80 10 client.25112 fetch_fsmap learned FSMap version 31 2024-05-09T02:12:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.479+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3123876333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a9544f0810 con 0x55a95446c400 2024-05-09T02:12:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.479+0000 7f79bcff9700 1 -- 172.21.15.19:0/3123876333 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79cc0a96f0 con 0x55a95446c400 2024-05-09T02:12:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.479+0000 7f79bcff9700 1 -- 172.21.15.19:0/3123876333 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f79cc0a83c0 con 0x55a95446c400 2024-05-09T02:12:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.479+0000 7f79e9d4db80 10 client.25112 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T02:12:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.479+0000 7f79e9d4db80 10 client.25112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:12:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.479+0000 7f79e9d4db80 20 client.25112 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.479+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3123876333 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55a954485e00 0x55a9544f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.479+0000 7f79e9d4db80 4 client.25112 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:12:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.479+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3123876333 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a954396b20 con 0x55a954485e00 2024-05-09T02:12:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.479+0000 7f79d90d3700 1 --2- 172.21.15.19:0/3123876333 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55a954485e00 0x55a9544f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.480+0000 7f79bcff9700 10 client.25112 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.480+0000 7f79d90d3700 1 --2- 172.21.15.19:0/3123876333 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55a954485e00 0x55a9544f2e20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4530dbf000acb0f2 server_cookie=4b2b1eb9cb3121d2 in_seq=0 out_seq=0 2024-05-09T02:12:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.480+0000 7f79bcff9700 1 -- 172.21.15.19:0/3123876333 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a954396b20 con 0x55a954485e00 2024-05-09T02:12:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.480+0000 7f79bcff9700 10 client.25112 handle_command_reply: tid=0 2024-05-09T02:12:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.480+0000 7f79e9d4db80 1 client.25112 shutdown 2024-05-09T02:12:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.480+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3123876333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a95446c400 msgr2=0x55a9544738e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.480+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3123876333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a95446c400 0x55a9544738e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0df7e0 tx=0x7f79cc0dfbe0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.480+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3123876333 shutdown_connections 2024-05-09T02:12:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.480+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3123876333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a95446c400 0x55a9544738e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.480+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3123876333 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55a954485e00 0x55a9544f2e20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.480+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3123876333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9542ef4a0 0x55a954472e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.480+0000 7f79e9d4db80 1 --2- 172.21.15.19:0/3123876333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a954369d00 0x55a9544733a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.480+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3123876333 >> 172.21.15.19:0/3123876333 conn(0x55a9542f2f70 msgr2=0x55a954461780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.480+0000 7f79bcff9700 0 client.25112 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.481+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3123876333 shutdown_connections 2024-05-09T02:12:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.481+0000 7f79e9d4db80 1 -- 172.21.15.19:0/3123876333 wait complete. 2024-05-09T02:12:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.481+0000 7f79e9d4db80 20 client.25112 trim_cache size 0 max 16384 2024-05-09T02:12:05.490 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:12:05.490 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T02:12:05.490 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T02:12:05.490 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:12:05.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.483+0000 7f79db337700 1 -- 172.21.15.19:0/2414283292 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79c00b2ca0 msgr2=0x7f79c00b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:05.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.483+0000 7f79db337700 1 --2- 172.21.15.19:0/2414283292 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79c00b2ca0 0x7f79c00b51b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79c4000c00 tx=0x7f79c40058e0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.483+0000 7f79db337700 1 -- 172.21.15.19:0/2414283292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79d4108740 msgr2=0x7f79d4111790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:05.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.483+0000 7f79db337700 1 --2- 172.21.15.19:0/2414283292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79d4108740 0x7f79d4111790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00efc0 tx=0x7f79cc00c3a0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.483+0000 7f79db337700 1 -- 172.21.15.19:0/2414283292 shutdown_connections 2024-05-09T02:12:05.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.483+0000 7f79db337700 1 --2- 172.21.15.19:0/2414283292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79d4108740 0x7f79d4111790 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.483+0000 7f79db337700 1 --2- 172.21.15.19:0/2414283292 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79c00b2ca0 0x7f79c00b51b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.483+0000 7f79db337700 1 --2- 172.21.15.19:0/2414283292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d41034c0 0x7f79d4110d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.483+0000 7f79db337700 1 --2- 172.21.15.19:0/2414283292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79d4107df0 0x7f79d4111250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:05.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.483+0000 7f79db337700 1 -- 172.21.15.19:0/2414283292 >> 172.21.15.19:0/2414283292 conn(0x7f79d40fedc0 msgr2=0x7f79d4107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:05.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.484+0000 7f79db337700 1 -- 172.21.15.19:0/2414283292 shutdown_connections 2024-05-09T02:12:05.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:05.484+0000 7f79db337700 1 -- 172.21.15.19:0/2414283292 wait complete. 2024-05-09T02:12:06.310 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T02:12:06.310 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:12:06.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:06 smithi059 ceph-mon[34781]: pgmap v27: 97 pgs: 97 active+clean; 969 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 45 MiB/s wr, 1.36k op/s 2024-05-09T02:12:06.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:06 smithi196 ceph-mon[34718]: pgmap v27: 97 pgs: 97 active+clean; 969 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 45 MiB/s wr, 1.36k op/s 2024-05-09T02:12:06.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:06 smithi019 ceph-mon[30151]: pgmap v27: 97 pgs: 97 active+clean; 969 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 45 MiB/s wr, 1.36k op/s 2024-05-09T02:12:06.702 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:12:07.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:07 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:12:07.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:07 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:12:07.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:07 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:12:07.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 25.0 s, 3484.2 tps, lat 28.733 ms stddev 28.386 2024-05-09T02:12:08.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:08 smithi059 ceph-mon[34781]: pgmap v28: 97 pgs: 97 active+clean; 969 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 42 MiB/s wr, 1.16k op/s 2024-05-09T02:12:08.524 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:08 smithi019 ceph-mon[30151]: pgmap v28: 97 pgs: 97 active+clean; 969 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 42 MiB/s wr, 1.16k op/s 2024-05-09T02:12:08.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:08 smithi196 ceph-mon[34718]: pgmap v28: 97 pgs: 97 active+clean; 969 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 42 MiB/s wr, 1.16k op/s 2024-05-09T02:12:09.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.227+0000 7efdfffff700 1 -- 172.21.15.19:0/1600863025 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe040092e0 con 0x7efe08110970 2024-05-09T02:12:09.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.229+0000 7efe0f5b4700 1 -- 172.21.15.19:0/1600863025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe08110970 msgr2=0x7efe08110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:09.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.229+0000 7efe0f5b4700 1 --2- 172.21.15.19:0/1600863025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe08110970 0x7efe08110d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efe0400b0a0 tx=0x7efe0400b3b0 comp rx=0 tx=0).stop 2024-05-09T02:12:09.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.229+0000 7efe0f5b4700 1 -- 172.21.15.19:0/1600863025 shutdown_connections 2024-05-09T02:12:09.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.229+0000 7efe0f5b4700 1 --2- 172.21.15.19:0/1600863025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe0810b2e0 0x7efe08117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:09.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.229+0000 7efe0f5b4700 1 --2- 172.21.15.19:0/1600863025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe08110970 0x7efe08110d80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:09.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.229+0000 7efe0f5b4700 1 --2- 172.21.15.19:0/1600863025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe0810a990 0x7efe0810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:09.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.229+0000 7efe0f5b4700 1 -- 172.21.15.19:0/1600863025 >> 172.21.15.19:0/1600863025 conn(0x7efe08077c80 msgr2=0x7efe080780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:09.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.229+0000 7efe0f5b4700 1 -- 172.21.15.19:0/1600863025 shutdown_connections 2024-05-09T02:12:09.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.229+0000 7efe0f5b4700 1 -- 172.21.15.19:0/1600863025 wait complete. 2024-05-09T02:12:09.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.230+0000 7efe0f5b4700 1 Processor -- start 2024-05-09T02:12:09.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.230+0000 7efe0f5b4700 1 -- start start 2024-05-09T02:12:09.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.230+0000 7efe0f5b4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe0810a990 0x7efe081a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:09.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.230+0000 7efe0f5b4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe0810b2e0 0x7efe081a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:09.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.230+0000 7efe0f5b4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe08110970 0x7efe081a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:09.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.230+0000 7efe0f5b4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe0811ab90 con 0x7efe0810a990 2024-05-09T02:12:09.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.230+0000 7efe0f5b4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efe0811aa10 con 0x7efe0810b2e0 2024-05-09T02:12:09.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.230+0000 7efe0f5b4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe0811ad10 con 0x7efe08110970 2024-05-09T02:12:09.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.230+0000 7efe0cb4f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe0810b2e0 0x7efe081a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:09.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.230+0000 7efe0cb4f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe0810b2e0 0x7efe081a1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:43814/0 (socket says 172.21.15.19:43814) 2024-05-09T02:12:09.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.230+0000 7efe0cb4f700 1 -- 172.21.15.19:0/953694870 learned_addr learned my addr 172.21.15.19:0/953694870 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:09.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.231+0000 7efe0db51700 1 --2- 172.21.15.19:0/953694870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe08110970 0x7efe081a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:09.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.231+0000 7efe0cb4f700 1 -- 172.21.15.19:0/953694870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe08110970 msgr2=0x7efe081a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:09.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.231+0000 7efe0cb4f700 1 --2- 172.21.15.19:0/953694870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe08110970 0x7efe081a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:09.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.231+0000 7efe0cb4f700 1 -- 172.21.15.19:0/953694870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe0810a990 msgr2=0x7efe081a1460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:12:09.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.231+0000 7efe0cb4f700 1 --2- 172.21.15.19:0/953694870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe0810a990 0x7efe081a1460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:09.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.231+0000 7efe0cb4f700 1 -- 172.21.15.19:0/953694870 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe081a6330 con 0x7efe0810b2e0 2024-05-09T02:12:09.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.231+0000 7efe0cb4f700 1 --2- 172.21.15.19:0/953694870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe0810b2e0 0x7efe081a1a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efdf800b500 tx=0x7efdf800b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:09.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.231+0000 7efdfe7fc700 1 -- 172.21.15.19:0/953694870 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8015020 con 0x7efe0810b2e0 2024-05-09T02:12:09.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.231+0000 7efdfe7fc700 1 -- 172.21.15.19:0/953694870 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdf8011ce0 con 0x7efe0810b2e0 2024-05-09T02:12:09.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.231+0000 7efe0f5b4700 1 -- 172.21.15.19:0/953694870 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe081a6620 con 0x7efe0810b2e0 2024-05-09T02:12:09.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.232+0000 7efdfe7fc700 1 -- 172.21.15.19:0/953694870 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8010d40 con 0x7efe0810b2e0 2024-05-09T02:12:09.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.232+0000 7efe0f5b4700 1 -- 172.21.15.19:0/953694870 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe0807d120 con 0x7efe0810b2e0 2024-05-09T02:12:09.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.233+0000 7efe0f5b4700 1 -- 172.21.15.19:0/953694870 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe0810c080 con 0x7efe0810b2e0 2024-05-09T02:12:09.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.235+0000 7efdfe7fc700 1 -- 172.21.15.19:0/953694870 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7efdf8018020 con 0x7efe0810b2e0 2024-05-09T02:12:09.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.237+0000 7efdfe7fc700 1 --2- 172.21.15.19:0/953694870 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efdf40b2c60 0x7efdf40b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:09.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.237+0000 7efdfe7fc700 1 -- 172.21.15.19:0/953694870 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7efdf80e1e60 con 0x7efe0810b2e0 2024-05-09T02:12:09.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.238+0000 7efe0d350700 1 --2- 172.21.15.19:0/953694870 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efdf40b2c60 0x7efdf40b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:09.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.239+0000 7efe0d350700 1 --2- 172.21.15.19:0/953694870 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efdf40b2c60 0x7efdf40b5170 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe04009900 tx=0x7efe040060c0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:09.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.242+0000 7efdfe7fc700 1 -- 172.21.15.19:0/953694870 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efdf801f020 con 0x7efe0810b2e0 2024-05-09T02:12:09.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.649+0000 7efe0f5b4700 1 -- 172.21.15.19:0/953694870 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe081a27c0 con 0x7efe0810b2e0 2024-05-09T02:12:09.653 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:12:09.654 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:12:05.724282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24455,24457,24473,24484,24463]},"id":1}]} 2024-05-09T02:12:09.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.651+0000 7efdfe7fc700 1 -- 172.21.15.19:0/953694870 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11922 (secure 0 0 0) 0x7efdf80a96a0 con 0x7efe0810b2e0 2024-05-09T02:12:09.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.654+0000 7efe0f5b4700 1 -- 172.21.15.19:0/953694870 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efdf40b2c60 msgr2=0x7efdf40b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:09.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.654+0000 7efe0f5b4700 1 --2- 172.21.15.19:0/953694870 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efdf40b2c60 0x7efdf40b5170 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe04009900 tx=0x7efe040060c0 comp rx=0 tx=0).stop 2024-05-09T02:12:09.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.654+0000 7efe0f5b4700 1 -- 172.21.15.19:0/953694870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe0810b2e0 msgr2=0x7efe081a1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:09.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.654+0000 7efe0f5b4700 1 --2- 172.21.15.19:0/953694870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe0810b2e0 0x7efe081a1a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efdf800b500 tx=0x7efdf800b810 comp rx=0 tx=0).stop 2024-05-09T02:12:09.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.654+0000 7efe0f5b4700 1 -- 172.21.15.19:0/953694870 shutdown_connections 2024-05-09T02:12:09.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.654+0000 7efe0f5b4700 1 --2- 172.21.15.19:0/953694870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe08110970 0x7efe081a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:09.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.654+0000 7efe0f5b4700 1 --2- 172.21.15.19:0/953694870 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efdf40b2c60 0x7efdf40b5170 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:09.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.654+0000 7efe0f5b4700 1 --2- 172.21.15.19:0/953694870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe0810a990 0x7efe081a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:09.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.654+0000 7efe0f5b4700 1 --2- 172.21.15.19:0/953694870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe0810b2e0 0x7efe081a1a00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:09.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.654+0000 7efe0f5b4700 1 -- 172.21.15.19:0/953694870 >> 172.21.15.19:0/953694870 conn(0x7efe08077c80 msgr2=0x7efe0807bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:09.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.654+0000 7efe0f5b4700 1 -- 172.21.15.19:0/953694870 shutdown_connections 2024-05-09T02:12:09.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:09.654+0000 7efe0f5b4700 1 -- 172.21.15.19:0/953694870 wait complete. 2024-05-09T02:12:09.658 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-05-09T02:12:10.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:10 smithi019 ceph-mon[30151]: pgmap v29: 97 pgs: 97 active+clean; 997 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 46 MiB/s wr, 1.48k op/s 2024-05-09T02:12:10.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:10 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/953694870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:12:10.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T02:12:10.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:10 smithi059 ceph-mon[34781]: pgmap v29: 97 pgs: 97 active+clean; 997 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 46 MiB/s wr, 1.48k op/s 2024-05-09T02:12:10.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:10 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/953694870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:12:10.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:10 smithi196 ceph-mon[34718]: pgmap v29: 97 pgs: 97 active+clean; 997 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 46 MiB/s wr, 1.48k op/s 2024-05-09T02:12:10.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:10 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/953694870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:12:10.825 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:12:12.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:12 smithi019 ceph-mon[30151]: pgmap v30: 97 pgs: 97 active+clean; 997 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 42 MiB/s wr, 1.16k op/s 2024-05-09T02:12:12.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:12 smithi059 ceph-mon[34781]: pgmap v30: 97 pgs: 97 active+clean; 997 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 42 MiB/s wr, 1.16k op/s 2024-05-09T02:12:12.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:12 smithi196 ceph-mon[34718]: pgmap v30: 97 pgs: 97 active+clean; 997 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 42 MiB/s wr, 1.16k op/s 2024-05-09T02:12:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 30.0 s, 3785.2 tps, lat 26.455 ms stddev 27.877 2024-05-09T02:12:12.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.676+0000 7f3aee4d3700 1 -- 172.21.15.19:0/335945159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ae81034e0 msgr2=0x7f3ae81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:12.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.676+0000 7f3aee4d3700 1 --2- 172.21.15.19:0/335945159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ae81034e0 0x7f3ae81038f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3ad80099c0 tx=0x7f3ad8009cd0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.676+0000 7f3aee4d3700 1 -- 172.21.15.19:0/335945159 shutdown_connections 2024-05-09T02:12:12.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.676+0000 7f3aee4d3700 1 --2- 172.21.15.19:0/335945159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ae8108760 0x7f3ae81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.676+0000 7f3aee4d3700 1 --2- 172.21.15.19:0/335945159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ae81034e0 0x7f3ae81038f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.676+0000 7f3aee4d3700 1 --2- 172.21.15.19:0/335945159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae8107e10 0x7f3ae8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.676+0000 7f3aee4d3700 1 -- 172.21.15.19:0/335945159 >> 172.21.15.19:0/335945159 conn(0x7f3ae80fef30 msgr2=0x7f3ae8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:12.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.676+0000 7f3aee4d3700 1 -- 172.21.15.19:0/335945159 shutdown_connections 2024-05-09T02:12:12.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.676+0000 7f3aee4d3700 1 -- 172.21.15.19:0/335945159 wait complete. 2024-05-09T02:12:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.677+0000 7f3aee4d3700 1 Processor -- start 2024-05-09T02:12:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.677+0000 7f3aee4d3700 1 -- start start 2024-05-09T02:12:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.678+0000 7f3aee4d3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ae81034e0 0x7f3ae819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.678+0000 7f3aee4d3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ae8107e10 0x7f3ae819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.678+0000 7f3aee4d3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae8108760 0x7f3ae819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.678+0000 7f3aee4d3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8114530 con 0x7f3ae8108760 2024-05-09T02:12:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.678+0000 7f3aee4d3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81143b0 con 0x7f3ae8107e10 2024-05-09T02:12:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.678+0000 7f3aee4d3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81146b0 con 0x7f3ae81034e0 2024-05-09T02:12:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.678+0000 7f3adffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ae8107e10 0x7f3ae819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.678+0000 7f3adffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ae8107e10 0x7f3ae819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:43832/0 (socket says 172.21.15.19:43832) 2024-05-09T02:12:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.678+0000 7f3adffff700 1 -- 172.21.15.19:0/2900072185 learned_addr learned my addr 172.21.15.19:0/2900072185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.679+0000 7f3adffff700 1 -- 172.21.15.19:0/2900072185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ae81034e0 msgr2=0x7f3ae819aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.679+0000 7f3adffff700 1 --2- 172.21.15.19:0/2900072185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ae81034e0 0x7f3ae819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.679+0000 7f3adffff700 1 -- 172.21.15.19:0/2900072185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae8108760 msgr2=0x7f3ae819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.679+0000 7f3adffff700 1 --2- 172.21.15.19:0/2900072185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae8108760 0x7f3ae819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.679+0000 7f3adffff700 1 -- 172.21.15.19:0/2900072185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae819fd70 con 0x7f3ae8107e10 2024-05-09T02:12:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.679+0000 7f3adffff700 1 --2- 172.21.15.19:0/2900072185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ae8107e10 0x7f3ae819b440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000ea30 tx=0x7f3ad000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.680+0000 7f3ae5ffb700 1 -- 172.21.15.19:0/2900072185 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad000cc90 con 0x7f3ae8107e10 2024-05-09T02:12:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.680+0000 7f3aee4d3700 1 -- 172.21.15.19:0/2900072185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae81a0060 con 0x7f3ae8107e10 2024-05-09T02:12:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.680+0000 7f3ae5ffb700 1 -- 172.21.15.19:0/2900072185 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ad000ce30 con 0x7f3ae8107e10 2024-05-09T02:12:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.680+0000 7f3ae5ffb700 1 -- 172.21.15.19:0/2900072185 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0010430 con 0x7f3ae8107e10 2024-05-09T02:12:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.680+0000 7f3aee4d3700 1 -- 172.21.15.19:0/2900072185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae81118b0 con 0x7f3ae8107e10 2024-05-09T02:12:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.683+0000 7f3ae5ffb700 1 -- 172.21.15.19:0/2900072185 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f3ad00105d0 con 0x7f3ae8107e10 2024-05-09T02:12:12.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.685+0000 7f3ae5ffb700 1 --2- 172.21.15.19:0/2900072185 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3ad40b2c50 0x7f3ad40b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.685+0000 7f3ae5ffb700 1 -- 172.21.15.19:0/2900072185 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f3ad00e2be0 con 0x7f3ae8107e10 2024-05-09T02:12:12.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.685+0000 7f3ae7fff700 1 --2- 172.21.15.19:0/2900072185 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3ad40b2c50 0x7f3ad40b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:12.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.685+0000 7f3afcee9b80 1 Processor -- start 2024-05-09T02:12:12.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3afcee9b80 1 -- start start 2024-05-09T02:12:12.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3afcee9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 0x55fffefbc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3afcee9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffee394a0 0x55fffefbca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3afcee9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffefb66a0 0x55fffefb6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3afcee9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fffeed8ff0 con 0x55fffefb66a0 2024-05-09T02:12:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3afcee9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fffee777f0 con 0x55fffef23160 2024-05-09T02:12:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3afcee9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fffeed4fb0 con 0x55fffee394a0 2024-05-09T02:12:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3ae7fff700 1 --2- 172.21.15.19:0/2900072185 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3ad40b2c50 0x7f3ad40b5160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3ad80098e0 tx=0x7f3ad8000bc0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3aeca70700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffefb66a0 0x55fffefb6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3aeca70700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffefb66a0 0x55fffefb6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39696/0 (socket says 172.21.15.19:39696) 2024-05-09T02:12:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3aeca70700 1 -- 172.21.15.19:0/903808543 learned_addr learned my addr 172.21.15.19:0/903808543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3adffff700 1 --2- 172.21.15.19:0/903808543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffee394a0 0x55fffefbca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3ae7fff700 1 --2- 172.21.15.19:0/903808543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 0x55fffefbc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3adffff700 1 -- 172.21.15.19:0/903808543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 msgr2=0x55fffefbc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3adffff700 1 --2- 172.21.15.19:0/903808543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 0x55fffefbc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3adffff700 1 -- 172.21.15.19:0/903808543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffefb66a0 msgr2=0x55fffefb6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:12.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3adffff700 1 --2- 172.21.15.19:0/903808543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffefb66a0 0x55fffefb6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.686+0000 7f3adffff700 1 -- 172.21.15.19:0/903808543 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fffee6f370 con 0x55fffee394a0 2024-05-09T02:12:12.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.687+0000 7f3adffff700 1 --2- 172.21.15.19:0/903808543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffee394a0 0x55fffefbca30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00e0150 tx=0x7f3ad0009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9030ad929718c67 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.687+0000 7f3ade7fc700 1 -- 172.21.15.19:0/903808543 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00a62d0 con 0x55fffee394a0 2024-05-09T02:12:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.687+0000 7f3ade7fc700 1 -- 172.21.15.19:0/903808543 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ad00a7130 con 0x55fffee394a0 2024-05-09T02:12:12.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.693+0000 7f3afcee9b80 1 -- 172.21.15.19:0/903808543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffee394a0 msgr2=0x55fffefbca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:12.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.693+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/903808543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffee394a0 0x55fffefbca30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00e0150 tx=0x7f3ad0009620 comp rx=0 tx=0).stop 2024-05-09T02:12:12.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.693+0000 7f3afcee9b80 1 -- 172.21.15.19:0/903808543 shutdown_connections 2024-05-09T02:12:12.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.693+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/903808543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffefb66a0 0x55fffefb6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.693+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/903808543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 0x55fffefbc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.693+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/903808543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffee394a0 0x55fffefbca30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.693+0000 7f3afcee9b80 1 -- 172.21.15.19:0/903808543 >> 172.21.15.19:0/903808543 conn(0x55fffee996a0 msgr2=0x55fffefb1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:12.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.698+0000 7f3afcee9b80 1 -- 172.21.15.19:0/903808543 shutdown_connections 2024-05-09T02:12:12.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.699+0000 7f3afcee9b80 1 -- 172.21.15.19:0/903808543 wait complete. 2024-05-09T02:12:12.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.699+0000 7f3afcee9b80 1 -- start start 2024-05-09T02:12:12.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.700+0000 7f3afcee9b80 1 Processor -- start 2024-05-09T02:12:12.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.700+0000 7f3afcee9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffee394a0 0x55fffefc02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.700+0000 7f3afcee9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 0x55fffefc0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.700+0000 7f3afcee9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffefb66a0 0x55fffefc0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.700+0000 7f3afcee9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fffed54ba0 con 0x55fffee394a0 2024-05-09T02:12:12.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.700+0000 7f3afcee9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fffee93150 con 0x55fffef23160 2024-05-09T02:12:12.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.700+0000 7f3afcee9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fffefc31d0 con 0x55fffefb66a0 2024-05-09T02:12:12.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.701+0000 7f3afcee9b80 10 client.0 _ll_register_callbacks cb 0x55fffed4a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:12.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.701+0000 7f3aeca70700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffefb66a0 0x55fffefc0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:12.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.701+0000 7f3aeca70700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffefb66a0 0x55fffefc0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40956/0 (socket says 172.21.15.19:40956) 2024-05-09T02:12:12.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.701+0000 7f3aeca70700 1 -- 172.21.15.19:0/3602733822 learned_addr learned my addr 172.21.15.19:0/3602733822 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:12.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.701+0000 7f3adffff700 1 --2- 172.21.15.19:0/3602733822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 0x55fffefc0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:12.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.701+0000 7f3aeca70700 1 -- 172.21.15.19:0/3602733822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 msgr2=0x55fffefc0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:12.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.701+0000 7f3aeca70700 1 --2- 172.21.15.19:0/3602733822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 0x55fffefc0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.701+0000 7f3aeca70700 1 -- 172.21.15.19:0/3602733822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffee394a0 msgr2=0x55fffefc02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:12.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.701+0000 7f3aeca70700 1 --2- 172.21.15.19:0/3602733822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffee394a0 0x55fffefc02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.701+0000 7f3aeca70700 1 -- 172.21.15.19:0/3602733822 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fffed54490 con 0x55fffefb66a0 2024-05-09T02:12:12.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.702+0000 7f3aeca70700 1 --2- 172.21.15.19:0/3602733822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffefb66a0 0x55fffefc0d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0011c20 tx=0x7f3ae0011f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2deeeac78934a695 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:12.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.702+0000 7f3ac77fe700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:12:12.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.703+0000 7f3ac77fe700 1 -- 172.21.15.19:0/3602733822 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae001a030 con 0x55fffefb66a0 2024-05-09T02:12:12.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.703+0000 7f3ac77fe700 1 -- 172.21.15.19:0/3602733822 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f3ae0004bb0 con 0x55fffefb66a0 2024-05-09T02:12:12.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.703+0000 7f3afcee9b80 1 -- 172.21.15.19:0/3602733822 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ad00a7130 con 0x55fffefb66a0 2024-05-09T02:12:12.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.704+0000 7f3ac77fe700 1 -- 172.21.15.19:0/3602733822 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3ae0004db0 con 0x55fffefb66a0 2024-05-09T02:12:12.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.704+0000 7f3afcee9b80 10 client.25162 fetch_fsmap learned FSMap version 32 2024-05-09T02:12:12.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.704+0000 7f3afcee9b80 1 -- 172.21.15.19:0/3602733822 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fffefe7390 con 0x55fffefb66a0 2024-05-09T02:12:12.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.704+0000 7f3ac77fe700 1 -- 172.21.15.19:0/3602733822 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ad00a7130 con 0x55fffefb66a0 2024-05-09T02:12:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.704+0000 7f3ac77fe700 1 -- 172.21.15.19:0/3602733822 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f3ae0007570 con 0x55fffefb66a0 2024-05-09T02:12:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.705+0000 7f3afcee9b80 10 client.25162 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T02:12:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.705+0000 7f3afcee9b80 10 client.25162 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:12:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.705+0000 7f3afcee9b80 20 client.25162 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.705+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/3602733822 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55fffefe99d0 0x55fffefebe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.705+0000 7f3afcee9b80 4 client.25162 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:12:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.705+0000 7f3afcee9b80 1 -- 172.21.15.19:0/3602733822 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fffeee69d0 con 0x55fffefe99d0 2024-05-09T02:12:12.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.706+0000 7f3ae7fff700 1 --2- 172.21.15.19:0/3602733822 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55fffefe99d0 0x55fffefebe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:12.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.709+0000 7f3ae7fff700 1 --2- 172.21.15.19:0/3602733822 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55fffefe99d0 0x55fffefebe50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=47b2dc024049eeb8 server_cookie=e370ce35f824f90b in_seq=0 out_seq=0 2024-05-09T02:12:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.709+0000 7f3ac77fe700 10 client.25162 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:12:12.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.710+0000 7f3ac77fe700 1 -- 172.21.15.19:0/3602733822 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fffeee69d0 con 0x55fffefe99d0 2024-05-09T02:12:12.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.710+0000 7f3ac77fe700 10 client.25162 handle_command_reply: tid=0 2024-05-09T02:12:12.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.710+0000 7f3afcee9b80 1 client.25162 shutdown 2024-05-09T02:12:12.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.710+0000 7f3afcee9b80 1 -- 172.21.15.19:0/3602733822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffefb66a0 msgr2=0x55fffefc0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:12.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.710+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/3602733822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffefb66a0 0x55fffefc0d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0011c20 tx=0x7f3ae0011f30 comp rx=0 tx=0).stop 2024-05-09T02:12:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.719+0000 7f3afcee9b80 1 -- 172.21.15.19:0/3602733822 shutdown_connections 2024-05-09T02:12:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.719+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/3602733822 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55fffefe99d0 0x55fffefebe50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.719+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/3602733822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffefb66a0 0x55fffefc0d40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.719+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/3602733822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffee394a0 0x55fffefc02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.719+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/3602733822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 0x55fffefc0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.719+0000 7f3afcee9b80 1 -- 172.21.15.19:0/3602733822 >> 172.21.15.19:0/3602733822 conn(0x55fffee996a0 msgr2=0x55fffefb11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:12.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.721+0000 7f3ac77fe700 0 client.25162 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:12:12.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.722+0000 7f3afcee9b80 1 -- 172.21.15.19:0/3602733822 shutdown_connections 2024-05-09T02:12:12.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.723+0000 7f3afcee9b80 1 -- 172.21.15.19:0/3602733822 wait complete. 2024-05-09T02:12:12.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.725+0000 7f3afcee9b80 20 client.25162 trim_cache size 0 max 16384 2024-05-09T02:12:12.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.748+0000 7f3afcee9b80 1 Processor -- start 2024-05-09T02:12:12.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.748+0000 7f3afcee9b80 1 -- start start 2024-05-09T02:12:12.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.748+0000 7f3afcee9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffee394a0 0x55fffefb9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.748+0000 7f3afcee9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffef23160 0x55fffefba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.748+0000 7f3afcee9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffefb66a0 0x55fffefba880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.748+0000 7f3afcee9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae80727b0 con 0x55fffef23160 2024-05-09T02:12:12.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.748+0000 7f3afcee9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8074380 con 0x55fffee394a0 2024-05-09T02:12:12.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.748+0000 7f3afcee9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8072e80 con 0x55fffefb66a0 2024-05-09T02:12:12.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.749+0000 7f3adffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffef23160 0x55fffefba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:12.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.749+0000 7f3adffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffef23160 0x55fffefba340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39718/0 (socket says 172.21.15.19:39718) 2024-05-09T02:12:12.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.749+0000 7f3adffff700 1 -- 172.21.15.19:0/604994215 learned_addr learned my addr 172.21.15.19:0/604994215 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:12.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.749+0000 7f3aeca70700 1 --2- 172.21.15.19:0/604994215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffefb66a0 0x55fffefba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:12.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.749+0000 7f3ae7fff700 1 --2- 172.21.15.19:0/604994215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffee394a0 0x55fffefb9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:12.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.749+0000 7f3adffff700 1 -- 172.21.15.19:0/604994215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffefb66a0 msgr2=0x55fffefba880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:12.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.749+0000 7f3adffff700 1 --2- 172.21.15.19:0/604994215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffefb66a0 0x55fffefba880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.749+0000 7f3adffff700 1 -- 172.21.15.19:0/604994215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffee394a0 msgr2=0x55fffefb9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:12.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.749+0000 7f3adffff700 1 --2- 172.21.15.19:0/604994215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffee394a0 0x55fffefb9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.749+0000 7f3adffff700 1 -- 172.21.15.19:0/604994215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fffefb2cc0 con 0x55fffef23160 2024-05-09T02:12:12.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.749+0000 7f3adffff700 1 --2- 172.21.15.19:0/604994215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffef23160 0x55fffefba340 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00aa340 tx=0x7f3ad00caf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1c3bba1ebc2d72 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:12.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.749+0000 7f3addffb700 1 -- 172.21.15.19:0/604994215 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00ae040 con 0x55fffef23160 2024-05-09T02:12:12.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.750+0000 7f3addffb700 1 -- 172.21.15.19:0/604994215 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ad00a7130 con 0x55fffef23160 2024-05-09T02:12:12.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.750+0000 7f3addffb700 1 -- 172.21.15.19:0/604994215 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00b2910 con 0x55fffef23160 2024-05-09T02:12:12.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.751+0000 7f3afcee9b80 1 -- 172.21.15.19:0/604994215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffef23160 msgr2=0x55fffefba340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:12.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.751+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/604994215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffef23160 0x55fffefba340 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00aa340 tx=0x7f3ad00caf90 comp rx=0 tx=0).stop 2024-05-09T02:12:12.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.751+0000 7f3afcee9b80 1 -- 172.21.15.19:0/604994215 shutdown_connections 2024-05-09T02:12:12.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.751+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/604994215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffefb66a0 0x55fffefba880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.751+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/604994215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffee394a0 0x55fffefb9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.751+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/604994215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffef23160 0x55fffefba340 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.751+0000 7f3afcee9b80 1 -- 172.21.15.19:0/604994215 >> 172.21.15.19:0/604994215 conn(0x55fffee996a0 msgr2=0x55fffefbebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:12.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.751+0000 7f3afcee9b80 1 -- 172.21.15.19:0/604994215 shutdown_connections 2024-05-09T02:12:12.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.751+0000 7f3afcee9b80 1 -- 172.21.15.19:0/604994215 wait complete. 2024-05-09T02:12:12.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.752+0000 7f3afcee9b80 1 -- start start 2024-05-09T02:12:12.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.752+0000 7f3afcee9b80 1 Processor -- start 2024-05-09T02:12:12.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.753+0000 7f3afcee9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffee394a0 0x55fffefd0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.753+0000 7f3afcee9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 0x55fffefd11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.753+0000 7f3ae7fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffee394a0 0x55fffefd0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:12.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.753+0000 7f3ae7fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffee394a0 0x55fffefd0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40968/0 (socket says 172.21.15.19:40968) 2024-05-09T02:12:12.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.753+0000 7f3adffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 0x55fffefd11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:12.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.753+0000 7f3adffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 0x55fffefd11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:43864/0 (socket says 172.21.15.19:43864) 2024-05-09T02:12:12.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.754+0000 7f3ae7fff700 1 -- 172.21.15.19:0/4189221573 learned_addr learned my addr 172.21.15.19:0/4189221573 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:12.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.754+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/4189221573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffefb66a0 0x55fffeff0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.755+0000 7f3aeca70700 1 --2- 172.21.15.19:0/4189221573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffefb66a0 0x55fffeff0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:12.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.755+0000 7f3afcee9b80 1 -- 172.21.15.19:0/4189221573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8073c20 con 0x55fffefb66a0 2024-05-09T02:12:12.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.755+0000 7f3afcee9b80 1 -- 172.21.15.19:0/4189221573 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8073550 con 0x55fffef23160 2024-05-09T02:12:12.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.755+0000 7f3afcee9b80 1 -- 172.21.15.19:0/4189221573 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8070750 con 0x55fffee394a0 2024-05-09T02:12:12.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.755+0000 7f3ae7fff700 1 -- 172.21.15.19:0/4189221573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 msgr2=0x55fffefd11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:12.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.755+0000 7f3ae7fff700 1 --2- 172.21.15.19:0/4189221573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 0x55fffefd11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.755+0000 7f3ae7fff700 1 -- 172.21.15.19:0/4189221573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffefb66a0 msgr2=0x55fffeff0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:12.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.755+0000 7f3ae7fff700 1 --2- 172.21.15.19:0/4189221573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffefb66a0 0x55fffeff0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.755+0000 7f3ae7fff700 1 -- 172.21.15.19:0/4189221573 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fffeff0b80 con 0x55fffee394a0 2024-05-09T02:12:12.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.756+0000 7f3afcee9b80 10 client.0 _ll_register_callbacks cb 0x55fffed4a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:12.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.756+0000 7f3ae7fff700 1 --2- 172.21.15.19:0/4189221573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffee394a0 0x55fffefd0c70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8015000 tx=0x7f3ad8005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3746ea7544b2cb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:12.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.756+0000 7f3afcee9b80 1 -- 172.21.15.19:0/4189221573 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ad00a7130 con 0x55fffee394a0 2024-05-09T02:12:12.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.756+0000 7f3ac77fe700 10 client.25168 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:12:12.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.756+0000 7f3ac77fe700 1 -- 172.21.15.19:0/4189221573 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80039d0 con 0x55fffee394a0 2024-05-09T02:12:12.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.756+0000 7f3ac77fe700 1 -- 172.21.15.19:0/4189221573 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f3ad8003b70 con 0x55fffee394a0 2024-05-09T02:12:12.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.756+0000 7f3ac77fe700 1 -- 172.21.15.19:0/4189221573 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3ad8003d90 con 0x55fffee394a0 2024-05-09T02:12:12.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.756+0000 7f3afcee9b80 10 client.25168 fetch_fsmap learned FSMap version 32 2024-05-09T02:12:12.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.756+0000 7f3afcee9b80 1 -- 172.21.15.19:0/4189221573 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55ffff053700 con 0x55fffee394a0 2024-05-09T02:12:12.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.757+0000 7f3ac77fe700 1 -- 172.21.15.19:0/4189221573 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f3ad802e390 con 0x55fffee394a0 2024-05-09T02:12:12.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.757+0000 7f3afcee9b80 10 client.25168 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T02:12:12.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.757+0000 7f3afcee9b80 10 client.25168 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:12:12.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.757+0000 7f3afcee9b80 20 client.25168 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:12.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.757+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/4189221573 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55ffff053d00 0x55ffff0561f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:12.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.757+0000 7f3afcee9b80 4 client.25168 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:12:12.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.757+0000 7f3afcee9b80 1 -- 172.21.15.19:0/4189221573 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fffeee0a70 con 0x55ffff053d00 2024-05-09T02:12:12.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.758+0000 7f3adffff700 1 --2- 172.21.15.19:0/4189221573 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55ffff053d00 0x55ffff0561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:12.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.758+0000 7f3adffff700 1 --2- 172.21.15.19:0/4189221573 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55ffff053d00 0x55ffff0561f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=44a1c1941cd1fdc7 server_cookie=fd92437da4d4d2a9 in_seq=0 out_seq=0 2024-05-09T02:12:12.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.758+0000 7f3ac77fe700 10 client.25168 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:12:12.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.759+0000 7f3ac77fe700 1 -- 172.21.15.19:0/4189221573 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fffeee0a70 con 0x55ffff053d00 2024-05-09T02:12:12.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.759+0000 7f3ac77fe700 10 client.25168 handle_command_reply: tid=0 2024-05-09T02:12:12.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.760+0000 7f3afcee9b80 1 client.25168 shutdown 2024-05-09T02:12:12.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.761+0000 7f3afcee9b80 1 -- 172.21.15.19:0/4189221573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffee394a0 msgr2=0x55fffefd0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:12.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.761+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/4189221573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffee394a0 0x55fffefd0c70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8015000 tx=0x7f3ad8005050 comp rx=0 tx=0).stop 2024-05-09T02:12:12.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.761+0000 7f3afcee9b80 1 -- 172.21.15.19:0/4189221573 shutdown_connections 2024-05-09T02:12:12.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.761+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/4189221573 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55ffff053d00 0x55ffff0561f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.761+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/4189221573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fffefb66a0 0x55fffeff0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.761+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/4189221573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fffee394a0 0x55fffefd0c70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.761+0000 7f3afcee9b80 1 --2- 172.21.15.19:0/4189221573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fffef23160 0x55fffefd11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.761+0000 7f3ac77fe700 0 client.25168 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:12:12.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.761+0000 7f3afcee9b80 1 -- 172.21.15.19:0/4189221573 >> 172.21.15.19:0/4189221573 conn(0x55fffee996a0 msgr2=0x55fffefb9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:12.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.761+0000 7f3afcee9b80 1 -- 172.21.15.19:0/4189221573 shutdown_connections 2024-05-09T02:12:12.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.761+0000 7f3afcee9b80 1 -- 172.21.15.19:0/4189221573 wait complete. 2024-05-09T02:12:12.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.762+0000 7f3afcee9b80 20 client.25168 trim_cache size 0 max 16384 2024-05-09T02:12:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:12:12.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.762+0000 7f3aee4d3700 1 -- 172.21.15.19:0/2900072185 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3ad40b2c50 msgr2=0x7f3ad40b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:12.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.762+0000 7f3aee4d3700 1 --2- 172.21.15.19:0/2900072185 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3ad40b2c50 0x7f3ad40b5160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3ad80098e0 tx=0x7f3ad8000bc0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.762+0000 7f3aee4d3700 1 -- 172.21.15.19:0/2900072185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ae8107e10 msgr2=0x7f3ae819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:12.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.762+0000 7f3aee4d3700 1 --2- 172.21.15.19:0/2900072185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ae8107e10 0x7f3ae819b440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000ea30 tx=0x7f3ad000edf0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.763+0000 7f3aee4d3700 1 -- 172.21.15.19:0/2900072185 shutdown_connections 2024-05-09T02:12:12.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.763+0000 7f3aee4d3700 1 --2- 172.21.15.19:0/2900072185 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3ad40b2c50 0x7f3ad40b5160 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.763+0000 7f3aee4d3700 1 --2- 172.21.15.19:0/2900072185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae8108760 0x7f3ae819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.763+0000 7f3aee4d3700 1 --2- 172.21.15.19:0/2900072185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ae81034e0 0x7f3ae819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.763+0000 7f3aee4d3700 1 --2- 172.21.15.19:0/2900072185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ae8107e10 0x7f3ae819b440 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:12.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.763+0000 7f3aee4d3700 1 -- 172.21.15.19:0/2900072185 >> 172.21.15.19:0/2900072185 conn(0x7f3ae80fef30 msgr2=0x7f3ae8107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:12.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.763+0000 7f3aee4d3700 1 -- 172.21.15.19:0/2900072185 shutdown_connections 2024-05-09T02:12:12.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:12.763+0000 7f3aee4d3700 1 -- 172.21.15.19:0/2900072185 wait complete. 2024-05-09T02:12:13.612 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T02:12:14.002 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:12:14.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:14 smithi019 ceph-mon[30151]: pgmap v31: 97 pgs: 97 active+clean; 1012 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 44 MiB/s wr, 1.29k op/s 2024-05-09T02:12:14.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:14 smithi059 ceph-mon[34781]: pgmap v31: 97 pgs: 97 active+clean; 1012 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 44 MiB/s wr, 1.29k op/s 2024-05-09T02:12:14.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:14 smithi196 ceph-mon[34718]: pgmap v31: 97 pgs: 97 active+clean; 1012 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 44 MiB/s wr, 1.29k op/s 2024-05-09T02:12:15.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.938+0000 7f6039c43700 1 -- 172.21.15.19:0/1972571780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6034106ed0 msgr2=0x7f6034113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:15.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.939+0000 7f6039c43700 1 --2- 172.21.15.19:0/1972571780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6034106ed0 0x7f6034113360 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f602c00b0a0 tx=0x7f602c00b3b0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.939+0000 7f6039c43700 1 -- 172.21.15.19:0/1972571780 shutdown_connections 2024-05-09T02:12:15.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.939+0000 7f6039c43700 1 --2- 172.21.15.19:0/1972571780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6034106ed0 0x7f6034113360 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.939+0000 7f6039c43700 1 --2- 172.21.15.19:0/1972571780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f603410c5c0 0x7f603410c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.939+0000 7f6039c43700 1 --2- 172.21.15.19:0/1972571780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6034106580 0x7f6034106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.939+0000 7f6039c43700 1 -- 172.21.15.19:0/1972571780 >> 172.21.15.19:0/1972571780 conn(0x7f6034102080 msgr2=0x7f60341044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:15.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.939+0000 7f6039c43700 1 -- 172.21.15.19:0/1972571780 shutdown_connections 2024-05-09T02:12:15.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.939+0000 7f6039c43700 1 -- 172.21.15.19:0/1972571780 wait complete. 2024-05-09T02:12:15.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.941+0000 7f6039c43700 1 Processor -- start 2024-05-09T02:12:15.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.941+0000 7f6039c43700 1 -- start start 2024-05-09T02:12:15.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.942+0000 7f6039c43700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6034106580 0x7f6034110790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.942+0000 7f6039c43700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6034106ed0 0x7f6034110cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.942+0000 7f6039c43700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f603410c5c0 0x7f6034111210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.942+0000 7f6039c43700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60341166f0 con 0x7f6034106580 2024-05-09T02:12:15.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.942+0000 7f6039c43700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6034116570 con 0x7f603410c5c0 2024-05-09T02:12:15.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.942+0000 7f6039c43700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6034116870 con 0x7f6034106ed0 2024-05-09T02:12:15.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.942+0000 7f60337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6034106580 0x7f6034110790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.942+0000 7f6032ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6034106ed0 0x7f6034110cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.942+0000 7f60337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6034106580 0x7f6034110790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39736/0 (socket says 172.21.15.19:39736) 2024-05-09T02:12:15.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.942+0000 7f60337fe700 1 -- 172.21.15.19:0/1816982194 learned_addr learned my addr 172.21.15.19:0/1816982194 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:15.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.942+0000 7f6033fff700 1 --2- 172.21.15.19:0/1816982194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f603410c5c0 0x7f6034111210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.943+0000 7f60337fe700 1 -- 172.21.15.19:0/1816982194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6034106ed0 msgr2=0x7f6034110cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:15.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.943+0000 7f60337fe700 1 --2- 172.21.15.19:0/1816982194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6034106ed0 0x7f6034110cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.943+0000 7f60337fe700 1 -- 172.21.15.19:0/1816982194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f603410c5c0 msgr2=0x7f6034111210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:15.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.943+0000 7f60337fe700 1 --2- 172.21.15.19:0/1816982194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f603410c5c0 0x7f6034111210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.943+0000 7f60337fe700 1 -- 172.21.15.19:0/1816982194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60341a2c90 con 0x7f6034106580 2024-05-09T02:12:15.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.943+0000 7f6032ffd700 1 --2- 172.21.15.19:0/1816982194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6034106ed0 0x7f6034110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:12:15.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.943+0000 7f6033fff700 1 --2- 172.21.15.19:0/1816982194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f603410c5c0 0x7f6034111210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:12:15.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.943+0000 7f60337fe700 1 --2- 172.21.15.19:0/1816982194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6034106580 0x7f6034110790 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f602400d980 tx=0x7f602400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:15.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.943+0000 7f6030ff9700 1 -- 172.21.15.19:0/1816982194 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024017070 con 0x7f6034106580 2024-05-09T02:12:15.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.943+0000 7f6030ff9700 1 -- 172.21.15.19:0/1816982194 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602400fd40 con 0x7f6034106580 2024-05-09T02:12:15.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.943+0000 7f6039c43700 1 -- 172.21.15.19:0/1816982194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60341a2f80 con 0x7f6034106580 2024-05-09T02:12:15.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.943+0000 7f6030ff9700 1 -- 172.21.15.19:0/1816982194 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602400e3b0 con 0x7f6034106580 2024-05-09T02:12:15.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.943+0000 7f6039c43700 1 -- 172.21.15.19:0/1816982194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60341a3510 con 0x7f6034106580 2024-05-09T02:12:15.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.948+0000 7f6030ff9700 1 -- 172.21.15.19:0/1816982194 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f602400f890 con 0x7f6034106580 2024-05-09T02:12:15.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.948+0000 7f6048659b80 1 Processor -- start 2024-05-09T02:12:15.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.948+0000 7f6048659b80 1 -- start start 2024-05-09T02:12:15.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.949+0000 7f6048659b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e9340cd00 0x557e93515240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.949+0000 7f6048659b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e933924a0 0x557e93515780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.949+0000 7f6048659b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e9350f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.949+0000 7f6048659b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557e934320a0 con 0x557e9340cd00 2024-05-09T02:12:15.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.949+0000 7f6048659b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557e933d79a0 con 0x557e9350f400 2024-05-09T02:12:15.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.949+0000 7f6048659b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e9342e060 con 0x557e933924a0 2024-05-09T02:12:15.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.949+0000 7f6033fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e9340cd00 0x557e93515240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.949+0000 7f6033fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e9340cd00 0x557e93515240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39752/0 (socket says 172.21.15.19:39752) 2024-05-09T02:12:15.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.949+0000 7f6033fff700 1 -- 172.21.15.19:0/2616669423 learned_addr learned my addr 172.21.15.19:0/2616669423 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:15.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.949+0000 7f6030ff9700 1 --2- 172.21.15.19:0/1816982194 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f601c0b2b90 0x7f601c0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.949+0000 7f6030ff9700 1 -- 172.21.15.19:0/1816982194 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f60240e2680 con 0x7f6034106580 2024-05-09T02:12:15.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.949+0000 7f60337fe700 1 --2- 172.21.15.19:0/2616669423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e933924a0 0x557e93515780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.950+0000 7f6033fff700 1 --2- 172.21.15.19:0/1816982194 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f601c0b2b90 0x7f601c0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.950+0000 7f6033fff700 1 -- 172.21.15.19:0/2616669423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e933924a0 msgr2=0x557e93515780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.951+0000 7f6033fff700 1 --2- 172.21.15.19:0/2616669423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e933924a0 0x557e93515780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.951+0000 7f6033fff700 1 -- 172.21.15.19:0/2616669423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 msgr2=0x557e9350f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.951+0000 7f6033fff700 1 --2- 172.21.15.19:0/2616669423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e9350f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.951+0000 7f6033fff700 1 -- 172.21.15.19:0/2616669423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e933c8500 con 0x557e9340cd00 2024-05-09T02:12:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.951+0000 7f60337fe700 1 --2- 172.21.15.19:0/2616669423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e933924a0 0x557e93515780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:12:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.952+0000 7f6033fff700 1 --2- 172.21.15.19:0/2616669423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e9340cd00 0x557e93515240 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f602c015000 tx=0x7f602c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd804f1e2f96ee03 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.952+0000 7f6033fff700 1 --2- 172.21.15.19:0/1816982194 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f601c0b2b90 0x7f601c0b50a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f602c026000 tx=0x7f602c02a000 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:15.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.952+0000 7f6019ffb700 1 -- 172.21.15.19:0/2616669423 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c007aa0 con 0x557e9340cd00 2024-05-09T02:12:15.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.952+0000 7f6019ffb700 1 -- 172.21.15.19:0/2616669423 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602c007c40 con 0x557e9340cd00 2024-05-09T02:12:15.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.952+0000 7f6019ffb700 1 -- 172.21.15.19:0/2616669423 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c005040 con 0x557e9340cd00 2024-05-09T02:12:15.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.953+0000 7f6048659b80 1 -- 172.21.15.19:0/2616669423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e9340cd00 msgr2=0x557e93515240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:15.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.953+0000 7f6048659b80 1 --2- 172.21.15.19:0/2616669423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e9340cd00 0x557e93515240 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f602c015000 tx=0x7f602c004b50 comp rx=0 tx=0).stop 2024-05-09T02:12:15.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.953+0000 7f6048659b80 1 -- 172.21.15.19:0/2616669423 shutdown_connections 2024-05-09T02:12:15.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.953+0000 7f6048659b80 1 --2- 172.21.15.19:0/2616669423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e9350f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.953+0000 7f6048659b80 1 --2- 172.21.15.19:0/2616669423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e9340cd00 0x557e93515240 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.953+0000 7f6048659b80 1 --2- 172.21.15.19:0/2616669423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e933924a0 0x557e93515780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.953+0000 7f6048659b80 1 -- 172.21.15.19:0/2616669423 >> 172.21.15.19:0/2616669423 conn(0x557e93395f70 msgr2=0x557e9350a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:15.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.954+0000 7f6048659b80 1 -- 172.21.15.19:0/2616669423 shutdown_connections 2024-05-09T02:12:15.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.954+0000 7f6048659b80 1 -- 172.21.15.19:0/2616669423 wait complete. 2024-05-09T02:12:15.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.954+0000 7f6048659b80 1 -- start start 2024-05-09T02:12:15.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.955+0000 7f6048659b80 1 Processor -- start 2024-05-09T02:12:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.955+0000 7f6048659b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e933924a0 0x557e935190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.955+0000 7f6048659b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e9340cd00 0x557e93519630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.955+0000 7f6048659b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e93519b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.955+0000 7f6048659b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557e93510100 con 0x557e9340cd00 2024-05-09T02:12:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.955+0000 7f6048659b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557e933ccb20 con 0x557e9350f400 2024-05-09T02:12:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.955+0000 7f6048659b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e93525250 con 0x557e933924a0 2024-05-09T02:12:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.955+0000 7f6048659b80 10 client.0 _ll_register_callbacks cb 0x557e932a36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.956+0000 7f6033fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e93519b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.956+0000 7f6033fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e93519b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:43904/0 (socket says 172.21.15.19:43904) 2024-05-09T02:12:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.956+0000 7f6033fff700 1 -- 172.21.15.19:0/380084683 learned_addr learned my addr 172.21.15.19:0/380084683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.956+0000 7f6032ffd700 1 --2- 172.21.15.19:0/380084683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e9340cd00 0x557e93519630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.956+0000 7f60337fe700 1 --2- 172.21.15.19:0/380084683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e933924a0 0x557e935190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.957+0000 7f6033fff700 1 -- 172.21.15.19:0/380084683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e933924a0 msgr2=0x557e935190f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.957+0000 7f6033fff700 1 --2- 172.21.15.19:0/380084683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e933924a0 0x557e935190f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.957+0000 7f6033fff700 1 -- 172.21.15.19:0/380084683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e9340cd00 msgr2=0x557e93519630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.957+0000 7f6033fff700 1 --2- 172.21.15.19:0/380084683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e9340cd00 0x557e93519630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.957+0000 7f6033fff700 1 -- 172.21.15.19:0/380084683 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e932ad490 con 0x557e9350f400 2024-05-09T02:12:15.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.957+0000 7f6033fff700 1 --2- 172.21.15.19:0/380084683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e93519b70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f602c0061e0 tx=0x7f602c004f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e31b7911b4e8aea6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:15.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.957+0000 7f6012ffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:12:15.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.957+0000 7f6012ffd700 1 -- 172.21.15.19:0/380084683 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c02a660 con 0x557e9350f400 2024-05-09T02:12:15.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.958+0000 7f6012ffd700 1 -- 172.21.15.19:0/380084683 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f602c02a800 con 0x557e9350f400 2024-05-09T02:12:15.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.958+0000 7f6032ffd700 1 --2- 172.21.15.19:0/380084683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e9340cd00 0x557e93519630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:12:15.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.958+0000 7f6048659b80 1 -- 172.21.15.19:0/380084683 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f602c007c40 con 0x557e9350f400 2024-05-09T02:12:15.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.958+0000 7f60337fe700 1 --2- 172.21.15.19:0/380084683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e933924a0 0x557e935190f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:12:15.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.958+0000 7f6012ffd700 1 -- 172.21.15.19:0/380084683 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f602c033480 con 0x557e9350f400 2024-05-09T02:12:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.958+0000 7f6048659b80 10 client.25142 fetch_fsmap learned FSMap version 32 2024-05-09T02:12:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.958+0000 7f6048659b80 1 -- 172.21.15.19:0/380084683 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557e935269c0 con 0x557e9350f400 2024-05-09T02:12:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.959+0000 7f6012ffd700 1 -- 172.21.15.19:0/380084683 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602c007c40 con 0x557e9350f400 2024-05-09T02:12:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.959+0000 7f6012ffd700 1 -- 172.21.15.19:0/380084683 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f602c011070 con 0x557e9350f400 2024-05-09T02:12:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.959+0000 7f6048659b80 10 client.25142 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T02:12:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.959+0000 7f6048659b80 10 client.25142 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:12:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.959+0000 7f6048659b80 20 client.25142 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.959+0000 7f6048659b80 1 --2- 172.21.15.19:0/380084683 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557e93528e90 0x557e9352b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.959+0000 7f6048659b80 4 client.25142 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:12:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.959+0000 7f6048659b80 1 -- 172.21.15.19:0/380084683 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e9343fa80 con 0x557e93528e90 2024-05-09T02:12:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.960+0000 7f60337fe700 1 --2- 172.21.15.19:0/380084683 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557e93528e90 0x557e9352b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.960+0000 7f6012ffd700 10 client.25142 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:12:15.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.960+0000 7f60337fe700 1 --2- 172.21.15.19:0/380084683 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557e93528e90 0x557e9352b2b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=566a588be60e9bb5 server_cookie=62d2234298cae655 in_seq=0 out_seq=0 2024-05-09T02:12:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.961+0000 7f6012ffd700 1 -- 172.21.15.19:0/380084683 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e9343fa80 con 0x557e93528e90 2024-05-09T02:12:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.961+0000 7f6012ffd700 10 client.25142 handle_command_reply: tid=0 2024-05-09T02:12:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.961+0000 7f6048659b80 1 client.25142 shutdown 2024-05-09T02:12:15.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.963+0000 7f6048659b80 1 -- 172.21.15.19:0/380084683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 msgr2=0x557e93519b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:15.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.963+0000 7f6048659b80 1 --2- 172.21.15.19:0/380084683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e93519b70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f602c0061e0 tx=0x7f602c004f50 comp rx=0 tx=0).stop 2024-05-09T02:12:15.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.963+0000 7f6048659b80 1 -- 172.21.15.19:0/380084683 shutdown_connections 2024-05-09T02:12:15.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.963+0000 7f6048659b80 1 --2- 172.21.15.19:0/380084683 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557e93528e90 0x557e9352b2b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.963+0000 7f6048659b80 1 --2- 172.21.15.19:0/380084683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e93519b70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.963+0000 7f6048659b80 1 --2- 172.21.15.19:0/380084683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e933924a0 0x557e935190f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.963+0000 7f6048659b80 1 --2- 172.21.15.19:0/380084683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e9340cd00 0x557e93519630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.963+0000 7f6012ffd700 0 client.25142 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:12:15.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.963+0000 7f6048659b80 1 -- 172.21.15.19:0/380084683 >> 172.21.15.19:0/380084683 conn(0x557e93395f70 msgr2=0x557e93503ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:15.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.963+0000 7f6048659b80 1 -- 172.21.15.19:0/380084683 shutdown_connections 2024-05-09T02:12:15.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.963+0000 7f6048659b80 1 -- 172.21.15.19:0/380084683 wait complete. 2024-05-09T02:12:15.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.963+0000 7f6048659b80 20 client.25142 trim_cache size 0 max 16384 2024-05-09T02:12:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.987+0000 7f6048659b80 1 Processor -- start 2024-05-09T02:12:15.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.988+0000 7f6048659b80 1 -- start start 2024-05-09T02:12:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.988+0000 7f6048659b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e933924a0 0x557e935199b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f6048659b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e9340cd00 0x557e9351b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f6048659b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e93504700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f6048659b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60341bdce0 con 0x557e933924a0 2024-05-09T02:12:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f6048659b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f60341bf8b0 con 0x557e9350f400 2024-05-09T02:12:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f6048659b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60341be3b0 con 0x557e9340cd00 2024-05-09T02:12:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f60337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e933924a0 0x557e935199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f6033fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e93504700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f6032ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e9340cd00 0x557e9351b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f6033fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e93504700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:43912/0 (socket says 172.21.15.19:43912) 2024-05-09T02:12:15.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f6033fff700 1 -- 172.21.15.19:0/4179622443 learned_addr learned my addr 172.21.15.19:0/4179622443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:15.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f6033fff700 1 -- 172.21.15.19:0/4179622443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e9340cd00 msgr2=0x557e9351b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:15.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f6033fff700 1 --2- 172.21.15.19:0/4179622443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e9340cd00 0x557e9351b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f6033fff700 1 -- 172.21.15.19:0/4179622443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e933924a0 msgr2=0x557e935199b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:15.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f6033fff700 1 --2- 172.21.15.19:0/4179622443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e933924a0 0x557e935199b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f6033fff700 1 -- 172.21.15.19:0/4179622443 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e935069d0 con 0x557e9350f400 2024-05-09T02:12:15.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.989+0000 7f6033fff700 1 --2- 172.21.15.19:0/4179622443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e93504700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f602c002230 tx=0x7f602c009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7243ca0c37fe8a91 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:15.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.990+0000 7f60197fa700 1 -- 172.21.15.19:0/4179622443 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c007c40 con 0x557e9350f400 2024-05-09T02:12:15.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.990+0000 7f60197fa700 1 -- 172.21.15.19:0/4179622443 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602c038070 con 0x557e9350f400 2024-05-09T02:12:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.990+0000 7f6048659b80 1 -- 172.21.15.19:0/4179622443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 msgr2=0x557e93504700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.990+0000 7f6048659b80 1 --2- 172.21.15.19:0/4179622443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e93504700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f602c002230 tx=0x7f602c009300 comp rx=0 tx=0).stop 2024-05-09T02:12:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.991+0000 7f6048659b80 1 -- 172.21.15.19:0/4179622443 shutdown_connections 2024-05-09T02:12:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.991+0000 7f6048659b80 1 --2- 172.21.15.19:0/4179622443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e93504700 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.991+0000 7f6048659b80 1 --2- 172.21.15.19:0/4179622443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e933924a0 0x557e935199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.991+0000 7f6048659b80 1 --2- 172.21.15.19:0/4179622443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e9340cd00 0x557e9351b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.991+0000 7f6048659b80 1 -- 172.21.15.19:0/4179622443 >> 172.21.15.19:0/4179622443 conn(0x557e93395f70 msgr2=0x557e93503ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.991+0000 7f6048659b80 1 -- 172.21.15.19:0/4179622443 shutdown_connections 2024-05-09T02:12:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.991+0000 7f6048659b80 1 -- 172.21.15.19:0/4179622443 wait complete. 2024-05-09T02:12:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.991+0000 7f6048659b80 1 -- start start 2024-05-09T02:12:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.991+0000 7f6048659b80 1 Processor -- start 2024-05-09T02:12:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.993+0000 7f6048659b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e933924a0 0x557e9352b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.993+0000 7f6048659b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e9340cd00 0x557e9352b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.993+0000 7f6048659b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e9352bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.993+0000 7f6048659b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60341bf150 con 0x557e933924a0 2024-05-09T02:12:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.993+0000 7f6048659b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f60341bea80 con 0x557e9350f400 2024-05-09T02:12:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.993+0000 7f6048659b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60341bbc80 con 0x557e9340cd00 2024-05-09T02:12:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.993+0000 7f6048659b80 10 client.0 _ll_register_callbacks cb 0x557e9350fdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.994+0000 7f6033fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e9352bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.994+0000 7f60337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e933924a0 0x557e9352b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.994+0000 7f6033fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e9352bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:43922/0 (socket says 172.21.15.19:43922) 2024-05-09T02:12:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.994+0000 7f60337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e933924a0 0x557e9352b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39774/0 (socket says 172.21.15.19:39774) 2024-05-09T02:12:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.994+0000 7f6033fff700 1 -- 172.21.15.19:0/3595286236 learned_addr learned my addr 172.21.15.19:0/3595286236 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.994+0000 7f6032ffd700 1 --2- 172.21.15.19:0/3595286236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e9340cd00 0x557e9352b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.994+0000 7f6033fff700 1 -- 172.21.15.19:0/3595286236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e9340cd00 msgr2=0x557e9352b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.994+0000 7f6033fff700 1 --2- 172.21.15.19:0/3595286236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e9340cd00 0x557e9352b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.994+0000 7f6033fff700 1 -- 172.21.15.19:0/3595286236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e933924a0 msgr2=0x557e9352b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.994+0000 7f6033fff700 1 --2- 172.21.15.19:0/3595286236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e933924a0 0x557e9352b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.994+0000 7f6033fff700 1 -- 172.21.15.19:0/3595286236 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9350bea0 con 0x557e9350f400 2024-05-09T02:12:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.995+0000 7f6033fff700 1 --2- 172.21.15.19:0/3595286236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e9352bd30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f602c009a20 tx=0x7f602c01b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8659461484d9305 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.995+0000 7f6012ffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:12:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.995+0000 7f6012ffd700 1 -- 172.21.15.19:0/3595286236 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c053060 con 0x557e9350f400 2024-05-09T02:12:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.995+0000 7f6012ffd700 1 -- 172.21.15.19:0/3595286236 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f602c005e60 con 0x557e9350f400 2024-05-09T02:12:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.995+0000 7f6048659b80 1 -- 172.21.15.19:0/3595286236 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557e93439b20 con 0x557e9350f400 2024-05-09T02:12:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.995+0000 7f6012ffd700 1 -- 172.21.15.19:0/3595286236 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f602c007c40 con 0x557e9350f400 2024-05-09T02:12:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.995+0000 7f6048659b80 10 client.25148 fetch_fsmap learned FSMap version 32 2024-05-09T02:12:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.995+0000 7f6048659b80 1 -- 172.21.15.19:0/3595286236 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557e935926b0 con 0x557e9350f400 2024-05-09T02:12:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.995+0000 7f6012ffd700 1 -- 172.21.15.19:0/3595286236 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557e93439b20 con 0x557e9350f400 2024-05-09T02:12:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.996+0000 7f6012ffd700 1 -- 172.21.15.19:0/3595286236 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f602c011040 con 0x557e9350f400 2024-05-09T02:12:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.996+0000 7f6048659b80 10 client.25148 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T02:12:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.996+0000 7f6048659b80 10 client.25148 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:12:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.996+0000 7f6048659b80 20 client.25148 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.996+0000 7f6048659b80 1 --2- 172.21.15.19:0/3595286236 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557e93592ca0 0x557e935950c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.996+0000 7f6048659b80 4 client.25148 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:12:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.996+0000 7f6048659b80 1 -- 172.21.15.19:0/3595286236 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e932ad970 con 0x557e93592ca0 2024-05-09T02:12:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.996+0000 7f60337fe700 1 --2- 172.21.15.19:0/3595286236 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557e93592ca0 0x557e935950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:16.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.996+0000 7f6012ffd700 10 client.25148 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:12:16.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.997+0000 7f60337fe700 1 --2- 172.21.15.19:0/3595286236 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557e93592ca0 0x557e935950c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a194e07163e56698 server_cookie=6ec740e1fbce2134 in_seq=0 out_seq=0 2024-05-09T02:12:16.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.998+0000 7f6012ffd700 1 -- 172.21.15.19:0/3595286236 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e932ad970 con 0x557e93592ca0 2024-05-09T02:12:16.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.998+0000 7f6012ffd700 10 client.25148 handle_command_reply: tid=0 2024-05-09T02:12:16.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.998+0000 7f6048659b80 1 client.25148 shutdown 2024-05-09T02:12:16.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.998+0000 7f6048659b80 1 -- 172.21.15.19:0/3595286236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 msgr2=0x557e9352bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:16.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.998+0000 7f6048659b80 1 --2- 172.21.15.19:0/3595286236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e9352bd30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f602c009a20 tx=0x7f602c01b9d0 comp rx=0 tx=0).stop 2024-05-09T02:12:16.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.998+0000 7f6048659b80 1 -- 172.21.15.19:0/3595286236 shutdown_connections 2024-05-09T02:12:16.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.998+0000 7f6048659b80 1 --2- 172.21.15.19:0/3595286236 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557e93592ca0 0x557e935950c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:16.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.998+0000 7f6048659b80 1 --2- 172.21.15.19:0/3595286236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557e9350f400 0x557e9352bd30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:16.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.998+0000 7f6048659b80 1 --2- 172.21.15.19:0/3595286236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e933924a0 0x557e9352b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:16.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.998+0000 7f6048659b80 1 --2- 172.21.15.19:0/3595286236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e9340cd00 0x557e9352b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:16.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.998+0000 7f6048659b80 1 -- 172.21.15.19:0/3595286236 >> 172.21.15.19:0/3595286236 conn(0x557e93395f70 msgr2=0x557e9350e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:16.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.998+0000 7f6012ffd700 0 client.25148 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:12:16.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.999+0000 7f6048659b80 1 -- 172.21.15.19:0/3595286236 shutdown_connections 2024-05-09T02:12:16.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.999+0000 7f6048659b80 1 -- 172.21.15.19:0/3595286236 wait complete. 2024-05-09T02:12:16.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:15.999+0000 7f6048659b80 20 client.25148 trim_cache size 0 max 16384 2024-05-09T02:12:16.003 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:12:16.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:16.001+0000 7f6039c43700 1 -- 172.21.15.19:0/1816982194 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f601c0b2b90 msgr2=0x7f601c0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:16.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:16.001+0000 7f6039c43700 1 --2- 172.21.15.19:0/1816982194 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f601c0b2b90 0x7f601c0b50a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f602c026000 tx=0x7f602c02a000 comp rx=0 tx=0).stop 2024-05-09T02:12:16.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:16.001+0000 7f6039c43700 1 -- 172.21.15.19:0/1816982194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6034106580 msgr2=0x7f6034110790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:16.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:16.001+0000 7f6039c43700 1 --2- 172.21.15.19:0/1816982194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6034106580 0x7f6034110790 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f602400d980 tx=0x7f602400dc90 comp rx=0 tx=0).stop 2024-05-09T02:12:16.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:16.001+0000 7f6039c43700 1 -- 172.21.15.19:0/1816982194 shutdown_connections 2024-05-09T02:12:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:16.001+0000 7f6039c43700 1 --2- 172.21.15.19:0/1816982194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f603410c5c0 0x7f6034111210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:16.001+0000 7f6039c43700 1 --2- 172.21.15.19:0/1816982194 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f601c0b2b90 0x7f601c0b50a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:16.001+0000 7f6039c43700 1 --2- 172.21.15.19:0/1816982194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6034106580 0x7f6034110790 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:16.001+0000 7f6039c43700 1 --2- 172.21.15.19:0/1816982194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6034106ed0 0x7f6034110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:16.001+0000 7f6039c43700 1 -- 172.21.15.19:0/1816982194 >> 172.21.15.19:0/1816982194 conn(0x7f6034102080 msgr2=0x7f60341046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:16.002+0000 7f6039c43700 1 -- 172.21.15.19:0/1816982194 shutdown_connections 2024-05-09T02:12:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:16.002+0000 7f6039c43700 1 -- 172.21.15.19:0/1816982194 wait complete. 2024-05-09T02:12:16.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:16 smithi059 ceph-mon[34781]: pgmap v32: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 36 MiB/s wr, 1.32k op/s 2024-05-09T02:12:16.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:16 smithi019 ceph-mon[30151]: pgmap v32: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 36 MiB/s wr, 1.32k op/s 2024-05-09T02:12:16.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:16 smithi196 ceph-mon[34718]: pgmap v32: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 36 MiB/s wr, 1.32k op/s 2024-05-09T02:12:16.940 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T02:12:17.337 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:12:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 35.0 s, 3357.2 tps, lat 29.671 ms stddev 30.018 2024-05-09T02:12:18.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:18 smithi019 ceph-mon[30151]: pgmap v33: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.1 MiB/s wr, 740 op/s 2024-05-09T02:12:18.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:18 smithi196 ceph-mon[34718]: pgmap v33: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.1 MiB/s wr, 740 op/s 2024-05-09T02:12:18.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:18 smithi059 ceph-mon[34781]: pgmap v33: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.1 MiB/s wr, 740 op/s 2024-05-09T02:12:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.426+0000 7faa177cb700 1 -- 172.21.15.19:0/689469358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10107df0 msgr2=0x7faa10108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.426+0000 7faa177cb700 1 --2- 172.21.15.19:0/689469358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10107df0 0x7faa10108200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0098f0 tx=0x7faa0c009c00 comp rx=0 tx=0).stop 2024-05-09T02:12:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.426+0000 7faa177cb700 1 -- 172.21.15.19:0/689469358 shutdown_connections 2024-05-09T02:12:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.426+0000 7faa177cb700 1 --2- 172.21.15.19:0/689469358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa10108740 0x7faa1006b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.426+0000 7faa177cb700 1 --2- 172.21.15.19:0/689469358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa101034c0 0x7faa101038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.426+0000 7faa177cb700 1 --2- 172.21.15.19:0/689469358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10107df0 0x7faa10108200 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.426+0000 7faa177cb700 1 -- 172.21.15.19:0/689469358 >> 172.21.15.19:0/689469358 conn(0x7faa100fedc0 msgr2=0x7faa101011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.426+0000 7faa177cb700 1 -- 172.21.15.19:0/689469358 shutdown_connections 2024-05-09T02:12:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.426+0000 7faa177cb700 1 -- 172.21.15.19:0/689469358 wait complete. 2024-05-09T02:12:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.427+0000 7faa177cb700 1 Processor -- start 2024-05-09T02:12:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.427+0000 7faa177cb700 1 -- start start 2024-05-09T02:12:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.427+0000 7faa177cb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa101034c0 0x7faa10112380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.427+0000 7faa177cb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10107df0 0x7faa101128c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.427+0000 7faa177cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa10108740 0x7faa10112e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.427+0000 7faa177cb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa10116840 con 0x7faa101034c0 2024-05-09T02:12:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.427+0000 7faa177cb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa101166c0 con 0x7faa10108740 2024-05-09T02:12:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.427+0000 7faa177cb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa101169c0 con 0x7faa10107df0 2024-05-09T02:12:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.428+0000 7faa14d66700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10107df0 0x7faa101128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.428+0000 7faa14d66700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10107df0 0x7faa101128c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:47228/0 (socket says 172.21.15.19:47228) 2024-05-09T02:12:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.428+0000 7faa14d66700 1 -- 172.21.15.19:0/3174336698 learned_addr learned my addr 172.21.15.19:0/3174336698 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.428+0000 7faa15d68700 1 --2- 172.21.15.19:0/3174336698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa10108740 0x7faa10112e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.428+0000 7faa15567700 1 --2- 172.21.15.19:0/3174336698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa101034c0 0x7faa10112380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.428+0000 7faa15d68700 1 -- 172.21.15.19:0/3174336698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10107df0 msgr2=0x7faa101128c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.428+0000 7faa15d68700 1 --2- 172.21.15.19:0/3174336698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10107df0 0x7faa101128c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.428+0000 7faa15d68700 1 -- 172.21.15.19:0/3174336698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa101034c0 msgr2=0x7faa10112380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.428+0000 7faa15d68700 1 --2- 172.21.15.19:0/3174336698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa101034c0 0x7faa10112380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.428+0000 7faa15d68700 1 -- 172.21.15.19:0/3174336698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa101ad670 con 0x7faa10108740 2024-05-09T02:12:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.428+0000 7faa15d68700 1 --2- 172.21.15.19:0/3174336698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa10108740 0x7faa10112e00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faa08009cd0 tx=0x7faa0800c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.429+0000 7faa067fc700 1 -- 172.21.15.19:0/3174336698 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0800cdc0 con 0x7faa10108740 2024-05-09T02:12:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.429+0000 7faa177cb700 1 -- 172.21.15.19:0/3174336698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa101ad840 con 0x7faa10108740 2024-05-09T02:12:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.429+0000 7faa067fc700 1 -- 172.21.15.19:0/3174336698 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa08003a00 con 0x7faa10108740 2024-05-09T02:12:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.429+0000 7faa067fc700 1 -- 172.21.15.19:0/3174336698 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa080045d0 con 0x7faa10108740 2024-05-09T02:12:19.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.429+0000 7faa177cb700 1 -- 172.21.15.19:0/3174336698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa101add50 con 0x7faa10108740 2024-05-09T02:12:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.433+0000 7faa067fc700 1 -- 172.21.15.19:0/3174336698 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7faa0801a020 con 0x7faa10108740 2024-05-09T02:12:19.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.435+0000 7faa067fc700 1 --2- 172.21.15.19:0/3174336698 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa9fc0b2c10 0x7fa9fc0b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.436+0000 7faa261e1b80 1 Processor -- start 2024-05-09T02:12:19.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.436+0000 7faa261e1b80 1 -- start start 2024-05-09T02:12:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.436+0000 7faa15567700 1 --2- 172.21.15.19:0/3174336698 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa9fc0b2c10 0x7fa9fc0b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.436+0000 7faa067fc700 1 -- 172.21.15.19:0/3174336698 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7faa080e21d0 con 0x7faa10108740 2024-05-09T02:12:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.436+0000 7faa261e1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed496f160 0x55bed4a084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.437+0000 7faa261e1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed48854a0 0x55bed4a08a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.437+0000 7faa261e1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed4a026a0 0x55bed4a02ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.437+0000 7faa261e1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bed4924ff0 con 0x55bed496f160 2024-05-09T02:12:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.437+0000 7faa261e1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bed48c37f0 con 0x55bed48854a0 2024-05-09T02:12:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.437+0000 7faa261e1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bed4920fb0 con 0x55bed4a026a0 2024-05-09T02:12:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.437+0000 7faa14d66700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed48854a0 0x55bed4a08a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.437+0000 7faa15d68700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed4a026a0 0x55bed4a02ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.437+0000 7faa14d66700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed48854a0 0x55bed4a08a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:38308/0 (socket says 172.21.15.19:38308) 2024-05-09T02:12:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.437+0000 7faa14d66700 1 -- 172.21.15.19:0/2538057103 learned_addr learned my addr 172.21.15.19:0/2538057103 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:19.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.437+0000 7faa14d66700 1 -- 172.21.15.19:0/2538057103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed4a026a0 msgr2=0x55bed4a02ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:19.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.437+0000 7faa14d66700 1 --2- 172.21.15.19:0/2538057103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed4a026a0 0x55bed4a02ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.438+0000 7faa14d66700 1 -- 172.21.15.19:0/2538057103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed496f160 msgr2=0x55bed4a084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:19.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.438+0000 7faa14d66700 1 --2- 172.21.15.19:0/2538057103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed496f160 0x55bed4a084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.438+0000 7faa14d66700 1 -- 172.21.15.19:0/2538057103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bed48bb370 con 0x55bed48854a0 2024-05-09T02:12:19.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.438+0000 7faa15567700 1 --2- 172.21.15.19:0/3174336698 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa9fc0b2c10 0x7fa9fc0b5120 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faa00005ea0 tx=0x7faa00005e30 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:19.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.438+0000 7faa14d66700 1 --2- 172.21.15.19:0/2538057103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed48854a0 0x55bed4a08a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009230 tx=0x7faa0c00bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44c3c3b00de4fc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:19.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.439+0000 7fa9fb7fe700 1 -- 172.21.15.19:0/2538057103 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c026070 con 0x55bed48854a0 2024-05-09T02:12:19.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.439+0000 7fa9fb7fe700 1 -- 172.21.15.19:0/2538057103 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa0c005040 con 0x55bed48854a0 2024-05-09T02:12:19.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.439+0000 7fa9fb7fe700 1 -- 172.21.15.19:0/2538057103 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0038e0 con 0x55bed48854a0 2024-05-09T02:12:19.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.440+0000 7faa261e1b80 1 -- 172.21.15.19:0/2538057103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed48854a0 msgr2=0x55bed4a08a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:19.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.440+0000 7faa261e1b80 1 --2- 172.21.15.19:0/2538057103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed48854a0 0x55bed4a08a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009230 tx=0x7faa0c00bf60 comp rx=0 tx=0).stop 2024-05-09T02:12:19.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.440+0000 7faa261e1b80 1 -- 172.21.15.19:0/2538057103 shutdown_connections 2024-05-09T02:12:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.440+0000 7faa261e1b80 1 --2- 172.21.15.19:0/2538057103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed4a026a0 0x55bed4a02ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.440+0000 7faa261e1b80 1 --2- 172.21.15.19:0/2538057103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed496f160 0x55bed4a084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.440+0000 7faa261e1b80 1 --2- 172.21.15.19:0/2538057103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed48854a0 0x55bed4a08a30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.440+0000 7faa261e1b80 1 -- 172.21.15.19:0/2538057103 >> 172.21.15.19:0/2538057103 conn(0x55bed48e56a0 msgr2=0x55bed49fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.440+0000 7faa261e1b80 1 -- 172.21.15.19:0/2538057103 shutdown_connections 2024-05-09T02:12:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.440+0000 7faa261e1b80 1 -- 172.21.15.19:0/2538057103 wait complete. 2024-05-09T02:12:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.440+0000 7faa261e1b80 1 -- start start 2024-05-09T02:12:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.441+0000 7faa261e1b80 1 Processor -- start 2024-05-09T02:12:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa261e1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed48854a0 0x55bed4a0c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa261e1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed496f160 0x55bed4a0c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa261e1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed4a026a0 0x55bed4a0cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa261e1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bed47a0ba0 con 0x55bed4a026a0 2024-05-09T02:12:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa15567700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed48854a0 0x55bed4a0c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa14d66700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed496f160 0x55bed4a0c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa14d66700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed496f160 0x55bed4a0c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:38322/0 (socket says 172.21.15.19:38322) 2024-05-09T02:12:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa15567700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed48854a0 0x55bed4a0c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:47260/0 (socket says 172.21.15.19:47260) 2024-05-09T02:12:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa14d66700 1 -- 172.21.15.19:0/3233368237 learned_addr learned my addr 172.21.15.19:0/3233368237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa261e1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bed48df150 con 0x55bed496f160 2024-05-09T02:12:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa261e1b80 1 -- 172.21.15.19:0/3233368237 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bed4a0f1d0 con 0x55bed48854a0 2024-05-09T02:12:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa261e1b80 10 client.0 _ll_register_callbacks cb 0x55bed47966c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa15d68700 1 --2- 172.21.15.19:0/3233368237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed4a026a0 0x55bed4a0cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa14d66700 1 -- 172.21.15.19:0/3233368237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed48854a0 msgr2=0x55bed4a0c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa14d66700 1 --2- 172.21.15.19:0/3233368237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed48854a0 0x55bed4a0c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa14d66700 1 -- 172.21.15.19:0/3233368237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed4a026a0 msgr2=0x55bed4a0cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.442+0000 7faa14d66700 1 --2- 172.21.15.19:0/3233368237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed4a026a0 0x55bed4a0cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.443+0000 7faa14d66700 1 -- 172.21.15.19:0/3233368237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bed47a0490 con 0x55bed496f160 2024-05-09T02:12:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.443+0000 7faa14d66700 1 --2- 172.21.15.19:0/3233368237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed496f160 0x55bed4a0c740 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009810 tx=0x7faa0c009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=385030c9d0263574 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:19.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.443+0000 7fa9f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:12:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.443+0000 7fa9f8ff9700 1 -- 172.21.15.19:0/3233368237 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c026070 con 0x55bed496f160 2024-05-09T02:12:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.443+0000 7fa9f8ff9700 1 -- 172.21.15.19:0/3233368237 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7faa0c003680 con 0x55bed496f160 2024-05-09T02:12:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.443+0000 7faa261e1b80 1 -- 172.21.15.19:0/3233368237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa0c005040 con 0x55bed496f160 2024-05-09T02:12:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.443+0000 7fa9f8ff9700 1 -- 172.21.15.19:0/3233368237 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faa0c038050 con 0x55bed496f160 2024-05-09T02:12:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.443+0000 7faa261e1b80 10 client.25160 fetch_fsmap learned FSMap version 32 2024-05-09T02:12:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.443+0000 7faa261e1b80 1 -- 172.21.15.19:0/3233368237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bed4a19b60 con 0x55bed496f160 2024-05-09T02:12:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.443+0000 7fa9f8ff9700 1 -- 172.21.15.19:0/3233368237 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa0c005040 con 0x55bed496f160 2024-05-09T02:12:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.444+0000 7fa9f8ff9700 1 -- 172.21.15.19:0/3233368237 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7faa0c033030 con 0x55bed496f160 2024-05-09T02:12:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.444+0000 7faa261e1b80 10 client.25160 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T02:12:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.444+0000 7faa261e1b80 10 client.25160 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:12:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.444+0000 7faa261e1b80 20 client.25160 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.444+0000 7faa261e1b80 1 --2- 172.21.15.19:0/3233368237 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55bed4a1c030 0x55bed4a1e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.444+0000 7faa261e1b80 4 client.25160 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:12:19.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.444+0000 7faa261e1b80 1 -- 172.21.15.19:0/3233368237 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bed49329d0 con 0x55bed4a1c030 2024-05-09T02:12:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.444+0000 7faa15567700 1 --2- 172.21.15.19:0/3233368237 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55bed4a1c030 0x55bed4a1e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.444+0000 7fa9f8ff9700 10 client.25160 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:12:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.445+0000 7faa15567700 1 --2- 172.21.15.19:0/3233368237 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55bed4a1c030 0x55bed4a1e450 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23ba93c54d8d2de4 server_cookie=b44e717542beb29b in_seq=0 out_seq=0 2024-05-09T02:12:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.446+0000 7fa9f8ff9700 1 -- 172.21.15.19:0/3233368237 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bed49329d0 con 0x55bed4a1c030 2024-05-09T02:12:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.446+0000 7fa9f8ff9700 10 client.25160 handle_command_reply: tid=0 2024-05-09T02:12:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.446+0000 7faa261e1b80 1 client.25160 shutdown 2024-05-09T02:12:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.449+0000 7faa261e1b80 1 -- 172.21.15.19:0/3233368237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed496f160 msgr2=0x55bed4a0c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.449+0000 7faa261e1b80 1 --2- 172.21.15.19:0/3233368237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed496f160 0x55bed4a0c740 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009810 tx=0x7faa0c009f50 comp rx=0 tx=0).stop 2024-05-09T02:12:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.449+0000 7faa261e1b80 1 -- 172.21.15.19:0/3233368237 shutdown_connections 2024-05-09T02:12:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.449+0000 7faa261e1b80 1 --2- 172.21.15.19:0/3233368237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed4a026a0 0x55bed4a0cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.449+0000 7faa261e1b80 1 --2- 172.21.15.19:0/3233368237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed48854a0 0x55bed4a0c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.449+0000 7faa261e1b80 1 --2- 172.21.15.19:0/3233368237 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55bed4a1c030 0x55bed4a1e450 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.449+0000 7faa261e1b80 1 --2- 172.21.15.19:0/3233368237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed496f160 0x55bed4a0c740 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.449+0000 7faa261e1b80 1 -- 172.21.15.19:0/3233368237 >> 172.21.15.19:0/3233368237 conn(0x55bed48e56a0 msgr2=0x55bed49fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:19.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.451+0000 7fa9f8ff9700 0 client.25160 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:12:19.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.451+0000 7faa261e1b80 1 -- 172.21.15.19:0/3233368237 shutdown_connections 2024-05-09T02:12:19.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.451+0000 7faa261e1b80 1 -- 172.21.15.19:0/3233368237 wait complete. 2024-05-09T02:12:19.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.451+0000 7faa261e1b80 20 client.25160 trim_cache size 0 max 16384 2024-05-09T02:12:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.474+0000 7faa261e1b80 1 Processor -- start 2024-05-09T02:12:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.474+0000 7faa261e1b80 1 -- start start 2024-05-09T02:12:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.474+0000 7faa261e1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed48854a0 0x55bed4a07940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.474+0000 7faa261e1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed496f160 0x55bed49fd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.474+0000 7faa15567700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed48854a0 0x55bed4a07940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.474+0000 7faa15567700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed48854a0 0x55bed4a07940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:47268/0 (socket says 172.21.15.19:47268) 2024-05-09T02:12:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.475+0000 7faa15567700 1 -- 172.21.15.19:0/4160788950 learned_addr learned my addr 172.21.15.19:0/4160788950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.475+0000 7faa261e1b80 1 --2- 172.21.15.19:0/4160788950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed4a026a0 0x55bed4a0cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.475+0000 7faa261e1b80 1 -- 172.21.15.19:0/4160788950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa101b53d0 con 0x55bed496f160 2024-05-09T02:12:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.475+0000 7faa261e1b80 1 -- 172.21.15.19:0/4160788950 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa101b6fa0 con 0x55bed4a026a0 2024-05-09T02:12:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.475+0000 7faa261e1b80 1 -- 172.21.15.19:0/4160788950 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa101b5aa0 con 0x55bed48854a0 2024-05-09T02:12:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.475+0000 7faa14d66700 1 --2- 172.21.15.19:0/4160788950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed496f160 0x55bed49fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.475+0000 7faa15567700 1 -- 172.21.15.19:0/4160788950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed4a026a0 msgr2=0x55bed4a0cac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:12:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.475+0000 7faa15567700 1 --2- 172.21.15.19:0/4160788950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed4a026a0 0x55bed4a0cac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.475+0000 7faa15567700 1 -- 172.21.15.19:0/4160788950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed496f160 msgr2=0x55bed49fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.475+0000 7faa15567700 1 --2- 172.21.15.19:0/4160788950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed496f160 0x55bed49fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.475+0000 7faa15567700 1 -- 172.21.15.19:0/4160788950 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bed4a0f000 con 0x55bed48854a0 2024-05-09T02:12:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.476+0000 7faa15567700 1 --2- 172.21.15.19:0/4160788950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed48854a0 0x55bed4a07940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa0000e6e0 tx=0x7faa0000af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceafde4d3d9b0dc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.476+0000 7fa9faffd700 1 -- 172.21.15.19:0/4160788950 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0000eea0 con 0x55bed48854a0 2024-05-09T02:12:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.476+0000 7fa9faffd700 1 -- 172.21.15.19:0/4160788950 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa0002d070 con 0x55bed48854a0 2024-05-09T02:12:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.476+0000 7fa9faffd700 1 -- 172.21.15.19:0/4160788950 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa000109d0 con 0x55bed48854a0 2024-05-09T02:12:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.477+0000 7faa261e1b80 1 -- 172.21.15.19:0/4160788950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed48854a0 msgr2=0x55bed4a07940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.477+0000 7faa261e1b80 1 --2- 172.21.15.19:0/4160788950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed48854a0 0x55bed4a07940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa0000e6e0 tx=0x7faa0000af90 comp rx=0 tx=0).stop 2024-05-09T02:12:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.477+0000 7faa261e1b80 1 -- 172.21.15.19:0/4160788950 shutdown_connections 2024-05-09T02:12:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.477+0000 7faa261e1b80 1 --2- 172.21.15.19:0/4160788950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed4a026a0 0x55bed4a0cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.477+0000 7faa261e1b80 1 --2- 172.21.15.19:0/4160788950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed48854a0 0x55bed4a07940 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.477+0000 7faa261e1b80 1 --2- 172.21.15.19:0/4160788950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed496f160 0x55bed49fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.477+0000 7faa261e1b80 1 -- 172.21.15.19:0/4160788950 >> 172.21.15.19:0/4160788950 conn(0x55bed48e56a0 msgr2=0x55bed4a07230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.478+0000 7faa261e1b80 1 -- 172.21.15.19:0/4160788950 shutdown_connections 2024-05-09T02:12:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.478+0000 7faa261e1b80 1 -- 172.21.15.19:0/4160788950 wait complete. 2024-05-09T02:12:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.478+0000 7faa261e1b80 1 -- start start 2024-05-09T02:12:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.478+0000 7faa261e1b80 1 Processor -- start 2024-05-09T02:12:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa261e1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed48854a0 0x55bed4a1e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa261e1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed496f160 0x55bed4a1e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa261e1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed4a026a0 0x55bed4a1eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa261e1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa101b6840 con 0x55bed4a026a0 2024-05-09T02:12:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa261e1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa101b6170 con 0x55bed48854a0 2024-05-09T02:12:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa261e1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa101b3370 con 0x55bed496f160 2024-05-09T02:12:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa261e1b80 10 client.0 _ll_register_callbacks cb 0x55bed47966c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa15d68700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed4a026a0 0x55bed4a1eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa15567700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed48854a0 0x55bed4a1e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa15d68700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed4a026a0 0x55bed4a1eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60744/0 (socket says 172.21.15.19:60744) 2024-05-09T02:12:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa15d68700 1 -- 172.21.15.19:0/2325768025 learned_addr learned my addr 172.21.15.19:0/2325768025 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa14d66700 1 --2- 172.21.15.19:0/2325768025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed496f160 0x55bed4a1e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa15d68700 1 -- 172.21.15.19:0/2325768025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed496f160 msgr2=0x55bed4a1e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa15d68700 1 --2- 172.21.15.19:0/2325768025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed496f160 0x55bed4a1e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa15d68700 1 -- 172.21.15.19:0/2325768025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed48854a0 msgr2=0x55bed4a1e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa15d68700 1 --2- 172.21.15.19:0/2325768025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed48854a0 0x55bed4a1e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.479+0000 7faa15d68700 1 -- 172.21.15.19:0/2325768025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bed49f6f60 con 0x55bed4a026a0 2024-05-09T02:12:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.481+0000 7faa15d68700 1 --2- 172.21.15.19:0/2325768025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed4a026a0 0x55bed4a1eac0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faa080a9190 tx=0x7faa080b9cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504a4a9cb9a8ffb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.481+0000 7fa9f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:12:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.481+0000 7fa9f8ff9700 1 -- 172.21.15.19:0/2325768025 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa080a8fd0 con 0x55bed4a026a0 2024-05-09T02:12:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.481+0000 7fa9f8ff9700 1 -- 172.21.15.19:0/2325768025 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7faa080be2b0 con 0x55bed4a026a0 2024-05-09T02:12:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.482+0000 7faa261e1b80 1 -- 172.21.15.19:0/2325768025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa0002d070 con 0x55bed4a026a0 2024-05-09T02:12:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.482+0000 7fa9f8ff9700 1 -- 172.21.15.19:0/2325768025 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faa080a8970 con 0x55bed4a026a0 2024-05-09T02:12:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.482+0000 7faa261e1b80 10 client.15483 fetch_fsmap learned FSMap version 32 2024-05-09T02:12:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.482+0000 7faa261e1b80 1 -- 172.21.15.19:0/2325768025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bed4a856d0 con 0x55bed4a026a0 2024-05-09T02:12:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.482+0000 7fa9f8ff9700 1 -- 172.21.15.19:0/2325768025 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa0002d070 con 0x55bed4a026a0 2024-05-09T02:12:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.482+0000 7fa9f8ff9700 1 -- 172.21.15.19:0/2325768025 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7faa080b1bd0 con 0x55bed4a026a0 2024-05-09T02:12:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.482+0000 7faa261e1b80 10 client.15483 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T02:12:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.482+0000 7faa261e1b80 10 client.15483 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:12:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.483+0000 7faa261e1b80 20 client.15483 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.483+0000 7faa261e1b80 1 --2- 172.21.15.19:0/2325768025 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55bed4a85d60 0x55bed4a88180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.483+0000 7faa261e1b80 4 client.15483 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:12:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.483+0000 7faa261e1b80 1 -- 172.21.15.19:0/2325768025 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bed492ca70 con 0x55bed4a85d60 2024-05-09T02:12:19.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.483+0000 7faa15567700 1 --2- 172.21.15.19:0/2325768025 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55bed4a85d60 0x55bed4a88180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:19.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.484+0000 7fa9f8ff9700 10 client.15483 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:12:19.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7faa15567700 1 --2- 172.21.15.19:0/2325768025 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55bed4a85d60 0x55bed4a88180 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36177fa0a91b0253 server_cookie=40d3e2b8aa453bed in_seq=0 out_seq=0 2024-05-09T02:12:19.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7fa9f8ff9700 1 -- 172.21.15.19:0/2325768025 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bed492ca70 con 0x55bed4a85d60 2024-05-09T02:12:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7fa9f8ff9700 10 client.15483 handle_command_reply: tid=0 2024-05-09T02:12:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7faa261e1b80 1 client.15483 shutdown 2024-05-09T02:12:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7faa261e1b80 1 -- 172.21.15.19:0/2325768025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed4a026a0 msgr2=0x55bed4a1eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7faa261e1b80 1 --2- 172.21.15.19:0/2325768025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed4a026a0 0x55bed4a1eac0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faa080a9190 tx=0x7faa080b9cb0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7faa261e1b80 1 -- 172.21.15.19:0/2325768025 shutdown_connections 2024-05-09T02:12:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7faa261e1b80 1 --2- 172.21.15.19:0/2325768025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bed4a026a0 0x55bed4a1eac0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7faa261e1b80 1 --2- 172.21.15.19:0/2325768025 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55bed4a85d60 0x55bed4a88180 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7faa261e1b80 1 --2- 172.21.15.19:0/2325768025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bed48854a0 0x55bed4a1e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7faa261e1b80 1 --2- 172.21.15.19:0/2325768025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bed496f160 0x55bed4a1e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7faa261e1b80 1 -- 172.21.15.19:0/2325768025 >> 172.21.15.19:0/2325768025 conn(0x55bed48e56a0 msgr2=0x55bed4a05e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7fa9f8ff9700 0 client.15483 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:12:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7faa261e1b80 1 -- 172.21.15.19:0/2325768025 shutdown_connections 2024-05-09T02:12:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7faa261e1b80 1 -- 172.21.15.19:0/2325768025 wait complete. 2024-05-09T02:12:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.485+0000 7faa261e1b80 20 client.15483 trim_cache size 0 max 16384 2024-05-09T02:12:19.490 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:12:19.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.486+0000 7faa177cb700 1 -- 172.21.15.19:0/3174336698 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa9fc0b2c10 msgr2=0x7fa9fc0b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:19.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.486+0000 7faa177cb700 1 --2- 172.21.15.19:0/3174336698 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa9fc0b2c10 0x7fa9fc0b5120 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faa00005ea0 tx=0x7faa00005e30 comp rx=0 tx=0).stop 2024-05-09T02:12:19.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.486+0000 7faa177cb700 1 -- 172.21.15.19:0/3174336698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa10108740 msgr2=0x7faa10112e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:19.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.486+0000 7faa177cb700 1 --2- 172.21.15.19:0/3174336698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa10108740 0x7faa10112e00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faa08009cd0 tx=0x7faa0800c620 comp rx=0 tx=0).stop 2024-05-09T02:12:19.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.486+0000 7faa177cb700 1 -- 172.21.15.19:0/3174336698 shutdown_connections 2024-05-09T02:12:19.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.486+0000 7faa177cb700 1 --2- 172.21.15.19:0/3174336698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa10108740 0x7faa10112e00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.486+0000 7faa177cb700 1 --2- 172.21.15.19:0/3174336698 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa9fc0b2c10 0x7fa9fc0b5120 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.486+0000 7faa177cb700 1 --2- 172.21.15.19:0/3174336698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa101034c0 0x7faa10112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.486+0000 7faa177cb700 1 --2- 172.21.15.19:0/3174336698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10107df0 0x7faa101128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:19.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.486+0000 7faa177cb700 1 -- 172.21.15.19:0/3174336698 >> 172.21.15.19:0/3174336698 conn(0x7faa100fedc0 msgr2=0x7faa10105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:19.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.486+0000 7faa177cb700 1 -- 172.21.15.19:0/3174336698 shutdown_connections 2024-05-09T02:12:19.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:19.486+0000 7faa177cb700 1 -- 172.21.15.19:0/3174336698 wait complete. 2024-05-09T02:12:20.290 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T02:12:20.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:20 smithi196 ceph-mon[34718]: pgmap v34: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-05-09T02:12:20.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:20 smithi019 ceph-mon[30151]: pgmap v34: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-05-09T02:12:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:12:20.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:20 smithi059 ceph-mon[34781]: pgmap v34: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-05-09T02:12:22.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:22 smithi196 ceph-mon[34718]: pgmap v35: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 757 op/s 2024-05-09T02:12:22.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:22 smithi019 ceph-mon[30151]: pgmap v35: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 757 op/s 2024-05-09T02:12:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 40.0 s, 3476.4 tps, lat 28.878 ms stddev 30.187 2024-05-09T02:12:22.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:22 smithi059 ceph-mon[34781]: pgmap v35: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 757 op/s 2024-05-09T02:12:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.922+0000 7f04161dc700 1 -- 172.21.15.19:0/4230561419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0410107df0 msgr2=0x7f0410108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.922+0000 7f04161dc700 1 --2- 172.21.15.19:0/4230561419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0410107df0 0x7f0410108200 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f04000099c0 tx=0x7f0400009cd0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.922+0000 7f04161dc700 1 -- 172.21.15.19:0/4230561419 shutdown_connections 2024-05-09T02:12:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.922+0000 7f04161dc700 1 --2- 172.21.15.19:0/4230561419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0410108740 0x7f04101100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.922+0000 7f04161dc700 1 --2- 172.21.15.19:0/4230561419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04101034c0 0x7f04101038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.922+0000 7f04161dc700 1 --2- 172.21.15.19:0/4230561419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0410107df0 0x7f0410108200 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.922+0000 7f04161dc700 1 -- 172.21.15.19:0/4230561419 >> 172.21.15.19:0/4230561419 conn(0x7f04100fedc0 msgr2=0x7f04101011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.922+0000 7f04161dc700 1 -- 172.21.15.19:0/4230561419 shutdown_connections 2024-05-09T02:12:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.922+0000 7f04161dc700 1 -- 172.21.15.19:0/4230561419 wait complete. 2024-05-09T02:12:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.923+0000 7f04161dc700 1 Processor -- start 2024-05-09T02:12:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.923+0000 7f04161dc700 1 -- start start 2024-05-09T02:12:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f04161dc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04101034c0 0x7f041010bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f04161dc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0410107df0 0x7f041010a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f04161dc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0410108740 0x7f041010a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f04161dc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04101133a0 con 0x7f04101034c0 2024-05-09T02:12:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f04161dc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0410113220 con 0x7f0410107df0 2024-05-09T02:12:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f04161dc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0410113520 con 0x7f0410108740 2024-05-09T02:12:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f040f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04101034c0 0x7f041010bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f040f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04101034c0 0x7f041010bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60748/0 (socket says 172.21.15.19:60748) 2024-05-09T02:12:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f040f7fe700 1 -- 172.21.15.19:0/3870959534 learned_addr learned my addr 172.21.15.19:0/3870959534 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f040ffff700 1 --2- 172.21.15.19:0/3870959534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0410108740 0x7f041010a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f040effd700 1 --2- 172.21.15.19:0/3870959534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0410107df0 0x7f041010a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f040ffff700 1 -- 172.21.15.19:0/3870959534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0410107df0 msgr2=0x7f041010a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f040ffff700 1 --2- 172.21.15.19:0/3870959534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0410107df0 0x7f041010a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f040ffff700 1 -- 172.21.15.19:0/3870959534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04101034c0 msgr2=0x7f041010bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f040ffff700 1 --2- 172.21.15.19:0/3870959534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04101034c0 0x7f041010bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f040ffff700 1 -- 172.21.15.19:0/3870959534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041010af50 con 0x7f0410108740 2024-05-09T02:12:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.924+0000 7f040effd700 1 --2- 172.21.15.19:0/3870959534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0410107df0 0x7f041010a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:12:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.925+0000 7f040ffff700 1 --2- 172.21.15.19:0/3870959534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0410108740 0x7f041010a720 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f040400e970 tx=0x7f040400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.925+0000 7f040cff9700 1 -- 172.21.15.19:0/3870959534 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404004220 con 0x7f0410108740 2024-05-09T02:12:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.925+0000 7f040cff9700 1 -- 172.21.15.19:0/3870959534 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04040043c0 con 0x7f0410108740 2024-05-09T02:12:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.925+0000 7f04161dc700 1 -- 172.21.15.19:0/3870959534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04101ad910 con 0x7f0410108740 2024-05-09T02:12:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.925+0000 7f04161dc700 1 -- 172.21.15.19:0/3870959534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04101ade20 con 0x7f0410108740 2024-05-09T02:12:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.926+0000 7f040cff9700 1 -- 172.21.15.19:0/3870959534 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404010700 con 0x7f0410108740 2024-05-09T02:12:23.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.928+0000 7f040cff9700 1 -- 172.21.15.19:0/3870959534 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f04040108a0 con 0x7f0410108740 2024-05-09T02:12:23.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.930+0000 7f0424bf2b80 1 Processor -- start 2024-05-09T02:12:23.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.930+0000 7f0424bf2b80 1 -- start start 2024-05-09T02:12:23.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.931+0000 7f0424bf2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3b7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.931+0000 7f0424bf2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c2344a0 0x55e37c3b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.931+0000 7f040effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.931+0000 7f040effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3b7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60760/0 (socket says 172.21.15.19:60760) 2024-05-09T02:12:23.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.931+0000 7f040effd700 1 -- 172.21.15.19:0/3996332264 learned_addr learned my addr 172.21.15.19:0/3996332264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:23.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.931+0000 7f040f7fe700 1 --2- 172.21.15.19:0/3996332264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c2344a0 0x55e37c3b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.931+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/3996332264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c3b1400 0x55e37c3b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.931+0000 7f040cff9700 1 --2- 172.21.15.19:0/3870959534 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f03f80b2b90 0x7f03f80b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.931+0000 7f0424bf2b80 1 -- 172.21.15.19:0/3996332264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e37c2d40a0 con 0x55e37c2aed00 2024-05-09T02:12:23.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.931+0000 7f0424bf2b80 1 -- 172.21.15.19:0/3996332264 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e37c2799a0 con 0x55e37c3b1400 2024-05-09T02:12:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.931+0000 7f0424bf2b80 1 -- 172.21.15.19:0/3996332264 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e37c2d0060 con 0x55e37c2344a0 2024-05-09T02:12:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.932+0000 7f040ffff700 1 --2- 172.21.15.19:0/3870959534 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f03f80b2b90 0x7f03f80b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.932+0000 7f040effd700 1 --2- 172.21.15.19:0/3996332264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c3b1400 0x55e37c3b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.932+0000 7f040cff9700 1 -- 172.21.15.19:0/3870959534 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f04040e2580 con 0x7f0410108740 2024-05-09T02:12:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.932+0000 7f040effd700 1 -- 172.21.15.19:0/3996332264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c2344a0 msgr2=0x55e37c3b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.932+0000 7f040effd700 1 --2- 172.21.15.19:0/3996332264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c2344a0 0x55e37c3b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.932+0000 7f040effd700 1 -- 172.21.15.19:0/3996332264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c3b1400 msgr2=0x55e37c3b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.932+0000 7f040effd700 1 --2- 172.21.15.19:0/3996332264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c3b1400 0x55e37c3b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.932+0000 7f040effd700 1 -- 172.21.15.19:0/3996332264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e37c26a500 con 0x55e37c2aed00 2024-05-09T02:12:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.932+0000 7f040f7fe700 1 --2- 172.21.15.19:0/3996332264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c2344a0 0x55e37c3b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:12:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.932+0000 7f040effd700 1 --2- 172.21.15.19:0/3996332264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3b7240 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0400003760 tx=0x7f0400005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dba31da7f55eadd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.932+0000 7f040ffff700 1 --2- 172.21.15.19:0/3870959534 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f03f80b2b90 0x7f03f80b50a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f04040dfbb0 tx=0x7f040406fa20 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.932+0000 7f03f5ffb700 1 -- 172.21.15.19:0/3996332264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400021040 con 0x55e37c2aed00 2024-05-09T02:12:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.933+0000 7f03f5ffb700 1 -- 172.21.15.19:0/3996332264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04000037c0 con 0x55e37c2aed00 2024-05-09T02:12:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.933+0000 7f03f5ffb700 1 -- 172.21.15.19:0/3996332264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040000f6c0 con 0x55e37c2aed00 2024-05-09T02:12:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.933+0000 7f0424bf2b80 1 -- 172.21.15.19:0/3996332264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 msgr2=0x55e37c3b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.933+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/3996332264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3b7240 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0400003760 tx=0x7f0400005040 comp rx=0 tx=0).stop 2024-05-09T02:12:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.934+0000 7f0424bf2b80 1 -- 172.21.15.19:0/3996332264 shutdown_connections 2024-05-09T02:12:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.934+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/3996332264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c3b1400 0x55e37c3b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.934+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/3996332264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3b7240 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.934+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/3996332264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c2344a0 0x55e37c3b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.934+0000 7f0424bf2b80 1 -- 172.21.15.19:0/3996332264 >> 172.21.15.19:0/3996332264 conn(0x55e37c237f70 msgr2=0x55e37c3ac220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.934+0000 7f0424bf2b80 1 -- 172.21.15.19:0/3996332264 shutdown_connections 2024-05-09T02:12:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.934+0000 7f0424bf2b80 1 -- 172.21.15.19:0/3996332264 wait complete. 2024-05-09T02:12:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.935+0000 7f0424bf2b80 1 -- start start 2024-05-09T02:12:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.935+0000 7f0424bf2b80 1 Processor -- start 2024-05-09T02:12:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.936+0000 7f0424bf2b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c2344a0 0x55e37c3bafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.936+0000 7f0424bf2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3bb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.936+0000 7f0424bf2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c3b1400 0x55e37c3bba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.936+0000 7f0424bf2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e37c3b2100 con 0x55e37c2aed00 2024-05-09T02:12:23.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.936+0000 7f0424bf2b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e37c26eb20 con 0x55e37c2344a0 2024-05-09T02:12:23.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.936+0000 7f0424bf2b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e37c3c7260 con 0x55e37c3b1400 2024-05-09T02:12:23.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.936+0000 7f0424bf2b80 10 client.0 _ll_register_callbacks cb 0x55e37c1456c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:23.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.937+0000 7f040f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3bb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.937+0000 7f040f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3bb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60776/0 (socket says 172.21.15.19:60776) 2024-05-09T02:12:23.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.937+0000 7f040f7fe700 1 -- 172.21.15.19:0/1918312804 learned_addr learned my addr 172.21.15.19:0/1918312804 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:23.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.937+0000 7f040effd700 1 --2- 172.21.15.19:0/1918312804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c2344a0 0x55e37c3bafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.937+0000 7f040effd700 1 --2- 172.21.15.19:0/1918312804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c3b1400 0x55e37c3bba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.937+0000 7f040effd700 1 -- 172.21.15.19:0/1918312804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c3b1400 msgr2=0x55e37c3bba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:23.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.937+0000 7f040effd700 1 --2- 172.21.15.19:0/1918312804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c3b1400 0x55e37c3bba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.937+0000 7f040effd700 1 -- 172.21.15.19:0/1918312804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 msgr2=0x55e37c3bb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.937+0000 7f040effd700 1 --2- 172.21.15.19:0/1918312804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3bb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.937+0000 7f040effd700 1 -- 172.21.15.19:0/1918312804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e37c14f490 con 0x55e37c2344a0 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.938+0000 7f040effd700 1 --2- 172.21.15.19:0/1918312804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c2344a0 0x55e37c3bafc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f040000b910 tx=0x7f0400005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=964cd2ca4214f3c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.938+0000 7f03eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.938+0000 7f03eeffd700 1 -- 172.21.15.19:0/1918312804 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400003d80 con 0x55e37c2344a0 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.938+0000 7f03eeffd700 1 -- 172.21.15.19:0/1918312804 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f040000f8a0 con 0x55e37c2344a0 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.938+0000 7f0424bf2b80 1 -- 172.21.15.19:0/1918312804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04000037c0 con 0x55e37c2344a0 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.938+0000 7f03eeffd700 1 -- 172.21.15.19:0/1918312804 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f040000faa0 con 0x55e37c2344a0 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.939+0000 7f0424bf2b80 10 client.25175 fetch_fsmap learned FSMap version 32 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.939+0000 7f0424bf2b80 1 -- 172.21.15.19:0/1918312804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e37c3c8b30 con 0x55e37c2344a0 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.939+0000 7f03eeffd700 1 -- 172.21.15.19:0/1918312804 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0400005560 con 0x55e37c2344a0 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.939+0000 7f03eeffd700 1 -- 172.21.15.19:0/1918312804 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f0400027030 con 0x55e37c2344a0 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.940+0000 7f0424bf2b80 10 client.25175 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.940+0000 7f0424bf2b80 10 client.25175 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:12:23.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.940+0000 7f0424bf2b80 20 client.25175 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:23.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.940+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/1918312804 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e37c3cb000 0x55e37c3cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.940+0000 7f0424bf2b80 4 client.25175 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:12:23.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.940+0000 7f0424bf2b80 1 -- 172.21.15.19:0/1918312804 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f04000037c0 con 0x55e37c3cb000 2024-05-09T02:12:23.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.940+0000 7f040f7fe700 1 --2- 172.21.15.19:0/1918312804 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e37c3cb000 0x55e37c3cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.940+0000 7f03eeffd700 10 client.25175 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:12:23.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.940+0000 7f040f7fe700 1 --2- 172.21.15.19:0/1918312804 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e37c3cb000 0x55e37c3cd420 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e485709fd737e925 server_cookie=5c69dfed6432a88b in_seq=0 out_seq=0 2024-05-09T02:12:23.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.941+0000 7f03eeffd700 1 -- 172.21.15.19:0/1918312804 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f04000037c0 con 0x55e37c3cb000 2024-05-09T02:12:23.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.941+0000 7f03eeffd700 10 client.25175 handle_command_reply: tid=0 2024-05-09T02:12:23.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.941+0000 7f0424bf2b80 1 client.25175 shutdown 2024-05-09T02:12:23.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.942+0000 7f0424bf2b80 1 -- 172.21.15.19:0/1918312804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c2344a0 msgr2=0x55e37c3bafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:23.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.942+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/1918312804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c2344a0 0x55e37c3bafc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f040000b910 tx=0x7f0400005e80 comp rx=0 tx=0).stop 2024-05-09T02:12:23.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.942+0000 7f0424bf2b80 1 -- 172.21.15.19:0/1918312804 shutdown_connections 2024-05-09T02:12:23.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.942+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/1918312804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c3b1400 0x55e37c3bba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.942+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/1918312804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c2344a0 0x55e37c3bafc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.942+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/1918312804 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e37c3cb000 0x55e37c3cd420 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.942+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/1918312804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3bb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.942+0000 7f0424bf2b80 1 -- 172.21.15.19:0/1918312804 >> 172.21.15.19:0/1918312804 conn(0x55e37c237f70 msgr2=0x55e37c3a5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:23.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.942+0000 7f03eeffd700 0 client.25175 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:12:23.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.943+0000 7f0424bf2b80 1 -- 172.21.15.19:0/1918312804 shutdown_connections 2024-05-09T02:12:23.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.943+0000 7f0424bf2b80 1 -- 172.21.15.19:0/1918312804 wait complete. 2024-05-09T02:12:23.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.943+0000 7f0424bf2b80 20 client.25175 trim_cache size 0 max 16384 2024-05-09T02:12:23.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.966+0000 7f0424bf2b80 1 Processor -- start 2024-05-09T02:12:23.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.966+0000 7f0424bf2b80 1 -- start start 2024-05-09T02:12:23.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.966+0000 7f0424bf2b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c2344a0 0x55e37c3b9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.966+0000 7f0424bf2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3a6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.966+0000 7f0424bf2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c3b1400 0x55e37c3d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.966+0000 7f0424bf2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc00d600 con 0x55e37c2aed00 2024-05-09T02:12:23.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.966+0000 7f0424bf2b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc00f1d0 con 0x55e37c2344a0 2024-05-09T02:12:23.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.966+0000 7f0424bf2b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc00dcd0 con 0x55e37c3b1400 2024-05-09T02:12:23.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.966+0000 7f040f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.967+0000 7f040effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c2344a0 0x55e37c3b9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.967+0000 7f040f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3a6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60782/0 (socket says 172.21.15.19:60782) 2024-05-09T02:12:23.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.967+0000 7f040f7fe700 1 -- 172.21.15.19:0/151315269 learned_addr learned my addr 172.21.15.19:0/151315269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:23.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.967+0000 7f040effd700 1 --2- 172.21.15.19:0/151315269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c3b1400 0x55e37c3d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.967+0000 7f040effd700 1 -- 172.21.15.19:0/151315269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c3b1400 msgr2=0x55e37c3d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:23.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.967+0000 7f040effd700 1 --2- 172.21.15.19:0/151315269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c3b1400 0x55e37c3d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.967+0000 7f040effd700 1 -- 172.21.15.19:0/151315269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 msgr2=0x55e37c3a6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:23.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.967+0000 7f040effd700 1 --2- 172.21.15.19:0/151315269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3a6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.967+0000 7f040effd700 1 -- 172.21.15.19:0/151315269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e37c3a89d0 con 0x55e37c2344a0 2024-05-09T02:12:23.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.967+0000 7f040effd700 1 --2- 172.21.15.19:0/151315269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c2344a0 0x55e37c3b9c20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0400001eb0 tx=0x7f0400005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=201f6b645d09db27 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:23.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.967+0000 7f03f57fa700 1 -- 172.21.15.19:0/151315269 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400044040 con 0x55e37c2344a0 2024-05-09T02:12:23.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.967+0000 7f03f57fa700 1 -- 172.21.15.19:0/151315269 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0400045040 con 0x55e37c2344a0 2024-05-09T02:12:23.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.967+0000 7f03f57fa700 1 -- 172.21.15.19:0/151315269 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400005310 con 0x55e37c2344a0 2024-05-09T02:12:23.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.968+0000 7f0424bf2b80 1 -- 172.21.15.19:0/151315269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c2344a0 msgr2=0x55e37c3b9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:23.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.968+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/151315269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c2344a0 0x55e37c3b9c20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0400001eb0 tx=0x7f0400005040 comp rx=0 tx=0).stop 2024-05-09T02:12:23.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.968+0000 7f0424bf2b80 1 -- 172.21.15.19:0/151315269 shutdown_connections 2024-05-09T02:12:23.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.968+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/151315269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c3b1400 0x55e37c3d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.968+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/151315269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c2344a0 0x55e37c3b9c20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.968+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/151315269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.968+0000 7f0424bf2b80 1 -- 172.21.15.19:0/151315269 >> 172.21.15.19:0/151315269 conn(0x55e37c237f70 msgr2=0x55e37c3a5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:23.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.968+0000 7f0424bf2b80 1 -- 172.21.15.19:0/151315269 shutdown_connections 2024-05-09T02:12:23.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.968+0000 7f0424bf2b80 1 -- 172.21.15.19:0/151315269 wait complete. 2024-05-09T02:12:23.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.969+0000 7f0424bf2b80 1 -- start start 2024-05-09T02:12:23.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.969+0000 7f0424bf2b80 1 Processor -- start 2024-05-09T02:12:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.970+0000 7f0424bf2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c2344a0 0x55e37c3ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.970+0000 7f0424bf2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.970+0000 7f0424bf2b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c3b1400 0x55e37c433540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.970+0000 7f040f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.970+0000 7f040f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60798/0 (socket says 172.21.15.19:60798) 2024-05-09T02:12:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.970+0000 7f040f7fe700 1 -- 172.21.15.19:0/2211306659 learned_addr learned my addr 172.21.15.19:0/2211306659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.971+0000 7f0424bf2b80 1 -- 172.21.15.19:0/2211306659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc00ea70 con 0x55e37c2aed00 2024-05-09T02:12:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.971+0000 7f040effd700 1 --2- 172.21.15.19:0/2211306659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c2344a0 0x55e37c3ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.971+0000 7f0424bf2b80 1 -- 172.21.15.19:0/2211306659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc00e3a0 con 0x55e37c3b1400 2024-05-09T02:12:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.971+0000 7f0424bf2b80 1 -- 172.21.15.19:0/2211306659 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc00b5a0 con 0x55e37c2344a0 2024-05-09T02:12:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.971+0000 7f0424bf2b80 10 client.0 _ll_register_callbacks cb 0x55e37c1456c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.971+0000 7f040effd700 1 --2- 172.21.15.19:0/2211306659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c3b1400 0x55e37c433540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.971+0000 7f040effd700 1 -- 172.21.15.19:0/2211306659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c3b1400 msgr2=0x55e37c433540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.971+0000 7f040effd700 1 --2- 172.21.15.19:0/2211306659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c3b1400 0x55e37c433540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.971+0000 7f040effd700 1 -- 172.21.15.19:0/2211306659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 msgr2=0x55e37c3cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.971+0000 7f040effd700 1 --2- 172.21.15.19:0/2211306659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.971+0000 7f040effd700 1 -- 172.21.15.19:0/2211306659 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e37c3cdd30 con 0x55e37c2344a0 2024-05-09T02:12:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.972+0000 7f040effd700 1 --2- 172.21.15.19:0/2211306659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c2344a0 0x55e37c3ccf60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f040000be50 tx=0x7f0400003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=166b6ec5f4bdafe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.972+0000 7f03eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:12:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.972+0000 7f03eeffd700 1 -- 172.21.15.19:0/2211306659 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400044070 con 0x55e37c2344a0 2024-05-09T02:12:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.972+0000 7f03eeffd700 1 -- 172.21.15.19:0/2211306659 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f04000050b0 con 0x55e37c2344a0 2024-05-09T02:12:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.972+0000 7f0424bf2b80 1 -- 172.21.15.19:0/2211306659 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0400045040 con 0x55e37c2344a0 2024-05-09T02:12:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.972+0000 7f03eeffd700 1 -- 172.21.15.19:0/2211306659 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f04000037c0 con 0x55e37c2344a0 2024-05-09T02:12:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.972+0000 7f0424bf2b80 10 client.25222 fetch_fsmap learned FSMap version 32 2024-05-09T02:12:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.972+0000 7f0424bf2b80 1 -- 172.21.15.19:0/2211306659 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e37c434a50 con 0x55e37c2344a0 2024-05-09T02:12:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.973+0000 7f03eeffd700 1 -- 172.21.15.19:0/2211306659 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0400005760 con 0x55e37c2344a0 2024-05-09T02:12:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.973+0000 7f03eeffd700 1 -- 172.21.15.19:0/2211306659 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f040004a070 con 0x55e37c2344a0 2024-05-09T02:12:23.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.973+0000 7f0424bf2b80 10 client.25222 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T02:12:23.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.973+0000 7f0424bf2b80 10 client.25222 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:12:23.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.973+0000 7f0424bf2b80 20 client.25222 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:23.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.973+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/2211306659 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e37c435020 0x55e37c437440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:23.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.973+0000 7f0424bf2b80 4 client.25222 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:12:23.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.973+0000 7f0424bf2b80 1 -- 172.21.15.19:0/2211306659 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0400045040 con 0x55e37c435020 2024-05-09T02:12:23.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.973+0000 7f040f7fe700 1 --2- 172.21.15.19:0/2211306659 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e37c435020 0x55e37c437440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:23.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.974+0000 7f040f7fe700 1 --2- 172.21.15.19:0/2211306659 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e37c435020 0x55e37c437440 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ee4674daee41e2 server_cookie=e23c4019ba3b4e11 in_seq=0 out_seq=0 2024-05-09T02:12:23.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.974+0000 7f03eeffd700 10 client.25222 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:12:23.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.974+0000 7f03eeffd700 1 -- 172.21.15.19:0/2211306659 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0400045040 con 0x55e37c435020 2024-05-09T02:12:23.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.974+0000 7f03eeffd700 10 client.25222 handle_command_reply: tid=0 2024-05-09T02:12:23.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.974+0000 7f0424bf2b80 1 client.25222 shutdown 2024-05-09T02:12:23.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.975+0000 7f0424bf2b80 1 -- 172.21.15.19:0/2211306659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c2344a0 msgr2=0x55e37c3ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:23.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.975+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/2211306659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c2344a0 0x55e37c3ccf60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f040000be50 tx=0x7f0400003b80 comp rx=0 tx=0).stop 2024-05-09T02:12:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.976+0000 7f0424bf2b80 1 -- 172.21.15.19:0/2211306659 shutdown_connections 2024-05-09T02:12:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.976+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/2211306659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e37c3b1400 0x55e37c433540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.976+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/2211306659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e37c2344a0 0x55e37c3ccf60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.976+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/2211306659 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e37c435020 0x55e37c437440 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.976+0000 7f0424bf2b80 1 --2- 172.21.15.19:0/2211306659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e37c2aed00 0x55e37c3cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.976+0000 7f0424bf2b80 1 -- 172.21.15.19:0/2211306659 >> 172.21.15.19:0/2211306659 conn(0x55e37c237f70 msgr2=0x55e37c3b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.976+0000 7f03eeffd700 0 client.25222 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:12:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.976+0000 7f0424bf2b80 1 -- 172.21.15.19:0/2211306659 shutdown_connections 2024-05-09T02:12:23.896 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:12:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.976+0000 7f0424bf2b80 1 -- 172.21.15.19:0/2211306659 wait complete. 2024-05-09T02:12:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.976+0000 7f0424bf2b80 20 client.25222 trim_cache size 0 max 16384 2024-05-09T02:12:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.977+0000 7f04161dc700 1 -- 172.21.15.19:0/3870959534 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f03f80b2b90 msgr2=0x7f03f80b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.977+0000 7f04161dc700 1 --2- 172.21.15.19:0/3870959534 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f03f80b2b90 0x7f03f80b50a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f04040dfbb0 tx=0x7f040406fa20 comp rx=0 tx=0).stop 2024-05-09T02:12:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.977+0000 7f04161dc700 1 -- 172.21.15.19:0/3870959534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0410108740 msgr2=0x7f041010a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.977+0000 7f04161dc700 1 --2- 172.21.15.19:0/3870959534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0410108740 0x7f041010a720 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f040400e970 tx=0x7f040400ec80 comp rx=0 tx=0).stop 2024-05-09T02:12:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.977+0000 7f04161dc700 1 -- 172.21.15.19:0/3870959534 shutdown_connections 2024-05-09T02:12:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.977+0000 7f04161dc700 1 --2- 172.21.15.19:0/3870959534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0410108740 0x7f041010a720 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.977+0000 7f04161dc700 1 --2- 172.21.15.19:0/3870959534 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f03f80b2b90 0x7f03f80b50a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.977+0000 7f04161dc700 1 --2- 172.21.15.19:0/3870959534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04101034c0 0x7f041010bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.977+0000 7f04161dc700 1 --2- 172.21.15.19:0/3870959534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0410107df0 0x7f041010a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.977+0000 7f04161dc700 1 -- 172.21.15.19:0/3870959534 >> 172.21.15.19:0/3870959534 conn(0x7f04100fedc0 msgr2=0x7f0410106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.977+0000 7f04161dc700 1 -- 172.21.15.19:0/3870959534 shutdown_connections 2024-05-09T02:12:23.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:22.978+0000 7f04161dc700 1 -- 172.21.15.19:0/3870959534 wait complete. 2024-05-09T02:12:24.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:24 smithi019 ceph-mon[30151]: pgmap v36: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 882 op/s 2024-05-09T02:12:24.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:24 smithi196 ceph-mon[34718]: pgmap v36: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 882 op/s 2024-05-09T02:12:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:24 smithi059 ceph-mon[34781]: pgmap v36: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 882 op/s 2024-05-09T02:12:25.501 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T02:12:25.882 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:12:26.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:26 smithi019 ceph-mon[30151]: pgmap v37: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 946 op/s 2024-05-09T02:12:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:26 smithi196 ceph-mon[34718]: pgmap v37: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 946 op/s 2024-05-09T02:12:26.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:26 smithi059 ceph-mon[34781]: pgmap v37: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 946 op/s 2024-05-09T02:12:27.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 45.0 s, 3284.4 tps, lat 30.417 ms stddev 30.593 2024-05-09T02:12:28.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.036+0000 7f2629e7e700 1 -- 172.21.15.19:0/1132901683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f262410c5b0 msgr2=0x7f262410c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:28.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.036+0000 7f2629e7e700 1 --2- 172.21.15.19:0/1132901683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f262410c5b0 0x7f262410c9c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2614009990 tx=0x7f2614009ca0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.036+0000 7f2629e7e700 1 -- 172.21.15.19:0/1132901683 shutdown_connections 2024-05-09T02:12:28.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.036+0000 7f2629e7e700 1 --2- 172.21.15.19:0/1132901683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2624106ec0 0x7f2624113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.036+0000 7f2629e7e700 1 --2- 172.21.15.19:0/1132901683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f262410c5b0 0x7f262410c9c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.036+0000 7f2629e7e700 1 --2- 172.21.15.19:0/1132901683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2624106570 0x7f2624106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.036+0000 7f2629e7e700 1 -- 172.21.15.19:0/1132901683 >> 172.21.15.19:0/1132901683 conn(0x7f26241020d0 msgr2=0x7f2624104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:28.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.037+0000 7f2629e7e700 1 -- 172.21.15.19:0/1132901683 shutdown_connections 2024-05-09T02:12:28.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.037+0000 7f2629e7e700 1 -- 172.21.15.19:0/1132901683 wait complete. 2024-05-09T02:12:28.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.038+0000 7f2629e7e700 1 Processor -- start 2024-05-09T02:12:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.038+0000 7f2629e7e700 1 -- start start 2024-05-09T02:12:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.038+0000 7f2629e7e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2624106570 0x7f262419d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.038+0000 7f2629e7e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2624106ec0 0x7f262419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.038+0000 7f2629e7e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f262410c5b0 0x7f26241a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.039+0000 7f2629e7e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2624116760 con 0x7f2624106570 2024-05-09T02:12:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.039+0000 7f2629e7e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f26241165e0 con 0x7f2624106ec0 2024-05-09T02:12:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.039+0000 7f2629e7e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f26241168e0 con 0x7f262410c5b0 2024-05-09T02:12:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.039+0000 7f2623fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f262410c5b0 0x7f26241a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.039+0000 7f2623fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f262410c5b0 0x7f26241a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:54290/0 (socket says 172.21.15.19:54290) 2024-05-09T02:12:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.039+0000 7f2623fff700 1 -- 172.21.15.19:0/1331516938 learned_addr learned my addr 172.21.15.19:0/1331516938 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.039+0000 7f2623fff700 1 -- 172.21.15.19:0/1331516938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2624106ec0 msgr2=0x7f262419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:28.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.039+0000 7f2623fff700 1 --2- 172.21.15.19:0/1331516938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2624106ec0 0x7f262419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.039+0000 7f2623fff700 1 -- 172.21.15.19:0/1331516938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2624106570 msgr2=0x7f262419d090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:12:28.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.039+0000 7f2623fff700 1 --2- 172.21.15.19:0/1331516938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2624106570 0x7f262419d090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.039+0000 7f2623fff700 1 -- 172.21.15.19:0/1331516938 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26241a1f60 con 0x7f262410c5b0 2024-05-09T02:12:28.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.039+0000 7f2623fff700 1 --2- 172.21.15.19:0/1331516938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f262410c5b0 0x7f26241a18c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f261c00c370 tx=0x7f261c00c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:28.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.039+0000 7f2620ff9700 1 -- 172.21.15.19:0/1331516938 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c0090d0 con 0x7f262410c5b0 2024-05-09T02:12:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.040+0000 7f2620ff9700 1 -- 172.21.15.19:0/1331516938 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f261c009270 con 0x7f262410c5b0 2024-05-09T02:12:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.040+0000 7f2620ff9700 1 -- 172.21.15.19:0/1331516938 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c0047d0 con 0x7f262410c5b0 2024-05-09T02:12:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.040+0000 7f2629e7e700 1 -- 172.21.15.19:0/1331516938 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26241a21f0 con 0x7f262410c5b0 2024-05-09T02:12:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.040+0000 7f2629e7e700 1 -- 172.21.15.19:0/1331516938 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26241a26e0 con 0x7f262410c5b0 2024-05-09T02:12:28.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.045+0000 7f2620ff9700 1 -- 172.21.15.19:0/1331516938 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f261c01a020 con 0x7f262410c5b0 2024-05-09T02:12:28.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.047+0000 7f2638894b80 1 Processor -- start 2024-05-09T02:12:28.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.047+0000 7f2638894b80 1 -- start start 2024-05-09T02:12:28.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.048+0000 7f2620ff9700 1 --2- 172.21.15.19:0/1331516938 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f260c0b2bd0 0x7f260c0b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.048+0000 7f2638894b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c24063d00 0x557c2416c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.048+0000 7f2638894b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c23fe94a0 0x557c2416c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.048+0000 7f2638894b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24166400 0x557c24166810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.048+0000 7f2638894b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557c240890a0 con 0x557c24166400 2024-05-09T02:12:28.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.048+0000 7f2638894b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557c2402e9a0 con 0x557c23fe94a0 2024-05-09T02:12:28.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.048+0000 7f2638894b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557c24085060 con 0x557c24063d00 2024-05-09T02:12:28.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.048+0000 7f2620ff9700 1 -- 172.21.15.19:0/1331516938 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f261c0e21a0 con 0x7f262410c5b0 2024-05-09T02:12:28.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.048+0000 7f2622ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c23fe94a0 0x557c2416c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:28.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.048+0000 7f2622ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c23fe94a0 0x557c2416c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:35430/0 (socket says 172.21.15.19:35430) 2024-05-09T02:12:28.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.048+0000 7f2622ffd700 1 -- 172.21.15.19:0/2995936769 learned_addr learned my addr 172.21.15.19:0/2995936769 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:28.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.048+0000 7f2622ffd700 1 --2- 172.21.15.19:0/1331516938 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f260c0b2bd0 0x7f260c0b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:28.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.049+0000 7f2622ffd700 1 -- 172.21.15.19:0/2995936769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c24063d00 msgr2=0x557c2416c240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:12:28.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.049+0000 7f2622ffd700 1 --2- 172.21.15.19:0/2995936769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c24063d00 0x557c2416c240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.049+0000 7f2622ffd700 1 -- 172.21.15.19:0/2995936769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24166400 msgr2=0x557c24166810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:28.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.049+0000 7f2622ffd700 1 --2- 172.21.15.19:0/2995936769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24166400 0x557c24166810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.049+0000 7f2622ffd700 1 -- 172.21.15.19:0/2995936769 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c2401f500 con 0x557c23fe94a0 2024-05-09T02:12:28.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.050+0000 7f2622ffd700 1 --2- 172.21.15.19:0/2995936769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c23fe94a0 0x557c2416c780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2618013b30 tx=0x7f2618013e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a5919fd960f8ea3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:28.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.050+0000 7f2622ffd700 1 --2- 172.21.15.19:0/1331516938 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f260c0b2bd0 0x7f260c0b50e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f26180106b0 tx=0x7f261800b000 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:28.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.050+0000 7f2609ffb700 1 -- 172.21.15.19:0/2995936769 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618021070 con 0x557c23fe94a0 2024-05-09T02:12:28.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.050+0000 7f2609ffb700 1 -- 172.21.15.19:0/2995936769 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2618009960 con 0x557c23fe94a0 2024-05-09T02:12:28.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.051+0000 7f2609ffb700 1 -- 172.21.15.19:0/2995936769 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618005020 con 0x557c23fe94a0 2024-05-09T02:12:28.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.052+0000 7f2638894b80 1 -- 172.21.15.19:0/2995936769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c23fe94a0 msgr2=0x557c2416c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:28.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.052+0000 7f2638894b80 1 --2- 172.21.15.19:0/2995936769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c23fe94a0 0x557c2416c780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2618013b30 tx=0x7f2618013e40 comp rx=0 tx=0).stop 2024-05-09T02:12:28.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.052+0000 7f2638894b80 1 -- 172.21.15.19:0/2995936769 shutdown_connections 2024-05-09T02:12:28.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.052+0000 7f2638894b80 1 --2- 172.21.15.19:0/2995936769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24166400 0x557c24166810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.052+0000 7f2638894b80 1 --2- 172.21.15.19:0/2995936769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c24063d00 0x557c2416c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.052+0000 7f2638894b80 1 --2- 172.21.15.19:0/2995936769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c23fe94a0 0x557c2416c780 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.052+0000 7f2638894b80 1 -- 172.21.15.19:0/2995936769 >> 172.21.15.19:0/2995936769 conn(0x557c23fecf70 msgr2=0x557c24161220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:28.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.053+0000 7f2638894b80 1 -- 172.21.15.19:0/2995936769 shutdown_connections 2024-05-09T02:12:28.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.053+0000 7f2638894b80 1 -- 172.21.15.19:0/2995936769 wait complete. 2024-05-09T02:12:28.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.053+0000 7f2638894b80 1 -- start start 2024-05-09T02:12:28.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.053+0000 7f2638894b80 1 Processor -- start 2024-05-09T02:12:28.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.054+0000 7f2638894b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c24170060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.054+0000 7f2638894b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24063d00 0x557c241705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.054+0000 7f2638894b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24166400 0x557c24170ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.054+0000 7f2638894b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557c24167100 con 0x557c24063d00 2024-05-09T02:12:28.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.054+0000 7f2638894b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557c24023b20 con 0x557c24166400 2024-05-09T02:12:28.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.054+0000 7f2638894b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557c2417c260 con 0x557c23fe94a0 2024-05-09T02:12:28.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.054+0000 7f2638894b80 10 client.0 _ll_register_callbacks cb 0x557c23efa6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:28.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.054+0000 7f26237fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c24170060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:28.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.054+0000 7f26237fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c24170060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:54314/0 (socket says 172.21.15.19:54314) 2024-05-09T02:12:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.054+0000 7f26237fe700 1 -- 172.21.15.19:0/4059091161 learned_addr learned my addr 172.21.15.19:0/4059091161 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.054+0000 7f2622ffd700 1 --2- 172.21.15.19:0/4059091161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24063d00 0x557c241705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.054+0000 7f26237fe700 1 -- 172.21.15.19:0/4059091161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24166400 msgr2=0x557c24170ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:12:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.054+0000 7f26237fe700 1 --2- 172.21.15.19:0/4059091161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24166400 0x557c24170ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.055+0000 7f26237fe700 1 -- 172.21.15.19:0/4059091161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24063d00 msgr2=0x557c241705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.055+0000 7f26237fe700 1 --2- 172.21.15.19:0/4059091161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24063d00 0x557c241705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.055+0000 7f26237fe700 1 -- 172.21.15.19:0/4059091161 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c23f04490 con 0x557c23fe94a0 2024-05-09T02:12:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.055+0000 7f26237fe700 1 --2- 172.21.15.19:0/4059091161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c24170060 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2614000c00 tx=0x7f26140056a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=286ed058e4a67180 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.055+0000 7f2602ffd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:12:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.055+0000 7f2602ffd700 1 -- 172.21.15.19:0/4059091161 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261400bda0 con 0x557c23fe94a0 2024-05-09T02:12:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.055+0000 7f2638894b80 1 -- 172.21.15.19:0/4059091161 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557c24096a80 con 0x557c23fe94a0 2024-05-09T02:12:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.055+0000 7f2602ffd700 1 -- 172.21.15.19:0/4059091161 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f2614003680 con 0x557c23fe94a0 2024-05-09T02:12:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.056+0000 7f2602ffd700 1 -- 172.21.15.19:0/4059091161 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2614003880 con 0x557c23fe94a0 2024-05-09T02:12:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.056+0000 7f2638894b80 10 client.25231 fetch_fsmap learned FSMap version 32 2024-05-09T02:12:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.056+0000 7f2638894b80 1 -- 172.21.15.19:0/4059091161 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557c2417dc20 con 0x557c23fe94a0 2024-05-09T02:12:28.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.056+0000 7f2602ffd700 1 -- 172.21.15.19:0/4059091161 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557c24096a80 con 0x557c23fe94a0 2024-05-09T02:12:28.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.056+0000 7f2602ffd700 1 -- 172.21.15.19:0/4059091161 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f261402ab00 con 0x557c23fe94a0 2024-05-09T02:12:28.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.056+0000 7f2638894b80 10 client.25231 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T02:12:28.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.056+0000 7f2638894b80 10 client.25231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:12:28.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.056+0000 7f2638894b80 20 client.25231 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:28.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.056+0000 7f2638894b80 1 --2- 172.21.15.19:0/4059091161 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557c24180280 0x557c24182710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.056+0000 7f2638894b80 4 client.25231 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:12:28.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.056+0000 7f2638894b80 1 -- 172.21.15.19:0/4059091161 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c24090b20 con 0x557c24180280 2024-05-09T02:12:28.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.057+0000 7f2622ffd700 1 --2- 172.21.15.19:0/4059091161 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557c24180280 0x557c24182710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:28.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.057+0000 7f2602ffd700 10 client.25231 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:28.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.057+0000 7f2622ffd700 1 --2- 172.21.15.19:0/4059091161 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557c24180280 0x557c24182710 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4045b86841bb258 server_cookie=d6cd3e4a1e374c36 in_seq=0 out_seq=0 2024-05-09T02:12:28.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.058+0000 7f2602ffd700 1 -- 172.21.15.19:0/4059091161 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557c24090b20 con 0x557c24180280 2024-05-09T02:12:28.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.058+0000 7f2602ffd700 10 client.25231 handle_command_reply: tid=0 2024-05-09T02:12:28.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.058+0000 7f2638894b80 1 client.25231 shutdown 2024-05-09T02:12:28.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.058+0000 7f2638894b80 1 -- 172.21.15.19:0/4059091161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 msgr2=0x557c24170060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:28.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.058+0000 7f2638894b80 1 --2- 172.21.15.19:0/4059091161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c24170060 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2614000c00 tx=0x7f26140056a0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.058+0000 7f2638894b80 1 -- 172.21.15.19:0/4059091161 shutdown_connections 2024-05-09T02:12:28.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.058+0000 7f2638894b80 1 --2- 172.21.15.19:0/4059091161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24166400 0x557c24170ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.058+0000 7f2638894b80 1 --2- 172.21.15.19:0/4059091161 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557c24180280 0x557c24182710 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.058+0000 7f2638894b80 1 --2- 172.21.15.19:0/4059091161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c24170060 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.058+0000 7f2638894b80 1 --2- 172.21.15.19:0/4059091161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24063d00 0x557c241705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.058+0000 7f2638894b80 1 -- 172.21.15.19:0/4059091161 >> 172.21.15.19:0/4059091161 conn(0x557c23fecf70 msgr2=0x557c2415aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:28.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.058+0000 7f2602ffd700 0 client.25231 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:28.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.059+0000 7f2638894b80 1 -- 172.21.15.19:0/4059091161 shutdown_connections 2024-05-09T02:12:28.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.059+0000 7f2638894b80 1 -- 172.21.15.19:0/4059091161 wait complete. 2024-05-09T02:12:28.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.059+0000 7f2638894b80 20 client.25231 trim_cache size 0 max 16384 2024-05-09T02:12:28.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.085+0000 7f2638894b80 1 Processor -- start 2024-05-09T02:12:28.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.085+0000 7f2638894b80 1 -- start start 2024-05-09T02:12:28.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.086+0000 7f2638894b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c2416b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.086+0000 7f2638894b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24063d00 0x557c24182900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.086+0000 7f2638894b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24166400 0x557c24170920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.086+0000 7f26237fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c2416b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:28.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.086+0000 7f26237fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c2416b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:54326/0 (socket says 172.21.15.19:54326) 2024-05-09T02:12:28.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.086+0000 7f2622ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24063d00 0x557c24182900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:28.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.086+0000 7f26237fe700 1 -- 172.21.15.19:0/3547262636 learned_addr learned my addr 172.21.15.19:0/3547262636 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:28.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.086+0000 7f2638894b80 1 -- 172.21.15.19:0/3547262636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f261000d610 con 0x557c24166400 2024-05-09T02:12:28.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.086+0000 7f2638894b80 1 -- 172.21.15.19:0/3547262636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f261000f1e0 con 0x557c24063d00 2024-05-09T02:12:28.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.086+0000 7f2638894b80 1 -- 172.21.15.19:0/3547262636 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f261000dce0 con 0x557c23fe94a0 2024-05-09T02:12:28.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.086+0000 7f2623fff700 1 --2- 172.21.15.19:0/3547262636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24166400 0x557c24170920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:28.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.087+0000 7f2622ffd700 1 -- 172.21.15.19:0/3547262636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 msgr2=0x557c2416b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:28.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.087+0000 7f2622ffd700 1 --2- 172.21.15.19:0/3547262636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c2416b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.087+0000 7f2622ffd700 1 -- 172.21.15.19:0/3547262636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24166400 msgr2=0x557c24170920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:28.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.087+0000 7f2622ffd700 1 --2- 172.21.15.19:0/3547262636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24166400 0x557c24170920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.087+0000 7f2622ffd700 1 -- 172.21.15.19:0/3547262636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c2415d9d0 con 0x557c24063d00 2024-05-09T02:12:28.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.087+0000 7f26237fe700 1 --2- 172.21.15.19:0/3547262636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c2416b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:12:28.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.087+0000 7f2622ffd700 1 --2- 172.21.15.19:0/3547262636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24063d00 0x557c24182900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2618013b00 tx=0x7f2618013580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f0d9e9892189525 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:28.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.087+0000 7f26097fa700 1 -- 172.21.15.19:0/3547262636 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618021040 con 0x557c24063d00 2024-05-09T02:12:28.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.088+0000 7f26097fa700 1 -- 172.21.15.19:0/3547262636 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2618032e50 con 0x557c24063d00 2024-05-09T02:12:28.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.088+0000 7f26097fa700 1 -- 172.21.15.19:0/3547262636 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261800e6a0 con 0x557c24063d00 2024-05-09T02:12:28.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.089+0000 7f2638894b80 1 -- 172.21.15.19:0/3547262636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24063d00 msgr2=0x557c24182900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.089+0000 7f2638894b80 1 --2- 172.21.15.19:0/3547262636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24063d00 0x557c24182900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2618013b00 tx=0x7f2618013580 comp rx=0 tx=0).stop 2024-05-09T02:12:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.089+0000 7f2638894b80 1 -- 172.21.15.19:0/3547262636 shutdown_connections 2024-05-09T02:12:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.089+0000 7f2638894b80 1 --2- 172.21.15.19:0/3547262636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24166400 0x557c24170920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.089+0000 7f2638894b80 1 --2- 172.21.15.19:0/3547262636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c2416b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.089+0000 7f2638894b80 1 --2- 172.21.15.19:0/3547262636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24063d00 0x557c24182900 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.090+0000 7f2638894b80 1 -- 172.21.15.19:0/3547262636 >> 172.21.15.19:0/3547262636 conn(0x557c23fecf70 msgr2=0x557c2416b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.090+0000 7f2638894b80 1 -- 172.21.15.19:0/3547262636 shutdown_connections 2024-05-09T02:12:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.090+0000 7f2638894b80 1 -- 172.21.15.19:0/3547262636 wait complete. 2024-05-09T02:12:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.091+0000 7f2638894b80 1 -- start start 2024-05-09T02:12:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.091+0000 7f2638894b80 1 Processor -- start 2024-05-09T02:12:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.092+0000 7f2638894b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c241781c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.092+0000 7f2638894b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24063d00 0x557c24178700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.092+0000 7f26237fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c241781c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:28.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.092+0000 7f26237fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c241781c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:54332/0 (socket says 172.21.15.19:54332) 2024-05-09T02:12:28.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.092+0000 7f26237fe700 1 -- 172.21.15.19:0/1542681187 learned_addr learned my addr 172.21.15.19:0/1542681187 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:28.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.092+0000 7f2622ffd700 1 --2- 172.21.15.19:0/1542681187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24063d00 0x557c24178700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:28.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.092+0000 7f2638894b80 1 --2- 172.21.15.19:0/1542681187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24166400 0x557c24186f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.093+0000 7f2638894b80 1 -- 172.21.15.19:0/1542681187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f261000ea80 con 0x557c24166400 2024-05-09T02:12:28.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.093+0000 7f2638894b80 1 -- 172.21.15.19:0/1542681187 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f261000e3b0 con 0x557c24063d00 2024-05-09T02:12:28.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.093+0000 7f2623fff700 1 --2- 172.21.15.19:0/1542681187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24166400 0x557c24186f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:28.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.093+0000 7f2638894b80 1 -- 172.21.15.19:0/1542681187 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f261000b5b0 con 0x557c23fe94a0 2024-05-09T02:12:28.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.093+0000 7f2638894b80 10 client.0 _ll_register_callbacks cb 0x557c23efa6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:28.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.093+0000 7f26237fe700 1 -- 172.21.15.19:0/1542681187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24063d00 msgr2=0x557c24178700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:28.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.093+0000 7f26237fe700 1 --2- 172.21.15.19:0/1542681187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24063d00 0x557c24178700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.093+0000 7f26237fe700 1 -- 172.21.15.19:0/1542681187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24166400 msgr2=0x557c24186f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:28.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.093+0000 7f26237fe700 1 --2- 172.21.15.19:0/1542681187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24166400 0x557c24186f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.093+0000 7f26237fe700 1 -- 172.21.15.19:0/1542681187 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c24187440 con 0x557c23fe94a0 2024-05-09T02:12:28.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.093+0000 7f2622ffd700 1 --2- 172.21.15.19:0/1542681187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24063d00 0x557c24178700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:12:28.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.094+0000 7f2638894b80 1 -- 172.21.15.19:0/1542681187 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2618032e50 con 0x557c23fe94a0 2024-05-09T02:12:28.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.094+0000 7f26237fe700 1 --2- 172.21.15.19:0/1542681187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c241781c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f261402bfd0 tx=0x7f2614023270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bebeb46c39767db6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:28.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.094+0000 7f2602ffd700 10 client.25237 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:12:28.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.095+0000 7f2602ffd700 1 -- 172.21.15.19:0/1542681187 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261400bcb0 con 0x557c23fe94a0 2024-05-09T02:12:28.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.096+0000 7f2602ffd700 1 -- 172.21.15.19:0/1542681187 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f2614003a20 con 0x557c23fe94a0 2024-05-09T02:12:28.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.097+0000 7f2602ffd700 1 -- 172.21.15.19:0/1542681187 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2614003c80 con 0x557c23fe94a0 2024-05-09T02:12:28.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.097+0000 7f2638894b80 10 client.25237 fetch_fsmap learned FSMap version 32 2024-05-09T02:12:28.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.097+0000 7f2638894b80 1 -- 172.21.15.19:0/1542681187 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=32}) -- 0x557c241ea010 con 0x557c23fe94a0 2024-05-09T02:12:28.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.097+0000 7f2602ffd700 1 -- 172.21.15.19:0/1542681187 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f2614023970 con 0x557c23fe94a0 2024-05-09T02:12:28.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.097+0000 7f2638894b80 10 client.25237 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T02:12:28.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.097+0000 7f2638894b80 10 client.25237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:12:28.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.097+0000 7f2638894b80 20 client.25237 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:28.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.097+0000 7f2638894b80 1 --2- 172.21.15.19:0/1542681187 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557c241ea540 0x557c241ec960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:28.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.098+0000 7f2638894b80 4 client.25237 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:12:28.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.098+0000 7f2638894b80 1 -- 172.21.15.19:0/1542681187 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c23f04970 con 0x557c241ea540 2024-05-09T02:12:28.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.098+0000 7f2622ffd700 1 --2- 172.21.15.19:0/1542681187 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557c241ea540 0x557c241ec960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:28.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.098+0000 7f2622ffd700 1 --2- 172.21.15.19:0/1542681187 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557c241ea540 0x557c241ec960 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7991ea8ccf5cf4f server_cookie=869e0f2e4b7742ec in_seq=0 out_seq=0 2024-05-09T02:12:28.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.098+0000 7f2602ffd700 10 client.25237 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:28.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.098+0000 7f2602ffd700 1 -- 172.21.15.19:0/1542681187 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c23f04970 con 0x557c241ea540 2024-05-09T02:12:28.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.098+0000 7f2602ffd700 10 client.25237 handle_command_reply: tid=0 2024-05-09T02:12:28.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.098+0000 7f2638894b80 1 client.25237 shutdown 2024-05-09T02:12:28.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.099+0000 7f2638894b80 1 -- 172.21.15.19:0/1542681187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 msgr2=0x557c241781c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:28.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.099+0000 7f2638894b80 1 --2- 172.21.15.19:0/1542681187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c241781c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f261402bfd0 tx=0x7f2614023270 comp rx=0 tx=0).stop 2024-05-09T02:12:28.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.099+0000 7f2638894b80 1 -- 172.21.15.19:0/1542681187 shutdown_connections 2024-05-09T02:12:28.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.099+0000 7f2638894b80 1 --2- 172.21.15.19:0/1542681187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c24166400 0x557c24186f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.099+0000 7f2638894b80 1 --2- 172.21.15.19:0/1542681187 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557c241ea540 0x557c241ec960 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.099+0000 7f2638894b80 1 --2- 172.21.15.19:0/1542681187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c23fe94a0 0x557c241781c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.099+0000 7f2638894b80 1 --2- 172.21.15.19:0/1542681187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c24063d00 0x557c24178700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.099+0000 7f2602ffd700 0 client.25237 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:28.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.099+0000 7f2638894b80 1 -- 172.21.15.19:0/1542681187 >> 172.21.15.19:0/1542681187 conn(0x557c23fecf70 msgr2=0x557c2415aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:28.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.099+0000 7f2638894b80 1 -- 172.21.15.19:0/1542681187 shutdown_connections 2024-05-09T02:12:28.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.099+0000 7f2638894b80 1 -- 172.21.15.19:0/1542681187 wait complete. 2024-05-09T02:12:28.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.099+0000 7f2638894b80 20 client.25237 trim_cache size 0 max 16384 2024-05-09T02:12:28.103 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:12:28.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.100+0000 7f2629e7e700 1 -- 172.21.15.19:0/1331516938 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f260c0b2bd0 msgr2=0x7f260c0b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:28.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.100+0000 7f2629e7e700 1 --2- 172.21.15.19:0/1331516938 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f260c0b2bd0 0x7f260c0b50e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f26180106b0 tx=0x7f261800b000 comp rx=0 tx=0).stop 2024-05-09T02:12:28.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.100+0000 7f2629e7e700 1 -- 172.21.15.19:0/1331516938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f262410c5b0 msgr2=0x7f26241a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:28.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.100+0000 7f2629e7e700 1 --2- 172.21.15.19:0/1331516938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f262410c5b0 0x7f26241a18c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f261c00c370 tx=0x7f261c00c730 comp rx=0 tx=0).stop 2024-05-09T02:12:28.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.100+0000 7f2629e7e700 1 -- 172.21.15.19:0/1331516938 shutdown_connections 2024-05-09T02:12:28.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.100+0000 7f2629e7e700 1 --2- 172.21.15.19:0/1331516938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f262410c5b0 0x7f26241a18c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.100+0000 7f2629e7e700 1 --2- 172.21.15.19:0/1331516938 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f260c0b2bd0 0x7f260c0b50e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.100+0000 7f2629e7e700 1 --2- 172.21.15.19:0/1331516938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2624106570 0x7f262419d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.100+0000 7f2629e7e700 1 --2- 172.21.15.19:0/1331516938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2624106ec0 0x7f262419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:28.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.100+0000 7f2629e7e700 1 -- 172.21.15.19:0/1331516938 >> 172.21.15.19:0/1331516938 conn(0x7f26241020d0 msgr2=0x7f2624104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:28.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.100+0000 7f2629e7e700 1 -- 172.21.15.19:0/1331516938 shutdown_connections 2024-05-09T02:12:28.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:28.100+0000 7f2629e7e700 1 -- 172.21.15.19:0/1331516938 wait complete. 2024-05-09T02:12:28.506 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:28 smithi019 ceph-mon[30151]: pgmap v38: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 663 op/s 2024-05-09T02:12:28.506 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:28 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:12:28.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:28 smithi196 ceph-mon[34718]: pgmap v38: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 663 op/s 2024-05-09T02:12:28.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:28 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:12:28.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:28 smithi059 ceph-mon[34781]: pgmap v38: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 663 op/s 2024-05-09T02:12:28.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:28 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:12:30.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:30 smithi019 ceph-mon[30151]: pgmap v39: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 963 op/s 2024-05-09T02:12:30.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:30 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:12:30.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:30 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:12:30.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:30 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:12:30.536 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T02:12:30.536 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T02:12:30.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:30 smithi196 ceph-mon[34718]: pgmap v39: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 963 op/s 2024-05-09T02:12:30.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:30 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:12:30.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:30 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:12:30.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:30 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:12:30.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:30 smithi059 ceph-mon[34781]: pgmap v39: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 963 op/s 2024-05-09T02:12:30.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:30 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:12:30.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:30 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:12:30.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:30 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:12:30.898 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:12:32.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:32 smithi019 ceph-mon[30151]: pgmap v40: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 621 op/s 2024-05-09T02:12:32.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:32 smithi196 ceph-mon[34718]: pgmap v40: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 621 op/s 2024-05-09T02:12:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 50.0 s, 3537.8 tps, lat 28.158 ms stddev 28.002 2024-05-09T02:12:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:32 smithi059 ceph-mon[34781]: pgmap v40: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 621 op/s 2024-05-09T02:12:32.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.979+0000 7fe338439700 1 -- 172.21.15.19:0/2726731734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe330108740 msgr2=0x7fe3301100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:32.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.979+0000 7fe338439700 1 --2- 172.21.15.19:0/2726731734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe330108740 0x7fe3301100a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe3200099c0 tx=0x7fe320009cd0 comp rx=0 tx=0).stop 2024-05-09T02:12:32.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.979+0000 7fe338439700 1 -- 172.21.15.19:0/2726731734 shutdown_connections 2024-05-09T02:12:32.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.979+0000 7fe338439700 1 --2- 172.21.15.19:0/2726731734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe330108740 0x7fe3301100a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:32.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.979+0000 7fe338439700 1 --2- 172.21.15.19:0/2726731734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3301034c0 0x7fe3301038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:32.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.979+0000 7fe338439700 1 --2- 172.21.15.19:0/2726731734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe330107df0 0x7fe330108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:32.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.979+0000 7fe338439700 1 -- 172.21.15.19:0/2726731734 >> 172.21.15.19:0/2726731734 conn(0x7fe3300fedc0 msgr2=0x7fe3301011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:32.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.980+0000 7fe338439700 1 -- 172.21.15.19:0/2726731734 shutdown_connections 2024-05-09T02:12:32.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.980+0000 7fe338439700 1 -- 172.21.15.19:0/2726731734 wait complete. 2024-05-09T02:12:32.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.981+0000 7fe338439700 1 Processor -- start 2024-05-09T02:12:32.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.982+0000 7fe338439700 1 -- start start 2024-05-09T02:12:32.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.982+0000 7fe338439700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3301034c0 0x7fe3301a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:32.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.982+0000 7fe338439700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe330107df0 0x7fe330198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.982+0000 7fe338439700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe330108740 0x7fe3301990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.982+0000 7fe338439700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe330113370 con 0x7fe3301034c0 2024-05-09T02:12:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.982+0000 7fe338439700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3301131f0 con 0x7fe330107df0 2024-05-09T02:12:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.982+0000 7fe338439700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3301134f0 con 0x7fe330108740 2024-05-09T02:12:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.982+0000 7fe3359d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe330107df0 0x7fe330198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.982+0000 7fe3359d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe330107df0 0x7fe330198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:35490/0 (socket says 172.21.15.19:35490) 2024-05-09T02:12:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.982+0000 7fe3359d4700 1 -- 172.21.15.19:0/2976563422 learned_addr learned my addr 172.21.15.19:0/2976563422 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.982+0000 7fe3369d6700 1 --2- 172.21.15.19:0/2976563422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe330108740 0x7fe3301990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.983+0000 7fe3359d4700 1 -- 172.21.15.19:0/2976563422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe330108740 msgr2=0x7fe3301990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.983+0000 7fe3359d4700 1 --2- 172.21.15.19:0/2976563422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe330108740 0x7fe3301990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.983+0000 7fe3359d4700 1 -- 172.21.15.19:0/2976563422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3301034c0 msgr2=0x7fe3301a0950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:12:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.983+0000 7fe3359d4700 1 --2- 172.21.15.19:0/2976563422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3301034c0 0x7fe3301a0950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.983+0000 7fe3359d4700 1 -- 172.21.15.19:0/2976563422 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3301998f0 con 0x7fe330107df0 2024-05-09T02:12:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.983+0000 7fe3359d4700 1 --2- 172.21.15.19:0/2976563422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe330107df0 0x7fe330198b70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe32800efc0 tx=0x7fe32800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.984+0000 7fe3277fe700 1 -- 172.21.15.19:0/2976563422 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32800cad0 con 0x7fe330107df0 2024-05-09T02:12:32.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.984+0000 7fe3277fe700 1 -- 172.21.15.19:0/2976563422 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe328019070 con 0x7fe330107df0 2024-05-09T02:12:32.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.984+0000 7fe3277fe700 1 -- 172.21.15.19:0/2976563422 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3280146e0 con 0x7fe330107df0 2024-05-09T02:12:32.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.984+0000 7fe338439700 1 -- 172.21.15.19:0/2976563422 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3301b6080 con 0x7fe330107df0 2024-05-09T02:12:32.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.984+0000 7fe338439700 1 -- 172.21.15.19:0/2976563422 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3301b6570 con 0x7fe330107df0 2024-05-09T02:12:32.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.988+0000 7fe3277fe700 1 -- 172.21.15.19:0/2976563422 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fe328012050 con 0x7fe330107df0 2024-05-09T02:12:32.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.992+0000 7fe3277fe700 1 --2- 172.21.15.19:0/2976563422 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe31c0b2c60 0x7fe31c0b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:32.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.992+0000 7fe3277fe700 1 -- 172.21.15.19:0/2976563422 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fe3280e2830 con 0x7fe330107df0 2024-05-09T02:12:32.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.992+0000 7fe3361d5700 1 --2- 172.21.15.19:0/2976563422 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe31c0b2c60 0x7fe31c0b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:32.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.993+0000 7fe3361d5700 1 --2- 172.21.15.19:0/2976563422 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe31c0b2c60 0x7fe31c0b5170 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe33006c1a0 tx=0x7fe32c009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:32.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.995+0000 7fe346e4fb80 1 Processor -- start 2024-05-09T02:12:32.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.995+0000 7fe346e4fb80 1 -- start start 2024-05-09T02:12:32.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.995+0000 7fe346e4fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02ed5d80 0x561a02fde2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:32.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.995+0000 7fe346e4fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02e5b520 0x561a02fde800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:32.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.995+0000 7fe346e4fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02fd8480 0x561a02fd8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:32.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.995+0000 7fe346e4fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561a02efb120 con 0x561a02fd8480 2024-05-09T02:12:32.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe346e4fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561a02ea0a20 con 0x561a02ed5d80 2024-05-09T02:12:32.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe3369d6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02fd8480 0x561a02fd8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:32.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe3369d6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02fd8480 0x561a02fd8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41196/0 (socket says 172.21.15.19:41196) 2024-05-09T02:12:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe3361d5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02ed5d80 0x561a02fde2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe3369d6700 1 -- 172.21.15.19:0/382366103 learned_addr learned my addr 172.21.15.19:0/382366103 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe346e4fb80 1 -- 172.21.15.19:0/382366103 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a02ef70e0 con 0x561a02e5b520 2024-05-09T02:12:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe3359d4700 1 --2- 172.21.15.19:0/382366103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02e5b520 0x561a02fde800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe3369d6700 1 -- 172.21.15.19:0/382366103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02e5b520 msgr2=0x561a02fde800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe3369d6700 1 --2- 172.21.15.19:0/382366103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02e5b520 0x561a02fde800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe3369d6700 1 -- 172.21.15.19:0/382366103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02ed5d80 msgr2=0x561a02fde2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe3369d6700 1 --2- 172.21.15.19:0/382366103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02ed5d80 0x561a02fde2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe3369d6700 1 -- 172.21.15.19:0/382366103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a02e91580 con 0x561a02fd8480 2024-05-09T02:12:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe3361d5700 1 --2- 172.21.15.19:0/382366103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02ed5d80 0x561a02fde2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:12:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe3369d6700 1 --2- 172.21.15.19:0/382366103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02fd8480 0x561a02fd8890 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe320015000 tx=0x7fe320003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71267c5ee6c64cc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe324d47700 1 -- 172.21.15.19:0/382366103 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320027070 con 0x561a02fd8480 2024-05-09T02:12:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe324d47700 1 -- 172.21.15.19:0/382366103 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe320003c70 con 0x561a02fd8480 2024-05-09T02:12:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.996+0000 7fe324d47700 1 -- 172.21.15.19:0/382366103 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320022480 con 0x561a02fd8480 2024-05-09T02:12:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.997+0000 7fe346e4fb80 1 -- 172.21.15.19:0/382366103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02fd8480 msgr2=0x561a02fd8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.997+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/382366103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02fd8480 0x561a02fd8890 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe320015000 tx=0x7fe320003680 comp rx=0 tx=0).stop 2024-05-09T02:12:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.997+0000 7fe346e4fb80 1 -- 172.21.15.19:0/382366103 shutdown_connections 2024-05-09T02:12:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.997+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/382366103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02fd8480 0x561a02fd8890 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.997+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/382366103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02ed5d80 0x561a02fde2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.997+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/382366103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02e5b520 0x561a02fde800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.997+0000 7fe346e4fb80 1 -- 172.21.15.19:0/382366103 >> 172.21.15.19:0/382366103 conn(0x561a02e5eff0 msgr2=0x561a02fd32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.998+0000 7fe346e4fb80 1 -- 172.21.15.19:0/382366103 shutdown_connections 2024-05-09T02:12:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:32.999+0000 7fe346e4fb80 1 -- 172.21.15.19:0/382366103 wait complete. 2024-05-09T02:12:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.000+0000 7fe346e4fb80 1 -- start start 2024-05-09T02:12:33.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.000+0000 7fe346e4fb80 1 Processor -- start 2024-05-09T02:12:33.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe346e4fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02e5b520 0x561a02fe2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:33.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe346e4fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02ed5d80 0x561a02fe2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:33.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe346e4fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02fd8480 0x561a02fe2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:33.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe346e4fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561a02fd9180 con 0x561a02fd8480 2024-05-09T02:12:33.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe346e4fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561a02e95ba0 con 0x561a02e5b520 2024-05-09T02:12:33.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe346e4fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a02fee250 con 0x561a02ed5d80 2024-05-09T02:12:33.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe3369d6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02fd8480 0x561a02fe2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:33.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe346e4fb80 10 client.0 _ll_register_callbacks cb 0x561a02d6c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:33.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe3361d5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02e5b520 0x561a02fe2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:33.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe3369d6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02fd8480 0x561a02fe2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41202/0 (socket says 172.21.15.19:41202) 2024-05-09T02:12:33.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe3369d6700 1 -- 172.21.15.19:0/1975990605 learned_addr learned my addr 172.21.15.19:0/1975990605 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:33.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe3359d4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02ed5d80 0x561a02fe2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:33.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe3369d6700 1 -- 172.21.15.19:0/1975990605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02ed5d80 msgr2=0x561a02fe2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:33.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe3369d6700 1 --2- 172.21.15.19:0/1975990605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02ed5d80 0x561a02fe2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe3369d6700 1 -- 172.21.15.19:0/1975990605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02e5b520 msgr2=0x561a02fe2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:33.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe3369d6700 1 --2- 172.21.15.19:0/1975990605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02e5b520 0x561a02fe2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.001+0000 7fe3369d6700 1 -- 172.21.15.19:0/1975990605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a02d76510 con 0x561a02fd8480 2024-05-09T02:12:33.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.002+0000 7fe3359d4700 1 --2- 172.21.15.19:0/1975990605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02ed5d80 0x561a02fe2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:12:33.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.002+0000 7fe3361d5700 1 --2- 172.21.15.19:0/1975990605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02e5b520 0x561a02fe2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:12:33.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.002+0000 7fe3369d6700 1 --2- 172.21.15.19:0/1975990605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02fd8480 0x561a02fe2b90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe320025040 tx=0x7fe32000b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90bad8eb733c776 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:33.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.002+0000 7fe319ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:12:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.002+0000 7fe319ffb700 1 -- 172.21.15.19:0/1975990605 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320027040 con 0x561a02fd8480 2024-05-09T02:12:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.002+0000 7fe319ffb700 1 -- 172.21.15.19:0/1975990605 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fe3200037c0 con 0x561a02fd8480 2024-05-09T02:12:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.002+0000 7fe346e4fb80 1 -- 172.21.15.19:0/1975990605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe320003c70 con 0x561a02fd8480 2024-05-09T02:12:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.002+0000 7fe319ffb700 1 -- 172.21.15.19:0/1975990605 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe3200039c0 con 0x561a02fd8480 2024-05-09T02:12:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.002+0000 7fe346e4fb80 10 client.15522 fetch_fsmap learned FSMap version 32 2024-05-09T02:12:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.002+0000 7fe346e4fb80 1 -- 172.21.15.19:0/1975990605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561a02fefbd0 con 0x561a02fd8480 2024-05-09T02:12:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.002+0000 7fe319ffb700 1 -- 172.21.15.19:0/1975990605 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe320030070 con 0x561a02fd8480 2024-05-09T02:12:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.003+0000 7fe319ffb700 1 -- 172.21.15.19:0/1975990605 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fe3200227b0 con 0x561a02fd8480 2024-05-09T02:12:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.003+0000 7fe346e4fb80 10 client.15522 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T02:12:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.003+0000 7fe346e4fb80 10 client.15522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:12:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.003+0000 7fe346e4fb80 20 client.15522 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.003+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/1975990605 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x561a02ff20a0 0x561a02ff44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.003+0000 7fe346e4fb80 4 client.15522 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:12:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.003+0000 7fe346e4fb80 1 -- 172.21.15.19:0/1975990605 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe320003c70 con 0x561a02ff20a0 2024-05-09T02:12:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.003+0000 7fe3361d5700 1 --2- 172.21.15.19:0/1975990605 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x561a02ff20a0 0x561a02ff44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:33.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.004+0000 7fe319ffb700 10 client.15522 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.004+0000 7fe3361d5700 1 --2- 172.21.15.19:0/1975990605 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x561a02ff20a0 0x561a02ff44c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ad11e5922fe764 server_cookie=e04a5a16e19b4634 in_seq=0 out_seq=0 2024-05-09T02:12:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.005+0000 7fe319ffb700 1 -- 172.21.15.19:0/1975990605 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fe320003c70 con 0x561a02ff20a0 2024-05-09T02:12:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.005+0000 7fe319ffb700 10 client.15522 handle_command_reply: tid=0 2024-05-09T02:12:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.005+0000 7fe346e4fb80 1 client.15522 shutdown 2024-05-09T02:12:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.005+0000 7fe346e4fb80 1 -- 172.21.15.19:0/1975990605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02fd8480 msgr2=0x561a02fe2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.005+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/1975990605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02fd8480 0x561a02fe2b90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe320025040 tx=0x7fe32000b9c0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.005+0000 7fe346e4fb80 1 -- 172.21.15.19:0/1975990605 shutdown_connections 2024-05-09T02:12:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.005+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/1975990605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02fd8480 0x561a02fe2b90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.005+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/1975990605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02e5b520 0x561a02fe2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.005+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/1975990605 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x561a02ff20a0 0x561a02ff44c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.005+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/1975990605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02ed5d80 0x561a02fe2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.005+0000 7fe319ffb700 0 client.15522 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.005+0000 7fe346e4fb80 1 -- 172.21.15.19:0/1975990605 >> 172.21.15.19:0/1975990605 conn(0x561a02e5eff0 msgr2=0x561a02fcd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:33.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.005+0000 7fe346e4fb80 1 -- 172.21.15.19:0/1975990605 shutdown_connections 2024-05-09T02:12:33.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.006+0000 7fe346e4fb80 1 -- 172.21.15.19:0/1975990605 wait complete. 2024-05-09T02:12:33.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.006+0000 7fe346e4fb80 20 client.15522 trim_cache size 0 max 16384 2024-05-09T02:12:33.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.040+0000 7fe346e4fb80 1 Processor -- start 2024-05-09T02:12:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.040+0000 7fe346e4fb80 1 -- start start 2024-05-09T02:12:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.040+0000 7fe346e4fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02e5b520 0x561a02fe1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.040+0000 7fe346e4fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02ed5d80 0x561a02fee1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.040+0000 7fe346e4fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02fd8480 0x561a02fcd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.040+0000 7fe346e4fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3301bdc70 con 0x561a02ed5d80 2024-05-09T02:12:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.040+0000 7fe346e4fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3301bf840 con 0x561a02e5b520 2024-05-09T02:12:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.040+0000 7fe3359d4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02ed5d80 0x561a02fee1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.040+0000 7fe346e4fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3301be340 con 0x561a02fd8480 2024-05-09T02:12:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.040+0000 7fe3361d5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02e5b520 0x561a02fe1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.040+0000 7fe3359d4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02ed5d80 0x561a02fee1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41204/0 (socket says 172.21.15.19:41204) 2024-05-09T02:12:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.040+0000 7fe3359d4700 1 -- 172.21.15.19:0/3175199898 learned_addr learned my addr 172.21.15.19:0/3175199898 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.041+0000 7fe3369d6700 1 --2- 172.21.15.19:0/3175199898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02fd8480 0x561a02fcd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.041+0000 7fe3361d5700 1 -- 172.21.15.19:0/3175199898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02fd8480 msgr2=0x561a02fcd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.041+0000 7fe3361d5700 1 --2- 172.21.15.19:0/3175199898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02fd8480 0x561a02fcd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.041+0000 7fe3361d5700 1 -- 172.21.15.19:0/3175199898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02ed5d80 msgr2=0x561a02fee1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.041+0000 7fe3361d5700 1 --2- 172.21.15.19:0/3175199898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02ed5d80 0x561a02fee1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.041+0000 7fe3361d5700 1 -- 172.21.15.19:0/3175199898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a02fe06a0 con 0x561a02e5b520 2024-05-09T02:12:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.041+0000 7fe3369d6700 1 --2- 172.21.15.19:0/3175199898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02fd8480 0x561a02fcd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:12:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.041+0000 7fe3361d5700 1 --2- 172.21.15.19:0/3175199898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02e5b520 0x561a02fe1090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x561a02e95c30 tx=0x7fe32c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dfaa6862dacbfd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.041+0000 7fe31bfff700 1 -- 172.21.15.19:0/3175199898 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c01e190 con 0x561a02e5b520 2024-05-09T02:12:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.041+0000 7fe31bfff700 1 -- 172.21.15.19:0/3175199898 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe32c01ede0 con 0x561a02e5b520 2024-05-09T02:12:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.041+0000 7fe31bfff700 1 -- 172.21.15.19:0/3175199898 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c005280 con 0x561a02e5b520 2024-05-09T02:12:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.043+0000 7fe346e4fb80 1 -- 172.21.15.19:0/3175199898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02e5b520 msgr2=0x561a02fe1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.043+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/3175199898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02e5b520 0x561a02fe1090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x561a02e95c30 tx=0x7fe32c011000 comp rx=0 tx=0).stop 2024-05-09T02:12:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.043+0000 7fe346e4fb80 1 -- 172.21.15.19:0/3175199898 shutdown_connections 2024-05-09T02:12:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.043+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/3175199898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02fd8480 0x561a02fcd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.043+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/3175199898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a02e5b520 0x561a02fe1090 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x561a02e95c30 tx=0x7fe32c011000 comp rx=0 tx=0).stop 2024-05-09T02:12:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.043+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/3175199898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02ed5d80 0x561a02fee1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.043+0000 7fe346e4fb80 1 -- 172.21.15.19:0/3175199898 >> 172.21.15.19:0/3175199898 conn(0x561a02e5eff0 msgr2=0x561a02fcd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.044+0000 7fe346e4fb80 1 -- 172.21.15.19:0/3175199898 shutdown_connections 2024-05-09T02:12:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.044+0000 7fe346e4fb80 1 -- 172.21.15.19:0/3175199898 wait complete. 2024-05-09T02:12:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.045+0000 7fe346e4fb80 1 -- start start 2024-05-09T02:12:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.045+0000 7fe346e4fb80 1 Processor -- start 2024-05-09T02:12:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.046+0000 7fe346e4fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02ed5d80 0x561a02ff54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.046+0000 7fe346e4fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02fd8480 0x561a02ff5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.046+0000 7fe346e4fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a030555c0 0x561a030559d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.046+0000 7fe346e4fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3301bf0e0 con 0x561a02ed5d80 2024-05-09T02:12:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.046+0000 7fe346e4fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3301bea10 con 0x561a030555c0 2024-05-09T02:12:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.046+0000 7fe346e4fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3301bbc10 con 0x561a02fd8480 2024-05-09T02:12:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.046+0000 7fe346e4fb80 10 client.0 _ll_register_callbacks cb 0x561a02d6c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.047+0000 7fe3361d5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02ed5d80 0x561a02ff54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.047+0000 7fe3361d5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02ed5d80 0x561a02ff54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41218/0 (socket says 172.21.15.19:41218) 2024-05-09T02:12:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.047+0000 7fe3361d5700 1 -- 172.21.15.19:0/3019090175 learned_addr learned my addr 172.21.15.19:0/3019090175 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.047+0000 7fe3359d4700 1 --2- 172.21.15.19:0/3019090175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02fd8480 0x561a02ff5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.047+0000 7fe3361d5700 1 -- 172.21.15.19:0/3019090175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02fd8480 msgr2=0x561a02ff5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.047+0000 7fe3361d5700 1 --2- 172.21.15.19:0/3019090175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02fd8480 0x561a02ff5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.047+0000 7fe3361d5700 1 -- 172.21.15.19:0/3019090175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a030555c0 msgr2=0x561a030559d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:12:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.047+0000 7fe3361d5700 1 --2- 172.21.15.19:0/3019090175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a030555c0 0x561a030559d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.047+0000 7fe3361d5700 1 -- 172.21.15.19:0/3019090175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a03055f40 con 0x561a02ed5d80 2024-05-09T02:12:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.048+0000 7fe3361d5700 1 --2- 172.21.15.19:0/3019090175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02ed5d80 0x561a02ff54f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe32c01e040 tx=0x7fe32c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629ee1134f1b75e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.048+0000 7fe319ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:12:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.048+0000 7fe319ffb700 1 -- 172.21.15.19:0/3019090175 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c01ec30 con 0x561a02ed5d80 2024-05-09T02:12:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.048+0000 7fe319ffb700 1 -- 172.21.15.19:0/3019090175 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fe32c032460 con 0x561a02ed5d80 2024-05-09T02:12:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.048+0000 7fe346e4fb80 1 -- 172.21.15.19:0/3019090175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe32c01ede0 con 0x561a02ed5d80 2024-05-09T02:12:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.051+0000 7fe319ffb700 1 -- 172.21.15.19:0/3019090175 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe32c032680 con 0x561a02ed5d80 2024-05-09T02:12:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.051+0000 7fe346e4fb80 10 client.15528 fetch_fsmap learned FSMap version 32 2024-05-09T02:12:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.051+0000 7fe346e4fb80 1 -- 172.21.15.19:0/3019090175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561a0305d1e0 con 0x561a02ed5d80 2024-05-09T02:12:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.052+0000 7fe319ffb700 1 -- 172.21.15.19:0/3019090175 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe32c01ede0 con 0x561a02ed5d80 2024-05-09T02:12:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.052+0000 7fe319ffb700 1 -- 172.21.15.19:0/3019090175 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fe32c027a60 con 0x561a02ed5d80 2024-05-09T02:12:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.052+0000 7fe346e4fb80 10 client.15528 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T02:12:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.052+0000 7fe346e4fb80 10 client.15528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:12:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.052+0000 7fe346e4fb80 20 client.15528 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.052+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/3019090175 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x561a0305d840 0x561a0305fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.052+0000 7fe346e4fb80 4 client.15528 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T02:12:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.052+0000 7fe346e4fb80 1 -- 172.21.15.19:0/3019090175 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561a02f08b00 con 0x561a0305d840 2024-05-09T02:12:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.053+0000 7fe3359d4700 1 --2- 172.21.15.19:0/3019090175 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x561a0305d840 0x561a0305fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.054+0000 7fe3359d4700 1 --2- 172.21.15.19:0/3019090175 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x561a0305d840 0x561a0305fc60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9c8c24e0f797404 server_cookie=e70ece785df2d5d4 in_seq=0 out_seq=0 2024-05-09T02:12:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.054+0000 7fe319ffb700 10 client.15528 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.057+0000 7fe319ffb700 1 -- 172.21.15.19:0/3019090175 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561a02f08b00 con 0x561a0305d840 2024-05-09T02:12:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.058+0000 7fe319ffb700 10 client.15528 handle_command_reply: tid=0 2024-05-09T02:12:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.058+0000 7fe346e4fb80 1 client.15528 shutdown 2024-05-09T02:12:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.058+0000 7fe346e4fb80 1 -- 172.21.15.19:0/3019090175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02ed5d80 msgr2=0x561a02ff54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.058+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/3019090175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02ed5d80 0x561a02ff54f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe32c01e040 tx=0x7fe32c00ee60 comp rx=0 tx=0).stop 2024-05-09T02:12:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.061+0000 7fe346e4fb80 1 -- 172.21.15.19:0/3019090175 shutdown_connections 2024-05-09T02:12:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.061+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/3019090175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a030555c0 0x561a030559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.061+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/3019090175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a02ed5d80 0x561a02ff54f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.061+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/3019090175 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x561a0305d840 0x561a0305fc60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.061+0000 7fe346e4fb80 1 --2- 172.21.15.19:0/3019090175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a02fd8480 0x561a02ff5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.061+0000 7fe346e4fb80 1 -- 172.21.15.19:0/3019090175 >> 172.21.15.19:0/3019090175 conn(0x561a02e5eff0 msgr2=0x561a02fdbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.061+0000 7fe319ffb700 0 client.15528 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.062+0000 7fe346e4fb80 1 -- 172.21.15.19:0/3019090175 shutdown_connections 2024-05-09T02:12:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.064+0000 7fe346e4fb80 1 -- 172.21.15.19:0/3019090175 wait complete. 2024-05-09T02:12:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.065+0000 7fe346e4fb80 20 client.15528 trim_cache size 0 max 16384 2024-05-09T02:12:33.071 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:12:33.071 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T02:12:33.071 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "ac109e30-f302-4696-9654-c0961d606e78", 2024-05-09T02:12:33.071 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T02:12:33.071 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:12:33.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.069+0000 7fe338439700 1 -- 172.21.15.19:0/2976563422 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe31c0b2c60 msgr2=0x7fe31c0b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:33.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.069+0000 7fe338439700 1 --2- 172.21.15.19:0/2976563422 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe31c0b2c60 0x7fe31c0b5170 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe33006c1a0 tx=0x7fe32c009260 comp rx=0 tx=0).stop 2024-05-09T02:12:33.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.069+0000 7fe338439700 1 -- 172.21.15.19:0/2976563422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe330107df0 msgr2=0x7fe330198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:33.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.069+0000 7fe338439700 1 --2- 172.21.15.19:0/2976563422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe330107df0 0x7fe330198b70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe32800efc0 tx=0x7fe32800c3a0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.070+0000 7fe338439700 1 -- 172.21.15.19:0/2976563422 shutdown_connections 2024-05-09T02:12:33.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.071+0000 7fe338439700 1 --2- 172.21.15.19:0/2976563422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe330108740 0x7fe3301990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.071+0000 7fe338439700 1 --2- 172.21.15.19:0/2976563422 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe31c0b2c60 0x7fe31c0b5170 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.071+0000 7fe338439700 1 --2- 172.21.15.19:0/2976563422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3301034c0 0x7fe3301a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.071+0000 7fe338439700 1 --2- 172.21.15.19:0/2976563422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe330107df0 0x7fe330198b70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:33.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.071+0000 7fe338439700 1 -- 172.21.15.19:0/2976563422 >> 172.21.15.19:0/2976563422 conn(0x7fe3300fedc0 msgr2=0x7fe3301096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:33.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.072+0000 7fe338439700 1 -- 172.21.15.19:0/2976563422 shutdown_connections 2024-05-09T02:12:33.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:33.072+0000 7fe338439700 1 -- 172.21.15.19:0/2976563422 wait complete. 2024-05-09T02:12:33.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:33 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:12:33.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:33 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:12:33.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:33 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:12:33.781 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:12:34.175 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:12:34.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:34 smithi196 ceph-mon[34718]: scrub queued for path: / 2024-05-09T02:12:34.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:34 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:12:34.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:34 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:12:34.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:34 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:12:34.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:34 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:12:34.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:34 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:12:34.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:34 smithi196 ceph-mon[34718]: pgmap v41: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.2 MiB/s wr, 766 op/s 2024-05-09T02:12:34.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:34 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:12:34.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:34 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:12:34.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:34 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:12:34.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:34 smithi019 ceph-mon[30151]: scrub queued for path: / 2024-05-09T02:12:34.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:34 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:12:34.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:34 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:12:34.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:34 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:12:34.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:34 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:12:34.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:34 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:12:34.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:34 smithi019 ceph-mon[30151]: pgmap v41: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.2 MiB/s wr, 766 op/s 2024-05-09T02:12:34.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:34 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:12:34.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:34 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:12:34.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:34 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:12:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:34 smithi059 ceph-mon[34781]: scrub queued for path: / 2024-05-09T02:12:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:34 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:12:34.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:34 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:12:34.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:34 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:12:34.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:34 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:12:34.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:34 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:12:34.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:34 smithi059 ceph-mon[34781]: pgmap v41: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.2 MiB/s wr, 766 op/s 2024-05-09T02:12:34.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:34 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:12:34.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:34 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:12:34.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:34 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:12:36.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:36 smithi019 ceph-mon[30151]: pgmap v42: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.9 MiB/s wr, 819 op/s 2024-05-09T02:12:36.518 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:36 smithi059 ceph-mon[34781]: pgmap v42: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.9 MiB/s wr, 819 op/s 2024-05-09T02:12:36.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:36 smithi196 ceph-mon[34718]: pgmap v42: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.9 MiB/s wr, 819 op/s 2024-05-09T02:12:36.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.774+0000 7f4c399ae700 1 -- 172.21.15.19:0/3658075344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c34103c00 msgr2=0x7f4c34104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:36.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.774+0000 7f4c399ae700 1 --2- 172.21.15.19:0/3658075344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c34103c00 0x7f4c34104010 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4c240099c0 tx=0x7f4c24009cd0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.774+0000 7f4c399ae700 1 -- 172.21.15.19:0/3658075344 shutdown_connections 2024-05-09T02:12:36.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.774+0000 7f4c399ae700 1 --2- 172.21.15.19:0/3658075344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c34108750 0x7f4c341100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.774+0000 7f4c399ae700 1 --2- 172.21.15.19:0/3658075344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c34103c00 0x7f4c34104010 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.774+0000 7f4c399ae700 1 --2- 172.21.15.19:0/3658075344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c34107e00 0x7f4c34108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.774+0000 7f4c399ae700 1 -- 172.21.15.19:0/3658075344 >> 172.21.15.19:0/3658075344 conn(0x7f4c340fe6f0 msgr2=0x7f4c34100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:36.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.775+0000 7f4c399ae700 1 -- 172.21.15.19:0/3658075344 shutdown_connections 2024-05-09T02:12:36.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.775+0000 7f4c399ae700 1 -- 172.21.15.19:0/3658075344 wait complete. 2024-05-09T02:12:36.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.776+0000 7f4c399ae700 1 Processor -- start 2024-05-09T02:12:36.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.776+0000 7f4c399ae700 1 -- start start 2024-05-09T02:12:36.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.776+0000 7f4c399ae700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c34103c00 0x7f4c3419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.776+0000 7f4c399ae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c34107e00 0x7f4c3419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.776+0000 7f4c399ae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c34108750 0x7f4c3419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.776+0000 7f4c399ae700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c34113380 con 0x7f4c34107e00 2024-05-09T02:12:36.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.776+0000 7f4c399ae700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c34113200 con 0x7f4c34108750 2024-05-09T02:12:36.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.776+0000 7f4c399ae700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c34113500 con 0x7f4c34103c00 2024-05-09T02:12:36.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c32ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c34103c00 0x7f4c3419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c337fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c34108750 0x7f4c3419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c327fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c34107e00 0x7f4c3419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c337fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c34108750 0x7f4c3419f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:48742/0 (socket says 172.21.15.19:48742) 2024-05-09T02:12:36.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c337fe700 1 -- 172.21.15.19:0/1920721665 learned_addr learned my addr 172.21.15.19:0/1920721665 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:36.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c32ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c34103c00 0x7f4c3419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:37282/0 (socket says 172.21.15.19:37282) 2024-05-09T02:12:36.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c337fe700 1 -- 172.21.15.19:0/1920721665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c34103c00 msgr2=0x7f4c3419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:36.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c337fe700 1 --2- 172.21.15.19:0/1920721665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c34103c00 0x7f4c3419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c337fe700 1 -- 172.21.15.19:0/1920721665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c34107e00 msgr2=0x7f4c3419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:36.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c337fe700 1 --2- 172.21.15.19:0/1920721665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c34107e00 0x7f4c3419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c337fe700 1 -- 172.21.15.19:0/1920721665 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3419fc90 con 0x7f4c34108750 2024-05-09T02:12:36.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c32ffd700 1 --2- 172.21.15.19:0/1920721665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c34103c00 0x7f4c3419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:12:36.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c337fe700 1 --2- 172.21.15.19:0/1920721665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c34108750 0x7f4c3419f5f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00ef30 tx=0x7f4c2c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:36.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c389ac700 1 -- 172.21.15.19:0/1920721665 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0092e0 con 0x7f4c34108750 2024-05-09T02:12:36.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c399ae700 1 -- 172.21.15.19:0/1920721665 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3419ff20 con 0x7f4c34108750 2024-05-09T02:12:36.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c389ac700 1 -- 172.21.15.19:0/1920721665 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c2c004bb0 con 0x7f4c34108750 2024-05-09T02:12:36.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.777+0000 7f4c389ac700 1 -- 172.21.15.19:0/1920721665 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c004020 con 0x7f4c34108750 2024-05-09T02:12:36.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.778+0000 7f4c399ae700 1 -- 172.21.15.19:0/1920721665 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c341a78a0 con 0x7f4c34108750 2024-05-09T02:12:36.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.782+0000 7f4c389ac700 1 -- 172.21.15.19:0/1920721665 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f4c2c01b030 con 0x7f4c34108750 2024-05-09T02:12:36.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.784+0000 7f4c483c4b80 1 Processor -- start 2024-05-09T02:12:36.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.784+0000 7f4c483c4b80 1 -- start start 2024-05-09T02:12:36.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.785+0000 7f4c389ac700 1 --2- 172.21.15.19:0/1920721665 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f4c1c0b2b90 0x7f4c1c0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.785+0000 7f4c483c4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3adb160 0x5597f3b744f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.785+0000 7f4c483c4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f39f14a0 0x5597f3b74a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.785+0000 7f4c483c4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f3b6e6a0 0x5597f3b6ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.785+0000 7f4c483c4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5597f3a90ff0 con 0x5597f3adb160 2024-05-09T02:12:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.785+0000 7f4c483c4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5597f3a2f7f0 con 0x5597f3b6e6a0 2024-05-09T02:12:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.785+0000 7f4c483c4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5597f3a8cfb0 con 0x5597f39f14a0 2024-05-09T02:12:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.785+0000 7f4c327fc700 1 --2- 172.21.15.19:0/1920721665 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f4c1c0b2b90 0x7f4c1c0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.785+0000 7f4c389ac700 1 -- 172.21.15.19:0/1920721665 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f4c2c0e2370 con 0x7f4c34108750 2024-05-09T02:12:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.785+0000 7f4c327fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f39f14a0 0x5597f3b74a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.785+0000 7f4c327fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f39f14a0 0x5597f3b74a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:37292/0 (socket says 172.21.15.19:37292) 2024-05-09T02:12:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.785+0000 7f4c327fc700 1 -- 172.21.15.19:0/2805475554 learned_addr learned my addr 172.21.15.19:0/2805475554 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.785+0000 7f4c337fe700 1 --2- 172.21.15.19:0/2805475554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f3b6e6a0 0x5597f3b6ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.785+0000 7f4c32ffd700 1 --2- 172.21.15.19:0/2805475554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3adb160 0x5597f3b744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.786+0000 7f4c327fc700 1 -- 172.21.15.19:0/2805475554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f3b6e6a0 msgr2=0x5597f3b6ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.786+0000 7f4c327fc700 1 --2- 172.21.15.19:0/2805475554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f3b6e6a0 0x5597f3b6ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.786+0000 7f4c327fc700 1 -- 172.21.15.19:0/2805475554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3adb160 msgr2=0x5597f3b744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.786+0000 7f4c327fc700 1 --2- 172.21.15.19:0/2805475554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3adb160 0x5597f3b744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.786+0000 7f4c327fc700 1 -- 172.21.15.19:0/2805475554 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597f3a27370 con 0x5597f39f14a0 2024-05-09T02:12:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.786+0000 7f4c32ffd700 1 --2- 172.21.15.19:0/2805475554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3adb160 0x5597f3b744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:12:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.786+0000 7f4c327fc700 1 --2- 172.21.15.19:0/1920721665 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f4c1c0b2b90 0x7f4c1c0b50a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009fd0 tx=0x7f4c28009cf0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.786+0000 7f4c327fc700 1 --2- 172.21.15.19:0/2805475554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f39f14a0 0x5597f3b74a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4c28013ae0 tx=0x7f4c28021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e25c561caedfdf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.786+0000 7f4c19ffb700 1 -- 172.21.15.19:0/2805475554 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28013df0 con 0x5597f39f14a0 2024-05-09T02:12:36.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.786+0000 7f4c19ffb700 1 -- 172.21.15.19:0/2805475554 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c28004ce0 con 0x5597f39f14a0 2024-05-09T02:12:36.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.787+0000 7f4c19ffb700 1 -- 172.21.15.19:0/2805475554 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28005280 con 0x5597f39f14a0 2024-05-09T02:12:36.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.787+0000 7f4c483c4b80 1 -- 172.21.15.19:0/2805475554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f39f14a0 msgr2=0x5597f3b74a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:36.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.787+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/2805475554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f39f14a0 0x5597f3b74a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4c28013ae0 tx=0x7f4c28021000 comp rx=0 tx=0).stop 2024-05-09T02:12:36.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.788+0000 7f4c483c4b80 1 -- 172.21.15.19:0/2805475554 shutdown_connections 2024-05-09T02:12:36.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.788+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/2805475554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f3b6e6a0 0x5597f3b6ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.788+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/2805475554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3adb160 0x5597f3b744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.788+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/2805475554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f39f14a0 0x5597f3b74a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.788+0000 7f4c483c4b80 1 -- 172.21.15.19:0/2805475554 >> 172.21.15.19:0/2805475554 conn(0x5597f3a516a0 msgr2=0x5597f3b69400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:36.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.788+0000 7f4c483c4b80 1 -- 172.21.15.19:0/2805475554 shutdown_connections 2024-05-09T02:12:36.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.788+0000 7f4c483c4b80 1 -- 172.21.15.19:0/2805475554 wait complete. 2024-05-09T02:12:36.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.788+0000 7f4c483c4b80 1 -- start start 2024-05-09T02:12:36.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.789+0000 7f4c483c4b80 1 Processor -- start 2024-05-09T02:12:36.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.789+0000 7f4c483c4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f39f14a0 0x5597f3b78170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.790+0000 7f4c483c4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 0x5597f3b786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.790+0000 7f4c32ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f39f14a0 0x5597f3b78170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.790+0000 7f4c32ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f39f14a0 0x5597f3b78170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:48760/0 (socket says 172.21.15.19:48760) 2024-05-09T02:12:36.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.790+0000 7f4c32ffd700 1 -- 172.21.15.19:0/4152488763 learned_addr learned my addr 172.21.15.19:0/4152488763 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:36.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.790+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/4152488763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3b6e6a0 0x5597f3b78bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.790+0000 7f4c483c4b80 1 -- 172.21.15.19:0/4152488763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5597f390cba0 con 0x5597f3b6e6a0 2024-05-09T02:12:36.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.790+0000 7f4c483c4b80 1 -- 172.21.15.19:0/4152488763 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5597f3a4b150 con 0x5597f39f14a0 2024-05-09T02:12:36.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.790+0000 7f4c483c4b80 1 -- 172.21.15.19:0/4152488763 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5597f3b7b1d0 con 0x5597f3adb160 2024-05-09T02:12:36.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.790+0000 7f4c327fc700 1 --2- 172.21.15.19:0/4152488763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 0x5597f3b786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.791+0000 7f4c483c4b80 10 client.0 _ll_register_callbacks cb 0x5597f39026c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:36.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.791+0000 7f4c32ffd700 1 -- 172.21.15.19:0/4152488763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 msgr2=0x5597f3b786b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:36.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.791+0000 7f4c32ffd700 1 --2- 172.21.15.19:0/4152488763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 0x5597f3b786b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.791+0000 7f4c337fe700 1 --2- 172.21.15.19:0/4152488763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3b6e6a0 0x5597f3b78bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.791+0000 7f4c32ffd700 1 -- 172.21.15.19:0/4152488763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3b6e6a0 msgr2=0x5597f3b78bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:36.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.791+0000 7f4c32ffd700 1 --2- 172.21.15.19:0/4152488763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3b6e6a0 0x5597f3b78bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.791+0000 7f4c32ffd700 1 -- 172.21.15.19:0/4152488763 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597f390c490 con 0x5597f39f14a0 2024-05-09T02:12:36.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.791+0000 7f4c483c4b80 1 -- 172.21.15.19:0/4152488763 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c28004ce0 con 0x5597f39f14a0 2024-05-09T02:12:36.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.791+0000 7f4c337fe700 1 --2- 172.21.15.19:0/4152488763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3b6e6a0 0x5597f3b78bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:12:36.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.791+0000 7f4c32ffd700 1 --2- 172.21.15.19:0/4152488763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f39f14a0 0x5597f3b78170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4c24015000 tx=0x7f4c2400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=320f47e5ca363122 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:36.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.791+0000 7f4c12ffd700 10 client.25220 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:12:36.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.792+0000 7f4c12ffd700 1 -- 172.21.15.19:0/4152488763 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c240037c0 con 0x5597f39f14a0 2024-05-09T02:12:36.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.792+0000 7f4c12ffd700 1 -- 172.21.15.19:0/4152488763 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f4c24003960 con 0x5597f39f14a0 2024-05-09T02:12:36.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.792+0000 7f4c12ffd700 1 -- 172.21.15.19:0/4152488763 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c24003ba0 con 0x5597f39f14a0 2024-05-09T02:12:36.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.792+0000 7f4c483c4b80 10 client.25220 fetch_fsmap learned FSMap version 32 2024-05-09T02:12:36.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.792+0000 7f4c483c4b80 1 -- 172.21.15.19:0/4152488763 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=32}) -- 0x5597f3b85aa0 con 0x5597f39f14a0 2024-05-09T02:12:36.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.793+0000 7f4c12ffd700 1 -- 172.21.15.19:0/4152488763 <== mon.2 v2:172.21.15.59:3300/0 4 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f4c24024470 con 0x5597f39f14a0 2024-05-09T02:12:36.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.793+0000 7f4c483c4b80 10 client.25220 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T02:12:36.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.793+0000 7f4c483c4b80 10 client.25220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:12:36.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.793+0000 7f4c483c4b80 20 client.25220 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:36.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.793+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/4152488763 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5597f3b88310 0x5597f3b8a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.793+0000 7f4c483c4b80 4 client.25220 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:12:36.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.793+0000 7f4c483c4b80 1 -- 172.21.15.19:0/4152488763 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597f3a9e9d0 con 0x5597f3b88310 2024-05-09T02:12:36.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.793+0000 7f4c327fc700 1 --2- 172.21.15.19:0/4152488763 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5597f3b88310 0x5597f3b8a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.794+0000 7f4c12ffd700 10 client.25220 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:36.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.794+0000 7f4c327fc700 1 --2- 172.21.15.19:0/4152488763 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5597f3b88310 0x5597f3b8a7a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3193d0137834d64a server_cookie=e8968468c147806b in_seq=0 out_seq=0 2024-05-09T02:12:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.800+0000 7f4c12ffd700 1 -- 172.21.15.19:0/4152488763 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5597f3a9e9d0 con 0x5597f3b88310 2024-05-09T02:12:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.800+0000 7f4c12ffd700 10 client.25220 handle_command_reply: tid=0 2024-05-09T02:12:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.800+0000 7f4c483c4b80 1 client.25220 shutdown 2024-05-09T02:12:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.801+0000 7f4c483c4b80 1 -- 172.21.15.19:0/4152488763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f39f14a0 msgr2=0x5597f3b78170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.801+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/4152488763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f39f14a0 0x5597f3b78170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4c24015000 tx=0x7f4c2400b8d0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.801+0000 7f4c483c4b80 1 -- 172.21.15.19:0/4152488763 shutdown_connections 2024-05-09T02:12:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.801+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/4152488763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3b6e6a0 0x5597f3b78bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.801+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/4152488763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f39f14a0 0x5597f3b78170 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.801+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/4152488763 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5597f3b88310 0x5597f3b8a7a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.801+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/4152488763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 0x5597f3b786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.801+0000 7f4c483c4b80 1 -- 172.21.15.19:0/4152488763 >> 172.21.15.19:0/4152488763 conn(0x5597f3a516a0 msgr2=0x5597f3b691c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:36.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.801+0000 7f4c12ffd700 0 client.25220 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:36.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.801+0000 7f4c483c4b80 1 -- 172.21.15.19:0/4152488763 shutdown_connections 2024-05-09T02:12:36.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.802+0000 7f4c483c4b80 1 -- 172.21.15.19:0/4152488763 wait complete. 2024-05-09T02:12:36.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.802+0000 7f4c483c4b80 20 client.25220 trim_cache size 0 max 16384 2024-05-09T02:12:36.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.840+0000 7f4c483c4b80 1 Processor -- start 2024-05-09T02:12:36.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.840+0000 7f4c483c4b80 1 -- start start 2024-05-09T02:12:36.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.840+0000 7f4c483c4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f39f14a0 0x5597f3b77490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.840+0000 7f4c483c4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 0x5597f3b78a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.840+0000 7f4c483c4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f3b6e6a0 0x5597f3b83fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.840+0000 7f4c483c4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c341acba0 con 0x5597f39f14a0 2024-05-09T02:12:36.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.840+0000 7f4c483c4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c341ae770 con 0x5597f3b6e6a0 2024-05-09T02:12:36.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.840+0000 7f4c483c4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c341ad270 con 0x5597f3adb160 2024-05-09T02:12:36.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.840+0000 7f4c32ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f39f14a0 0x5597f3b77490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.840+0000 7f4c337fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f3b6e6a0 0x5597f3b83fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.840+0000 7f4c32ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f39f14a0 0x5597f3b77490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34260/0 (socket says 172.21.15.19:34260) 2024-05-09T02:12:36.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.841+0000 7f4c32ffd700 1 -- 172.21.15.19:0/2695905963 learned_addr learned my addr 172.21.15.19:0/2695905963 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:36.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.841+0000 7f4c327fc700 1 --2- 172.21.15.19:0/2695905963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 0x5597f3b78a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.841+0000 7f4c32ffd700 1 -- 172.21.15.19:0/2695905963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 msgr2=0x5597f3b78a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:36.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.841+0000 7f4c32ffd700 1 --2- 172.21.15.19:0/2695905963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 0x5597f3b78a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.841+0000 7f4c32ffd700 1 -- 172.21.15.19:0/2695905963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f3b6e6a0 msgr2=0x5597f3b83fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:36.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.841+0000 7f4c32ffd700 1 --2- 172.21.15.19:0/2695905963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f3b6e6a0 0x5597f3b83fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.841+0000 7f4c32ffd700 1 -- 172.21.15.19:0/2695905963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597f3b74050 con 0x5597f39f14a0 2024-05-09T02:12:36.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.841+0000 7f4c32ffd700 1 --2- 172.21.15.19:0/2695905963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f39f14a0 0x5597f3b77490 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4c240094e0 tx=0x7f4c2401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8ae8771f1dbf98 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:36.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.841+0000 7f4c197fa700 1 -- 172.21.15.19:0/2695905963 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24005500 con 0x5597f39f14a0 2024-05-09T02:12:36.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.841+0000 7f4c197fa700 1 -- 172.21.15.19:0/2695905963 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c28004ce0 con 0x5597f39f14a0 2024-05-09T02:12:36.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.841+0000 7f4c197fa700 1 -- 172.21.15.19:0/2695905963 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400f910 con 0x5597f39f14a0 2024-05-09T02:12:36.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.842+0000 7f4c483c4b80 1 -- 172.21.15.19:0/2695905963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f39f14a0 msgr2=0x5597f3b77490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:36.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.842+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/2695905963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f39f14a0 0x5597f3b77490 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4c240094e0 tx=0x7f4c2401e000 comp rx=0 tx=0).stop 2024-05-09T02:12:36.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.843+0000 7f4c483c4b80 1 -- 172.21.15.19:0/2695905963 shutdown_connections 2024-05-09T02:12:36.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.843+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/2695905963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f3b6e6a0 0x5597f3b83fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.843+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/2695905963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f39f14a0 0x5597f3b77490 secure :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4c240094e0 tx=0x7f4c2401e000 comp rx=0 tx=0).stop 2024-05-09T02:12:36.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.843+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/2695905963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 0x5597f3b78a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.843+0000 7f4c483c4b80 1 -- 172.21.15.19:0/2695905963 >> 172.21.15.19:0/2695905963 conn(0x5597f3a516a0 msgr2=0x5597f3b71e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:36.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.845+0000 7f4c483c4b80 1 -- 172.21.15.19:0/2695905963 shutdown_connections 2024-05-09T02:12:36.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.845+0000 7f4c483c4b80 1 -- 172.21.15.19:0/2695905963 wait complete. 2024-05-09T02:12:36.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.846+0000 7f4c483c4b80 1 -- start start 2024-05-09T02:12:36.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.846+0000 7f4c483c4b80 1 Processor -- start 2024-05-09T02:12:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.847+0000 7f4c483c4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 0x5597f3b8f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.847+0000 7f4c483c4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f3b6e6a0 0x5597f3b8fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.847+0000 7f4c32ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 0x5597f3b8f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.847+0000 7f4c32ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 0x5597f3b8f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:37322/0 (socket says 172.21.15.19:37322) 2024-05-09T02:12:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.847+0000 7f4c32ffd700 1 -- 172.21.15.19:0/72800269 learned_addr learned my addr 172.21.15.19:0/72800269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:12:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.848+0000 7f4c327fc700 1 --2- 172.21.15.19:0/72800269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f3b6e6a0 0x5597f3b8fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.848+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/72800269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3b90240 0x5597f3bf1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.848+0000 7f4c483c4b80 1 -- 172.21.15.19:0/72800269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c341ae010 con 0x5597f3b90240 2024-05-09T02:12:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.848+0000 7f4c483c4b80 1 -- 172.21.15.19:0/72800269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c341ad940 con 0x5597f3b6e6a0 2024-05-09T02:12:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.848+0000 7f4c483c4b80 1 -- 172.21.15.19:0/72800269 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c341aab40 con 0x5597f3adb160 2024-05-09T02:12:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.849+0000 7f4c483c4b80 10 client.0 _ll_register_callbacks cb 0x5597f39026c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:12:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.849+0000 7f4c32ffd700 1 -- 172.21.15.19:0/72800269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f3b6e6a0 msgr2=0x5597f3b8fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.849+0000 7f4c32ffd700 1 --2- 172.21.15.19:0/72800269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f3b6e6a0 0x5597f3b8fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.849+0000 7f4c32ffd700 1 -- 172.21.15.19:0/72800269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3b90240 msgr2=0x5597f3bf1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:12:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.849+0000 7f4c32ffd700 1 --2- 172.21.15.19:0/72800269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3b90240 0x5597f3bf1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.849+0000 7f4c32ffd700 1 -- 172.21.15.19:0/72800269 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597f3bf1ef0 con 0x5597f3adb160 2024-05-09T02:12:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.849+0000 7f4c483c4b80 1 -- 172.21.15.19:0/72800269 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c28004ce0 con 0x5597f3adb160 2024-05-09T02:12:36.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.851+0000 7f4c32ffd700 1 --2- 172.21.15.19:0/72800269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 0x5597f3b8f7c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c24005a80 tx=0x7f4c24024880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fd17b2b04954fc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:12:36.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.852+0000 7f4c12ffd700 10 client.25270 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:12:36.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.852+0000 7f4c12ffd700 1 -- 172.21.15.19:0/72800269 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c240037c0 con 0x5597f3adb160 2024-05-09T02:12:36.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.852+0000 7f4c12ffd700 1 -- 172.21.15.19:0/72800269 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f4c24003960 con 0x5597f3adb160 2024-05-09T02:12:36.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.852+0000 7f4c12ffd700 1 -- 172.21.15.19:0/72800269 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c2400f460 con 0x5597f3adb160 2024-05-09T02:12:36.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.853+0000 7f4c483c4b80 10 client.25270 fetch_fsmap learned FSMap version 32 2024-05-09T02:12:36.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.853+0000 7f4c483c4b80 1 -- 172.21.15.19:0/72800269 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=32}) -- 0x5597f3bf3000 con 0x5597f3adb160 2024-05-09T02:12:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.861+0000 7f4c12ffd700 1 -- 172.21.15.19:0/72800269 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f4c2400f600 con 0x5597f3adb160 2024-05-09T02:12:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.861+0000 7f4c483c4b80 10 client.25270 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T02:12:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.861+0000 7f4c483c4b80 10 client.25270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:12:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.861+0000 7f4c483c4b80 20 client.25270 populate_metadata read hostname 'smithi019' 2024-05-09T02:12:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.861+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/72800269 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5597f3bf3620 0x5597f3bf5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:12:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.861+0000 7f4c483c4b80 4 client.25270 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:12:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.862+0000 7f4c483c4b80 1 -- 172.21.15.19:0/72800269 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5597f3a98a70 con 0x5597f3bf3620 2024-05-09T02:12:36.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.866+0000 7f4c327fc700 1 --2- 172.21.15.19:0/72800269 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5597f3bf3620 0x5597f3bf5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:12:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.869+0000 7f4c327fc700 1 --2- 172.21.15.19:0/72800269 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5597f3bf3620 0x5597f3bf5a60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f1c3119ab47438d server_cookie=22f0cdd6d2f0eaa4 in_seq=0 out_seq=0 2024-05-09T02:12:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.869+0000 7f4c12ffd700 10 client.25270 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.870+0000 7f4c12ffd700 1 -- 172.21.15.19:0/72800269 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5597f3a98a70 con 0x5597f3bf3620 2024-05-09T02:12:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.870+0000 7f4c12ffd700 10 client.25270 handle_command_reply: tid=0 2024-05-09T02:12:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.870+0000 7f4c483c4b80 1 client.25270 shutdown 2024-05-09T02:12:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.870+0000 7f4c483c4b80 1 -- 172.21.15.19:0/72800269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 msgr2=0x5597f3b8f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.870+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/72800269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 0x5597f3b8f7c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c24005a80 tx=0x7f4c24024880 comp rx=0 tx=0).stop 2024-05-09T02:12:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.870+0000 7f4c483c4b80 1 -- 172.21.15.19:0/72800269 shutdown_connections 2024-05-09T02:12:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.870+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/72800269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f3b90240 0x5597f3bf1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.870+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/72800269 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5597f3bf3620 0x5597f3bf5a60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.871+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/72800269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597f3adb160 0x5597f3b8f7c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.871+0000 7f4c12ffd700 0 client.25270 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:12:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.871+0000 7f4c483c4b80 1 --2- 172.21.15.19:0/72800269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597f3b6e6a0 0x5597f3b8fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.871+0000 7f4c483c4b80 1 -- 172.21.15.19:0/72800269 >> 172.21.15.19:0/72800269 conn(0x5597f3a516a0 msgr2=0x5597f3b6d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.871+0000 7f4c483c4b80 1 -- 172.21.15.19:0/72800269 shutdown_connections 2024-05-09T02:12:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.872+0000 7f4c483c4b80 1 -- 172.21.15.19:0/72800269 wait complete. 2024-05-09T02:12:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.872+0000 7f4c483c4b80 20 client.25270 trim_cache size 0 max 16384 2024-05-09T02:12:36.875 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:12:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T02:12:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T02:12:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: "ac109e30-f302-4696-9654-c0961d606e78": { 2024-05-09T02:12:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T02:12:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "ac109e30-f302-4696-9654-c0961d606e78", 2024-05-09T02:12:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T02:12:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:12:36.876 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:12:36.876 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:12:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.875+0000 7f4c399ae700 1 -- 172.21.15.19:0/1920721665 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f4c1c0b2b90 msgr2=0x7f4c1c0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.875+0000 7f4c399ae700 1 --2- 172.21.15.19:0/1920721665 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f4c1c0b2b90 0x7f4c1c0b50a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009fd0 tx=0x7f4c28009cf0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.875+0000 7f4c399ae700 1 -- 172.21.15.19:0/1920721665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c34108750 msgr2=0x7f4c3419f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:12:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.875+0000 7f4c399ae700 1 --2- 172.21.15.19:0/1920721665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c34108750 0x7f4c3419f5f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00ef30 tx=0x7f4c2c00c3a0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.875+0000 7f4c399ae700 1 -- 172.21.15.19:0/1920721665 shutdown_connections 2024-05-09T02:12:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.875+0000 7f4c399ae700 1 --2- 172.21.15.19:0/1920721665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c34108750 0x7f4c3419f5f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.875+0000 7f4c399ae700 1 --2- 172.21.15.19:0/1920721665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c34103c00 0x7f4c3419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.875+0000 7f4c399ae700 1 --2- 172.21.15.19:0/1920721665 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f4c1c0b2b90 0x7f4c1c0b50a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.875+0000 7f4c399ae700 1 --2- 172.21.15.19:0/1920721665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c34107e00 0x7f4c3419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:12:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.875+0000 7f4c399ae700 1 -- 172.21.15.19:0/1920721665 >> 172.21.15.19:0/1920721665 conn(0x7f4c340fe6f0 msgr2=0x7f4c34102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:12:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.875+0000 7f4c399ae700 1 -- 172.21.15.19:0/1920721665 shutdown_connections 2024-05-09T02:12:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:12:36.876+0000 7f4c399ae700 1 -- 172.21.15.19:0/1920721665 wait complete. 2024-05-09T02:12:37.650 INFO:tasks.cephfs.filesystem:scrub status for tag:ac109e30-f302-4696-9654-c0961d606e78 - {'path': '/', 'tag': 'ac109e30-f302-4696-9654-c0961d606e78', 'options': 'recursive,force'} 2024-05-09T02:12:37.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 55.0 s, 3153.2 tps, lat 31.820 ms stddev 33.123 2024-05-09T02:12:38.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:38 smithi196 ceph-mon[34718]: pgmap v43: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.4 MiB/s wr, 624 op/s 2024-05-09T02:12:38.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:38 smithi019 ceph-mon[30151]: pgmap v43: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.4 MiB/s wr, 624 op/s 2024-05-09T02:12:38.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:38 smithi059 ceph-mon[34781]: pgmap v43: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.4 MiB/s wr, 624 op/s 2024-05-09T02:12:40.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:40 smithi196 ceph-mon[34718]: pgmap v44: 97 pgs: 97 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 10 MiB/s wr, 932 op/s 2024-05-09T02:12:40.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:40 smithi019 ceph-mon[30151]: pgmap v44: 97 pgs: 97 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 10 MiB/s wr, 932 op/s 2024-05-09T02:12:40.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:40 smithi059 ceph-mon[34781]: pgmap v44: 97 pgs: 97 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 10 MiB/s wr, 932 op/s 2024-05-09T02:12:41.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:41 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:12:41.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:41 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:12:41.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:41 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:12:41.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:41 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:12:41.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:41 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:12:41.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:41 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:12:41.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:41 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:12:41.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:41 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:12:41.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:41 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:12:41.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:41 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:12:41.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:41 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:12:41.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:41 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:12:42.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:42 smithi196 ceph-mon[34718]: pgmap v45: 97 pgs: 97 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 7.5 MiB/s wr, 632 op/s 2024-05-09T02:12:42.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:42 smithi019 ceph-mon[30151]: pgmap v45: 97 pgs: 97 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 7.5 MiB/s wr, 632 op/s 2024-05-09T02:12:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 60.0 s, 3446.0 tps, lat 29.036 ms stddev 30.652 2024-05-09T02:12:42.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:42 smithi059 ceph-mon[34781]: pgmap v45: 97 pgs: 97 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 7.5 MiB/s wr, 632 op/s 2024-05-09T02:12:44.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:44 smithi196 ceph-mon[34718]: pgmap v46: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 9.9 MiB/s wr, 780 op/s 2024-05-09T02:12:44.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:44 smithi019 ceph-mon[30151]: pgmap v46: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 9.9 MiB/s wr, 780 op/s 2024-05-09T02:12:44.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:44 smithi059 ceph-mon[34781]: pgmap v46: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 9.9 MiB/s wr, 780 op/s 2024-05-09T02:12:46.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:46 smithi196 ceph-mon[34718]: pgmap v47: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 10 MiB/s wr, 809 op/s 2024-05-09T02:12:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:46 smithi019 ceph-mon[30151]: pgmap v47: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 10 MiB/s wr, 809 op/s 2024-05-09T02:12:46.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:46 smithi059 ceph-mon[34781]: pgmap v47: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 10 MiB/s wr, 809 op/s 2024-05-09T02:12:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 65.0 s, 3563.6 tps, lat 28.039 ms stddev 26.683 2024-05-09T02:12:48.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:48 smithi196 ceph-mon[34718]: pgmap v48: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 7.8 MiB/s wr, 631 op/s 2024-05-09T02:12:48.574 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:48 smithi019 ceph-mon[30151]: pgmap v48: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 7.8 MiB/s wr, 631 op/s 2024-05-09T02:12:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:48 smithi059 ceph-mon[34781]: pgmap v48: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 7.8 MiB/s wr, 631 op/s 2024-05-09T02:12:50.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:50 smithi019 ceph-mon[30151]: pgmap v49: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 13 MiB/s wr, 967 op/s 2024-05-09T02:12:50.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:50 smithi019 ceph-mon[30151]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-09T02:12:50.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:50 smithi059 ceph-mon[34781]: pgmap v49: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 13 MiB/s wr, 967 op/s 2024-05-09T02:12:50.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:50 smithi059 ceph-mon[34781]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-09T02:12:50.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:50 smithi196 ceph-mon[34718]: pgmap v49: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 13 MiB/s wr, 967 op/s 2024-05-09T02:12:50.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:50 smithi196 ceph-mon[34718]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-09T02:12:52.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:52 smithi019 ceph-mon[30151]: pgmap v50: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 9.5 MiB/s wr, 659 op/s 2024-05-09T02:12:52.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:52 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:12:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 70.0 s, 3385.4 tps, lat 29.537 ms stddev 31.507 2024-05-09T02:12:52.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:52 smithi059 ceph-mon[34781]: pgmap v50: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 9.5 MiB/s wr, 659 op/s 2024-05-09T02:12:52.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:52 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:12:52.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:52 smithi196 ceph-mon[34718]: pgmap v50: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 9.5 MiB/s wr, 659 op/s 2024-05-09T02:12:52.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:52 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:12:54.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:54 smithi019 ceph-mon[30151]: pgmap v51: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 13 MiB/s wr, 925 op/s 2024-05-09T02:12:54.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:54 smithi059 ceph-mon[34781]: pgmap v51: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 13 MiB/s wr, 925 op/s 2024-05-09T02:12:54.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:54 smithi196 ceph-mon[34718]: pgmap v51: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 13 MiB/s wr, 925 op/s 2024-05-09T02:12:56.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:56 smithi019 ceph-mon[30151]: pgmap v52: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 12 MiB/s wr, 826 op/s 2024-05-09T02:12:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:56 smithi059 ceph-mon[34781]: pgmap v52: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 12 MiB/s wr, 826 op/s 2024-05-09T02:12:56.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:56 smithi196 ceph-mon[34718]: pgmap v52: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 12 MiB/s wr, 826 op/s 2024-05-09T02:12:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 75.0 s, 3342.6 tps, lat 29.857 ms stddev 34.145 2024-05-09T02:12:58.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:12:58 smithi019 ceph-mon[30151]: pgmap v53: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 9.2 MiB/s wr, 651 op/s 2024-05-09T02:12:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:12:58 smithi059 ceph-mon[34781]: pgmap v53: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 9.2 MiB/s wr, 651 op/s 2024-05-09T02:12:58.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:12:58 smithi196 ceph-mon[34718]: pgmap v53: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 9.2 MiB/s wr, 651 op/s 2024-05-09T02:13:00.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:00 smithi019 ceph-mon[30151]: pgmap v54: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 961 op/s 2024-05-09T02:13:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:00 smithi059 ceph-mon[34781]: pgmap v54: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 961 op/s 2024-05-09T02:13:00.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:00 smithi196 ceph-mon[34718]: pgmap v54: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 961 op/s 2024-05-09T02:13:02.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:02 smithi019 ceph-mon[30151]: pgmap v55: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.4 MiB/s wr, 625 op/s 2024-05-09T02:13:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 80.0 s, 2173.0 tps, lat 45.176 ms stddev 76.703 2024-05-09T02:13:02.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:02 smithi059 ceph-mon[34781]: pgmap v55: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.4 MiB/s wr, 625 op/s 2024-05-09T02:13:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:02 smithi196 ceph-mon[34718]: pgmap v55: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.4 MiB/s wr, 625 op/s 2024-05-09T02:13:04.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:04 smithi019 ceph-mon[30151]: pgmap v56: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.4 MiB/s wr, 625 op/s 2024-05-09T02:13:04.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:04 smithi059 ceph-mon[34781]: pgmap v56: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.4 MiB/s wr, 625 op/s 2024-05-09T02:13:04.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:04 smithi196 ceph-mon[34718]: pgmap v56: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.4 MiB/s wr, 625 op/s 2024-05-09T02:13:06.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:06 smithi019 ceph-mon[30151]: pgmap v57: 97 pgs: 97 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 MiB/s wr, 1.19k op/s 2024-05-09T02:13:06.595 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:06 smithi196 ceph-mon[34718]: pgmap v57: 97 pgs: 97 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 MiB/s wr, 1.19k op/s 2024-05-09T02:13:06.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:06 smithi059 ceph-mon[34781]: pgmap v57: 97 pgs: 97 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 MiB/s wr, 1.19k op/s 2024-05-09T02:13:07.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:13:07.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 85.0 s, 2310.6 tps, lat 44.077 ms stddev 78.534 2024-05-09T02:13:08.020 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:13:08.431 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:08 smithi019 ceph-mon[30151]: pgmap v58: 97 pgs: 97 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 MiB/s wr, 1.14k op/s 2024-05-09T02:13:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:08 smithi059 ceph-mon[34781]: pgmap v58: 97 pgs: 97 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 MiB/s wr, 1.14k op/s 2024-05-09T02:13:08.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:08 smithi196 ceph-mon[34718]: pgmap v58: 97 pgs: 97 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 MiB/s wr, 1.14k op/s 2024-05-09T02:13:10.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.036+0000 7fbebc929700 1 -- 172.21.15.19:0/3040499493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb81045d0 msgr2=0x7fbeb81049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:10.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.036+0000 7fbebc929700 1 --2- 172.21.15.19:0/3040499493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb81045d0 0x7fbeb81049e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbea80099c0 tx=0x7fbea8009cd0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.036+0000 7fbebc929700 1 -- 172.21.15.19:0/3040499493 shutdown_connections 2024-05-09T02:13:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.036+0000 7fbebc929700 1 --2- 172.21.15.19:0/3040499493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbeb8109850 0x7fbeb81111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.036+0000 7fbebc929700 1 --2- 172.21.15.19:0/3040499493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb81045d0 0x7fbeb81049e0 secure :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbea80099c0 tx=0x7fbea8009cd0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.036+0000 7fbebc929700 1 --2- 172.21.15.19:0/3040499493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeb8108f00 0x7fbeb8109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.036+0000 7fbebc929700 1 -- 172.21.15.19:0/3040499493 >> 172.21.15.19:0/3040499493 conn(0x7fbeb80ffea0 msgr2=0x7fbeb81022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.037+0000 7fbebc929700 1 -- 172.21.15.19:0/3040499493 shutdown_connections 2024-05-09T02:13:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.037+0000 7fbebc929700 1 -- 172.21.15.19:0/3040499493 wait complete. 2024-05-09T02:13:10.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.039+0000 7fbebc929700 1 Processor -- start 2024-05-09T02:13:10.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.039+0000 7fbebc929700 1 -- start start 2024-05-09T02:13:10.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.039+0000 7fbebc929700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbeb8108f00 0x7fbeb819afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.039+0000 7fbebc929700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeb8109850 0x7fbeb819b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.039+0000 7fbebc929700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb819f720 0x7fbeb819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.039+0000 7fbebc929700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8114480 con 0x7fbeb819f720 2024-05-09T02:13:10.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.039+0000 7fbebc929700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8114300 con 0x7fbeb8108f00 2024-05-09T02:13:10.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.039+0000 7fbebc929700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8114600 con 0x7fbeb8109850 2024-05-09T02:13:10.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.039+0000 7fbeb5d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeb8109850 0x7fbeb819b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbeb5d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeb8109850 0x7fbeb819b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:41442/0 (socket says 172.21.15.19:41442) 2024-05-09T02:13:10.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbeb6d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb819f720 0x7fbeb819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbeb5d9b700 1 -- 172.21.15.19:0/3143112184 learned_addr learned my addr 172.21.15.19:0/3143112184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:10.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbeb659c700 1 --2- 172.21.15.19:0/3143112184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbeb8108f00 0x7fbeb819afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbeb5d9b700 1 -- 172.21.15.19:0/3143112184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbeb8108f00 msgr2=0x7fbeb819afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:10.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbeb5d9b700 1 --2- 172.21.15.19:0/3143112184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbeb8108f00 0x7fbeb819afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbeb5d9b700 1 -- 172.21.15.19:0/3143112184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb819f720 msgr2=0x7fbeb819fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:10.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbeb5d9b700 1 --2- 172.21.15.19:0/3143112184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb819f720 0x7fbeb819fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbeb5d9b700 1 -- 172.21.15.19:0/3143112184 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb81a0270 con 0x7fbeb8109850 2024-05-09T02:13:10.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbeb659c700 1 --2- 172.21.15.19:0/3143112184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbeb8108f00 0x7fbeb819afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:13:10.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbeb5d9b700 1 --2- 172.21.15.19:0/3143112184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeb8109850 0x7fbeb819b550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00e970 tx=0x7fbeac00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:10.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbea77fe700 1 -- 172.21.15.19:0/3143112184 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac004a30 con 0x7fbeb8109850 2024-05-09T02:13:10.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbebc929700 1 -- 172.21.15.19:0/3143112184 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb81a0560 con 0x7fbeb8109850 2024-05-09T02:13:10.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbea77fe700 1 -- 172.21.15.19:0/3143112184 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeac00cc90 con 0x7fbeb8109850 2024-05-09T02:13:10.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbea77fe700 1 -- 172.21.15.19:0/3143112184 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac005610 con 0x7fbeb8109850 2024-05-09T02:13:10.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.040+0000 7fbebc929700 1 -- 172.21.15.19:0/3143112184 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb81a7d40 con 0x7fbeb8109850 2024-05-09T02:13:10.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.044+0000 7fbea77fe700 1 -- 172.21.15.19:0/3143112184 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbeac00c7e0 con 0x7fbeb8109850 2024-05-09T02:13:10.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.045+0000 7fbecb33fb80 1 Processor -- start 2024-05-09T02:13:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.045+0000 7fbecb33fb80 1 -- start start 2024-05-09T02:13:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.045+0000 7fbecb33fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d438160 0x55ed7d4d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.045+0000 7fbecb33fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.045+0000 7fbea77fe700 1 --2- 172.21.15.19:0/3143112184 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fbea00b29f0 0x7fbea00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.045+0000 7fbecb33fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4cb6a0 0x55ed7d4cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.045+0000 7fbecb33fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ed7d3edff0 con 0x55ed7d34e4a0 2024-05-09T02:13:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.046+0000 7fbecb33fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ed7d38c7f0 con 0x55ed7d4cb6a0 2024-05-09T02:13:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.046+0000 7fbecb33fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ed7d3e9fb0 con 0x55ed7d438160 2024-05-09T02:13:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.046+0000 7fbeb6d9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d438160 0x55ed7d4d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.046+0000 7fbeb6d9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d438160 0x55ed7d4d14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:41452/0 (socket says 172.21.15.19:41452) 2024-05-09T02:13:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.046+0000 7fbeb6d9d700 1 -- 172.21.15.19:0/1983885294 learned_addr learned my addr 172.21.15.19:0/1983885294 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:10.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.046+0000 7fbeb5d9b700 1 --2- 172.21.15.19:0/3143112184 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fbea00b29f0 0x7fbea00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.046+0000 7fbeb659c700 1 --2- 172.21.15.19:0/1983885294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.046+0000 7fbea77fe700 1 -- 172.21.15.19:0/3143112184 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fbeac0e2840 con 0x7fbeb8109850 2024-05-09T02:13:10.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.046+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/1983885294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4cb6a0 0x55ed7d4cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.046+0000 7fbeb6d9d700 1 -- 172.21.15.19:0/1983885294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4cb6a0 msgr2=0x55ed7d4cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:10.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.046+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/1983885294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4cb6a0 0x55ed7d4cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.046+0000 7fbeb6d9d700 1 -- 172.21.15.19:0/1983885294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d34e4a0 msgr2=0x55ed7d4d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:10.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.046+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/1983885294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.046+0000 7fbeb6d9d700 1 -- 172.21.15.19:0/1983885294 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed7d384370 con 0x55ed7d438160 2024-05-09T02:13:10.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.047+0000 7fbeb659c700 1 --2- 172.21.15.19:0/1983885294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:13:10.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.047+0000 7fbeb5d9b700 1 --2- 172.21.15.19:0/3143112184 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fbea00b29f0 0x7fbea00b4f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0a6d30 tx=0x7fbeac06efb0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:10.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.047+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/1983885294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d438160 0x55ed7d4d14f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009ec0 tx=0x7fbeb0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff48efb79dc44b2f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:10.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.048+0000 7fbea4ff9700 1 -- 172.21.15.19:0/1983885294 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0009310 con 0x55ed7d438160 2024-05-09T02:13:10.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.048+0000 7fbea4ff9700 1 -- 172.21.15.19:0/1983885294 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeb00158f0 con 0x55ed7d438160 2024-05-09T02:13:10.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.048+0000 7fbea4ff9700 1 -- 172.21.15.19:0/1983885294 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0004280 con 0x55ed7d438160 2024-05-09T02:13:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.050+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1983885294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d438160 msgr2=0x55ed7d4d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.050+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/1983885294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d438160 0x55ed7d4d14f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009ec0 tx=0x7fbeb0009920 comp rx=0 tx=0).stop 2024-05-09T02:13:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.050+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1983885294 shutdown_connections 2024-05-09T02:13:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.050+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/1983885294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4cb6a0 0x55ed7d4cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.050+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/1983885294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d438160 0x55ed7d4d14f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.050+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/1983885294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.050+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1983885294 >> 172.21.15.19:0/1983885294 conn(0x55ed7d3ae6a0 msgr2=0x55ed7d4c6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.050+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1983885294 shutdown_connections 2024-05-09T02:13:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.050+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1983885294 wait complete. 2024-05-09T02:13:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.050+0000 7fbecb33fb80 1 -- start start 2024-05-09T02:13:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.050+0000 7fbecb33fb80 1 Processor -- start 2024-05-09T02:13:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbecb33fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4d51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbecb33fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d438160 0x55ed7d4d56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbecb33fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d4cb6a0 0x55ed7d4d5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbecb33fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ed7d269ba0 con 0x55ed7d438160 2024-05-09T02:13:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbecb33fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ed7d3a8150 con 0x55ed7d34e4a0 2024-05-09T02:13:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbecb33fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ed7d4d81d0 con 0x55ed7d4cb6a0 2024-05-09T02:13:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbecb33fb80 10 client.0 _ll_register_callbacks cb 0x55ed7d25f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbeb659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d438160 0x55ed7d4d56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbeb659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d438160 0x55ed7d4d56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38354/0 (socket says 172.21.15.19:38354) 2024-05-09T02:13:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbeb659c700 1 -- 172.21.15.19:0/1716584215 learned_addr learned my addr 172.21.15.19:0/1716584215 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/1716584215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d4cb6a0 0x55ed7d4d5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/1716584215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4d51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbeb659c700 1 -- 172.21.15.19:0/1716584215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d4cb6a0 msgr2=0x55ed7d4d5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbeb659c700 1 --2- 172.21.15.19:0/1716584215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d4cb6a0 0x55ed7d4d5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbeb659c700 1 -- 172.21.15.19:0/1716584215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d34e4a0 msgr2=0x55ed7d4d51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.051+0000 7fbeb659c700 1 --2- 172.21.15.19:0/1716584215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4d51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.052+0000 7fbeb659c700 1 -- 172.21.15.19:0/1716584215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7d269490 con 0x55ed7d438160 2024-05-09T02:13:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.052+0000 7fbeb659c700 1 --2- 172.21.15.19:0/1716584215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d438160 0x55ed7d4d56e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8104270 tx=0x7fbea800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1471e82fb2be0480 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.052+0000 7fbe9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:13:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.052+0000 7fbe9dffb700 1 -- 172.21.15.19:0/1716584215 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8005610 con 0x55ed7d438160 2024-05-09T02:13:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.052+0000 7fbe9dffb700 1 -- 172.21.15.19:0/1716584215 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fbea80037c0 con 0x55ed7d438160 2024-05-09T02:13:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.053+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1716584215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeb00158f0 con 0x55ed7d438160 2024-05-09T02:13:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.053+0000 7fbe9dffb700 1 -- 172.21.15.19:0/1716584215 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbea80039c0 con 0x55ed7d438160 2024-05-09T02:13:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.053+0000 7fbecb33fb80 10 client.15552 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.053+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1716584215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ed7d4e6dc0 con 0x55ed7d438160 2024-05-09T02:13:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.053+0000 7fbe9dffb700 1 -- 172.21.15.19:0/1716584215 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea8028070 con 0x55ed7d438160 2024-05-09T02:13:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.053+0000 7fbe9dffb700 1 -- 172.21.15.19:0/1716584215 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fbea800fc90 con 0x55ed7d438160 2024-05-09T02:13:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.054+0000 7fbecb33fb80 10 client.15552 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.054+0000 7fbecb33fb80 10 client.15552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:13:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.054+0000 7fbecb33fb80 20 client.15552 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.054+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/1716584215 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55ed7d4e9400 0x55ed7d4eb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.054+0000 7fbecb33fb80 4 client.15552 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:13:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.054+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1716584215 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbeb00158f0 con 0x55ed7d4e9400 2024-05-09T02:13:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.054+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/1716584215 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55ed7d4e9400 0x55ed7d4eb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.054+0000 7fbe9dffb700 10 client.15552 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.054+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/1716584215 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55ed7d4e9400 0x55ed7d4eb820 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=259cd5032056c5bf server_cookie=ec41ec1e86e5a0cb in_seq=0 out_seq=0 2024-05-09T02:13:10.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.055+0000 7fbe9dffb700 1 -- 172.21.15.19:0/1716584215 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fbeb00158f0 con 0x55ed7d4e9400 2024-05-09T02:13:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.055+0000 7fbe9dffb700 10 client.15552 handle_command_reply: tid=0 2024-05-09T02:13:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.055+0000 7fbecb33fb80 1 client.15552 shutdown 2024-05-09T02:13:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.055+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1716584215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d438160 msgr2=0x55ed7d4d56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.055+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/1716584215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d438160 0x55ed7d4d56e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8104270 tx=0x7fbea800b8b0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.055+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1716584215 shutdown_connections 2024-05-09T02:13:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.055+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/1716584215 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55ed7d4e9400 0x55ed7d4eb820 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.056+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/1716584215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d4cb6a0 0x55ed7d4d5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.056+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/1716584215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4d51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.056+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/1716584215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d438160 0x55ed7d4d56e0 secure :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8104270 tx=0x7fbea800b8b0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.056+0000 7fbe9dffb700 0 client.15552 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.056+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1716584215 >> 172.21.15.19:0/1716584215 conn(0x55ed7d3ae6a0 msgr2=0x55ed7d4c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.056+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1716584215 shutdown_connections 2024-05-09T02:13:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.056+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1716584215 wait complete. 2024-05-09T02:13:10.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.061+0000 7fbecb33fb80 20 client.15552 trim_cache size 0 max 16384 2024-05-09T02:13:10.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.116+0000 7fbecb33fb80 1 Processor -- start 2024-05-09T02:13:10.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.116+0000 7fbecb33fb80 1 -- start start 2024-05-09T02:13:10.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.116+0000 7fbecb33fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.117+0000 7fbecb33fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d4cb6a0 0x55ed7d4e10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.117+0000 7fbecb33fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4e15e0 0x55ed7d4f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.117+0000 7fbecb33fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81ad390 con 0x55ed7d34e4a0 2024-05-09T02:13:10.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.117+0000 7fbecb33fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81aef90 con 0x55ed7d4e15e0 2024-05-09T02:13:10.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.117+0000 7fbecb33fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81ada60 con 0x55ed7d4cb6a0 2024-05-09T02:13:10.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.117+0000 7fbeb6d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4e15e0 0x55ed7d4f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.117+0000 7fbeb6d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4e15e0 0x55ed7d4f0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:60882/0 (socket says 172.21.15.19:60882) 2024-05-09T02:13:10.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.117+0000 7fbeb6d9d700 1 -- 172.21.15.19:0/1748660951 learned_addr learned my addr 172.21.15.19:0/1748660951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:10.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.118+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/1748660951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.118+0000 7fbeb6d9d700 1 -- 172.21.15.19:0/1748660951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d4cb6a0 msgr2=0x55ed7d4e10a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:13:10.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.118+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/1748660951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d4cb6a0 0x55ed7d4e10a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.118+0000 7fbeb6d9d700 1 -- 172.21.15.19:0/1748660951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d34e4a0 msgr2=0x55ed7d4d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:10.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.118+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/1748660951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.118+0000 7fbeb6d9d700 1 -- 172.21.15.19:0/1748660951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed7d4bff60 con 0x55ed7d4e15e0 2024-05-09T02:13:10.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.118+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/1748660951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4e15e0 0x55ed7d4f0fe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00093e0 tx=0x7fbeb0009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6402557e1a94983 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:10.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.119+0000 7fbe9ffff700 1 -- 172.21.15.19:0/1748660951 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0003b30 con 0x55ed7d4e15e0 2024-05-09T02:13:10.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.119+0000 7fbe9ffff700 1 -- 172.21.15.19:0/1748660951 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeb0015da0 con 0x55ed7d4e15e0 2024-05-09T02:13:10.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.119+0000 7fbe9ffff700 1 -- 172.21.15.19:0/1748660951 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb00056c0 con 0x55ed7d4e15e0 2024-05-09T02:13:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.120+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1748660951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4e15e0 msgr2=0x55ed7d4f0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.120+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/1748660951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4e15e0 0x55ed7d4f0fe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00093e0 tx=0x7fbeb0009f30 comp rx=0 tx=0).stop 2024-05-09T02:13:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.120+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1748660951 shutdown_connections 2024-05-09T02:13:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.120+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/1748660951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4e15e0 0x55ed7d4f0fe0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.120+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/1748660951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.120+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/1748660951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d4cb6a0 0x55ed7d4e10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.120+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1748660951 >> 172.21.15.19:0/1748660951 conn(0x55ed7d3ae6a0 msgr2=0x55ed7d4d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.120+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1748660951 shutdown_connections 2024-05-09T02:13:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.120+0000 7fbecb33fb80 1 -- 172.21.15.19:0/1748660951 wait complete. 2024-05-09T02:13:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.120+0000 7fbecb33fb80 1 -- start start 2024-05-09T02:13:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.120+0000 7fbecb33fb80 1 Processor -- start 2024-05-09T02:13:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.121+0000 7fbecb33fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4e10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.121+0000 7fbecb33fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d4cb6a0 0x55ed7d4dd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.121+0000 7fbeb6d9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4e10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.121+0000 7fbeb6d9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4e10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:41470/0 (socket says 172.21.15.19:41470) 2024-05-09T02:13:10.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.121+0000 7fbeb6d9d700 1 -- 172.21.15.19:0/2197761061 learned_addr learned my addr 172.21.15.19:0/2197761061 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:10.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.121+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/2197761061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4e15e0 0x55ed7d4dd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.121+0000 7fbecb33fb80 1 -- 172.21.15.19:0/2197761061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81ae800 con 0x55ed7d4cb6a0 2024-05-09T02:13:10.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.121+0000 7fbecb33fb80 1 -- 172.21.15.19:0/2197761061 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81ae130 con 0x55ed7d4e15e0 2024-05-09T02:13:10.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.121+0000 7fbecb33fb80 1 -- 172.21.15.19:0/2197761061 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81ab330 con 0x55ed7d34e4a0 2024-05-09T02:13:10.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.122+0000 7fbeb659c700 1 --2- 172.21.15.19:0/2197761061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d4cb6a0 0x55ed7d4dd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.122+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/2197761061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4e15e0 0x55ed7d4dd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.122+0000 7fbeb6d9d700 1 -- 172.21.15.19:0/2197761061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4e15e0 msgr2=0x55ed7d4dd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:10.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.122+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/2197761061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4e15e0 0x55ed7d4dd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.122+0000 7fbeb6d9d700 1 -- 172.21.15.19:0/2197761061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d4cb6a0 msgr2=0x55ed7d4dd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.122+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/2197761061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d4cb6a0 0x55ed7d4dd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.122+0000 7fbeb6d9d700 1 -- 172.21.15.19:0/2197761061 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7d4e40b0 con 0x55ed7d34e4a0 2024-05-09T02:13:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.122+0000 7fbeb6d9d700 1 --2- 172.21.15.19:0/2197761061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4e10a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009fa0 tx=0x7fbeb0013b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=487909d06303d8b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.122+0000 7fbe9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:13:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.122+0000 7fbecb33fb80 10 client.0 _ll_register_callbacks cb 0x55ed7d25f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.122+0000 7fbecb33fb80 1 -- 172.21.15.19:0/2197761061 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeb0015da0 con 0x55ed7d34e4a0 2024-05-09T02:13:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.123+0000 7fbe9dffb700 1 -- 172.21.15.19:0/2197761061 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0005540 con 0x55ed7d34e4a0 2024-05-09T02:13:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.123+0000 7fbe9dffb700 1 -- 172.21.15.19:0/2197761061 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fbeb00056e0 con 0x55ed7d34e4a0 2024-05-09T02:13:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.123+0000 7fbe9dffb700 1 -- 172.21.15.19:0/2197761061 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbeb0005960 con 0x55ed7d34e4a0 2024-05-09T02:13:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.123+0000 7fbecb33fb80 10 client.25285 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.123+0000 7fbecb33fb80 1 -- 172.21.15.19:0/2197761061 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55ed7d569390 con 0x55ed7d34e4a0 2024-05-09T02:13:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.123+0000 7fbe9dffb700 1 -- 172.21.15.19:0/2197761061 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fbeb0007500 con 0x55ed7d34e4a0 2024-05-09T02:13:10.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.123+0000 7fbecb33fb80 10 client.25285 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:10.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.123+0000 7fbecb33fb80 10 client.25285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:13:10.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.123+0000 7fbecb33fb80 20 client.25285 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:10.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.123+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/2197761061 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55ed7d569790 0x55ed7d56bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:10.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.123+0000 7fbecb33fb80 4 client.25285 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:13:10.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.123+0000 7fbecb33fb80 1 -- 172.21.15.19:0/2197761061 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fbeb0015da0 con 0x55ed7d569790 2024-05-09T02:13:10.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.124+0000 7fbeb659c700 1 --2- 172.21.15.19:0/2197761061 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55ed7d569790 0x55ed7d56bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:10.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.125+0000 7fbeb659c700 1 --2- 172.21.15.19:0/2197761061 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55ed7d569790 0x55ed7d56bc40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6acabafe9c81d91 server_cookie=8ec9d0e48c3dad8f in_seq=0 out_seq=0 2024-05-09T02:13:10.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.125+0000 7fbe9dffb700 10 client.25285 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:10.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.125+0000 7fbe9dffb700 1 -- 172.21.15.19:0/2197761061 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fbeb0015da0 con 0x55ed7d569790 2024-05-09T02:13:10.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.125+0000 7fbe9dffb700 10 client.25285 handle_command_reply: tid=0 2024-05-09T02:13:10.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.125+0000 7fbecb33fb80 1 client.25285 shutdown 2024-05-09T02:13:10.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.125+0000 7fbecb33fb80 1 -- 172.21.15.19:0/2197761061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d34e4a0 msgr2=0x55ed7d4e10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:10.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.125+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/2197761061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4e10a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009fa0 tx=0x7fbeb0013b10 comp rx=0 tx=0).stop 2024-05-09T02:13:10.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.126+0000 7fbecb33fb80 1 -- 172.21.15.19:0/2197761061 shutdown_connections 2024-05-09T02:13:10.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.126+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/2197761061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed7d4e15e0 0x55ed7d4dd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.126+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/2197761061 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55ed7d569790 0x55ed7d56bc40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.126+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/2197761061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed7d34e4a0 0x55ed7d4e10a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.126+0000 7fbecb33fb80 1 --2- 172.21.15.19:0/2197761061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed7d4cb6a0 0x55ed7d4dd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.126+0000 7fbe9dffb700 0 client.25285 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:10.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.126+0000 7fbecb33fb80 1 -- 172.21.15.19:0/2197761061 >> 172.21.15.19:0/2197761061 conn(0x55ed7d3ae6a0 msgr2=0x55ed7d4cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:10.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.126+0000 7fbecb33fb80 1 -- 172.21.15.19:0/2197761061 shutdown_connections 2024-05-09T02:13:10.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.126+0000 7fbecb33fb80 1 -- 172.21.15.19:0/2197761061 wait complete. 2024-05-09T02:13:10.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.127+0000 7fbecb33fb80 20 client.25285 trim_cache size 0 max 16384 2024-05-09T02:13:10.131 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:13:10.132 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T02:13:10.132 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T02:13:10.132 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:13:10.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.127+0000 7fbebc929700 1 -- 172.21.15.19:0/3143112184 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fbea00b29f0 msgr2=0x7fbea00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:10.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.127+0000 7fbebc929700 1 --2- 172.21.15.19:0/3143112184 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fbea00b29f0 0x7fbea00b4f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0a6d30 tx=0x7fbeac06efb0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.127+0000 7fbebc929700 1 -- 172.21.15.19:0/3143112184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeb8109850 msgr2=0x7fbeb819b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:10.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.127+0000 7fbebc929700 1 --2- 172.21.15.19:0/3143112184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeb8109850 0x7fbeb819b550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00e970 tx=0x7fbeac00ec80 comp rx=0 tx=0).stop 2024-05-09T02:13:10.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.127+0000 7fbebc929700 1 -- 172.21.15.19:0/3143112184 shutdown_connections 2024-05-09T02:13:10.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.127+0000 7fbebc929700 1 --2- 172.21.15.19:0/3143112184 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fbea00b29f0 0x7fbea00b4f00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.127+0000 7fbebc929700 1 --2- 172.21.15.19:0/3143112184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb819f720 0x7fbeb819fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.127+0000 7fbebc929700 1 --2- 172.21.15.19:0/3143112184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbeb8108f00 0x7fbeb819afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.127+0000 7fbebc929700 1 --2- 172.21.15.19:0/3143112184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeb8109850 0x7fbeb819b550 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:10.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.127+0000 7fbebc929700 1 -- 172.21.15.19:0/3143112184 >> 172.21.15.19:0/3143112184 conn(0x7fbeb80ffea0 msgr2=0x7fbeb8102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:10.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.127+0000 7fbebc929700 1 -- 172.21.15.19:0/3143112184 shutdown_connections 2024-05-09T02:13:10.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:10.127+0000 7fbebc929700 1 -- 172.21.15.19:0/3143112184 wait complete. 2024-05-09T02:13:10.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:10 smithi019 ceph-mon[30151]: pgmap v59: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 MiB/s wr, 1.69k op/s 2024-05-09T02:13:10.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:10 smithi059 ceph-mon[34781]: pgmap v59: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 MiB/s wr, 1.69k op/s 2024-05-09T02:13:10.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:10 smithi196 ceph-mon[34718]: pgmap v59: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 MiB/s wr, 1.69k op/s 2024-05-09T02:13:11.459 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T02:13:11.460 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:13:11.843 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:13:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 90.0 s, 3535.1 tps, lat 28.329 ms stddev 30.205 2024-05-09T02:13:12.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:12 smithi059 ceph-mon[34781]: pgmap v60: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 MiB/s wr, 1.38k op/s 2024-05-09T02:13:12.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:12 smithi196 ceph-mon[34718]: pgmap v60: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 MiB/s wr, 1.38k op/s 2024-05-09T02:13:12.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:12 smithi019 ceph-mon[30151]: pgmap v60: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 MiB/s wr, 1.38k op/s 2024-05-09T02:13:14.488 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:14 smithi019 ceph-mon[30151]: pgmap v61: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 MiB/s wr, 1.38k op/s 2024-05-09T02:13:14.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:14 smithi059 ceph-mon[34781]: pgmap v61: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 MiB/s wr, 1.38k op/s 2024-05-09T02:13:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.765+0000 7fa84ddf5700 1 -- 172.21.15.19:0/168526829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8481045d0 msgr2=0x7fa8481049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.765+0000 7fa84ddf5700 1 --2- 172.21.15.19:0/168526829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8481045d0 0x7fa8481049e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0099c0 tx=0x7fa83c009cd0 comp rx=0 tx=0).stop 2024-05-09T02:13:14.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.767+0000 7fa84ddf5700 1 -- 172.21.15.19:0/168526829 shutdown_connections 2024-05-09T02:13:14.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.767+0000 7fa84ddf5700 1 --2- 172.21.15.19:0/168526829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa848109850 0x7fa8481111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:14.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.767+0000 7fa84ddf5700 1 --2- 172.21.15.19:0/168526829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8481045d0 0x7fa8481049e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:14.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.767+0000 7fa84ddf5700 1 --2- 172.21.15.19:0/168526829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa848108f00 0x7fa848109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:14.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.767+0000 7fa84ddf5700 1 -- 172.21.15.19:0/168526829 >> 172.21.15.19:0/168526829 conn(0x7fa8480ffe80 msgr2=0x7fa8481022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:14.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.768+0000 7fa84ddf5700 1 -- 172.21.15.19:0/168526829 shutdown_connections 2024-05-09T02:13:14.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.768+0000 7fa84ddf5700 1 -- 172.21.15.19:0/168526829 wait complete. 2024-05-09T02:13:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.770+0000 7fa84ddf5700 1 Processor -- start 2024-05-09T02:13:14.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.771+0000 7fa84ddf5700 1 -- start start 2024-05-09T02:13:14.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.771+0000 7fa84ddf5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8481045d0 0x7fa84810e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:14.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.771+0000 7fa84ddf5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa848108f00 0x7fa84810e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:14.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.771+0000 7fa84ddf5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa848109850 0x7fa84810ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:14.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.771+0000 7fa84ddf5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa848078350 con 0x7fa848109850 2024-05-09T02:13:14.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.771+0000 7fa84ddf5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa8480781d0 con 0x7fa848108f00 2024-05-09T02:13:14.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.771+0000 7fa84ddf5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8480784d0 con 0x7fa8481045d0 2024-05-09T02:13:14.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.771+0000 7fa8477fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8481045d0 0x7fa84810e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:14.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.771+0000 7fa846ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa848108f00 0x7fa84810e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:14.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.771+0000 7fa8477fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8481045d0 0x7fa84810e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:41478/0 (socket says 172.21.15.19:41478) 2024-05-09T02:13:14.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.771+0000 7fa8477fe700 1 -- 172.21.15.19:0/363884557 learned_addr learned my addr 172.21.15.19:0/363884557 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.772+0000 7fa846ffd700 1 -- 172.21.15.19:0/363884557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8481045d0 msgr2=0x7fa84810e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.772+0000 7fa846ffd700 1 --2- 172.21.15.19:0/363884557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8481045d0 0x7fa84810e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.772+0000 7fa846ffd700 1 -- 172.21.15.19:0/363884557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa848109850 msgr2=0x7fa84810ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.772+0000 7fa846ffd700 1 --2- 172.21.15.19:0/363884557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa848109850 0x7fa84810ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.772+0000 7fa846ffd700 1 -- 172.21.15.19:0/363884557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8481ad6d0 con 0x7fa848108f00 2024-05-09T02:13:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.773+0000 7fa846ffd700 1 --2- 172.21.15.19:0/363884557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa848108f00 0x7fa84810e8d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa84000bd40 tx=0x7fa84000bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.773+0000 7fa844ff9700 1 -- 172.21.15.19:0/363884557 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840019070 con 0x7fa848108f00 2024-05-09T02:13:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.773+0000 7fa84ddf5700 1 -- 172.21.15.19:0/363884557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8481ad8a0 con 0x7fa848108f00 2024-05-09T02:13:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.773+0000 7fa84ddf5700 1 -- 172.21.15.19:0/363884557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8481ade30 con 0x7fa848108f00 2024-05-09T02:13:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.773+0000 7fa844ff9700 1 -- 172.21.15.19:0/363884557 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa84000cc90 con 0x7fa848108f00 2024-05-09T02:13:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.773+0000 7fa844ff9700 1 -- 172.21.15.19:0/363884557 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8400125a0 con 0x7fa848108f00 2024-05-09T02:13:14.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.775+0000 7fa84ddf5700 1 -- 172.21.15.19:0/363884557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa834005330 con 0x7fa848108f00 2024-05-09T02:13:14.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.777+0000 7fa844ff9700 1 -- 172.21.15.19:0/363884557 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa84001c070 con 0x7fa848108f00 2024-05-09T02:13:14.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.778+0000 7fa844ff9700 1 --2- 172.21.15.19:0/363884557 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8300b2aa0 0x7fa8300b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:14.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.779+0000 7fa8477fe700 1 --2- 172.21.15.19:0/363884557 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8300b2aa0 0x7fa8300b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:14.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.779+0000 7fa844ff9700 1 -- 172.21.15.19:0/363884557 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fa8400e19c0 con 0x7fa848108f00 2024-05-09T02:13:14.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.780+0000 7fa8477fe700 1 --2- 172.21.15.19:0/363884557 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8300b2aa0 0x7fa8300b4fb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa83c003800 tx=0x7fa83c0095f0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:14.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:14.784+0000 7fa844ff9700 1 -- 172.21.15.19:0/363884557 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa8400e3e20 con 0x7fa848108f00 2024-05-09T02:13:14.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:14 smithi196 ceph-mon[34718]: pgmap v61: 97 pgs: 97 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 MiB/s wr, 1.38k op/s 2024-05-09T02:13:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:15.177+0000 7fa84ddf5700 1 -- 172.21.15.19:0/363884557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8340058f0 con 0x7fa848108f00 2024-05-09T02:13:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:15.178+0000 7fa844ff9700 1 -- 172.21.15.19:0/363884557 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11924 (secure 0 0 0) 0x7fa8400a94b0 con 0x7fa848108f00 2024-05-09T02:13:15.181 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:13:15.182 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:12:50.723377+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24463,24484,24473,24455,24457]},"id":1}]} 2024-05-09T02:13:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:15.184+0000 7fa84ddf5700 1 -- 172.21.15.19:0/363884557 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8300b2aa0 msgr2=0x7fa8300b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:15.184+0000 7fa84ddf5700 1 --2- 172.21.15.19:0/363884557 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8300b2aa0 0x7fa8300b4fb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa83c003800 tx=0x7fa83c0095f0 comp rx=0 tx=0).stop 2024-05-09T02:13:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:15.184+0000 7fa84ddf5700 1 -- 172.21.15.19:0/363884557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa848108f00 msgr2=0x7fa84810e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:15.184+0000 7fa84ddf5700 1 --2- 172.21.15.19:0/363884557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa848108f00 0x7fa84810e8d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa84000bd40 tx=0x7fa84000bd70 comp rx=0 tx=0).stop 2024-05-09T02:13:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:15.184+0000 7fa84ddf5700 1 -- 172.21.15.19:0/363884557 shutdown_connections 2024-05-09T02:13:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:15.184+0000 7fa84ddf5700 1 --2- 172.21.15.19:0/363884557 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8300b2aa0 0x7fa8300b4fb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:15.184+0000 7fa84ddf5700 1 --2- 172.21.15.19:0/363884557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa848109850 0x7fa84810ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:15.184+0000 7fa84ddf5700 1 --2- 172.21.15.19:0/363884557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8481045d0 0x7fa84810e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:15.184+0000 7fa84ddf5700 1 --2- 172.21.15.19:0/363884557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa848108f00 0x7fa84810e8d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:15.184+0000 7fa84ddf5700 1 -- 172.21.15.19:0/363884557 >> 172.21.15.19:0/363884557 conn(0x7fa8480ffe80 msgr2=0x7fa848107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:15.185+0000 7fa84ddf5700 1 -- 172.21.15.19:0/363884557 shutdown_connections 2024-05-09T02:13:15.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:15.186+0000 7fa84ddf5700 1 -- 172.21.15.19:0/363884557 wait complete. 2024-05-09T02:13:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 33 2024-05-09T02:13:15.526 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:15 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/363884557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:13:15.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:15 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/363884557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:13:15.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:15 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/363884557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:13:15.985 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T02:13:16.349 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:13:16.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:16 smithi019 ceph-mon[30151]: pgmap v62: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 MiB/s wr, 1.79k op/s 2024-05-09T02:13:16.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:16 smithi059 ceph-mon[34781]: pgmap v62: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 MiB/s wr, 1.79k op/s 2024-05-09T02:13:16.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:16 smithi196 ceph-mon[34718]: pgmap v62: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 MiB/s wr, 1.79k op/s 2024-05-09T02:13:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 95.0 s, 3240.6 tps, lat 30.808 ms stddev 31.183 2024-05-09T02:13:18.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.288+0000 7f1593fe5700 1 -- 172.21.15.19:0/2891496679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f158c107e10 msgr2=0x7f158c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:18.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.288+0000 7f1593fe5700 1 --2- 172.21.15.19:0/2891496679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f158c107e10 0x7f158c108220 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f157c0099c0 tx=0x7f157c009cd0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.289+0000 7f1593fe5700 1 -- 172.21.15.19:0/2891496679 shutdown_connections 2024-05-09T02:13:18.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.289+0000 7f1593fe5700 1 --2- 172.21.15.19:0/2891496679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f158c108760 0x7f158c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.289+0000 7f1593fe5700 1 --2- 172.21.15.19:0/2891496679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f158c1034e0 0x7f158c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.289+0000 7f1593fe5700 1 --2- 172.21.15.19:0/2891496679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f158c107e10 0x7f158c108220 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.289+0000 7f1593fe5700 1 -- 172.21.15.19:0/2891496679 >> 172.21.15.19:0/2891496679 conn(0x7f158c0fef30 msgr2=0x7f158c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:18.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.289+0000 7f1593fe5700 1 -- 172.21.15.19:0/2891496679 shutdown_connections 2024-05-09T02:13:18.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.289+0000 7f1593fe5700 1 -- 172.21.15.19:0/2891496679 wait complete. 2024-05-09T02:13:18.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.289+0000 7f1593fe5700 1 Processor -- start 2024-05-09T02:13:18.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.290+0000 7f1593fe5700 1 -- start start 2024-05-09T02:13:18.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.290+0000 7f1593fe5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f158c1034e0 0x7f158c10c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.290+0000 7f1593fe5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f158c107e10 0x7f158c10c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.290+0000 7f1593fe5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f158c108760 0x7f158c10cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.290+0000 7f1593fe5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f158c112260 con 0x7f158c108760 2024-05-09T02:13:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.290+0000 7f1592582700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f158c108760 0x7f158c10cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.290+0000 7f1592582700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f158c108760 0x7f158c10cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48480/0 (socket says 172.21.15.19:48480) 2024-05-09T02:13:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.290+0000 7f1592582700 1 -- 172.21.15.19:0/3638695651 learned_addr learned my addr 172.21.15.19:0/3638695651 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.290+0000 7f1593fe5700 1 -- 172.21.15.19:0/3638695651 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1120e0 con 0x7f158c107e10 2024-05-09T02:13:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.290+0000 7f1593fe5700 1 -- 172.21.15.19:0/3638695651 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1123e0 con 0x7f158c1034e0 2024-05-09T02:13:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.290+0000 7f1591d81700 1 --2- 172.21.15.19:0/3638695651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f158c1034e0 0x7f158c10c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.290+0000 7f1591580700 1 --2- 172.21.15.19:0/3638695651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f158c107e10 0x7f158c10c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.291+0000 7f1591d81700 1 -- 172.21.15.19:0/3638695651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f158c107e10 msgr2=0x7f158c10c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.291+0000 7f1591d81700 1 --2- 172.21.15.19:0/3638695651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f158c107e10 0x7f158c10c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.291+0000 7f1591d81700 1 -- 172.21.15.19:0/3638695651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f158c108760 msgr2=0x7f158c10cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.291+0000 7f1591d81700 1 --2- 172.21.15.19:0/3638695651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f158c108760 0x7f158c10cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.291+0000 7f1591d81700 1 -- 172.21.15.19:0/3638695651 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c1ad3c0 con 0x7f158c1034e0 2024-05-09T02:13:18.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.291+0000 7f1591d81700 1 --2- 172.21.15.19:0/3638695651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f158c1034e0 0x7f158c10c330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f158800ef10 tx=0x7f158800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:18.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.291+0000 7f1582ffd700 1 -- 172.21.15.19:0/3638695651 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158800c9c0 con 0x7f158c1034e0 2024-05-09T02:13:18.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.291+0000 7f1582ffd700 1 -- 172.21.15.19:0/3638695651 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1588019070 con 0x7f158c1034e0 2024-05-09T02:13:18.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.291+0000 7f1593fe5700 1 -- 172.21.15.19:0/3638695651 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158c1ad650 con 0x7f158c1034e0 2024-05-09T02:13:18.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.292+0000 7f1582ffd700 1 -- 172.21.15.19:0/3638695651 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588014480 con 0x7f158c1034e0 2024-05-09T02:13:18.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.292+0000 7f1593fe5700 1 -- 172.21.15.19:0/3638695651 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158c1adb20 con 0x7f158c1034e0 2024-05-09T02:13:18.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.297+0000 7f1582ffd700 1 -- 172.21.15.19:0/3638695651 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1588012050 con 0x7f158c1034e0 2024-05-09T02:13:18.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.297+0000 7f15a29fbb80 1 Processor -- start 2024-05-09T02:13:18.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.299+0000 7f1582ffd700 1 --2- 172.21.15.19:0/3638695651 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f15780b2a70 0x7f15780b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.299+0000 7f1582ffd700 1 -- 172.21.15.19:0/3638695651 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f15880e2580 con 0x7f158c1034e0 2024-05-09T02:13:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.299+0000 7f15a29fbb80 1 -- start start 2024-05-09T02:13:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.299+0000 7f1592582700 1 --2- 172.21.15.19:0/3638695651 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f15780b2a70 0x7f15780b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f15a29fbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a8dd00 0x55a5b8b96240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f15a29fbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a134a0 0x55a5b8b96780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f15a29fbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 0x55a5b8b90810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f15a29fbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a5b8ab30a0 con 0x55a5b8a8dd00 2024-05-09T02:13:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f15a29fbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a5b8a589a0 con 0x55a5b8a134a0 2024-05-09T02:13:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f15a29fbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a5b8aaf060 con 0x55a5b8b90400 2024-05-09T02:13:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f1591d81700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a8dd00 0x55a5b8b96240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f1591d81700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a8dd00 0x55a5b8b96240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48490/0 (socket says 172.21.15.19:48490) 2024-05-09T02:13:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f1591d81700 1 -- 172.21.15.19:0/1326098026 learned_addr learned my addr 172.21.15.19:0/1326098026 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f1592582700 1 --2- 172.21.15.19:0/1326098026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 0x55a5b8b90810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f1591580700 1 --2- 172.21.15.19:0/1326098026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a134a0 0x55a5b8b96780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f1592582700 1 --2- 172.21.15.19:0/3638695651 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f15780b2a70 0x7f15780b4f80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f158400bca0 tx=0x7f15840106b0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f1591d81700 1 -- 172.21.15.19:0/1326098026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 msgr2=0x55a5b8b90810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f1591d81700 1 --2- 172.21.15.19:0/1326098026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 0x55a5b8b90810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f1591d81700 1 -- 172.21.15.19:0/1326098026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a134a0 msgr2=0x55a5b8b96780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f1591d81700 1 --2- 172.21.15.19:0/1326098026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a134a0 0x55a5b8b96780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.300+0000 7f1591d81700 1 -- 172.21.15.19:0/1326098026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b8a49500 con 0x55a5b8a8dd00 2024-05-09T02:13:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.301+0000 7f1591d81700 1 --2- 172.21.15.19:0/1326098026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a8dd00 0x55a5b8b96240 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f15880a9c50 tx=0x7f15880a9ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a7bcdef6251d97 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:18.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.301+0000 7f1577fff700 1 -- 172.21.15.19:0/1326098026 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15880aa3c0 con 0x55a5b8a8dd00 2024-05-09T02:13:18.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.303+0000 7f1577fff700 1 -- 172.21.15.19:0/1326098026 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15880a9cc0 con 0x55a5b8a8dd00 2024-05-09T02:13:18.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.303+0000 7f1577fff700 1 -- 172.21.15.19:0/1326098026 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15880aa180 con 0x55a5b8a8dd00 2024-05-09T02:13:18.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.304+0000 7f15a29fbb80 1 -- 172.21.15.19:0/1326098026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a8dd00 msgr2=0x55a5b8b96240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:18.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.304+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/1326098026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a8dd00 0x55a5b8b96240 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f15880a9c50 tx=0x7f15880a9ba0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.304+0000 7f15a29fbb80 1 -- 172.21.15.19:0/1326098026 shutdown_connections 2024-05-09T02:13:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.304+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/1326098026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 0x55a5b8b90810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.304+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/1326098026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a8dd00 0x55a5b8b96240 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.304+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/1326098026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a134a0 0x55a5b8b96780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.304+0000 7f15a29fbb80 1 -- 172.21.15.19:0/1326098026 >> 172.21.15.19:0/1326098026 conn(0x55a5b8a16f70 msgr2=0x55a5b8b8b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.305+0000 7f15a29fbb80 1 -- 172.21.15.19:0/1326098026 shutdown_connections 2024-05-09T02:13:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.305+0000 7f15a29fbb80 1 -- 172.21.15.19:0/1326098026 wait complete. 2024-05-09T02:13:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.305+0000 7f15a29fbb80 1 -- start start 2024-05-09T02:13:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.305+0000 7f15a29fbb80 1 Processor -- start 2024-05-09T02:13:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f15a29fbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a134a0 0x55a5b8b99ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f15a29fbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a8dd00 0x55a5b8b9a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f15a29fbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 0x55a5b8b9aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f1591d81700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a134a0 0x55a5b8b99ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f15a29fbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a5b8b91160 con 0x55a5b8a134a0 2024-05-09T02:13:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f1591d81700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a134a0 0x55a5b8b99ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48496/0 (socket says 172.21.15.19:48496) 2024-05-09T02:13:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f1591d81700 1 -- 172.21.15.19:0/539983137 learned_addr learned my addr 172.21.15.19:0/539983137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f15a29fbb80 1 -- 172.21.15.19:0/539983137 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a5b8a4db20 con 0x55a5b8a8dd00 2024-05-09T02:13:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f15a29fbb80 1 -- 172.21.15.19:0/539983137 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a5b8b9d020 con 0x55a5b8b90400 2024-05-09T02:13:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f15a29fbb80 10 client.0 _ll_register_callbacks cb 0x55a5b89246c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f1592582700 1 --2- 172.21.15.19:0/539983137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 0x55a5b8b9aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f1591580700 1 --2- 172.21.15.19:0/539983137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a8dd00 0x55a5b8b9a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f1591d81700 1 -- 172.21.15.19:0/539983137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 msgr2=0x55a5b8b9aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f1591d81700 1 --2- 172.21.15.19:0/539983137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 0x55a5b8b9aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f1591d81700 1 -- 172.21.15.19:0/539983137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a8dd00 msgr2=0x55a5b8b9a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f1591d81700 1 --2- 172.21.15.19:0/539983137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a8dd00 0x55a5b8b9a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f1591d81700 1 -- 172.21.15.19:0/539983137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b892e490 con 0x55a5b8a134a0 2024-05-09T02:13:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.306+0000 7f1592582700 1 --2- 172.21.15.19:0/539983137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 0x55a5b8b9aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:13:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.307+0000 7f1591d81700 1 --2- 172.21.15.19:0/539983137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a134a0 0x55a5b8b99ff0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f15880a5d40 tx=0x7f15880b2970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea0500b834ebf8b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.307+0000 7f15757fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:13:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.307+0000 7f15757fa700 1 -- 172.21.15.19:0/539983137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15880c9070 con 0x55a5b8a134a0 2024-05-09T02:13:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.307+0000 7f15a29fbb80 1 -- 172.21.15.19:0/539983137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15880a9cc0 con 0x55a5b8a134a0 2024-05-09T02:13:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.307+0000 7f15757fa700 1 -- 172.21.15.19:0/539983137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f15880e2260 con 0x55a5b8a134a0 2024-05-09T02:13:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.307+0000 7f15757fa700 1 -- 172.21.15.19:0/539983137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f15880ae070 con 0x55a5b8a134a0 2024-05-09T02:13:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.307+0000 7f15a29fbb80 10 client.15573 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.307+0000 7f15a29fbb80 1 -- 172.21.15.19:0/539983137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a5b8ba79a0 con 0x55a5b8a134a0 2024-05-09T02:13:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.307+0000 7f15757fa700 1 -- 172.21.15.19:0/539983137 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15880a9cc0 con 0x55a5b8a134a0 2024-05-09T02:13:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.308+0000 7f15757fa700 1 -- 172.21.15.19:0/539983137 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f15880a5780 con 0x55a5b8a134a0 2024-05-09T02:13:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.308+0000 7f15a29fbb80 10 client.15573 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.308+0000 7f15a29fbb80 10 client.15573 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:13:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.308+0000 7f15a29fbb80 20 client.15573 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.308+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/539983137 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55a5b8ba9e70 0x55a5b8bac290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.308+0000 7f15a29fbb80 4 client.15573 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:13:18.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.308+0000 7f15a29fbb80 1 -- 172.21.15.19:0/539983137 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5b8ac0a80 con 0x55a5b8ba9e70 2024-05-09T02:13:18.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.308+0000 7f1591580700 1 --2- 172.21.15.19:0/539983137 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55a5b8ba9e70 0x55a5b8bac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.309+0000 7f15757fa700 10 client.15573 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:13:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.309+0000 7f1591580700 1 --2- 172.21.15.19:0/539983137 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55a5b8ba9e70 0x55a5b8bac290 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4b89bf3f52b01af0 server_cookie=6a9008e0022ee64d in_seq=0 out_seq=0 2024-05-09T02:13:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.310+0000 7f15757fa700 1 -- 172.21.15.19:0/539983137 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a5b8ac0a80 con 0x55a5b8ba9e70 2024-05-09T02:13:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.310+0000 7f15757fa700 10 client.15573 handle_command_reply: tid=0 2024-05-09T02:13:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.310+0000 7f15a29fbb80 1 client.15573 shutdown 2024-05-09T02:13:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.310+0000 7f15a29fbb80 1 -- 172.21.15.19:0/539983137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a134a0 msgr2=0x55a5b8b99ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.310+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/539983137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a134a0 0x55a5b8b99ff0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f15880a5d40 tx=0x7f15880b2970 comp rx=0 tx=0).stop 2024-05-09T02:13:18.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.310+0000 7f15a29fbb80 1 -- 172.21.15.19:0/539983137 shutdown_connections 2024-05-09T02:13:18.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.310+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/539983137 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55a5b8ba9e70 0x55a5b8bac290 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.310+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/539983137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 0x55a5b8b9aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.311+0000 7f15757fa700 0 client.15573 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:13:18.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.311+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/539983137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a134a0 0x55a5b8b99ff0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.311+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/539983137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a8dd00 0x55a5b8b9a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.311+0000 7f15a29fbb80 1 -- 172.21.15.19:0/539983137 >> 172.21.15.19:0/539983137 conn(0x55a5b8a16f70 msgr2=0x55a5b8b84ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:18.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.311+0000 7f15a29fbb80 1 -- 172.21.15.19:0/539983137 shutdown_connections 2024-05-09T02:13:18.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.311+0000 7f15a29fbb80 1 -- 172.21.15.19:0/539983137 wait complete. 2024-05-09T02:13:18.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.311+0000 7f15a29fbb80 20 client.15573 trim_cache size 0 max 16384 2024-05-09T02:13:18.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.331+0000 7f15a29fbb80 1 Processor -- start 2024-05-09T02:13:18.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.331+0000 7f15a29fbb80 1 -- start start 2024-05-09T02:13:18.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.331+0000 7f15a29fbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a134a0 0x55a5b8b85a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.331+0000 7f15a29fbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a8dd00 0x55a5b8ba60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.331+0000 7f15a29fbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 0x55a5b8b97250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.331+0000 7f15a29fbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1b5390 con 0x55a5b8a8dd00 2024-05-09T02:13:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.331+0000 7f15a29fbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1b6f90 con 0x55a5b8a134a0 2024-05-09T02:13:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.331+0000 7f15a29fbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1b5a60 con 0x55a5b8b90400 2024-05-09T02:13:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.332+0000 7f1591580700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a8dd00 0x55a5b8ba60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.332+0000 7f1591580700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a8dd00 0x55a5b8ba60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48510/0 (socket says 172.21.15.19:48510) 2024-05-09T02:13:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.332+0000 7f1591580700 1 -- 172.21.15.19:0/4130754619 learned_addr learned my addr 172.21.15.19:0/4130754619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.332+0000 7f1592582700 1 --2- 172.21.15.19:0/4130754619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 0x55a5b8b97250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.332+0000 7f1591d81700 1 --2- 172.21.15.19:0/4130754619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a134a0 0x55a5b8b85a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.332+0000 7f1591580700 1 -- 172.21.15.19:0/4130754619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 msgr2=0x55a5b8b97250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.332+0000 7f1591580700 1 --2- 172.21.15.19:0/4130754619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 0x55a5b8b97250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.332+0000 7f1591580700 1 -- 172.21.15.19:0/4130754619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a134a0 msgr2=0x55a5b8b85a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:18.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.332+0000 7f1591580700 1 --2- 172.21.15.19:0/4130754619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a134a0 0x55a5b8b85a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.332+0000 7f1591580700 1 -- 172.21.15.19:0/4130754619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b8b87510 con 0x55a5b8a8dd00 2024-05-09T02:13:18.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.332+0000 7f1591d81700 1 --2- 172.21.15.19:0/4130754619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a134a0 0x55a5b8b85a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:13:18.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.332+0000 7f1591580700 1 --2- 172.21.15.19:0/4130754619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a8dd00 0x55a5b8ba60a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f157c003760 tx=0x7f157c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d444474c4c3c2bd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:18.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.332+0000 7f15777fe700 1 -- 172.21.15.19:0/4130754619 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c0054e0 con 0x55a5b8a8dd00 2024-05-09T02:13:18.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.332+0000 7f15777fe700 1 -- 172.21.15.19:0/4130754619 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f157c02f070 con 0x55a5b8a8dd00 2024-05-09T02:13:18.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.332+0000 7f15777fe700 1 -- 172.21.15.19:0/4130754619 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c00baf0 con 0x55a5b8a8dd00 2024-05-09T02:13:18.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.333+0000 7f15a29fbb80 1 -- 172.21.15.19:0/4130754619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a8dd00 msgr2=0x55a5b8ba60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:18.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.333+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/4130754619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a8dd00 0x55a5b8ba60a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f157c003760 tx=0x7f157c005040 comp rx=0 tx=0).stop 2024-05-09T02:13:18.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.335+0000 7f15a29fbb80 1 -- 172.21.15.19:0/4130754619 shutdown_connections 2024-05-09T02:13:18.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.335+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/4130754619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8b90400 0x55a5b8b97250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.335+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/4130754619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a134a0 0x55a5b8b85a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.335+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/4130754619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8a8dd00 0x55a5b8ba60a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.335+0000 7f15a29fbb80 1 -- 172.21.15.19:0/4130754619 >> 172.21.15.19:0/4130754619 conn(0x55a5b8a16f70 msgr2=0x55a5b8b98f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:18.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.335+0000 7f15a29fbb80 1 -- 172.21.15.19:0/4130754619 shutdown_connections 2024-05-09T02:13:18.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.335+0000 7f15a29fbb80 1 -- 172.21.15.19:0/4130754619 wait complete. 2024-05-09T02:13:18.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.336+0000 7f15a29fbb80 1 -- start start 2024-05-09T02:13:18.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.336+0000 7f15a29fbb80 1 Processor -- start 2024-05-09T02:13:18.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.337+0000 7f15a29fbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8a134a0 0x55a5b8baf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.337+0000 7f15a29fbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a8dd00 0x55a5b8bacaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.337+0000 7f15a29fbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8b90400 0x55a5b8bacfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.338+0000 7f1591d81700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8a134a0 0x55a5b8baf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.338+0000 7f15a29fbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1b6800 con 0x55a5b8b90400 2024-05-09T02:13:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.338+0000 7f1592582700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8b90400 0x55a5b8bacfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.338+0000 7f1591d81700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8a134a0 0x55a5b8baf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:39066/0 (socket says 172.21.15.19:39066) 2024-05-09T02:13:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.338+0000 7f1591d81700 1 -- 172.21.15.19:0/3813276912 learned_addr learned my addr 172.21.15.19:0/3813276912 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.338+0000 7f15a29fbb80 1 -- 172.21.15.19:0/3813276912 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1b6130 con 0x55a5b8a8dd00 2024-05-09T02:13:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.338+0000 7f15a29fbb80 1 -- 172.21.15.19:0/3813276912 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1b3330 con 0x55a5b8a134a0 2024-05-09T02:13:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.338+0000 7f15a29fbb80 10 client.0 _ll_register_callbacks cb 0x55a5b89246c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.338+0000 7f1591580700 1 --2- 172.21.15.19:0/3813276912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a8dd00 0x55a5b8bacaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.339+0000 7f1592582700 1 -- 172.21.15.19:0/3813276912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8a134a0 msgr2=0x55a5b8baf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:18.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.339+0000 7f1592582700 1 --2- 172.21.15.19:0/3813276912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8a134a0 0x55a5b8baf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.339+0000 7f1592582700 1 -- 172.21.15.19:0/3813276912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a8dd00 msgr2=0x55a5b8bacaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:18.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.339+0000 7f1592582700 1 --2- 172.21.15.19:0/3813276912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a8dd00 0x55a5b8bacaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.339+0000 7f1592582700 1 -- 172.21.15.19:0/3813276912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b8bad5f0 con 0x55a5b8b90400 2024-05-09T02:13:18.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.339+0000 7f1592582700 1 --2- 172.21.15.19:0/3813276912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8b90400 0x55a5b8bacfe0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1584011860 tx=0x7f15840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9ad43f1d5fa3fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:18.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.339+0000 7f15757fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:13:18.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.341+0000 7f15757fa700 1 -- 172.21.15.19:0/3813276912 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584004a30 con 0x55a5b8b90400 2024-05-09T02:13:18.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.341+0000 7f15757fa700 1 -- 172.21.15.19:0/3813276912 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f1584004bd0 con 0x55a5b8b90400 2024-05-09T02:13:18.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.341+0000 7f15a29fbb80 1 -- 172.21.15.19:0/3813276912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f157c02f070 con 0x55a5b8b90400 2024-05-09T02:13:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.342+0000 7f15757fa700 1 -- 172.21.15.19:0/3813276912 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1584004dd0 con 0x55a5b8b90400 2024-05-09T02:13:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.343+0000 7f15a29fbb80 10 client.15579 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:18.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.343+0000 7f15a29fbb80 1 -- 172.21.15.19:0/3813276912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a5b8c141c0 con 0x55a5b8b90400 2024-05-09T02:13:18.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.345+0000 7f15757fa700 1 -- 172.21.15.19:0/3813276912 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15840054d0 con 0x55a5b8b90400 2024-05-09T02:13:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.346+0000 7f15757fa700 1 -- 172.21.15.19:0/3813276912 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f1584028070 con 0x55a5b8b90400 2024-05-09T02:13:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.347+0000 7f15a29fbb80 10 client.15579 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:18.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.347+0000 7f15a29fbb80 10 client.15579 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:13:18.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.347+0000 7f15a29fbb80 20 client.15579 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:18.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.347+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/3813276912 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55a5b8c14850 0x55a5b8c16c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:18.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.347+0000 7f15a29fbb80 4 client.15579 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:13:18.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.347+0000 7f15a29fbb80 1 -- 172.21.15.19:0/3813276912 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f157c02f070 con 0x55a5b8c14850 2024-05-09T02:13:18.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.347+0000 7f1591d81700 1 --2- 172.21.15.19:0/3813276912 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55a5b8c14850 0x55a5b8c16c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:18.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.348+0000 7f1591d81700 1 --2- 172.21.15.19:0/3813276912 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55a5b8c14850 0x55a5b8c16c70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=49463e7c2048d59a server_cookie=747e26783a288812 in_seq=0 out_seq=0 2024-05-09T02:13:18.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.348+0000 7f15757fa700 10 client.15579 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:13:18.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.350+0000 7f15757fa700 1 -- 172.21.15.19:0/3813276912 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f157c02f070 con 0x55a5b8c14850 2024-05-09T02:13:18.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.350+0000 7f15757fa700 10 client.15579 handle_command_reply: tid=0 2024-05-09T02:13:18.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.350+0000 7f15a29fbb80 1 client.15579 shutdown 2024-05-09T02:13:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.353+0000 7f15a29fbb80 1 -- 172.21.15.19:0/3813276912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8b90400 msgr2=0x55a5b8bacfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.353+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/3813276912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8b90400 0x55a5b8bacfe0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1584011860 tx=0x7f15840047c0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.354+0000 7f15a29fbb80 1 -- 172.21.15.19:0/3813276912 shutdown_connections 2024-05-09T02:13:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.354+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/3813276912 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55a5b8c14850 0x55a5b8c16c70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.354+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/3813276912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b8b90400 0x55a5b8bacfe0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.354+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/3813276912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5b8a134a0 0x55a5b8baf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.354+0000 7f15a29fbb80 1 --2- 172.21.15.19:0/3813276912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5b8a8dd00 0x55a5b8bacaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.354+0000 7f15a29fbb80 1 -- 172.21.15.19:0/3813276912 >> 172.21.15.19:0/3813276912 conn(0x55a5b8a16f70 msgr2=0x55a5b8b93cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.354+0000 7f15757fa700 0 client.15579 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:13:18.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.356+0000 7f15a29fbb80 1 -- 172.21.15.19:0/3813276912 shutdown_connections 2024-05-09T02:13:18.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.356+0000 7f15a29fbb80 1 -- 172.21.15.19:0/3813276912 wait complete. 2024-05-09T02:13:18.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.356+0000 7f15a29fbb80 20 client.15579 trim_cache size 0 max 16384 2024-05-09T02:13:18.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:18 smithi019 ceph-mon[30151]: pgmap v63: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 MiB/s wr, 958 op/s 2024-05-09T02:13:18.362 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:13:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.365+0000 7f1593fe5700 1 -- 172.21.15.19:0/3638695651 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f15780b2a70 msgr2=0x7f15780b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.365+0000 7f1593fe5700 1 --2- 172.21.15.19:0/3638695651 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f15780b2a70 0x7f15780b4f80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f158400bca0 tx=0x7f15840106b0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.365+0000 7f1593fe5700 1 -- 172.21.15.19:0/3638695651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f158c1034e0 msgr2=0x7f158c10c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.365+0000 7f1593fe5700 1 --2- 172.21.15.19:0/3638695651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f158c1034e0 0x7f158c10c330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f158800ef10 tx=0x7f158800c3a0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.365+0000 7f1593fe5700 1 -- 172.21.15.19:0/3638695651 shutdown_connections 2024-05-09T02:13:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.365+0000 7f1593fe5700 1 --2- 172.21.15.19:0/3638695651 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f15780b2a70 0x7f15780b4f80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.365+0000 7f1593fe5700 1 --2- 172.21.15.19:0/3638695651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f158c108760 0x7f158c10cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.365+0000 7f1593fe5700 1 --2- 172.21.15.19:0/3638695651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f158c1034e0 0x7f158c10c330 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.365+0000 7f1593fe5700 1 --2- 172.21.15.19:0/3638695651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f158c107e10 0x7f158c10c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.365+0000 7f1593fe5700 1 -- 172.21.15.19:0/3638695651 >> 172.21.15.19:0/3638695651 conn(0x7f158c0fef30 msgr2=0x7f158c107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.366+0000 7f1593fe5700 1 -- 172.21.15.19:0/3638695651 shutdown_connections 2024-05-09T02:13:18.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:18.366+0000 7f1593fe5700 1 -- 172.21.15.19:0/3638695651 wait complete. 2024-05-09T02:13:18.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:18 smithi059 ceph-mon[34781]: pgmap v63: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 MiB/s wr, 958 op/s 2024-05-09T02:13:18.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:18 smithi196 ceph-mon[34718]: pgmap v63: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 MiB/s wr, 958 op/s 2024-05-09T02:13:20.002 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T02:13:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:13:20.534 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:20 smithi019 ceph-mon[30151]: pgmap v64: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 MiB/s wr, 1.33k op/s 2024-05-09T02:13:20.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:20 smithi059 ceph-mon[34781]: pgmap v64: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 MiB/s wr, 1.33k op/s 2024-05-09T02:13:20.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:20 smithi196 ceph-mon[34718]: pgmap v64: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 MiB/s wr, 1.33k op/s 2024-05-09T02:13:22.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.361+0000 7fb179787700 1 -- 172.21.15.19:0/3702433956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb174113900 msgr2=0x7fb174113d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:22.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.361+0000 7fb179787700 1 --2- 172.21.15.19:0/3702433956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb174113900 0x7fb174113d10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb1640099c0 tx=0x7fb164009cd0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.363+0000 7fb179787700 1 -- 172.21.15.19:0/3702433956 shutdown_connections 2024-05-09T02:13:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.363+0000 7fb179787700 1 --2- 172.21.15.19:0/3702433956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb174114250 0x7fb17411bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.363+0000 7fb179787700 1 --2- 172.21.15.19:0/3702433956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb17410efd0 0x7fb17410f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.363+0000 7fb179787700 1 --2- 172.21.15.19:0/3702433956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb174113900 0x7fb174113d10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.363+0000 7fb179787700 1 -- 172.21.15.19:0/3702433956 >> 172.21.15.19:0/3702433956 conn(0x7fb17410ab60 msgr2=0x7fb17410cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.363+0000 7fb179787700 1 -- 172.21.15.19:0/3702433956 shutdown_connections 2024-05-09T02:13:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.363+0000 7fb179787700 1 -- 172.21.15.19:0/3702433956 wait complete. 2024-05-09T02:13:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.364+0000 7fb179787700 1 Processor -- start 2024-05-09T02:13:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.365+0000 7fb179787700 1 -- start start 2024-05-09T02:13:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.365+0000 7fb179787700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb17410efd0 0x7fb1741ab530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.365+0000 7fb179787700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb174113900 0x7fb1741aba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.366+0000 7fb172ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb17410efd0 0x7fb1741ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.366+0000 7fb172ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb17410efd0 0x7fb1741ab530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:49752/0 (socket says 172.21.15.19:49752) 2024-05-09T02:13:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.366+0000 7fb179787700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb174114250 0x7fb1741a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.366+0000 7fb172ffd700 1 -- 172.21.15.19:0/4214212275 learned_addr learned my addr 172.21.15.19:0/4214212275 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.366+0000 7fb179787700 1 -- 172.21.15.19:0/4214212275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb17411ee60 con 0x7fb174114250 2024-05-09T02:13:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.366+0000 7fb179787700 1 -- 172.21.15.19:0/4214212275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb17411ece0 con 0x7fb17410efd0 2024-05-09T02:13:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.366+0000 7fb179787700 1 -- 172.21.15.19:0/4214212275 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb17411efe0 con 0x7fb174113900 2024-05-09T02:13:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.367+0000 7fb172ffd700 1 -- 172.21.15.19:0/4214212275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb174113900 msgr2=0x7fb1741aba70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T02:13:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.367+0000 7fb172ffd700 1 --2- 172.21.15.19:0/4214212275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb174113900 0x7fb1741aba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.367+0000 7fb172ffd700 1 -- 172.21.15.19:0/4214212275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb174114250 msgr2=0x7fb1741a56c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:13:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.367+0000 7fb172ffd700 1 --2- 172.21.15.19:0/4214212275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb174114250 0x7fb1741a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.367+0000 7fb172ffd700 1 -- 172.21.15.19:0/4214212275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1741a5df0 con 0x7fb17410efd0 2024-05-09T02:13:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.367+0000 7fb172ffd700 1 --2- 172.21.15.19:0/4214212275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb17410efd0 0x7fb1741ab530 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00e9a0 tx=0x7fb15c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.368+0000 7fb16bfff700 1 -- 172.21.15.19:0/4214212275 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c00cbd0 con 0x7fb17410efd0 2024-05-09T02:13:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.368+0000 7fb16bfff700 1 -- 172.21.15.19:0/4214212275 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb15c00cd70 con 0x7fb17410efd0 2024-05-09T02:13:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.368+0000 7fb16bfff700 1 -- 172.21.15.19:0/4214212275 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c00cbd0 con 0x7fb17410efd0 2024-05-09T02:13:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.368+0000 7fb179787700 1 -- 172.21.15.19:0/4214212275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1741a60e0 con 0x7fb17410efd0 2024-05-09T02:13:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.368+0000 7fb179787700 1 -- 172.21.15.19:0/4214212275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1741a6510 con 0x7fb17410efd0 2024-05-09T02:13:22.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.378+0000 7fb18819db80 1 Processor -- start 2024-05-09T02:13:22.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.379+0000 7fb18819db80 1 -- start start 2024-05-09T02:13:22.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.379+0000 7fb18819db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fea70d00 0x5582feb79240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.380+0000 7fb18819db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fe9f64a0 0x5582feb79780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.380+0000 7fb18819db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb73400 0x5582feb73810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.380+0000 7fb18819db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5582fea960a0 con 0x5582fea70d00 2024-05-09T02:13:22.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.380+0000 7fb18819db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5582fea3b9a0 con 0x5582fe9f64a0 2024-05-09T02:13:22.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.380+0000 7fb18819db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5582fea92060 con 0x5582feb73400 2024-05-09T02:13:22.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.380+0000 7fb1737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fea70d00 0x5582feb79240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:22.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.380+0000 7fb1737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fea70d00 0x5582feb79240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48550/0 (socket says 172.21.15.19:48550) 2024-05-09T02:13:22.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.380+0000 7fb1727fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb73400 0x5582feb73810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:22.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.380+0000 7fb1737fe700 1 -- 172.21.15.19:0/3479967640 learned_addr learned my addr 172.21.15.19:0/3479967640 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:22.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.380+0000 7fb1737fe700 1 -- 172.21.15.19:0/3479967640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb73400 msgr2=0x5582feb73810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:22.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.380+0000 7fb1737fe700 1 --2- 172.21.15.19:0/3479967640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb73400 0x5582feb73810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.381+0000 7fb1737fe700 1 -- 172.21.15.19:0/3479967640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fe9f64a0 msgr2=0x5582feb79780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:13:22.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.381+0000 7fb1737fe700 1 --2- 172.21.15.19:0/3479967640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fe9f64a0 0x5582feb79780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.381+0000 7fb1737fe700 1 -- 172.21.15.19:0/3479967640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582fea2c500 con 0x5582fea70d00 2024-05-09T02:13:22.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.381+0000 7fb1737fe700 1 --2- 172.21.15.19:0/3479967640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fea70d00 0x5582feb79240 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009d10 tx=0x7fb16c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6259f80177ec2210 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:22.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.382+0000 7fb16bfff700 1 -- 172.21.15.19:0/4214212275 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb15c010430 con 0x7fb17410efd0 2024-05-09T02:13:22.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.383+0000 7fb1697fa700 1 -- 172.21.15.19:0/3479967640 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c003a00 con 0x5582fea70d00 2024-05-09T02:13:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.383+0000 7fb1697fa700 1 -- 172.21.15.19:0/3479967640 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb16c003ba0 con 0x5582fea70d00 2024-05-09T02:13:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.383+0000 7fb16bfff700 1 --2- 172.21.15.19:0/4214212275 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb1600b2e70 0x7fb1600b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.384+0000 7fb16bfff700 1 -- 172.21.15.19:0/4214212275 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fb15c00c9d0 con 0x7fb17410efd0 2024-05-09T02:13:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.384+0000 7fb1697fa700 1 -- 172.21.15.19:0/3479967640 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c013650 con 0x5582fea70d00 2024-05-09T02:13:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.384+0000 7fb18819db80 1 -- 172.21.15.19:0/3479967640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fea70d00 msgr2=0x5582feb79240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.384+0000 7fb18819db80 1 --2- 172.21.15.19:0/3479967640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fea70d00 0x5582feb79240 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009d10 tx=0x7fb16c0077e0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.385+0000 7fb18819db80 1 -- 172.21.15.19:0/3479967640 shutdown_connections 2024-05-09T02:13:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.385+0000 7fb18819db80 1 --2- 172.21.15.19:0/3479967640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb73400 0x5582feb73810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.385+0000 7fb18819db80 1 --2- 172.21.15.19:0/3479967640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fea70d00 0x5582feb79240 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.385+0000 7fb18819db80 1 --2- 172.21.15.19:0/3479967640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fe9f64a0 0x5582feb79780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.385+0000 7fb18819db80 1 -- 172.21.15.19:0/3479967640 >> 172.21.15.19:0/3479967640 conn(0x5582fe9f9f70 msgr2=0x5582feb6e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.385+0000 7fb18819db80 1 -- 172.21.15.19:0/3479967640 shutdown_connections 2024-05-09T02:13:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.385+0000 7fb18819db80 1 -- 172.21.15.19:0/3479967640 wait complete. 2024-05-09T02:13:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.386+0000 7fb18819db80 1 -- start start 2024-05-09T02:13:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.386+0000 7fb18819db80 1 Processor -- start 2024-05-09T02:13:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.387+0000 7fb18819db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f64a0 0x5582feb7d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.387+0000 7fb18819db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fea70d00 0x5582feb7d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.387+0000 7fb18819db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb73400 0x5582feb7db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.387+0000 7fb18819db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5582feb741b0 con 0x5582fe9f64a0 2024-05-09T02:13:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.387+0000 7fb18819db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5582fea30b20 con 0x5582fea70d00 2024-05-09T02:13:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.387+0000 7fb18819db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5582feb80020 con 0x5582feb73400 2024-05-09T02:13:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.387+0000 7fb18819db80 10 client.0 _ll_register_callbacks cb 0x5582fe9076c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.387+0000 7fb1737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f64a0 0x5582feb7d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.387+0000 7fb1737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f64a0 0x5582feb7d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48566/0 (socket says 172.21.15.19:48566) 2024-05-09T02:13:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.387+0000 7fb1737fe700 1 -- 172.21.15.19:0/3400912806 learned_addr learned my addr 172.21.15.19:0/3400912806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.387+0000 7fb1727fc700 1 --2- 172.21.15.19:0/3400912806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fea70d00 0x5582feb7d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.387+0000 7fb1737fe700 1 --2- 172.21.15.19:0/3400912806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb73400 0x5582feb7db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.388+0000 7fb1727fc700 1 -- 172.21.15.19:0/3400912806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb73400 msgr2=0x5582feb7db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.388+0000 7fb1727fc700 1 --2- 172.21.15.19:0/3400912806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb73400 0x5582feb7db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.388+0000 7fb1727fc700 1 -- 172.21.15.19:0/3400912806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f64a0 msgr2=0x5582feb7d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.388+0000 7fb1727fc700 1 --2- 172.21.15.19:0/3400912806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f64a0 0x5582feb7d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.388+0000 7fb1727fc700 1 -- 172.21.15.19:0/3400912806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582fe911490 con 0x5582fea70d00 2024-05-09T02:13:22.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.388+0000 7fb1737fe700 1 --2- 172.21.15.19:0/3400912806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb73400 0x5582feb7db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:13:22.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.388+0000 7fb1727fc700 1 --2- 172.21.15.19:0/3400912806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fea70d00 0x5582feb7d5f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb1741a6e60 tx=0x7fb164005790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feff5942e080ee11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:22.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.388+0000 7fb15a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:13:22.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.388+0000 7fb15a7fc700 1 -- 172.21.15.19:0/3400912806 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640039c0 con 0x5582fea70d00 2024-05-09T02:13:22.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.388+0000 7fb15a7fc700 1 -- 172.21.15.19:0/3400912806 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fb164003b60 con 0x5582fea70d00 2024-05-09T02:13:22.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.388+0000 7fb172ffd700 1 --2- 172.21.15.19:0/4214212275 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb1600b2e70 0x7fb1600b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:22.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.388+0000 7fb18819db80 1 -- 172.21.15.19:0/3400912806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb16c003ba0 con 0x5582fea70d00 2024-05-09T02:13:22.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.389+0000 7fb15a7fc700 1 -- 172.21.15.19:0/3400912806 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb164003d80 con 0x5582fea70d00 2024-05-09T02:13:22.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.389+0000 7fb18819db80 10 client.25274 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:22.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.389+0000 7fb18819db80 1 -- 172.21.15.19:0/3400912806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5582feb8a8f0 con 0x5582fea70d00 2024-05-09T02:13:22.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.390+0000 7fb15a7fc700 1 -- 172.21.15.19:0/3400912806 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb16c003ba0 con 0x5582fea70d00 2024-05-09T02:13:22.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.390+0000 7fb172ffd700 1 --2- 172.21.15.19:0/4214212275 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb1600b2e70 0x7fb1600b5380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x5582feb6fc90 tx=0x7fb15c000bc0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:22.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.390+0000 7fb15a7fc700 1 -- 172.21.15.19:0/3400912806 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fb16400f4f0 con 0x5582fea70d00 2024-05-09T02:13:22.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.390+0000 7fb18819db80 10 client.25274 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:22.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.390+0000 7fb18819db80 10 client.25274 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:13:22.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.390+0000 7fb18819db80 20 client.25274 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:22.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.390+0000 7fb18819db80 1 --2- 172.21.15.19:0/3400912806 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5582feb8cf30 0x5582feb8f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.390+0000 7fb18819db80 4 client.25274 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:13:22.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.390+0000 7fb18819db80 1 -- 172.21.15.19:0/3400912806 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582feaa3a80 con 0x5582feb8cf30 2024-05-09T02:13:22.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.391+0000 7fb1737fe700 1 --2- 172.21.15.19:0/3400912806 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5582feb8cf30 0x5582feb8f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:22.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.392+0000 7fb15a7fc700 10 client.25274 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:13:22.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.392+0000 7fb1737fe700 1 --2- 172.21.15.19:0/3400912806 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5582feb8cf30 0x5582feb8f350 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7554a73abc0d48a7 server_cookie=2d8b6ed18403ef13 in_seq=0 out_seq=0 2024-05-09T02:13:22.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.393+0000 7fb15a7fc700 1 -- 172.21.15.19:0/3400912806 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5582feaa3a80 con 0x5582feb8cf30 2024-05-09T02:13:22.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.393+0000 7fb15a7fc700 10 client.25274 handle_command_reply: tid=0 2024-05-09T02:13:22.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.393+0000 7fb18819db80 1 client.25274 shutdown 2024-05-09T02:13:22.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.394+0000 7fb18819db80 1 -- 172.21.15.19:0/3400912806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fea70d00 msgr2=0x5582feb7d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:22.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.394+0000 7fb18819db80 1 --2- 172.21.15.19:0/3400912806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fea70d00 0x5582feb7d5f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb1741a6e60 tx=0x7fb164005790 comp rx=0 tx=0).stop 2024-05-09T02:13:22.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.394+0000 7fb18819db80 1 -- 172.21.15.19:0/3400912806 shutdown_connections 2024-05-09T02:13:22.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.394+0000 7fb18819db80 1 --2- 172.21.15.19:0/3400912806 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5582feb8cf30 0x5582feb8f350 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.394+0000 7fb18819db80 1 --2- 172.21.15.19:0/3400912806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb73400 0x5582feb7db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.394+0000 7fb18819db80 1 --2- 172.21.15.19:0/3400912806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f64a0 0x5582feb7d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.394+0000 7fb18819db80 1 --2- 172.21.15.19:0/3400912806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fea70d00 0x5582feb7d5f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.394+0000 7fb18819db80 1 -- 172.21.15.19:0/3400912806 >> 172.21.15.19:0/3400912806 conn(0x5582fe9f9f70 msgr2=0x5582feb67ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:22.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.394+0000 7fb15a7fc700 0 client.25274 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:13:22.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.394+0000 7fb18819db80 1 -- 172.21.15.19:0/3400912806 shutdown_connections 2024-05-09T02:13:22.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.394+0000 7fb18819db80 1 -- 172.21.15.19:0/3400912806 wait complete. 2024-05-09T02:13:22.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.394+0000 7fb18819db80 20 client.25274 trim_cache size 0 max 16384 2024-05-09T02:13:22.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.415+0000 7fb18819db80 1 Processor -- start 2024-05-09T02:13:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.415+0000 7fb18819db80 1 -- start start 2024-05-09T02:13:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb18819db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb6e520 0x5582feb78880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb18819db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f9f70 0x5582feb68700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb1737fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb6e520 0x5582feb78880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb1727fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f9f70 0x5582feb68700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb1737fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb6e520 0x5582feb78880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:39110/0 (socket says 172.21.15.19:39110) 2024-05-09T02:13:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb1727fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f9f70 0x5582feb68700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48582/0 (socket says 172.21.15.19:48582) 2024-05-09T02:13:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb18819db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fe9f64a0 0x5582feb94a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb18819db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1741bdef0 con 0x5582fe9f9f70 2024-05-09T02:13:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb18819db80 1 -- 172.21.15.19:0/527437616 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1741bfaf0 con 0x5582fe9f64a0 2024-05-09T02:13:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb1737fe700 1 -- 172.21.15.19:0/527437616 learned_addr learned my addr 172.21.15.19:0/527437616 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb18819db80 1 -- 172.21.15.19:0/527437616 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1741be5c0 con 0x5582feb6e520 2024-05-09T02:13:22.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb1727fc700 1 -- 172.21.15.19:0/527437616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb6e520 msgr2=0x5582feb78880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:22.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb1727fc700 1 --2- 172.21.15.19:0/527437616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb6e520 0x5582feb78880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb1727fc700 1 -- 172.21.15.19:0/527437616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fe9f64a0 msgr2=0x5582feb94a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:22.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb1727fc700 1 --2- 172.21.15.19:0/527437616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fe9f64a0 0x5582feb94a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb1727fc700 1 -- 172.21.15.19:0/527437616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582feb8f7f0 con 0x5582fe9f9f70 2024-05-09T02:13:22.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.416+0000 7fb1737fe700 1 --2- 172.21.15.19:0/527437616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb6e520 0x5582feb78880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:13:22.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.417+0000 7fb1727fc700 1 --2- 172.21.15.19:0/527437616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f9f70 0x5582feb68700 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb164005f50 tx=0x7fb164009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21747b29c6b27c1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:22.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.417+0000 7fb168ff9700 1 -- 172.21.15.19:0/527437616 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16400fcc0 con 0x5582fe9f9f70 2024-05-09T02:13:22.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.417+0000 7fb168ff9700 1 -- 172.21.15.19:0/527437616 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb164026070 con 0x5582fe9f9f70 2024-05-09T02:13:22.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.417+0000 7fb168ff9700 1 -- 172.21.15.19:0/527437616 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16402b450 con 0x5582fe9f9f70 2024-05-09T02:13:22.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.418+0000 7fb18819db80 1 -- 172.21.15.19:0/527437616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f9f70 msgr2=0x5582feb68700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:22.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.418+0000 7fb18819db80 1 --2- 172.21.15.19:0/527437616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f9f70 0x5582feb68700 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb164005f50 tx=0x7fb164009cd0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.419+0000 7fb18819db80 1 -- 172.21.15.19:0/527437616 shutdown_connections 2024-05-09T02:13:22.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.419+0000 7fb18819db80 1 --2- 172.21.15.19:0/527437616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fe9f64a0 0x5582feb94a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.419+0000 7fb18819db80 1 --2- 172.21.15.19:0/527437616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb6e520 0x5582feb78880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.419+0000 7fb18819db80 1 --2- 172.21.15.19:0/527437616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f9f70 0x5582feb68700 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.419+0000 7fb18819db80 1 -- 172.21.15.19:0/527437616 >> 172.21.15.19:0/527437616 conn(0x5582feb7dc80 msgr2=0x5582feb67ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:22.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.419+0000 7fb18819db80 1 -- 172.21.15.19:0/527437616 shutdown_connections 2024-05-09T02:13:22.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.419+0000 7fb18819db80 1 -- 172.21.15.19:0/527437616 wait complete. 2024-05-09T02:13:22.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.420+0000 7fb18819db80 1 -- start start 2024-05-09T02:13:22.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.420+0000 7fb18819db80 1 Processor -- start 2024-05-09T02:13:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.420+0000 7fb18819db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f64a0 0x5582feb85130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.420+0000 7fb18819db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fe9f9f70 0x5582feb85670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.420+0000 7fb18819db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb6e520 0x5582feb825b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.420+0000 7fb18819db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1741bf360 con 0x5582fe9f64a0 2024-05-09T02:13:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.420+0000 7fb18819db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1741bec90 con 0x5582fe9f9f70 2024-05-09T02:13:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.420+0000 7fb18819db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1741bbe90 con 0x5582feb6e520 2024-05-09T02:13:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.420+0000 7fb18819db80 10 client.0 _ll_register_callbacks cb 0x5582fe9076c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.421+0000 7fb1737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f64a0 0x5582feb85130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.421+0000 7fb1727fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fe9f9f70 0x5582feb85670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.421+0000 7fb1737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f64a0 0x5582feb85130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48588/0 (socket says 172.21.15.19:48588) 2024-05-09T02:13:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.421+0000 7fb1737fe700 1 -- 172.21.15.19:0/2666805936 learned_addr learned my addr 172.21.15.19:0/2666805936 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.421+0000 7fb1737fe700 1 --2- 172.21.15.19:0/2666805936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb6e520 0x5582feb825b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:22.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.421+0000 7fb1737fe700 1 -- 172.21.15.19:0/2666805936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb6e520 msgr2=0x5582feb825b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:22.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.421+0000 7fb1737fe700 1 --2- 172.21.15.19:0/2666805936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb6e520 0x5582feb825b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.421+0000 7fb1737fe700 1 -- 172.21.15.19:0/2666805936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fe9f9f70 msgr2=0x5582feb85670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:22.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.421+0000 7fb1737fe700 1 --2- 172.21.15.19:0/2666805936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fe9f9f70 0x5582feb85670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.421+0000 7fb1737fe700 1 -- 172.21.15.19:0/2666805936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582feb82cb0 con 0x5582fe9f64a0 2024-05-09T02:13:22.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.421+0000 7fb1727fc700 1 --2- 172.21.15.19:0/2666805936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fe9f9f70 0x5582feb85670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:13:22.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.421+0000 7fb1737fe700 1 --2- 172.21.15.19:0/2666805936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f64a0 0x5582feb85130 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb16c007820 tx=0x7fb16c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4e8454e66b8859 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:22.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.421+0000 7fb15a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:13:22.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.422+0000 7fb15a7fc700 1 -- 172.21.15.19:0/2666805936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c0045e0 con 0x5582fe9f64a0 2024-05-09T02:13:22.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.422+0000 7fb18819db80 1 -- 172.21.15.19:0/2666805936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb164026070 con 0x5582fe9f64a0 2024-05-09T02:13:22.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.422+0000 7fb15a7fc700 1 -- 172.21.15.19:0/2666805936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fb16c009e00 con 0x5582fe9f64a0 2024-05-09T02:13:22.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.422+0000 7fb15a7fc700 1 -- 172.21.15.19:0/2666805936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb16c004bb0 con 0x5582fe9f64a0 2024-05-09T02:13:22.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.422+0000 7fb18819db80 10 client.15594 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:22.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.422+0000 7fb18819db80 1 -- 172.21.15.19:0/2666805936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5582febf6f90 con 0x5582fe9f64a0 2024-05-09T02:13:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.423+0000 7fb15a7fc700 1 -- 172.21.15.19:0/2666805936 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb164026070 con 0x5582fe9f64a0 2024-05-09T02:13:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.423+0000 7fb15a7fc700 1 -- 172.21.15.19:0/2666805936 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fb16c032030 con 0x5582fe9f64a0 2024-05-09T02:13:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.423+0000 7fb18819db80 10 client.15594 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.423+0000 7fb18819db80 10 client.15594 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:13:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.423+0000 7fb18819db80 20 client.15594 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.423+0000 7fb18819db80 1 --2- 172.21.15.19:0/2666805936 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5582fea70d00 0x5582febf94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.423+0000 7fb18819db80 4 client.15594 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:13:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.423+0000 7fb18819db80 1 -- 172.21.15.19:0/2666805936 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582fea9db20 con 0x5582fea70d00 2024-05-09T02:13:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.423+0000 7fb1727fc700 1 --2- 172.21.15.19:0/2666805936 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5582fea70d00 0x5582febf94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.424+0000 7fb15a7fc700 10 client.15594 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:13:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.425+0000 7fb1727fc700 1 --2- 172.21.15.19:0/2666805936 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5582fea70d00 0x5582febf94f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e434afc2ed2f11f0 server_cookie=1489cc64f36d1aea in_seq=0 out_seq=0 2024-05-09T02:13:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.425+0000 7fb15a7fc700 1 -- 172.21.15.19:0/2666805936 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582fea9db20 con 0x5582fea70d00 2024-05-09T02:13:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.425+0000 7fb15a7fc700 10 client.15594 handle_command_reply: tid=0 2024-05-09T02:13:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.425+0000 7fb18819db80 1 client.15594 shutdown 2024-05-09T02:13:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.425+0000 7fb18819db80 1 -- 172.21.15.19:0/2666805936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f64a0 msgr2=0x5582feb85130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.425+0000 7fb18819db80 1 --2- 172.21.15.19:0/2666805936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f64a0 0x5582feb85130 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb16c007820 tx=0x7fb16c0047c0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.425+0000 7fb18819db80 1 -- 172.21.15.19:0/2666805936 shutdown_connections 2024-05-09T02:13:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.425+0000 7fb18819db80 1 --2- 172.21.15.19:0/2666805936 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5582fea70d00 0x5582febf94f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.425+0000 7fb15a7fc700 0 client.15594 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:13:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb18819db80 1 --2- 172.21.15.19:0/2666805936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582feb6e520 0x5582feb825b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb18819db80 1 --2- 172.21.15.19:0/2666805936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5582fe9f64a0 0x5582feb85130 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb18819db80 1 --2- 172.21.15.19:0/2666805936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582fe9f9f70 0x5582feb85670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb18819db80 1 -- 172.21.15.19:0/2666805936 >> 172.21.15.19:0/2666805936 conn(0x5582feb7dc80 msgr2=0x5582feb68780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:22.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb18819db80 1 -- 172.21.15.19:0/2666805936 shutdown_connections 2024-05-09T02:13:22.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb18819db80 1 -- 172.21.15.19:0/2666805936 wait complete. 2024-05-09T02:13:22.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb18819db80 20 client.15594 trim_cache size 0 max 16384 2024-05-09T02:13:22.431 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:13:22.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb179787700 1 -- 172.21.15.19:0/4214212275 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb1600b2e70 msgr2=0x7fb1600b5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:22.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb179787700 1 --2- 172.21.15.19:0/4214212275 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb1600b2e70 0x7fb1600b5380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x5582feb6fc90 tx=0x7fb15c000bc0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb179787700 1 -- 172.21.15.19:0/4214212275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb17410efd0 msgr2=0x7fb1741ab530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:22.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb179787700 1 --2- 172.21.15.19:0/4214212275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb17410efd0 0x7fb1741ab530 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00e9a0 tx=0x7fb15c00ed60 comp rx=0 tx=0).stop 2024-05-09T02:13:22.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb179787700 1 -- 172.21.15.19:0/4214212275 shutdown_connections 2024-05-09T02:13:22.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb179787700 1 --2- 172.21.15.19:0/4214212275 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb1600b2e70 0x7fb1600b5380 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb179787700 1 --2- 172.21.15.19:0/4214212275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb174114250 0x7fb1741a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb179787700 1 --2- 172.21.15.19:0/4214212275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb17410efd0 0x7fb1741ab530 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.426+0000 7fb179787700 1 --2- 172.21.15.19:0/4214212275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb174113900 0x7fb1741aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:22.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.427+0000 7fb179787700 1 -- 172.21.15.19:0/4214212275 >> 172.21.15.19:0/4214212275 conn(0x7fb17410ab60 msgr2=0x7fb1741130a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:22.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.427+0000 7fb179787700 1 -- 172.21.15.19:0/4214212275 shutdown_connections 2024-05-09T02:13:22.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:22.427+0000 7fb179787700 1 -- 172.21.15.19:0/4214212275 wait complete. 2024-05-09T02:13:22.445 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:22 smithi019 ceph-mon[30151]: pgmap v65: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.4 MiB/s wr, 783 op/s 2024-05-09T02:13:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 100.0 s, 2628.6 tps, lat 37.943 ms stddev 39.365 2024-05-09T02:13:22.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:22 smithi059 ceph-mon[34781]: pgmap v65: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.4 MiB/s wr, 783 op/s 2024-05-09T02:13:22.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:22 smithi196 ceph-mon[34718]: pgmap v65: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.4 MiB/s wr, 783 op/s 2024-05-09T02:13:23.272 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T02:13:23.641 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:13:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:24 smithi059 ceph-mon[34781]: pgmap v66: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.4 MiB/s wr, 783 op/s 2024-05-09T02:13:24.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:24 smithi196 ceph-mon[34718]: pgmap v66: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.4 MiB/s wr, 783 op/s 2024-05-09T02:13:24.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:24 smithi019 ceph-mon[30151]: pgmap v66: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.4 MiB/s wr, 783 op/s 2024-05-09T02:13:25.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.618+0000 7f2f97b1e700 1 -- 172.21.15.19:0/940831335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90109860 msgr2=0x7f2f901111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:25.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.618+0000 7f2f97b1e700 1 --2- 172.21.15.19:0/940831335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90109860 0x7f2f901111c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800b0d0 tx=0x7f2f8800b3e0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.618+0000 7f2f97b1e700 1 -- 172.21.15.19:0/940831335 shutdown_connections 2024-05-09T02:13:25.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.618+0000 7f2f97b1e700 1 --2- 172.21.15.19:0/940831335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90109860 0x7f2f901111c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.618+0000 7f2f97b1e700 1 --2- 172.21.15.19:0/940831335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f901045e0 0x7f2f901049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.618+0000 7f2f97b1e700 1 --2- 172.21.15.19:0/940831335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f90108f10 0x7f2f90109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.618+0000 7f2f97b1e700 1 -- 172.21.15.19:0/940831335 >> 172.21.15.19:0/940831335 conn(0x7f2f900ffed0 msgr2=0x7f2f90102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:25.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.618+0000 7f2f97b1e700 1 -- 172.21.15.19:0/940831335 shutdown_connections 2024-05-09T02:13:25.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.618+0000 7f2f97b1e700 1 -- 172.21.15.19:0/940831335 wait complete. 2024-05-09T02:13:25.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.619+0000 7f2f97b1e700 1 Processor -- start 2024-05-09T02:13:25.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.619+0000 7f2f97b1e700 1 -- start start 2024-05-09T02:13:25.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.619+0000 7f2f97b1e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f901045e0 0x7f2f9019d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.619+0000 7f2f97b1e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f90108f10 0x7f2f9019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.619+0000 7f2f97b1e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90109860 0x7f2f901a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.619+0000 7f2f97b1e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90077900 con 0x7f2f90109860 2024-05-09T02:13:25.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.620+0000 7f2f97b1e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90077780 con 0x7f2f90108f10 2024-05-09T02:13:25.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.620+0000 7f2f97b1e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90077a80 con 0x7f2f901045e0 2024-05-09T02:13:25.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.620+0000 7f2f950b9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f90108f10 0x7f2f9019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.620+0000 7f2f950b9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f90108f10 0x7f2f9019d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:49804/0 (socket says 172.21.15.19:49804) 2024-05-09T02:13:25.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.620+0000 7f2f950b9700 1 -- 172.21.15.19:0/3226911148 learned_addr learned my addr 172.21.15.19:0/3226911148 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:25.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.620+0000 7f2f958ba700 1 --2- 172.21.15.19:0/3226911148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f901045e0 0x7f2f9019d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.620+0000 7f2f950b9700 1 -- 172.21.15.19:0/3226911148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f901045e0 msgr2=0x7f2f9019d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:25.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.620+0000 7f2f950b9700 1 --2- 172.21.15.19:0/3226911148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f901045e0 0x7f2f9019d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.620+0000 7f2f950b9700 1 -- 172.21.15.19:0/3226911148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90109860 msgr2=0x7f2f901a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:25.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.620+0000 7f2f950b9700 1 --2- 172.21.15.19:0/3226911148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90109860 0x7f2f901a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.620+0000 7f2f950b9700 1 -- 172.21.15.19:0/3226911148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f901a1f40 con 0x7f2f90108f10 2024-05-09T02:13:25.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.620+0000 7f2f950b9700 1 --2- 172.21.15.19:0/3226911148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f90108f10 0x7f2f9019d610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000b500 tx=0x7f2f8000b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:25.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.621+0000 7f2f86ffd700 1 -- 172.21.15.19:0/3226911148 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80021070 con 0x7f2f90108f10 2024-05-09T02:13:25.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.621+0000 7f2f86ffd700 1 -- 172.21.15.19:0/3226911148 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f80014e30 con 0x7f2f90108f10 2024-05-09T02:13:25.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.621+0000 7f2f86ffd700 1 -- 172.21.15.19:0/3226911148 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8001a740 con 0x7f2f90108f10 2024-05-09T02:13:25.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.621+0000 7f2f97b1e700 1 -- 172.21.15.19:0/3226911148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f901a2230 con 0x7f2f90108f10 2024-05-09T02:13:25.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.621+0000 7f2f97b1e700 1 -- 172.21.15.19:0/3226911148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f90113750 con 0x7f2f90108f10 2024-05-09T02:13:25.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.624+0000 7f2f86ffd700 1 -- 172.21.15.19:0/3226911148 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2f80024070 con 0x7f2f90108f10 2024-05-09T02:13:25.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.624+0000 7f2f86ffd700 1 --2- 172.21.15.19:0/3226911148 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2f7c0b2a70 0x7f2f7c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.624+0000 7f2f86ffd700 1 -- 172.21.15.19:0/3226911148 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f2f800e9a00 con 0x7f2f90108f10 2024-05-09T02:13:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.625+0000 7f2f958ba700 1 --2- 172.21.15.19:0/3226911148 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2f7c0b2a70 0x7f2f7c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.625+0000 7f2fa6534b80 1 Processor -- start 2024-05-09T02:13:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.625+0000 7f2fa6534b80 1 -- start start 2024-05-09T02:13:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.625+0000 7f2fa6534b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc73e7160 0x557bc74804f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.625+0000 7f2fa6534b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 0x557bc7480a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.625+0000 7f2fa6534b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc747a6a0 0x557bc747aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.625+0000 7f2f958ba700 1 --2- 172.21.15.19:0/3226911148 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2f7c0b2a70 0x7f2f7c0b4f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c005fd0 tx=0x7f2f8c005f60 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.625+0000 7f2fa6534b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557bc739cff0 con 0x557bc747a6a0 2024-05-09T02:13:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.625+0000 7f2fa6534b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557bc733b7f0 con 0x557bc73e7160 2024-05-09T02:13:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.625+0000 7f2fa6534b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557bc7398fb0 con 0x557bc72fd4a0 2024-05-09T02:13:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.625+0000 7f2f960bb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc747a6a0 0x557bc747aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.625+0000 7f2f960bb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc747a6a0 0x557bc747aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48632/0 (socket says 172.21.15.19:48632) 2024-05-09T02:13:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.625+0000 7f2f960bb700 1 -- 172.21.15.19:0/592834978 learned_addr learned my addr 172.21.15.19:0/592834978 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:25.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.626+0000 7f2f950b9700 1 --2- 172.21.15.19:0/592834978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 0x557bc7480a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.626+0000 7f2f958ba700 1 --2- 172.21.15.19:0/592834978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc73e7160 0x557bc74804f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.626+0000 7f2f960bb700 1 -- 172.21.15.19:0/592834978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 msgr2=0x557bc7480a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:25.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.626+0000 7f2f960bb700 1 --2- 172.21.15.19:0/592834978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 0x557bc7480a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.626+0000 7f2f960bb700 1 -- 172.21.15.19:0/592834978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc73e7160 msgr2=0x557bc74804f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:25.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.626+0000 7f2f960bb700 1 --2- 172.21.15.19:0/592834978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc73e7160 0x557bc74804f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.626+0000 7f2f960bb700 1 -- 172.21.15.19:0/592834978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bc7333370 con 0x557bc747a6a0 2024-05-09T02:13:25.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.626+0000 7f2f960bb700 1 --2- 172.21.15.19:0/592834978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc747a6a0 0x557bc747aad0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800cd70 tx=0x7f2f8800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680361604bcff8af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:25.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.627+0000 7f2f950b9700 1 --2- 172.21.15.19:0/592834978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 0x557bc7480a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:13:25.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.627+0000 7f2f7bfff700 1 -- 172.21.15.19:0/592834978 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f880090d0 con 0x557bc747a6a0 2024-05-09T02:13:25.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.627+0000 7f2f7bfff700 1 -- 172.21.15.19:0/592834978 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f88009270 con 0x557bc747a6a0 2024-05-09T02:13:25.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.627+0000 7f2f7bfff700 1 -- 172.21.15.19:0/592834978 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88007e00 con 0x557bc747a6a0 2024-05-09T02:13:25.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.628+0000 7f2fa6534b80 1 -- 172.21.15.19:0/592834978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc747a6a0 msgr2=0x557bc747aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:25.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.628+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/592834978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc747a6a0 0x557bc747aad0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800cd70 tx=0x7f2f8800cda0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.629+0000 7f2fa6534b80 1 -- 172.21.15.19:0/592834978 shutdown_connections 2024-05-09T02:13:25.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.629+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/592834978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc747a6a0 0x557bc747aad0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.629+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/592834978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc73e7160 0x557bc74804f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.629+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/592834978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 0x557bc7480a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.629+0000 7f2fa6534b80 1 -- 172.21.15.19:0/592834978 >> 172.21.15.19:0/592834978 conn(0x557bc735d6a0 msgr2=0x557bc7475400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:25.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.629+0000 7f2fa6534b80 1 -- 172.21.15.19:0/592834978 shutdown_connections 2024-05-09T02:13:25.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.629+0000 7f2fa6534b80 1 -- 172.21.15.19:0/592834978 wait complete. 2024-05-09T02:13:25.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.629+0000 7f2fa6534b80 1 -- start start 2024-05-09T02:13:25.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.630+0000 7f2fa6534b80 1 Processor -- start 2024-05-09T02:13:25.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.630+0000 7f2fa6534b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc72fd4a0 0x557bc7484240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.630+0000 7f2fa6534b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc73e7160 0x557bc7484780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.630+0000 7f2fa6534b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc747a6a0 0x557bc7484cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.630+0000 7f2fa6534b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557bc7218ba0 con 0x557bc72fd4a0 2024-05-09T02:13:25.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.630+0000 7f2fa6534b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557bc7357150 con 0x557bc747a6a0 2024-05-09T02:13:25.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.630+0000 7f2fa6534b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557bc74871d0 con 0x557bc73e7160 2024-05-09T02:13:25.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.630+0000 7f2fa6534b80 10 client.0 _ll_register_callbacks cb 0x557bc720e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:25.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.630+0000 7f2f958ba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc72fd4a0 0x557bc7484240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.630+0000 7f2f958ba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc72fd4a0 0x557bc7484240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48646/0 (socket says 172.21.15.19:48646) 2024-05-09T02:13:25.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.630+0000 7f2f958ba700 1 -- 172.21.15.19:0/3827135471 learned_addr learned my addr 172.21.15.19:0/3827135471 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:25.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.630+0000 7f2f950b9700 1 --2- 172.21.15.19:0/3827135471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc73e7160 0x557bc7484780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.630+0000 7f2f960bb700 1 --2- 172.21.15.19:0/3827135471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc747a6a0 0x557bc7484cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.631+0000 7f2f958ba700 1 -- 172.21.15.19:0/3827135471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc73e7160 msgr2=0x557bc7484780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:25.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.631+0000 7f2f958ba700 1 --2- 172.21.15.19:0/3827135471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc73e7160 0x557bc7484780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.631+0000 7f2f958ba700 1 -- 172.21.15.19:0/3827135471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc747a6a0 msgr2=0x557bc7484cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:25.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.631+0000 7f2f958ba700 1 --2- 172.21.15.19:0/3827135471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc747a6a0 0x557bc7484cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.631+0000 7f2f958ba700 1 -- 172.21.15.19:0/3827135471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bc7218490 con 0x557bc72fd4a0 2024-05-09T02:13:25.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.631+0000 7f2f960bb700 1 --2- 172.21.15.19:0/3827135471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc747a6a0 0x557bc7484cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:13:25.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.631+0000 7f2f950b9700 1 --2- 172.21.15.19:0/3827135471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc73e7160 0x557bc7484780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:13:25.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.631+0000 7f2f958ba700 1 --2- 172.21.15.19:0/3827135471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc72fd4a0 0x557bc7484240 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00ff60 tx=0x7f2f8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22237ec6085a84f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:25.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.631+0000 7f2f797fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:13:25.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.631+0000 7f2f797fa700 1 -- 172.21.15.19:0/3827135471 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c028470 con 0x557bc72fd4a0 2024-05-09T02:13:25.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.631+0000 7f2f797fa700 1 -- 172.21.15.19:0/3827135471 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f2f8c028610 con 0x557bc72fd4a0 2024-05-09T02:13:25.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.631+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3827135471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f88009270 con 0x557bc72fd4a0 2024-05-09T02:13:25.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.631+0000 7f2f797fa700 1 -- 172.21.15.19:0/3827135471 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2f8c028810 con 0x557bc72fd4a0 2024-05-09T02:13:25.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.632+0000 7f2fa6534b80 10 client.15603 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:25.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.632+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3827135471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557bc7491a20 con 0x557bc72fd4a0 2024-05-09T02:13:25.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.632+0000 7f2f797fa700 1 -- 172.21.15.19:0/3827135471 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f88009270 con 0x557bc72fd4a0 2024-05-09T02:13:25.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.632+0000 7f2f797fa700 1 -- 172.21.15.19:0/3827135471 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f2f8c01f550 con 0x557bc72fd4a0 2024-05-09T02:13:25.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.632+0000 7f2fa6534b80 10 client.15603 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:25.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.632+0000 7f2fa6534b80 10 client.15603 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:13:25.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.632+0000 7f2fa6534b80 20 client.15603 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:25.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.632+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/3827135471 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557bc7493fb0 0x557bc74963f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.632+0000 7f2fa6534b80 4 client.15603 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:13:25.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.632+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3827135471 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bc73aa9d0 con 0x557bc7493fb0 2024-05-09T02:13:25.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.632+0000 7f2f950b9700 1 --2- 172.21.15.19:0/3827135471 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557bc7493fb0 0x557bc74963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.633+0000 7f2f797fa700 10 client.15603 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:13:25.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.633+0000 7f2f950b9700 1 --2- 172.21.15.19:0/3827135471 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557bc7493fb0 0x557bc74963f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab61defb1f67a28a server_cookie=65d357b114f89cdb in_seq=0 out_seq=0 2024-05-09T02:13:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.634+0000 7f2f797fa700 1 -- 172.21.15.19:0/3827135471 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557bc73aa9d0 con 0x557bc7493fb0 2024-05-09T02:13:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.634+0000 7f2f797fa700 10 client.15603 handle_command_reply: tid=0 2024-05-09T02:13:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.634+0000 7f2fa6534b80 1 client.15603 shutdown 2024-05-09T02:13:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.634+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3827135471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc72fd4a0 msgr2=0x557bc7484240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.634+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/3827135471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc72fd4a0 0x557bc7484240 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00ff60 tx=0x7f2f8c004750 comp rx=0 tx=0).stop 2024-05-09T02:13:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.634+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3827135471 shutdown_connections 2024-05-09T02:13:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.634+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/3827135471 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557bc7493fb0 0x557bc74963f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.634+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/3827135471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc747a6a0 0x557bc7484cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.634+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/3827135471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc72fd4a0 0x557bc7484240 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.634+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/3827135471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc73e7160 0x557bc7484780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.634+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3827135471 >> 172.21.15.19:0/3827135471 conn(0x557bc735d6a0 msgr2=0x557bc74751c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.634+0000 7f2f797fa700 0 client.15603 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:13:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.635+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3827135471 shutdown_connections 2024-05-09T02:13:25.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.635+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3827135471 wait complete. 2024-05-09T02:13:25.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.635+0000 7f2fa6534b80 20 client.15603 trim_cache size 0 max 16384 2024-05-09T02:13:25.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.655+0000 7f2fa6534b80 1 Processor -- start 2024-05-09T02:13:25.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.655+0000 7f2fa6534b80 1 -- start start 2024-05-09T02:13:25.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.655+0000 7f2fa6534b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 0x557bc7483260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.655+0000 7f2fa6534b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc73e7160 0x557bc7490010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.655+0000 7f2fa6534b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc747a6a0 0x557bc7484200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.655+0000 7f2fa6534b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901acc70 con 0x557bc747a6a0 2024-05-09T02:13:25.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.655+0000 7f2fa6534b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901ae870 con 0x557bc73e7160 2024-05-09T02:13:25.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.655+0000 7f2fa6534b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901ad340 con 0x557bc72fd4a0 2024-05-09T02:13:25.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.655+0000 7f2f950b9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc73e7160 0x557bc7490010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.655+0000 7f2f960bb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc747a6a0 0x557bc7484200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.655+0000 7f2f958ba700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 0x557bc7483260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.655+0000 7f2f950b9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc73e7160 0x557bc7490010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:49838/0 (socket says 172.21.15.19:49838) 2024-05-09T02:13:25.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.655+0000 7f2f950b9700 1 -- 172.21.15.19:0/2823759356 learned_addr learned my addr 172.21.15.19:0/2823759356 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:25.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.656+0000 7f2f950b9700 1 -- 172.21.15.19:0/2823759356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 msgr2=0x557bc7483260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:25.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.656+0000 7f2f950b9700 1 --2- 172.21.15.19:0/2823759356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 0x557bc7483260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.656+0000 7f2f950b9700 1 -- 172.21.15.19:0/2823759356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc747a6a0 msgr2=0x557bc7484200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:25.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.656+0000 7f2f950b9700 1 --2- 172.21.15.19:0/2823759356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc747a6a0 0x557bc7484200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.656+0000 7f2f950b9700 1 -- 172.21.15.19:0/2823759356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bc747ad00 con 0x557bc73e7160 2024-05-09T02:13:25.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.656+0000 7f2f950b9700 1 --2- 172.21.15.19:0/2823759356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc73e7160 0x557bc7490010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f800b0a80 tx=0x7f2f800acd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=724b9d3b3b52a32f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:25.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.656+0000 7f2f7b7fe700 1 -- 172.21.15.19:0/2823759356 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f800cf070 con 0x557bc73e7160 2024-05-09T02:13:25.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.656+0000 7f2f7b7fe700 1 -- 172.21.15.19:0/2823759356 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f800b5180 con 0x557bc73e7160 2024-05-09T02:13:25.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.656+0000 7f2f7b7fe700 1 -- 172.21.15.19:0/2823759356 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f800b4040 con 0x557bc73e7160 2024-05-09T02:13:25.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.657+0000 7f2fa6534b80 1 -- 172.21.15.19:0/2823759356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc73e7160 msgr2=0x557bc7490010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:25.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.657+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/2823759356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc73e7160 0x557bc7490010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f800b0a80 tx=0x7f2f800acd10 comp rx=0 tx=0).stop 2024-05-09T02:13:25.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.657+0000 7f2fa6534b80 1 -- 172.21.15.19:0/2823759356 shutdown_connections 2024-05-09T02:13:25.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.657+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/2823759356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc747a6a0 0x557bc7484200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.657+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/2823759356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 0x557bc7483260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.657+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/2823759356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc73e7160 0x557bc7490010 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.657+0000 7f2fa6534b80 1 -- 172.21.15.19:0/2823759356 >> 172.21.15.19:0/2823759356 conn(0x557bc735d6a0 msgr2=0x557bc7482b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:25.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.658+0000 7f2fa6534b80 1 -- 172.21.15.19:0/2823759356 shutdown_connections 2024-05-09T02:13:25.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.658+0000 7f2fa6534b80 1 -- 172.21.15.19:0/2823759356 wait complete. 2024-05-09T02:13:25.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.658+0000 7f2fa6534b80 1 -- start start 2024-05-09T02:13:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.658+0000 7f2fa6534b80 1 Processor -- start 2024-05-09T02:13:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.659+0000 7f2fa6534b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 0x557bc748c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.659+0000 7f2fa6534b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc73e7160 0x557bc748c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.659+0000 7f2fa6534b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc747a6a0 0x557bc749aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.659+0000 7f2f958ba700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 0x557bc748c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.659+0000 7f2f958ba700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 0x557bc748c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:39160/0 (socket says 172.21.15.19:39160) 2024-05-09T02:13:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.659+0000 7f2f960bb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc747a6a0 0x557bc749aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.659+0000 7f2f950b9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc73e7160 0x557bc748c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.659+0000 7f2f950b9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc73e7160 0x557bc748c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48672/0 (socket says 172.21.15.19:48672) 2024-05-09T02:13:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.659+0000 7f2f960bb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc747a6a0 0x557bc749aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:49840/0 (socket says 172.21.15.19:49840) 2024-05-09T02:13:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.659+0000 7f2f958ba700 1 -- 172.21.15.19:0/3034705826 learned_addr learned my addr 172.21.15.19:0/3034705826 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.659+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3034705826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901ae0e0 con 0x557bc73e7160 2024-05-09T02:13:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.659+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3034705826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901ada10 con 0x557bc747a6a0 2024-05-09T02:13:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.659+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3034705826 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901aad60 con 0x557bc72fd4a0 2024-05-09T02:13:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.660+0000 7f2fa6534b80 10 client.0 _ll_register_callbacks cb 0x557bc720e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.660+0000 7f2f958ba700 1 -- 172.21.15.19:0/3034705826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc747a6a0 msgr2=0x557bc749aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.660+0000 7f2f958ba700 1 --2- 172.21.15.19:0/3034705826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc747a6a0 0x557bc749aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.660+0000 7f2f958ba700 1 -- 172.21.15.19:0/3034705826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc73e7160 msgr2=0x557bc748c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.660+0000 7f2f958ba700 1 --2- 172.21.15.19:0/3034705826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc73e7160 0x557bc748c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.660+0000 7f2f958ba700 1 -- 172.21.15.19:0/3034705826 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bc746ef60 con 0x557bc72fd4a0 2024-05-09T02:13:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.660+0000 7f2f950b9700 1 --2- 172.21.15.19:0/3034705826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc73e7160 0x557bc748c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:13:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.660+0000 7f2f958ba700 1 --2- 172.21.15.19:0/3034705826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 0x557bc748c300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00f8d0 tx=0x7f2f8c00f760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c1318bec32ff732 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.660+0000 7f2f797fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:13:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.662+0000 7f2f797fa700 1 -- 172.21.15.19:0/3034705826 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c01f630 con 0x557bc72fd4a0 2024-05-09T02:13:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.662+0000 7f2f797fa700 1 -- 172.21.15.19:0/3034705826 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f2f8c0289b0 con 0x557bc72fd4a0 2024-05-09T02:13:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.662+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3034705826 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f800b5180 con 0x557bc72fd4a0 2024-05-09T02:13:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.662+0000 7f2f797fa700 1 -- 172.21.15.19:0/3034705826 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2f8c028bd0 con 0x557bc72fd4a0 2024-05-09T02:13:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.662+0000 7f2fa6534b80 10 client.25339 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.662+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3034705826 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557bc74fd9c0 con 0x557bc72fd4a0 2024-05-09T02:13:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.663+0000 7f2f797fa700 1 -- 172.21.15.19:0/3034705826 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f800b5180 con 0x557bc72fd4a0 2024-05-09T02:13:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.663+0000 7f2f797fa700 1 -- 172.21.15.19:0/3034705826 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f2f8c031430 con 0x557bc72fd4a0 2024-05-09T02:13:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.663+0000 7f2fa6534b80 10 client.25339 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.663+0000 7f2fa6534b80 10 client.25339 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:13:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.663+0000 7f2fa6534b80 20 client.25339 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.663+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/3034705826 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557bc74fe070 0x557bc7500490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.663+0000 7f2fa6534b80 4 client.25339 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:13:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.663+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3034705826 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557bc73a4a70 con 0x557bc74fe070 2024-05-09T02:13:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.664+0000 7f2f950b9700 1 --2- 172.21.15.19:0/3034705826 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557bc74fe070 0x557bc7500490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.664+0000 7f2f797fa700 10 client.25339 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:13:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.664+0000 7f2f950b9700 1 --2- 172.21.15.19:0/3034705826 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557bc74fe070 0x557bc7500490 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf744db6db65ed79 server_cookie=5d531093fb7fed1b in_seq=0 out_seq=0 2024-05-09T02:13:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.664+0000 7f2f797fa700 1 -- 172.21.15.19:0/3034705826 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557bc73a4a70 con 0x557bc74fe070 2024-05-09T02:13:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.664+0000 7f2f797fa700 10 client.25339 handle_command_reply: tid=0 2024-05-09T02:13:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.664+0000 7f2fa6534b80 1 client.25339 shutdown 2024-05-09T02:13:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.664+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3034705826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 msgr2=0x557bc748c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.664+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/3034705826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 0x557bc748c300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00f8d0 tx=0x7f2f8c00f760 comp rx=0 tx=0).stop 2024-05-09T02:13:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.664+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3034705826 shutdown_connections 2024-05-09T02:13:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.664+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/3034705826 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557bc74fe070 0x557bc7500490 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.664+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/3034705826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bc747a6a0 0x557bc749aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.664+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/3034705826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bc72fd4a0 0x557bc748c300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.664+0000 7f2fa6534b80 1 --2- 172.21.15.19:0/3034705826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bc73e7160 0x557bc748c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.664+0000 7f2f797fa700 0 client.25339 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:13:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.664+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3034705826 >> 172.21.15.19:0/3034705826 conn(0x557bc735d6a0 msgr2=0x557bc747fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.665+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3034705826 shutdown_connections 2024-05-09T02:13:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.665+0000 7f2fa6534b80 1 -- 172.21.15.19:0/3034705826 wait complete. 2024-05-09T02:13:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.665+0000 7f2fa6534b80 20 client.25339 trim_cache size 0 max 16384 2024-05-09T02:13:25.670 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:13:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.665+0000 7f2f97b1e700 1 -- 172.21.15.19:0/3226911148 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2f7c0b2a70 msgr2=0x7f2f7c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.665+0000 7f2f97b1e700 1 --2- 172.21.15.19:0/3226911148 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2f7c0b2a70 0x7f2f7c0b4f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c005fd0 tx=0x7f2f8c005f60 comp rx=0 tx=0).stop 2024-05-09T02:13:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.665+0000 7f2f97b1e700 1 -- 172.21.15.19:0/3226911148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f90108f10 msgr2=0x7f2f9019d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.665+0000 7f2f97b1e700 1 --2- 172.21.15.19:0/3226911148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f90108f10 0x7f2f9019d610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000b500 tx=0x7f2f8000b8c0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.665+0000 7f2f97b1e700 1 -- 172.21.15.19:0/3226911148 shutdown_connections 2024-05-09T02:13:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.665+0000 7f2f97b1e700 1 --2- 172.21.15.19:0/3226911148 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2f7c0b2a70 0x7f2f7c0b4f80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.665+0000 7f2f97b1e700 1 --2- 172.21.15.19:0/3226911148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f90109860 0x7f2f901a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.665+0000 7f2f97b1e700 1 --2- 172.21.15.19:0/3226911148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f901045e0 0x7f2f9019d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.665+0000 7f2f97b1e700 1 --2- 172.21.15.19:0/3226911148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f90108f10 0x7f2f9019d610 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:25.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.665+0000 7f2f97b1e700 1 -- 172.21.15.19:0/3226911148 >> 172.21.15.19:0/3226911148 conn(0x7f2f900ffed0 msgr2=0x7f2f901061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:25.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.666+0000 7f2f97b1e700 1 -- 172.21.15.19:0/3226911148 shutdown_connections 2024-05-09T02:13:25.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:25.666+0000 7f2f97b1e700 1 -- 172.21.15.19:0/3226911148 wait complete. 2024-05-09T02:13:26.604 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T02:13:26.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:26 smithi019 ceph-mon[30151]: pgmap v67: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-05-09T02:13:26.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:26 smithi059 ceph-mon[34781]: pgmap v67: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-05-09T02:13:26.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:26 smithi196 ceph-mon[34718]: pgmap v67: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-05-09T02:13:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:13:27.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 105.0 s, 2736.6 tps, lat 36.604 ms stddev 38.893 2024-05-09T02:13:28.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:28 smithi019 ceph-mon[30151]: pgmap v68: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 MiB/s wr, 664 op/s 2024-05-09T02:13:28.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:28 smithi059 ceph-mon[34781]: pgmap v68: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 MiB/s wr, 664 op/s 2024-05-09T02:13:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:28 smithi196 ceph-mon[34718]: pgmap v68: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 MiB/s wr, 664 op/s 2024-05-09T02:13:28.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.874+0000 7fb98d59a700 1 -- 172.21.15.19:0/2832217153 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988009480 con 0x7fb990108740 2024-05-09T02:13:28.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.877+0000 7fb994f85700 1 -- 172.21.15.19:0/2832217153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb990108740 msgr2=0x7fb9901100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:28.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.877+0000 7fb994f85700 1 --2- 172.21.15.19:0/2832217153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb990108740 0x7fb9901100a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb98800b0d0 tx=0x7fb98800b3e0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.877+0000 7fb994f85700 1 -- 172.21.15.19:0/2832217153 shutdown_connections 2024-05-09T02:13:28.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.877+0000 7fb994f85700 1 --2- 172.21.15.19:0/2832217153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb990108740 0x7fb9901100a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.877+0000 7fb994f85700 1 --2- 172.21.15.19:0/2832217153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9901034c0 0x7fb9901038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.877+0000 7fb994f85700 1 --2- 172.21.15.19:0/2832217153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb990107df0 0x7fb990108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.877+0000 7fb994f85700 1 -- 172.21.15.19:0/2832217153 >> 172.21.15.19:0/2832217153 conn(0x7fb9900fedc0 msgr2=0x7fb9901011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.878+0000 7fb994f85700 1 -- 172.21.15.19:0/2832217153 shutdown_connections 2024-05-09T02:13:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.878+0000 7fb994f85700 1 -- 172.21.15.19:0/2832217153 wait complete. 2024-05-09T02:13:28.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.879+0000 7fb994f85700 1 Processor -- start 2024-05-09T02:13:28.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.879+0000 7fb994f85700 1 -- start start 2024-05-09T02:13:28.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.879+0000 7fb994f85700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9901034c0 0x7fb99010a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.879+0000 7fb994f85700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb990107df0 0x7fb99010a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.879+0000 7fb994f85700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb990108740 0x7fb99010aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.879+0000 7fb994f85700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb990113410 con 0x7fb990107df0 2024-05-09T02:13:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.879+0000 7fb98e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9901034c0 0x7fb99010a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.879+0000 7fb994f85700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb990113290 con 0x7fb9901034c0 2024-05-09T02:13:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.879+0000 7fb994f85700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb990113590 con 0x7fb990108740 2024-05-09T02:13:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.879+0000 7fb98dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb990107df0 0x7fb99010a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.879+0000 7fb98e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9901034c0 0x7fb99010a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:47162/0 (socket says 172.21.15.19:47162) 2024-05-09T02:13:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.879+0000 7fb98e59c700 1 -- 172.21.15.19:0/2425856513 learned_addr learned my addr 172.21.15.19:0/2425856513 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.880+0000 7fb98ed9d700 1 --2- 172.21.15.19:0/2425856513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb990108740 0x7fb99010aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.880+0000 7fb98ed9d700 1 -- 172.21.15.19:0/2425856513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9901034c0 msgr2=0x7fb99010a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:28.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.880+0000 7fb98ed9d700 1 --2- 172.21.15.19:0/2425856513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9901034c0 0x7fb99010a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.880+0000 7fb98ed9d700 1 -- 172.21.15.19:0/2425856513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb990107df0 msgr2=0x7fb99010a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.880+0000 7fb98ed9d700 1 --2- 172.21.15.19:0/2425856513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb990107df0 0x7fb99010a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.880+0000 7fb98ed9d700 1 -- 172.21.15.19:0/2425856513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9901ad560 con 0x7fb990108740 2024-05-09T02:13:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.880+0000 7fb98dd9b700 1 --2- 172.21.15.19:0/2425856513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb990107df0 0x7fb99010a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:13:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.881+0000 7fb98e59c700 1 --2- 172.21.15.19:0/2425856513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9901034c0 0x7fb99010a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:13:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.881+0000 7fb98ed9d700 1 --2- 172.21.15.19:0/2425856513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb990108740 0x7fb99010aba0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb98800b0d0 tx=0x7fb988007ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.881+0000 7fb97f7fe700 1 -- 172.21.15.19:0/2425856513 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98801f070 con 0x7fb990108740 2024-05-09T02:13:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.881+0000 7fb97f7fe700 1 -- 172.21.15.19:0/2425856513 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb988007d90 con 0x7fb990108740 2024-05-09T02:13:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.881+0000 7fb97f7fe700 1 -- 172.21.15.19:0/2425856513 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98801b690 con 0x7fb990108740 2024-05-09T02:13:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.881+0000 7fb994f85700 1 -- 172.21.15.19:0/2425856513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9901ad790 con 0x7fb990108740 2024-05-09T02:13:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.882+0000 7fb994f85700 1 -- 172.21.15.19:0/2425856513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9901adb90 con 0x7fb990108740 2024-05-09T02:13:28.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.886+0000 7fb97f7fe700 1 -- 172.21.15.19:0/2425856513 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb988022070 con 0x7fb990108740 2024-05-09T02:13:28.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.887+0000 7fb97f7fe700 1 --2- 172.21.15.19:0/2425856513 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb9780b2b40 0x7fb9780b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.887+0000 7fb97f7fe700 1 -- 172.21.15.19:0/2425856513 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fb98800f020 con 0x7fb990108740 2024-05-09T02:13:28.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.888+0000 7fb98e59c700 1 --2- 172.21.15.19:0/2425856513 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb9780b2b40 0x7fb9780b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.889+0000 7fb9a399bb80 1 Processor -- start 2024-05-09T02:13:28.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.889+0000 7fb98e59c700 1 --2- 172.21.15.19:0/2425856513 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb9780b2b40 0x7fb9780b5050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb980009d80 tx=0x7fb980009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.889+0000 7fb9a399bb80 1 -- start start 2024-05-09T02:13:28.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.889+0000 7fb9a399bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e252974f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.889+0000 7fb9a399bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e251144a0 0x559e25297a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.889+0000 7fb9a399bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e252916a0 0x559e25291ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.890+0000 7fb98dd9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e251144a0 0x559e25297a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.890+0000 7fb98dd9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e251144a0 0x559e25297a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:33376/0 (socket says 172.21.15.19:33376) 2024-05-09T02:13:28.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.890+0000 7fb98dd9b700 1 -- 172.21.15.19:0/3486230755 learned_addr learned my addr 172.21.15.19:0/3486230755 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:28.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.890+0000 7fb98e59c700 1 --2- 172.21.15.19:0/3486230755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e252974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.890+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3486230755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559e251b3ff0 con 0x559e251fe160 2024-05-09T02:13:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.890+0000 7fb98ed9d700 1 --2- 172.21.15.19:0/3486230755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e252916a0 0x559e25291ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.890+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3486230755 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559e251527f0 con 0x559e252916a0 2024-05-09T02:13:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.890+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3486230755 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559e251affb0 con 0x559e251144a0 2024-05-09T02:13:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.890+0000 7fb98dd9b700 1 -- 172.21.15.19:0/3486230755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e252916a0 msgr2=0x559e25291ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.890+0000 7fb98dd9b700 1 --2- 172.21.15.19:0/3486230755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e252916a0 0x559e25291ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.890+0000 7fb98dd9b700 1 -- 172.21.15.19:0/3486230755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 msgr2=0x559e252974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.890+0000 7fb98dd9b700 1 --2- 172.21.15.19:0/3486230755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e252974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.890+0000 7fb98dd9b700 1 -- 172.21.15.19:0/3486230755 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e2514a370 con 0x559e251144a0 2024-05-09T02:13:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.891+0000 7fb98e59c700 1 --2- 172.21.15.19:0/3486230755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e252974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:13:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.891+0000 7fb98dd9b700 1 --2- 172.21.15.19:0/3486230755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e251144a0 0x559e25297a30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb984009d20 tx=0x7fb9840106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa363fcc7656d266 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.891+0000 7fb97cff9700 1 -- 172.21.15.19:0/3486230755 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984010e10 con 0x559e251144a0 2024-05-09T02:13:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.891+0000 7fb97cff9700 1 -- 172.21.15.19:0/3486230755 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9840044d0 con 0x559e251144a0 2024-05-09T02:13:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.891+0000 7fb97cff9700 1 -- 172.21.15.19:0/3486230755 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984011440 con 0x559e251144a0 2024-05-09T02:13:28.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.893+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3486230755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e251144a0 msgr2=0x559e25297a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:28.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.893+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3486230755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e251144a0 0x559e25297a30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb984009d20 tx=0x7fb9840106b0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.893+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3486230755 shutdown_connections 2024-05-09T02:13:28.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.893+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3486230755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e252916a0 0x559e25291ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.893+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3486230755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e252974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.893+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3486230755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e251144a0 0x559e25297a30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.893+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3486230755 >> 172.21.15.19:0/3486230755 conn(0x559e251746a0 msgr2=0x559e2528c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:28.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.893+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3486230755 shutdown_connections 2024-05-09T02:13:28.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.893+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3486230755 wait complete. 2024-05-09T02:13:28.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.894+0000 7fb9a399bb80 1 -- start start 2024-05-09T02:13:28.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.894+0000 7fb9a399bb80 1 Processor -- start 2024-05-09T02:13:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.897+0000 7fb9a399bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 0x559e2529b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.897+0000 7fb9a399bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e2529b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.897+0000 7fb9a399bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e2529bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.897+0000 7fb9a399bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559e2502fba0 con 0x559e251fe160 2024-05-09T02:13:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.897+0000 7fb9a399bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559e2516e150 con 0x559e251144a0 2024-05-09T02:13:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.897+0000 7fb9a399bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559e2529e1d0 con 0x559e252916a0 2024-05-09T02:13:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.897+0000 7fb9a399bb80 10 client.0 _ll_register_callbacks cb 0x559e250256c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.897+0000 7fb98ed9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e2529bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.897+0000 7fb98ed9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e2529bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:33380/0 (socket says 172.21.15.19:33380) 2024-05-09T02:13:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.897+0000 7fb98ed9d700 1 -- 172.21.15.19:0/3460010585 learned_addr learned my addr 172.21.15.19:0/3460010585 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.898+0000 7fb98dd9b700 1 --2- 172.21.15.19:0/3460010585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e2529b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.898+0000 7fb98ed9d700 1 -- 172.21.15.19:0/3460010585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 msgr2=0x559e2529b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.898+0000 7fb98e59c700 1 --2- 172.21.15.19:0/3460010585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 0x559e2529b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.898+0000 7fb98ed9d700 1 --2- 172.21.15.19:0/3460010585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 0x559e2529b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.898+0000 7fb98ed9d700 1 -- 172.21.15.19:0/3460010585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 msgr2=0x559e2529b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.898+0000 7fb98ed9d700 1 --2- 172.21.15.19:0/3460010585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e2529b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.898+0000 7fb98ed9d700 1 -- 172.21.15.19:0/3460010585 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e2502f490 con 0x559e252916a0 2024-05-09T02:13:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.898+0000 7fb98dd9b700 1 --2- 172.21.15.19:0/3460010585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e2529b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:13:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.898+0000 7fb98ed9d700 1 --2- 172.21.15.19:0/3460010585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e2529bc30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb988007f40 tx=0x7fb9880aa190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=701977176f4f2e86 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.898+0000 7fb975ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:13:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.898+0000 7fb975ffb700 1 -- 172.21.15.19:0/3460010585 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9880075c0 con 0x559e252916a0 2024-05-09T02:13:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.899+0000 7fb975ffb700 1 -- 172.21.15.19:0/3460010585 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fb9880b10e0 con 0x559e252916a0 2024-05-09T02:13:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.899+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3460010585 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9840044d0 con 0x559e252916a0 2024-05-09T02:13:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.899+0000 7fb975ffb700 1 -- 172.21.15.19:0/3460010585 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb9880cd2d0 con 0x559e252916a0 2024-05-09T02:13:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.899+0000 7fb9a399bb80 10 client.25351 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.899+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3460010585 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559e252a8980 con 0x559e252916a0 2024-05-09T02:13:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.899+0000 7fb975ffb700 1 -- 172.21.15.19:0/3460010585 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9880aa530 con 0x559e252916a0 2024-05-09T02:13:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.899+0000 7fb975ffb700 1 -- 172.21.15.19:0/3460010585 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fb988011070 con 0x559e252916a0 2024-05-09T02:13:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.900+0000 7fb9a399bb80 10 client.25351 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.900+0000 7fb9a399bb80 10 client.25351 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:13:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.900+0000 7fb9a399bb80 20 client.25351 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.900+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3460010585 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x559e252aaf90 0x559e252ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.900+0000 7fb9a399bb80 4 client.25351 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:13:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.900+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3460010585 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb9840044d0 con 0x559e252aaf90 2024-05-09T02:13:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.900+0000 7fb98e59c700 1 --2- 172.21.15.19:0/3460010585 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x559e252aaf90 0x559e252ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.900+0000 7fb98e59c700 1 --2- 172.21.15.19:0/3460010585 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x559e252aaf90 0x559e252ad3b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=465c0704ee32e537 server_cookie=d3d2a399849e40a3 in_seq=0 out_seq=0 2024-05-09T02:13:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.900+0000 7fb975ffb700 10 client.25351 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:13:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.901+0000 7fb975ffb700 1 -- 172.21.15.19:0/3460010585 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fb9840044d0 con 0x559e252aaf90 2024-05-09T02:13:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.901+0000 7fb975ffb700 10 client.25351 handle_command_reply: tid=0 2024-05-09T02:13:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.901+0000 7fb9a399bb80 1 client.25351 shutdown 2024-05-09T02:13:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.901+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3460010585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 msgr2=0x559e2529bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.901+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3460010585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e2529bc30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb988007f40 tx=0x7fb9880aa190 comp rx=0 tx=0).stop 2024-05-09T02:13:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.901+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3460010585 shutdown_connections 2024-05-09T02:13:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.901+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3460010585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e2529bc30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.901+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3460010585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 0x559e2529b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.901+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3460010585 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x559e252aaf90 0x559e252ad3b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.901+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3460010585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e2529b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.901+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3460010585 >> 172.21.15.19:0/3460010585 conn(0x559e251746a0 msgr2=0x559e2528c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.901+0000 7fb975ffb700 0 client.25351 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:13:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.902+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3460010585 shutdown_connections 2024-05-09T02:13:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.902+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3460010585 wait complete. 2024-05-09T02:13:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.902+0000 7fb9a399bb80 20 client.25351 trim_cache size 0 max 16384 2024-05-09T02:13:28.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.934+0000 7fb9a399bb80 1 Processor -- start 2024-05-09T02:13:28.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.934+0000 7fb9a399bb80 1 -- start start 2024-05-09T02:13:28.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.934+0000 7fb9a399bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 0x559e252969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.934+0000 7fb9a399bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e2529b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.935+0000 7fb9a399bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e252a6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.935+0000 7fb9a399bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9901b52f0 con 0x559e251fe160 2024-05-09T02:13:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.935+0000 7fb9a399bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9901b6ef0 con 0x559e251144a0 2024-05-09T02:13:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.935+0000 7fb9a399bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9901b59c0 con 0x559e252916a0 2024-05-09T02:13:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.935+0000 7fb98ed9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e252a6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.935+0000 7fb98dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e2529b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.935+0000 7fb98ed9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e252a6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:33394/0 (socket says 172.21.15.19:33394) 2024-05-09T02:13:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.935+0000 7fb98ed9d700 1 -- 172.21.15.19:0/3886701742 learned_addr learned my addr 172.21.15.19:0/3886701742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.935+0000 7fb98ed9d700 1 -- 172.21.15.19:0/3886701742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 msgr2=0x559e252969f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:13:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.935+0000 7fb98ed9d700 1 --2- 172.21.15.19:0/3886701742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 0x559e252969f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.935+0000 7fb98ed9d700 1 -- 172.21.15.19:0/3886701742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 msgr2=0x559e2529b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.935+0000 7fb98ed9d700 1 --2- 172.21.15.19:0/3886701742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e2529b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.935+0000 7fb98ed9d700 1 -- 172.21.15.19:0/3886701742 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e25291fd0 con 0x559e252916a0 2024-05-09T02:13:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.936+0000 7fb98ed9d700 1 --2- 172.21.15.19:0/3886701742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e252a6fd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb9880b1330 tx=0x7fb988009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e78c3265ac6e69a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.936+0000 7fb977fff700 1 -- 172.21.15.19:0/3886701742 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9880aa530 con 0x559e252916a0 2024-05-09T02:13:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.936+0000 7fb977fff700 1 -- 172.21.15.19:0/3886701742 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9880ab120 con 0x559e252916a0 2024-05-09T02:13:28.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.936+0000 7fb977fff700 1 -- 172.21.15.19:0/3886701742 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9880b33b0 con 0x559e252916a0 2024-05-09T02:13:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.938+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3886701742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 msgr2=0x559e252a6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.938+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3886701742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e252a6fd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb9880b1330 tx=0x7fb988009960 comp rx=0 tx=0).stop 2024-05-09T02:13:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.938+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3886701742 shutdown_connections 2024-05-09T02:13:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.938+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3886701742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e252a6fd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.938+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3886701742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 0x559e252969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.938+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3886701742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e2529b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.938+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3886701742 >> 172.21.15.19:0/3886701742 conn(0x559e251746a0 msgr2=0x559e2529b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.940+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3886701742 shutdown_connections 2024-05-09T02:13:28.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.940+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3886701742 wait complete. 2024-05-09T02:13:28.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.941+0000 7fb9a399bb80 1 -- start start 2024-05-09T02:13:28.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.941+0000 7fb9a399bb80 1 Processor -- start 2024-05-09T02:13:28.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb9a399bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 0x559e252a3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb9a399bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e252b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb9a399bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e252b2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb9a399bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9901b6760 con 0x559e251fe160 2024-05-09T02:13:28.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb9a399bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9901b6090 con 0x559e251144a0 2024-05-09T02:13:28.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb9a399bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9901b3290 con 0x559e252916a0 2024-05-09T02:13:28.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb98e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 0x559e252a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb9a399bb80 10 client.0 _ll_register_callbacks cb 0x559e250256c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:28.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb98e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 0x559e252a3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:47200/0 (socket says 172.21.15.19:47200) 2024-05-09T02:13:28.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb98ed9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e252b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb98e59c700 1 -- 172.21.15.19:0/3759534454 learned_addr learned my addr 172.21.15.19:0/3759534454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:28.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb98e59c700 1 -- 172.21.15.19:0/3759534454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 msgr2=0x559e252b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:28.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb98e59c700 1 --2- 172.21.15.19:0/3759534454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e252b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb98e59c700 1 -- 172.21.15.19:0/3759534454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 msgr2=0x559e252b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:28.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb98e59c700 1 --2- 172.21.15.19:0/3759534454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e252b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.942+0000 7fb98e59c700 1 -- 172.21.15.19:0/3759534454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e252b27a0 con 0x559e251144a0 2024-05-09T02:13:28.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.943+0000 7fb98e59c700 1 --2- 172.21.15.19:0/3759534454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 0x559e252a3550 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x559e2529aa30 tx=0x7fb9800047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93735990262ef66a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:28.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.943+0000 7fb975ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:13:28.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.943+0000 7fb975ffb700 1 -- 172.21.15.19:0/3759534454 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98002a7b0 con 0x559e251144a0 2024-05-09T02:13:28.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.943+0000 7fb975ffb700 1 -- 172.21.15.19:0/3759534454 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fb98002a950 con 0x559e251144a0 2024-05-09T02:13:28.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.943+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3759534454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9880ab120 con 0x559e251144a0 2024-05-09T02:13:28.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.944+0000 7fb975ffb700 1 -- 172.21.15.19:0/3759534454 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb98003a050 con 0x559e251144a0 2024-05-09T02:13:28.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.944+0000 7fb9a399bb80 10 client.25304 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:28.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.944+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3759534454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559e2530d970 con 0x559e251144a0 2024-05-09T02:13:28.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.945+0000 7fb975ffb700 1 -- 172.21.15.19:0/3759534454 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9880ab120 con 0x559e251144a0 2024-05-09T02:13:28.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.945+0000 7fb975ffb700 1 -- 172.21.15.19:0/3759534454 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fb980020460 con 0x559e251144a0 2024-05-09T02:13:28.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.945+0000 7fb9a399bb80 10 client.25304 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:28.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.945+0000 7fb9a399bb80 10 client.25304 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:13:28.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.945+0000 7fb9a399bb80 20 client.25304 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:28.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.945+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3759534454 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x559e2530e080 0x559e253104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:28.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.945+0000 7fb9a399bb80 4 client.25304 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:13:28.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.945+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3759534454 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e251c19d0 con 0x559e2530e080 2024-05-09T02:13:28.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.946+0000 7fb98dd9b700 1 --2- 172.21.15.19:0/3759534454 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x559e2530e080 0x559e253104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:28.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.946+0000 7fb975ffb700 10 client.25304 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:13:28.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.947+0000 7fb98dd9b700 1 --2- 172.21.15.19:0/3759534454 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x559e2530e080 0x559e253104a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aac1e20e12bb3e64 server_cookie=d9d69f3bcb892b48 in_seq=0 out_seq=0 2024-05-09T02:13:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.947+0000 7fb975ffb700 1 -- 172.21.15.19:0/3759534454 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e251c19d0 con 0x559e2530e080 2024-05-09T02:13:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.947+0000 7fb975ffb700 10 client.25304 handle_command_reply: tid=0 2024-05-09T02:13:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.947+0000 7fb9a399bb80 1 client.25304 shutdown 2024-05-09T02:13:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.947+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3759534454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 msgr2=0x559e252a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.947+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3759534454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 0x559e252a3550 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x559e2529aa30 tx=0x7fb9800047c0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.947+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3759534454 shutdown_connections 2024-05-09T02:13:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.947+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3759534454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e252916a0 0x559e252b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.947+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3759534454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e251144a0 0x559e252a3550 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.947+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3759534454 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x559e2530e080 0x559e253104a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.947+0000 7fb9a399bb80 1 --2- 172.21.15.19:0/3759534454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e251fe160 0x559e252b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.947+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3759534454 >> 172.21.15.19:0/3759534454 conn(0x559e251746a0 msgr2=0x559e25290e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:28.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.947+0000 7fb975ffb700 0 client.25304 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:13:28.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.948+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3759534454 shutdown_connections 2024-05-09T02:13:28.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.949+0000 7fb9a399bb80 1 -- 172.21.15.19:0/3759534454 wait complete. 2024-05-09T02:13:28.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.949+0000 7fb9a399bb80 20 client.25304 trim_cache size 0 max 16384 2024-05-09T02:13:28.953 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:13:28.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.950+0000 7fb994f85700 1 -- 172.21.15.19:0/2425856513 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb9780b2b40 msgr2=0x7fb9780b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:28.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.950+0000 7fb994f85700 1 --2- 172.21.15.19:0/2425856513 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb9780b2b40 0x7fb9780b5050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb980009d80 tx=0x7fb980009260 comp rx=0 tx=0).stop 2024-05-09T02:13:28.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.950+0000 7fb994f85700 1 -- 172.21.15.19:0/2425856513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb990108740 msgr2=0x7fb99010aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:28.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.950+0000 7fb994f85700 1 --2- 172.21.15.19:0/2425856513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb990108740 0x7fb99010aba0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb98800b0d0 tx=0x7fb988007ab0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.950+0000 7fb994f85700 1 -- 172.21.15.19:0/2425856513 shutdown_connections 2024-05-09T02:13:28.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.950+0000 7fb994f85700 1 --2- 172.21.15.19:0/2425856513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb990108740 0x7fb99010aba0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.950+0000 7fb994f85700 1 --2- 172.21.15.19:0/2425856513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9901034c0 0x7fb99010a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.950+0000 7fb994f85700 1 --2- 172.21.15.19:0/2425856513 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb9780b2b40 0x7fb9780b5050 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.950+0000 7fb994f85700 1 --2- 172.21.15.19:0/2425856513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb990107df0 0x7fb99010a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:28.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.950+0000 7fb994f85700 1 -- 172.21.15.19:0/2425856513 >> 172.21.15.19:0/2425856513 conn(0x7fb9900fedc0 msgr2=0x7fb990109730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:28.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.950+0000 7fb994f85700 1 -- 172.21.15.19:0/2425856513 shutdown_connections 2024-05-09T02:13:28.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:28.950+0000 7fb994f85700 1 -- 172.21.15.19:0/2425856513 wait complete. 2024-05-09T02:13:29.724 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T02:13:30.174 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:13:30.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:30 smithi019 ceph-mon[30151]: pgmap v69: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 953 op/s 2024-05-09T02:13:30.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:30 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:13:30.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:30 smithi059 ceph-mon[34781]: pgmap v69: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 953 op/s 2024-05-09T02:13:30.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:30 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:13:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:30 smithi196 ceph-mon[34718]: pgmap v69: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 953 op/s 2024-05-09T02:13:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:30 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:13:31.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:31 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:13:31.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:31 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:13:31.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:31 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:13:31.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:31 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:13:31.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:31 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:13:31.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:31 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:13:31.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:31 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:13:31.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:31 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:13:31.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:31 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:13:32.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.475+0000 7f6ec959a700 1 -- 172.21.15.19:0/748510252 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4009480 con 0x7f6ecc108740 2024-05-09T02:13:32.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.476+0000 7f6ed0a93700 1 -- 172.21.15.19:0/748510252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc108740 msgr2=0x7f6ecc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:32.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.476+0000 7f6ed0a93700 1 --2- 172.21.15.19:0/748510252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc108740 0x7f6ecc1100a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400b0d0 tx=0x7f6ec400b3e0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.477+0000 7f6ed0a93700 1 -- 172.21.15.19:0/748510252 shutdown_connections 2024-05-09T02:13:32.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.477+0000 7f6ed0a93700 1 --2- 172.21.15.19:0/748510252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc108740 0x7f6ecc1100a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.477+0000 7f6ed0a93700 1 --2- 172.21.15.19:0/748510252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ecc1034c0 0x7f6ecc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.477+0000 7f6ed0a93700 1 --2- 172.21.15.19:0/748510252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ecc107df0 0x7f6ecc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.477+0000 7f6ed0a93700 1 -- 172.21.15.19:0/748510252 >> 172.21.15.19:0/748510252 conn(0x7f6ecc0fedc0 msgr2=0x7f6ecc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:32.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.477+0000 7f6ed0a93700 1 -- 172.21.15.19:0/748510252 shutdown_connections 2024-05-09T02:13:32.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.477+0000 7f6ed0a93700 1 -- 172.21.15.19:0/748510252 wait complete. 2024-05-09T02:13:32.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.478+0000 7f6ed0a93700 1 Processor -- start 2024-05-09T02:13:32.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.478+0000 7f6ed0a93700 1 -- start start 2024-05-09T02:13:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.478+0000 7f6ed0a93700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc1034c0 0x7f6ecc10d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.478+0000 7f6ed0a93700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ecc107df0 0x7f6ecc10d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.478+0000 7f6ed0a93700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ecc108740 0x7f6ecc10dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.478+0000 7f6ed0a93700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc112250 con 0x7f6ecc1034c0 2024-05-09T02:13:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.478+0000 7f6ed0a93700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1120d0 con 0x7f6ecc108740 2024-05-09T02:13:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.478+0000 7f6ed0a93700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1123d0 con 0x7f6ecc107df0 2024-05-09T02:13:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.478+0000 7f6eca59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc1034c0 0x7f6ecc10d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.478+0000 7f6eca59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc1034c0 0x7f6ecc10d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43048/0 (socket says 172.21.15.19:43048) 2024-05-09T02:13:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.478+0000 7f6eca59c700 1 -- 172.21.15.19:0/1754680917 learned_addr learned my addr 172.21.15.19:0/1754680917 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.479+0000 7f6ec9d9b700 1 --2- 172.21.15.19:0/1754680917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ecc107df0 0x7f6ecc10d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:32.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.479+0000 7f6eca59c700 1 -- 172.21.15.19:0/1754680917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ecc107df0 msgr2=0x7f6ecc10d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:32.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.479+0000 7f6eca59c700 1 --2- 172.21.15.19:0/1754680917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ecc107df0 0x7f6ecc10d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.479+0000 7f6eca59c700 1 -- 172.21.15.19:0/1754680917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ecc108740 msgr2=0x7f6ecc10dce0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T02:13:32.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.479+0000 7f6eca59c700 1 --2- 172.21.15.19:0/1754680917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ecc108740 0x7f6ecc10dce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.479+0000 7f6eca59c700 1 -- 172.21.15.19:0/1754680917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc1ad550 con 0x7f6ecc1034c0 2024-05-09T02:13:32.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.480+0000 7f6eca59c700 1 --2- 172.21.15.19:0/1754680917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc1034c0 0x7f6ecc10d260 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00e9a0 tx=0x7f6ebc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:32.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.480+0000 7f6ebb7fe700 1 -- 172.21.15.19:0/1754680917 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc00cbd0 con 0x7f6ecc1034c0 2024-05-09T02:13:32.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.480+0000 7f6ed0a93700 1 -- 172.21.15.19:0/1754680917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc1ad780 con 0x7f6ecc1034c0 2024-05-09T02:13:32.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.480+0000 7f6ed0a93700 1 -- 172.21.15.19:0/1754680917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc1add10 con 0x7f6ecc1034c0 2024-05-09T02:13:32.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.481+0000 7f6ebb7fe700 1 -- 172.21.15.19:0/1754680917 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ebc00cd70 con 0x7f6ecc1034c0 2024-05-09T02:13:32.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.481+0000 7f6ebb7fe700 1 -- 172.21.15.19:0/1754680917 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc018680 con 0x7f6ecc1034c0 2024-05-09T02:13:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.486+0000 7f6ebb7fe700 1 -- 172.21.15.19:0/1754680917 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6ebc018820 con 0x7f6ecc1034c0 2024-05-09T02:13:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.487+0000 7f6ebb7fe700 1 --2- 172.21.15.19:0/1754680917 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6eb40b29f0 0x7f6eb40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.487+0000 7f6edf4a9b80 1 Processor -- start 2024-05-09T02:13:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.487+0000 7f6ebb7fe700 1 -- 172.21.15.19:0/1754680917 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f6ebc0e27b0 con 0x7f6ecc1034c0 2024-05-09T02:13:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.488+0000 7f6edf4a9b80 1 -- start start 2024-05-09T02:13:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.488+0000 7f6edf4a9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e42b8d00 0x5568e43c1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.488+0000 7f6edf4a9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e423e4a0 0x5568e43c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.488+0000 7f6edf4a9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e43bb400 0x5568e43bb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.488+0000 7f6edf4a9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568e42de0a0 con 0x5568e42b8d00 2024-05-09T02:13:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.488+0000 7f6edf4a9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5568e42839a0 con 0x5568e423e4a0 2024-05-09T02:13:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.488+0000 7f6edf4a9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5568e42da060 con 0x5568e43bb400 2024-05-09T02:13:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.489+0000 7f6eca59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e42b8d00 0x5568e43c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.489+0000 7f6eca59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e42b8d00 0x5568e43c1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43060/0 (socket says 172.21.15.19:43060) 2024-05-09T02:13:32.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.489+0000 7f6eca59c700 1 -- 172.21.15.19:0/4220080515 learned_addr learned my addr 172.21.15.19:0/4220080515 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:32.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.489+0000 7f6ecad9d700 1 --2- 172.21.15.19:0/4220080515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e43bb400 0x5568e43bb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:32.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.489+0000 7f6eca59c700 1 -- 172.21.15.19:0/4220080515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e43bb400 msgr2=0x5568e43bb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:32.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.489+0000 7f6eca59c700 1 --2- 172.21.15.19:0/4220080515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e43bb400 0x5568e43bb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.489+0000 7f6eca59c700 1 -- 172.21.15.19:0/4220080515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e423e4a0 msgr2=0x5568e43c1780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:13:32.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.489+0000 7f6eca59c700 1 --2- 172.21.15.19:0/4220080515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e423e4a0 0x5568e43c1780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.489+0000 7f6eca59c700 1 -- 172.21.15.19:0/4220080515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568e4274500 con 0x5568e42b8d00 2024-05-09T02:13:32.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.489+0000 7f6eca59c700 1 --2- 172.21.15.19:0/4220080515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e42b8d00 0x5568e43c1240 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0a9790 tx=0x7f6ebc0a9990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ceb48b7224f484 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.490+0000 7f6eb8ff9700 1 -- 172.21.15.19:0/4220080515 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0b2240 con 0x5568e42b8d00 2024-05-09T02:13:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.490+0000 7f6eb8ff9700 1 -- 172.21.15.19:0/4220080515 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ebc0aa190 con 0x5568e42b8d00 2024-05-09T02:13:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.490+0000 7f6eb8ff9700 1 -- 172.21.15.19:0/4220080515 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0b26c0 con 0x5568e42b8d00 2024-05-09T02:13:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.490+0000 7f6ec9d9b700 1 --2- 172.21.15.19:0/1754680917 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6eb40b29f0 0x7f6eb40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.491+0000 7f6ec9d9b700 1 --2- 172.21.15.19:0/1754680917 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6eb40b29f0 0x7f6eb40b4f00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5568e43b7a00 tx=0x7f6ec000b410 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.491+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/4220080515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e42b8d00 msgr2=0x5568e43c1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.491+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/4220080515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e42b8d00 0x5568e43c1240 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0a9790 tx=0x7f6ebc0a9990 comp rx=0 tx=0).stop 2024-05-09T02:13:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.491+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/4220080515 shutdown_connections 2024-05-09T02:13:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.491+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/4220080515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e43bb400 0x5568e43bb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.491+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/4220080515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e42b8d00 0x5568e43c1240 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.491+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/4220080515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e423e4a0 0x5568e43c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.491+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/4220080515 >> 172.21.15.19:0/4220080515 conn(0x5568e4241f70 msgr2=0x5568e43b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.491+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/4220080515 shutdown_connections 2024-05-09T02:13:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.491+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/4220080515 wait complete. 2024-05-09T02:13:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.492+0000 7f6edf4a9b80 1 -- start start 2024-05-09T02:13:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.492+0000 7f6edf4a9b80 1 Processor -- start 2024-05-09T02:13:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.493+0000 7f6edf4a9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.493+0000 7f6edf4a9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e42b8d00 0x5568e43c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.493+0000 7f6edf4a9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e43bb400 0x5568e43c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.493+0000 7f6edf4a9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568e43bc1b0 con 0x5568e43bb400 2024-05-09T02:13:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.493+0000 7f6edf4a9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5568e4278b20 con 0x5568e42b8d00 2024-05-09T02:13:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.493+0000 7f6edf4a9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5568e43c8020 con 0x5568e423e4a0 2024-05-09T02:13:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.493+0000 7f6edf4a9b80 10 client.0 _ll_register_callbacks cb 0x5568e414f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.493+0000 7f6eca59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.493+0000 7f6ecad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e43bb400 0x5568e43c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.493+0000 7f6ecad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e43bb400 0x5568e43c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43070/0 (socket says 172.21.15.19:43070) 2024-05-09T02:13:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.493+0000 7f6ecad9d700 1 -- 172.21.15.19:0/2220986645 learned_addr learned my addr 172.21.15.19:0/2220986645 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.494+0000 7f6eca59c700 1 -- 172.21.15.19:0/2220986645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e42b8d00 msgr2=0x5568e43c54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.494+0000 7f6eca59c700 1 --2- 172.21.15.19:0/2220986645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e42b8d00 0x5568e43c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.494+0000 7f6eca59c700 1 -- 172.21.15.19:0/2220986645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e43bb400 msgr2=0x5568e43c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.494+0000 7f6eca59c700 1 --2- 172.21.15.19:0/2220986645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e43bb400 0x5568e43c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.494+0000 7f6eca59c700 1 -- 172.21.15.19:0/2220986645 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568e4159490 con 0x5568e423e4a0 2024-05-09T02:13:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.494+0000 7f6eca59c700 1 --2- 172.21.15.19:0/2220986645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43c4f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0a6d00 tx=0x7f6ebc0b26f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f000867f24fda607 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.494+0000 7f6eb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:13:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.494+0000 7f6eb1ffb700 1 -- 172.21.15.19:0/2220986645 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0b2240 con 0x5568e423e4a0 2024-05-09T02:13:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.494+0000 7f6eb1ffb700 1 -- 172.21.15.19:0/2220986645 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f6ebc0a8f00 con 0x5568e423e4a0 2024-05-09T02:13:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.494+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/2220986645 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ebc0aa190 con 0x5568e423e4a0 2024-05-09T02:13:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.495+0000 7f6eb1ffb700 1 -- 172.21.15.19:0/2220986645 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6ebc0a9a90 con 0x5568e423e4a0 2024-05-09T02:13:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.495+0000 7f6edf4a9b80 10 client.25369 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.495+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/2220986645 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5568e43d29f0 con 0x5568e423e4a0 2024-05-09T02:13:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.495+0000 7f6eb1ffb700 1 -- 172.21.15.19:0/2220986645 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ebc0aa190 con 0x5568e423e4a0 2024-05-09T02:13:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.495+0000 7f6eb1ffb700 1 -- 172.21.15.19:0/2220986645 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f6ebc0a8760 con 0x5568e423e4a0 2024-05-09T02:13:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.495+0000 7f6edf4a9b80 10 client.25369 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.495+0000 7f6edf4a9b80 10 client.25369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:13:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.495+0000 7f6edf4a9b80 20 client.25369 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.495+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/2220986645 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5568e43d4ec0 0x5568e43d72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.495+0000 7f6edf4a9b80 4 client.25369 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:13:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.495+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/2220986645 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568e42eba80 con 0x5568e43d4ec0 2024-05-09T02:13:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.496+0000 7f6ec9d9b700 1 --2- 172.21.15.19:0/2220986645 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5568e43d4ec0 0x5568e43d72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.496+0000 7f6eb1ffb700 10 client.25369 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.496+0000 7f6ec9d9b700 1 --2- 172.21.15.19:0/2220986645 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5568e43d4ec0 0x5568e43d72e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a20a56faa1820750 server_cookie=a32a36abc43044a1 in_seq=0 out_seq=0 2024-05-09T02:13:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.498+0000 7f6eb1ffb700 1 -- 172.21.15.19:0/2220986645 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5568e42eba80 con 0x5568e43d4ec0 2024-05-09T02:13:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.498+0000 7f6eb1ffb700 10 client.25369 handle_command_reply: tid=0 2024-05-09T02:13:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.498+0000 7f6edf4a9b80 1 client.25369 shutdown 2024-05-09T02:13:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.498+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/2220986645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 msgr2=0x5568e43c4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.498+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/2220986645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43c4f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0a6d00 tx=0x7f6ebc0b26f0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.498+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/2220986645 shutdown_connections 2024-05-09T02:13:32.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.498+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/2220986645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e43bb400 0x5568e43c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.498+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/2220986645 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5568e43d4ec0 0x5568e43d72e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.498+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/2220986645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43c4f90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.498+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/2220986645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e42b8d00 0x5568e43c54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.498+0000 7f6eb1ffb700 0 client.25369 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:32.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.498+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/2220986645 >> 172.21.15.19:0/2220986645 conn(0x5568e4241f70 msgr2=0x5568e43afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:32.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.498+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/2220986645 shutdown_connections 2024-05-09T02:13:32.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.499+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/2220986645 wait complete. 2024-05-09T02:13:32.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.499+0000 7f6edf4a9b80 20 client.25369 trim_cache size 0 max 16384 2024-05-09T02:13:32.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.528+0000 7f6edf4a9b80 1 Processor -- start 2024-05-09T02:13:32.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.529+0000 7f6edf4a9b80 1 -- start start 2024-05-09T02:13:32.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.529+0000 7f6edf4a9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.529+0000 7f6edf4a9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e42b8d00 0x5568e43bf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.529+0000 7f6edf4a9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e43bb400 0x5568e43c2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.529+0000 7f6edf4a9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac00d590 con 0x5568e42b8d00 2024-05-09T02:13:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.529+0000 7f6edf4a9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac00f190 con 0x5568e43bb400 2024-05-09T02:13:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.529+0000 7f6edf4a9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac00dc60 con 0x5568e423e4a0 2024-05-09T02:13:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.529+0000 7f6eca59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.529+0000 7f6eca59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:33454/0 (socket says 172.21.15.19:33454) 2024-05-09T02:13:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.529+0000 7f6eca59c700 1 -- 172.21.15.19:0/1997240683 learned_addr learned my addr 172.21.15.19:0/1997240683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.530+0000 7f6ecad9d700 1 --2- 172.21.15.19:0/1997240683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e43bb400 0x5568e43c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.530+0000 7f6eca59c700 1 -- 172.21.15.19:0/1997240683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e43bb400 msgr2=0x5568e43c2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.530+0000 7f6eca59c700 1 --2- 172.21.15.19:0/1997240683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e43bb400 0x5568e43c2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.530+0000 7f6eca59c700 1 -- 172.21.15.19:0/1997240683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e42b8d00 msgr2=0x5568e43bf3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:13:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.530+0000 7f6eca59c700 1 --2- 172.21.15.19:0/1997240683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e42b8d00 0x5568e43bf3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.530+0000 7f6eca59c700 1 -- 172.21.15.19:0/1997240683 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568e43b71b0 con 0x5568e423e4a0 2024-05-09T02:13:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.531+0000 7f6eca59c700 1 --2- 172.21.15.19:0/1997240683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43d1280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0aa0f0 tx=0x7f6ebc018de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc9de428f168513a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.531+0000 7f6eb3fff700 1 -- 172.21.15.19:0/1997240683 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0b2240 con 0x5568e423e4a0 2024-05-09T02:13:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.531+0000 7f6eb3fff700 1 -- 172.21.15.19:0/1997240683 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ebc0af070 con 0x5568e423e4a0 2024-05-09T02:13:32.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.531+0000 7f6eb3fff700 1 -- 172.21.15.19:0/1997240683 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0a8760 con 0x5568e423e4a0 2024-05-09T02:13:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.533+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/1997240683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 msgr2=0x5568e43d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.533+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/1997240683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43d1280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0aa0f0 tx=0x7f6ebc018de0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.533+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/1997240683 shutdown_connections 2024-05-09T02:13:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.533+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/1997240683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e43bb400 0x5568e43c2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.533+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/1997240683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43d1280 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.533+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/1997240683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e42b8d00 0x5568e43bf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.533+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/1997240683 >> 172.21.15.19:0/1997240683 conn(0x5568e4241f70 msgr2=0x5568e43beca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.533+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/1997240683 shutdown_connections 2024-05-09T02:13:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.533+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/1997240683 wait complete. 2024-05-09T02:13:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.533+0000 7f6edf4a9b80 1 -- start start 2024-05-09T02:13:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.534+0000 7f6edf4a9b80 1 Processor -- start 2024-05-09T02:13:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.534+0000 7f6edf4a9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43d72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.534+0000 7f6edf4a9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e42b8d00 0x5568e43d7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.534+0000 7f6edf4a9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e43bb400 0x5568e443d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.534+0000 7f6eca59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43d72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.534+0000 7f6edf4a9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac00ea00 con 0x5568e43bb400 2024-05-09T02:13:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.534+0000 7f6eca59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43d72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:33470/0 (socket says 172.21.15.19:33470) 2024-05-09T02:13:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.534+0000 7f6edf4a9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac00e330 con 0x5568e42b8d00 2024-05-09T02:13:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.534+0000 7f6eca59c700 1 -- 172.21.15.19:0/295483979 learned_addr learned my addr 172.21.15.19:0/295483979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.534+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/295483979 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac00b530 con 0x5568e423e4a0 2024-05-09T02:13:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.535+0000 7f6edf4a9b80 10 client.0 _ll_register_callbacks cb 0x5568e414f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.535+0000 7f6ec9d9b700 1 --2- 172.21.15.19:0/295483979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e42b8d00 0x5568e43d7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.535+0000 7f6ecad9d700 1 --2- 172.21.15.19:0/295483979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e43bb400 0x5568e443d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.535+0000 7f6ec9d9b700 1 -- 172.21.15.19:0/295483979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 msgr2=0x5568e43d72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.535+0000 7f6ec9d9b700 1 --2- 172.21.15.19:0/295483979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43d72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.535+0000 7f6ec9d9b700 1 -- 172.21.15.19:0/295483979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e43bb400 msgr2=0x5568e443d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.535+0000 7f6ec9d9b700 1 --2- 172.21.15.19:0/295483979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e43bb400 0x5568e443d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.535+0000 7f6ec9d9b700 1 -- 172.21.15.19:0/295483979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568e443d940 con 0x5568e42b8d00 2024-05-09T02:13:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.535+0000 7f6ec9d9b700 1 --2- 172.21.15.19:0/295483979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e42b8d00 0x5568e43d7820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5568e43b2a80 tx=0x7f6ec00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=299aa04da6baace0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.535+0000 7f6eb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:13:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.535+0000 7f6eb1ffb700 1 -- 172.21.15.19:0/295483979 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0025460 con 0x5568e42b8d00 2024-05-09T02:13:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.535+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/295483979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ebc0af070 con 0x5568e42b8d00 2024-05-09T02:13:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.535+0000 7f6eb1ffb700 1 -- 172.21.15.19:0/295483979 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f6ec0025050 con 0x5568e42b8d00 2024-05-09T02:13:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.536+0000 7f6eb1ffb700 1 -- 172.21.15.19:0/295483979 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6ec0034050 con 0x5568e42b8d00 2024-05-09T02:13:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.536+0000 7f6edf4a9b80 10 client.25313 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.536+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/295483979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5568e443ea20 con 0x5568e42b8d00 2024-05-09T02:13:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.536+0000 7f6eb1ffb700 1 -- 172.21.15.19:0/295483979 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ebc0af070 con 0x5568e42b8d00 2024-05-09T02:13:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.536+0000 7f6eb1ffb700 1 -- 172.21.15.19:0/295483979 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f6ec0025ce0 con 0x5568e42b8d00 2024-05-09T02:13:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.536+0000 7f6edf4a9b80 10 client.25313 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.536+0000 7f6edf4a9b80 10 client.25313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:13:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.536+0000 7f6edf4a9b80 20 client.25313 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.536+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/295483979 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5568e443f1c0 0x5568e44415e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.536+0000 7f6edf4a9b80 4 client.25313 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:13:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.536+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/295483979 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568e42e5b20 con 0x5568e443f1c0 2024-05-09T02:13:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.537+0000 7f6eca59c700 1 --2- 172.21.15.19:0/295483979 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5568e443f1c0 0x5568e44415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:32.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.538+0000 7f6eca59c700 1 --2- 172.21.15.19:0/295483979 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5568e443f1c0 0x5568e44415e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d8aeefaa274ec1b server_cookie=ce44907135bef7bb in_seq=0 out_seq=0 2024-05-09T02:13:32.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.538+0000 7f6eb1ffb700 10 client.25313 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:32.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.538+0000 7f6eb1ffb700 1 -- 172.21.15.19:0/295483979 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568e42e5b20 con 0x5568e443f1c0 2024-05-09T02:13:32.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.538+0000 7f6eb1ffb700 10 client.25313 handle_command_reply: tid=0 2024-05-09T02:13:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.538+0000 7f6edf4a9b80 1 client.25313 shutdown 2024-05-09T02:13:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.539+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/295483979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e42b8d00 msgr2=0x5568e43d7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.539+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/295483979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e42b8d00 0x5568e43d7820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5568e43b2a80 tx=0x7f6ec00047c0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.539+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/295483979 shutdown_connections 2024-05-09T02:13:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.539+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/295483979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568e43bb400 0x5568e443d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.539+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/295483979 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5568e443f1c0 0x5568e44415e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.539+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/295483979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568e423e4a0 0x5568e43d72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.539+0000 7f6edf4a9b80 1 --2- 172.21.15.19:0/295483979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568e42b8d00 0x5568e43d7820 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.539+0000 7f6eb1ffb700 0 client.25313 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.539+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/295483979 >> 172.21.15.19:0/295483979 conn(0x5568e4241f70 msgr2=0x5568e43c0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.539+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/295483979 shutdown_connections 2024-05-09T02:13:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.539+0000 7f6edf4a9b80 1 -- 172.21.15.19:0/295483979 wait complete. 2024-05-09T02:13:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.539+0000 7f6edf4a9b80 20 client.25313 trim_cache size 0 max 16384 2024-05-09T02:13:32.546 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:13:32.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.540+0000 7f6ed0a93700 1 -- 172.21.15.19:0/1754680917 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6eb40b29f0 msgr2=0x7f6eb40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:32.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.540+0000 7f6ed0a93700 1 --2- 172.21.15.19:0/1754680917 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6eb40b29f0 0x7f6eb40b4f00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5568e43b7a00 tx=0x7f6ec000b410 comp rx=0 tx=0).stop 2024-05-09T02:13:32.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.540+0000 7f6ed0a93700 1 -- 172.21.15.19:0/1754680917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc1034c0 msgr2=0x7f6ecc10d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:32.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.540+0000 7f6ed0a93700 1 --2- 172.21.15.19:0/1754680917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc1034c0 0x7f6ecc10d260 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00e9a0 tx=0x7f6ebc00ecb0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.540+0000 7f6ed0a93700 1 -- 172.21.15.19:0/1754680917 shutdown_connections 2024-05-09T02:13:32.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.540+0000 7f6ed0a93700 1 --2- 172.21.15.19:0/1754680917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ecc108740 0x7f6ecc10dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.540+0000 7f6ed0a93700 1 --2- 172.21.15.19:0/1754680917 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6eb40b29f0 0x7f6eb40b4f00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.540+0000 7f6ed0a93700 1 --2- 172.21.15.19:0/1754680917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc1034c0 0x7f6ecc10d260 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.540+0000 7f6ed0a93700 1 --2- 172.21.15.19:0/1754680917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ecc107df0 0x7f6ecc10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:32.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.540+0000 7f6ed0a93700 1 -- 172.21.15.19:0/1754680917 >> 172.21.15.19:0/1754680917 conn(0x7f6ecc0fedc0 msgr2=0x7f6ecc107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:32.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.540+0000 7f6ed0a93700 1 -- 172.21.15.19:0/1754680917 shutdown_connections 2024-05-09T02:13:32.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:32.540+0000 7f6ed0a93700 1 -- 172.21.15.19:0/1754680917 wait complete. 2024-05-09T02:13:32.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:32 smithi019 ceph-mon[30151]: pgmap v70: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.5 MiB/s wr, 577 op/s 2024-05-09T02:13:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 110.0 s, 2721.6 tps, lat 36.600 ms stddev 40.297 2024-05-09T02:13:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:32 smithi059 ceph-mon[34781]: pgmap v70: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.5 MiB/s wr, 577 op/s 2024-05-09T02:13:32.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:32 smithi196 ceph-mon[34718]: pgmap v70: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.5 MiB/s wr, 577 op/s 2024-05-09T02:13:34.364 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T02:13:34.364 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T02:13:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:34 smithi059 ceph-mon[34781]: pgmap v71: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.5 MiB/s wr, 577 op/s 2024-05-09T02:13:34.756 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:13:34.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:34 smithi196 ceph-mon[34718]: pgmap v71: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.5 MiB/s wr, 577 op/s 2024-05-09T02:13:34.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:34 smithi019 ceph-mon[30151]: pgmap v71: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.5 MiB/s wr, 577 op/s 2024-05-09T02:13:36.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:36 smithi019 ceph-mon[30151]: pgmap v72: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.6 MiB/s wr, 871 op/s 2024-05-09T02:13:36.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:36 smithi059 ceph-mon[34781]: pgmap v72: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.6 MiB/s wr, 871 op/s 2024-05-09T02:13:36.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:36 smithi196 ceph-mon[34718]: pgmap v72: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.6 MiB/s wr, 871 op/s 2024-05-09T02:13:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.435+0000 7fb0b1f56700 1 -- 172.21.15.19:0/3598573130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0ac106570 msgr2=0x7fb0ac106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.435+0000 7fb0b1f56700 1 --2- 172.21.15.19:0/3598573130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0ac106570 0x7fb0ac106980 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00098f0 tx=0x7fb0a0009c00 comp rx=0 tx=0).stop 2024-05-09T02:13:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.435+0000 7fb0b1f56700 1 -- 172.21.15.19:0/3598573130 shutdown_connections 2024-05-09T02:13:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.435+0000 7fb0b1f56700 1 --2- 172.21.15.19:0/3598573130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ac106ec0 0x7fb0ac113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.435+0000 7fb0b1f56700 1 --2- 172.21.15.19:0/3598573130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0ac10c5b0 0x7fb0ac10c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.435+0000 7fb0b1f56700 1 --2- 172.21.15.19:0/3598573130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0ac106570 0x7fb0ac106980 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.435+0000 7fb0b1f56700 1 -- 172.21.15.19:0/3598573130 >> 172.21.15.19:0/3598573130 conn(0x7fb0ac102090 msgr2=0x7fb0ac1044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.435+0000 7fb0b1f56700 1 -- 172.21.15.19:0/3598573130 shutdown_connections 2024-05-09T02:13:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.436+0000 7fb0b1f56700 1 -- 172.21.15.19:0/3598573130 wait complete. 2024-05-09T02:13:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.437+0000 7fb0b1f56700 1 Processor -- start 2024-05-09T02:13:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.437+0000 7fb0b1f56700 1 -- start start 2024-05-09T02:13:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.437+0000 7fb0b1f56700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0ac106570 0x7fb0ac19d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.437+0000 7fb0b1f56700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0ac106ec0 0x7fb0ac19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.438+0000 7fb0b1f56700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ac10c5b0 0x7fb0ac1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.438+0000 7fb0aaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0ac106ec0 0x7fb0ac19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.438+0000 7fb0aaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0ac106ec0 0x7fb0ac19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39820/0 (socket says 172.21.15.19:39820) 2024-05-09T02:13:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.438+0000 7fb0ab7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0ac106570 0x7fb0ac19d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.438+0000 7fb0ab7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0ac106570 0x7fb0ac19d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:42934/0 (socket says 172.21.15.19:42934) 2024-05-09T02:13:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.438+0000 7fb0aaffd700 1 -- 172.21.15.19:0/673489319 learned_addr learned my addr 172.21.15.19:0/673489319 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.438+0000 7fb0b1f56700 1 -- 172.21.15.19:0/673489319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1166e0 con 0x7fb0ac106ec0 2024-05-09T02:13:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.438+0000 7fb0b1f56700 1 -- 172.21.15.19:0/673489319 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac116560 con 0x7fb0ac106570 2024-05-09T02:13:37.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.438+0000 7fb0b1f56700 1 -- 172.21.15.19:0/673489319 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac116860 con 0x7fb0ac10c5b0 2024-05-09T02:13:37.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.438+0000 7fb0abfff700 1 --2- 172.21.15.19:0/673489319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ac10c5b0 0x7fb0ac1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.438+0000 7fb0aaffd700 1 -- 172.21.15.19:0/673489319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ac10c5b0 msgr2=0x7fb0ac1a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:37.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.438+0000 7fb0aaffd700 1 --2- 172.21.15.19:0/673489319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ac10c5b0 0x7fb0ac1a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.438+0000 7fb0aaffd700 1 -- 172.21.15.19:0/673489319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0ac106570 msgr2=0x7fb0ac19d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:37.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.438+0000 7fb0aaffd700 1 --2- 172.21.15.19:0/673489319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0ac106570 0x7fb0ac19d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.438+0000 7fb0aaffd700 1 -- 172.21.15.19:0/673489319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac1a1f80 con 0x7fb0ac106ec0 2024-05-09T02:13:37.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.439+0000 7fb0aaffd700 1 --2- 172.21.15.19:0/673489319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0ac106ec0 0x7fb0ac19d650 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000c010 tx=0x7fb0a000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:37.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.439+0000 7fb0a8ff9700 1 -- 172.21.15.19:0/673489319 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00051c0 con 0x7fb0ac106ec0 2024-05-09T02:13:37.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.440+0000 7fb0b1f56700 1 -- 172.21.15.19:0/673489319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ac1a2210 con 0x7fb0ac106ec0 2024-05-09T02:13:37.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.440+0000 7fb0a8ff9700 1 -- 172.21.15.19:0/673489319 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a000b560 con 0x7fb0ac106ec0 2024-05-09T02:13:37.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.440+0000 7fb0a8ff9700 1 -- 172.21.15.19:0/673489319 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0003950 con 0x7fb0ac106ec0 2024-05-09T02:13:37.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.440+0000 7fb0b1f56700 1 -- 172.21.15.19:0/673489319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ac1a2630 con 0x7fb0ac106ec0 2024-05-09T02:13:37.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.443+0000 7fb0a8ff9700 1 -- 172.21.15.19:0/673489319 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb0a000b910 con 0x7fb0ac106ec0 2024-05-09T02:13:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.446+0000 7fb0a8ff9700 1 --2- 172.21.15.19:0/673489319 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb0940b2af0 0x7fb0940b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.446+0000 7fb0ab7fe700 1 --2- 172.21.15.19:0/673489319 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb0940b2af0 0x7fb0940b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.446+0000 7fb0a8ff9700 1 -- 172.21.15.19:0/673489319 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fb0a00e34a0 con 0x7fb0ac106ec0 2024-05-09T02:13:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.447+0000 7fb0ab7fe700 1 --2- 172.21.15.19:0/673489319 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb0940b2af0 0x7fb0940b5000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac1079e0 tx=0x7fb09c009310 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.447+0000 7fb0c096cb80 1 Processor -- start 2024-05-09T02:13:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.447+0000 7fb0c096cb80 1 -- start start 2024-05-09T02:13:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0c096cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d1391c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0c096cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 0x560d1391c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0c096cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13916480 0x560d13916890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0c096cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560d13839120 con 0x560d13916480 2024-05-09T02:13:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0c096cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560d137dea20 con 0x560d13813d80 2024-05-09T02:13:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0c096cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560d138350e0 con 0x560d13799520 2024-05-09T02:13:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0abfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13916480 0x560d13916890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0abfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13916480 0x560d13916890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39836/0 (socket says 172.21.15.19:39836) 2024-05-09T02:13:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0abfff700 1 -- 172.21.15.19:0/4072859656 learned_addr learned my addr 172.21.15.19:0/4072859656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0aaffd700 1 --2- 172.21.15.19:0/4072859656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 0x560d1391c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0ab7fe700 1 --2- 172.21.15.19:0/4072859656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d1391c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0abfff700 1 -- 172.21.15.19:0/4072859656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 msgr2=0x560d1391c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0abfff700 1 --2- 172.21.15.19:0/4072859656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 0x560d1391c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0abfff700 1 -- 172.21.15.19:0/4072859656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 msgr2=0x560d1391c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0abfff700 1 --2- 172.21.15.19:0/4072859656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d1391c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0abfff700 1 -- 172.21.15.19:0/4072859656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d137cf580 con 0x560d13916480 2024-05-09T02:13:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb0abfff700 1 --2- 172.21.15.19:0/4072859656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13916480 0x560d13916890 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4011680 tx=0x7fb0a4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24134b8ecd3e6afe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.448+0000 7fb091ffb700 1 -- 172.21.15.19:0/4072859656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4019030 con 0x560d13916480 2024-05-09T02:13:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.449+0000 7fb091ffb700 1 -- 172.21.15.19:0/4072859656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a40092e0 con 0x560d13916480 2024-05-09T02:13:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.449+0000 7fb091ffb700 1 -- 172.21.15.19:0/4072859656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4004880 con 0x560d13916480 2024-05-09T02:13:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.450+0000 7fb0c096cb80 1 -- 172.21.15.19:0/4072859656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13916480 msgr2=0x560d13916890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.450+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/4072859656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13916480 0x560d13916890 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4011680 tx=0x7fb0a4009920 comp rx=0 tx=0).stop 2024-05-09T02:13:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.451+0000 7fb0c096cb80 1 -- 172.21.15.19:0/4072859656 shutdown_connections 2024-05-09T02:13:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.451+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/4072859656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13916480 0x560d13916890 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.451+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/4072859656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d1391c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.451+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/4072859656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 0x560d1391c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.451+0000 7fb0c096cb80 1 -- 172.21.15.19:0/4072859656 >> 172.21.15.19:0/4072859656 conn(0x560d1379cff0 msgr2=0x560d139112a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.451+0000 7fb0c096cb80 1 -- 172.21.15.19:0/4072859656 shutdown_connections 2024-05-09T02:13:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.451+0000 7fb0c096cb80 1 -- 172.21.15.19:0/4072859656 wait complete. 2024-05-09T02:13:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.452+0000 7fb0c096cb80 1 -- start start 2024-05-09T02:13:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.452+0000 7fb0c096cb80 1 Processor -- start 2024-05-09T02:13:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.453+0000 7fb0c096cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 0x560d13920160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.453+0000 7fb0c096cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d139206a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.453+0000 7fb0ab7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 0x560d13920160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.453+0000 7fb0ab7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 0x560d13920160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40906/0 (socket says 172.21.15.19:40906) 2024-05-09T02:13:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.453+0000 7fb0ab7fe700 1 -- 172.21.15.19:0/3445875692 learned_addr learned my addr 172.21.15.19:0/3445875692 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.453+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13916480 0x560d13920be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.453+0000 7fb0c096cb80 1 -- 172.21.15.19:0/3445875692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560d139171e0 con 0x560d13916480 2024-05-09T02:13:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.453+0000 7fb0c096cb80 1 -- 172.21.15.19:0/3445875692 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560d137d3ba0 con 0x560d13813d80 2024-05-09T02:13:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.453+0000 7fb0c096cb80 1 -- 172.21.15.19:0/3445875692 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560d139230a0 con 0x560d13799520 2024-05-09T02:13:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.454+0000 7fb0c096cb80 10 client.0 _ll_register_callbacks cb 0x560d136aa740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.454+0000 7fb0aaffd700 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d139206a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.454+0000 7fb0ab7fe700 1 -- 172.21.15.19:0/3445875692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 msgr2=0x560d139206a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.454+0000 7fb0abfff700 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13916480 0x560d13920be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.454+0000 7fb0ab7fe700 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d139206a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.454+0000 7fb0ab7fe700 1 -- 172.21.15.19:0/3445875692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13916480 msgr2=0x560d13920be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.454+0000 7fb0ab7fe700 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13916480 0x560d13920be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.454+0000 7fb0ab7fe700 1 -- 172.21.15.19:0/3445875692 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d136b4510 con 0x560d13799520 2024-05-09T02:13:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.454+0000 7fb0aaffd700 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d139206a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:13:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.454+0000 7fb0abfff700 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13916480 0x560d13920be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:13:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.454+0000 7fb0ab7fe700 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 0x560d13920160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0046e0 tx=0x7fb09c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c04c3b16986c0fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.454+0000 7fb08affd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:13:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.455+0000 7fb08affd700 1 -- 172.21.15.19:0/3445875692 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c01fac0 con 0x560d13799520 2024-05-09T02:13:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.455+0000 7fb08affd700 1 -- 172.21.15.19:0/3445875692 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fb09c01fc60 con 0x560d13799520 2024-05-09T02:13:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.455+0000 7fb0c096cb80 1 -- 172.21.15.19:0/3445875692 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a40092e0 con 0x560d13799520 2024-05-09T02:13:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.455+0000 7fb08affd700 1 -- 172.21.15.19:0/3445875692 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb09c01fe60 con 0x560d13799520 2024-05-09T02:13:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.455+0000 7fb0c096cb80 10 client.25387 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.455+0000 7fb0c096cb80 1 -- 172.21.15.19:0/3445875692 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560d1393eb80 con 0x560d13799520 2024-05-09T02:13:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.455+0000 7fb08affd700 1 -- 172.21.15.19:0/3445875692 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a40092e0 con 0x560d13799520 2024-05-09T02:13:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.455+0000 7fb08affd700 1 -- 172.21.15.19:0/3445875692 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fb09c026070 con 0x560d13799520 2024-05-09T02:13:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.455+0000 7fb0c096cb80 10 client.25387 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.455+0000 7fb0c096cb80 10 client.25387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:13:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.455+0000 7fb0c096cb80 20 client.25387 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.455+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560d13941260 0x560d13943710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.455+0000 7fb0c096cb80 4 client.25387 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:13:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.455+0000 7fb0c096cb80 1 -- 172.21.15.19:0/3445875692 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d13846b00 con 0x560d13941260 2024-05-09T02:13:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.456+0000 7fb0aaffd700 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560d13941260 0x560d13943710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.456+0000 7fb08affd700 10 client.25387 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.456+0000 7fb0aaffd700 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560d13941260 0x560d13943710 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=993045ec7bbea48a server_cookie=2e9040a0c6c5c49a in_seq=0 out_seq=0 2024-05-09T02:13:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.458+0000 7fb08affd700 1 -- 172.21.15.19:0/3445875692 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560d13846b00 con 0x560d13941260 2024-05-09T02:13:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.458+0000 7fb08affd700 10 client.25387 handle_command_reply: tid=0 2024-05-09T02:13:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.458+0000 7fb0c096cb80 1 client.25387 shutdown 2024-05-09T02:13:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.458+0000 7fb0c096cb80 1 -- 172.21.15.19:0/3445875692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 msgr2=0x560d13920160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.458+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 0x560d13920160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0046e0 tx=0x7fb09c0047c0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.458+0000 7fb0c096cb80 1 -- 172.21.15.19:0/3445875692 shutdown_connections 2024-05-09T02:13:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.458+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13916480 0x560d13920be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.458+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560d13941260 0x560d13943710 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.458+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 0x560d13920160 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.458+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/3445875692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d139206a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.458+0000 7fb08affd700 0 client.25387 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.458+0000 7fb0c096cb80 1 -- 172.21.15.19:0/3445875692 >> 172.21.15.19:0/3445875692 conn(0x560d1379cff0 msgr2=0x560d1390b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.459+0000 7fb0c096cb80 1 -- 172.21.15.19:0/3445875692 shutdown_connections 2024-05-09T02:13:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.459+0000 7fb0c096cb80 1 -- 172.21.15.19:0/3445875692 wait complete. 2024-05-09T02:13:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.459+0000 7fb0c096cb80 20 client.25387 trim_cache size 0 max 16384 2024-05-09T02:13:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.489+0000 7fb0c096cb80 1 Processor -- start 2024-05-09T02:13:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.489+0000 7fb0c096cb80 1 -- start start 2024-05-09T02:13:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.489+0000 7fb0c096cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d1379cff0 0x560d1390baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.489+0000 7fb0c096cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13799520 0x560d1391b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.489+0000 7fb0c096cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d13915b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.489+0000 7fb0c096cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1acc80 con 0x560d13799520 2024-05-09T02:13:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.490+0000 7fb0c096cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1ae880 con 0x560d13813d80 2024-05-09T02:13:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.490+0000 7fb0c096cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1ad350 con 0x560d1379cff0 2024-05-09T02:13:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.490+0000 7fb0aaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13799520 0x560d1391b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.490+0000 7fb0ab7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d1379cff0 0x560d1390baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.490+0000 7fb0ab7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d1379cff0 0x560d1390baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40910/0 (socket says 172.21.15.19:40910) 2024-05-09T02:13:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.490+0000 7fb0aaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13799520 0x560d1391b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39866/0 (socket says 172.21.15.19:39866) 2024-05-09T02:13:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.490+0000 7fb0ab7fe700 1 -- 172.21.15.19:0/924339706 learned_addr learned my addr 172.21.15.19:0/924339706 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.490+0000 7fb0abfff700 1 --2- 172.21.15.19:0/924339706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d13915b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.490+0000 7fb0ab7fe700 1 -- 172.21.15.19:0/924339706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 msgr2=0x560d13915b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.490+0000 7fb0ab7fe700 1 --2- 172.21.15.19:0/924339706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d13915b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.490+0000 7fb0ab7fe700 1 -- 172.21.15.19:0/924339706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13799520 msgr2=0x560d1391b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.490+0000 7fb0ab7fe700 1 --2- 172.21.15.19:0/924339706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13799520 0x560d1391b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.490+0000 7fb0ab7fe700 1 -- 172.21.15.19:0/924339706 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d13922f50 con 0x560d1379cff0 2024-05-09T02:13:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.490+0000 7fb0ab7fe700 1 --2- 172.21.15.19:0/924339706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d1379cff0 0x560d1390baa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb09c013010 tx=0x7fb09c003890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=734791c0724761a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.490+0000 7fb0917fa700 1 -- 172.21.15.19:0/924339706 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c03c070 con 0x560d1379cff0 2024-05-09T02:13:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.491+0000 7fb0917fa700 1 -- 172.21.15.19:0/924339706 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb09c02f6e0 con 0x560d1379cff0 2024-05-09T02:13:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.491+0000 7fb0917fa700 1 -- 172.21.15.19:0/924339706 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c037900 con 0x560d1379cff0 2024-05-09T02:13:37.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.492+0000 7fb0c096cb80 1 -- 172.21.15.19:0/924339706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d1379cff0 msgr2=0x560d1390baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:37.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.492+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/924339706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d1379cff0 0x560d1390baa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb09c013010 tx=0x7fb09c003890 comp rx=0 tx=0).stop 2024-05-09T02:13:37.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.493+0000 7fb0c096cb80 1 -- 172.21.15.19:0/924339706 shutdown_connections 2024-05-09T02:13:37.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.493+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/924339706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d13915b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.493+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/924339706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d1379cff0 0x560d1390baa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.493+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/924339706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d13799520 0x560d1391b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.493+0000 7fb0c096cb80 1 -- 172.21.15.19:0/924339706 >> 172.21.15.19:0/924339706 conn(0x560d139115a0 msgr2=0x560d13915470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:37.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.494+0000 7fb0c096cb80 1 -- 172.21.15.19:0/924339706 shutdown_connections 2024-05-09T02:13:37.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.494+0000 7fb0c096cb80 1 -- 172.21.15.19:0/924339706 wait complete. 2024-05-09T02:13:37.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.494+0000 7fb0c096cb80 1 -- start start 2024-05-09T02:13:37.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.495+0000 7fb0c096cb80 1 Processor -- start 2024-05-09T02:13:37.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.496+0000 7fb0c096cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 0x560d13928190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.496+0000 7fb0c096cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d1379cff0 0x560d139286d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.496+0000 7fb0c096cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d13925160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.496+0000 7fb0c096cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1ae0f0 con 0x560d1379cff0 2024-05-09T02:13:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.496+0000 7fb0c096cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1ada20 con 0x560d13813d80 2024-05-09T02:13:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.496+0000 7fb0c096cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac07be90 con 0x560d13799520 2024-05-09T02:13:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.496+0000 7fb0aaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d1379cff0 0x560d139286d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.496+0000 7fb0ab7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 0x560d13928190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.496+0000 7fb0aaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d1379cff0 0x560d139286d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39870/0 (socket says 172.21.15.19:39870) 2024-05-09T02:13:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.496+0000 7fb0aaffd700 1 -- 172.21.15.19:0/172990620 learned_addr learned my addr 172.21.15.19:0/172990620 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.496+0000 7fb0c096cb80 10 client.0 _ll_register_callbacks cb 0x560d136aa740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.497+0000 7fb0abfff700 1 --2- 172.21.15.19:0/172990620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d13925160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.497+0000 7fb0aaffd700 1 -- 172.21.15.19:0/172990620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 msgr2=0x560d13928190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.497+0000 7fb0aaffd700 1 --2- 172.21.15.19:0/172990620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 0x560d13928190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.497+0000 7fb0aaffd700 1 -- 172.21.15.19:0/172990620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 msgr2=0x560d13925160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.497+0000 7fb0aaffd700 1 --2- 172.21.15.19:0/172990620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d13925160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.497+0000 7fb0aaffd700 1 -- 172.21.15.19:0/172990620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d13912d70 con 0x560d1379cff0 2024-05-09T02:13:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.497+0000 7fb0ab7fe700 1 --2- 172.21.15.19:0/172990620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 0x560d13928190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:13:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.497+0000 7fb0aaffd700 1 --2- 172.21.15.19:0/172990620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d1379cff0 0x560d139286d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00a9ff0 tx=0x7fb0a00b1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f58d70185b48e63 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.497+0000 7fb08affd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:13:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.497+0000 7fb08affd700 1 -- 172.21.15.19:0/172990620 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00e1020 con 0x560d1379cff0 2024-05-09T02:13:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.497+0000 7fb08affd700 1 -- 172.21.15.19:0/172990620 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fb0a00aaa10 con 0x560d1379cff0 2024-05-09T02:13:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.497+0000 7fb0c096cb80 1 -- 172.21.15.19:0/172990620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb09c02f6e0 con 0x560d1379cff0 2024-05-09T02:13:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.497+0000 7fb08affd700 1 -- 172.21.15.19:0/172990620 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb0a00aaeb0 con 0x560d1379cff0 2024-05-09T02:13:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.498+0000 7fb0c096cb80 10 client.15654 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.498+0000 7fb0c096cb80 1 -- 172.21.15.19:0/172990620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560d1390da50 con 0x560d1379cff0 2024-05-09T02:13:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.498+0000 7fb08affd700 1 -- 172.21.15.19:0/172990620 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb09c02f6e0 con 0x560d1379cff0 2024-05-09T02:13:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.498+0000 7fb08affd700 1 -- 172.21.15.19:0/172990620 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fb0a00cb030 con 0x560d1379cff0 2024-05-09T02:13:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.498+0000 7fb0c096cb80 10 client.15654 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.498+0000 7fb0c096cb80 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:13:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.498+0000 7fb0c096cb80 20 client.15654 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.498+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/172990620 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560d13916480 0x560d139ae630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.498+0000 7fb0c096cb80 4 client.15654 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T02:13:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.498+0000 7fb0c096cb80 1 -- 172.21.15.19:0/172990620 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560d13840ba0 con 0x560d13916480 2024-05-09T02:13:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.499+0000 7fb0ab7fe700 1 --2- 172.21.15.19:0/172990620 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560d13916480 0x560d139ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.499+0000 7fb0ab7fe700 1 --2- 172.21.15.19:0/172990620 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560d13916480 0x560d139ae630 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99d9c29ae789fb29 server_cookie=fab0a2e15a6c4cf7 in_seq=0 out_seq=0 2024-05-09T02:13:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.499+0000 7fb08affd700 10 client.15654 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.500+0000 7fb08affd700 1 -- 172.21.15.19:0/172990620 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560d13840ba0 con 0x560d13916480 2024-05-09T02:13:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.500+0000 7fb08affd700 10 client.15654 handle_command_reply: tid=0 2024-05-09T02:13:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.500+0000 7fb0c096cb80 1 client.15654 shutdown 2024-05-09T02:13:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.501+0000 7fb0c096cb80 1 -- 172.21.15.19:0/172990620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d1379cff0 msgr2=0x560d139286d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.501+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/172990620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d1379cff0 0x560d139286d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00a9ff0 tx=0x7fb0a00b1040 comp rx=0 tx=0).stop 2024-05-09T02:13:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.501+0000 7fb0c096cb80 1 -- 172.21.15.19:0/172990620 shutdown_connections 2024-05-09T02:13:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.501+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/172990620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d13813d80 0x560d13925160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.501+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/172990620 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560d13916480 0x560d139ae630 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.501+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/172990620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d13799520 0x560d13928190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.501+0000 7fb0c096cb80 1 --2- 172.21.15.19:0/172990620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560d1379cff0 0x560d139286d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.501+0000 7fb0c096cb80 1 -- 172.21.15.19:0/172990620 >> 172.21.15.19:0/172990620 conn(0x560d139115a0 msgr2=0x560d1390b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.501+0000 7fb08affd700 0 client.15654 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.502+0000 7fb0c096cb80 1 -- 172.21.15.19:0/172990620 shutdown_connections 2024-05-09T02:13:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.502+0000 7fb0c096cb80 1 -- 172.21.15.19:0/172990620 wait complete. 2024-05-09T02:13:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.502+0000 7fb0c096cb80 20 client.15654 trim_cache size 0 max 16384 2024-05-09T02:13:37.507 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:13:37.507 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T02:13:37.507 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "5db2a022-983b-4efc-b0bb-e2e7800c4e8e", 2024-05-09T02:13:37.507 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T02:13:37.507 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:13:37.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.502+0000 7fb0b1f56700 1 -- 172.21.15.19:0/673489319 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb0940b2af0 msgr2=0x7fb0940b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:37.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.502+0000 7fb0b1f56700 1 --2- 172.21.15.19:0/673489319 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb0940b2af0 0x7fb0940b5000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac1079e0 tx=0x7fb09c009310 comp rx=0 tx=0).stop 2024-05-09T02:13:37.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.502+0000 7fb0b1f56700 1 -- 172.21.15.19:0/673489319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0ac106ec0 msgr2=0x7fb0ac19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:37.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.502+0000 7fb0b1f56700 1 --2- 172.21.15.19:0/673489319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0ac106ec0 0x7fb0ac19d650 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000c010 tx=0x7fb0a000fa00 comp rx=0 tx=0).stop 2024-05-09T02:13:37.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.502+0000 7fb0b1f56700 1 -- 172.21.15.19:0/673489319 shutdown_connections 2024-05-09T02:13:37.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.502+0000 7fb0b1f56700 1 --2- 172.21.15.19:0/673489319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ac10c5b0 0x7fb0ac1a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.502+0000 7fb0b1f56700 1 --2- 172.21.15.19:0/673489319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0ac106570 0x7fb0ac19d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.502+0000 7fb0b1f56700 1 --2- 172.21.15.19:0/673489319 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb0940b2af0 0x7fb0940b5000 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.502+0000 7fb0b1f56700 1 --2- 172.21.15.19:0/673489319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0ac106ec0 0x7fb0ac19d650 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:37.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.502+0000 7fb0b1f56700 1 -- 172.21.15.19:0/673489319 >> 172.21.15.19:0/673489319 conn(0x7fb0ac102090 msgr2=0x7fb0ac104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:37.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.503+0000 7fb0b1f56700 1 -- 172.21.15.19:0/673489319 shutdown_connections 2024-05-09T02:13:37.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:37.503+0000 7fb0b1f56700 1 -- 172.21.15.19:0/673489319 wait complete. 2024-05-09T02:13:37.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 115.0 s, 2600.6 tps, lat 38.589 ms stddev 42.914 2024-05-09T02:13:38.346 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:13:38.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: pgmap v73: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 MiB/s wr, 583 op/s 2024-05-09T02:13:38.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub queued for path: / 2024-05-09T02:13:38.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:13:38.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:13:38.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:13:38.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:13:38.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:13:38.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:13:38.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:13:38.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:38 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: pgmap v73: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 MiB/s wr, 583 op/s 2024-05-09T02:13:38.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub queued for path: / 2024-05-09T02:13:38.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:13:38.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:13:38.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:13:38.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:13:38.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:13:38.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:13:38.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:13:38.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:38 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.733 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:13:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: pgmap v73: 97 pgs: 97 active+clean; 1.7 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 MiB/s wr, 583 op/s 2024-05-09T02:13:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub queued for path: / 2024-05-09T02:13:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:13:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:13:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:13:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:13:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:13:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:13:38.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:13:38.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:38.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:38 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:13:39.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:39 smithi059 ceph-mon[34781]: pgmap v74: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.5 MiB/s wr, 874 op/s 2024-05-09T02:13:39.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:39 smithi196 ceph-mon[34718]: pgmap v74: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.5 MiB/s wr, 874 op/s 2024-05-09T02:13:39.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:39 smithi019 ceph-mon[30151]: pgmap v74: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.5 MiB/s wr, 874 op/s 2024-05-09T02:13:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.000+0000 7efca6667700 1 -- 172.21.15.19:0/865094411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca01045f0 msgr2=0x7efca0104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.000+0000 7efca6667700 1 --2- 172.21.15.19:0/865094411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca01045f0 0x7efca0104a00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efc900099c0 tx=0x7efc90009cd0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.000+0000 7efca6667700 1 -- 172.21.15.19:0/865094411 shutdown_connections 2024-05-09T02:13:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.000+0000 7efca6667700 1 --2- 172.21.15.19:0/865094411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca0109870 0x7efca01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.000+0000 7efca6667700 1 --2- 172.21.15.19:0/865094411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca01045f0 0x7efca0104a00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.000+0000 7efca6667700 1 --2- 172.21.15.19:0/865094411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca0108f20 0x7efca0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.000+0000 7efca6667700 1 -- 172.21.15.19:0/865094411 >> 172.21.15.19:0/865094411 conn(0x7efca0100040 msgr2=0x7efca0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:42.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.001+0000 7efca6667700 1 -- 172.21.15.19:0/865094411 shutdown_connections 2024-05-09T02:13:42.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.001+0000 7efca6667700 1 -- 172.21.15.19:0/865094411 wait complete. 2024-05-09T02:13:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.002+0000 7efca6667700 1 Processor -- start 2024-05-09T02:13:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.002+0000 7efca6667700 1 -- start start 2024-05-09T02:13:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.002+0000 7efca6667700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca01045f0 0x7efca019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.002+0000 7efca6667700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca0108f20 0x7efca019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.002+0000 7efca6667700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca0109870 0x7efca019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.002+0000 7efca6667700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efca01144a0 con 0x7efca0108f20 2024-05-09T02:13:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.002+0000 7efca6667700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efca0114320 con 0x7efca0109870 2024-05-09T02:13:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.002+0000 7efca6667700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efca0114620 con 0x7efca01045f0 2024-05-09T02:13:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.003+0000 7efc9ffff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca01045f0 0x7efca019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.003+0000 7efc9ffff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca01045f0 0x7efca019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40942/0 (socket says 172.21.15.19:40942) 2024-05-09T02:13:42.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.003+0000 7efc9ffff700 1 -- 172.21.15.19:0/1806289202 learned_addr learned my addr 172.21.15.19:0/1806289202 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:42.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.003+0000 7efc97fff700 1 --2- 172.21.15.19:0/1806289202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca0108f20 0x7efca019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.003+0000 7efca4c04700 1 --2- 172.21.15.19:0/1806289202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca0109870 0x7efca019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.003+0000 7efc9ffff700 1 -- 172.21.15.19:0/1806289202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca0109870 msgr2=0x7efca019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:42.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.003+0000 7efc9ffff700 1 --2- 172.21.15.19:0/1806289202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca0109870 0x7efca019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.003+0000 7efc9ffff700 1 -- 172.21.15.19:0/1806289202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca0108f20 msgr2=0x7efca019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:42.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.003+0000 7efc9ffff700 1 --2- 172.21.15.19:0/1806289202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca0108f20 0x7efca019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.003+0000 7efc9ffff700 1 -- 172.21.15.19:0/1806289202 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca019fc30 con 0x7efca01045f0 2024-05-09T02:13:42.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.003+0000 7efc9ffff700 1 --2- 172.21.15.19:0/1806289202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca01045f0 0x7efca019ae10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efc900098e0 tx=0x7efc900055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:42.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.003+0000 7efc9dffb700 1 -- 172.21.15.19:0/1806289202 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90026070 con 0x7efca01045f0 2024-05-09T02:13:42.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.003+0000 7efc9dffb700 1 -- 172.21.15.19:0/1806289202 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc9000f460 con 0x7efca01045f0 2024-05-09T02:13:42.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.003+0000 7efca6667700 1 -- 172.21.15.19:0/1806289202 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca019fec0 con 0x7efca01045f0 2024-05-09T02:13:42.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.004+0000 7efc9dffb700 1 -- 172.21.15.19:0/1806289202 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90009e60 con 0x7efca01045f0 2024-05-09T02:13:42.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.004+0000 7efca6667700 1 -- 172.21.15.19:0/1806289202 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca01a0330 con 0x7efca01045f0 2024-05-09T02:13:42.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.009+0000 7efc9dffb700 1 -- 172.21.15.19:0/1806289202 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efc90029070 con 0x7efca01045f0 2024-05-09T02:13:42.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.010+0000 7efc9dffb700 1 --2- 172.21.15.19:0/1806289202 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc8c0b2a70 0x7efc8c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.010+0000 7efcb507db80 1 Processor -- start 2024-05-09T02:13:42.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.010+0000 7efcb507db80 1 -- start start 2024-05-09T02:13:42.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efc97fff700 1 --2- 172.21.15.19:0/1806289202 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc8c0b2a70 0x7efc8c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efc9dffb700 1 -- 172.21.15.19:0/1806289202 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7efc900e2e20 con 0x7efca01045f0 2024-05-09T02:13:42.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efcb507db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fadc160 0x563e6fb754f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efcb507db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6f9f24a0 0x563e6fb75a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efcb507db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6fb6f6a0 0x563e6fb6fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efcb507db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563e6fa91ff0 con 0x563e6fadc160 2024-05-09T02:13:42.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efcb507db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563e6fa307f0 con 0x563e6fb6f6a0 2024-05-09T02:13:42.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efcb507db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e6fa8dfb0 con 0x563e6f9f24a0 2024-05-09T02:13:42.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efc9ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fadc160 0x563e6fb754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efc97fff700 1 --2- 172.21.15.19:0/1806289202 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc8c0b2a70 0x7efc8c0b4f80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efca01075f0 tx=0x7efc8800a380 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:42.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efc9ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fadc160 0x563e6fb754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39888/0 (socket says 172.21.15.19:39888) 2024-05-09T02:13:42.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efc9ffff700 1 -- 172.21.15.19:0/2635262713 learned_addr learned my addr 172.21.15.19:0/2635262713 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efc97fff700 1 --2- 172.21.15.19:0/2635262713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6f9f24a0 0x563e6fb75a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efca4c04700 1 --2- 172.21.15.19:0/2635262713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6fb6f6a0 0x563e6fb6fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efc9ffff700 1 -- 172.21.15.19:0/2635262713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6f9f24a0 msgr2=0x563e6fb75a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efc9ffff700 1 --2- 172.21.15.19:0/2635262713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6f9f24a0 0x563e6fb75a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efc9ffff700 1 -- 172.21.15.19:0/2635262713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6fb6f6a0 msgr2=0x563e6fb6fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efc9ffff700 1 --2- 172.21.15.19:0/2635262713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6fb6f6a0 0x563e6fb6fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.011+0000 7efc9ffff700 1 -- 172.21.15.19:0/2635262713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6fa28370 con 0x563e6fadc160 2024-05-09T02:13:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.012+0000 7efc97fff700 1 --2- 172.21.15.19:0/2635262713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6f9f24a0 0x563e6fb75a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:13:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.012+0000 7efc9ffff700 1 --2- 172.21.15.19:0/2635262713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fadc160 0x563e6fb754f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efc900a6210 tx=0x7efc900a9bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad60500ce6d089ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.012+0000 7efc967fc700 1 -- 172.21.15.19:0/2635262713 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc900aa190 con 0x563e6fadc160 2024-05-09T02:13:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.012+0000 7efc967fc700 1 -- 172.21.15.19:0/2635262713 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc900ae540 con 0x563e6fadc160 2024-05-09T02:13:42.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.012+0000 7efc967fc700 1 -- 172.21.15.19:0/2635262713 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc900ae340 con 0x563e6fadc160 2024-05-09T02:13:42.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.013+0000 7efcb507db80 1 -- 172.21.15.19:0/2635262713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fadc160 msgr2=0x563e6fb754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:42.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.013+0000 7efcb507db80 1 --2- 172.21.15.19:0/2635262713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fadc160 0x563e6fb754f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efc900a6210 tx=0x7efc900a9bb0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.013+0000 7efcb507db80 1 -- 172.21.15.19:0/2635262713 shutdown_connections 2024-05-09T02:13:42.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.013+0000 7efcb507db80 1 --2- 172.21.15.19:0/2635262713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6fb6f6a0 0x563e6fb6fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.013+0000 7efcb507db80 1 --2- 172.21.15.19:0/2635262713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fadc160 0x563e6fb754f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.013+0000 7efcb507db80 1 --2- 172.21.15.19:0/2635262713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6f9f24a0 0x563e6fb75a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.013+0000 7efcb507db80 1 -- 172.21.15.19:0/2635262713 >> 172.21.15.19:0/2635262713 conn(0x563e6fa526a0 msgr2=0x563e6fb6a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:42.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.013+0000 7efcb507db80 1 -- 172.21.15.19:0/2635262713 shutdown_connections 2024-05-09T02:13:42.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.013+0000 7efcb507db80 1 -- 172.21.15.19:0/2635262713 wait complete. 2024-05-09T02:13:42.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.014+0000 7efcb507db80 1 -- start start 2024-05-09T02:13:42.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.014+0000 7efcb507db80 1 Processor -- start 2024-05-09T02:13:42.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efcb507db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6f9f24a0 0x563e6fb87580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efcb507db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6fadc160 0x563e6fb87ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efcb507db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fb6f6a0 0x563e6fb81980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efcb507db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563e6f90dba0 con 0x563e6fb6f6a0 2024-05-09T02:13:42.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efca4c04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fb6f6a0 0x563e6fb81980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efca4c04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fb6f6a0 0x563e6fb81980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39894/0 (socket says 172.21.15.19:39894) 2024-05-09T02:13:42.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efca4c04700 1 -- 172.21.15.19:0/3717159986 learned_addr learned my addr 172.21.15.19:0/3717159986 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:42.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efc9ffff700 1 --2- 172.21.15.19:0/3717159986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6f9f24a0 0x563e6fb87580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efcb507db80 1 -- 172.21.15.19:0/3717159986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563e6fa4c150 con 0x563e6fadc160 2024-05-09T02:13:42.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efcb507db80 1 -- 172.21.15.19:0/3717159986 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e6fb70360 con 0x563e6f9f24a0 2024-05-09T02:13:42.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efc97fff700 1 --2- 172.21.15.19:0/3717159986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6fadc160 0x563e6fb87ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efcb507db80 10 client.0 _ll_register_callbacks cb 0x563e6f9036c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:42.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efc9ffff700 1 -- 172.21.15.19:0/3717159986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6fadc160 msgr2=0x563e6fb87ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:42.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efc9ffff700 1 --2- 172.21.15.19:0/3717159986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6fadc160 0x563e6fb87ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efc9ffff700 1 -- 172.21.15.19:0/3717159986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fb6f6a0 msgr2=0x563e6fb81980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:42.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efc9ffff700 1 --2- 172.21.15.19:0/3717159986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fb6f6a0 0x563e6fb81980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.015+0000 7efc9ffff700 1 -- 172.21.15.19:0/3717159986 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6f90d490 con 0x563e6f9f24a0 2024-05-09T02:13:42.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.016+0000 7efc9ffff700 1 --2- 172.21.15.19:0/3717159986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6f9f24a0 0x563e6fb87580 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x563e6fb70170 tx=0x7efc900a9a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59c961a4012e881a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:42.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.016+0000 7efc7f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:13:42.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.016+0000 7efc7f7fe700 1 -- 172.21.15.19:0/3717159986 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc900c3070 con 0x563e6f9f24a0 2024-05-09T02:13:42.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.016+0000 7efcb507db80 1 -- 172.21.15.19:0/3717159986 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc900ae540 con 0x563e6f9f24a0 2024-05-09T02:13:42.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.016+0000 7efc7f7fe700 1 -- 172.21.15.19:0/3717159986 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7efc900aa3f0 con 0x563e6f9f24a0 2024-05-09T02:13:42.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.016+0000 7efc7f7fe700 1 -- 172.21.15.19:0/3717159986 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efc900ae330 con 0x563e6f9f24a0 2024-05-09T02:13:42.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.016+0000 7efcb507db80 10 client.25405 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:42.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.016+0000 7efcb507db80 1 -- 172.21.15.19:0/3717159986 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563e6fb76ea0 con 0x563e6f9f24a0 2024-05-09T02:13:42.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.017+0000 7efc7f7fe700 1 -- 172.21.15.19:0/3717159986 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc900ae540 con 0x563e6f9f24a0 2024-05-09T02:13:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.018+0000 7efc7f7fe700 1 -- 172.21.15.19:0/3717159986 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7efc900a5b00 con 0x563e6f9f24a0 2024-05-09T02:13:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.018+0000 7efcb507db80 10 client.25405 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.018+0000 7efcb507db80 10 client.25405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:13:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.018+0000 7efcb507db80 20 client.25405 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.018+0000 7efcb507db80 1 --2- 172.21.15.19:0/3717159986 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x563e6fb793d0 0x563e6fb7b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.018+0000 7efcb507db80 4 client.25405 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:13:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.019+0000 7efcb507db80 1 -- 172.21.15.19:0/3717159986 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e6fa9f9d0 con 0x563e6fb793d0 2024-05-09T02:13:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.019+0000 7efc97fff700 1 --2- 172.21.15.19:0/3717159986 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x563e6fb793d0 0x563e6fb7b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.019+0000 7efc7f7fe700 10 client.25405 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:42.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.019+0000 7efc97fff700 1 --2- 172.21.15.19:0/3717159986 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x563e6fb793d0 0x563e6fb7b7f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=986e89802052e88c server_cookie=3dba7f0b46ec5e71 in_seq=0 out_seq=0 2024-05-09T02:13:42.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.020+0000 7efc7f7fe700 1 -- 172.21.15.19:0/3717159986 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563e6fa9f9d0 con 0x563e6fb793d0 2024-05-09T02:13:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.020+0000 7efc7f7fe700 10 client.25405 handle_command_reply: tid=0 2024-05-09T02:13:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.020+0000 7efcb507db80 1 client.25405 shutdown 2024-05-09T02:13:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.021+0000 7efcb507db80 1 -- 172.21.15.19:0/3717159986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6f9f24a0 msgr2=0x563e6fb87580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.021+0000 7efcb507db80 1 --2- 172.21.15.19:0/3717159986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6f9f24a0 0x563e6fb87580 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x563e6fb70170 tx=0x7efc900a9a30 comp rx=0 tx=0).stop 2024-05-09T02:13:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.021+0000 7efcb507db80 1 -- 172.21.15.19:0/3717159986 shutdown_connections 2024-05-09T02:13:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.021+0000 7efcb507db80 1 --2- 172.21.15.19:0/3717159986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fb6f6a0 0x563e6fb81980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.021+0000 7efcb507db80 1 --2- 172.21.15.19:0/3717159986 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x563e6fb793d0 0x563e6fb7b7f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.021+0000 7efcb507db80 1 --2- 172.21.15.19:0/3717159986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6f9f24a0 0x563e6fb87580 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.021+0000 7efcb507db80 1 --2- 172.21.15.19:0/3717159986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6fadc160 0x563e6fb87ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.021+0000 7efcb507db80 1 -- 172.21.15.19:0/3717159986 >> 172.21.15.19:0/3717159986 conn(0x563e6fa526a0 msgr2=0x563e6fb6a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.021+0000 7efc7f7fe700 0 client.25405 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.022+0000 7efcb507db80 1 -- 172.21.15.19:0/3717159986 shutdown_connections 2024-05-09T02:13:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.022+0000 7efcb507db80 1 -- 172.21.15.19:0/3717159986 wait complete. 2024-05-09T02:13:42.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.022+0000 7efcb507db80 20 client.25405 trim_cache size 0 max 16384 2024-05-09T02:13:42.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.050+0000 7efcb507db80 1 Processor -- start 2024-05-09T02:13:42.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.050+0000 7efcb507db80 1 -- start start 2024-05-09T02:13:42.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efcb507db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6f9f24a0 0x563e6fb75860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efcb507db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6fadc160 0x563e6fb73860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efcb507db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fb6f6a0 0x563e6fb818a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efcb507db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efca01acbb0 con 0x563e6fb6f6a0 2024-05-09T02:13:42.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efcb507db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efca01ae7b0 con 0x563e6f9f24a0 2024-05-09T02:13:42.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efcb507db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efca01ad280 con 0x563e6fadc160 2024-05-09T02:13:42.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efca4c04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fb6f6a0 0x563e6fb818a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efca4c04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fb6f6a0 0x563e6fb818a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39908/0 (socket says 172.21.15.19:39908) 2024-05-09T02:13:42.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efca4c04700 1 -- 172.21.15.19:0/3827524632 learned_addr learned my addr 172.21.15.19:0/3827524632 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:42.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efc9ffff700 1 --2- 172.21.15.19:0/3827524632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6f9f24a0 0x563e6fb75860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efc97fff700 1 --2- 172.21.15.19:0/3827524632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6fadc160 0x563e6fb73860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efc9ffff700 1 -- 172.21.15.19:0/3827524632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6fadc160 msgr2=0x563e6fb73860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:42.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efc9ffff700 1 --2- 172.21.15.19:0/3827524632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6fadc160 0x563e6fb73860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efc9ffff700 1 -- 172.21.15.19:0/3827524632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fb6f6a0 msgr2=0x563e6fb818a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:42.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efc9ffff700 1 --2- 172.21.15.19:0/3827524632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fb6f6a0 0x563e6fb818a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.051+0000 7efc9ffff700 1 -- 172.21.15.19:0/3827524632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6fb70350 con 0x563e6f9f24a0 2024-05-09T02:13:42.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.052+0000 7efc9ffff700 1 --2- 172.21.15.19:0/3827524632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6f9f24a0 0x563e6fb75860 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efc900aa120 tx=0x7efc900095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f35d3cb90f0a043e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:42.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.052+0000 7efc95ffb700 1 -- 172.21.15.19:0/3827524632 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc900ae330 con 0x563e6f9f24a0 2024-05-09T02:13:42.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.052+0000 7efc95ffb700 1 -- 172.21.15.19:0/3827524632 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc900b2ad0 con 0x563e6f9f24a0 2024-05-09T02:13:42.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.052+0000 7efc95ffb700 1 -- 172.21.15.19:0/3827524632 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc900aa3f0 con 0x563e6f9f24a0 2024-05-09T02:13:42.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.053+0000 7efcb507db80 1 -- 172.21.15.19:0/3827524632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6f9f24a0 msgr2=0x563e6fb75860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:42.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.053+0000 7efcb507db80 1 --2- 172.21.15.19:0/3827524632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6f9f24a0 0x563e6fb75860 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efc900aa120 tx=0x7efc900095f0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.053+0000 7efcb507db80 1 -- 172.21.15.19:0/3827524632 shutdown_connections 2024-05-09T02:13:42.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.053+0000 7efcb507db80 1 --2- 172.21.15.19:0/3827524632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fb6f6a0 0x563e6fb818a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.053+0000 7efcb507db80 1 --2- 172.21.15.19:0/3827524632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6f9f24a0 0x563e6fb75860 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.053+0000 7efcb507db80 1 --2- 172.21.15.19:0/3827524632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6fadc160 0x563e6fb73860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.053+0000 7efcb507db80 1 -- 172.21.15.19:0/3827524632 >> 172.21.15.19:0/3827524632 conn(0x563e6fa526a0 msgr2=0x563e6fb6e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.053+0000 7efcb507db80 1 -- 172.21.15.19:0/3827524632 shutdown_connections 2024-05-09T02:13:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.053+0000 7efcb507db80 1 -- 172.21.15.19:0/3827524632 wait complete. 2024-05-09T02:13:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.053+0000 7efcb507db80 1 -- start start 2024-05-09T02:13:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.054+0000 7efcb507db80 1 Processor -- start 2024-05-09T02:13:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.054+0000 7efcb507db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6f9f24a0 0x563e6fb88400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.054+0000 7efcb507db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fadc160 0x563e6fb88940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.054+0000 7efcb507db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6fb6f6a0 0x563e6fb88e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.054+0000 7efcb507db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efca01ae020 con 0x563e6fadc160 2024-05-09T02:13:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.054+0000 7efc9ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6f9f24a0 0x563e6fb88400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.054+0000 7efc97fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fadc160 0x563e6fb88940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.054+0000 7efcb507db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efca01ad950 con 0x563e6f9f24a0 2024-05-09T02:13:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.054+0000 7efcb507db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efca01aab50 con 0x563e6fb6f6a0 2024-05-09T02:13:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.054+0000 7efc9ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6f9f24a0 0x563e6fb88400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:43014/0 (socket says 172.21.15.19:43014) 2024-05-09T02:13:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.054+0000 7efc9ffff700 1 -- 172.21.15.19:0/3410846072 learned_addr learned my addr 172.21.15.19:0/3410846072 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:13:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.054+0000 7efcb507db80 10 client.0 _ll_register_callbacks cb 0x563e6f9036c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:13:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.055+0000 7efca4c04700 1 --2- 172.21.15.19:0/3410846072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6fb6f6a0 0x563e6fb88e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.055+0000 7efc9ffff700 1 -- 172.21.15.19:0/3410846072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6fb6f6a0 msgr2=0x563e6fb88e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.055+0000 7efc9ffff700 1 --2- 172.21.15.19:0/3410846072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6fb6f6a0 0x563e6fb88e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.055+0000 7efc9ffff700 1 -- 172.21.15.19:0/3410846072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fadc160 msgr2=0x563e6fb88940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:13:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.055+0000 7efc9ffff700 1 --2- 172.21.15.19:0/3410846072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fadc160 0x563e6fb88940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.055+0000 7efc9ffff700 1 -- 172.21.15.19:0/3410846072 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6fb63f60 con 0x563e6f9f24a0 2024-05-09T02:13:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.055+0000 7efca4c04700 1 --2- 172.21.15.19:0/3410846072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6fb6f6a0 0x563e6fb88e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:13:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.055+0000 7efc9ffff700 1 --2- 172.21.15.19:0/3410846072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6f9f24a0 0x563e6fb88400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efc900a9e90 tx=0x7efc900a99b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38439420d856718d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:13:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.055+0000 7efc7f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:13:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.055+0000 7efc7f7fe700 1 -- 172.21.15.19:0/3410846072 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc900ae540 con 0x563e6f9f24a0 2024-05-09T02:13:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.055+0000 7efcb507db80 1 -- 172.21.15.19:0/3410846072 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc900b2ad0 con 0x563e6f9f24a0 2024-05-09T02:13:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.055+0000 7efc7f7fe700 1 -- 172.21.15.19:0/3410846072 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7efc900a8ee0 con 0x563e6f9f24a0 2024-05-09T02:13:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.055+0000 7efc7f7fe700 1 -- 172.21.15.19:0/3410846072 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efc900b2f60 con 0x563e6f9f24a0 2024-05-09T02:13:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.056+0000 7efcb507db80 10 client.25343 fetch_fsmap learned FSMap version 33 2024-05-09T02:13:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.056+0000 7efcb507db80 1 -- 172.21.15.19:0/3410846072 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563e6fb6bd00 con 0x563e6f9f24a0 2024-05-09T02:13:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.056+0000 7efc7f7fe700 1 -- 172.21.15.19:0/3410846072 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc900b2ad0 con 0x563e6f9f24a0 2024-05-09T02:13:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.056+0000 7efc7f7fe700 1 -- 172.21.15.19:0/3410846072 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7efc900c4af0 con 0x563e6f9f24a0 2024-05-09T02:13:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.056+0000 7efcb507db80 10 client.25343 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:13:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.056+0000 7efcb507db80 10 client.25343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:13:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.056+0000 7efcb507db80 20 client.25343 populate_metadata read hostname 'smithi019' 2024-05-09T02:13:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.057+0000 7efcb507db80 1 --2- 172.21.15.19:0/3410846072 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x563e6fc042b0 0x563e6fc066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:13:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.057+0000 7efcb507db80 4 client.25343 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:13:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.057+0000 7efcb507db80 1 -- 172.21.15.19:0/3410846072 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e6fa99a70 con 0x563e6fc042b0 2024-05-09T02:13:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.059+0000 7efc97fff700 1 --2- 172.21.15.19:0/3410846072 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x563e6fc042b0 0x563e6fc066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:13:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.059+0000 7efc7f7fe700 10 client.25343 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.059+0000 7efc97fff700 1 --2- 172.21.15.19:0/3410846072 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x563e6fc042b0 0x563e6fc066d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ee2ee9f0dd5cd0 server_cookie=91060ada834f52f6 in_seq=0 out_seq=0 2024-05-09T02:13:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.060+0000 7efc7f7fe700 1 -- 172.21.15.19:0/3410846072 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563e6fa99a70 con 0x563e6fc042b0 2024-05-09T02:13:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.060+0000 7efc7f7fe700 10 client.25343 handle_command_reply: tid=0 2024-05-09T02:13:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.060+0000 7efcb507db80 1 client.25343 shutdown 2024-05-09T02:13:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.060+0000 7efcb507db80 1 -- 172.21.15.19:0/3410846072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6f9f24a0 msgr2=0x563e6fb88400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.060+0000 7efcb507db80 1 --2- 172.21.15.19:0/3410846072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6f9f24a0 0x563e6fb88400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efc900a9e90 tx=0x7efc900a99b0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.060+0000 7efcb507db80 1 -- 172.21.15.19:0/3410846072 shutdown_connections 2024-05-09T02:13:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.060+0000 7efcb507db80 1 --2- 172.21.15.19:0/3410846072 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x563e6fc042b0 0x563e6fc066d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.060+0000 7efcb507db80 1 --2- 172.21.15.19:0/3410846072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6fb6f6a0 0x563e6fb88e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.060+0000 7efcb507db80 1 --2- 172.21.15.19:0/3410846072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6f9f24a0 0x563e6fb88400 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.060+0000 7efc7f7fe700 0 client.25343 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:13:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.060+0000 7efcb507db80 1 --2- 172.21.15.19:0/3410846072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6fadc160 0x563e6fb88940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.060+0000 7efcb507db80 1 -- 172.21.15.19:0/3410846072 >> 172.21.15.19:0/3410846072 conn(0x563e6fa526a0 msgr2=0x563e6fb72e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.060+0000 7efcb507db80 1 -- 172.21.15.19:0/3410846072 shutdown_connections 2024-05-09T02:13:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.060+0000 7efcb507db80 1 -- 172.21.15.19:0/3410846072 wait complete. 2024-05-09T02:13:42.067 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:13:42.067 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T02:13:42.067 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T02:13:42.067 INFO:teuthology.orchestra.run.smithi019.stdout: "5db2a022-983b-4efc-b0bb-e2e7800c4e8e": { 2024-05-09T02:13:42.067 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T02:13:42.067 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "5db2a022-983b-4efc-b0bb-e2e7800c4e8e", 2024-05-09T02:13:42.067 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T02:13:42.067 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:13:42.067 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:13:42.067 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:13:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.060+0000 7efcb507db80 20 client.25343 trim_cache size 0 max 16384 2024-05-09T02:13:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.061+0000 7efca6667700 1 -- 172.21.15.19:0/1806289202 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc8c0b2a70 msgr2=0x7efc8c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.061+0000 7efca6667700 1 --2- 172.21.15.19:0/1806289202 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc8c0b2a70 0x7efc8c0b4f80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efca01075f0 tx=0x7efc8800a380 comp rx=0 tx=0).stop 2024-05-09T02:13:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.061+0000 7efca6667700 1 -- 172.21.15.19:0/1806289202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca01045f0 msgr2=0x7efca019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:13:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.061+0000 7efca6667700 1 --2- 172.21.15.19:0/1806289202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca01045f0 0x7efca019ae10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efc900098e0 tx=0x7efc900055f0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.061+0000 7efca6667700 1 -- 172.21.15.19:0/1806289202 shutdown_connections 2024-05-09T02:13:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.061+0000 7efca6667700 1 --2- 172.21.15.19:0/1806289202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca0109870 0x7efca019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.061+0000 7efca6667700 1 --2- 172.21.15.19:0/1806289202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca01045f0 0x7efca019ae10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.061+0000 7efca6667700 1 --2- 172.21.15.19:0/1806289202 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc8c0b2a70 0x7efc8c0b4f80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.061+0000 7efca6667700 1 --2- 172.21.15.19:0/1806289202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca0108f20 0x7efca019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:13:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.061+0000 7efca6667700 1 -- 172.21.15.19:0/1806289202 >> 172.21.15.19:0/1806289202 conn(0x7efca0100040 msgr2=0x7efca0102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:13:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.061+0000 7efca6667700 1 -- 172.21.15.19:0/1806289202 shutdown_connections 2024-05-09T02:13:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:13:42.061+0000 7efca6667700 1 -- 172.21.15.19:0/1806289202 wait complete. 2024-05-09T02:13:42.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:42 smithi059 ceph-mon[34781]: pgmap v75: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.8 MiB/s wr, 585 op/s 2024-05-09T02:13:42.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:42 smithi196 ceph-mon[34718]: pgmap v75: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.8 MiB/s wr, 585 op/s 2024-05-09T02:13:42.553 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:42 smithi019 ceph-mon[30151]: pgmap v75: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.8 MiB/s wr, 585 op/s 2024-05-09T02:13:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 120.0 s, 2628.6 tps, lat 38.053 ms stddev 43.310 2024-05-09T02:13:43.146 INFO:tasks.cephfs.filesystem:scrub status for tag:5db2a022-983b-4efc-b0bb-e2e7800c4e8e - {'path': '/', 'tag': '5db2a022-983b-4efc-b0bb-e2e7800c4e8e', 'options': 'recursive,force'} 2024-05-09T02:13:44.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:44 smithi059 ceph-mon[34781]: pgmap v76: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.8 MiB/s wr, 585 op/s 2024-05-09T02:13:44.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:44 smithi196 ceph-mon[34718]: pgmap v76: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.8 MiB/s wr, 585 op/s 2024-05-09T02:13:44.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:44 smithi019 ceph-mon[30151]: pgmap v76: 97 pgs: 97 active+clean; 1.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.8 MiB/s wr, 585 op/s 2024-05-09T02:13:46.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:46 smithi059 ceph-mon[34781]: pgmap v77: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.6 MiB/s wr, 872 op/s 2024-05-09T02:13:46.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:46 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:13:46.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:46 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:13:46.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:46 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:13:46.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:46 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:13:46.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:46 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:13:46.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:46 smithi196 ceph-mon[34718]: pgmap v77: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.6 MiB/s wr, 872 op/s 2024-05-09T02:13:46.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:46 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:13:46.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:46 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:13:46.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:46 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:13:46.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:46 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:13:46.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:46 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:13:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:46 smithi019 ceph-mon[30151]: pgmap v77: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.6 MiB/s wr, 872 op/s 2024-05-09T02:13:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:46 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:13:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:46 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:13:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:46 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:13:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:46 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:13:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:46 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:13:47.656 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 125.0 s, 3110.8 tps, lat 32.165 ms stddev 34.316 2024-05-09T02:13:48.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:48 smithi059 ceph-mon[34781]: pgmap v78: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.5 MiB/s wr, 578 op/s 2024-05-09T02:13:48.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:48 smithi196 ceph-mon[34718]: pgmap v78: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.5 MiB/s wr, 578 op/s 2024-05-09T02:13:48.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:48 smithi019 ceph-mon[30151]: pgmap v78: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.5 MiB/s wr, 578 op/s 2024-05-09T02:13:50.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:50 smithi196 ceph-mon[34718]: pgmap v79: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 10 MiB/s wr, 888 op/s 2024-05-09T02:13:50.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:50 smithi019 ceph-mon[30151]: pgmap v79: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 10 MiB/s wr, 888 op/s 2024-05-09T02:13:50.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:50 smithi059 ceph-mon[34781]: pgmap v79: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 10 MiB/s wr, 888 op/s 2024-05-09T02:13:52.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:52 smithi196 ceph-mon[34718]: pgmap v80: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 597 op/s 2024-05-09T02:13:52.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:52 smithi019 ceph-mon[30151]: pgmap v80: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 597 op/s 2024-05-09T02:13:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 130.0 s, 2722.8 tps, lat 36.662 ms stddev 38.477 2024-05-09T02:13:52.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:52 smithi059 ceph-mon[34781]: pgmap v80: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 597 op/s 2024-05-09T02:13:54.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:54 smithi196 ceph-mon[34718]: pgmap v81: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 597 op/s 2024-05-09T02:13:54.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:54 smithi019 ceph-mon[30151]: pgmap v81: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 597 op/s 2024-05-09T02:13:54.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:54 smithi059 ceph-mon[34781]: pgmap v81: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 597 op/s 2024-05-09T02:13:56.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:56 smithi196 ceph-mon[34718]: pgmap v82: 97 pgs: 97 active+clean; 1.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 942 op/s 2024-05-09T02:13:56.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:56 smithi019 ceph-mon[30151]: pgmap v82: 97 pgs: 97 active+clean; 1.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 942 op/s 2024-05-09T02:13:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:56 smithi059 ceph-mon[34781]: pgmap v82: 97 pgs: 97 active+clean; 1.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 942 op/s 2024-05-09T02:13:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 135.0 s, 2674.2 tps, lat 37.402 ms stddev 38.887 2024-05-09T02:13:58.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:13:58 smithi196 ceph-mon[34718]: pgmap v83: 97 pgs: 97 active+clean; 1.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.0 MiB/s wr, 655 op/s 2024-05-09T02:13:58.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:13:58 smithi019 ceph-mon[30151]: pgmap v83: 97 pgs: 97 active+clean; 1.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.0 MiB/s wr, 655 op/s 2024-05-09T02:13:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:13:58 smithi059 ceph-mon[34781]: pgmap v83: 97 pgs: 97 active+clean; 1.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.0 MiB/s wr, 655 op/s 2024-05-09T02:14:00.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:00 smithi196 ceph-mon[34718]: pgmap v84: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 944 op/s 2024-05-09T02:14:00.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:00 smithi019 ceph-mon[30151]: pgmap v84: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 944 op/s 2024-05-09T02:14:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:00 smithi059 ceph-mon[34781]: pgmap v84: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 944 op/s 2024-05-09T02:14:02.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:02 smithi196 ceph-mon[34718]: pgmap v85: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 634 op/s 2024-05-09T02:14:02.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:02 smithi019 ceph-mon[30151]: pgmap v85: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 634 op/s 2024-05-09T02:14:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 140.0 s, 2015.2 tps, lat 48.461 ms stddev 84.129 2024-05-09T02:14:02.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:02 smithi059 ceph-mon[34781]: pgmap v85: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 634 op/s 2024-05-09T02:14:04.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:04 smithi196 ceph-mon[34718]: pgmap v86: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 634 op/s 2024-05-09T02:14:04.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:04 smithi019 ceph-mon[30151]: pgmap v86: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 634 op/s 2024-05-09T02:14:04.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:04 smithi059 ceph-mon[34781]: pgmap v86: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 634 op/s 2024-05-09T02:14:06.518 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:06 smithi059 ceph-mon[34781]: pgmap v87: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 933 op/s 2024-05-09T02:14:06.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:06 smithi196 ceph-mon[34718]: pgmap v87: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 933 op/s 2024-05-09T02:14:06.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:06 smithi019 ceph-mon[30151]: pgmap v87: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 933 op/s 2024-05-09T02:14:07.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 145.0 s, 323.8 tps, lat 303.799 ms stddev 178.299 2024-05-09T02:14:08.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:08 smithi196 ceph-mon[34718]: pgmap v88: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.2 MiB/s wr, 588 op/s 2024-05-09T02:14:08.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:08 smithi019 ceph-mon[30151]: pgmap v88: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.2 MiB/s wr, 588 op/s 2024-05-09T02:14:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:08 smithi059 ceph-mon[34781]: pgmap v88: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.2 MiB/s wr, 588 op/s 2024-05-09T02:14:10.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:10 smithi196 ceph-mon[34718]: pgmap v89: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.4 MiB/s wr, 832 op/s 2024-05-09T02:14:10.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:10 smithi019 ceph-mon[30151]: pgmap v89: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.4 MiB/s wr, 832 op/s 2024-05-09T02:14:10.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:10 smithi059 ceph-mon[34781]: pgmap v89: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.4 MiB/s wr, 832 op/s 2024-05-09T02:14:12.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:12 smithi196 ceph-mon[34718]: pgmap v90: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.4 MiB/s wr, 542 op/s 2024-05-09T02:14:12.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:12 smithi019 ceph-mon[30151]: pgmap v90: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.4 MiB/s wr, 542 op/s 2024-05-09T02:14:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 150.0 s, 1319.8 tps, lat 78.968 ms stddev 150.640 2024-05-09T02:14:12.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:12 smithi059 ceph-mon[34781]: pgmap v90: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.4 MiB/s wr, 542 op/s 2024-05-09T02:14:13.148 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:14:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:14:14.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:14 smithi196 ceph-mon[34718]: pgmap v91: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.4 MiB/s wr, 542 op/s 2024-05-09T02:14:14.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:14 smithi019 ceph-mon[30151]: pgmap v91: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.4 MiB/s wr, 542 op/s 2024-05-09T02:14:14.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:14 smithi059 ceph-mon[34781]: pgmap v91: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.4 MiB/s wr, 542 op/s 2024-05-09T02:14:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.460+0000 7f5884df4700 1 -- 172.21.15.19:0/742192067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5880108740 msgr2=0x7f588006b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.460+0000 7f5884df4700 1 --2- 172.21.15.19:0/742192067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5880108740 0x7f588006b610 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f587000b0a0 tx=0x7f587000b3b0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.460+0000 7f5884df4700 1 -- 172.21.15.19:0/742192067 shutdown_connections 2024-05-09T02:14:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.460+0000 7f5884df4700 1 --2- 172.21.15.19:0/742192067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5880108740 0x7f588006b610 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.460+0000 7f5884df4700 1 --2- 172.21.15.19:0/742192067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58801034c0 0x7f58801038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.460+0000 7f5884df4700 1 --2- 172.21.15.19:0/742192067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5880107df0 0x7f5880108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.460+0000 7f5884df4700 1 -- 172.21.15.19:0/742192067 >> 172.21.15.19:0/742192067 conn(0x7f58800fedc0 msgr2=0x7f58801011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.460+0000 7f5884df4700 1 -- 172.21.15.19:0/742192067 shutdown_connections 2024-05-09T02:14:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.460+0000 7f5884df4700 1 -- 172.21.15.19:0/742192067 wait complete. 2024-05-09T02:14:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.461+0000 7f5884df4700 1 Processor -- start 2024-05-09T02:14:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.461+0000 7f5884df4700 1 -- start start 2024-05-09T02:14:15.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.462+0000 7f5884df4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58801034c0 0x7f588019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.462+0000 7f5884df4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5880107df0 0x7f588019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.462+0000 7f5884df4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5880108740 0x7f588019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.462+0000 7f5884df4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58801144e0 con 0x7f58801034c0 2024-05-09T02:14:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.462+0000 7f5884df4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5880114360 con 0x7f5880108740 2024-05-09T02:14:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.462+0000 7f5884df4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5880114660 con 0x7f5880107df0 2024-05-09T02:14:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.462+0000 7f587e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58801034c0 0x7f588019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.462+0000 7f587e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58801034c0 0x7f588019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35626/0 (socket says 172.21.15.19:35626) 2024-05-09T02:14:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.462+0000 7f587dd9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5880107df0 0x7f588019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.462+0000 7f587dd9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5880107df0 0x7f588019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40190/0 (socket says 172.21.15.19:40190) 2024-05-09T02:14:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.462+0000 7f587dd9b700 1 -- 172.21.15.19:0/344729898 learned_addr learned my addr 172.21.15.19:0/344729898 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.462+0000 7f587ed9d700 1 --2- 172.21.15.19:0/344729898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5880108740 0x7f588019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.463+0000 7f587dd9b700 1 -- 172.21.15.19:0/344729898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5880108740 msgr2=0x7f588019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.463+0000 7f587dd9b700 1 --2- 172.21.15.19:0/344729898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5880108740 0x7f588019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.463+0000 7f587dd9b700 1 -- 172.21.15.19:0/344729898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58801034c0 msgr2=0x7f588019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.463+0000 7f587dd9b700 1 --2- 172.21.15.19:0/344729898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58801034c0 0x7f588019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.463+0000 7f587dd9b700 1 -- 172.21.15.19:0/344729898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588019fce0 con 0x7f5880107df0 2024-05-09T02:14:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.463+0000 7f587e59c700 1 --2- 172.21.15.19:0/344729898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58801034c0 0x7f588019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:14:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.463+0000 7f587dd9b700 1 --2- 172.21.15.19:0/344729898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5880107df0 0x7f588019b3b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f587400b500 tx=0x7f587400b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.463+0000 7f586f7fe700 1 -- 172.21.15.19:0/344729898 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874015020 con 0x7f5880107df0 2024-05-09T02:14:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.463+0000 7f586f7fe700 1 -- 172.21.15.19:0/344729898 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5874011ce0 con 0x7f5880107df0 2024-05-09T02:14:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.463+0000 7f586f7fe700 1 -- 172.21.15.19:0/344729898 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874010d00 con 0x7f5880107df0 2024-05-09T02:14:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.463+0000 7f5884df4700 1 -- 172.21.15.19:0/344729898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588019ffd0 con 0x7f5880107df0 2024-05-09T02:14:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.463+0000 7f5884df4700 1 -- 172.21.15.19:0/344729898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58801a78f0 con 0x7f5880107df0 2024-05-09T02:14:15.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.468+0000 7f586f7fe700 1 -- 172.21.15.19:0/344729898 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5874018020 con 0x7f5880107df0 2024-05-09T02:14:15.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.469+0000 7f589380ab80 1 Processor -- start 2024-05-09T02:14:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.470+0000 7f589380ab80 1 -- start start 2024-05-09T02:14:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.470+0000 7f586f7fe700 1 --2- 172.21.15.19:0/344729898 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f58680b2a70 0x7f58680b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.470+0000 7f587ed9d700 1 --2- 172.21.15.19:0/344729898 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f58680b2a70 0x7f58680b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.470+0000 7f589380ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563692786d00 0x56369288f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.470+0000 7f586f7fe700 1 -- 172.21.15.19:0/344729898 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f58740e1bb0 con 0x7f5880107df0 2024-05-09T02:14:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.470+0000 7f589380ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56369270c4a0 0x56369288f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.470+0000 7f589380ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692889400 0x563692889810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.470+0000 7f589380ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5636927ac0a0 con 0x563692889400 2024-05-09T02:14:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.470+0000 7f589380ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5636927519a0 con 0x56369270c4a0 2024-05-09T02:14:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.470+0000 7f589380ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5636927a8060 con 0x563692786d00 2024-05-09T02:14:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.470+0000 7f587e59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563692786d00 0x56369288f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.470+0000 7f587dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56369270c4a0 0x56369288f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.470+0000 7f587e59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563692786d00 0x56369288f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40204/0 (socket says 172.21.15.19:40204) 2024-05-09T02:14:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.470+0000 7f587ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692889400 0x563692889810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.470+0000 7f587e59c700 1 -- 172.21.15.19:0/169962358 learned_addr learned my addr 172.21.15.19:0/169962358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.471+0000 7f587ed9d700 1 --2- 172.21.15.19:0/344729898 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f58680b2a70 0x7f58680b4f80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5870009900 tx=0x7f5870009410 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.471+0000 7f587dd9b700 1 -- 172.21.15.19:0/169962358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563692786d00 msgr2=0x56369288f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.471+0000 7f587dd9b700 1 --2- 172.21.15.19:0/169962358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563692786d00 0x56369288f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.471+0000 7f587dd9b700 1 -- 172.21.15.19:0/169962358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692889400 msgr2=0x563692889810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.471+0000 7f587dd9b700 1 --2- 172.21.15.19:0/169962358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692889400 0x563692889810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.471+0000 7f587dd9b700 1 -- 172.21.15.19:0/169962358 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563692742500 con 0x56369270c4a0 2024-05-09T02:14:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.471+0000 7f587dd9b700 1 --2- 172.21.15.19:0/169962358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56369270c4a0 0x56369288f780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f58740df410 tx=0x7f58740a87e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bf9b8c4dfbd9ce3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.471+0000 7f586cff9700 1 -- 172.21.15.19:0/169962358 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58740a8fa0 con 0x56369270c4a0 2024-05-09T02:14:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.471+0000 7f586cff9700 1 -- 172.21.15.19:0/169962358 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58740ad350 con 0x56369270c4a0 2024-05-09T02:14:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.471+0000 7f586cff9700 1 -- 172.21.15.19:0/169962358 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58740a8fa0 con 0x56369270c4a0 2024-05-09T02:14:15.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.473+0000 7f589380ab80 1 -- 172.21.15.19:0/169962358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56369270c4a0 msgr2=0x56369288f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:15.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.473+0000 7f589380ab80 1 --2- 172.21.15.19:0/169962358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56369270c4a0 0x56369288f780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f58740df410 tx=0x7f58740a87e0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.473+0000 7f589380ab80 1 -- 172.21.15.19:0/169962358 shutdown_connections 2024-05-09T02:14:15.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.473+0000 7f589380ab80 1 --2- 172.21.15.19:0/169962358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692889400 0x563692889810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.473+0000 7f589380ab80 1 --2- 172.21.15.19:0/169962358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563692786d00 0x56369288f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.473+0000 7f589380ab80 1 --2- 172.21.15.19:0/169962358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56369270c4a0 0x56369288f780 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.473+0000 7f589380ab80 1 -- 172.21.15.19:0/169962358 >> 172.21.15.19:0/169962358 conn(0x56369270ff70 msgr2=0x563692884220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:15.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.473+0000 7f589380ab80 1 -- 172.21.15.19:0/169962358 shutdown_connections 2024-05-09T02:14:15.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.473+0000 7f589380ab80 1 -- 172.21.15.19:0/169962358 wait complete. 2024-05-09T02:14:15.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.474+0000 7f589380ab80 1 -- start start 2024-05-09T02:14:15.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.474+0000 7f589380ab80 1 Processor -- start 2024-05-09T02:14:15.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.474+0000 7f589380ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56369270c4a0 0x563692892ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.474+0000 7f589380ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 0x563692893530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f589380ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563692889400 0x563692893a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f589380ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56369288a100 con 0x563692786d00 2024-05-09T02:14:15.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f589380ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563692746b20 con 0x563692889400 2024-05-09T02:14:15.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f589380ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56369289f1d0 con 0x56369270c4a0 2024-05-09T02:14:15.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f587dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 0x563692893530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f587e59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56369270c4a0 0x563692892ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f587ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563692889400 0x563692893a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f587e59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56369270c4a0 0x563692892ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40216/0 (socket says 172.21.15.19:40216) 2024-05-09T02:14:15.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f587ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563692889400 0x563692893a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56274/0 (socket says 172.21.15.19:56274) 2024-05-09T02:14:15.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f587dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 0x563692893530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35648/0 (socket says 172.21.15.19:35648) 2024-05-09T02:14:15.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f587e59c700 1 -- 172.21.15.19:0/1333053482 learned_addr learned my addr 172.21.15.19:0/1333053482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:15.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f587e59c700 1 -- 172.21.15.19:0/1333053482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563692889400 msgr2=0x563692893a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:15.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f587e59c700 1 --2- 172.21.15.19:0/1333053482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563692889400 0x563692893a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f587e59c700 1 -- 172.21.15.19:0/1333053482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 msgr2=0x563692893530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:15.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f587e59c700 1 --2- 172.21.15.19:0/1333053482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 0x563692893530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f587e59c700 1 -- 172.21.15.19:0/1333053482 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563692627490 con 0x56369270c4a0 2024-05-09T02:14:15.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f589380ab80 10 client.0 _ll_register_callbacks cb 0x56369261d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:15.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f589380ab80 1 -- 172.21.15.19:0/1333053482 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58740ad350 con 0x56369270c4a0 2024-05-09T02:14:15.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f587ed9d700 1 --2- 172.21.15.19:0/1333053482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563692889400 0x563692893a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:14:15.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.475+0000 7f587dd9b700 1 --2- 172.21.15.19:0/1333053482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 0x563692893530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:14:15.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.476+0000 7f587e59c700 1 --2- 172.21.15.19:0/1333053482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56369270c4a0 0x563692892ff0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5878012950 tx=0x7f5878012c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6140eedfd703171b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:15.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.476+0000 7f5865ffb700 10 client.25417 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:14:15.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.476+0000 7f5865ffb700 1 -- 172.21.15.19:0/1333053482 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878011ac0 con 0x56369270c4a0 2024-05-09T02:14:15.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.476+0000 7f5865ffb700 1 -- 172.21.15.19:0/1333053482 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f5878011c60 con 0x56369270c4a0 2024-05-09T02:14:15.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.476+0000 7f5865ffb700 1 -- 172.21.15.19:0/1333053482 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f587801c420 con 0x56369270c4a0 2024-05-09T02:14:15.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.478+0000 7f589380ab80 10 client.25417 fetch_fsmap learned FSMap version 33 2024-05-09T02:14:15.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.478+0000 7f589380ab80 1 -- 172.21.15.19:0/1333053482 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=33}) -- 0x5636928a0ac0 con 0x56369270c4a0 2024-05-09T02:14:15.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.478+0000 7f5865ffb700 1 -- 172.21.15.19:0/1333053482 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f587801c5c0 con 0x56369270c4a0 2024-05-09T02:14:15.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.479+0000 7f589380ab80 10 client.25417 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:14:15.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.479+0000 7f589380ab80 10 client.25417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:14:15.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.479+0000 7f589380ab80 20 client.25417 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:15.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.479+0000 7f589380ab80 1 --2- 172.21.15.19:0/1333053482 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5636928a3040 0x5636928a5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.479+0000 7f589380ab80 4 client.25417 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:14:15.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.479+0000 7f589380ab80 1 -- 172.21.15.19:0/1333053482 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636927b9a80 con 0x5636928a3040 2024-05-09T02:14:15.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.479+0000 7f587dd9b700 1 --2- 172.21.15.19:0/1333053482 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5636928a3040 0x5636928a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.480+0000 7f587dd9b700 1 --2- 172.21.15.19:0/1333053482 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5636928a3040 0x5636928a5460 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba939a78f6b95e8c server_cookie=efb300c76328182d in_seq=0 out_seq=0 2024-05-09T02:14:15.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.480+0000 7f5865ffb700 10 client.25417 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:15.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.488+0000 7f5865ffb700 1 -- 172.21.15.19:0/1333053482 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5636927b9a80 con 0x5636928a3040 2024-05-09T02:14:15.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.488+0000 7f5865ffb700 10 client.25417 handle_command_reply: tid=0 2024-05-09T02:14:15.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.488+0000 7f589380ab80 1 client.25417 shutdown 2024-05-09T02:14:15.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.492+0000 7f589380ab80 1 -- 172.21.15.19:0/1333053482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56369270c4a0 msgr2=0x563692892ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:15.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.492+0000 7f589380ab80 1 --2- 172.21.15.19:0/1333053482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56369270c4a0 0x563692892ff0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5878012950 tx=0x7f5878012c60 comp rx=0 tx=0).stop 2024-05-09T02:14:15.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.493+0000 7f589380ab80 1 -- 172.21.15.19:0/1333053482 shutdown_connections 2024-05-09T02:14:15.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.493+0000 7f589380ab80 1 --2- 172.21.15.19:0/1333053482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563692889400 0x563692893a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.493+0000 7f589380ab80 1 --2- 172.21.15.19:0/1333053482 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5636928a3040 0x5636928a5460 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.493+0000 7f589380ab80 1 --2- 172.21.15.19:0/1333053482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56369270c4a0 0x563692892ff0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.493+0000 7f589380ab80 1 --2- 172.21.15.19:0/1333053482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 0x563692893530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.493+0000 7f589380ab80 1 -- 172.21.15.19:0/1333053482 >> 172.21.15.19:0/1333053482 conn(0x56369270ff70 msgr2=0x56369287dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:15.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.495+0000 7f5865ffb700 0 client.25417 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:15.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.495+0000 7f589380ab80 1 -- 172.21.15.19:0/1333053482 shutdown_connections 2024-05-09T02:14:15.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.496+0000 7f589380ab80 1 -- 172.21.15.19:0/1333053482 wait complete. 2024-05-09T02:14:15.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.496+0000 7f589380ab80 20 client.25417 trim_cache size 0 max 16384 2024-05-09T02:14:15.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.526+0000 7f589380ab80 1 Processor -- start 2024-05-09T02:14:15.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.526+0000 7f589380ab80 1 -- start start 2024-05-09T02:14:15.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.527+0000 7f589380ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56369270c4a0 0x5636928938b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.527+0000 7f589380ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 0x56369287dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.527+0000 7f589380ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563692889400 0x56369287e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.527+0000 7f589380ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58801acd70 con 0x563692786d00 2024-05-09T02:14:15.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.527+0000 7f589380ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58801ae970 con 0x56369270c4a0 2024-05-09T02:14:15.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.527+0000 7f589380ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58801ad440 con 0x563692889400 2024-05-09T02:14:15.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.527+0000 7f587e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56369270c4a0 0x5636928938b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.527+0000 7f587e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56369270c4a0 0x5636928938b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56280/0 (socket says 172.21.15.19:56280) 2024-05-09T02:14:15.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.527+0000 7f587e59c700 1 -- 172.21.15.19:0/2129862085 learned_addr learned my addr 172.21.15.19:0/2129862085 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:15.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.527+0000 7f587dd9b700 1 --2- 172.21.15.19:0/2129862085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 0x56369287dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.528+0000 7f587ed9d700 1 --2- 172.21.15.19:0/2129862085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563692889400 0x56369287e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.528+0000 7f587dd9b700 1 -- 172.21.15.19:0/2129862085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563692889400 msgr2=0x56369287e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:15.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.528+0000 7f587dd9b700 1 --2- 172.21.15.19:0/2129862085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563692889400 0x56369287e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.528+0000 7f587dd9b700 1 -- 172.21.15.19:0/2129862085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56369270c4a0 msgr2=0x5636928938b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:15.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.528+0000 7f587dd9b700 1 --2- 172.21.15.19:0/2129862085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56369270c4a0 0x5636928938b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.528+0000 7f587dd9b700 1 -- 172.21.15.19:0/2129862085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563692895890 con 0x563692786d00 2024-05-09T02:14:15.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.528+0000 7f587e59c700 1 --2- 172.21.15.19:0/2129862085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56369270c4a0 0x5636928938b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:14:15.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.528+0000 7f587dd9b700 1 --2- 172.21.15.19:0/2129862085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 0x56369287dff0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f587400bc10 tx=0x7f58740b1970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a76b3e6c49bc59 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:15.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.528+0000 7f5867fff700 1 -- 172.21.15.19:0/2129862085 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58740c2070 con 0x563692786d00 2024-05-09T02:14:15.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.528+0000 7f5867fff700 1 -- 172.21.15.19:0/2129862085 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58740b1e00 con 0x563692786d00 2024-05-09T02:14:15.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.528+0000 7f5867fff700 1 -- 172.21.15.19:0/2129862085 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58740a8fa0 con 0x563692786d00 2024-05-09T02:14:15.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.529+0000 7f589380ab80 1 -- 172.21.15.19:0/2129862085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 msgr2=0x56369287dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:15.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.529+0000 7f589380ab80 1 --2- 172.21.15.19:0/2129862085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 0x56369287dff0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f587400bc10 tx=0x7f58740b1970 comp rx=0 tx=0).stop 2024-05-09T02:14:15.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.529+0000 7f589380ab80 1 -- 172.21.15.19:0/2129862085 shutdown_connections 2024-05-09T02:14:15.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.529+0000 7f589380ab80 1 --2- 172.21.15.19:0/2129862085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563692889400 0x56369287e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.529+0000 7f589380ab80 1 --2- 172.21.15.19:0/2129862085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56369270c4a0 0x5636928938b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.529+0000 7f589380ab80 1 --2- 172.21.15.19:0/2129862085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 0x56369287dff0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.529+0000 7f589380ab80 1 -- 172.21.15.19:0/2129862085 >> 172.21.15.19:0/2129862085 conn(0x56369270ff70 msgr2=0x5636928918e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:15.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.530+0000 7f589380ab80 1 -- 172.21.15.19:0/2129862085 shutdown_connections 2024-05-09T02:14:15.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.530+0000 7f589380ab80 1 -- 172.21.15.19:0/2129862085 wait complete. 2024-05-09T02:14:15.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.530+0000 7f589380ab80 1 -- start start 2024-05-09T02:14:15.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.530+0000 7f589380ab80 1 Processor -- start 2024-05-09T02:14:15.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.531+0000 7f589380ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56369270c4a0 0x56369289b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.531+0000 7f589380ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 0x56369289b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.531+0000 7f589380ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563692889400 0x5636928a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.531+0000 7f589380ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58801ae1e0 con 0x563692786d00 2024-05-09T02:14:15.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.531+0000 7f589380ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58801adb10 con 0x563692889400 2024-05-09T02:14:15.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.531+0000 7f589380ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58801aad10 con 0x56369270c4a0 2024-05-09T02:14:15.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.531+0000 7f589380ab80 10 client.0 _ll_register_callbacks cb 0x56369261d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:15.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.531+0000 7f587ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563692889400 0x5636928a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.531+0000 7f587ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563692889400 0x5636928a9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56286/0 (socket says 172.21.15.19:56286) 2024-05-09T02:14:15.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.531+0000 7f587ed9d700 1 -- 172.21.15.19:0/2649851226 learned_addr learned my addr 172.21.15.19:0/2649851226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:15.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.531+0000 7f587e59c700 1 --2- 172.21.15.19:0/2649851226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56369270c4a0 0x56369289b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.532+0000 7f587ed9d700 1 -- 172.21.15.19:0/2649851226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56369270c4a0 msgr2=0x56369289b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:15.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.532+0000 7f587ed9d700 1 --2- 172.21.15.19:0/2649851226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56369270c4a0 0x56369289b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.532+0000 7f587ed9d700 1 -- 172.21.15.19:0/2649851226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 msgr2=0x56369289b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:15.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.532+0000 7f587ed9d700 1 --2- 172.21.15.19:0/2649851226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 0x56369289b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.532+0000 7f587ed9d700 1 -- 172.21.15.19:0/2649851226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636928aa150 con 0x563692889400 2024-05-09T02:14:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.532+0000 7f587ed9d700 1 --2- 172.21.15.19:0/2649851226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563692889400 0x5636928a9c10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5870015000 tx=0x7f587000b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae5d6b77b501f716 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.532+0000 7f5865ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:14:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.533+0000 7f5865ffb700 1 -- 172.21.15.19:0/2649851226 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870005450 con 0x563692889400 2024-05-09T02:14:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.533+0000 7f5865ffb700 1 -- 172.21.15.19:0/2649851226 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f58700055f0 con 0x563692889400 2024-05-09T02:14:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.533+0000 7f589380ab80 1 -- 172.21.15.19:0/2649851226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58740b1e00 con 0x563692889400 2024-05-09T02:14:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.533+0000 7f5865ffb700 1 -- 172.21.15.19:0/2649851226 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f58700057f0 con 0x563692889400 2024-05-09T02:14:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.533+0000 7f589380ab80 10 client.25358 fetch_fsmap learned FSMap version 33 2024-05-09T02:14:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.533+0000 7f589380ab80 1 -- 172.21.15.19:0/2649851226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56369290ce70 con 0x563692889400 2024-05-09T02:14:15.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.534+0000 7f5865ffb700 1 -- 172.21.15.19:0/2649851226 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5870005990 con 0x563692889400 2024-05-09T02:14:15.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.534+0000 7f5865ffb700 1 -- 172.21.15.19:0/2649851226 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f5870011070 con 0x563692889400 2024-05-09T02:14:15.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.534+0000 7f589380ab80 10 client.25358 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T02:14:15.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.534+0000 7f589380ab80 10 client.25358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:14:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.534+0000 7f589380ab80 20 client.25358 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.534+0000 7f589380ab80 1 --2- 172.21.15.19:0/2649851226 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56369290d610 0x56369290fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.534+0000 7f589380ab80 4 client.25358 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:14:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.534+0000 7f589380ab80 1 -- 172.21.15.19:0/2649851226 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f58740b1e00 con 0x56369290d610 2024-05-09T02:14:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.535+0000 7f587e59c700 1 --2- 172.21.15.19:0/2649851226 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56369290d610 0x56369290fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.535+0000 7f587e59c700 1 --2- 172.21.15.19:0/2649851226 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56369290d610 0x56369290fa30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cbe7e15e996658e server_cookie=5aee9ff764d747ea in_seq=0 out_seq=0 2024-05-09T02:14:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.535+0000 7f5865ffb700 10 client.25358 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.535+0000 7f5865ffb700 1 -- 172.21.15.19:0/2649851226 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f58740b1e00 con 0x56369290d610 2024-05-09T02:14:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.535+0000 7f5865ffb700 10 client.25358 handle_command_reply: tid=0 2024-05-09T02:14:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.535+0000 7f589380ab80 1 client.25358 shutdown 2024-05-09T02:14:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.536+0000 7f589380ab80 1 -- 172.21.15.19:0/2649851226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563692889400 msgr2=0x5636928a9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.536+0000 7f589380ab80 1 --2- 172.21.15.19:0/2649851226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563692889400 0x5636928a9c10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5870015000 tx=0x7f587000b3b0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.536+0000 7f589380ab80 1 -- 172.21.15.19:0/2649851226 shutdown_connections 2024-05-09T02:14:15.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.536+0000 7f589380ab80 1 --2- 172.21.15.19:0/2649851226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563692889400 0x5636928a9c10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.536+0000 7f589380ab80 1 --2- 172.21.15.19:0/2649851226 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56369290d610 0x56369290fa30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.536+0000 7f589380ab80 1 --2- 172.21.15.19:0/2649851226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56369270c4a0 0x56369289b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.536+0000 7f589380ab80 1 --2- 172.21.15.19:0/2649851226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563692786d00 0x56369289b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.536+0000 7f589380ab80 1 -- 172.21.15.19:0/2649851226 >> 172.21.15.19:0/2649851226 conn(0x56369270ff70 msgr2=0x56369287dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:15.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.536+0000 7f5865ffb700 0 client.25358 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:15.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.536+0000 7f589380ab80 1 -- 172.21.15.19:0/2649851226 shutdown_connections 2024-05-09T02:14:15.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.536+0000 7f589380ab80 1 -- 172.21.15.19:0/2649851226 wait complete. 2024-05-09T02:14:15.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.536+0000 7f589380ab80 20 client.25358 trim_cache size 0 max 16384 2024-05-09T02:14:15.541 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:14:15.541 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T02:14:15.542 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T02:14:15.542 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:14:15.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.537+0000 7f5884df4700 1 -- 172.21.15.19:0/344729898 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f58680b2a70 msgr2=0x7f58680b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:15.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.537+0000 7f5884df4700 1 --2- 172.21.15.19:0/344729898 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f58680b2a70 0x7f58680b4f80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5870009900 tx=0x7f5870009410 comp rx=0 tx=0).stop 2024-05-09T02:14:15.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.537+0000 7f5884df4700 1 -- 172.21.15.19:0/344729898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5880107df0 msgr2=0x7f588019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:15.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.537+0000 7f5884df4700 1 --2- 172.21.15.19:0/344729898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5880107df0 0x7f588019b3b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f587400b500 tx=0x7f587400b810 comp rx=0 tx=0).stop 2024-05-09T02:14:15.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.537+0000 7f5884df4700 1 -- 172.21.15.19:0/344729898 shutdown_connections 2024-05-09T02:14:15.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.537+0000 7f5884df4700 1 --2- 172.21.15.19:0/344729898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5880108740 0x7f588019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.538+0000 7f5884df4700 1 --2- 172.21.15.19:0/344729898 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f58680b2a70 0x7f58680b4f80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.538+0000 7f5884df4700 1 --2- 172.21.15.19:0/344729898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58801034c0 0x7f588019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.538+0000 7f5884df4700 1 --2- 172.21.15.19:0/344729898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5880107df0 0x7f588019b3b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:15.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.538+0000 7f5884df4700 1 -- 172.21.15.19:0/344729898 >> 172.21.15.19:0/344729898 conn(0x7f58800fedc0 msgr2=0x7f5880111570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:15.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.538+0000 7f5884df4700 1 -- 172.21.15.19:0/344729898 shutdown_connections 2024-05-09T02:14:15.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:15.538+0000 7f5884df4700 1 -- 172.21.15.19:0/344729898 wait complete. 2024-05-09T02:14:16.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:16 smithi019 ceph-mon[30151]: pgmap v92: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 1.65k op/s 2024-05-09T02:14:16.509 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T02:14:16.510 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:14:16.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:16 smithi196 ceph-mon[34718]: pgmap v92: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 1.65k op/s 2024-05-09T02:14:16.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:16 smithi059 ceph-mon[34781]: pgmap v92: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 1.65k op/s 2024-05-09T02:14:16.889 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:14:17.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:17 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:14:17.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:17 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:14:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 155.0 s, 2527.6 tps, lat 39.480 ms stddev 44.535 2024-05-09T02:14:17.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:17 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:14:18.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:18 smithi196 ceph-mon[34718]: pgmap v93: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 MiB/s wr, 1.35k op/s 2024-05-09T02:14:18.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:18 smithi019 ceph-mon[30151]: pgmap v93: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 MiB/s wr, 1.35k op/s 2024-05-09T02:14:18.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:18 smithi059 ceph-mon[34781]: pgmap v93: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 MiB/s wr, 1.35k op/s 2024-05-09T02:14:20.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:20 smithi019 ceph-mon[30151]: pgmap v94: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 1.71k op/s 2024-05-09T02:14:20.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.526+0000 7efc6c657700 1 -- 172.21.15.19:0/2296489865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc64108f10 msgr2=0x7efc64109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:20.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.526+0000 7efc6c657700 1 --2- 172.21.15.19:0/2296489865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc64108f10 0x7efc64109320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efc540098f0 tx=0x7efc54009c00 comp rx=0 tx=0).stop 2024-05-09T02:14:20.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.527+0000 7efc6c657700 1 -- 172.21.15.19:0/2296489865 shutdown_connections 2024-05-09T02:14:20.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.527+0000 7efc6c657700 1 --2- 172.21.15.19:0/2296489865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc64109860 0x7efc641111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:20.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.527+0000 7efc6c657700 1 --2- 172.21.15.19:0/2296489865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc641045e0 0x7efc641049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:20.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.527+0000 7efc6c657700 1 --2- 172.21.15.19:0/2296489865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc64108f10 0x7efc64109320 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:20.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.527+0000 7efc6c657700 1 -- 172.21.15.19:0/2296489865 >> 172.21.15.19:0/2296489865 conn(0x7efc640ffe70 msgr2=0x7efc641022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:20.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.527+0000 7efc6c657700 1 -- 172.21.15.19:0/2296489865 shutdown_connections 2024-05-09T02:14:20.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.530+0000 7efc6c657700 1 -- 172.21.15.19:0/2296489865 wait complete. 2024-05-09T02:14:20.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.532+0000 7efc6c657700 1 Processor -- start 2024-05-09T02:14:20.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.532+0000 7efc6c657700 1 -- start start 2024-05-09T02:14:20.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.533+0000 7efc6c657700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc641045e0 0x7efc6419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:20.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.533+0000 7efc6c657700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc64108f10 0x7efc6419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:20.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.533+0000 7efc6c657700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc64109860 0x7efc6419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:20.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.533+0000 7efc6c657700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc64114510 con 0x7efc64108f10 2024-05-09T02:14:20.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.533+0000 7efc6c657700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efc64114390 con 0x7efc64109860 2024-05-09T02:14:20.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.533+0000 7efc6c657700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efc64114690 con 0x7efc641045e0 2024-05-09T02:14:20.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.535+0000 7efc6abf4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc64109860 0x7efc6419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:20.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.535+0000 7efc6abf4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc64109860 0x7efc6419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:44896/0 (socket says 172.21.15.19:44896) 2024-05-09T02:14:20.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.535+0000 7efc6abf4700 1 -- 172.21.15.19:0/3827844190 learned_addr learned my addr 172.21.15.19:0/3827844190 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:20.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.535+0000 7efc69bf2700 1 --2- 172.21.15.19:0/3827844190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc64108f10 0x7efc6419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:20.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.535+0000 7efc6a3f3700 1 --2- 172.21.15.19:0/3827844190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc641045e0 0x7efc6419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:20.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.536+0000 7efc69bf2700 1 -- 172.21.15.19:0/3827844190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc641045e0 msgr2=0x7efc6419aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:20.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.536+0000 7efc69bf2700 1 --2- 172.21.15.19:0/3827844190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc641045e0 0x7efc6419aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:20.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.536+0000 7efc69bf2700 1 -- 172.21.15.19:0/3827844190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc64109860 msgr2=0x7efc6419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:20.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.536+0000 7efc69bf2700 1 --2- 172.21.15.19:0/3827844190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc64109860 0x7efc6419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:20.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.536+0000 7efc69bf2700 1 -- 172.21.15.19:0/3827844190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6419fd90 con 0x7efc64108f10 2024-05-09T02:14:20.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.536+0000 7efc69bf2700 1 --2- 172.21.15.19:0/3827844190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc64108f10 0x7efc6419b460 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efc54000c00 tx=0x7efc5400fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:20.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:20 smithi196 ceph-mon[34718]: pgmap v94: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 1.71k op/s 2024-05-09T02:14:20.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.542+0000 7efc5b7fe700 1 -- 172.21.15.19:0/3827844190 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc540050c0 con 0x7efc64108f10 2024-05-09T02:14:20.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.543+0000 7efc5b7fe700 1 -- 172.21.15.19:0/3827844190 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc5400b560 con 0x7efc64108f10 2024-05-09T02:14:20.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.543+0000 7efc5b7fe700 1 -- 172.21.15.19:0/3827844190 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54003980 con 0x7efc64108f10 2024-05-09T02:14:20.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.544+0000 7efc6c657700 1 -- 172.21.15.19:0/3827844190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc641a0020 con 0x7efc64108f10 2024-05-09T02:14:20.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.544+0000 7efc6c657700 1 -- 172.21.15.19:0/3827844190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc641a03f0 con 0x7efc64108f10 2024-05-09T02:14:20.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.546+0000 7efc597fa700 1 -- 172.21.15.19:0/3827844190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc640512c0 con 0x7efc64108f10 2024-05-09T02:14:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.548+0000 7efc5b7fe700 1 -- 172.21.15.19:0/3827844190 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efc54003b80 con 0x7efc64108f10 2024-05-09T02:14:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.550+0000 7efc5b7fe700 1 --2- 172.21.15.19:0/3827844190 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc500b2b80 0x7efc500b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:20.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.551+0000 7efc5b7fe700 1 -- 172.21.15.19:0/3827844190 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7efc540e3460 con 0x7efc64108f10 2024-05-09T02:14:20.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.554+0000 7efc6a3f3700 1 --2- 172.21.15.19:0/3827844190 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc500b2b80 0x7efc500b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:20.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.555+0000 7efc6a3f3700 1 --2- 172.21.15.19:0/3827844190 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc500b2b80 0x7efc500b5090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efc60009750 tx=0x7efc60009310 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:20.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.560+0000 7efc5b7fe700 1 -- 172.21.15.19:0/3827844190 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc54013030 con 0x7efc64108f10 2024-05-09T02:14:20.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:20 smithi059 ceph-mon[34781]: pgmap v94: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 1.71k op/s 2024-05-09T02:14:20.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.994+0000 7efc597fa700 1 -- 172.21.15.19:0/3827844190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc640643d0 con 0x7efc64108f10 2024-05-09T02:14:20.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.994+0000 7efc5b7fe700 1 -- 172.21.15.19:0/3827844190 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11922 (secure 0 0 0) 0x7efc540aaed0 con 0x7efc64108f10 2024-05-09T02:14:20.997 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:14:20.997 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:14:15.724494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24457,24455,24473,24463,24484]},"id":1}]} 2024-05-09T02:14:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.998+0000 7efc597fa700 1 -- 172.21.15.19:0/3827844190 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc500b2b80 msgr2=0x7efc500b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.998+0000 7efc597fa700 1 --2- 172.21.15.19:0/3827844190 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc500b2b80 0x7efc500b5090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efc60009750 tx=0x7efc60009310 comp rx=0 tx=0).stop 2024-05-09T02:14:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.998+0000 7efc597fa700 1 -- 172.21.15.19:0/3827844190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc64108f10 msgr2=0x7efc6419b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.998+0000 7efc597fa700 1 --2- 172.21.15.19:0/3827844190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc64108f10 0x7efc6419b460 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efc54000c00 tx=0x7efc5400fa20 comp rx=0 tx=0).stop 2024-05-09T02:14:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.998+0000 7efc597fa700 1 -- 172.21.15.19:0/3827844190 shutdown_connections 2024-05-09T02:14:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.998+0000 7efc597fa700 1 --2- 172.21.15.19:0/3827844190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc64109860 0x7efc6419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.998+0000 7efc597fa700 1 --2- 172.21.15.19:0/3827844190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc641045e0 0x7efc6419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.998+0000 7efc597fa700 1 --2- 172.21.15.19:0/3827844190 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc500b2b80 0x7efc500b5090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.998+0000 7efc597fa700 1 --2- 172.21.15.19:0/3827844190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc64108f10 0x7efc6419b460 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.998+0000 7efc597fa700 1 -- 172.21.15.19:0/3827844190 >> 172.21.15.19:0/3827844190 conn(0x7efc640ffe70 msgr2=0x7efc641086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:21.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.998+0000 7efc597fa700 1 -- 172.21.15.19:0/3827844190 shutdown_connections 2024-05-09T02:14:21.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:20.998+0000 7efc597fa700 1 -- 172.21.15.19:0/3827844190 wait complete. 2024-05-09T02:14:21.002 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 34 2024-05-09T02:14:21.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:21 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3827844190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:14:21.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:21 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3827844190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:14:21.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:21 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3827844190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:14:21.828 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T02:14:22.184 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:14:22.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:22 smithi019 ceph-mon[30151]: pgmap v95: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 37 MiB/s wr, 1.46k op/s 2024-05-09T02:14:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 160.0 s, 2773.6 tps, lat 35.994 ms stddev 38.773 2024-05-09T02:14:22.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:22 smithi059 ceph-mon[34781]: pgmap v95: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 37 MiB/s wr, 1.46k op/s 2024-05-09T02:14:22.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:22 smithi196 ceph-mon[34718]: pgmap v95: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 37 MiB/s wr, 1.46k op/s 2024-05-09T02:14:24.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:24 smithi019 ceph-mon[30151]: pgmap v96: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 37 MiB/s wr, 1.47k op/s 2024-05-09T02:14:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:24 smithi059 ceph-mon[34781]: pgmap v96: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 37 MiB/s wr, 1.47k op/s 2024-05-09T02:14:24.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:24 smithi196 ceph-mon[34718]: pgmap v96: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 37 MiB/s wr, 1.47k op/s 2024-05-09T02:14:26.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.488+0000 7f0f38eb5700 1 -- 172.21.15.19:0/2346148599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f34104650 msgr2=0x7f0f34104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:26.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.488+0000 7f0f38eb5700 1 --2- 172.21.15.19:0/2346148599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f34104650 0x7f0f34104a60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0f240099c0 tx=0x7f0f24009cd0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.488+0000 7f0f38eb5700 1 -- 172.21.15.19:0/2346148599 shutdown_connections 2024-05-09T02:14:26.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.488+0000 7f0f38eb5700 1 --2- 172.21.15.19:0/2346148599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f34109880 0x7f0f341111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.488+0000 7f0f38eb5700 1 --2- 172.21.15.19:0/2346148599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f34104650 0x7f0f34104a60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.488+0000 7f0f38eb5700 1 --2- 172.21.15.19:0/2346148599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f34108f30 0x7f0f34109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.488+0000 7f0f38eb5700 1 -- 172.21.15.19:0/2346148599 >> 172.21.15.19:0/2346148599 conn(0x7f0f34079f00 msgr2=0x7f0f3407a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:26.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.488+0000 7f0f38eb5700 1 -- 172.21.15.19:0/2346148599 shutdown_connections 2024-05-09T02:14:26.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.488+0000 7f0f38eb5700 1 -- 172.21.15.19:0/2346148599 wait complete. 2024-05-09T02:14:26.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.489+0000 7f0f38eb5700 1 Processor -- start 2024-05-09T02:14:26.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.489+0000 7f0f38eb5700 1 -- start start 2024-05-09T02:14:26.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f38eb5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f34108f30 0x7f0f3419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f38eb5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f34109880 0x7f0f3419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f38eb5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f3419f6f0 0x7f0f3419fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f38eb5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341144b0 con 0x7f0f34109880 2024-05-09T02:14:26.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f38eb5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f34114330 con 0x7f0f34108f30 2024-05-09T02:14:26.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f38eb5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f34114630 con 0x7f0f3419f6f0 2024-05-09T02:14:26.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f31d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f34109880 0x7f0f3419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f3259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f34108f30 0x7f0f3419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f3259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f34108f30 0x7f0f3419af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:44918/0 (socket says 172.21.15.19:44918) 2024-05-09T02:14:26.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f3259c700 1 -- 172.21.15.19:0/3191904377 learned_addr learned my addr 172.21.15.19:0/3191904377 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:26.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f32d9d700 1 --2- 172.21.15.19:0/3191904377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f3419f6f0 0x7f0f3419fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f32d9d700 1 -- 172.21.15.19:0/3191904377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f34108f30 msgr2=0x7f0f3419af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:26.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f32d9d700 1 --2- 172.21.15.19:0/3191904377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f34108f30 0x7f0f3419af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f32d9d700 1 -- 172.21.15.19:0/3191904377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f34109880 msgr2=0x7f0f3419b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:26.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f32d9d700 1 --2- 172.21.15.19:0/3191904377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f34109880 0x7f0f3419b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.490+0000 7f0f32d9d700 1 -- 172.21.15.19:0/3191904377 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f341a0240 con 0x7f0f3419f6f0 2024-05-09T02:14:26.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.491+0000 7f0f32d9d700 1 --2- 172.21.15.19:0/3191904377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f3419f6f0 0x7f0f3419fb00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00efc0 tx=0x7f0f2c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:26.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.491+0000 7f0f237fe700 1 -- 172.21.15.19:0/3191904377 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c004bb0 con 0x7f0f3419f6f0 2024-05-09T02:14:26.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.491+0000 7f0f237fe700 1 -- 172.21.15.19:0/3191904377 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f2c004d50 con 0x7f0f3419f6f0 2024-05-09T02:14:26.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.491+0000 7f0f237fe700 1 -- 172.21.15.19:0/3191904377 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c004310 con 0x7f0f3419f6f0 2024-05-09T02:14:26.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.492+0000 7f0f38eb5700 1 -- 172.21.15.19:0/3191904377 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f341a0530 con 0x7f0f3419f6f0 2024-05-09T02:14:26.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.492+0000 7f0f38eb5700 1 -- 172.21.15.19:0/3191904377 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f34075d10 con 0x7f0f3419f6f0 2024-05-09T02:14:26.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.497+0000 7f0f237fe700 1 -- 172.21.15.19:0/3191904377 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0f2c00c9f0 con 0x7f0f3419f6f0 2024-05-09T02:14:26.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.497+0000 7f0f478cbb80 1 Processor -- start 2024-05-09T02:14:26.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.498+0000 7f0f478cbb80 1 -- start start 2024-05-09T02:14:26.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.498+0000 7f0f237fe700 1 --2- 172.21.15.19:0/3191904377 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0f1c0b2a70 0x7f0f1c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.498+0000 7f0f237fe700 1 -- 172.21.15.19:0/3191904377 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f0f2c0e5e40 con 0x7f0f3419f6f0 2024-05-09T02:14:26.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.498+0000 7f0f478cbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be0444e160 0x55be044e74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.498+0000 7f0f478cbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 0x55be044e7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.498+0000 7f0f31d9b700 1 --2- 172.21.15.19:0/3191904377 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0f1c0b2a70 0x7f0f1c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.498+0000 7f0f478cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be044e16a0 0x55be044e1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.498+0000 7f0f478cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55be04403ff0 con 0x55be044e16a0 2024-05-09T02:14:26.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.498+0000 7f0f478cbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55be043a27f0 con 0x55be0444e160 2024-05-09T02:14:26.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.498+0000 7f0f478cbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55be043fffb0 con 0x55be043644a0 2024-05-09T02:14:26.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.498+0000 7f0f3259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be0444e160 0x55be044e74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.498+0000 7f0f3259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be0444e160 0x55be044e74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:44932/0 (socket says 172.21.15.19:44932) 2024-05-09T02:14:26.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.498+0000 7f0f3259c700 1 -- 172.21.15.19:0/2114282101 learned_addr learned my addr 172.21.15.19:0/2114282101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:26.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.498+0000 7f0f31d9b700 1 --2- 172.21.15.19:0/2114282101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 0x55be044e7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.499+0000 7f0f32d9d700 1 --2- 172.21.15.19:0/2114282101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be044e16a0 0x55be044e1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.500+0000 7f0f31d9b700 1 --2- 172.21.15.19:0/3191904377 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0f1c0b2a70 0x7f0f1c0b4f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0f3407a4a0 tx=0x7f0f2800b890 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:26.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.500+0000 7f0f3259c700 1 -- 172.21.15.19:0/2114282101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 msgr2=0x55be044e7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:26.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.500+0000 7f0f3259c700 1 --2- 172.21.15.19:0/2114282101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 0x55be044e7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.500+0000 7f0f3259c700 1 -- 172.21.15.19:0/2114282101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be044e16a0 msgr2=0x55be044e1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:26.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.500+0000 7f0f3259c700 1 --2- 172.21.15.19:0/2114282101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be044e16a0 0x55be044e1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.500+0000 7f0f3259c700 1 -- 172.21.15.19:0/2114282101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be0439a370 con 0x55be0444e160 2024-05-09T02:14:26.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.500+0000 7f0f3259c700 1 --2- 172.21.15.19:0/2114282101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be0444e160 0x55be044e74f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0f24015000 tx=0x7f0f24005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6196f8f3f70a3d3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:26.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.500+0000 7f0f20ff9700 1 -- 172.21.15.19:0/2114282101 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f240052b0 con 0x55be0444e160 2024-05-09T02:14:26.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.501+0000 7f0f20ff9700 1 -- 172.21.15.19:0/2114282101 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f24003c70 con 0x55be0444e160 2024-05-09T02:14:26.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.501+0000 7f0f20ff9700 1 -- 172.21.15.19:0/2114282101 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2400bad0 con 0x55be0444e160 2024-05-09T02:14:26.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.502+0000 7f0f478cbb80 1 -- 172.21.15.19:0/2114282101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be0444e160 msgr2=0x55be044e74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:26.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.502+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/2114282101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be0444e160 0x55be044e74f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0f24015000 tx=0x7f0f24005040 comp rx=0 tx=0).stop 2024-05-09T02:14:26.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.502+0000 7f0f478cbb80 1 -- 172.21.15.19:0/2114282101 shutdown_connections 2024-05-09T02:14:26.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.502+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/2114282101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be044e16a0 0x55be044e1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.502+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/2114282101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be0444e160 0x55be044e74f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.502+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/2114282101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 0x55be044e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.502+0000 7f0f478cbb80 1 -- 172.21.15.19:0/2114282101 >> 172.21.15.19:0/2114282101 conn(0x55be043c46a0 msgr2=0x55be044dc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.502+0000 7f0f478cbb80 1 -- 172.21.15.19:0/2114282101 shutdown_connections 2024-05-09T02:14:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.503+0000 7f0f478cbb80 1 -- 172.21.15.19:0/2114282101 wait complete. 2024-05-09T02:14:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.503+0000 7f0f478cbb80 1 -- start start 2024-05-09T02:14:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.504+0000 7f0f478cbb80 1 Processor -- start 2024-05-09T02:14:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.504+0000 7f0f478cbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 0x55be044eb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.504+0000 7f0f478cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0444e160 0x55be044eb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.504+0000 7f0f478cbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be044e16a0 0x55be044ebc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.504+0000 7f0f478cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55be0427fba0 con 0x55be0444e160 2024-05-09T02:14:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.504+0000 7f0f478cbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55be043be150 con 0x55be044e16a0 2024-05-09T02:14:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.504+0000 7f0f478cbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55be044e24e0 con 0x55be043644a0 2024-05-09T02:14:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.504+0000 7f0f478cbb80 10 client.0 _ll_register_callbacks cb 0x55be042756c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:26.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.505+0000 7f0f3259c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 0x55be044eb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.505+0000 7f0f32d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be044e16a0 0x55be044ebc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.505+0000 7f0f3259c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 0x55be044eb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:60086/0 (socket says 172.21.15.19:60086) 2024-05-09T02:14:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.505+0000 7f0f3259c700 1 -- 172.21.15.19:0/3395694704 learned_addr learned my addr 172.21.15.19:0/3395694704 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.505+0000 7f0f31d9b700 1 --2- 172.21.15.19:0/3395694704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0444e160 0x55be044eb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.505+0000 7f0f3259c700 1 -- 172.21.15.19:0/3395694704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be044e16a0 msgr2=0x55be044ebc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.505+0000 7f0f3259c700 1 --2- 172.21.15.19:0/3395694704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be044e16a0 0x55be044ebc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.505+0000 7f0f3259c700 1 -- 172.21.15.19:0/3395694704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0444e160 msgr2=0x55be044eb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.505+0000 7f0f3259c700 1 --2- 172.21.15.19:0/3395694704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0444e160 0x55be044eb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.505+0000 7f0f3259c700 1 -- 172.21.15.19:0/3395694704 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be0427f490 con 0x55be043644a0 2024-05-09T02:14:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.505+0000 7f0f31d9b700 1 --2- 172.21.15.19:0/3395694704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0444e160 0x55be044eb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:14:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.506+0000 7f0f3259c700 1 --2- 172.21.15.19:0/3395694704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 0x55be044eb1a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0f24024040 tx=0x7f0f24009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1163c807166b604 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.506+0000 7f0f19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:14:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.506+0000 7f0f19ffb700 1 -- 172.21.15.19:0/3395694704 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2400b790 con 0x55be043644a0 2024-05-09T02:14:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.506+0000 7f0f19ffb700 1 -- 172.21.15.19:0/3395694704 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f0f2400b930 con 0x55be043644a0 2024-05-09T02:14:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.506+0000 7f0f478cbb80 1 -- 172.21.15.19:0/3395694704 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f24003c70 con 0x55be043644a0 2024-05-09T02:14:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.507+0000 7f0f19ffb700 1 -- 172.21.15.19:0/3395694704 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0f24039050 con 0x55be043644a0 2024-05-09T02:14:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.507+0000 7f0f478cbb80 10 client.25441 fetch_fsmap learned FSMap version 34 2024-05-09T02:14:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.507+0000 7f0f478cbb80 1 -- 172.21.15.19:0/3395694704 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55be044f8b00 con 0x55be043644a0 2024-05-09T02:14:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.507+0000 7f0f19ffb700 1 -- 172.21.15.19:0/3395694704 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f24030070 con 0x55be043644a0 2024-05-09T02:14:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.508+0000 7f0f19ffb700 1 -- 172.21.15.19:0/3395694704 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f0f2402bbb0 con 0x55be043644a0 2024-05-09T02:14:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.509+0000 7f0f478cbb80 10 client.25441 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T02:14:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.509+0000 7f0f478cbb80 10 client.25441 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:14:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.509+0000 7f0f478cbb80 20 client.25441 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.509+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/3395694704 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be044fafd0 0x55be044fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.509+0000 7f0f478cbb80 4 client.25441 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:14:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.509+0000 7f0f478cbb80 1 -- 172.21.15.19:0/3395694704 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0f24003c70 con 0x55be044fafd0 2024-05-09T02:14:26.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.510+0000 7f0f31d9b700 1 --2- 172.21.15.19:0/3395694704 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be044fafd0 0x55be044fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.510+0000 7f0f31d9b700 1 --2- 172.21.15.19:0/3395694704 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be044fafd0 0x55be044fd3f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ce8e6af915ebbb57 server_cookie=3aced47356814382 in_seq=0 out_seq=0 2024-05-09T02:14:26.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.510+0000 7f0f19ffb700 10 client.25441 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:14:26.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.512+0000 7f0f19ffb700 1 -- 172.21.15.19:0/3395694704 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f0f24003c70 con 0x55be044fafd0 2024-05-09T02:14:26.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.512+0000 7f0f19ffb700 10 client.25441 handle_command_reply: tid=0 2024-05-09T02:14:26.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.512+0000 7f0f478cbb80 1 client.25441 shutdown 2024-05-09T02:14:26.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.513+0000 7f0f478cbb80 1 -- 172.21.15.19:0/3395694704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 msgr2=0x55be044eb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:26.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.513+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/3395694704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 0x55be044eb1a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0f24024040 tx=0x7f0f24009310 comp rx=0 tx=0).stop 2024-05-09T02:14:26.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.513+0000 7f0f478cbb80 1 -- 172.21.15.19:0/3395694704 shutdown_connections 2024-05-09T02:14:26.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.513+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/3395694704 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be044fafd0 0x55be044fd3f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.513+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/3395694704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be044e16a0 0x55be044ebc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.513+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/3395694704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 0x55be044eb1a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.513+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/3395694704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0444e160 0x55be044eb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.513+0000 7f0f478cbb80 1 -- 172.21.15.19:0/3395694704 >> 172.21.15.19:0/3395694704 conn(0x55be043c46a0 msgr2=0x55be044dc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:26.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.513+0000 7f0f19ffb700 0 client.25441 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:14:26.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.513+0000 7f0f478cbb80 1 -- 172.21.15.19:0/3395694704 shutdown_connections 2024-05-09T02:14:26.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.513+0000 7f0f478cbb80 1 -- 172.21.15.19:0/3395694704 wait complete. 2024-05-09T02:14:26.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.513+0000 7f0f478cbb80 20 client.25441 trim_cache size 0 max 16384 2024-05-09T02:14:26.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.535+0000 7f0f478cbb80 1 Processor -- start 2024-05-09T02:14:26.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.535+0000 7f0f478cbb80 1 -- start start 2024-05-09T02:14:26.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.536+0000 7f0f478cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be043644a0 0x55be044f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.536+0000 7f0f478cbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be0444e160 0x55be044eba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.536+0000 7f0f478cbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be044e16a0 0x55be044e5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.536+0000 7f0f478cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341ad200 con 0x55be043644a0 2024-05-09T02:14:26.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.536+0000 7f0f478cbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341aedd0 con 0x55be0444e160 2024-05-09T02:14:26.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.536+0000 7f0f478cbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341ad8d0 con 0x55be044e16a0 2024-05-09T02:14:26.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.536+0000 7f0f31d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be0444e160 0x55be044eba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.536+0000 7f0f3259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be043644a0 0x55be044f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.536+0000 7f0f3259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be043644a0 0x55be044f72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42588/0 (socket says 172.21.15.19:42588) 2024-05-09T02:14:26.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.536+0000 7f0f3259c700 1 -- 172.21.15.19:0/673050315 learned_addr learned my addr 172.21.15.19:0/673050315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:26.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.536+0000 7f0f32d9d700 1 --2- 172.21.15.19:0/673050315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be044e16a0 0x55be044e5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.537+0000 7f0f3259c700 1 -- 172.21.15.19:0/673050315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be044e16a0 msgr2=0x55be044e5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:26.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.537+0000 7f0f3259c700 1 --2- 172.21.15.19:0/673050315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be044e16a0 0x55be044e5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.537+0000 7f0f3259c700 1 -- 172.21.15.19:0/673050315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be0444e160 msgr2=0x55be044eba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:26.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.537+0000 7f0f3259c700 1 --2- 172.21.15.19:0/673050315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be0444e160 0x55be044eba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.537+0000 7f0f3259c700 1 -- 172.21.15.19:0/673050315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be044ddd00 con 0x55be043644a0 2024-05-09T02:14:26.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.537+0000 7f0f3259c700 1 --2- 172.21.15.19:0/673050315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be043644a0 0x55be044f72b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0f240057f0 tx=0x7f0f24005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b169f1f4ff7312fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:26.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.537+0000 7f0f1bfff700 1 -- 172.21.15.19:0/673050315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24003bd0 con 0x55be043644a0 2024-05-09T02:14:26.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.537+0000 7f0f1bfff700 1 -- 172.21.15.19:0/673050315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f24030040 con 0x55be043644a0 2024-05-09T02:14:26.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.538+0000 7f0f1bfff700 1 -- 172.21.15.19:0/673050315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2400bb10 con 0x55be043644a0 2024-05-09T02:14:26.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.538+0000 7f0f478cbb80 1 -- 172.21.15.19:0/673050315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be043644a0 msgr2=0x55be044f72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:26.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.538+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/673050315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be043644a0 0x55be044f72b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0f240057f0 tx=0x7f0f24005610 comp rx=0 tx=0).stop 2024-05-09T02:14:26.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.538+0000 7f0f478cbb80 1 -- 172.21.15.19:0/673050315 shutdown_connections 2024-05-09T02:14:26.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.538+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/673050315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be044e16a0 0x55be044e5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.538+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/673050315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be043644a0 0x55be044f72b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.538+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/673050315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be0444e160 0x55be044eba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.538+0000 7f0f478cbb80 1 -- 172.21.15.19:0/673050315 >> 172.21.15.19:0/673050315 conn(0x55be043c46a0 msgr2=0x55be044e4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.539+0000 7f0f478cbb80 1 -- 172.21.15.19:0/673050315 shutdown_connections 2024-05-09T02:14:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.539+0000 7f0f478cbb80 1 -- 172.21.15.19:0/673050315 wait complete. 2024-05-09T02:14:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.539+0000 7f0f478cbb80 1 -- start start 2024-05-09T02:14:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.539+0000 7f0f478cbb80 1 Processor -- start 2024-05-09T02:14:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.540+0000 7f0f478cbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 0x55be044fff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.540+0000 7f0f478cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0444e160 0x55be04500460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.540+0000 7f0f478cbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be044e16a0 0x55be044fd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.540+0000 7f0f478cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341ae670 con 0x55be0444e160 2024-05-09T02:14:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.540+0000 7f0f478cbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341adfa0 con 0x55be044e16a0 2024-05-09T02:14:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.540+0000 7f0f478cbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341ab1a0 con 0x55be043644a0 2024-05-09T02:14:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.540+0000 7f0f478cbb80 10 client.0 _ll_register_callbacks cb 0x55be042756c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.540+0000 7f0f31d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0444e160 0x55be04500460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.540+0000 7f0f31d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0444e160 0x55be04500460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42604/0 (socket says 172.21.15.19:42604) 2024-05-09T02:14:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.540+0000 7f0f32d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be044e16a0 0x55be044fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.540+0000 7f0f31d9b700 1 -- 172.21.15.19:0/2879812072 learned_addr learned my addr 172.21.15.19:0/2879812072 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.541+0000 7f0f3259c700 1 --2- 172.21.15.19:0/2879812072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 0x55be044fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.541+0000 7f0f32d9d700 1 -- 172.21.15.19:0/2879812072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 msgr2=0x55be044fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.541+0000 7f0f32d9d700 1 --2- 172.21.15.19:0/2879812072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 0x55be044fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.541+0000 7f0f32d9d700 1 -- 172.21.15.19:0/2879812072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0444e160 msgr2=0x55be04500460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.541+0000 7f0f32d9d700 1 --2- 172.21.15.19:0/2879812072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0444e160 0x55be04500460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.541+0000 7f0f32d9d700 1 -- 172.21.15.19:0/2879812072 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be044fda50 con 0x55be044e16a0 2024-05-09T02:14:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.541+0000 7f0f31d9b700 1 --2- 172.21.15.19:0/2879812072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0444e160 0x55be04500460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:14:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.541+0000 7f0f32d9d700 1 --2- 172.21.15.19:0/2879812072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be044e16a0 0x55be044fd410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0e3670 tx=0x7f0f2c0e3a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8068b03b30737150 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.541+0000 7f0f19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:14:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.541+0000 7f0f19ffb700 1 -- 172.21.15.19:0/2879812072 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c01f040 con 0x55be044e16a0 2024-05-09T02:14:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.541+0000 7f0f19ffb700 1 -- 172.21.15.19:0/2879812072 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f0f2c0b5e10 con 0x55be044e16a0 2024-05-09T02:14:26.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.541+0000 7f0f478cbb80 1 -- 172.21.15.19:0/2879812072 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f24030040 con 0x55be044e16a0 2024-05-09T02:14:26.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.542+0000 7f0f19ffb700 1 -- 172.21.15.19:0/2879812072 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0f2c0ad930 con 0x55be044e16a0 2024-05-09T02:14:26.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.542+0000 7f0f478cbb80 10 client.25382 fetch_fsmap learned FSMap version 34 2024-05-09T02:14:26.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.542+0000 7f0f478cbb80 1 -- 172.21.15.19:0/2879812072 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55be0456d770 con 0x55be044e16a0 2024-05-09T02:14:26.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.542+0000 7f0f19ffb700 1 -- 172.21.15.19:0/2879812072 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f24030040 con 0x55be044e16a0 2024-05-09T02:14:26.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.542+0000 7f0f19ffb700 1 -- 172.21.15.19:0/2879812072 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f0f2c0c7b20 con 0x55be044e16a0 2024-05-09T02:14:26.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.542+0000 7f0f478cbb80 10 client.25382 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T02:14:26.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.542+0000 7f0f478cbb80 10 client.25382 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:14:26.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.542+0000 7f0f478cbb80 20 client.25382 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:26.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.542+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/2879812072 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be0456de80 0x55be045702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:26.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.542+0000 7f0f478cbb80 4 client.25382 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:14:26.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.542+0000 7f0f478cbb80 1 -- 172.21.15.19:0/2879812072 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be044119d0 con 0x55be0456de80 2024-05-09T02:14:26.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.543+0000 7f0f3259c700 1 --2- 172.21.15.19:0/2879812072 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be0456de80 0x55be045702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:26.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.544+0000 7f0f3259c700 1 --2- 172.21.15.19:0/2879812072 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be0456de80 0x55be045702a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f06e72908490d807 server_cookie=8bc75f30ed59f216 in_seq=0 out_seq=0 2024-05-09T02:14:26.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.544+0000 7f0f19ffb700 10 client.25382 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:14:26.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.544+0000 7f0f19ffb700 1 -- 172.21.15.19:0/2879812072 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be044119d0 con 0x55be0456de80 2024-05-09T02:14:26.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.544+0000 7f0f19ffb700 10 client.25382 handle_command_reply: tid=0 2024-05-09T02:14:26.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.544+0000 7f0f478cbb80 1 client.25382 shutdown 2024-05-09T02:14:26.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.545+0000 7f0f478cbb80 1 -- 172.21.15.19:0/2879812072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be044e16a0 msgr2=0x55be044fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:26.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.545+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/2879812072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be044e16a0 0x55be044fd410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0e3670 tx=0x7f0f2c0e3a90 comp rx=0 tx=0).stop 2024-05-09T02:14:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.545+0000 7f0f478cbb80 1 -- 172.21.15.19:0/2879812072 shutdown_connections 2024-05-09T02:14:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.545+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/2879812072 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be0456de80 0x55be045702a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.545+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/2879812072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be044e16a0 0x55be044fd410 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.545+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/2879812072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be043644a0 0x55be044fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.545+0000 7f0f478cbb80 1 --2- 172.21.15.19:0/2879812072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0444e160 0x55be04500460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.545+0000 7f0f478cbb80 1 -- 172.21.15.19:0/2879812072 >> 172.21.15.19:0/2879812072 conn(0x55be043c46a0 msgr2=0x55be044eb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.545+0000 7f0f19ffb700 0 client.25382 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:14:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.545+0000 7f0f478cbb80 1 -- 172.21.15.19:0/2879812072 shutdown_connections 2024-05-09T02:14:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.545+0000 7f0f478cbb80 1 -- 172.21.15.19:0/2879812072 wait complete. 2024-05-09T02:14:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.545+0000 7f0f478cbb80 20 client.25382 trim_cache size 0 max 16384 2024-05-09T02:14:26.551 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:14:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.546+0000 7f0f38eb5700 1 -- 172.21.15.19:0/3191904377 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0f1c0b2a70 msgr2=0x7f0f1c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.546+0000 7f0f38eb5700 1 --2- 172.21.15.19:0/3191904377 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0f1c0b2a70 0x7f0f1c0b4f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0f3407a4a0 tx=0x7f0f2800b890 comp rx=0 tx=0).stop 2024-05-09T02:14:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.546+0000 7f0f38eb5700 1 -- 172.21.15.19:0/3191904377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f3419f6f0 msgr2=0x7f0f3419fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.546+0000 7f0f38eb5700 1 --2- 172.21.15.19:0/3191904377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f3419f6f0 0x7f0f3419fb00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00efc0 tx=0x7f0f2c00c3a0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.546+0000 7f0f38eb5700 1 -- 172.21.15.19:0/3191904377 shutdown_connections 2024-05-09T02:14:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.546+0000 7f0f38eb5700 1 --2- 172.21.15.19:0/3191904377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f3419f6f0 0x7f0f3419fb00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.546+0000 7f0f38eb5700 1 --2- 172.21.15.19:0/3191904377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f34108f30 0x7f0f3419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.546+0000 7f0f38eb5700 1 --2- 172.21.15.19:0/3191904377 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0f1c0b2a70 0x7f0f1c0b4f80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.546+0000 7f0f38eb5700 1 --2- 172.21.15.19:0/3191904377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f34109880 0x7f0f3419b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:26.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.546+0000 7f0f38eb5700 1 -- 172.21.15.19:0/3191904377 >> 172.21.15.19:0/3191904377 conn(0x7f0f34079f00 msgr2=0x7f0f34102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:26.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.546+0000 7f0f38eb5700 1 -- 172.21.15.19:0/3191904377 shutdown_connections 2024-05-09T02:14:26.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:26.546+0000 7f0f38eb5700 1 -- 172.21.15.19:0/3191904377 wait complete. 2024-05-09T02:14:26.562 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:26 smithi019 ceph-mon[30151]: pgmap v97: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 MiB/s wr, 1.82k op/s 2024-05-09T02:14:26.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:26 smithi059 ceph-mon[34781]: pgmap v97: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 MiB/s wr, 1.82k op/s 2024-05-09T02:14:26.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:26 smithi196 ceph-mon[34718]: pgmap v97: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 MiB/s wr, 1.82k op/s 2024-05-09T02:14:27.476 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T02:14:27.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 165.0 s, 2869.4 tps, lat 35.000 ms stddev 40.572 2024-05-09T02:14:27.868 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:14:28.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:28 smithi019 ceph-mon[30151]: pgmap v98: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 MiB/s wr, 717 op/s 2024-05-09T02:14:28.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:28 smithi059 ceph-mon[34781]: pgmap v98: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 MiB/s wr, 717 op/s 2024-05-09T02:14:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:28 smithi196 ceph-mon[34718]: pgmap v98: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 MiB/s wr, 717 op/s 2024-05-09T02:14:29.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.782+0000 7f6aa402e700 1 -- 172.21.15.19:0/3442697873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a9c108740 msgr2=0x7f6a9c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:29.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.782+0000 7f6aa402e700 1 --2- 172.21.15.19:0/3442697873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a9c108740 0x7f6a9c1100a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400b0d0 tx=0x7f6a9400b3e0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.782+0000 7f6aa402e700 1 -- 172.21.15.19:0/3442697873 shutdown_connections 2024-05-09T02:14:29.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.782+0000 7f6aa402e700 1 --2- 172.21.15.19:0/3442697873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a9c108740 0x7f6a9c1100a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.782+0000 7f6aa402e700 1 --2- 172.21.15.19:0/3442697873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a9c1034c0 0x7f6a9c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.782+0000 7f6aa402e700 1 --2- 172.21.15.19:0/3442697873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a9c107df0 0x7f6a9c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.782+0000 7f6aa402e700 1 -- 172.21.15.19:0/3442697873 >> 172.21.15.19:0/3442697873 conn(0x7f6a9c0fedc0 msgr2=0x7f6a9c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:29.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.783+0000 7f6aa402e700 1 -- 172.21.15.19:0/3442697873 shutdown_connections 2024-05-09T02:14:29.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.783+0000 7f6aa402e700 1 -- 172.21.15.19:0/3442697873 wait complete. 2024-05-09T02:14:29.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.783+0000 7f6aa402e700 1 Processor -- start 2024-05-09T02:14:29.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.783+0000 7f6aa402e700 1 -- start start 2024-05-09T02:14:29.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa402e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a9c1034c0 0x7f6a9c0ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa402e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a9c107df0 0x7f6a9c0ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa402e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a9c108740 0x7f6a9c0ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa402e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c113370 con 0x7f6a9c1034c0 2024-05-09T02:14:29.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa402e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1131f0 con 0x7f6a9c107df0 2024-05-09T02:14:29.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa402e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1134f0 con 0x7f6a9c108740 2024-05-09T02:14:29.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa15c9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a9c107df0 0x7f6a9c0ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa1dca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a9c1034c0 0x7f6a9c0ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa15c9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a9c107df0 0x7f6a9c0ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:33226/0 (socket says 172.21.15.19:33226) 2024-05-09T02:14:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa15c9700 1 -- 172.21.15.19:0/2655711516 learned_addr learned my addr 172.21.15.19:0/2655711516 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa25cb700 1 --2- 172.21.15.19:0/2655711516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a9c108740 0x7f6a9c0ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa15c9700 1 -- 172.21.15.19:0/2655711516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a9c108740 msgr2=0x7f6a9c0ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa15c9700 1 --2- 172.21.15.19:0/2655711516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a9c108740 0x7f6a9c0ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa15c9700 1 -- 172.21.15.19:0/2655711516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a9c1034c0 msgr2=0x7f6a9c0ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa15c9700 1 --2- 172.21.15.19:0/2655711516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a9c1034c0 0x7f6a9c0ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.784+0000 7f6aa15c9700 1 -- 172.21.15.19:0/2655711516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c1a91b0 con 0x7f6a9c107df0 2024-05-09T02:14:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.785+0000 7f6aa15c9700 1 --2- 172.21.15.19:0/2655711516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a9c107df0 0x7f6a9c0ff7e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00d980 tx=0x7f6a8c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.785+0000 7f6a92ffd700 1 -- 172.21.15.19:0/2655711516 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c017070 con 0x7f6a9c107df0 2024-05-09T02:14:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.785+0000 7f6aa402e700 1 -- 172.21.15.19:0/2655711516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9c19abd0 con 0x7f6a9c107df0 2024-05-09T02:14:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.785+0000 7f6aa402e700 1 -- 172.21.15.19:0/2655711516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a9c19b160 con 0x7f6a9c107df0 2024-05-09T02:14:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.785+0000 7f6a92ffd700 1 -- 172.21.15.19:0/2655711516 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a8c00fd40 con 0x7f6a9c107df0 2024-05-09T02:14:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.785+0000 7f6a92ffd700 1 -- 172.21.15.19:0/2655711516 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00e3b0 con 0x7f6a9c107df0 2024-05-09T02:14:29.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.788+0000 7f6a92ffd700 1 -- 172.21.15.19:0/2655711516 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6a8c00e550 con 0x7f6a9c107df0 2024-05-09T02:14:29.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.789+0000 7f6ab2a44b80 1 Processor -- start 2024-05-09T02:14:29.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.789+0000 7f6ab2a44b80 1 -- start start 2024-05-09T02:14:29.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.789+0000 7f6ab2a44b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 0x55f60af08240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.789+0000 7f6ab2a44b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af08780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.789+0000 7f6ab2a44b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 0x55f60af02810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.789+0000 7f6ab2a44b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f60ae250a0 con 0x55f60af02400 2024-05-09T02:14:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.789+0000 7f6ab2a44b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f60adca9a0 con 0x55f60adffd00 2024-05-09T02:14:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.789+0000 7f6ab2a44b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f60ae21060 con 0x55f60ad854a0 2024-05-09T02:14:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.789+0000 7f6a92ffd700 1 --2- 172.21.15.19:0/2655711516 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6a880b2a70 0x7f6a880b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.790+0000 7f6aa25cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 0x55f60af08240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.790+0000 7f6aa25cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 0x55f60af08240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:33238/0 (socket says 172.21.15.19:33238) 2024-05-09T02:14:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.790+0000 7f6aa25cb700 1 -- 172.21.15.19:0/3064423135 learned_addr learned my addr 172.21.15.19:0/3064423135 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.790+0000 7f6aa1dca700 1 --2- 172.21.15.19:0/3064423135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af08780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.790+0000 7f6aa25cb700 1 --2- 172.21.15.19:0/2655711516 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6a880b2a70 0x7f6a880b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.790+0000 7f6aa15c9700 1 --2- 172.21.15.19:0/3064423135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 0x55f60af02810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.790+0000 7f6a92ffd700 1 -- 172.21.15.19:0/2655711516 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f6a8c0e23b0 con 0x7f6a9c107df0 2024-05-09T02:14:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.790+0000 7f6aa1dca700 1 -- 172.21.15.19:0/3064423135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 msgr2=0x55f60af08240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.790+0000 7f6aa1dca700 1 --2- 172.21.15.19:0/3064423135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 0x55f60af08240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.790+0000 7f6aa1dca700 1 -- 172.21.15.19:0/3064423135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 msgr2=0x55f60af02810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.790+0000 7f6aa1dca700 1 --2- 172.21.15.19:0/3064423135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 0x55f60af02810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.790+0000 7f6aa1dca700 1 -- 172.21.15.19:0/3064423135 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f60adbb500 con 0x55f60ad854a0 2024-05-09T02:14:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.790+0000 7f6aa15c9700 1 --2- 172.21.15.19:0/3064423135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 0x55f60af02810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:14:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.790+0000 7f6aa25cb700 1 --2- 172.21.15.19:0/2655711516 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6a880b2a70 0x7f6a880b4f80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009e60 tx=0x7f6a9400b040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.790+0000 7f6aa1dca700 1 --2- 172.21.15.19:0/3064423135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af08780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009cb0 tx=0x7f6a98011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b785ed95954cf727 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.791+0000 7f6a87fff700 1 -- 172.21.15.19:0/3064423135 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98011d20 con 0x55f60ad854a0 2024-05-09T02:14:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.791+0000 7f6a87fff700 1 -- 172.21.15.19:0/3064423135 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a98004500 con 0x55f60ad854a0 2024-05-09T02:14:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.791+0000 7f6a87fff700 1 -- 172.21.15.19:0/3064423135 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98005310 con 0x55f60ad854a0 2024-05-09T02:14:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.792+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/3064423135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 msgr2=0x55f60af08780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.792+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/3064423135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af08780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009cb0 tx=0x7f6a98011440 comp rx=0 tx=0).stop 2024-05-09T02:14:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.792+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/3064423135 shutdown_connections 2024-05-09T02:14:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.792+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/3064423135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 0x55f60af02810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.792+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/3064423135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 0x55f60af08240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.792+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/3064423135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af08780 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.792+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/3064423135 >> 172.21.15.19:0/3064423135 conn(0x55f60ad88f70 msgr2=0x55f60aefd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.792+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/3064423135 shutdown_connections 2024-05-09T02:14:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.792+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/3064423135 wait complete. 2024-05-09T02:14:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.792+0000 7f6ab2a44b80 1 -- start start 2024-05-09T02:14:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.793+0000 7f6ab2a44b80 1 Processor -- start 2024-05-09T02:14:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.793+0000 7f6ab2a44b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af0bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.793+0000 7f6ab2a44b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 0x55f60af0c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.793+0000 7f6ab2a44b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 0x55f60af0c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.793+0000 7f6ab2a44b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f60af03160 con 0x55f60af02400 2024-05-09T02:14:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.793+0000 7f6ab2a44b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f60adbfb20 con 0x55f60adffd00 2024-05-09T02:14:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.793+0000 7f6ab2a44b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f60af0f020 con 0x55f60ad854a0 2024-05-09T02:14:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.793+0000 7f6ab2a44b80 10 client.0 _ll_register_callbacks cb 0x55f60ac966c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.793+0000 7f6aa1dca700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af0bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.793+0000 7f6aa1dca700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af0bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:38290/0 (socket says 172.21.15.19:38290) 2024-05-09T02:14:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.793+0000 7f6aa25cb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 0x55f60af0c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.793+0000 7f6aa1dca700 1 -- 172.21.15.19:0/1519421092 learned_addr learned my addr 172.21.15.19:0/1519421092 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.793+0000 7f6aa15c9700 1 --2- 172.21.15.19:0/1519421092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 0x55f60af0c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.794+0000 7f6aa1dca700 1 -- 172.21.15.19:0/1519421092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 msgr2=0x55f60af0c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.794+0000 7f6aa1dca700 1 --2- 172.21.15.19:0/1519421092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 0x55f60af0c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.794+0000 7f6aa1dca700 1 -- 172.21.15.19:0/1519421092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 msgr2=0x55f60af0c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.794+0000 7f6aa1dca700 1 --2- 172.21.15.19:0/1519421092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 0x55f60af0c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.794+0000 7f6aa1dca700 1 -- 172.21.15.19:0/1519421092 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f60aca0490 con 0x55f60ad854a0 2024-05-09T02:14:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.794+0000 7f6aa15c9700 1 --2- 172.21.15.19:0/1519421092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 0x55f60af0c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:14:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.794+0000 7f6aa1dca700 1 --2- 172.21.15.19:0/1519421092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af0bf00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6a98004d10 tx=0x7f6a98004d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70be76d88de0dff9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.794+0000 7f6a857fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:14:29.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.794+0000 7f6a857fa700 1 -- 172.21.15.19:0/1519421092 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98004a70 con 0x55f60ad854a0 2024-05-09T02:14:29.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.794+0000 7f6a857fa700 1 -- 172.21.15.19:0/1519421092 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f6a9801e070 con 0x55f60ad854a0 2024-05-09T02:14:29.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.794+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1519421092 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a98004500 con 0x55f60ad854a0 2024-05-09T02:14:29.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.794+0000 7f6a857fa700 1 -- 172.21.15.19:0/1519421092 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6a9802f050 con 0x55f60ad854a0 2024-05-09T02:14:29.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.794+0000 7f6ab2a44b80 10 client.25459 fetch_fsmap learned FSMap version 34 2024-05-09T02:14:29.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.794+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1519421092 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f60af19960 con 0x55f60ad854a0 2024-05-09T02:14:29.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.795+0000 7f6a857fa700 1 -- 172.21.15.19:0/1519421092 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a98004500 con 0x55f60ad854a0 2024-05-09T02:14:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.795+0000 7f6a857fa700 1 -- 172.21.15.19:0/1519421092 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f6a98018070 con 0x55f60ad854a0 2024-05-09T02:14:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.795+0000 7f6ab2a44b80 10 client.25459 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T02:14:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.795+0000 7f6ab2a44b80 10 client.25459 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:14:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.795+0000 7f6ab2a44b80 20 client.25459 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.795+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/1519421092 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55f60af1be30 0x55f60af1e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.795+0000 7f6ab2a44b80 4 client.25459 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:14:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.795+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1519421092 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f60ae32a80 con 0x55f60af1be30 2024-05-09T02:14:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.796+0000 7f6aa15c9700 1 --2- 172.21.15.19:0/1519421092 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55f60af1be30 0x55f60af1e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.797+0000 7f6a857fa700 10 client.25459 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:14:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.797+0000 7f6aa15c9700 1 --2- 172.21.15.19:0/1519421092 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55f60af1be30 0x55f60af1e250 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e2251095d63705d1 server_cookie=3a35c61257e777db in_seq=0 out_seq=0 2024-05-09T02:14:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.798+0000 7f6a857fa700 1 -- 172.21.15.19:0/1519421092 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f60ae32a80 con 0x55f60af1be30 2024-05-09T02:14:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.798+0000 7f6a857fa700 10 client.25459 handle_command_reply: tid=0 2024-05-09T02:14:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.798+0000 7f6ab2a44b80 1 client.25459 shutdown 2024-05-09T02:14:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.798+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1519421092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 msgr2=0x55f60af0bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.798+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/1519421092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af0bf00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6a98004d10 tx=0x7f6a98004d40 comp rx=0 tx=0).stop 2024-05-09T02:14:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.798+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1519421092 shutdown_connections 2024-05-09T02:14:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.798+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/1519421092 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55f60af1be30 0x55f60af1e250 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.798+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/1519421092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 0x55f60af0c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.798+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/1519421092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af0bf00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.798+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/1519421092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 0x55f60af0c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.798+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1519421092 >> 172.21.15.19:0/1519421092 conn(0x55f60ad88f70 msgr2=0x55f60aef6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:29.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.798+0000 7f6a857fa700 0 client.25459 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:14:29.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.799+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1519421092 shutdown_connections 2024-05-09T02:14:29.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.799+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1519421092 wait complete. 2024-05-09T02:14:29.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.799+0000 7f6ab2a44b80 20 client.25459 trim_cache size 0 max 16384 2024-05-09T02:14:29.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.835+0000 7f6ab2a44b80 1 Processor -- start 2024-05-09T02:14:29.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.835+0000 7f6ab2a44b80 1 -- start start 2024-05-09T02:14:29.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.835+0000 7f6ab2a44b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af0c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.835+0000 7f6ab2a44b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60adffd00 0x55f60af17ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.835+0000 7f6ab2a44b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60af02400 0x55f60aef7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.835+0000 7f6ab2a44b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8000d600 con 0x55f60adffd00 2024-05-09T02:14:29.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.835+0000 7f6ab2a44b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8000f1d0 con 0x55f60af02400 2024-05-09T02:14:29.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.835+0000 7f6ab2a44b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8000dcd0 con 0x55f60ad854a0 2024-05-09T02:14:29.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.835+0000 7f6aa15c9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60adffd00 0x55f60af17ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.835+0000 7f6aa1dca700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af0c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.836+0000 7f6aa15c9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60adffd00 0x55f60af17ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60442/0 (socket says 172.21.15.19:60442) 2024-05-09T02:14:29.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.836+0000 7f6aa15c9700 1 -- 172.21.15.19:0/32177687 learned_addr learned my addr 172.21.15.19:0/32177687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:29.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.836+0000 7f6aa1dca700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af0c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:38300/0 (socket says 172.21.15.19:38300) 2024-05-09T02:14:29.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.836+0000 7f6aa25cb700 1 --2- 172.21.15.19:0/32177687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60af02400 0x55f60aef7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.836+0000 7f6aa1dca700 1 -- 172.21.15.19:0/32177687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60af02400 msgr2=0x55f60aef7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:29.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.836+0000 7f6aa1dca700 1 --2- 172.21.15.19:0/32177687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60af02400 0x55f60aef7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.836+0000 7f6aa1dca700 1 -- 172.21.15.19:0/32177687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60adffd00 msgr2=0x55f60af17ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:29.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.836+0000 7f6aa1dca700 1 --2- 172.21.15.19:0/32177687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60adffd00 0x55f60af17ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.836+0000 7f6aa1dca700 1 -- 172.21.15.19:0/32177687 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f60aefdf40 con 0x55f60ad854a0 2024-05-09T02:14:29.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.836+0000 7f6aa15c9700 1 --2- 172.21.15.19:0/32177687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60adffd00 0x55f60af17ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:14:29.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.836+0000 7f6aa1dca700 1 --2- 172.21.15.19:0/32177687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af0c7c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6a98011480 tx=0x7f6a98009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bd50cb71118f857 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:29.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.836+0000 7f6a877fe700 1 -- 172.21.15.19:0/32177687 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98004800 con 0x55f60ad854a0 2024-05-09T02:14:29.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.836+0000 7f6a877fe700 1 -- 172.21.15.19:0/32177687 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a98003680 con 0x55f60ad854a0 2024-05-09T02:14:29.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.836+0000 7f6a877fe700 1 -- 172.21.15.19:0/32177687 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98005260 con 0x55f60ad854a0 2024-05-09T02:14:29.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.837+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/32177687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 msgr2=0x55f60af0c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:29.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.837+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/32177687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af0c7c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6a98011480 tx=0x7f6a98009830 comp rx=0 tx=0).stop 2024-05-09T02:14:29.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.837+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/32177687 shutdown_connections 2024-05-09T02:14:29.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.837+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/32177687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60af02400 0x55f60aef7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.838+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/32177687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af0c7c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.838+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/32177687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60adffd00 0x55f60af17ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.838+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/32177687 >> 172.21.15.19:0/32177687 conn(0x55f60ad88f70 msgr2=0x55f60aef6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:29.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.838+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/32177687 shutdown_connections 2024-05-09T02:14:29.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.838+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/32177687 wait complete. 2024-05-09T02:14:29.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.838+0000 7f6ab2a44b80 1 -- start start 2024-05-09T02:14:29.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.838+0000 7f6ab2a44b80 1 Processor -- start 2024-05-09T02:14:29.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.839+0000 7f6ab2a44b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af14470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.839+0000 7f6ab2a44b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 0x55f60af22960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.839+0000 7f6ab2a44b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 0x55f60af22ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.839+0000 7f6ab2a44b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8000ea70 con 0x55f60af02400 2024-05-09T02:14:29.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.839+0000 7f6ab2a44b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8000e3a0 con 0x55f60adffd00 2024-05-09T02:14:29.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.839+0000 7f6ab2a44b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8000b5a0 con 0x55f60ad854a0 2024-05-09T02:14:29.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.839+0000 7f6ab2a44b80 10 client.0 _ll_register_callbacks cb 0x55f60ac966c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:29.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.839+0000 7f6aa1dca700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af14470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.839+0000 7f6aa1dca700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af14470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:38304/0 (socket says 172.21.15.19:38304) 2024-05-09T02:14:29.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.839+0000 7f6aa1dca700 1 -- 172.21.15.19:0/1664513127 learned_addr learned my addr 172.21.15.19:0/1664513127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:29.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.839+0000 7f6aa15c9700 1 --2- 172.21.15.19:0/1664513127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 0x55f60af22960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.839+0000 7f6aa25cb700 1 --2- 172.21.15.19:0/1664513127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 0x55f60af22ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.840+0000 7f6aa1dca700 1 -- 172.21.15.19:0/1664513127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 msgr2=0x55f60af22960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:29.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.840+0000 7f6aa1dca700 1 --2- 172.21.15.19:0/1664513127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 0x55f60af22960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.840+0000 7f6aa1dca700 1 -- 172.21.15.19:0/1664513127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 msgr2=0x55f60af22ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:29.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.840+0000 7f6aa1dca700 1 --2- 172.21.15.19:0/1664513127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 0x55f60af22ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.840+0000 7f6aa1dca700 1 -- 172.21.15.19:0/1664513127 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f60af14bb0 con 0x55f60ad854a0 2024-05-09T02:14:29.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.840+0000 7f6aa1dca700 1 --2- 172.21.15.19:0/1664513127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af14470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6a98003850 tx=0x7f6a98003d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a7377ad5b9f0f7f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:29.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.840+0000 7f6a857fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:14:29.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.840+0000 7f6a857fa700 1 -- 172.21.15.19:0/1664513127 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98004800 con 0x55f60ad854a0 2024-05-09T02:14:29.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.840+0000 7f6a857fa700 1 -- 172.21.15.19:0/1664513127 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f6a98004f50 con 0x55f60ad854a0 2024-05-09T02:14:29.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.840+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1664513127 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a98003680 con 0x55f60ad854a0 2024-05-09T02:14:29.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.840+0000 7f6a857fa700 1 -- 172.21.15.19:0/1664513127 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6a98044050 con 0x55f60ad854a0 2024-05-09T02:14:29.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.841+0000 7f6ab2a44b80 10 client.25465 fetch_fsmap learned FSMap version 34 2024-05-09T02:14:29.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.841+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1664513127 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f60af24330 con 0x55f60ad854a0 2024-05-09T02:14:29.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.841+0000 7f6a857fa700 1 -- 172.21.15.19:0/1664513127 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a98003e30 con 0x55f60ad854a0 2024-05-09T02:14:29.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.841+0000 7f6a857fa700 1 -- 172.21.15.19:0/1664513127 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f6a98018040 con 0x55f60ad854a0 2024-05-09T02:14:29.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.841+0000 7f6ab2a44b80 10 client.25465 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T02:14:29.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.841+0000 7f6ab2a44b80 10 client.25465 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:14:29.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.841+0000 7f6ab2a44b80 20 client.25465 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:29.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.841+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/1664513127 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55f60af858e0 0x55f60af87d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:29.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.841+0000 7f6ab2a44b80 4 client.25465 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:14:29.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.841+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1664513127 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6a98003680 con 0x55f60af858e0 2024-05-09T02:14:29.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.841+0000 7f6aa15c9700 1 --2- 172.21.15.19:0/1664513127 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55f60af858e0 0x55f60af87d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:29.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.842+0000 7f6a857fa700 10 client.25465 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:14:29.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.842+0000 7f6aa15c9700 1 --2- 172.21.15.19:0/1664513127 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55f60af858e0 0x55f60af87d00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b6ea00561dd2e3ac server_cookie=4c965c9014d5c6c8 in_seq=0 out_seq=0 2024-05-09T02:14:29.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.842+0000 7f6a857fa700 1 -- 172.21.15.19:0/1664513127 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6a98003680 con 0x55f60af858e0 2024-05-09T02:14:29.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.842+0000 7f6a857fa700 10 client.25465 handle_command_reply: tid=0 2024-05-09T02:14:29.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.842+0000 7f6ab2a44b80 1 client.25465 shutdown 2024-05-09T02:14:29.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.842+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1664513127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 msgr2=0x55f60af14470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:29.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.842+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/1664513127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af14470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6a98003850 tx=0x7f6a98003d40 comp rx=0 tx=0).stop 2024-05-09T02:14:29.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.843+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1664513127 shutdown_connections 2024-05-09T02:14:29.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.843+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/1664513127 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55f60af858e0 0x55f60af87d00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.843+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/1664513127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f60af02400 0x55f60af22ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.843+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/1664513127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60ad854a0 0x55f60af14470 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.843+0000 7f6ab2a44b80 1 --2- 172.21.15.19:0/1664513127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f60adffd00 0x55f60af22960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.843+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1664513127 >> 172.21.15.19:0/1664513127 conn(0x55f60ad88f70 msgr2=0x55f60af077f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:29.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.843+0000 7f6a857fa700 0 client.25465 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:14:29.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.843+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1664513127 shutdown_connections 2024-05-09T02:14:29.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.843+0000 7f6ab2a44b80 1 -- 172.21.15.19:0/1664513127 wait complete. 2024-05-09T02:14:29.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.843+0000 7f6ab2a44b80 20 client.25465 trim_cache size 0 max 16384 2024-05-09T02:14:29.851 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:14:29.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.844+0000 7f6aa402e700 1 -- 172.21.15.19:0/2655711516 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6a880b2a70 msgr2=0x7f6a880b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:29.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.844+0000 7f6aa402e700 1 --2- 172.21.15.19:0/2655711516 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6a880b2a70 0x7f6a880b4f80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009e60 tx=0x7f6a9400b040 comp rx=0 tx=0).stop 2024-05-09T02:14:29.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.844+0000 7f6aa402e700 1 -- 172.21.15.19:0/2655711516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a9c107df0 msgr2=0x7f6a9c0ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:29.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.844+0000 7f6aa402e700 1 --2- 172.21.15.19:0/2655711516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a9c107df0 0x7f6a9c0ff7e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00d980 tx=0x7f6a8c00dc90 comp rx=0 tx=0).stop 2024-05-09T02:14:29.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.844+0000 7f6aa402e700 1 -- 172.21.15.19:0/2655711516 shutdown_connections 2024-05-09T02:14:29.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.844+0000 7f6aa402e700 1 --2- 172.21.15.19:0/2655711516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a9c108740 0x7f6a9c0ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.844+0000 7f6aa402e700 1 --2- 172.21.15.19:0/2655711516 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6a880b2a70 0x7f6a880b4f80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.844+0000 7f6aa402e700 1 --2- 172.21.15.19:0/2655711516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a9c1034c0 0x7f6a9c0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.844+0000 7f6aa402e700 1 --2- 172.21.15.19:0/2655711516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a9c107df0 0x7f6a9c0ff7e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:29.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.844+0000 7f6aa402e700 1 -- 172.21.15.19:0/2655711516 >> 172.21.15.19:0/2655711516 conn(0x7f6a9c0fedc0 msgr2=0x7f6a9c107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:29.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.844+0000 7f6aa402e700 1 -- 172.21.15.19:0/2655711516 shutdown_connections 2024-05-09T02:14:29.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:29.844+0000 7f6aa402e700 1 -- 172.21.15.19:0/2655711516 wait complete. 2024-05-09T02:14:30.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:30 smithi019 ceph-mon[30151]: pgmap v99: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-05-09T02:14:30.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:30 smithi059 ceph-mon[34781]: pgmap v99: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-05-09T02:14:30.786 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T02:14:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:30 smithi196 ceph-mon[34718]: pgmap v99: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-05-09T02:14:31.210 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:14:32.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:32 smithi019 ceph-mon[30151]: pgmap v100: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.3 MiB/s wr, 662 op/s 2024-05-09T02:14:32.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:32 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:14:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 170.0 s, 3242.1 tps, lat 30.787 ms stddev 32.399 2024-05-09T02:14:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:32 smithi059 ceph-mon[34781]: pgmap v100: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.3 MiB/s wr, 662 op/s 2024-05-09T02:14:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:32 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:14:32.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:32 smithi196 ceph-mon[34718]: pgmap v100: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.3 MiB/s wr, 662 op/s 2024-05-09T02:14:32.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:32 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:14:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.151+0000 7f5d9f2ed700 1 -- 172.21.15.19:0/2070379294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d98110fa0 msgr2=0x7f5d98118900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.151+0000 7f5d9f2ed700 1 --2- 172.21.15.19:0/2070379294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d98110fa0 0x7f5d98118900 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000b0d0 tx=0x7f5d9000b3e0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.152+0000 7f5d9f2ed700 1 -- 172.21.15.19:0/2070379294 shutdown_connections 2024-05-09T02:14:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.152+0000 7f5d9f2ed700 1 --2- 172.21.15.19:0/2070379294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d98110fa0 0x7f5d98118900 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.152+0000 7f5d9f2ed700 1 --2- 172.21.15.19:0/2070379294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d9810c830 0x7f5d9810cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.152+0000 7f5d9f2ed700 1 --2- 172.21.15.19:0/2070379294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d98110650 0x7f5d98110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.152+0000 7f5d9f2ed700 1 -- 172.21.15.19:0/2070379294 >> 172.21.15.19:0/2070379294 conn(0x7f5d9806f990 msgr2=0x7f5d9806fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.152+0000 7f5d9f2ed700 1 -- 172.21.15.19:0/2070379294 shutdown_connections 2024-05-09T02:14:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.152+0000 7f5d9f2ed700 1 -- 172.21.15.19:0/2070379294 wait complete. 2024-05-09T02:14:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.152+0000 7f5d9f2ed700 1 Processor -- start 2024-05-09T02:14:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.152+0000 7f5d9f2ed700 1 -- start start 2024-05-09T02:14:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9f2ed700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d9810c830 0x7f5d981a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9f2ed700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d98110650 0x7f5d981a2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9f2ed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d98110fa0 0x7f5d981a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9f2ed700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9811bbe0 con 0x7f5d98110fa0 2024-05-09T02:14:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9f2ed700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9811ba60 con 0x7f5d98110650 2024-05-09T02:14:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9f2ed700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9811bd60 con 0x7f5d9810c830 2024-05-09T02:14:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9d089700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d9810c830 0x7f5d981a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9d089700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d9810c830 0x7f5d981a21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:38326/0 (socket says 172.21.15.19:38326) 2024-05-09T02:14:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9d089700 1 -- 172.21.15.19:0/2045998303 learned_addr learned my addr 172.21.15.19:0/2045998303 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9d88a700 1 --2- 172.21.15.19:0/2045998303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d98110fa0 0x7f5d981a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9c888700 1 --2- 172.21.15.19:0/2045998303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d98110650 0x7f5d981a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9d089700 1 -- 172.21.15.19:0/2045998303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d98110650 msgr2=0x7f5d981a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9d089700 1 --2- 172.21.15.19:0/2045998303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d98110650 0x7f5d981a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9d089700 1 -- 172.21.15.19:0/2045998303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d98110fa0 msgr2=0x7f5d981a2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9d089700 1 --2- 172.21.15.19:0/2045998303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d98110fa0 0x7f5d981a2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.153+0000 7f5d9d089700 1 -- 172.21.15.19:0/2045998303 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d981b1900 con 0x7f5d9810c830 2024-05-09T02:14:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.154+0000 7f5d9d089700 1 --2- 172.21.15.19:0/2045998303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d9810c830 0x7f5d981a21d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e970 tx=0x7f5d9400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.154+0000 7f5d8e7fc700 1 -- 172.21.15.19:0/2045998303 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9400cb70 con 0x7f5d9810c830 2024-05-09T02:14:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.154+0000 7f5d9f2ed700 1 -- 172.21.15.19:0/2045998303 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d981b1bf0 con 0x7f5d9810c830 2024-05-09T02:14:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.154+0000 7f5d8e7fc700 1 -- 172.21.15.19:0/2045998303 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d940044d0 con 0x7f5d9810c830 2024-05-09T02:14:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.154+0000 7f5d8e7fc700 1 -- 172.21.15.19:0/2045998303 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94003df0 con 0x7f5d9810c830 2024-05-09T02:14:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.154+0000 7f5d9f2ed700 1 -- 172.21.15.19:0/2045998303 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d981b2180 con 0x7f5d9810c830 2024-05-09T02:14:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.157+0000 7f5d8e7fc700 1 -- 172.21.15.19:0/2045998303 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5d94010480 con 0x7f5d9810c830 2024-05-09T02:14:33.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.158+0000 7f5d8e7fc700 1 --2- 172.21.15.19:0/2045998303 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5d840b29f0 0x7f5d840b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.158+0000 7f5d9c888700 1 --2- 172.21.15.19:0/2045998303 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5d840b29f0 0x7f5d840b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.158+0000 7f5d8e7fc700 1 -- 172.21.15.19:0/2045998303 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f5d940e2ab0 con 0x7f5d9810c830 2024-05-09T02:14:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.159+0000 7f5dadd03b80 1 Processor -- start 2024-05-09T02:14:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.159+0000 7f5dadd03b80 1 -- start start 2024-05-09T02:14:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.159+0000 7f5d9c888700 1 --2- 172.21.15.19:0/2045998303 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5d840b29f0 0x7f5d840b4f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5d9810d660 tx=0x7f5d88009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.159+0000 7f5dadd03b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 0x555c1a1d6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.159+0000 7f5dadd03b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0534a0 0x555c1a1d6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.159+0000 7f5dadd03b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a1d0400 0x555c1a1d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.159+0000 7f5dadd03b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555c1a0f30a0 con 0x555c1a0534a0 2024-05-09T02:14:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.159+0000 7f5dadd03b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555c1a0989a0 con 0x555c1a0cdd00 2024-05-09T02:14:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.159+0000 7f5dadd03b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555c1a0ef060 con 0x555c1a1d0400 2024-05-09T02:14:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.159+0000 7f5d9c888700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0534a0 0x555c1a1d6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.159+0000 7f5d9c888700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0534a0 0x555c1a1d6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60482/0 (socket says 172.21.15.19:60482) 2024-05-09T02:14:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.159+0000 7f5d9c888700 1 -- 172.21.15.19:0/2049949594 learned_addr learned my addr 172.21.15.19:0/2049949594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.159+0000 7f5d9d089700 1 --2- 172.21.15.19:0/2049949594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 0x555c1a1d6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.159+0000 7f5d9d88a700 1 --2- 172.21.15.19:0/2049949594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a1d0400 0x555c1a1d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.160+0000 7f5d9d089700 1 -- 172.21.15.19:0/2049949594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a1d0400 msgr2=0x555c1a1d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.160+0000 7f5d9d089700 1 --2- 172.21.15.19:0/2049949594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a1d0400 0x555c1a1d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.160+0000 7f5d9d089700 1 -- 172.21.15.19:0/2049949594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0534a0 msgr2=0x555c1a1d6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.160+0000 7f5d9d089700 1 --2- 172.21.15.19:0/2049949594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0534a0 0x555c1a1d6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.160+0000 7f5d9d089700 1 -- 172.21.15.19:0/2049949594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1a089500 con 0x555c1a0cdd00 2024-05-09T02:14:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.160+0000 7f5d9c888700 1 --2- 172.21.15.19:0/2049949594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0534a0 0x555c1a1d6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:14:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.161+0000 7f5d9d089700 1 --2- 172.21.15.19:0/2049949594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 0x555c1a1d6240 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5d940e01a0 tx=0x7f5d9400ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c804849221449b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.161+0000 7f5d837fe700 1 -- 172.21.15.19:0/2049949594 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d940a9840 con 0x555c1a0cdd00 2024-05-09T02:14:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.162+0000 7f5d837fe700 1 -- 172.21.15.19:0/2049949594 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d940c0070 con 0x555c1a0cdd00 2024-05-09T02:14:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.162+0000 7f5d837fe700 1 -- 172.21.15.19:0/2049949594 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d940a9f90 con 0x555c1a0cdd00 2024-05-09T02:14:33.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.163+0000 7f5dadd03b80 1 -- 172.21.15.19:0/2049949594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 msgr2=0x555c1a1d6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:33.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.163+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/2049949594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 0x555c1a1d6240 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5d940e01a0 tx=0x7f5d9400ef70 comp rx=0 tx=0).stop 2024-05-09T02:14:33.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.163+0000 7f5dadd03b80 1 -- 172.21.15.19:0/2049949594 shutdown_connections 2024-05-09T02:14:33.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.163+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/2049949594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a1d0400 0x555c1a1d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.163+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/2049949594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 0x555c1a1d6240 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.163+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/2049949594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0534a0 0x555c1a1d6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.163+0000 7f5dadd03b80 1 -- 172.21.15.19:0/2049949594 >> 172.21.15.19:0/2049949594 conn(0x555c1a056f70 msgr2=0x555c1a1cb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.163+0000 7f5dadd03b80 1 -- 172.21.15.19:0/2049949594 shutdown_connections 2024-05-09T02:14:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.163+0000 7f5dadd03b80 1 -- 172.21.15.19:0/2049949594 wait complete. 2024-05-09T02:14:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.163+0000 7f5dadd03b80 1 -- start start 2024-05-09T02:14:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.164+0000 7f5dadd03b80 1 Processor -- start 2024-05-09T02:14:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.164+0000 7f5dadd03b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a0534a0 0x555c1a1da120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.164+0000 7f5dadd03b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0cdd00 0x555c1a1da660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.164+0000 7f5dadd03b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a1d0400 0x555c1a1daba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.164+0000 7f5dadd03b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555c1a1d1160 con 0x555c1a0cdd00 2024-05-09T02:14:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.164+0000 7f5dadd03b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555c1a08db20 con 0x555c1a1d0400 2024-05-09T02:14:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.165+0000 7f5dadd03b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555c1a1dd020 con 0x555c1a0534a0 2024-05-09T02:14:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.165+0000 7f5dadd03b80 10 client.0 _ll_register_callbacks cb 0x555c19f646c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.165+0000 7f5d9d88a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a1d0400 0x555c1a1daba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.165+0000 7f5d9d089700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a0534a0 0x555c1a1da120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.165+0000 7f5d9d88a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a1d0400 0x555c1a1daba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:33292/0 (socket says 172.21.15.19:33292) 2024-05-09T02:14:33.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.165+0000 7f5d9d88a700 1 -- 172.21.15.19:0/2509786258 learned_addr learned my addr 172.21.15.19:0/2509786258 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:33.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.165+0000 7f5d9c888700 1 --2- 172.21.15.19:0/2509786258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0cdd00 0x555c1a1da660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.165+0000 7f5d9d88a700 1 -- 172.21.15.19:0/2509786258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a0534a0 msgr2=0x555c1a1da120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:33.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.165+0000 7f5d9d88a700 1 --2- 172.21.15.19:0/2509786258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a0534a0 0x555c1a1da120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.165+0000 7f5d9d88a700 1 -- 172.21.15.19:0/2509786258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0cdd00 msgr2=0x555c1a1da660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.165+0000 7f5d9d88a700 1 --2- 172.21.15.19:0/2509786258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0cdd00 0x555c1a1da660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.165+0000 7f5d9d88a700 1 -- 172.21.15.19:0/2509786258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c19f6e490 con 0x555c1a1d0400 2024-05-09T02:14:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.166+0000 7f5d9d88a700 1 --2- 172.21.15.19:0/2509786258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a1d0400 0x555c1a1daba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009580 tx=0x7f5d9000bb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13cd32683a1c61ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.166+0000 7f5d80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:14:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.167+0000 7f5d80ff9700 1 -- 172.21.15.19:0/2509786258 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d900079d0 con 0x555c1a1d0400 2024-05-09T02:14:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.167+0000 7f5d80ff9700 1 -- 172.21.15.19:0/2509786258 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f5d90007b70 con 0x555c1a1d0400 2024-05-09T02:14:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.167+0000 7f5dadd03b80 1 -- 172.21.15.19:0/2509786258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d940c0070 con 0x555c1a1d0400 2024-05-09T02:14:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.167+0000 7f5d80ff9700 1 -- 172.21.15.19:0/2509786258 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5d9000be20 con 0x555c1a1d0400 2024-05-09T02:14:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.167+0000 7f5dadd03b80 10 client.25412 fetch_fsmap learned FSMap version 34 2024-05-09T02:14:33.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.167+0000 7f5dadd03b80 1 -- 172.21.15.19:0/2509786258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555c1a1e7ae0 con 0x555c1a1d0400 2024-05-09T02:14:33.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.167+0000 7f5d80ff9700 1 -- 172.21.15.19:0/2509786258 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d940c0070 con 0x555c1a1d0400 2024-05-09T02:14:33.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.167+0000 7f5d80ff9700 1 -- 172.21.15.19:0/2509786258 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f5d90011070 con 0x555c1a1d0400 2024-05-09T02:14:33.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.168+0000 7f5dadd03b80 10 client.25412 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T02:14:33.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.168+0000 7f5dadd03b80 10 client.25412 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:14:33.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.168+0000 7f5dadd03b80 20 client.25412 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:33.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.168+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/2509786258 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x555c1a1ea170 0x555c1a1ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.168+0000 7f5dadd03b80 4 client.25412 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:14:33.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.168+0000 7f5dadd03b80 1 -- 172.21.15.19:0/2509786258 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c1a100a80 con 0x555c1a1ea170 2024-05-09T02:14:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.169+0000 7f5d9d089700 1 --2- 172.21.15.19:0/2509786258 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x555c1a1ea170 0x555c1a1ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.170+0000 7f5d80ff9700 10 client.25412 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:14:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.170+0000 7f5d9d089700 1 --2- 172.21.15.19:0/2509786258 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x555c1a1ea170 0x555c1a1ec5f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b7362c67ae743b server_cookie=7b7806901c0f9e2a in_seq=0 out_seq=0 2024-05-09T02:14:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.171+0000 7f5d80ff9700 1 -- 172.21.15.19:0/2509786258 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555c1a100a80 con 0x555c1a1ea170 2024-05-09T02:14:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.171+0000 7f5d80ff9700 10 client.25412 handle_command_reply: tid=0 2024-05-09T02:14:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.171+0000 7f5dadd03b80 1 client.25412 shutdown 2024-05-09T02:14:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.172+0000 7f5dadd03b80 1 -- 172.21.15.19:0/2509786258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a1d0400 msgr2=0x555c1a1daba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.172+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/2509786258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a1d0400 0x555c1a1daba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009580 tx=0x7f5d9000bb50 comp rx=0 tx=0).stop 2024-05-09T02:14:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.172+0000 7f5dadd03b80 1 -- 172.21.15.19:0/2509786258 shutdown_connections 2024-05-09T02:14:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.172+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/2509786258 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x555c1a1ea170 0x555c1a1ec5f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.172+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/2509786258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a1d0400 0x555c1a1daba0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.172+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/2509786258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a0534a0 0x555c1a1da120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.172+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/2509786258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0cdd00 0x555c1a1da660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.172+0000 7f5dadd03b80 1 -- 172.21.15.19:0/2509786258 >> 172.21.15.19:0/2509786258 conn(0x555c1a056f70 msgr2=0x555c1a1c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.173+0000 7f5d80ff9700 0 client.25412 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:14:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.173+0000 7f5dadd03b80 1 -- 172.21.15.19:0/2509786258 shutdown_connections 2024-05-09T02:14:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.173+0000 7f5dadd03b80 1 -- 172.21.15.19:0/2509786258 wait complete. 2024-05-09T02:14:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.173+0000 7f5dadd03b80 20 client.25412 trim_cache size 0 max 16384 2024-05-09T02:14:33.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.193+0000 7f5dadd03b80 1 Processor -- start 2024-05-09T02:14:33.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.193+0000 7f5dadd03b80 1 -- start start 2024-05-09T02:14:33.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.193+0000 7f5dadd03b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a0534a0 0x555c1a1d7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.193+0000 7f5dadd03b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 0x555c1a1ec4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.193+0000 7f5dadd03b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a1d0400 0x555c1a1d58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.193+0000 7f5dadd03b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981b98d0 con 0x555c1a1d0400 2024-05-09T02:14:33.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.193+0000 7f5dadd03b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981bb4a0 con 0x555c1a0cdd00 2024-05-09T02:14:33.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.193+0000 7f5dadd03b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981b9fa0 con 0x555c1a0534a0 2024-05-09T02:14:33.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.193+0000 7f5d9d88a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a1d0400 0x555c1a1d58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.194+0000 7f5d9d88a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a1d0400 0x555c1a1d58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60490/0 (socket says 172.21.15.19:60490) 2024-05-09T02:14:33.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.194+0000 7f5d9d88a700 1 -- 172.21.15.19:0/1252251095 learned_addr learned my addr 172.21.15.19:0/1252251095 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:33.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.194+0000 7f5d9d089700 1 --2- 172.21.15.19:0/1252251095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a0534a0 0x555c1a1d7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.194+0000 7f5d9c888700 1 --2- 172.21.15.19:0/1252251095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 0x555c1a1ec4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.194+0000 7f5d9d88a700 1 -- 172.21.15.19:0/1252251095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a0534a0 msgr2=0x555c1a1d7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:33.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.194+0000 7f5d9d88a700 1 --2- 172.21.15.19:0/1252251095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a0534a0 0x555c1a1d7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.194+0000 7f5d9d88a700 1 -- 172.21.15.19:0/1252251095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 msgr2=0x555c1a1ec4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:33.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.194+0000 7f5d9d88a700 1 --2- 172.21.15.19:0/1252251095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 0x555c1a1ec4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.194+0000 7f5d9d88a700 1 -- 172.21.15.19:0/1252251095 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1a1d9460 con 0x555c1a1d0400 2024-05-09T02:14:33.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.194+0000 7f5d9d089700 1 --2- 172.21.15.19:0/1252251095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a0534a0 0x555c1a1d7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:14:33.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.194+0000 7f5d9d88a700 1 --2- 172.21.15.19:0/1252251095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a1d0400 0x555c1a1d58f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000b0c0 tx=0x7f5d900099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6408be82e4058813 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:33.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.194+0000 7f5d82ffd700 1 -- 172.21.15.19:0/1252251095 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90034070 con 0x555c1a1d0400 2024-05-09T02:14:33.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.194+0000 7f5d82ffd700 1 -- 172.21.15.19:0/1252251095 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d90005c60 con 0x555c1a1d0400 2024-05-09T02:14:33.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.194+0000 7f5d82ffd700 1 -- 172.21.15.19:0/1252251095 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90007cf0 con 0x555c1a1d0400 2024-05-09T02:14:33.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.195+0000 7f5dadd03b80 1 -- 172.21.15.19:0/1252251095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a1d0400 msgr2=0x555c1a1d58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:33.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.195+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/1252251095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a1d0400 0x555c1a1d58f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000b0c0 tx=0x7f5d900099e0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.195+0000 7f5dadd03b80 1 -- 172.21.15.19:0/1252251095 shutdown_connections 2024-05-09T02:14:33.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.195+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/1252251095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a1d0400 0x555c1a1d58f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.195+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/1252251095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a0534a0 0x555c1a1d7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.195+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/1252251095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 0x555c1a1ec4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.195+0000 7f5dadd03b80 1 -- 172.21.15.19:0/1252251095 >> 172.21.15.19:0/1252251095 conn(0x555c1a056f70 msgr2=0x555c1a1c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:33.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.195+0000 7f5dadd03b80 1 -- 172.21.15.19:0/1252251095 shutdown_connections 2024-05-09T02:14:33.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.195+0000 7f5dadd03b80 1 -- 172.21.15.19:0/1252251095 wait complete. 2024-05-09T02:14:33.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.196+0000 7f5dadd03b80 1 -- start start 2024-05-09T02:14:33.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.196+0000 7f5dadd03b80 1 Processor -- start 2024-05-09T02:14:33.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.198+0000 7f5dadd03b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0534a0 0x555c1a1f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.198+0000 7f5dadd03b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 0x555c1a1e20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.198+0000 7f5dadd03b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a1d0400 0x555c1a1e25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.198+0000 7f5d9d089700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0534a0 0x555c1a1f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.198+0000 7f5dadd03b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981bad40 con 0x555c1a0534a0 2024-05-09T02:14:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.198+0000 7f5dadd03b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981ba670 con 0x555c1a0cdd00 2024-05-09T02:14:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.198+0000 7f5dadd03b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981b7870 con 0x555c1a1d0400 2024-05-09T02:14:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.198+0000 7f5d9d089700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0534a0 0x555c1a1f20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60492/0 (socket says 172.21.15.19:60492) 2024-05-09T02:14:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.198+0000 7f5d9d089700 1 -- 172.21.15.19:0/1278499065 learned_addr learned my addr 172.21.15.19:0/1278499065 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.198+0000 7f5d9d88a700 1 --2- 172.21.15.19:0/1278499065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a1d0400 0x555c1a1e25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.198+0000 7f5d9c888700 1 --2- 172.21.15.19:0/1278499065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 0x555c1a1e20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.198+0000 7f5dadd03b80 10 client.0 _ll_register_callbacks cb 0x555c19f646c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.199+0000 7f5d9c888700 1 -- 172.21.15.19:0/1278499065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a1d0400 msgr2=0x555c1a1e25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.199+0000 7f5d9c888700 1 --2- 172.21.15.19:0/1278499065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a1d0400 0x555c1a1e25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.199+0000 7f5d9c888700 1 -- 172.21.15.19:0/1278499065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0534a0 msgr2=0x555c1a1f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:33.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.199+0000 7f5d9c888700 1 --2- 172.21.15.19:0/1278499065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0534a0 0x555c1a1f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.199+0000 7f5d9c888700 1 -- 172.21.15.19:0/1278499065 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c1a1e2b30 con 0x555c1a0cdd00 2024-05-09T02:14:33.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.199+0000 7f5d9d88a700 1 --2- 172.21.15.19:0/1278499065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a1d0400 0x555c1a1e25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:14:33.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.199+0000 7f5d9d089700 1 --2- 172.21.15.19:0/1278499065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0534a0 0x555c1a1f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:14:33.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.199+0000 7f5d9c888700 1 --2- 172.21.15.19:0/1278499065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 0x555c1a1e20b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x555c1a1cc320 tx=0x7f5d88013070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8819a8b1f8c0309a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:33.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.199+0000 7f5d80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:14:33.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.199+0000 7f5d80ff9700 1 -- 172.21.15.19:0/1278499065 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8802e060 con 0x555c1a0cdd00 2024-05-09T02:14:33.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.199+0000 7f5d80ff9700 1 -- 172.21.15.19:0/1278499065 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f5d8801f840 con 0x555c1a0cdd00 2024-05-09T02:14:33.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.199+0000 7f5dadd03b80 1 -- 172.21.15.19:0/1278499065 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d90005c60 con 0x555c1a0cdd00 2024-05-09T02:14:33.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.199+0000 7f5d80ff9700 1 -- 172.21.15.19:0/1278499065 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5d8801fa40 con 0x555c1a0cdd00 2024-05-09T02:14:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.199+0000 7f5dadd03b80 10 client.25418 fetch_fsmap learned FSMap version 34 2024-05-09T02:14:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.199+0000 7f5dadd03b80 1 -- 172.21.15.19:0/1278499065 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555c1a25c1b0 con 0x555c1a0cdd00 2024-05-09T02:14:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.200+0000 7f5d80ff9700 1 -- 172.21.15.19:0/1278499065 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d90005c60 con 0x555c1a0cdd00 2024-05-09T02:14:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.200+0000 7f5d80ff9700 1 -- 172.21.15.19:0/1278499065 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f5d88028070 con 0x555c1a0cdd00 2024-05-09T02:14:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.201+0000 7f5dadd03b80 10 client.25418 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T02:14:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.201+0000 7f5dadd03b80 10 client.25418 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:14:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.201+0000 7f5dadd03b80 20 client.25418 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.201+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/1278499065 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x555c1a25c7d0 0x555c1a25ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.201+0000 7f5dadd03b80 4 client.25418 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:14:33.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.201+0000 7f5dadd03b80 1 -- 172.21.15.19:0/1278499065 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c1a0fab20 con 0x555c1a25c7d0 2024-05-09T02:14:33.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.201+0000 7f5d9d089700 1 --2- 172.21.15.19:0/1278499065 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x555c1a25c7d0 0x555c1a25ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.202+0000 7f5d9d089700 1 --2- 172.21.15.19:0/1278499065 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x555c1a25c7d0 0x555c1a25ebf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61eb222120706d39 server_cookie=fd8887d626fdddb4 in_seq=0 out_seq=0 2024-05-09T02:14:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.202+0000 7f5d80ff9700 10 client.25418 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:14:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.203+0000 7f5d80ff9700 1 -- 172.21.15.19:0/1278499065 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c1a0fab20 con 0x555c1a25c7d0 2024-05-09T02:14:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.203+0000 7f5d80ff9700 10 client.25418 handle_command_reply: tid=0 2024-05-09T02:14:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.203+0000 7f5dadd03b80 1 client.25418 shutdown 2024-05-09T02:14:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.203+0000 7f5dadd03b80 1 -- 172.21.15.19:0/1278499065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 msgr2=0x555c1a1e20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.203+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/1278499065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 0x555c1a1e20b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x555c1a1cc320 tx=0x7f5d88013070 comp rx=0 tx=0).stop 2024-05-09T02:14:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.203+0000 7f5dadd03b80 1 -- 172.21.15.19:0/1278499065 shutdown_connections 2024-05-09T02:14:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.203+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/1278499065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1a1d0400 0x555c1a1e25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.203+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/1278499065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c1a0534a0 0x555c1a1f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.203+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/1278499065 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x555c1a25c7d0 0x555c1a25ebf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.203+0000 7f5dadd03b80 1 --2- 172.21.15.19:0/1278499065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c1a0cdd00 0x555c1a1e20b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.203+0000 7f5d80ff9700 0 client.25418 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:14:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.203+0000 7f5dadd03b80 1 -- 172.21.15.19:0/1278499065 >> 172.21.15.19:0/1278499065 conn(0x555c1a056f70 msgr2=0x555c1a1cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.203+0000 7f5dadd03b80 1 -- 172.21.15.19:0/1278499065 shutdown_connections 2024-05-09T02:14:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.203+0000 7f5dadd03b80 1 -- 172.21.15.19:0/1278499065 wait complete. 2024-05-09T02:14:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.203+0000 7f5dadd03b80 20 client.25418 trim_cache size 0 max 16384 2024-05-09T02:14:33.209 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:14:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.204+0000 7f5d9f2ed700 1 -- 172.21.15.19:0/2045998303 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5d840b29f0 msgr2=0x7f5d840b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.204+0000 7f5d9f2ed700 1 --2- 172.21.15.19:0/2045998303 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5d840b29f0 0x7f5d840b4f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5d9810d660 tx=0x7f5d88009260 comp rx=0 tx=0).stop 2024-05-09T02:14:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.204+0000 7f5d9f2ed700 1 -- 172.21.15.19:0/2045998303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d9810c830 msgr2=0x7f5d981a21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.204+0000 7f5d9f2ed700 1 --2- 172.21.15.19:0/2045998303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d9810c830 0x7f5d981a21d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e970 tx=0x7f5d9400ed30 comp rx=0 tx=0).stop 2024-05-09T02:14:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.206+0000 7f5d9f2ed700 1 -- 172.21.15.19:0/2045998303 shutdown_connections 2024-05-09T02:14:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.206+0000 7f5d9f2ed700 1 --2- 172.21.15.19:0/2045998303 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5d840b29f0 0x7f5d840b4f00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.206+0000 7f5d9f2ed700 1 --2- 172.21.15.19:0/2045998303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d98110fa0 0x7f5d981a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.206+0000 7f5d9f2ed700 1 --2- 172.21.15.19:0/2045998303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d9810c830 0x7f5d981a21d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.206+0000 7f5d9f2ed700 1 --2- 172.21.15.19:0/2045998303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d98110650 0x7f5d981a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.206+0000 7f5d9f2ed700 1 -- 172.21.15.19:0/2045998303 >> 172.21.15.19:0/2045998303 conn(0x7f5d9806f990 msgr2=0x7f5d9810e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.206+0000 7f5d9f2ed700 1 -- 172.21.15.19:0/2045998303 shutdown_connections 2024-05-09T02:14:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:33.206+0000 7f5d9f2ed700 1 -- 172.21.15.19:0/2045998303 wait complete. 2024-05-09T02:14:33.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:33 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:14:33.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:33 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:14:33.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:33 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:14:33.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:33 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:14:33.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:33 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:14:33.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:33 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:14:33.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:33 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:14:33.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:33 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:14:33.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:33 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:14:34.464 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:34 smithi019 ceph-mon[30151]: pgmap v101: 97 pgs: 97 active+clean; 2.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 729 op/s 2024-05-09T02:14:34.506 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T02:14:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:34 smithi059 ceph-mon[34781]: pgmap v101: 97 pgs: 97 active+clean; 2.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 729 op/s 2024-05-09T02:14:34.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:34 smithi196 ceph-mon[34718]: pgmap v101: 97 pgs: 97 active+clean; 2.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 729 op/s 2024-05-09T02:14:34.842 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:14:36.385 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:36 smithi019 ceph-mon[30151]: pgmap v102: 97 pgs: 97 active+clean; 2.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 956 op/s 2024-05-09T02:14:36.596 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:36 smithi196 ceph-mon[34718]: pgmap v102: 97 pgs: 97 active+clean; 2.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 956 op/s 2024-05-09T02:14:36.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:36 smithi059 ceph-mon[34781]: pgmap v102: 97 pgs: 97 active+clean; 2.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 956 op/s 2024-05-09T02:14:36.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.843+0000 7f8f56168700 1 -- 172.21.15.19:0/1732562646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5011d930 msgr2=0x7f8f5011fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:36.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.843+0000 7f8f56168700 1 --2- 172.21.15.19:0/1732562646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5011d930 0x7f8f5011fd50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8f480098f0 tx=0x7f8f48009c00 comp rx=0 tx=0).stop 2024-05-09T02:14:36.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.843+0000 7f8f56168700 1 -- 172.21.15.19:0/1732562646 shutdown_connections 2024-05-09T02:14:36.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.843+0000 7f8f56168700 1 --2- 172.21.15.19:0/1732562646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5011d930 0x7f8f5011fd50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.843+0000 7f8f56168700 1 --2- 172.21.15.19:0/1732562646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f5010a930 0x7f8f5010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.843+0000 7f8f56168700 1 --2- 172.21.15.19:0/1732562646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5010b280 0x7f8f5011d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.843+0000 7f8f56168700 1 -- 172.21.15.19:0/1732562646 >> 172.21.15.19:0/1732562646 conn(0x7f8f5006f4d0 msgr2=0x7f8f5006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:36.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.844+0000 7f8f56168700 1 -- 172.21.15.19:0/1732562646 shutdown_connections 2024-05-09T02:14:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.844+0000 7f8f56168700 1 -- 172.21.15.19:0/1732562646 wait complete. 2024-05-09T02:14:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.845+0000 7f8f56168700 1 Processor -- start 2024-05-09T02:14:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.845+0000 7f8f56168700 1 -- start start 2024-05-09T02:14:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.846+0000 7f8f56168700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f5010a930 0x7f8f5011bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.846+0000 7f8f56168700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5010b280 0x7f8f50114f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.846+0000 7f8f56168700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5011d930 0x7f8f50115550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.846+0000 7f8f56168700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f50123350 con 0x7f8f5010b280 2024-05-09T02:14:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.846+0000 7f8f56168700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f501231d0 con 0x7f8f5011d930 2024-05-09T02:14:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.846+0000 7f8f56168700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f501234d0 con 0x7f8f5010a930 2024-05-09T02:14:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.846+0000 7f8f4f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f5010a930 0x7f8f5011bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.846+0000 7f8f4f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f5010a930 0x7f8f5011bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50114/0 (socket says 172.21.15.19:50114) 2024-05-09T02:14:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.846+0000 7f8f4f7fe700 1 -- 172.21.15.19:0/3860866526 learned_addr learned my addr 172.21.15.19:0/3860866526 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.846+0000 7f8f4effd700 1 --2- 172.21.15.19:0/3860866526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5010b280 0x7f8f50114f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.846+0000 7f8f4ffff700 1 --2- 172.21.15.19:0/3860866526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5011d930 0x7f8f50115550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.847+0000 7f8f4f7fe700 1 -- 172.21.15.19:0/3860866526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5011d930 msgr2=0x7f8f50115550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.847+0000 7f8f4f7fe700 1 --2- 172.21.15.19:0/3860866526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5011d930 0x7f8f50115550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.847+0000 7f8f4f7fe700 1 -- 172.21.15.19:0/3860866526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5010b280 msgr2=0x7f8f50114f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.847+0000 7f8f4f7fe700 1 --2- 172.21.15.19:0/3860866526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5010b280 0x7f8f50114f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.847+0000 7f8f4f7fe700 1 -- 172.21.15.19:0/3860866526 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f50119900 con 0x7f8f5010a930 2024-05-09T02:14:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.847+0000 7f8f4ffff700 1 --2- 172.21.15.19:0/3860866526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5011d930 0x7f8f50115550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:14:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.848+0000 7f8f4f7fe700 1 --2- 172.21.15.19:0/3860866526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f5010a930 0x7f8f5011bdd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000b840 tx=0x7f8f4000bb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:36.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.851+0000 7f8f4cff9700 1 -- 172.21.15.19:0/3860866526 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000f840 con 0x7f8f5010a930 2024-05-09T02:14:36.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.852+0000 7f8f4cff9700 1 -- 172.21.15.19:0/3860866526 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f40017400 con 0x7f8f5010a930 2024-05-09T02:14:36.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.852+0000 7f8f4cff9700 1 -- 172.21.15.19:0/3860866526 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000e610 con 0x7f8f5010a930 2024-05-09T02:14:36.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.852+0000 7f8f56168700 1 -- 172.21.15.19:0/3860866526 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f50119bf0 con 0x7f8f5010a930 2024-05-09T02:14:36.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.852+0000 7f8f56168700 1 -- 172.21.15.19:0/3860866526 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f5011a130 con 0x7f8f5010a930 2024-05-09T02:14:36.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.858+0000 7f8f64b7eb80 1 Processor -- start 2024-05-09T02:14:36.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.860+0000 7f8f64b7eb80 1 -- start start 2024-05-09T02:14:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.860+0000 7f8f64b7eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c5f71160 0x5577c600a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.861+0000 7f8f64b7eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c5e874a0 0x5577c600aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.861+0000 7f8f64b7eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c60046a0 0x5577c6004ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.861+0000 7f8f64b7eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5577c5f26ff0 con 0x5577c5f71160 2024-05-09T02:14:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.861+0000 7f8f64b7eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5577c5ec57f0 con 0x5577c5e874a0 2024-05-09T02:14:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.861+0000 7f8f64b7eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5577c5f22fb0 con 0x5577c60046a0 2024-05-09T02:14:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.861+0000 7f8f4ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c5f71160 0x5577c600a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.861+0000 7f8f4ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c5f71160 0x5577c600a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37382/0 (socket says 172.21.15.19:37382) 2024-05-09T02:14:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.861+0000 7f8f4ffff700 1 -- 172.21.15.19:0/1306779008 learned_addr learned my addr 172.21.15.19:0/1306779008 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.861+0000 7f8f4cff9700 1 -- 172.21.15.19:0/3860866526 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8f400175b0 con 0x7f8f5010a930 2024-05-09T02:14:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.861+0000 7f8f4f7fe700 1 --2- 172.21.15.19:0/1306779008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c5e874a0 0x5577c600aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.861+0000 7f8f4effd700 1 --2- 172.21.15.19:0/1306779008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c60046a0 0x5577c6004ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.862+0000 7f8f4ffff700 1 -- 172.21.15.19:0/1306779008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c60046a0 msgr2=0x5577c6004ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.862+0000 7f8f4ffff700 1 --2- 172.21.15.19:0/1306779008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c60046a0 0x5577c6004ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.862+0000 7f8f4ffff700 1 -- 172.21.15.19:0/1306779008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c5e874a0 msgr2=0x5577c600aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.862+0000 7f8f4ffff700 1 --2- 172.21.15.19:0/1306779008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c5e874a0 0x5577c600aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.862+0000 7f8f4ffff700 1 -- 172.21.15.19:0/1306779008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577c5ebd370 con 0x5577c5f71160 2024-05-09T02:14:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.862+0000 7f8f4effd700 1 --2- 172.21.15.19:0/1306779008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c60046a0 0x5577c6004ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:14:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.863+0000 7f8f4ffff700 1 --2- 172.21.15.19:0/1306779008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c5f71160 0x5577c600a4f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8f48005f40 tx=0x7f8f48005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63729943638f449e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.865+0000 7f8f4cff9700 1 --2- 172.21.15.19:0/3860866526 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8f3c0b2b80 0x7f8f3c0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.866+0000 7f8f45ffb700 1 -- 172.21.15.19:0/1306779008 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4800f9a0 con 0x5577c5f71160 2024-05-09T02:14:36.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.866+0000 7f8f45ffb700 1 -- 172.21.15.19:0/1306779008 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f480054e0 con 0x5577c5f71160 2024-05-09T02:14:36.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.866+0000 7f8f4cff9700 1 -- 172.21.15.19:0/3860866526 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f8f400e20e0 con 0x7f8f5010a930 2024-05-09T02:14:36.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.866+0000 7f8f45ffb700 1 -- 172.21.15.19:0/1306779008 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4800b560 con 0x5577c5f71160 2024-05-09T02:14:36.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.866+0000 7f8f4f7fe700 1 --2- 172.21.15.19:0/3860866526 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8f3c0b2b80 0x7f8f3c0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.867+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/1306779008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c5f71160 msgr2=0x5577c600a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.867+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/1306779008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c5f71160 0x5577c600a4f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8f48005f40 tx=0x7f8f48005d00 comp rx=0 tx=0).stop 2024-05-09T02:14:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.867+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/1306779008 shutdown_connections 2024-05-09T02:14:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.867+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/1306779008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c60046a0 0x5577c6004ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.867+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/1306779008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c5f71160 0x5577c600a4f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.867+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/1306779008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c5e874a0 0x5577c600aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.867+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/1306779008 >> 172.21.15.19:0/1306779008 conn(0x5577c5ee76a0 msgr2=0x5577c5fff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.868+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/1306779008 shutdown_connections 2024-05-09T02:14:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.868+0000 7f8f4f7fe700 1 --2- 172.21.15.19:0/3860866526 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8f3c0b2b80 0x7f8f3c0b5090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x5577c6000ed0 tx=0x7f8f400e5b90 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.868+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/1306779008 wait complete. 2024-05-09T02:14:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.868+0000 7f8f64b7eb80 1 -- start start 2024-05-09T02:14:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.869+0000 7f8f64b7eb80 1 Processor -- start 2024-05-09T02:14:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.869+0000 7f8f64b7eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c5e874a0 0x5577c600e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.869+0000 7f8f64b7eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c5f71160 0x5577c600e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.869+0000 7f8f64b7eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c60046a0 0x5577c600ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.869+0000 7f8f64b7eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5577c5da2ba0 con 0x5577c60046a0 2024-05-09T02:14:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.869+0000 7f8f64b7eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5577c5ee1150 con 0x5577c5f71160 2024-05-09T02:14:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.869+0000 7f8f64b7eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5577c60111d0 con 0x5577c5e874a0 2024-05-09T02:14:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.869+0000 7f8f64b7eb80 10 client.0 _ll_register_callbacks cb 0x5577c5d986c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.869+0000 7f8f4effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c5f71160 0x5577c600e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.870+0000 7f8f4effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c5f71160 0x5577c600e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:47662/0 (socket says 172.21.15.19:47662) 2024-05-09T02:14:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.870+0000 7f8f4effd700 1 -- 172.21.15.19:0/1764142113 learned_addr learned my addr 172.21.15.19:0/1764142113 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.870+0000 7f8f4effd700 1 -- 172.21.15.19:0/1764142113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c5e874a0 msgr2=0x5577c600e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.870+0000 7f8f4ffff700 1 --2- 172.21.15.19:0/1764142113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c5e874a0 0x5577c600e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.871+0000 7f8f4effd700 1 --2- 172.21.15.19:0/1764142113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c5e874a0 0x5577c600e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.871+0000 7f8f4effd700 1 -- 172.21.15.19:0/1764142113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c60046a0 msgr2=0x5577c600ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.871+0000 7f8f4effd700 1 --2- 172.21.15.19:0/1764142113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c60046a0 0x5577c600ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.871+0000 7f8f4effd700 1 -- 172.21.15.19:0/1764142113 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577c5da2490 con 0x5577c5f71160 2024-05-09T02:14:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.871+0000 7f8f4ffff700 1 --2- 172.21.15.19:0/1764142113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c5e874a0 0x5577c600e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:14:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.871+0000 7f8f4effd700 1 --2- 172.21.15.19:0/1764142113 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c5f71160 0x5577c600e680 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f38012680 tx=0x7f8f38012a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40321698300b9862 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.873+0000 7f8f2effd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:14:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.873+0000 7f8f2effd700 1 -- 172.21.15.19:0/1764142113 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38004030 con 0x5577c5f71160 2024-05-09T02:14:36.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.873+0000 7f8f2effd700 1 -- 172.21.15.19:0/1764142113 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f8f380041d0 con 0x5577c5f71160 2024-05-09T02:14:36.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.874+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/1764142113 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f480054e0 con 0x5577c5f71160 2024-05-09T02:14:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.874+0000 7f8f2effd700 1 -- 172.21.15.19:0/1764142113 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8f380043d0 con 0x5577c5f71160 2024-05-09T02:14:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.874+0000 7f8f64b7eb80 10 client.25430 fetch_fsmap learned FSMap version 35 2024-05-09T02:14:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.874+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/1764142113 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5577c601bba0 con 0x5577c5f71160 2024-05-09T02:14:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.875+0000 7f8f2effd700 1 -- 172.21.15.19:0/1764142113 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f480054e0 con 0x5577c5f71160 2024-05-09T02:14:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.875+0000 7f8f2effd700 1 -- 172.21.15.19:0/1764142113 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f8f3800f770 con 0x5577c5f71160 2024-05-09T02:14:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.875+0000 7f8f64b7eb80 10 client.25430 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T02:14:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.875+0000 7f8f64b7eb80 10 client.25430 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:14:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.875+0000 7f8f64b7eb80 20 client.25430 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.876+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/1764142113 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5577c601e130 0x5577c6020550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.876+0000 7f8f64b7eb80 4 client.25430 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:14:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.876+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/1764142113 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577c5f349d0 con 0x5577c601e130 2024-05-09T02:14:36.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.878+0000 7f8f4ffff700 1 --2- 172.21.15.19:0/1764142113 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5577c601e130 0x5577c6020550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:36.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.879+0000 7f8f4ffff700 1 --2- 172.21.15.19:0/1764142113 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5577c601e130 0x5577c6020550 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6f1c4fa99b1e8c1 server_cookie=35fd388aaf2233dc in_seq=0 out_seq=0 2024-05-09T02:14:36.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.879+0000 7f8f2effd700 10 client.25430 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:14:36.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.879+0000 7f8f2effd700 1 -- 172.21.15.19:0/1764142113 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5577c5f349d0 con 0x5577c601e130 2024-05-09T02:14:36.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.879+0000 7f8f2effd700 10 client.25430 handle_command_reply: tid=0 2024-05-09T02:14:36.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.880+0000 7f8f64b7eb80 1 client.25430 shutdown 2024-05-09T02:14:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.880+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/1764142113 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c5f71160 msgr2=0x5577c600e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.880+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/1764142113 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c5f71160 0x5577c600e680 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f38012680 tx=0x7f8f38012a40 comp rx=0 tx=0).stop 2024-05-09T02:14:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.880+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/1764142113 shutdown_connections 2024-05-09T02:14:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.880+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/1764142113 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5577c601e130 0x5577c6020550 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.880+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/1764142113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c60046a0 0x5577c600ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.880+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/1764142113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c5e874a0 0x5577c600e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.880+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/1764142113 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c5f71160 0x5577c600e680 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f38012680 tx=0x7f8f38012a40 comp rx=0 tx=0).stop 2024-05-09T02:14:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.880+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/1764142113 >> 172.21.15.19:0/1764142113 conn(0x5577c5ee76a0 msgr2=0x5577c5fff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.880+0000 7f8f2effd700 0 client.25430 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:14:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.881+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/1764142113 shutdown_connections 2024-05-09T02:14:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.882+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/1764142113 wait complete. 2024-05-09T02:14:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.882+0000 7f8f64b7eb80 20 client.25430 trim_cache size 0 max 16384 2024-05-09T02:14:36.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.909+0000 7f8f64b7eb80 1 Processor -- start 2024-05-09T02:14:36.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.909+0000 7f8f64b7eb80 1 -- start start 2024-05-09T02:14:36.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.910+0000 7f8f64b7eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c5e874a0 0x5577c600bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.910+0000 7f8f64b7eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c60046a0 0x5577c600ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.910+0000 7f8f4ffff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c5e874a0 0x5577c600bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:36.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.910+0000 7f8f4ffff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c5e874a0 0x5577c600bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50144/0 (socket says 172.21.15.19:50144) 2024-05-09T02:14:36.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.910+0000 7f8f4ffff700 1 -- 172.21.15.19:0/2389542015 learned_addr learned my addr 172.21.15.19:0/2389542015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:36.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.910+0000 7f8f4effd700 1 --2- 172.21.15.19:0/2389542015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c60046a0 0x5577c600ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:36.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.911+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/2389542015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c6008510 0x5577c6019f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.911+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/2389542015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3000d6e0 con 0x5577c6008510 2024-05-09T02:14:36.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.911+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/2389542015 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3000f280 con 0x5577c60046a0 2024-05-09T02:14:36.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.911+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/2389542015 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3000ddb0 con 0x5577c5e874a0 2024-05-09T02:14:36.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.911+0000 7f8f4ffff700 1 -- 172.21.15.19:0/2389542015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c60046a0 msgr2=0x5577c600ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:36.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.911+0000 7f8f4ffff700 1 --2- 172.21.15.19:0/2389542015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c60046a0 0x5577c600ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.911+0000 7f8f4ffff700 1 -- 172.21.15.19:0/2389542015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c6008510 msgr2=0x5577c6019f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:36.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.911+0000 7f8f4ffff700 1 --2- 172.21.15.19:0/2389542015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c6008510 0x5577c6019f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.912+0000 7f8f4ffff700 1 -- 172.21.15.19:0/2389542015 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577c600ef40 con 0x5577c5e874a0 2024-05-09T02:14:36.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.912+0000 7f8f4ffff700 1 --2- 172.21.15.19:0/2389542015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c5e874a0 0x5577c600bd70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5577c600d9c0 tx=0x7f8f48005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaa84eeb750d1569 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:36.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.913+0000 7f8f457fa700 1 -- 172.21.15.19:0/2389542015 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4803a070 con 0x5577c5e874a0 2024-05-09T02:14:36.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.913+0000 7f8f457fa700 1 -- 172.21.15.19:0/2389542015 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f48035050 con 0x5577c5e874a0 2024-05-09T02:14:36.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.913+0000 7f8f457fa700 1 -- 172.21.15.19:0/2389542015 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48003ae0 con 0x5577c5e874a0 2024-05-09T02:14:36.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.914+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/2389542015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c5e874a0 msgr2=0x5577c600bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:36.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.914+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/2389542015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c5e874a0 0x5577c600bd70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5577c600d9c0 tx=0x7f8f48005d90 comp rx=0 tx=0).stop 2024-05-09T02:14:36.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.914+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/2389542015 shutdown_connections 2024-05-09T02:14:36.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.914+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/2389542015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c6008510 0x5577c6019f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.914+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/2389542015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c5e874a0 0x5577c600bd70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.914+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/2389542015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c60046a0 0x5577c600ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.914+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/2389542015 >> 172.21.15.19:0/2389542015 conn(0x5577c5ee76a0 msgr2=0x5577c6007e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:36.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.915+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/2389542015 shutdown_connections 2024-05-09T02:14:36.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.915+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/2389542015 wait complete. 2024-05-09T02:14:36.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.916+0000 7f8f64b7eb80 1 -- start start 2024-05-09T02:14:36.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.916+0000 7f8f64b7eb80 1 Processor -- start 2024-05-09T02:14:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f64b7eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c5e874a0 0x5577c60167c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f64b7eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c60046a0 0x5577c60252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f64b7eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c6008510 0x5577c6025800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f64b7eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3000eb50 con 0x5577c5e874a0 2024-05-09T02:14:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f64b7eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3000e480 con 0x5577c6008510 2024-05-09T02:14:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f64b7eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3000b680 con 0x5577c60046a0 2024-05-09T02:14:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f64b7eb80 10 client.0 _ll_register_callbacks cb 0x5577c5d986c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f4effd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c60046a0 0x5577c60252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f4effd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c60046a0 0x5577c60252c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50152/0 (socket says 172.21.15.19:50152) 2024-05-09T02:14:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f4effd700 1 -- 172.21.15.19:0/3359028240 learned_addr learned my addr 172.21.15.19:0/3359028240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f4ffff700 1 --2- 172.21.15.19:0/3359028240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c6008510 0x5577c6025800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:36.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f4effd700 1 -- 172.21.15.19:0/3359028240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c6008510 msgr2=0x5577c6025800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:36.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f4effd700 1 --2- 172.21.15.19:0/3359028240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c6008510 0x5577c6025800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f4effd700 1 -- 172.21.15.19:0/3359028240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c5e874a0 msgr2=0x5577c60167c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:36.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f4effd700 1 --2- 172.21.15.19:0/3359028240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c5e874a0 0x5577c60167c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.917+0000 7f8f4effd700 1 -- 172.21.15.19:0/3359028240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577c6025ee0 con 0x5577c60046a0 2024-05-09T02:14:36.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.918+0000 7f8f4effd700 1 --2- 172.21.15.19:0/3359028240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c60046a0 0x5577c60252c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8f38012420 tx=0x7f8f3800fde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c0b3152957a7be2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:36.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.920+0000 7f8f2effd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:14:36.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.920+0000 7f8f2effd700 1 -- 172.21.15.19:0/3359028240 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3802c070 con 0x5577c60046a0 2024-05-09T02:14:36.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.920+0000 7f8f2effd700 1 -- 172.21.15.19:0/3359028240 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f8f3800f7d0 con 0x5577c60046a0 2024-05-09T02:14:36.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.920+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/3359028240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f48035050 con 0x5577c60046a0 2024-05-09T02:14:36.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.921+0000 7f8f2effd700 1 -- 172.21.15.19:0/3359028240 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8f38005040 con 0x5577c60046a0 2024-05-09T02:14:36.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.921+0000 7f8f64b7eb80 10 client.25495 fetch_fsmap learned FSMap version 35 2024-05-09T02:14:36.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.921+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/3359028240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5577c6088790 con 0x5577c60046a0 2024-05-09T02:14:36.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.921+0000 7f8f2effd700 1 -- 172.21.15.19:0/3359028240 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f48035050 con 0x5577c60046a0 2024-05-09T02:14:36.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.922+0000 7f8f2effd700 1 -- 172.21.15.19:0/3359028240 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f8f38003c60 con 0x5577c60046a0 2024-05-09T02:14:36.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.922+0000 7f8f64b7eb80 10 client.25495 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T02:14:36.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.922+0000 7f8f64b7eb80 10 client.25495 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:14:36.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.922+0000 7f8f64b7eb80 20 client.25495 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:36.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.922+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/3359028240 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5577c6088e20 0x5577c608b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:36.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.922+0000 7f8f64b7eb80 4 client.25495 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:14:36.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.922+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/3359028240 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577c5f2ea70 con 0x5577c6088e20 2024-05-09T02:14:36.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.922+0000 7f8f4ffff700 1 --2- 172.21.15.19:0/3359028240 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5577c6088e20 0x5577c608b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:36.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.923+0000 7f8f2effd700 10 client.25495 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:14:36.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.923+0000 7f8f4ffff700 1 --2- 172.21.15.19:0/3359028240 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5577c6088e20 0x5577c608b240 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=558b5d7a8b201341 server_cookie=d924978c7e235e7 in_seq=0 out_seq=0 2024-05-09T02:14:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.923+0000 7f8f2effd700 1 -- 172.21.15.19:0/3359028240 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577c5f2ea70 con 0x5577c6088e20 2024-05-09T02:14:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.923+0000 7f8f2effd700 10 client.25495 handle_command_reply: tid=0 2024-05-09T02:14:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.923+0000 7f8f64b7eb80 1 client.25495 shutdown 2024-05-09T02:14:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.924+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/3359028240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c60046a0 msgr2=0x5577c60252c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.924+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/3359028240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c60046a0 0x5577c60252c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8f38012420 tx=0x7f8f3800fde0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.924+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/3359028240 shutdown_connections 2024-05-09T02:14:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.924+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/3359028240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577c6008510 0x5577c6025800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.924+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/3359028240 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x5577c6088e20 0x5577c608b240 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.924+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/3359028240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577c5e874a0 0x5577c60167c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.924+0000 7f8f64b7eb80 1 --2- 172.21.15.19:0/3359028240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577c60046a0 0x5577c60252c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.924+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/3359028240 >> 172.21.15.19:0/3359028240 conn(0x5577c5ee76a0 msgr2=0x5577c6007e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.924+0000 7f8f2effd700 0 client.25495 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:14:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.924+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/3359028240 shutdown_connections 2024-05-09T02:14:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.924+0000 7f8f64b7eb80 1 -- 172.21.15.19:0/3359028240 wait complete. 2024-05-09T02:14:36.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.925+0000 7f8f64b7eb80 20 client.25495 trim_cache size 0 max 16384 2024-05-09T02:14:36.930 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:14:36.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.926+0000 7f8f56168700 1 -- 172.21.15.19:0/3860866526 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8f3c0b2b80 msgr2=0x7f8f3c0b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:36.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.926+0000 7f8f56168700 1 --2- 172.21.15.19:0/3860866526 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8f3c0b2b80 0x7f8f3c0b5090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x5577c6000ed0 tx=0x7f8f400e5b90 comp rx=0 tx=0).stop 2024-05-09T02:14:36.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.926+0000 7f8f56168700 1 -- 172.21.15.19:0/3860866526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f5010a930 msgr2=0x7f8f5011bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:36.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.926+0000 7f8f56168700 1 --2- 172.21.15.19:0/3860866526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f5010a930 0x7f8f5011bdd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000b840 tx=0x7f8f4000bb50 comp rx=0 tx=0).stop 2024-05-09T02:14:36.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.926+0000 7f8f56168700 1 -- 172.21.15.19:0/3860866526 shutdown_connections 2024-05-09T02:14:36.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.926+0000 7f8f56168700 1 --2- 172.21.15.19:0/3860866526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5011d930 0x7f8f50115550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.926+0000 7f8f56168700 1 --2- 172.21.15.19:0/3860866526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f5010a930 0x7f8f5011bdd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.926+0000 7f8f56168700 1 --2- 172.21.15.19:0/3860866526 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8f3c0b2b80 0x7f8f3c0b5090 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.926+0000 7f8f56168700 1 --2- 172.21.15.19:0/3860866526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5010b280 0x7f8f50114f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:36.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.926+0000 7f8f56168700 1 -- 172.21.15.19:0/3860866526 >> 172.21.15.19:0/3860866526 conn(0x7f8f5006f4d0 msgr2=0x7f8f50072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:36.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.927+0000 7f8f56168700 1 -- 172.21.15.19:0/3860866526 shutdown_connections 2024-05-09T02:14:36.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:36.927+0000 7f8f56168700 1 -- 172.21.15.19:0/3860866526 wait complete. 2024-05-09T02:14:37.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 175.0 s, 2832.5 tps, lat 35.316 ms stddev 36.876 2024-05-09T02:14:37.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:37 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:14:37.719 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:37 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:14:37.766 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T02:14:37.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:37 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:14:38.169 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:14:38.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:38 smithi019 ceph-mon[30151]: pgmap v103: 97 pgs: 97 active+clean; 2.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 603 op/s 2024-05-09T02:14:38.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:38 smithi059 ceph-mon[34781]: pgmap v103: 97 pgs: 97 active+clean; 2.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 603 op/s 2024-05-09T02:14:38.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:38 smithi196 ceph-mon[34718]: pgmap v103: 97 pgs: 97 active+clean; 2.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 603 op/s 2024-05-09T02:14:40.403 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:40 smithi019 ceph-mon[30151]: pgmap v104: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.9 MiB/s wr, 896 op/s 2024-05-09T02:14:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.472+0000 7f8995048700 1 -- 172.21.15.19:0/3668430687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f899010b2b0 msgr2=0x7f8990117830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.472+0000 7f8995048700 1 --2- 172.21.15.19:0/3668430687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f899010b2b0 0x7f8990117830 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f898800b0a0 tx=0x7f898800b3b0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.472+0000 7f8995048700 1 -- 172.21.15.19:0/3668430687 shutdown_connections 2024-05-09T02:14:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.472+0000 7f8995048700 1 --2- 172.21.15.19:0/3668430687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f899010b2b0 0x7f8990117830 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.472+0000 7f8995048700 1 --2- 172.21.15.19:0/3668430687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8990110990 0x7f8990110da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.472+0000 7f8995048700 1 --2- 172.21.15.19:0/3668430687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f899010a960 0x7f899010ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.472+0000 7f8995048700 1 -- 172.21.15.19:0/3668430687 >> 172.21.15.19:0/3668430687 conn(0x7f8990076d30 msgr2=0x7f8990077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.478+0000 7f8995048700 1 -- 172.21.15.19:0/3668430687 shutdown_connections 2024-05-09T02:14:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.479+0000 7f8995048700 1 -- 172.21.15.19:0/3668430687 wait complete. 2024-05-09T02:14:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.480+0000 7f8995048700 1 Processor -- start 2024-05-09T02:14:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.480+0000 7f8995048700 1 -- start start 2024-05-09T02:14:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.481+0000 7f8995048700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899010b2b0 0x7f89901a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.481+0000 7f8995048700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8990110990 0x7f89901a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.481+0000 7f8995048700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89901a5e00 0x7f89901a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.481+0000 7f8995048700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f899011ac00 con 0x7f899010b2b0 2024-05-09T02:14:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.481+0000 7f8995048700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f899011aa80 con 0x7f89901a5e00 2024-05-09T02:14:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.481+0000 7f8995048700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f899011ad80 con 0x7f8990110990 2024-05-09T02:14:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.481+0000 7f898ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899010b2b0 0x7f89901a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.481+0000 7f898ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899010b2b0 0x7f89901a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37418/0 (socket says 172.21.15.19:37418) 2024-05-09T02:14:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.481+0000 7f898ed9d700 1 -- 172.21.15.19:0/1909378281 learned_addr learned my addr 172.21.15.19:0/1909378281 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.481+0000 7f898f59e700 1 --2- 172.21.15.19:0/1909378281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89901a5e00 0x7f89901a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.481+0000 7f898e59c700 1 --2- 172.21.15.19:0/1909378281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8990110990 0x7f89901a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.481+0000 7f898ed9d700 1 -- 172.21.15.19:0/1909378281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8990110990 msgr2=0x7f89901a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.481+0000 7f898ed9d700 1 --2- 172.21.15.19:0/1909378281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8990110990 0x7f89901a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.482+0000 7f898ed9d700 1 -- 172.21.15.19:0/1909378281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89901a5e00 msgr2=0x7f89901a6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.482+0000 7f898ed9d700 1 --2- 172.21.15.19:0/1909378281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89901a5e00 0x7f89901a6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.482+0000 7f898ed9d700 1 -- 172.21.15.19:0/1909378281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89901a68b0 con 0x7f899010b2b0 2024-05-09T02:14:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.482+0000 7f898ed9d700 1 --2- 172.21.15.19:0/1909378281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899010b2b0 0x7f89901a15d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f898000b550 tx=0x7f898000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.482+0000 7f8977fff700 1 -- 172.21.15.19:0/1909378281 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980015020 con 0x7f899010b2b0 2024-05-09T02:14:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.482+0000 7f8995048700 1 -- 172.21.15.19:0/1909378281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89901a6ba0 con 0x7f899010b2b0 2024-05-09T02:14:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.482+0000 7f8995048700 1 -- 172.21.15.19:0/1909378281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89901ae390 con 0x7f899010b2b0 2024-05-09T02:14:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.482+0000 7f8977fff700 1 -- 172.21.15.19:0/1909378281 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8980011ce0 con 0x7f899010b2b0 2024-05-09T02:14:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.482+0000 7f8977fff700 1 -- 172.21.15.19:0/1909378281 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980010cc0 con 0x7f899010b2b0 2024-05-09T02:14:40.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.487+0000 7f89a3a5eb80 1 Processor -- start 2024-05-09T02:14:40.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.487+0000 7f89a3a5eb80 1 -- start start 2024-05-09T02:14:40.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.487+0000 7f8977fff700 1 -- 172.21.15.19:0/1909378281 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8980018020 con 0x7f899010b2b0 2024-05-09T02:14:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f89a3a5eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d8828d00 0x55f4d8931240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f89a3a5eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d87ae4a0 0x55f4d8931780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f89a3a5eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d892b400 0x55f4d892b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f89a3a5eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f4d884e0a0 con 0x55f4d87ae4a0 2024-05-09T02:14:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f89a3a5eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f4d87f39a0 con 0x55f4d8828d00 2024-05-09T02:14:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f89a3a5eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f4d884a060 con 0x55f4d892b400 2024-05-09T02:14:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f898ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d87ae4a0 0x55f4d8931780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f898f59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d8828d00 0x55f4d8931240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f898f59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d8828d00 0x55f4d8931240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:47702/0 (socket says 172.21.15.19:47702) 2024-05-09T02:14:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f898f59e700 1 -- 172.21.15.19:0/1300454353 learned_addr learned my addr 172.21.15.19:0/1300454353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f898e59c700 1 --2- 172.21.15.19:0/1300454353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d892b400 0x55f4d892b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f898f59e700 1 -- 172.21.15.19:0/1300454353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d892b400 msgr2=0x55f4d892b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f898f59e700 1 --2- 172.21.15.19:0/1300454353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d892b400 0x55f4d892b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f898f59e700 1 -- 172.21.15.19:0/1300454353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d87ae4a0 msgr2=0x55f4d8931780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:40.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f898f59e700 1 --2- 172.21.15.19:0/1300454353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d87ae4a0 0x55f4d8931780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.488+0000 7f898f59e700 1 -- 172.21.15.19:0/1300454353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4d87e4500 con 0x55f4d8828d00 2024-05-09T02:14:40.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.489+0000 7f8977fff700 1 --2- 172.21.15.19:0/1909378281 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f89780b2aa0 0x7f89780b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.489+0000 7f898ed9d700 1 --2- 172.21.15.19:0/1909378281 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f89780b2aa0 0x7f89780b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.489+0000 7f898f59e700 1 --2- 172.21.15.19:0/1300454353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d8828d00 0x55f4d8931240 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8988015000 tx=0x7f89880093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88eea59d33bd8e62 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:40.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.489+0000 7f8977fff700 1 -- 172.21.15.19:0/1909378281 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f89800e1ba0 con 0x7f899010b2b0 2024-05-09T02:14:40.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.490+0000 7f89757fa700 1 -- 172.21.15.19:0/1300454353 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988003c00 con 0x55f4d8828d00 2024-05-09T02:14:40.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.490+0000 7f89757fa700 1 -- 172.21.15.19:0/1300454353 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8988003da0 con 0x55f4d8828d00 2024-05-09T02:14:40.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.490+0000 7f89757fa700 1 -- 172.21.15.19:0/1300454353 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89880053c0 con 0x55f4d8828d00 2024-05-09T02:14:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.491+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/1300454353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d8828d00 msgr2=0x55f4d8931240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.491+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/1300454353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d8828d00 0x55f4d8931240 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8988015000 tx=0x7f89880093f0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.491+0000 7f898ed9d700 1 --2- 172.21.15.19:0/1909378281 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f89780b2aa0 0x7f89780b4fb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8980011cb0 tx=0x7f89800e55f0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.491+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/1300454353 shutdown_connections 2024-05-09T02:14:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.491+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/1300454353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d892b400 0x55f4d892b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.491+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/1300454353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d8828d00 0x55f4d8931240 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.491+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/1300454353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d87ae4a0 0x55f4d8931780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.491+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/1300454353 >> 172.21.15.19:0/1300454353 conn(0x55f4d87b1f70 msgr2=0x55f4d8926220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.491+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/1300454353 shutdown_connections 2024-05-09T02:14:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.491+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/1300454353 wait complete. 2024-05-09T02:14:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.491+0000 7f89a3a5eb80 1 -- start start 2024-05-09T02:14:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.492+0000 7f89a3a5eb80 1 Processor -- start 2024-05-09T02:14:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.492+0000 7f89a3a5eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d87ae4a0 0x55f4d8935060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.492+0000 7f89a3a5eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d8828d00 0x55f4d89355a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.492+0000 7f89a3a5eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d892b400 0x55f4d8935ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.492+0000 7f89a3a5eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f4d892c100 con 0x55f4d87ae4a0 2024-05-09T02:14:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.492+0000 7f89a3a5eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f4d87e8b20 con 0x55f4d892b400 2024-05-09T02:14:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.492+0000 7f89a3a5eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f4d8941200 con 0x55f4d8828d00 2024-05-09T02:14:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.492+0000 7f89a3a5eb80 10 client.0 _ll_register_callbacks cb 0x55f4d86bf6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.492+0000 7f898e59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d8828d00 0x55f4d89355a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.492+0000 7f898f59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d87ae4a0 0x55f4d8935060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.492+0000 7f898e59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d8828d00 0x55f4d89355a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50200/0 (socket says 172.21.15.19:50200) 2024-05-09T02:14:40.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.492+0000 7f898e59c700 1 -- 172.21.15.19:0/2508261561 learned_addr learned my addr 172.21.15.19:0/2508261561 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.493+0000 7f898f59e700 1 --2- 172.21.15.19:0/2508261561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d892b400 0x55f4d8935ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.493+0000 7f898f59e700 1 -- 172.21.15.19:0/2508261561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d8828d00 msgr2=0x55f4d89355a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.493+0000 7f898f59e700 1 --2- 172.21.15.19:0/2508261561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d8828d00 0x55f4d89355a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.493+0000 7f898f59e700 1 -- 172.21.15.19:0/2508261561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d87ae4a0 msgr2=0x55f4d8935060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.493+0000 7f898f59e700 1 --2- 172.21.15.19:0/2508261561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d87ae4a0 0x55f4d8935060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.493+0000 7f898f59e700 1 -- 172.21.15.19:0/2508261561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4d86c9490 con 0x55f4d892b400 2024-05-09T02:14:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.493+0000 7f898f59e700 1 --2- 172.21.15.19:0/2508261561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d892b400 0x55f4d8935ae0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55f4d89348e0 tx=0x7f898800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e63bcf60a86c11b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.493+0000 7f8972ff5700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:14:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.493+0000 7f8972ff5700 1 -- 172.21.15.19:0/2508261561 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988031420 con 0x55f4d892b400 2024-05-09T02:14:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.494+0000 7f8972ff5700 1 -- 172.21.15.19:0/2508261561 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f898801c410 con 0x55f4d892b400 2024-05-09T02:14:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.494+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/2508261561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8988003da0 con 0x55f4d892b400 2024-05-09T02:14:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.494+0000 7f8972ff5700 1 -- 172.21.15.19:0/2508261561 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f898801c820 con 0x55f4d892b400 2024-05-09T02:14:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.494+0000 7f89a3a5eb80 10 client.25448 fetch_fsmap learned FSMap version 35 2024-05-09T02:14:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.494+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/2508261561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f4d8942b70 con 0x55f4d892b400 2024-05-09T02:14:40.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.494+0000 7f8972ff5700 1 -- 172.21.15.19:0/2508261561 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89880123f0 con 0x55f4d892b400 2024-05-09T02:14:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.494+0000 7f8972ff5700 1 -- 172.21.15.19:0/2508261561 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f8988011070 con 0x55f4d892b400 2024-05-09T02:14:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.494+0000 7f89a3a5eb80 10 client.25448 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T02:14:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.494+0000 7f89a3a5eb80 10 client.25448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:14:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.494+0000 7f89a3a5eb80 20 client.25448 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.494+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/2508261561 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55f4d8945040 0x55f4d8947460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.494+0000 7f89a3a5eb80 4 client.25448 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:14:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.494+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/2508261561 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8988003da0 con 0x55f4d8945040 2024-05-09T02:14:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.495+0000 7f898e59c700 1 --2- 172.21.15.19:0/2508261561 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55f4d8945040 0x55f4d8947460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.495+0000 7f8972ff5700 10 client.25448 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.495+0000 7f898e59c700 1 --2- 172.21.15.19:0/2508261561 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55f4d8945040 0x55f4d8947460 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b374c3c6a00c59f server_cookie=17d1306bee3f9d4d in_seq=0 out_seq=0 2024-05-09T02:14:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.496+0000 7f8972ff5700 1 -- 172.21.15.19:0/2508261561 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f8988003da0 con 0x55f4d8945040 2024-05-09T02:14:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.496+0000 7f8972ff5700 10 client.25448 handle_command_reply: tid=0 2024-05-09T02:14:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.496+0000 7f89a3a5eb80 1 client.25448 shutdown 2024-05-09T02:14:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.497+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/2508261561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d892b400 msgr2=0x55f4d8935ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.497+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/2508261561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d892b400 0x55f4d8935ae0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55f4d89348e0 tx=0x7f898800b3b0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.497+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/2508261561 shutdown_connections 2024-05-09T02:14:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.497+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/2508261561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d892b400 0x55f4d8935ae0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.497+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/2508261561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d87ae4a0 0x55f4d8935060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.497+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/2508261561 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55f4d8945040 0x55f4d8947460 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.497+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/2508261561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d8828d00 0x55f4d89355a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.497+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/2508261561 >> 172.21.15.19:0/2508261561 conn(0x55f4d87b1f70 msgr2=0x55f4d891fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:40.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.497+0000 7f8972ff5700 0 client.25448 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:40.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.497+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/2508261561 shutdown_connections 2024-05-09T02:14:40.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.497+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/2508261561 wait complete. 2024-05-09T02:14:40.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.497+0000 7f89a3a5eb80 20 client.25448 trim_cache size 0 max 16384 2024-05-09T02:14:40.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.521+0000 7f89a3a5eb80 1 Processor -- start 2024-05-09T02:14:40.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.521+0000 7f89a3a5eb80 1 -- start start 2024-05-09T02:14:40.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.521+0000 7f89a3a5eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d87ae4a0 0x55f4d89343a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.521+0000 7f89a3a5eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d8828d00 0x55f4d8920700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.521+0000 7f89a3a5eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d892b400 0x55f4d8932250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.521+0000 7f89a3a5eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f897c00d560 con 0x55f4d8828d00 2024-05-09T02:14:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.521+0000 7f89a3a5eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f897c00f100 con 0x55f4d87ae4a0 2024-05-09T02:14:40.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.521+0000 7f89a3a5eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f897c00dc30 con 0x55f4d892b400 2024-05-09T02:14:40.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.522+0000 7f898e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d8828d00 0x55f4d8920700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.522+0000 7f898f59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d87ae4a0 0x55f4d89343a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.522+0000 7f898e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d8828d00 0x55f4d8920700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37434/0 (socket says 172.21.15.19:37434) 2024-05-09T02:14:40.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.522+0000 7f898e59c700 1 -- 172.21.15.19:0/1464596790 learned_addr learned my addr 172.21.15.19:0/1464596790 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:40.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.522+0000 7f898f59e700 1 --2- 172.21.15.19:0/1464596790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d892b400 0x55f4d8932250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.522+0000 7f898e59c700 1 -- 172.21.15.19:0/1464596790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d892b400 msgr2=0x55f4d8932250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:40.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.522+0000 7f898e59c700 1 --2- 172.21.15.19:0/1464596790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d892b400 0x55f4d8932250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.522+0000 7f898e59c700 1 -- 172.21.15.19:0/1464596790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d87ae4a0 msgr2=0x55f4d89343a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:40.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.522+0000 7f898e59c700 1 --2- 172.21.15.19:0/1464596790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d87ae4a0 0x55f4d89343a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.522+0000 7f898e59c700 1 -- 172.21.15.19:0/1464596790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4d8937900 con 0x55f4d8828d00 2024-05-09T02:14:40.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.522+0000 7f898f59e700 1 --2- 172.21.15.19:0/1464596790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d87ae4a0 0x55f4d89343a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:14:40.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.523+0000 7f898e59c700 1 --2- 172.21.15.19:0/1464596790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d8828d00 0x55f4d8920700 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8990075b80 tx=0x7f8984012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e295e911359ccc78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:40.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.523+0000 7f8974ff9700 1 -- 172.21.15.19:0/1464596790 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984010d50 con 0x55f4d8828d00 2024-05-09T02:14:40.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.523+0000 7f8974ff9700 1 -- 172.21.15.19:0/1464596790 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8984024070 con 0x55f4d8828d00 2024-05-09T02:14:40.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.523+0000 7f8974ff9700 1 -- 172.21.15.19:0/1464596790 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89840200f0 con 0x55f4d8828d00 2024-05-09T02:14:40.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.524+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/1464596790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d8828d00 msgr2=0x55f4d8920700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:40.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.524+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/1464596790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d8828d00 0x55f4d8920700 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8990075b80 tx=0x7f8984012040 comp rx=0 tx=0).stop 2024-05-09T02:14:40.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.526+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/1464596790 shutdown_connections 2024-05-09T02:14:40.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.526+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/1464596790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d892b400 0x55f4d8932250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.526+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/1464596790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d87ae4a0 0x55f4d89343a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.526+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/1464596790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d8828d00 0x55f4d8920700 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.526+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/1464596790 >> 172.21.15.19:0/1464596790 conn(0x55f4d87b1f70 msgr2=0x55f4d891fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:40.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.526+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/1464596790 shutdown_connections 2024-05-09T02:14:40.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.526+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/1464596790 wait complete. 2024-05-09T02:14:40.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.527+0000 7f89a3a5eb80 1 -- start start 2024-05-09T02:14:40.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.527+0000 7f89a3a5eb80 1 Processor -- start 2024-05-09T02:14:40.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.527+0000 7f89a3a5eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d87ae4a0 0x55f4d8947880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.527+0000 7f89a3a5eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d8828d00 0x55f4d8947dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.527+0000 7f89a3a5eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d892b400 0x55f4d89a7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.527+0000 7f89a3a5eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f897c00e9d0 con 0x55f4d892b400 2024-05-09T02:14:40.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.527+0000 7f89a3a5eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f897c00e300 con 0x55f4d87ae4a0 2024-05-09T02:14:40.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.527+0000 7f89a3a5eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f897c00b500 con 0x55f4d8828d00 2024-05-09T02:14:40.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.527+0000 7f89a3a5eb80 10 client.0 _ll_register_callbacks cb 0x55f4d86bf6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:40.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.527+0000 7f898f59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d87ae4a0 0x55f4d8947880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.527+0000 7f898f59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d87ae4a0 0x55f4d8947880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:47716/0 (socket says 172.21.15.19:47716) 2024-05-09T02:14:40.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.527+0000 7f898f59e700 1 -- 172.21.15.19:0/3946448307 learned_addr learned my addr 172.21.15.19:0/3946448307 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:40.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.527+0000 7f898f59e700 1 --2- 172.21.15.19:0/3946448307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d892b400 0x55f4d89a7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.527+0000 7f898e59c700 1 --2- 172.21.15.19:0/3946448307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d8828d00 0x55f4d8947dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.528+0000 7f898f59e700 1 -- 172.21.15.19:0/3946448307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d8828d00 msgr2=0x55f4d8947dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:40.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.528+0000 7f898f59e700 1 --2- 172.21.15.19:0/3946448307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d8828d00 0x55f4d8947dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.528+0000 7f898f59e700 1 -- 172.21.15.19:0/3946448307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d892b400 msgr2=0x55f4d89a7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:40.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.528+0000 7f898f59e700 1 --2- 172.21.15.19:0/3946448307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d892b400 0x55f4d89a7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.528+0000 7f898f59e700 1 -- 172.21.15.19:0/3946448307 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4d89a7ac0 con 0x55f4d87ae4a0 2024-05-09T02:14:40.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.528+0000 7f898f59e700 1 --2- 172.21.15.19:0/3946448307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d87ae4a0 0x55f4d8947880 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f89880092a0 tx=0x7f8988007a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef76123e237ac7af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:40.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.528+0000 7f8972ff5700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:14:40.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.528+0000 7f8972ff5700 1 -- 172.21.15.19:0/3946448307 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898801c410 con 0x55f4d87ae4a0 2024-05-09T02:14:40.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.528+0000 7f8972ff5700 1 -- 172.21.15.19:0/3946448307 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f8988002030 con 0x55f4d87ae4a0 2024-05-09T02:14:40.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.528+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/3946448307 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8984024070 con 0x55f4d87ae4a0 2024-05-09T02:14:40.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.529+0000 7f8972ff5700 1 -- 172.21.15.19:0/3946448307 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f89880093f0 con 0x55f4d87ae4a0 2024-05-09T02:14:40.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.529+0000 7f89a3a5eb80 10 client.25454 fetch_fsmap learned FSMap version 35 2024-05-09T02:14:40.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.529+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/3946448307 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f4d89aeda0 con 0x55f4d87ae4a0 2024-05-09T02:14:40.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.529+0000 7f8972ff5700 1 -- 172.21.15.19:0/3946448307 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8988024db0 con 0x55f4d87ae4a0 2024-05-09T02:14:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.529+0000 7f8972ff5700 1 -- 172.21.15.19:0/3946448307 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f8988011040 con 0x55f4d87ae4a0 2024-05-09T02:14:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.529+0000 7f89a3a5eb80 10 client.25454 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T02:14:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.529+0000 7f89a3a5eb80 10 client.25454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:14:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.529+0000 7f89a3a5eb80 20 client.25454 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.529+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/3946448307 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55f4d89af330 0x55f4d89b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.529+0000 7f89a3a5eb80 4 client.25454 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:14:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.529+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/3946448307 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8984024070 con 0x55f4d89af330 2024-05-09T02:14:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.530+0000 7f898e59c700 1 --2- 172.21.15.19:0/3946448307 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55f4d89af330 0x55f4d89b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.530+0000 7f8972ff5700 10 client.25454 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.531+0000 7f898e59c700 1 --2- 172.21.15.19:0/3946448307 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55f4d89af330 0x55f4d89b1750 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbf2fe13ac447bd server_cookie=5c698383b4cdf58c in_seq=0 out_seq=0 2024-05-09T02:14:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.531+0000 7f8972ff5700 1 -- 172.21.15.19:0/3946448307 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8984024070 con 0x55f4d89af330 2024-05-09T02:14:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.531+0000 7f8972ff5700 10 client.25454 handle_command_reply: tid=0 2024-05-09T02:14:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.531+0000 7f89a3a5eb80 1 client.25454 shutdown 2024-05-09T02:14:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.531+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/3946448307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d87ae4a0 msgr2=0x55f4d8947880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.531+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/3946448307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d87ae4a0 0x55f4d8947880 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f89880092a0 tx=0x7f8988007a10 comp rx=0 tx=0).stop 2024-05-09T02:14:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.531+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/3946448307 shutdown_connections 2024-05-09T02:14:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.531+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/3946448307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4d892b400 0x55f4d89a7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.531+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/3946448307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4d87ae4a0 0x55f4d8947880 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.531+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/3946448307 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55f4d89af330 0x55f4d89b1750 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.532+0000 7f8972ff5700 0 client.25454 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.532+0000 7f89a3a5eb80 1 --2- 172.21.15.19:0/3946448307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4d8828d00 0x55f4d8947dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.532+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/3946448307 >> 172.21.15.19:0/3946448307 conn(0x55f4d87b1f70 msgr2=0x55f4d8934100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.532+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/3946448307 shutdown_connections 2024-05-09T02:14:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.532+0000 7f89a3a5eb80 1 -- 172.21.15.19:0/3946448307 wait complete. 2024-05-09T02:14:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.532+0000 7f89a3a5eb80 20 client.25454 trim_cache size 0 max 16384 2024-05-09T02:14:40.539 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:14:40.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.534+0000 7f8995048700 1 -- 172.21.15.19:0/1909378281 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f89780b2aa0 msgr2=0x7f89780b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:40.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.534+0000 7f8995048700 1 --2- 172.21.15.19:0/1909378281 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f89780b2aa0 0x7f89780b4fb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8980011cb0 tx=0x7f89800e55f0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.534+0000 7f8995048700 1 -- 172.21.15.19:0/1909378281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899010b2b0 msgr2=0x7f89901a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:40.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.534+0000 7f8995048700 1 --2- 172.21.15.19:0/1909378281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899010b2b0 0x7f89901a15d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f898000b550 tx=0x7f898000b860 comp rx=0 tx=0).stop 2024-05-09T02:14:40.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.534+0000 7f8995048700 1 -- 172.21.15.19:0/1909378281 shutdown_connections 2024-05-09T02:14:40.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.534+0000 7f8995048700 1 --2- 172.21.15.19:0/1909378281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89901a5e00 0x7f89901a6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.534+0000 7f8995048700 1 --2- 172.21.15.19:0/1909378281 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f89780b2aa0 0x7f89780b4fb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.534+0000 7f8995048700 1 --2- 172.21.15.19:0/1909378281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f899010b2b0 0x7f89901a15d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.534+0000 7f8995048700 1 --2- 172.21.15.19:0/1909378281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8990110990 0x7f89901a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:40.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.534+0000 7f8995048700 1 -- 172.21.15.19:0/1909378281 >> 172.21.15.19:0/1909378281 conn(0x7f8990076d30 msgr2=0x7f89901088e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:40.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.535+0000 7f8995048700 1 -- 172.21.15.19:0/1909378281 shutdown_connections 2024-05-09T02:14:40.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:40.535+0000 7f8995048700 1 -- 172.21.15.19:0/1909378281 wait complete. 2024-05-09T02:14:40.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:40 smithi059 ceph-mon[34781]: pgmap v104: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.9 MiB/s wr, 896 op/s 2024-05-09T02:14:40.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:40 smithi196 ceph-mon[34718]: pgmap v104: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.9 MiB/s wr, 896 op/s 2024-05-09T02:14:42.370 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T02:14:42.370 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T02:14:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 180.0 s, 2791.8 tps, lat 35.484 ms stddev 39.838 2024-05-09T02:14:42.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:42 smithi059 ceph-mon[34781]: pgmap v105: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 594 op/s 2024-05-09T02:14:42.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:42 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:14:42.735 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:14:42.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:42 smithi196 ceph-mon[34718]: pgmap v105: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 594 op/s 2024-05-09T02:14:42.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:42 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:14:42.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:42 smithi019 ceph-mon[30151]: pgmap v105: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 594 op/s 2024-05-09T02:14:42.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:42 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:14:44.536 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:44 smithi019 ceph-mon[30151]: pgmap v106: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.1 MiB/s wr, 601 op/s 2024-05-09T02:14:44.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:44 smithi059 ceph-mon[34781]: pgmap v106: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.1 MiB/s wr, 601 op/s 2024-05-09T02:14:44.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:44 smithi196 ceph-mon[34718]: pgmap v106: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.1 MiB/s wr, 601 op/s 2024-05-09T02:14:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:14:45.069 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T02:14:45.069 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "9a3a88a0-0c5f-44ee-bd9c-cf3a455ed10d", 2024-05-09T02:14:45.069 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T02:14:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:14:45.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.852+0000 7f8ff9911700 1 -- 172.21.15.19:0/3759447526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff4108740 msgr2=0x7f8ff41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:45.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.852+0000 7f8ff9911700 1 --2- 172.21.15.19:0/3759447526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff4108740 0x7f8ff41100a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009f90 tx=0x7f8fec00b330 comp rx=0 tx=0).stop 2024-05-09T02:14:45.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.853+0000 7f8ff9911700 1 -- 172.21.15.19:0/3759447526 shutdown_connections 2024-05-09T02:14:45.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.853+0000 7f8ff9911700 1 --2- 172.21.15.19:0/3759447526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff4108740 0x7f8ff41100a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.853+0000 7f8ff9911700 1 --2- 172.21.15.19:0/3759447526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ff41034c0 0x7f8ff41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.853+0000 7f8ff9911700 1 --2- 172.21.15.19:0/3759447526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ff4107df0 0x7f8ff4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.853+0000 7f8ff9911700 1 -- 172.21.15.19:0/3759447526 >> 172.21.15.19:0/3759447526 conn(0x7f8ff40fedc0 msgr2=0x7f8ff41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:45.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.853+0000 7f8ff9911700 1 -- 172.21.15.19:0/3759447526 shutdown_connections 2024-05-09T02:14:45.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.853+0000 7f8ff9911700 1 -- 172.21.15.19:0/3759447526 wait complete. 2024-05-09T02:14:45.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.854+0000 7f8ff9911700 1 Processor -- start 2024-05-09T02:14:45.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.854+0000 7f8ff9911700 1 -- start start 2024-05-09T02:14:45.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff9911700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ff41034c0 0x7f8ff419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff9911700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff4107df0 0x7f8ff419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff9911700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ff4108740 0x7f8ff4199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff9911700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4113370 con 0x7f8ff41034c0 2024-05-09T02:14:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff9911700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41131f0 con 0x7f8ff4107df0 2024-05-09T02:14:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff9911700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41134f0 con 0x7f8ff4108740 2024-05-09T02:14:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ff41034c0 0x7f8ff419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ff41034c0 0x7f8ff419f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37474/0 (socket says 172.21.15.19:37474) 2024-05-09T02:14:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff2ffd700 1 -- 172.21.15.19:0/1818686758 learned_addr learned my addr 172.21.15.19:0/1818686758 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff27fc700 1 --2- 172.21.15.19:0/1818686758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff4107df0 0x7f8ff419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff37fe700 1 --2- 172.21.15.19:0/1818686758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ff4108740 0x7f8ff4199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff2ffd700 1 -- 172.21.15.19:0/1818686758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ff4108740 msgr2=0x7f8ff4199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff2ffd700 1 --2- 172.21.15.19:0/1818686758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ff4108740 0x7f8ff4199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff2ffd700 1 -- 172.21.15.19:0/1818686758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff4107df0 msgr2=0x7f8ff419fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff2ffd700 1 --2- 172.21.15.19:0/1818686758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff4107df0 0x7f8ff419fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff2ffd700 1 -- 172.21.15.19:0/1818686758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff419a1d0 con 0x7f8ff41034c0 2024-05-09T02:14:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.855+0000 7f8ff2ffd700 1 --2- 172.21.15.19:0/1818686758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ff41034c0 0x7f8ff419f8b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400ec10 tx=0x7f8fe400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.856+0000 7f8ff890f700 1 -- 172.21.15.19:0/1818686758 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400cc90 con 0x7f8ff41034c0 2024-05-09T02:14:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.856+0000 7f8ff9911700 1 -- 172.21.15.19:0/1818686758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff419a4c0 con 0x7f8ff41034c0 2024-05-09T02:14:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.856+0000 7f8ff890f700 1 -- 172.21.15.19:0/1818686758 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe400ce30 con 0x7f8ff41034c0 2024-05-09T02:14:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.856+0000 7f8ff890f700 1 -- 172.21.15.19:0/1818686758 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4010430 con 0x7f8ff41034c0 2024-05-09T02:14:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.856+0000 7f8ff9911700 1 -- 172.21.15.19:0/1818686758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff406b610 con 0x7f8ff41034c0 2024-05-09T02:14:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.860+0000 7f8ff890f700 1 -- 172.21.15.19:0/1818686758 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8fe40105d0 con 0x7f8ff41034c0 2024-05-09T02:14:45.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.862+0000 7f8ff890f700 1 --2- 172.21.15.19:0/1818686758 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8fdc0b2a70 0x7f8fdc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.862+0000 7f8ff27fc700 1 --2- 172.21.15.19:0/1818686758 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8fdc0b2a70 0x7f8fdc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.862+0000 7f8ff890f700 1 -- 172.21.15.19:0/1818686758 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f8fe40e2970 con 0x7f8ff41034c0 2024-05-09T02:14:45.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.862+0000 7f9008327b80 1 Processor -- start 2024-05-09T02:14:45.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.862+0000 7f9008327b80 1 -- start start 2024-05-09T02:14:45.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.862+0000 7f8ff27fc700 1 --2- 172.21.15.19:0/1818686758 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8fdc0b2a70 0x7f8fdc0b4f80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4106c10 tx=0x7f8fe80073d0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:45.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.862+0000 7f9008327b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.862+0000 7f9008327b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d4834520 0x5612d49b7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.862+0000 7f9008327b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d49b1480 0x5612d49b1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.862+0000 7f9008327b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5612d48d4120 con 0x5612d4834520 2024-05-09T02:14:45.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.862+0000 7f9008327b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5612d4879a20 con 0x5612d49b1480 2024-05-09T02:14:45.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.862+0000 7f9008327b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5612d48d00e0 con 0x5612d48aed80 2024-05-09T02:14:45.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.862+0000 7f8ff2ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.863+0000 7f8ff2ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50248/0 (socket says 172.21.15.19:50248) 2024-05-09T02:14:45.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.863+0000 7f8ff2ffd700 1 -- 172.21.15.19:0/1012924005 learned_addr learned my addr 172.21.15.19:0/1012924005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:45.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.863+0000 7f8ff27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d4834520 0x5612d49b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.863+0000 7f8ff37fe700 1 --2- 172.21.15.19:0/1012924005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d49b1480 0x5612d49b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.863+0000 7f8ff2ffd700 1 -- 172.21.15.19:0/1012924005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d49b1480 msgr2=0x5612d49b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:45.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.863+0000 7f8ff2ffd700 1 --2- 172.21.15.19:0/1012924005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d49b1480 0x5612d49b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.863+0000 7f8ff2ffd700 1 -- 172.21.15.19:0/1012924005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d4834520 msgr2=0x5612d49b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.863+0000 7f8ff2ffd700 1 --2- 172.21.15.19:0/1012924005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d4834520 0x5612d49b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.863+0000 7f8ff2ffd700 1 -- 172.21.15.19:0/1012924005 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612d486a580 con 0x5612d48aed80 2024-05-09T02:14:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.863+0000 7f8ff2ffd700 1 --2- 172.21.15.19:0/1012924005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b72c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4005d70 tx=0x7f8fe40a6b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad10574be367deae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.864+0000 7f8fd9ffb700 1 -- 172.21.15.19:0/1012924005 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40abe60 con 0x5612d48aed80 2024-05-09T02:14:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.864+0000 7f8fd9ffb700 1 -- 172.21.15.19:0/1012924005 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe40b32e0 con 0x5612d48aed80 2024-05-09T02:14:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.864+0000 7f8fd9ffb700 1 -- 172.21.15.19:0/1012924005 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40b3990 con 0x5612d48aed80 2024-05-09T02:14:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.865+0000 7f9008327b80 1 -- 172.21.15.19:0/1012924005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 msgr2=0x5612d49b72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.865+0000 7f9008327b80 1 --2- 172.21.15.19:0/1012924005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b72c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4005d70 tx=0x7f8fe40a6b20 comp rx=0 tx=0).stop 2024-05-09T02:14:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.865+0000 7f9008327b80 1 -- 172.21.15.19:0/1012924005 shutdown_connections 2024-05-09T02:14:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.865+0000 7f9008327b80 1 --2- 172.21.15.19:0/1012924005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d49b1480 0x5612d49b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.865+0000 7f9008327b80 1 --2- 172.21.15.19:0/1012924005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b72c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.865+0000 7f9008327b80 1 --2- 172.21.15.19:0/1012924005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d4834520 0x5612d49b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.865+0000 7f9008327b80 1 -- 172.21.15.19:0/1012924005 >> 172.21.15.19:0/1012924005 conn(0x5612d4837ff0 msgr2=0x5612d49ac2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.865+0000 7f9008327b80 1 -- 172.21.15.19:0/1012924005 shutdown_connections 2024-05-09T02:14:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.865+0000 7f9008327b80 1 -- 172.21.15.19:0/1012924005 wait complete. 2024-05-09T02:14:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.866+0000 7f9008327b80 1 -- start start 2024-05-09T02:14:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.866+0000 7f9008327b80 1 Processor -- start 2024-05-09T02:14:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.867+0000 7f9008327b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d4834520 0x5612d49bb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.868+0000 7f9008327b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d48aed80 0x5612d49bb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.868+0000 7f9008327b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d49b1480 0x5612d49bbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.868+0000 7f9008327b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5612d49b21e0 con 0x5612d4834520 2024-05-09T02:14:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.868+0000 7f9008327b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5612d486eba0 con 0x5612d48aed80 2024-05-09T02:14:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.868+0000 7f9008327b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5612d49be0a0 con 0x5612d49b1480 2024-05-09T02:14:45.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.868+0000 7f9008327b80 10 client.0 _ll_register_callbacks cb 0x5612d4745740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:45.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.868+0000 7f8ff27fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d48aed80 0x5612d49bb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.868+0000 7f8ff2ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d4834520 0x5612d49bb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.868+0000 7f8ff27fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d48aed80 0x5612d49bb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:47740/0 (socket says 172.21.15.19:47740) 2024-05-09T02:14:45.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.868+0000 7f8ff27fc700 1 -- 172.21.15.19:0/3438239922 learned_addr learned my addr 172.21.15.19:0/3438239922 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:45.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.869+0000 7f8ff37fe700 1 --2- 172.21.15.19:0/3438239922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d49b1480 0x5612d49bbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.869+0000 7f8ff37fe700 1 -- 172.21.15.19:0/3438239922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d48aed80 msgr2=0x5612d49bb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:45.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.869+0000 7f8ff37fe700 1 --2- 172.21.15.19:0/3438239922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d48aed80 0x5612d49bb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.869+0000 7f8ff37fe700 1 -- 172.21.15.19:0/3438239922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d4834520 msgr2=0x5612d49bb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:45.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.869+0000 7f8ff37fe700 1 --2- 172.21.15.19:0/3438239922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d4834520 0x5612d49bb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.869+0000 7f8ff37fe700 1 -- 172.21.15.19:0/3438239922 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612d474f510 con 0x5612d49b1480 2024-05-09T02:14:45.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.870+0000 7f8ff37fe700 1 --2- 172.21.15.19:0/3438239922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d49b1480 0x5612d49bbb80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009790 tx=0x7f8fec003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2fc3a40aeba8c84 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:45.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.870+0000 7f8fd2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:14:45.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.870+0000 7f8fd2ffd700 1 -- 172.21.15.19:0/3438239922 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0079d0 con 0x5612d49b1480 2024-05-09T02:14:45.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.870+0000 7f8fd2ffd700 1 -- 172.21.15.19:0/3438239922 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f8fec007b70 con 0x5612d49b1480 2024-05-09T02:14:45.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.870+0000 7f9008327b80 1 -- 172.21.15.19:0/3438239922 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe40b32e0 con 0x5612d49b1480 2024-05-09T02:14:45.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.871+0000 7f8fd2ffd700 1 -- 172.21.15.19:0/3438239922 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8fec007db0 con 0x5612d49b1480 2024-05-09T02:14:45.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.871+0000 7f9008327b80 10 client.25522 fetch_fsmap learned FSMap version 36 2024-05-09T02:14:45.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.871+0000 7f9008327b80 1 -- 172.21.15.19:0/3438239922 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5612d49c79d0 con 0x5612d49b1480 2024-05-09T02:14:45.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.871+0000 7f8fd2ffd700 1 -- 172.21.15.19:0/3438239922 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fec021070 con 0x5612d49b1480 2024-05-09T02:14:45.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.871+0000 7f8fd2ffd700 1 -- 172.21.15.19:0/3438239922 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f8fec011070 con 0x5612d49b1480 2024-05-09T02:14:45.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.872+0000 7f9008327b80 10 client.25522 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T02:14:45.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.872+0000 7f9008327b80 10 client.25522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:14:45.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.872+0000 7f9008327b80 20 client.25522 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:45.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.872+0000 7f9008327b80 1 --2- 172.21.15.19:0/3438239922 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5612d49ca010 0x5612d49ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.872+0000 7f9008327b80 4 client.25522 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:14:45.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.872+0000 7f9008327b80 1 -- 172.21.15.19:0/3438239922 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8fe40b32e0 con 0x5612d49ca010 2024-05-09T02:14:45.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.873+0000 7f8ff2ffd700 1 --2- 172.21.15.19:0/3438239922 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5612d49ca010 0x5612d49ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.879+0000 7f8ff2ffd700 1 --2- 172.21.15.19:0/3438239922 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5612d49ca010 0x5612d49ca480 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=633a6d5e94dca6d8 server_cookie=46c42df2d83b7a6 in_seq=0 out_seq=0 2024-05-09T02:14:45.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.879+0000 7f8fd2ffd700 10 client.25522 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:45.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.892+0000 7f8fd2ffd700 1 -- 172.21.15.19:0/3438239922 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f8fe40b32e0 con 0x5612d49ca010 2024-05-09T02:14:45.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.892+0000 7f8fd2ffd700 10 client.25522 handle_command_reply: tid=0 2024-05-09T02:14:45.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.892+0000 7f9008327b80 1 client.25522 shutdown 2024-05-09T02:14:45.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.892+0000 7f9008327b80 1 -- 172.21.15.19:0/3438239922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d49b1480 msgr2=0x5612d49bbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:45.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.892+0000 7f9008327b80 1 --2- 172.21.15.19:0/3438239922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d49b1480 0x5612d49bbb80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009790 tx=0x7f8fec003f90 comp rx=0 tx=0).stop 2024-05-09T02:14:45.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.894+0000 7f9008327b80 1 -- 172.21.15.19:0/3438239922 shutdown_connections 2024-05-09T02:14:45.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.894+0000 7f9008327b80 1 --2- 172.21.15.19:0/3438239922 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5612d49ca010 0x5612d49ca480 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.894+0000 7f9008327b80 1 --2- 172.21.15.19:0/3438239922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d49b1480 0x5612d49bbb80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.894+0000 7f9008327b80 1 --2- 172.21.15.19:0/3438239922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d4834520 0x5612d49bb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.894+0000 7f9008327b80 1 --2- 172.21.15.19:0/3438239922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d48aed80 0x5612d49bb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.894+0000 7f9008327b80 1 -- 172.21.15.19:0/3438239922 >> 172.21.15.19:0/3438239922 conn(0x5612d4837ff0 msgr2=0x5612d49a6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:45.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.894+0000 7f8fd2ffd700 0 client.25522 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:45.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.895+0000 7f9008327b80 1 -- 172.21.15.19:0/3438239922 shutdown_connections 2024-05-09T02:14:45.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.895+0000 7f9008327b80 1 -- 172.21.15.19:0/3438239922 wait complete. 2024-05-09T02:14:45.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.895+0000 7f9008327b80 20 client.25522 trim_cache size 0 max 16384 2024-05-09T02:14:45.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.926+0000 7f9008327b80 1 Processor -- start 2024-05-09T02:14:45.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.926+0000 7f9008327b80 1 -- start start 2024-05-09T02:14:45.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f9008327b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d4834520 0x5612d49a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f9008327b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f9008327b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d49b1480 0x5612d49b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f9008327b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41acd40 con 0x5612d49b1480 2024-05-09T02:14:45.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f9008327b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41ae8b0 con 0x5612d4834520 2024-05-09T02:14:45.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f9008327b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41ad410 con 0x5612d48aed80 2024-05-09T02:14:45.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f8ff37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d49b1480 0x5612d49b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f8ff37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d49b1480 0x5612d49b09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37504/0 (socket says 172.21.15.19:37504) 2024-05-09T02:14:45.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f8ff37fe700 1 -- 172.21.15.19:0/3773984887 learned_addr learned my addr 172.21.15.19:0/3773984887 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:45.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f8ff27fc700 1 --2- 172.21.15.19:0/3773984887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f8ff2ffd700 1 --2- 172.21.15.19:0/3773984887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d4834520 0x5612d49a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f8ff37fe700 1 -- 172.21.15.19:0/3773984887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 msgr2=0x5612d49b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:45.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f8ff37fe700 1 --2- 172.21.15.19:0/3773984887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f8ff37fe700 1 -- 172.21.15.19:0/3773984887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d4834520 msgr2=0x5612d49a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:45.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f8ff37fe700 1 --2- 172.21.15.19:0/3773984887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d4834520 0x5612d49a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.927+0000 7f8ff37fe700 1 -- 172.21.15.19:0/3773984887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612d49baed0 con 0x5612d49b1480 2024-05-09T02:14:45.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.928+0000 7f8ff37fe700 1 --2- 172.21.15.19:0/3773984887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d49b1480 0x5612d49b09b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8fec007fd0 tx=0x7f8fec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df6c1a96e6e17e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:45.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.928+0000 7f8fd97fa700 1 -- 172.21.15.19:0/3773984887 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec032020 con 0x5612d49b1480 2024-05-09T02:14:45.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.928+0000 7f8fd97fa700 1 -- 172.21.15.19:0/3773984887 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fec021070 con 0x5612d49b1480 2024-05-09T02:14:45.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.928+0000 7f8fd97fa700 1 -- 172.21.15.19:0/3773984887 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0054e0 con 0x5612d49b1480 2024-05-09T02:14:45.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.929+0000 7f9008327b80 1 -- 172.21.15.19:0/3773984887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d49b1480 msgr2=0x5612d49b09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:45.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.929+0000 7f9008327b80 1 --2- 172.21.15.19:0/3773984887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d49b1480 0x5612d49b09b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8fec007fd0 tx=0x7f8fec00b330 comp rx=0 tx=0).stop 2024-05-09T02:14:45.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.929+0000 7f9008327b80 1 -- 172.21.15.19:0/3773984887 shutdown_connections 2024-05-09T02:14:45.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.929+0000 7f9008327b80 1 --2- 172.21.15.19:0/3773984887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d49b1480 0x5612d49b09b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.929+0000 7f9008327b80 1 --2- 172.21.15.19:0/3773984887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d4834520 0x5612d49a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.929+0000 7f9008327b80 1 --2- 172.21.15.19:0/3773984887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.929+0000 7f9008327b80 1 -- 172.21.15.19:0/3773984887 >> 172.21.15.19:0/3773984887 conn(0x5612d4837ff0 msgr2=0x5612d49ba060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:45.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.929+0000 7f9008327b80 1 -- 172.21.15.19:0/3773984887 shutdown_connections 2024-05-09T02:14:45.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.929+0000 7f9008327b80 1 -- 172.21.15.19:0/3773984887 wait complete. 2024-05-09T02:14:45.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.929+0000 7f9008327b80 1 -- start start 2024-05-09T02:14:45.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.929+0000 7f9008327b80 1 Processor -- start 2024-05-09T02:14:45.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.930+0000 7f9008327b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d4834520 0x5612d49b8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.930+0000 7f9008327b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.930+0000 7f9008327b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d49b1480 0x5612d4a3cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.930+0000 7f9008327b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41ae1b0 con 0x5612d49b1480 2024-05-09T02:14:45.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.930+0000 7f9008327b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41adae0 con 0x5612d4834520 2024-05-09T02:14:45.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.930+0000 7f9008327b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41aad00 con 0x5612d48aed80 2024-05-09T02:14:45.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.930+0000 7f9008327b80 10 client.0 _ll_register_callbacks cb 0x5612d49ac900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:45.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.930+0000 7f8ff37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d49b1480 0x5612d4a3cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.930+0000 7f8ff27fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.930+0000 7f8ff27fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50270/0 (socket says 172.21.15.19:50270) 2024-05-09T02:14:45.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.930+0000 7f8ff27fc700 1 -- 172.21.15.19:0/406223423 learned_addr learned my addr 172.21.15.19:0/406223423 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:45.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.930+0000 7f8ff2ffd700 1 --2- 172.21.15.19:0/406223423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d4834520 0x5612d49b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.931+0000 7f8ff37fe700 1 -- 172.21.15.19:0/406223423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 msgr2=0x5612d49b8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:45.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.931+0000 7f8ff37fe700 1 --2- 172.21.15.19:0/406223423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.931+0000 7f8ff37fe700 1 -- 172.21.15.19:0/406223423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d4834520 msgr2=0x5612d49b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:45.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.931+0000 7f8ff37fe700 1 --2- 172.21.15.19:0/406223423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d4834520 0x5612d49b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.931+0000 7f8ff37fe700 1 -- 172.21.15.19:0/406223423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612d4a3d190 con 0x5612d49b1480 2024-05-09T02:14:45.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.931+0000 7f8ff27fc700 1 --2- 172.21.15.19:0/406223423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:14:45.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.931+0000 7f8ff37fe700 1 --2- 172.21.15.19:0/406223423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d49b1480 0x5612d4a3cc20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8fec002230 tx=0x7f8fec005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23da5f011be8b65 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:45.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.931+0000 7f8fd2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:14:45.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.932+0000 7f8fd2ffd700 1 -- 172.21.15.19:0/406223423 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec03b030 con 0x5612d49b1480 2024-05-09T02:14:45.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.932+0000 7f8fd2ffd700 1 -- 172.21.15.19:0/406223423 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f8fec0079e0 con 0x5612d49b1480 2024-05-09T02:14:45.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.932+0000 7f9008327b80 1 -- 172.21.15.19:0/406223423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fec021070 con 0x5612d49b1480 2024-05-09T02:14:45.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.932+0000 7f8fd2ffd700 1 -- 172.21.15.19:0/406223423 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8fec045050 con 0x5612d49b1480 2024-05-09T02:14:45.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.932+0000 7f9008327b80 10 client.15786 fetch_fsmap learned FSMap version 36 2024-05-09T02:14:45.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.932+0000 7f9008327b80 1 -- 172.21.15.19:0/406223423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5612d4a3e410 con 0x5612d49b1480 2024-05-09T02:14:45.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.932+0000 7f8fd2ffd700 1 -- 172.21.15.19:0/406223423 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fec005610 con 0x5612d49b1480 2024-05-09T02:14:45.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.933+0000 7f8fd2ffd700 1 -- 172.21.15.19:0/406223423 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f8fec011040 con 0x5612d49b1480 2024-05-09T02:14:45.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.933+0000 7f9008327b80 10 client.15786 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T02:14:45.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.933+0000 7f9008327b80 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:14:45.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.933+0000 7f9008327b80 20 client.15786 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:45.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.933+0000 7f9008327b80 1 --2- 172.21.15.19:0/406223423 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5612d4a3e9e0 0x5612d4a40e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:45.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.933+0000 7f9008327b80 4 client.15786 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T02:14:45.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.933+0000 7f9008327b80 1 -- 172.21.15.19:0/406223423 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f8fec021070 con 0x5612d4a3e9e0 2024-05-09T02:14:45.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.933+0000 7f8ff2ffd700 1 --2- 172.21.15.19:0/406223423 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5612d4a3e9e0 0x5612d4a40e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:45.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.933+0000 7f8fd2ffd700 10 client.15786 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:45.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.934+0000 7f8ff2ffd700 1 --2- 172.21.15.19:0/406223423 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5612d4a3e9e0 0x5612d4a40e00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deda366894eeea6f server_cookie=5dcee458fca8f38c in_seq=0 out_seq=0 2024-05-09T02:14:45.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.935+0000 7f8fd2ffd700 1 -- 172.21.15.19:0/406223423 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f8fec021070 con 0x5612d4a3e9e0 2024-05-09T02:14:45.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.935+0000 7f8fd2ffd700 10 client.15786 handle_command_reply: tid=0 2024-05-09T02:14:45.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.935+0000 7f9008327b80 1 client.15786 shutdown 2024-05-09T02:14:45.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.935+0000 7f9008327b80 1 -- 172.21.15.19:0/406223423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d49b1480 msgr2=0x5612d4a3cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:45.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.935+0000 7f9008327b80 1 --2- 172.21.15.19:0/406223423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d49b1480 0x5612d4a3cc20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8fec002230 tx=0x7f8fec005f10 comp rx=0 tx=0).stop 2024-05-09T02:14:45.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.936+0000 7f9008327b80 1 -- 172.21.15.19:0/406223423 shutdown_connections 2024-05-09T02:14:45.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.936+0000 7f9008327b80 1 --2- 172.21.15.19:0/406223423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5612d49b1480 0x5612d4a3cc20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.936+0000 7f9008327b80 1 --2- 172.21.15.19:0/406223423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612d4834520 0x5612d49b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.936+0000 7f9008327b80 1 --2- 172.21.15.19:0/406223423 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5612d4a3e9e0 0x5612d4a40e00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.936+0000 7f8fd2ffd700 0 client.15786 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:45.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.936+0000 7f9008327b80 1 --2- 172.21.15.19:0/406223423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d48aed80 0x5612d49b8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.936+0000 7f9008327b80 1 -- 172.21.15.19:0/406223423 >> 172.21.15.19:0/406223423 conn(0x5612d4837ff0 msgr2=0x5612d49ba060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:45.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.936+0000 7f9008327b80 1 -- 172.21.15.19:0/406223423 shutdown_connections 2024-05-09T02:14:45.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.936+0000 7f9008327b80 1 -- 172.21.15.19:0/406223423 wait complete. 2024-05-09T02:14:45.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.936+0000 7f9008327b80 20 client.15786 trim_cache size 0 max 16384 2024-05-09T02:14:45.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.938+0000 7f8ff9911700 1 -- 172.21.15.19:0/1818686758 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8fdc0b2a70 msgr2=0x7f8fdc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:45.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.938+0000 7f8ff9911700 1 --2- 172.21.15.19:0/1818686758 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8fdc0b2a70 0x7f8fdc0b4f80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4106c10 tx=0x7f8fe80073d0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.938+0000 7f8ff9911700 1 -- 172.21.15.19:0/1818686758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ff41034c0 msgr2=0x7f8ff419f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:45.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.938+0000 7f8ff9911700 1 --2- 172.21.15.19:0/1818686758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ff41034c0 0x7f8ff419f8b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400ec10 tx=0x7f8fe400ef20 comp rx=0 tx=0).stop 2024-05-09T02:14:45.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.938+0000 7f8ff9911700 1 -- 172.21.15.19:0/1818686758 shutdown_connections 2024-05-09T02:14:45.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.938+0000 7f8ff9911700 1 --2- 172.21.15.19:0/1818686758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ff4108740 0x7f8ff4199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.938+0000 7f8ff9911700 1 --2- 172.21.15.19:0/1818686758 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8fdc0b2a70 0x7f8fdc0b4f80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.938+0000 7f8ff9911700 1 --2- 172.21.15.19:0/1818686758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ff41034c0 0x7f8ff419f8b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.938+0000 7f8ff9911700 1 --2- 172.21.15.19:0/1818686758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff4107df0 0x7f8ff419fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:45.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.938+0000 7f8ff9911700 1 -- 172.21.15.19:0/1818686758 >> 172.21.15.19:0/1818686758 conn(0x7f8ff40fedc0 msgr2=0x7f8ff4107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:45.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.938+0000 7f8ff9911700 1 -- 172.21.15.19:0/1818686758 shutdown_connections 2024-05-09T02:14:45.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:44.938+0000 7f8ff9911700 1 -- 172.21.15.19:0/1818686758 wait complete. 2024-05-09T02:14:45.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:45 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:14:45.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:45 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:45.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:45 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:14:45.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:45 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:45.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:45 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:14:45.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:45 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.268 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:14:46.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:46 smithi019 ceph-mon[30151]: scrub queued for path: / 2024-05-09T02:14:46.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:46 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:46 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:14:46.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:46 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:14:46.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:46 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:46 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:14:46.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:46 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:14:46.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:46 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:14:46.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:46 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:46 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:46 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:46 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:46 smithi019 ceph-mon[30151]: pgmap v107: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 821 op/s 2024-05-09T02:14:46.667 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:14:46.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:46 smithi059 ceph-mon[34781]: scrub queued for path: / 2024-05-09T02:14:46.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:46 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:46 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:14:46.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:46 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:14:46.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:46 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:46 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:14:46.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:46 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:14:46.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:46 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:14:46.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:46 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:46 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:46 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:46 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:46 smithi059 ceph-mon[34781]: pgmap v107: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 821 op/s 2024-05-09T02:14:46.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:46 smithi196 ceph-mon[34718]: scrub queued for path: / 2024-05-09T02:14:46.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:46 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:46 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:14:46.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:46 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:14:46.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:46 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:46 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:14:46.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:46 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:14:46.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:46 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:14:46.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:46 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:46 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:46 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:46 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:14:46.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:46 smithi196 ceph-mon[34718]: pgmap v107: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 821 op/s 2024-05-09T02:14:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 185.0 s, 3080.8 tps, lat 32.635 ms stddev 39.236 2024-05-09T02:14:48.632 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:48 smithi019 ceph-mon[30151]: pgmap v108: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.9 MiB/s wr, 586 op/s 2024-05-09T02:14:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:48 smithi059 ceph-mon[34781]: pgmap v108: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.9 MiB/s wr, 586 op/s 2024-05-09T02:14:48.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:48 smithi196 ceph-mon[34718]: pgmap v108: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.9 MiB/s wr, 586 op/s 2024-05-09T02:14:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.791+0000 7f2bcf878700 1 -- 172.21.15.19:0/833314828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc8110c90 msgr2=0x7f2bc81130b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.791+0000 7f2bcf878700 1 --2- 172.21.15.19:0/833314828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc8110c90 0x7f2bc81130b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009990 tx=0x7f2bb8009ca0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.791+0000 7f2bcf878700 1 -- 172.21.15.19:0/833314828 shutdown_connections 2024-05-09T02:14:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.791+0000 7f2bcf878700 1 --2- 172.21.15.19:0/833314828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc8110c90 0x7f2bc81130b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.791+0000 7f2bcf878700 1 --2- 172.21.15.19:0/833314828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bc81030e0 0x7f2bc81034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.791+0000 7f2bcf878700 1 --2- 172.21.15.19:0/833314828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bc8103a30 0x7f2bc8110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.792+0000 7f2bcf878700 1 -- 172.21.15.19:0/833314828 >> 172.21.15.19:0/833314828 conn(0x7f2bc80fedc0 msgr2=0x7f2bc81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.792+0000 7f2bcf878700 1 -- 172.21.15.19:0/833314828 shutdown_connections 2024-05-09T02:14:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.792+0000 7f2bcf878700 1 -- 172.21.15.19:0/833314828 wait complete. 2024-05-09T02:14:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.792+0000 7f2bcf878700 1 Processor -- start 2024-05-09T02:14:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.792+0000 7f2bcf878700 1 -- start start 2024-05-09T02:14:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcf878700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bc81030e0 0x7f2bc819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcf878700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bc8103a30 0x7f2bc819d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcf878700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc8110c90 0x7f2bc81a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcf878700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc8116680 con 0x7f2bc8110c90 2024-05-09T02:14:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcf878700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc8116500 con 0x7f2bc8103a30 2024-05-09T02:14:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcf878700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc8116800 con 0x7f2bc81030e0 2024-05-09T02:14:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcd614700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bc81030e0 0x7f2bc819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcce13700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bc8103a30 0x7f2bc819d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcd614700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bc81030e0 0x7f2bc819cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:59740/0 (socket says 172.21.15.19:59740) 2024-05-09T02:14:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcd614700 1 -- 172.21.15.19:0/3179316439 learned_addr learned my addr 172.21.15.19:0/3179316439 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcde15700 1 --2- 172.21.15.19:0/3179316439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc8110c90 0x7f2bc81a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcce13700 1 -- 172.21.15.19:0/3179316439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bc81030e0 msgr2=0x7f2bc819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcce13700 1 --2- 172.21.15.19:0/3179316439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bc81030e0 0x7f2bc819cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcce13700 1 -- 172.21.15.19:0/3179316439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc8110c90 msgr2=0x7f2bc81a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:48.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcce13700 1 --2- 172.21.15.19:0/3179316439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc8110c90 0x7f2bc81a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.793+0000 7f2bcce13700 1 -- 172.21.15.19:0/3179316439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc81a1e60 con 0x7f2bc8103a30 2024-05-09T02:14:48.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.794+0000 7f2bcce13700 1 --2- 172.21.15.19:0/3179316439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bc8103a30 0x7f2bc819d530 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000eee0 tx=0x7f2bc000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:48.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.794+0000 7f2bbe7fc700 1 -- 172.21.15.19:0/3179316439 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000c9c0 con 0x7f2bc8103a30 2024-05-09T02:14:48.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.794+0000 7f2bcf878700 1 -- 172.21.15.19:0/3179316439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc81a20f0 con 0x7f2bc8103a30 2024-05-09T02:14:48.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.794+0000 7f2bbe7fc700 1 -- 172.21.15.19:0/3179316439 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc0019070 con 0x7f2bc8103a30 2024-05-09T02:14:48.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.794+0000 7f2bbe7fc700 1 -- 172.21.15.19:0/3179316439 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0014480 con 0x7f2bc8103a30 2024-05-09T02:14:48.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.794+0000 7f2bcf878700 1 -- 172.21.15.19:0/3179316439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc81a9a70 con 0x7f2bc8103a30 2024-05-09T02:14:48.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.797+0000 7f2bbe7fc700 1 -- 172.21.15.19:0/3179316439 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2bc0012050 con 0x7f2bc8103a30 2024-05-09T02:14:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.799+0000 7f2bbe7fc700 1 --2- 172.21.15.19:0/3179316439 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2bb40b29f0 0x7f2bb40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.800+0000 7f2bbe7fc700 1 -- 172.21.15.19:0/3179316439 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f2bc00e2710 con 0x7f2bc8103a30 2024-05-09T02:14:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.800+0000 7f2bcd614700 1 --2- 172.21.15.19:0/3179316439 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2bb40b29f0 0x7f2bb40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.800+0000 7f2bde28eb80 1 Processor -- start 2024-05-09T02:14:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.800+0000 7f2bde28eb80 1 -- start start 2024-05-09T02:14:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.800+0000 7f2bde28eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 0x55e821d18240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.800+0000 7f2bde28eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821b954a0 0x55e821d18780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.800+0000 7f2bde28eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821d12400 0x55e821d12810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.800+0000 7f2bde28eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e821c350a0 con 0x55e821d12400 2024-05-09T02:14:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.800+0000 7f2bde28eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e821bda9a0 con 0x55e821b954a0 2024-05-09T02:14:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.800+0000 7f2bde28eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e821c31060 con 0x55e821c0fd00 2024-05-09T02:14:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.800+0000 7f2bcd614700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 0x55e821d18240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.801+0000 7f2bcde15700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821d12400 0x55e821d12810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.801+0000 7f2bcd614700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 0x55e821d18240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:59746/0 (socket says 172.21.15.19:59746) 2024-05-09T02:14:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.801+0000 7f2bcd614700 1 -- 172.21.15.19:0/2011198520 learned_addr learned my addr 172.21.15.19:0/2011198520 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.801+0000 7f2bcce13700 1 --2- 172.21.15.19:0/2011198520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821b954a0 0x55e821d18780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.801+0000 7f2bcd614700 1 --2- 172.21.15.19:0/3179316439 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2bb40b29f0 0x7f2bb40b4f00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4007ce0 tx=0x7f2bc4007480 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.801+0000 7f2bcde15700 1 -- 172.21.15.19:0/2011198520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 msgr2=0x55e821d18240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.801+0000 7f2bcde15700 1 --2- 172.21.15.19:0/2011198520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 0x55e821d18240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.801+0000 7f2bcde15700 1 -- 172.21.15.19:0/2011198520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821b954a0 msgr2=0x55e821d18780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.801+0000 7f2bcde15700 1 --2- 172.21.15.19:0/2011198520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821b954a0 0x55e821d18780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.801+0000 7f2bcde15700 1 -- 172.21.15.19:0/2011198520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e821bcb500 con 0x55e821d12400 2024-05-09T02:14:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.801+0000 7f2bcde15700 1 --2- 172.21.15.19:0/2011198520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821d12400 0x55e821d12810 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8006010 tx=0x7f2bb8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9340cd9d39f56a9e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.801+0000 7f2bb37fe700 1 -- 172.21.15.19:0/2011198520 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8027070 con 0x55e821d12400 2024-05-09T02:14:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.801+0000 7f2bb37fe700 1 -- 172.21.15.19:0/2011198520 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb80056b0 con 0x55e821d12400 2024-05-09T02:14:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.801+0000 7f2bb37fe700 1 -- 172.21.15.19:0/2011198520 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8003c30 con 0x55e821d12400 2024-05-09T02:14:48.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.802+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2011198520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821d12400 msgr2=0x55e821d12810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:48.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.802+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/2011198520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821d12400 0x55e821d12810 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8006010 tx=0x7f2bb8005e80 comp rx=0 tx=0).stop 2024-05-09T02:14:48.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.802+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2011198520 shutdown_connections 2024-05-09T02:14:48.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.802+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/2011198520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821d12400 0x55e821d12810 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.802+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/2011198520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 0x55e821d18240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.802+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/2011198520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821b954a0 0x55e821d18780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.802+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2011198520 >> 172.21.15.19:0/2011198520 conn(0x55e821b98f70 msgr2=0x55e821d0d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:48.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.803+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2011198520 shutdown_connections 2024-05-09T02:14:48.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.803+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2011198520 wait complete. 2024-05-09T02:14:48.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.803+0000 7f2bde28eb80 1 -- start start 2024-05-09T02:14:48.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.803+0000 7f2bde28eb80 1 Processor -- start 2024-05-09T02:14:48.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bde28eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821b954a0 0x55e821d1c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bde28eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 0x55e821d1c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bde28eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821d12400 0x55e821d1cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bde28eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e821d13100 con 0x55e821b954a0 2024-05-09T02:14:48.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bde28eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e821bcfb20 con 0x55e821d12400 2024-05-09T02:14:48.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bde28eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e821d28250 con 0x55e821c0fd00 2024-05-09T02:14:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bde28eb80 10 client.0 _ll_register_callbacks cb 0x55e821aa66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bcde15700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821d12400 0x55e821d1cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bcd614700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821b954a0 0x55e821d1c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bcce13700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 0x55e821d1c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bcce13700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 0x55e821d1c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:59762/0 (socket says 172.21.15.19:59762) 2024-05-09T02:14:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bcce13700 1 -- 172.21.15.19:0/2414414730 learned_addr learned my addr 172.21.15.19:0/2414414730 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bcd614700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821b954a0 0x55e821d1c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51730/0 (socket says 172.21.15.19:51730) 2024-05-09T02:14:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bcce13700 1 -- 172.21.15.19:0/2414414730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821d12400 msgr2=0x55e821d1cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bcce13700 1 --2- 172.21.15.19:0/2414414730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821d12400 0x55e821d1cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bcce13700 1 -- 172.21.15.19:0/2414414730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821b954a0 msgr2=0x55e821d1c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bcce13700 1 --2- 172.21.15.19:0/2414414730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821b954a0 0x55e821d1c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.804+0000 7f2bcce13700 1 -- 172.21.15.19:0/2414414730 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e821ab0490 con 0x55e821c0fd00 2024-05-09T02:14:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.805+0000 7f2bcce13700 1 --2- 172.21.15.19:0/2414414730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 0x55e821d1c630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000c6f0 tx=0x7f2bc00a9ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be876d77e02cdd50 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.805+0000 7f2bb0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:14:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.805+0000 7f2bb0ff9700 1 -- 172.21.15.19:0/2414414730 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00b2130 con 0x55e821c0fd00 2024-05-09T02:14:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.805+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2414414730 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bb80056b0 con 0x55e821c0fd00 2024-05-09T02:14:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.805+0000 7f2bb0ff9700 1 -- 172.21.15.19:0/2414414730 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f2bc00920e0 con 0x55e821c0fd00 2024-05-09T02:14:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.806+0000 7f2bb0ff9700 1 -- 172.21.15.19:0/2414414730 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2bc00cc2d0 con 0x55e821c0fd00 2024-05-09T02:14:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.806+0000 7f2bde28eb80 10 client.25540 fetch_fsmap learned FSMap version 36 2024-05-09T02:14:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.806+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2414414730 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e821d29c80 con 0x55e821c0fd00 2024-05-09T02:14:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.806+0000 7f2bb0ff9700 1 -- 172.21.15.19:0/2414414730 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb80056b0 con 0x55e821c0fd00 2024-05-09T02:14:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.806+0000 7f2bb0ff9700 1 -- 172.21.15.19:0/2414414730 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f2bc00a5300 con 0x55e821c0fd00 2024-05-09T02:14:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.806+0000 7f2bde28eb80 10 client.25540 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T02:14:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.806+0000 7f2bde28eb80 10 client.25540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:14:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.806+0000 7f2bde28eb80 20 client.25540 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.806+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/2414414730 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e821d2c210 0x55e821d2e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.806+0000 7f2bde28eb80 4 client.25540 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:14:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.806+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2414414730 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e821c42a80 con 0x55e821d2c210 2024-05-09T02:14:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.806+0000 7f2bcd614700 1 --2- 172.21.15.19:0/2414414730 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e821d2c210 0x55e821d2e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.807+0000 7f2bb0ff9700 10 client.25540 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.807+0000 7f2bcd614700 1 --2- 172.21.15.19:0/2414414730 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e821d2c210 0x55e821d2e630 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a17fdde61d858955 server_cookie=69bb7cdc0118604b in_seq=0 out_seq=0 2024-05-09T02:14:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.808+0000 7f2bb0ff9700 1 -- 172.21.15.19:0/2414414730 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e821c42a80 con 0x55e821d2c210 2024-05-09T02:14:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.808+0000 7f2bb0ff9700 10 client.25540 handle_command_reply: tid=0 2024-05-09T02:14:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.808+0000 7f2bde28eb80 1 client.25540 shutdown 2024-05-09T02:14:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.809+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2414414730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 msgr2=0x55e821d1c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.809+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/2414414730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 0x55e821d1c630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000c6f0 tx=0x7f2bc00a9ed0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.809+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2414414730 shutdown_connections 2024-05-09T02:14:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.809+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/2414414730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821d12400 0x55e821d1cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.809+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/2414414730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821b954a0 0x55e821d1c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.809+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/2414414730 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e821d2c210 0x55e821d2e630 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.809+0000 7f2bb0ff9700 0 client.25540 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.809+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/2414414730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 0x55e821d1c630 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.809+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2414414730 >> 172.21.15.19:0/2414414730 conn(0x55e821b98f70 msgr2=0x55e821d06ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.809+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2414414730 shutdown_connections 2024-05-09T02:14:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.809+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2414414730 wait complete. 2024-05-09T02:14:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.810+0000 7f2bde28eb80 20 client.25540 trim_cache size 0 max 16384 2024-05-09T02:14:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.840+0000 7f2bde28eb80 1 Processor -- start 2024-05-09T02:14:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.840+0000 7f2bde28eb80 1 -- start start 2024-05-09T02:14:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.840+0000 7f2bde28eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821b954a0 0x55e821d27db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.840+0000 7f2bde28eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821c0fd00 0x55e821d1b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.840+0000 7f2bde28eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821d12400 0x55e821d07700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.840+0000 7f2bde28eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81b1050 con 0x55e821c0fd00 2024-05-09T02:14:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.840+0000 7f2bde28eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81b2bc0 con 0x55e821d12400 2024-05-09T02:14:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.840+0000 7f2bde28eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81b1720 con 0x55e821b954a0 2024-05-09T02:14:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.840+0000 7f2bcce13700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821c0fd00 0x55e821d1b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.840+0000 7f2bcce13700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821c0fd00 0x55e821d1b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51732/0 (socket says 172.21.15.19:51732) 2024-05-09T02:14:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.840+0000 7f2bcde15700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821d12400 0x55e821d07700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.840+0000 7f2bcce13700 1 -- 172.21.15.19:0/2881227439 learned_addr learned my addr 172.21.15.19:0/2881227439 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.841+0000 7f2bcd614700 1 --2- 172.21.15.19:0/2881227439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821b954a0 0x55e821d27db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.841+0000 7f2bcce13700 1 -- 172.21.15.19:0/2881227439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821b954a0 msgr2=0x55e821d27db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.841+0000 7f2bcce13700 1 --2- 172.21.15.19:0/2881227439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821b954a0 0x55e821d27db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.841+0000 7f2bcce13700 1 -- 172.21.15.19:0/2881227439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821d12400 msgr2=0x55e821d07700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.841+0000 7f2bcce13700 1 --2- 172.21.15.19:0/2881227439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821d12400 0x55e821d07700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.841+0000 7f2bcce13700 1 -- 172.21.15.19:0/2881227439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e821d1bec0 con 0x55e821c0fd00 2024-05-09T02:14:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.841+0000 7f2bcd614700 1 --2- 172.21.15.19:0/2881227439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821b954a0 0x55e821d27db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:14:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.841+0000 7f2bcde15700 1 --2- 172.21.15.19:0/2881227439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821d12400 0x55e821d07700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:14:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.841+0000 7f2bcce13700 1 --2- 172.21.15.19:0/2881227439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821c0fd00 0x55e821d1b070 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00a5a10 tx=0x7f2bc000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a8f4cf8c1fbc0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.841+0000 7f2bb2ffd700 1 -- 172.21.15.19:0/2881227439 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00b2130 con 0x55e821c0fd00 2024-05-09T02:14:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.841+0000 7f2bb2ffd700 1 -- 172.21.15.19:0/2881227439 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc00a94a0 con 0x55e821c0fd00 2024-05-09T02:14:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.841+0000 7f2bb2ffd700 1 -- 172.21.15.19:0/2881227439 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00a9990 con 0x55e821c0fd00 2024-05-09T02:14:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.842+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2881227439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821c0fd00 msgr2=0x55e821d1b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.842+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/2881227439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821c0fd00 0x55e821d1b070 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00a5a10 tx=0x7f2bc000cf70 comp rx=0 tx=0).stop 2024-05-09T02:14:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.843+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2881227439 shutdown_connections 2024-05-09T02:14:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.843+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/2881227439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821d12400 0x55e821d07700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.843+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/2881227439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821b954a0 0x55e821d27db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.843+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/2881227439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821c0fd00 0x55e821d1b070 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.843+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2881227439 >> 172.21.15.19:0/2881227439 conn(0x55e821b98f70 msgr2=0x55e821d06ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.843+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2881227439 shutdown_connections 2024-05-09T02:14:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.843+0000 7f2bde28eb80 1 -- 172.21.15.19:0/2881227439 wait complete. 2024-05-09T02:14:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.843+0000 7f2bde28eb80 1 -- start start 2024-05-09T02:14:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.844+0000 7f2bde28eb80 1 Processor -- start 2024-05-09T02:14:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.844+0000 7f2bde28eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821b954a0 0x55e821d19240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.844+0000 7f2bde28eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 0x55e821d19780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.844+0000 7f2bde28eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821d12400 0x55e821d19cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.844+0000 7f2bde28eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81b24c0 con 0x55e821d12400 2024-05-09T02:14:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.844+0000 7f2bde28eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81b1df0 con 0x55e821b954a0 2024-05-09T02:14:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.844+0000 7f2bde28eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81aeff0 con 0x55e821c0fd00 2024-05-09T02:14:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.844+0000 7f2bde28eb80 10 client.0 _ll_register_callbacks cb 0x55e821aa66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:14:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.844+0000 7f2bcd614700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821b954a0 0x55e821d19240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.844+0000 7f2bcd614700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821b954a0 0x55e821d19240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:37480/0 (socket says 172.21.15.19:37480) 2024-05-09T02:14:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.844+0000 7f2bcd614700 1 -- 172.21.15.19:0/3285362952 learned_addr learned my addr 172.21.15.19:0/3285362952 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:14:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.844+0000 7f2bcde15700 1 --2- 172.21.15.19:0/3285362952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821d12400 0x55e821d19cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.844+0000 7f2bcce13700 1 --2- 172.21.15.19:0/3285362952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 0x55e821d19780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.845+0000 7f2bcd614700 1 -- 172.21.15.19:0/3285362952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 msgr2=0x55e821d19780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.845+0000 7f2bcd614700 1 --2- 172.21.15.19:0/3285362952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 0x55e821d19780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.845+0000 7f2bcd614700 1 -- 172.21.15.19:0/3285362952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821d12400 msgr2=0x55e821d19cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:14:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.845+0000 7f2bcd614700 1 --2- 172.21.15.19:0/3285362952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821d12400 0x55e821d19cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.845+0000 7f2bcd614700 1 -- 172.21.15.19:0/3285362952 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e821d95bf0 con 0x55e821b954a0 2024-05-09T02:14:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.845+0000 7f2bcd614700 1 --2- 172.21.15.19:0/3285362952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821b954a0 0x55e821d19240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4010a40 tx=0x7f2bc40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9ba2860d27bc8ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:14:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.845+0000 7f2bb0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:14:48.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.846+0000 7f2bb0ff9700 1 -- 172.21.15.19:0/3285362952 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4011e60 con 0x55e821b954a0 2024-05-09T02:14:48.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.846+0000 7f2bb0ff9700 1 -- 172.21.15.19:0/3285362952 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f2bc402c550 con 0x55e821b954a0 2024-05-09T02:14:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.847+0000 7f2bde28eb80 1 -- 172.21.15.19:0/3285362952 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc00a94a0 con 0x55e821b954a0 2024-05-09T02:14:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.847+0000 7f2bb0ff9700 1 -- 172.21.15.19:0/3285362952 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2bc402c750 con 0x55e821b954a0 2024-05-09T02:14:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.847+0000 7f2bde28eb80 10 client.25490 fetch_fsmap learned FSMap version 36 2024-05-09T02:14:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.847+0000 7f2bde28eb80 1 -- 172.21.15.19:0/3285362952 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e821d96ee0 con 0x55e821b954a0 2024-05-09T02:14:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.847+0000 7f2bb0ff9700 1 -- 172.21.15.19:0/3285362952 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc00a94a0 con 0x55e821b954a0 2024-05-09T02:14:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.848+0000 7f2bb0ff9700 1 -- 172.21.15.19:0/3285362952 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f2bc4035c70 con 0x55e821b954a0 2024-05-09T02:14:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.848+0000 7f2bde28eb80 10 client.25490 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T02:14:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.848+0000 7f2bde28eb80 10 client.25490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:14:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.848+0000 7f2bde28eb80 20 client.25490 populate_metadata read hostname 'smithi019' 2024-05-09T02:14:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.848+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/3285362952 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e821da8620 0x55e821daaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:14:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.848+0000 7f2bde28eb80 4 client.25490 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:14:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.848+0000 7f2bde28eb80 1 -- 172.21.15.19:0/3285362952 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e821c3cb20 con 0x55e821da8620 2024-05-09T02:14:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.848+0000 7f2bcce13700 1 --2- 172.21.15.19:0/3285362952 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e821da8620 0x55e821daaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:14:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.848+0000 7f2bb0ff9700 10 client.25490 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.848+0000 7f2bcce13700 1 --2- 172.21.15.19:0/3285362952 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e821da8620 0x55e821daaa40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=354d5bdf7e60a714 server_cookie=9ac42d109fc72436 in_seq=0 out_seq=0 2024-05-09T02:14:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.849+0000 7f2bb0ff9700 1 -- 172.21.15.19:0/3285362952 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e821c3cb20 con 0x55e821da8620 2024-05-09T02:14:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.849+0000 7f2bb0ff9700 10 client.25490 handle_command_reply: tid=0 2024-05-09T02:14:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.849+0000 7f2bde28eb80 1 client.25490 shutdown 2024-05-09T02:14:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.849+0000 7f2bde28eb80 1 -- 172.21.15.19:0/3285362952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821b954a0 msgr2=0x55e821d19240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.849+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/3285362952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821b954a0 0x55e821d19240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4010a40 tx=0x7f2bc40047c0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.849+0000 7f2bde28eb80 1 -- 172.21.15.19:0/3285362952 shutdown_connections 2024-05-09T02:14:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.849+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/3285362952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e821d12400 0x55e821d19cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.849+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/3285362952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e821b954a0 0x55e821d19240 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.849+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/3285362952 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55e821da8620 0x55e821daaa40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.849+0000 7f2bde28eb80 1 --2- 172.21.15.19:0/3285362952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e821c0fd00 0x55e821d19780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.849+0000 7f2bde28eb80 1 -- 172.21.15.19:0/3285362952 >> 172.21.15.19:0/3285362952 conn(0x55e821b98f70 msgr2=0x55e821d113f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.849+0000 7f2bb0ff9700 0 client.25490 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:14:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.849+0000 7f2bde28eb80 1 -- 172.21.15.19:0/3285362952 shutdown_connections 2024-05-09T02:14:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.849+0000 7f2bde28eb80 1 -- 172.21.15.19:0/3285362952 wait complete. 2024-05-09T02:14:48.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.849+0000 7f2bde28eb80 20 client.25490 trim_cache size 0 max 16384 2024-05-09T02:14:48.854 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:14:48.854 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T02:14:48.854 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T02:14:48.854 INFO:teuthology.orchestra.run.smithi019.stdout: "9a3a88a0-0c5f-44ee-bd9c-cf3a455ed10d": { 2024-05-09T02:14:48.855 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T02:14:48.855 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "9a3a88a0-0c5f-44ee-bd9c-cf3a455ed10d", 2024-05-09T02:14:48.855 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T02:14:48.855 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:14:48.855 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:14:48.855 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:14:48.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.850+0000 7f2bcf878700 1 -- 172.21.15.19:0/3179316439 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2bb40b29f0 msgr2=0x7f2bb40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:48.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.850+0000 7f2bcf878700 1 --2- 172.21.15.19:0/3179316439 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2bb40b29f0 0x7f2bb40b4f00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4007ce0 tx=0x7f2bc4007480 comp rx=0 tx=0).stop 2024-05-09T02:14:48.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.850+0000 7f2bcf878700 1 -- 172.21.15.19:0/3179316439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bc8103a30 msgr2=0x7f2bc819d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:14:48.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.850+0000 7f2bcf878700 1 --2- 172.21.15.19:0/3179316439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bc8103a30 0x7f2bc819d530 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000eee0 tx=0x7f2bc000c3a0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.850+0000 7f2bcf878700 1 -- 172.21.15.19:0/3179316439 shutdown_connections 2024-05-09T02:14:48.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.850+0000 7f2bcf878700 1 --2- 172.21.15.19:0/3179316439 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2bb40b29f0 0x7f2bb40b4f00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.850+0000 7f2bcf878700 1 --2- 172.21.15.19:0/3179316439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc8110c90 0x7f2bc81a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.850+0000 7f2bcf878700 1 --2- 172.21.15.19:0/3179316439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bc81030e0 0x7f2bc819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.850+0000 7f2bcf878700 1 --2- 172.21.15.19:0/3179316439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bc8103a30 0x7f2bc819d530 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:14:48.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.850+0000 7f2bcf878700 1 -- 172.21.15.19:0/3179316439 >> 172.21.15.19:0/3179316439 conn(0x7f2bc80fedc0 msgr2=0x7f2bc8112690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:14:48.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.850+0000 7f2bcf878700 1 -- 172.21.15.19:0/3179316439 shutdown_connections 2024-05-09T02:14:48.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:14:48.850+0000 7f2bcf878700 1 -- 172.21.15.19:0/3179316439 wait complete. 2024-05-09T02:14:49.605 INFO:tasks.cephfs.filesystem:scrub status for tag:9a3a88a0-0c5f-44ee-bd9c-cf3a455ed10d - {'path': '/', 'tag': '9a3a88a0-0c5f-44ee-bd9c-cf3a455ed10d', 'options': 'recursive,force'} 2024-05-09T02:14:50.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:50 smithi059 ceph-mon[34781]: pgmap v109: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 915 op/s 2024-05-09T02:14:50.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:50 smithi196 ceph-mon[34718]: pgmap v109: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 915 op/s 2024-05-09T02:14:50.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:50 smithi019 ceph-mon[30151]: pgmap v109: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 915 op/s 2024-05-09T02:14:51.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:51 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:14:51.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:51 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:14:51.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:51 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:14:51.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:51 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:14:51.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:51 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:14:51.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:51 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:14:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:51 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:14:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:51 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:14:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:51 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:14:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:51 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:14:51.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:51 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:14:51.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:51 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:14:51.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:51 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:14:51.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:51 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:14:51.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:51 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:14:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 190.0 s, 2700.2 tps, lat 37.191 ms stddev 39.080 2024-05-09T02:14:52.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:52 smithi059 ceph-mon[34781]: pgmap v110: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 622 op/s 2024-05-09T02:14:52.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:52 smithi196 ceph-mon[34718]: pgmap v110: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 622 op/s 2024-05-09T02:14:52.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:52 smithi019 ceph-mon[30151]: pgmap v110: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 622 op/s 2024-05-09T02:14:54.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:54 smithi059 ceph-mon[34781]: pgmap v111: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.8 MiB/s wr, 631 op/s 2024-05-09T02:14:54.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:54 smithi196 ceph-mon[34718]: pgmap v111: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.8 MiB/s wr, 631 op/s 2024-05-09T02:14:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:54 smithi019 ceph-mon[30151]: pgmap v111: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.8 MiB/s wr, 631 op/s 2024-05-09T02:14:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:56 smithi059 ceph-mon[34781]: pgmap v112: 97 pgs: 97 active+clean; 2.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 970 op/s 2024-05-09T02:14:56.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:56 smithi196 ceph-mon[34718]: pgmap v112: 97 pgs: 97 active+clean; 2.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 970 op/s 2024-05-09T02:14:56.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:56 smithi019 ceph-mon[30151]: pgmap v112: 97 pgs: 97 active+clean; 2.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 970 op/s 2024-05-09T02:14:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 195.0 s, 3281.2 tps, lat 30.465 ms stddev 32.950 2024-05-09T02:14:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:14:58 smithi059 ceph-mon[34781]: pgmap v113: 97 pgs: 97 active+clean; 2.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.6 MiB/s wr, 684 op/s 2024-05-09T02:14:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:14:58 smithi196 ceph-mon[34718]: pgmap v113: 97 pgs: 97 active+clean; 2.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.6 MiB/s wr, 684 op/s 2024-05-09T02:14:58.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:14:58 smithi019 ceph-mon[30151]: pgmap v113: 97 pgs: 97 active+clean; 2.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.6 MiB/s wr, 684 op/s 2024-05-09T02:15:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:00 smithi059 ceph-mon[34781]: pgmap v114: 97 pgs: 97 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 991 op/s 2024-05-09T02:15:00.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:00 smithi196 ceph-mon[34718]: pgmap v114: 97 pgs: 97 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 991 op/s 2024-05-09T02:15:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:00 smithi019 ceph-mon[30151]: pgmap v114: 97 pgs: 97 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 991 op/s 2024-05-09T02:15:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 200.0 s, 1850.4 tps, lat 53.352 ms stddev 117.978 2024-05-09T02:15:02.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:02 smithi059 ceph-mon[34781]: pgmap v115: 97 pgs: 97 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 661 op/s 2024-05-09T02:15:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:02 smithi196 ceph-mon[34718]: pgmap v115: 97 pgs: 97 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 661 op/s 2024-05-09T02:15:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:02 smithi019 ceph-mon[30151]: pgmap v115: 97 pgs: 97 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 661 op/s 2024-05-09T02:15:04.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:04 smithi059 ceph-mon[34781]: pgmap v116: 97 pgs: 97 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 682 op/s 2024-05-09T02:15:04.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:04 smithi196 ceph-mon[34718]: pgmap v116: 97 pgs: 97 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 682 op/s 2024-05-09T02:15:04.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:04 smithi019 ceph-mon[30151]: pgmap v116: 97 pgs: 97 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 682 op/s 2024-05-09T02:15:06.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:06 smithi059 ceph-mon[34781]: pgmap v117: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 MiB/s wr, 1.82k op/s 2024-05-09T02:15:06.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:06 smithi196 ceph-mon[34718]: pgmap v117: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 MiB/s wr, 1.82k op/s 2024-05-09T02:15:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:06 smithi019 ceph-mon[30151]: pgmap v117: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 MiB/s wr, 1.82k op/s 2024-05-09T02:15:07.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 205.0 s, 2911.6 tps, lat 34.810 ms stddev 39.926 2024-05-09T02:15:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:08 smithi059 ceph-mon[34781]: pgmap v118: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.48k op/s 2024-05-09T02:15:08.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:08 smithi196 ceph-mon[34718]: pgmap v118: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.48k op/s 2024-05-09T02:15:08.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:08 smithi019 ceph-mon[30151]: pgmap v118: 97 pgs: 97 active+clean; 2.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.48k op/s 2024-05-09T02:15:09.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:09 smithi059 ceph-mon[34781]: pgmap v119: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 MiB/s wr, 1.78k op/s 2024-05-09T02:15:09.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:09 smithi196 ceph-mon[34718]: pgmap v119: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 MiB/s wr, 1.78k op/s 2024-05-09T02:15:09.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:09 smithi019 ceph-mon[30151]: pgmap v119: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 MiB/s wr, 1.78k op/s 2024-05-09T02:15:12.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:11 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:15:12.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:11 smithi196 ceph-mon[34718]: pgmap v120: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.48k op/s 2024-05-09T02:15:12.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:11 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:15:12.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:11 smithi019 ceph-mon[30151]: pgmap v120: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.48k op/s 2024-05-09T02:15:12.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:11 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:15:12.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:11 smithi059 ceph-mon[34781]: pgmap v120: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.48k op/s 2024-05-09T02:15:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 210.0 s, 2803.8 tps, lat 35.293 ms stddev 41.576 2024-05-09T02:15:14.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:14 smithi196 ceph-mon[34718]: pgmap v121: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 MiB/s wr, 1.48k op/s 2024-05-09T02:15:14.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:14 smithi019 ceph-mon[30151]: pgmap v121: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 MiB/s wr, 1.48k op/s 2024-05-09T02:15:14.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:14 smithi059 ceph-mon[34781]: pgmap v121: 97 pgs: 97 active+clean; 2.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 MiB/s wr, 1.48k op/s 2024-05-09T02:15:16.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:16 smithi196 ceph-mon[34718]: pgmap v122: 97 pgs: 97 active+clean; 2.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 54 MiB/s wr, 1.87k op/s 2024-05-09T02:15:16.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:16 smithi019 ceph-mon[30151]: pgmap v122: 97 pgs: 97 active+clean; 2.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 54 MiB/s wr, 1.87k op/s 2024-05-09T02:15:16.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:16 smithi059 ceph-mon[34781]: pgmap v122: 97 pgs: 97 active+clean; 2.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 54 MiB/s wr, 1.87k op/s 2024-05-09T02:15:17.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:17 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:15:17.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:17 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:15:17.656 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 215.0 s, 2706.3 tps, lat 30.545 ms stddev 39.014 2024-05-09T02:15:17.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:17 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:15:18.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:18 smithi196 ceph-mon[34718]: pgmap v123: 97 pgs: 97 active+clean; 2.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 719 op/s 2024-05-09T02:15:18.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:18 smithi019 ceph-mon[30151]: pgmap v123: 97 pgs: 97 active+clean; 2.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 719 op/s 2024-05-09T02:15:18.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:18 smithi059 ceph-mon[34781]: pgmap v123: 97 pgs: 97 active+clean; 2.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 719 op/s 2024-05-09T02:15:19.606 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:15:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:15:20.284 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:20 smithi019 ceph-mon[30151]: pgmap v124: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 MiB/s wr, 1.13k op/s 2024-05-09T02:15:20.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:20 smithi196 ceph-mon[34718]: pgmap v124: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 MiB/s wr, 1.13k op/s 2024-05-09T02:15:20.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:20 smithi059 ceph-mon[34781]: pgmap v124: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 MiB/s wr, 1.13k op/s 2024-05-09T02:15:21.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.848+0000 7f5f6fd11700 1 -- 172.21.15.19:0/3377347074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f68110650 msgr2=0x7f5f68110a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:21.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.848+0000 7f5f6fd11700 1 --2- 172.21.15.19:0/3377347074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f68110650 0x7f5f68110a60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0099c0 tx=0x7f5f5c009cd0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.848+0000 7f5f6fd11700 1 -- 172.21.15.19:0/3377347074 shutdown_connections 2024-05-09T02:15:21.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.848+0000 7f5f6fd11700 1 --2- 172.21.15.19:0/3377347074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f68110fa0 0x7f5f68118900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.848+0000 7f5f6fd11700 1 --2- 172.21.15.19:0/3377347074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f6810c830 0x7f5f6810cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.848+0000 7f5f6fd11700 1 --2- 172.21.15.19:0/3377347074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f68110650 0x7f5f68110a60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.848+0000 7f5f6fd11700 1 -- 172.21.15.19:0/3377347074 >> 172.21.15.19:0/3377347074 conn(0x7f5f6806f990 msgr2=0x7f5f6806fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:21.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.848+0000 7f5f6fd11700 1 -- 172.21.15.19:0/3377347074 shutdown_connections 2024-05-09T02:15:21.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.848+0000 7f5f6fd11700 1 -- 172.21.15.19:0/3377347074 wait complete. 2024-05-09T02:15:21.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.849+0000 7f5f6fd11700 1 Processor -- start 2024-05-09T02:15:21.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.849+0000 7f5f6fd11700 1 -- start start 2024-05-09T02:15:21.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.850+0000 7f5f6fd11700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f6810c830 0x7f5f681a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.850+0000 7f5f6fd11700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f68110650 0x7f5f681a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.850+0000 7f5f6fd11700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f68110fa0 0x7f5f681a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.850+0000 7f5f6fd11700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6811bc60 con 0x7f5f68110650 2024-05-09T02:15:21.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.850+0000 7f5f6fd11700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6811bae0 con 0x7f5f68110fa0 2024-05-09T02:15:21.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.850+0000 7f5f6fd11700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6811bde0 con 0x7f5f6810c830 2024-05-09T02:15:21.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.850+0000 7f5f6daad700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f6810c830 0x7f5f681a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.850+0000 7f5f6daad700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f6810c830 0x7f5f681a44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50364/0 (socket says 172.21.15.19:50364) 2024-05-09T02:15:21.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.850+0000 7f5f6daad700 1 -- 172.21.15.19:0/3083245818 learned_addr learned my addr 172.21.15.19:0/3083245818 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:21.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.850+0000 7f5f6d2ac700 1 --2- 172.21.15.19:0/3083245818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f68110650 0x7f5f681a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.850+0000 7f5f6e2ae700 1 --2- 172.21.15.19:0/3083245818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f68110fa0 0x7f5f681a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.850+0000 7f5f6daad700 1 -- 172.21.15.19:0/3083245818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f68110fa0 msgr2=0x7f5f681a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:21.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.850+0000 7f5f6daad700 1 --2- 172.21.15.19:0/3083245818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f68110fa0 0x7f5f681a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.850+0000 7f5f6daad700 1 -- 172.21.15.19:0/3083245818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f68110650 msgr2=0x7f5f681a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:21.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.850+0000 7f5f6daad700 1 --2- 172.21.15.19:0/3083245818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f68110650 0x7f5f681a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.851+0000 7f5f6daad700 1 -- 172.21.15.19:0/3083245818 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f681af830 con 0x7f5f6810c830 2024-05-09T02:15:21.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.851+0000 7f5f6e2ae700 1 --2- 172.21.15.19:0/3083245818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f68110fa0 0x7f5f681a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:15:21.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.851+0000 7f5f6daad700 1 --2- 172.21.15.19:0/3083245818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f6810c830 0x7f5f681a44e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800efc0 tx=0x7f5f5800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:21.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.851+0000 7f5f66ffd700 1 -- 172.21.15.19:0/3083245818 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58003a00 con 0x7f5f6810c830 2024-05-09T02:15:21.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.851+0000 7f5f66ffd700 1 -- 172.21.15.19:0/3083245818 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f58003ba0 con 0x7f5f6810c830 2024-05-09T02:15:21.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.851+0000 7f5f66ffd700 1 -- 172.21.15.19:0/3083245818 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58004770 con 0x7f5f6810c830 2024-05-09T02:15:21.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.851+0000 7f5f6fd11700 1 -- 172.21.15.19:0/3083245818 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f681a2330 con 0x7f5f6810c830 2024-05-09T02:15:21.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.851+0000 7f5f6fd11700 1 -- 172.21.15.19:0/3083245818 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f681a27f0 con 0x7f5f6810c830 2024-05-09T02:15:21.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.856+0000 7f5f66ffd700 1 -- 172.21.15.19:0/3083245818 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5f580135c0 con 0x7f5f6810c830 2024-05-09T02:15:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.858+0000 7f5f66ffd700 1 --2- 172.21.15.19:0/3083245818 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5f500b2b80 0x7f5f500b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.858+0000 7f5f7e727b80 1 Processor -- start 2024-05-09T02:15:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.858+0000 7f5f66ffd700 1 -- 172.21.15.19:0/3083245818 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f5f580e6650 con 0x7f5f6810c830 2024-05-09T02:15:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.858+0000 7f5f6d2ac700 1 --2- 172.21.15.19:0/3083245818 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5f500b2b80 0x7f5f500b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.858+0000 7f5f7e727b80 1 -- start start 2024-05-09T02:15:21.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.859+0000 7f5f7e727b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5ea3160 0x5592a5f44d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.859+0000 7f5f7e727b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5db94a0 0x5592a5f45240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.859+0000 7f5f7e727b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5f3eeb0 0x5592a5f3f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.859+0000 7f5f7e727b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5592a5e58ff0 con 0x5592a5db94a0 2024-05-09T02:15:21.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.859+0000 7f5f6d2ac700 1 --2- 172.21.15.19:0/3083245818 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5f500b2b80 0x7f5f500b5090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009990 tx=0x7f5f5c01a040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:21.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.859+0000 7f5f7e727b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5592a5df77f0 con 0x5592a5f3eeb0 2024-05-09T02:15:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.859+0000 7f5f6daad700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5ea3160 0x5592a5f44d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.859+0000 7f5f7e727b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592a5e54fb0 con 0x5592a5ea3160 2024-05-09T02:15:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.859+0000 7f5f6daad700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5ea3160 0x5592a5f44d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50380/0 (socket says 172.21.15.19:50380) 2024-05-09T02:15:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.859+0000 7f5f6daad700 1 -- 172.21.15.19:0/1461065608 learned_addr learned my addr 172.21.15.19:0/1461065608 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.859+0000 7f5f6d2ac700 1 --2- 172.21.15.19:0/1461065608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5db94a0 0x5592a5f45240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.859+0000 7f5f6e2ae700 1 --2- 172.21.15.19:0/1461065608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5f3eeb0 0x5592a5f3f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.859+0000 7f5f6daad700 1 -- 172.21.15.19:0/1461065608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5f3eeb0 msgr2=0x5592a5f3f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.859+0000 7f5f6daad700 1 --2- 172.21.15.19:0/1461065608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5f3eeb0 0x5592a5f3f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.859+0000 7f5f6daad700 1 -- 172.21.15.19:0/1461065608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5db94a0 msgr2=0x5592a5f45240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.860+0000 7f5f6daad700 1 --2- 172.21.15.19:0/1461065608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5db94a0 0x5592a5f45240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.860+0000 7f5f6daad700 1 -- 172.21.15.19:0/1461065608 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592a5def370 con 0x5592a5ea3160 2024-05-09T02:15:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.860+0000 7f5f6d2ac700 1 --2- 172.21.15.19:0/1461065608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5db94a0 0x5592a5f45240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:15:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.860+0000 7f5f6daad700 1 --2- 172.21.15.19:0/1461065608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5ea3160 0x5592a5f44d00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5f580ae8d0 tx=0x7f5f580ae290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df8cf3e4bf9024fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.860+0000 7f5f4ffff700 1 -- 172.21.15.19:0/1461065608 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f580b70f0 con 0x5592a5ea3160 2024-05-09T02:15:21.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.860+0000 7f5f4ffff700 1 -- 172.21.15.19:0/1461065608 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f580ae450 con 0x5592a5ea3160 2024-05-09T02:15:21.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.860+0000 7f5f4ffff700 1 -- 172.21.15.19:0/1461065608 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f580d00d0 con 0x5592a5ea3160 2024-05-09T02:15:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.862+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1461065608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5ea3160 msgr2=0x5592a5f44d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.862+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/1461065608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5ea3160 0x5592a5f44d00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5f580ae8d0 tx=0x7f5f580ae290 comp rx=0 tx=0).stop 2024-05-09T02:15:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.862+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1461065608 shutdown_connections 2024-05-09T02:15:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.862+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/1461065608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5f3eeb0 0x5592a5f3f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.862+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/1461065608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5ea3160 0x5592a5f44d00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.862+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/1461065608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5db94a0 0x5592a5f45240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.862+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1461065608 >> 172.21.15.19:0/1461065608 conn(0x5592a5e196a0 msgr2=0x5592a5f31400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.862+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1461065608 shutdown_connections 2024-05-09T02:15:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.862+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1461065608 wait complete. 2024-05-09T02:15:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.863+0000 7f5f7e727b80 1 -- start start 2024-05-09T02:15:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.863+0000 7f5f7e727b80 1 Processor -- start 2024-05-09T02:15:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.864+0000 7f5f7e727b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5db94a0 0x5592a5f48a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.864+0000 7f5f7e727b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 0x5592a5f48f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.864+0000 7f5f7e727b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5f3eeb0 0x5592a5f494c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.864+0000 7f5f6daad700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5db94a0 0x5592a5f48a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.864+0000 7f5f6daad700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5db94a0 0x5592a5f48a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50394/0 (socket says 172.21.15.19:50394) 2024-05-09T02:15:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.864+0000 7f5f6daad700 1 -- 172.21.15.19:0/1986140841 learned_addr learned my addr 172.21.15.19:0/1986140841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.864+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1986140841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5592a5cd4ba0 con 0x5592a5f3eeb0 2024-05-09T02:15:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.864+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1986140841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5592a5e13150 con 0x5592a5ea3160 2024-05-09T02:15:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.864+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1986140841 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592a5f4b9e0 con 0x5592a5db94a0 2024-05-09T02:15:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.864+0000 7f5f6d2ac700 1 --2- 172.21.15.19:0/1986140841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 0x5592a5f48f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.864+0000 7f5f7e727b80 10 client.0 _ll_register_callbacks cb 0x5592a5cca6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.865+0000 7f5f6e2ae700 1 --2- 172.21.15.19:0/1986140841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5f3eeb0 0x5592a5f494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.865+0000 7f5f6daad700 1 -- 172.21.15.19:0/1986140841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 msgr2=0x5592a5f48f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:21.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.865+0000 7f5f6daad700 1 --2- 172.21.15.19:0/1986140841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 0x5592a5f48f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.865+0000 7f5f6daad700 1 -- 172.21.15.19:0/1986140841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5f3eeb0 msgr2=0x5592a5f494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:21.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.865+0000 7f5f6daad700 1 --2- 172.21.15.19:0/1986140841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5f3eeb0 0x5592a5f494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.865+0000 7f5f6daad700 1 -- 172.21.15.19:0/1986140841 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592a5cd4490 con 0x5592a5db94a0 2024-05-09T02:15:21.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.865+0000 7f5f6daad700 1 --2- 172.21.15.19:0/1986140841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5db94a0 0x5592a5f48a40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009f90 tx=0x7f5f5800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e42a0dcf795503a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:21.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.865+0000 7f5f4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:15:21.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.866+0000 7f5f4d7fa700 1 -- 172.21.15.19:0/1986140841 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f580c7060 con 0x5592a5db94a0 2024-05-09T02:15:21.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.866+0000 7f5f4d7fa700 1 -- 172.21.15.19:0/1986140841 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f5f580aaf30 con 0x5592a5db94a0 2024-05-09T02:15:21.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.866+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1986140841 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f580ae450 con 0x5592a5db94a0 2024-05-09T02:15:21.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.866+0000 7f5f4d7fa700 1 -- 172.21.15.19:0/1986140841 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5f580ae940 con 0x5592a5db94a0 2024-05-09T02:15:21.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.866+0000 7f5f7e727b80 10 client.25558 fetch_fsmap learned FSMap version 38 2024-05-09T02:15:21.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.866+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1986140841 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5592a5f561a0 con 0x5592a5db94a0 2024-05-09T02:15:21.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.867+0000 7f5f4d7fa700 1 -- 172.21.15.19:0/1986140841 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f580ae450 con 0x5592a5db94a0 2024-05-09T02:15:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.868+0000 7f5f4d7fa700 1 -- 172.21.15.19:0/1986140841 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f5f580b7310 con 0x5592a5db94a0 2024-05-09T02:15:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.868+0000 7f5f7e727b80 10 client.25558 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T02:15:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.868+0000 7f5f7e727b80 10 client.25558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:15:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.868+0000 7f5f7e727b80 20 client.25558 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.868+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/1986140841 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5592a5f58710 0x5592a5f5abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.868+0000 7f5f7e727b80 4 client.25558 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:15:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.868+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1986140841 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592a5e669d0 con 0x5592a5f58710 2024-05-09T02:15:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.868+0000 7f5f6d2ac700 1 --2- 172.21.15.19:0/1986140841 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5592a5f58710 0x5592a5f5abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.868+0000 7f5f6d2ac700 1 --2- 172.21.15.19:0/1986140841 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5592a5f58710 0x5592a5f5abe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbd2bb67a0d43882 server_cookie=dd2cb3a04eda8141 in_seq=0 out_seq=0 2024-05-09T02:15:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.868+0000 7f5f4d7fa700 10 client.25558 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:21.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.869+0000 7f5f4d7fa700 1 -- 172.21.15.19:0/1986140841 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592a5e669d0 con 0x5592a5f58710 2024-05-09T02:15:21.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.869+0000 7f5f4d7fa700 10 client.25558 handle_command_reply: tid=0 2024-05-09T02:15:21.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.869+0000 7f5f7e727b80 1 client.25558 shutdown 2024-05-09T02:15:21.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.869+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1986140841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5db94a0 msgr2=0x5592a5f48a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:21.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.869+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/1986140841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5db94a0 0x5592a5f48a40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009f90 tx=0x7f5f5800c8c0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.870+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1986140841 shutdown_connections 2024-05-09T02:15:21.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.870+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/1986140841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5f3eeb0 0x5592a5f494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.870+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/1986140841 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5592a5f58710 0x5592a5f5abe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.870+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/1986140841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5db94a0 0x5592a5f48a40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.870+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/1986140841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 0x5592a5f48f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.870+0000 7f5f4d7fa700 0 client.25558 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:21.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.870+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1986140841 >> 172.21.15.19:0/1986140841 conn(0x5592a5e196a0 msgr2=0x5592a5f311c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:21.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.870+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1986140841 shutdown_connections 2024-05-09T02:15:21.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.872+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1986140841 wait complete. 2024-05-09T02:15:21.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.872+0000 7f5f7e727b80 20 client.25558 trim_cache size 0 max 16384 2024-05-09T02:15:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.912+0000 7f5f7e727b80 1 Processor -- start 2024-05-09T02:15:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.912+0000 7f5f7e727b80 1 -- start start 2024-05-09T02:15:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.913+0000 7f5f7e727b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5db94a0 0x5592a5f476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.913+0000 7f5f7e727b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 0x5592a5f49300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.913+0000 7f5f7e727b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5f3eeb0 0x5592a5f42d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.913+0000 7f5f7e727b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5400d630 con 0x5592a5f3eeb0 2024-05-09T02:15:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.913+0000 7f5f7e727b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5400f1a0 con 0x5592a5ea3160 2024-05-09T02:15:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.913+0000 7f5f7e727b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5400dd00 con 0x5592a5db94a0 2024-05-09T02:15:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.913+0000 7f5f6e2ae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5f3eeb0 0x5592a5f42d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.913+0000 7f5f6e2ae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5f3eeb0 0x5592a5f42d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55872/0 (socket says 172.21.15.19:55872) 2024-05-09T02:15:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.913+0000 7f5f6e2ae700 1 -- 172.21.15.19:0/1095312760 learned_addr learned my addr 172.21.15.19:0/1095312760 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:21.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.914+0000 7f5f6daad700 1 --2- 172.21.15.19:0/1095312760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5db94a0 0x5592a5f476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.915+0000 7f5f6d2ac700 1 --2- 172.21.15.19:0/1095312760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 0x5592a5f49300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.916+0000 7f5f6e2ae700 1 -- 172.21.15.19:0/1095312760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5db94a0 msgr2=0x5592a5f476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.916+0000 7f5f6e2ae700 1 --2- 172.21.15.19:0/1095312760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5db94a0 0x5592a5f476a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.916+0000 7f5f6e2ae700 1 -- 172.21.15.19:0/1095312760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 msgr2=0x5592a5f49300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.916+0000 7f5f6e2ae700 1 --2- 172.21.15.19:0/1095312760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 0x5592a5f49300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.916+0000 7f5f6e2ae700 1 -- 172.21.15.19:0/1095312760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592a5f2af60 con 0x5592a5f3eeb0 2024-05-09T02:15:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.916+0000 7f5f6e2ae700 1 --2- 172.21.15.19:0/1095312760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5f3eeb0 0x5592a5f42d20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5f60004600 tx=0x7f5f6000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7df781d012dc99 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.918+0000 7f5f4f7fe700 1 -- 172.21.15.19:0/1095312760 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60003680 con 0x5592a5f3eeb0 2024-05-09T02:15:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.918+0000 7f5f4f7fe700 1 -- 172.21.15.19:0/1095312760 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f60003cd0 con 0x5592a5f3eeb0 2024-05-09T02:15:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.918+0000 7f5f4f7fe700 1 -- 172.21.15.19:0/1095312760 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60011dd0 con 0x5592a5f3eeb0 2024-05-09T02:15:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.919+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1095312760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5f3eeb0 msgr2=0x5592a5f42d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.919+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/1095312760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5f3eeb0 0x5592a5f42d20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5f60004600 tx=0x7f5f6000da90 comp rx=0 tx=0).stop 2024-05-09T02:15:21.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.920+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1095312760 shutdown_connections 2024-05-09T02:15:21.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.920+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/1095312760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5f3eeb0 0x5592a5f42d20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.920+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/1095312760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5db94a0 0x5592a5f476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.920+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/1095312760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 0x5592a5f49300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.920+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1095312760 >> 172.21.15.19:0/1095312760 conn(0x5592a5e196a0 msgr2=0x5592a5f42610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.925+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1095312760 shutdown_connections 2024-05-09T02:15:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.929+0000 7f5f7e727b80 1 -- 172.21.15.19:0/1095312760 wait complete. 2024-05-09T02:15:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.929+0000 7f5f7e727b80 1 -- start start 2024-05-09T02:15:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.929+0000 7f5f7e727b80 1 Processor -- start 2024-05-09T02:15:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.930+0000 7f5f7e727b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5db94a0 0x5592a5f51250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.931+0000 7f5f7e727b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 0x5592a5f5fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.931+0000 7f5f7e727b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5f3eeb0 0x5592a5f60480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.931+0000 7f5f7e727b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5400ead0 con 0x5592a5db94a0 2024-05-09T02:15:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.931+0000 7f5f7e727b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5400e3d0 con 0x5592a5ea3160 2024-05-09T02:15:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.931+0000 7f5f7e727b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5400b5d0 con 0x5592a5f3eeb0 2024-05-09T02:15:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.932+0000 7f5f6e2ae700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5f3eeb0 0x5592a5f60480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.932+0000 7f5f7e727b80 10 client.0 _ll_register_callbacks cb 0x5592a5cca6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.932+0000 7f5f6d2ac700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 0x5592a5f5fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.932+0000 7f5f6e2ae700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5f3eeb0 0x5592a5f60480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50406/0 (socket says 172.21.15.19:50406) 2024-05-09T02:15:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.932+0000 7f5f6d2ac700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 0x5592a5f5fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:45368/0 (socket says 172.21.15.19:45368) 2024-05-09T02:15:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.932+0000 7f5f6e2ae700 1 -- 172.21.15.19:0/3034223581 learned_addr learned my addr 172.21.15.19:0/3034223581 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.932+0000 7f5f6e2ae700 1 -- 172.21.15.19:0/3034223581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 msgr2=0x5592a5f5fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.932+0000 7f5f6e2ae700 1 --2- 172.21.15.19:0/3034223581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 0x5592a5f5fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.932+0000 7f5f6e2ae700 1 -- 172.21.15.19:0/3034223581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5db94a0 msgr2=0x5592a5f51250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.932+0000 7f5f6e2ae700 1 --2- 172.21.15.19:0/3034223581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5db94a0 0x5592a5f51250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.932+0000 7f5f6e2ae700 1 -- 172.21.15.19:0/3034223581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592a5f33180 con 0x5592a5f3eeb0 2024-05-09T02:15:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.933+0000 7f5f6e2ae700 1 --2- 172.21.15.19:0/3034223581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5f3eeb0 0x5592a5f60480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000d4d0 tx=0x7f5f60011d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86359676377e3551 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.933+0000 7f5f4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:15:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.933+0000 7f5f4d7fa700 1 -- 172.21.15.19:0/3034223581 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60011740 con 0x5592a5f3eeb0 2024-05-09T02:15:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.934+0000 7f5f4d7fa700 1 -- 172.21.15.19:0/3034223581 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f5f60018040 con 0x5592a5f3eeb0 2024-05-09T02:15:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.934+0000 7f5f7e727b80 1 -- 172.21.15.19:0/3034223581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f60003cd0 con 0x5592a5f3eeb0 2024-05-09T02:15:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.934+0000 7f5f4d7fa700 1 -- 172.21.15.19:0/3034223581 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5f60032050 con 0x5592a5f3eeb0 2024-05-09T02:15:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.936+0000 7f5f7e727b80 10 client.25561 fetch_fsmap learned FSMap version 38 2024-05-09T02:15:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.936+0000 7f5f7e727b80 1 -- 172.21.15.19:0/3034223581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5592a5f3b540 con 0x5592a5f3eeb0 2024-05-09T02:15:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.936+0000 7f5f4d7fa700 1 -- 172.21.15.19:0/3034223581 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f60029070 con 0x5592a5f3eeb0 2024-05-09T02:15:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.936+0000 7f5f4d7fa700 1 -- 172.21.15.19:0/3034223581 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f5f60005050 con 0x5592a5f3eeb0 2024-05-09T02:15:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.936+0000 7f5f7e727b80 10 client.25561 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T02:15:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.936+0000 7f5f7e727b80 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:15:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.937+0000 7f5f7e727b80 20 client.25561 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.937+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/3034223581 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5592a5f58710 0x5592a5fcdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.937+0000 7f5f7e727b80 4 client.25561 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:15:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.937+0000 7f5f7e727b80 1 -- 172.21.15.19:0/3034223581 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5f60003cd0 con 0x5592a5f58710 2024-05-09T02:15:21.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.937+0000 7f5f6daad700 1 --2- 172.21.15.19:0/3034223581 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5592a5f58710 0x5592a5fcdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:21.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.937+0000 7f5f4d7fa700 10 client.25561 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:21.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.938+0000 7f5f6daad700 1 --2- 172.21.15.19:0/3034223581 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5592a5f58710 0x5592a5fcdf30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dda5383b1d8c278b server_cookie=13a1b02e69e50ebb in_seq=0 out_seq=0 2024-05-09T02:15:21.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.938+0000 7f5f4d7fa700 1 -- 172.21.15.19:0/3034223581 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f5f60003cd0 con 0x5592a5f58710 2024-05-09T02:15:21.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.938+0000 7f5f4d7fa700 10 client.25561 handle_command_reply: tid=0 2024-05-09T02:15:21.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.938+0000 7f5f7e727b80 1 client.25561 shutdown 2024-05-09T02:15:21.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.939+0000 7f5f7e727b80 1 -- 172.21.15.19:0/3034223581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5f3eeb0 msgr2=0x5592a5f60480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.939+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/3034223581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5f3eeb0 0x5592a5f60480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000d4d0 tx=0x7f5f60011d30 comp rx=0 tx=0).stop 2024-05-09T02:15:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.939+0000 7f5f7e727b80 1 -- 172.21.15.19:0/3034223581 shutdown_connections 2024-05-09T02:15:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.939+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/3034223581 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5592a5f58710 0x5592a5fcdf30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.939+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/3034223581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592a5f3eeb0 0x5592a5f60480 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.939+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/3034223581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592a5db94a0 0x5592a5f51250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.939+0000 7f5f7e727b80 1 --2- 172.21.15.19:0/3034223581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592a5ea3160 0x5592a5f5fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.939+0000 7f5f7e727b80 1 -- 172.21.15.19:0/3034223581 >> 172.21.15.19:0/3034223581 conn(0x5592a5e196a0 msgr2=0x5592a5f42610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.939+0000 7f5f4d7fa700 0 client.25561 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:21.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.939+0000 7f5f7e727b80 1 -- 172.21.15.19:0/3034223581 shutdown_connections 2024-05-09T02:15:21.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.939+0000 7f5f7e727b80 1 -- 172.21.15.19:0/3034223581 wait complete. 2024-05-09T02:15:21.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.940+0000 7f5f7e727b80 20 client.25561 trim_cache size 0 max 16384 2024-05-09T02:15:21.953 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:15:21.953 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T02:15:21.954 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T02:15:21.954 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:15:21.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.940+0000 7f5f6fd11700 1 -- 172.21.15.19:0/3083245818 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5f500b2b80 msgr2=0x7f5f500b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:21.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.940+0000 7f5f6fd11700 1 --2- 172.21.15.19:0/3083245818 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5f500b2b80 0x7f5f500b5090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009990 tx=0x7f5f5c01a040 comp rx=0 tx=0).stop 2024-05-09T02:15:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.940+0000 7f5f6fd11700 1 -- 172.21.15.19:0/3083245818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f6810c830 msgr2=0x7f5f681a44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:21.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.940+0000 7f5f6fd11700 1 --2- 172.21.15.19:0/3083245818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f6810c830 0x7f5f681a44e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800efc0 tx=0x7f5f5800c3a0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.940+0000 7f5f6fd11700 1 -- 172.21.15.19:0/3083245818 shutdown_connections 2024-05-09T02:15:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.940+0000 7f5f6fd11700 1 --2- 172.21.15.19:0/3083245818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f68110fa0 0x7f5f681a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.940+0000 7f5f6fd11700 1 --2- 172.21.15.19:0/3083245818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f6810c830 0x7f5f681a44e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.940+0000 7f5f6fd11700 1 --2- 172.21.15.19:0/3083245818 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5f500b2b80 0x7f5f500b5090 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.940+0000 7f5f6fd11700 1 --2- 172.21.15.19:0/3083245818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f68110650 0x7f5f681a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.940+0000 7f5f6fd11700 1 -- 172.21.15.19:0/3083245818 >> 172.21.15.19:0/3083245818 conn(0x7f5f6806f990 msgr2=0x7f5f6810e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.941+0000 7f5f6fd11700 1 -- 172.21.15.19:0/3083245818 shutdown_connections 2024-05-09T02:15:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:21.941+0000 7f5f6fd11700 1 -- 172.21.15.19:0/3083245818 wait complete. 2024-05-09T02:15:22.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:22 smithi196 ceph-mon[34718]: pgmap v125: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 MiB/s wr, 825 op/s 2024-05-09T02:15:22.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:22 smithi019 ceph-mon[30151]: pgmap v125: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 MiB/s wr, 825 op/s 2024-05-09T02:15:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 220.0 s, 799.9 tps, lat 147.748 ms stddev 691.831 2024-05-09T02:15:22.706 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T02:15:22.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:15:22.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:22 smithi059 ceph-mon[34781]: pgmap v125: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 MiB/s wr, 825 op/s 2024-05-09T02:15:23.157 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:15:24.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:24 smithi196 ceph-mon[34718]: pgmap v126: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 MiB/s wr, 829 op/s 2024-05-09T02:15:24.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:24 smithi019 ceph-mon[30151]: pgmap v126: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 MiB/s wr, 829 op/s 2024-05-09T02:15:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:24 smithi059 ceph-mon[34781]: pgmap v126: 97 pgs: 97 active+clean; 2.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 MiB/s wr, 829 op/s 2024-05-09T02:15:25.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.704+0000 7fe3e31da700 1 -- 172.21.15.19:0/3948163240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3dc10b2e0 msgr2=0x7fe3dc117810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:25.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.704+0000 7fe3e31da700 1 --2- 172.21.15.19:0/3948163240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3dc10b2e0 0x7fe3dc117810 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400b0a0 tx=0x7fe3d400b3b0 comp rx=0 tx=0).stop 2024-05-09T02:15:25.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.706+0000 7fe3e31da700 1 -- 172.21.15.19:0/3948163240 shutdown_connections 2024-05-09T02:15:25.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.706+0000 7fe3e31da700 1 --2- 172.21.15.19:0/3948163240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3dc10b2e0 0x7fe3dc117810 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:25.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.706+0000 7fe3e31da700 1 --2- 172.21.15.19:0/3948163240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3dc110970 0x7fe3dc110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:25.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.706+0000 7fe3e31da700 1 --2- 172.21.15.19:0/3948163240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3dc10a990 0x7fe3dc10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:25.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.706+0000 7fe3e31da700 1 -- 172.21.15.19:0/3948163240 >> 172.21.15.19:0/3948163240 conn(0x7fe3dc077c80 msgr2=0x7fe3dc0780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:25.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.711+0000 7fe3e31da700 1 -- 172.21.15.19:0/3948163240 shutdown_connections 2024-05-09T02:15:25.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.715+0000 7fe3e31da700 1 -- 172.21.15.19:0/3948163240 wait complete. 2024-05-09T02:15:25.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.716+0000 7fe3e31da700 1 Processor -- start 2024-05-09T02:15:25.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.719+0000 7fe3e31da700 1 -- start start 2024-05-09T02:15:25.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.719+0000 7fe3e31da700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3dc10a990 0x7fe3dc0852f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:25.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.719+0000 7fe3e31da700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3dc10b2e0 0x7fe3dc07e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.719+0000 7fe3e31da700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3dc110970 0x7fe3dc07eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.719+0000 7fe3e31da700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc11ab60 con 0x7fe3dc10b2e0 2024-05-09T02:15:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.719+0000 7fe3e31da700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc11a9e0 con 0x7fe3dc10a990 2024-05-09T02:15:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.719+0000 7fe3e31da700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc11ace0 con 0x7fe3dc110970 2024-05-09T02:15:25.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.721+0000 7fe3e0f76700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3dc10a990 0x7fe3dc0852f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:25.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.721+0000 7fe3e1777700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3dc110970 0x7fe3dc07eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.721+0000 7fe3e1777700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3dc110970 0x7fe3dc07eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50432/0 (socket says 172.21.15.19:50432) 2024-05-09T02:15:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.721+0000 7fe3e1777700 1 -- 172.21.15.19:0/1609644315 learned_addr learned my addr 172.21.15.19:0/1609644315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.721+0000 7fe3dbfff700 1 --2- 172.21.15.19:0/1609644315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3dc10b2e0 0x7fe3dc07e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:25.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.723+0000 7fe3dbfff700 1 -- 172.21.15.19:0/1609644315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3dc110970 msgr2=0x7fe3dc07eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:25.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.723+0000 7fe3dbfff700 1 --2- 172.21.15.19:0/1609644315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3dc110970 0x7fe3dc07eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:25.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.723+0000 7fe3dbfff700 1 -- 172.21.15.19:0/1609644315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3dc10a990 msgr2=0x7fe3dc0852f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:25.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.723+0000 7fe3dbfff700 1 --2- 172.21.15.19:0/1609644315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3dc10a990 0x7fe3dc0852f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:25.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.723+0000 7fe3dbfff700 1 -- 172.21.15.19:0/1609644315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3dc082e20 con 0x7fe3dc10b2e0 2024-05-09T02:15:25.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.724+0000 7fe3dbfff700 1 --2- 172.21.15.19:0/1609644315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3dc10b2e0 0x7fe3dc07e470 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc06d120 tx=0x7fe3d000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:25.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.724+0000 7fe3d9ffb700 1 -- 172.21.15.19:0/1609644315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0016070 con 0x7fe3dc10b2e0 2024-05-09T02:15:25.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.724+0000 7fe3e31da700 1 -- 172.21.15.19:0/1609644315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3dc083110 con 0x7fe3dc10b2e0 2024-05-09T02:15:25.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.725+0000 7fe3e31da700 1 -- 172.21.15.19:0/1609644315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3dc0835f0 con 0x7fe3dc10b2e0 2024-05-09T02:15:25.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.725+0000 7fe3d9ffb700 1 -- 172.21.15.19:0/1609644315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3d000fd40 con 0x7fe3dc10b2e0 2024-05-09T02:15:25.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.725+0000 7fe3d9ffb700 1 -- 172.21.15.19:0/1609644315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d000e3b0 con 0x7fe3dc10b2e0 2024-05-09T02:15:25.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.729+0000 7fe3d9ffb700 1 -- 172.21.15.19:0/1609644315 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe3d0019070 con 0x7fe3dc10b2e0 2024-05-09T02:15:25.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.730+0000 7fe3d9ffb700 1 --2- 172.21.15.19:0/1609644315 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe3c40b2b80 0x7fe3c40b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:25.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.731+0000 7fe3e31da700 1 -- 172.21.15.19:0/1609644315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c8005330 con 0x7fe3dc10b2e0 2024-05-09T02:15:25.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.732+0000 7fe3d9ffb700 1 -- 172.21.15.19:0/1609644315 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fe3d00a5660 con 0x7fe3dc10b2e0 2024-05-09T02:15:25.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.733+0000 7fe3e0f76700 1 --2- 172.21.15.19:0/1609644315 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe3c40b2b80 0x7fe3c40b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.738+0000 7fe3e0f76700 1 --2- 172.21.15.19:0/1609644315 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe3c40b2b80 0x7fe3c40b5090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc07fc70 tx=0x7fe3cc009190 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:25.743+0000 7fe3d9ffb700 1 -- 172.21.15.19:0/1609644315 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe3d00a9770 con 0x7fe3dc10b2e0 2024-05-09T02:15:26.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:26.162+0000 7fe3e31da700 1 -- 172.21.15.19:0/1609644315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3c80058f0 con 0x7fe3dc10b2e0 2024-05-09T02:15:26.169 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:15:26.169 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:15:16.064731+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24457,24455,24473,24463,24484]},"id":1}]} 2024-05-09T02:15:26.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:26.166+0000 7fe3d9ffb700 1 -- 172.21.15.19:0/1609644315 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11918 (secure 0 0 0) 0x7fe3d00a9070 con 0x7fe3dc10b2e0 2024-05-09T02:15:26.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:26.171+0000 7fe3e31da700 1 -- 172.21.15.19:0/1609644315 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe3c40b2b80 msgr2=0x7fe3c40b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:26.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:26.171+0000 7fe3e31da700 1 --2- 172.21.15.19:0/1609644315 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe3c40b2b80 0x7fe3c40b5090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc07fc70 tx=0x7fe3cc009190 comp rx=0 tx=0).stop 2024-05-09T02:15:26.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:26.171+0000 7fe3e31da700 1 -- 172.21.15.19:0/1609644315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3dc10b2e0 msgr2=0x7fe3dc07e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:26.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:26.171+0000 7fe3e31da700 1 --2- 172.21.15.19:0/1609644315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3dc10b2e0 0x7fe3dc07e470 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc06d120 tx=0x7fe3d000dc30 comp rx=0 tx=0).stop 2024-05-09T02:15:26.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:26.173+0000 7fe3e31da700 1 -- 172.21.15.19:0/1609644315 shutdown_connections 2024-05-09T02:15:26.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:26.173+0000 7fe3e31da700 1 --2- 172.21.15.19:0/1609644315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3dc110970 0x7fe3dc07eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:26.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:26.173+0000 7fe3e31da700 1 --2- 172.21.15.19:0/1609644315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3dc10a990 0x7fe3dc0852f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:26.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:26.173+0000 7fe3e31da700 1 --2- 172.21.15.19:0/1609644315 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe3c40b2b80 0x7fe3c40b5090 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:26.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:26.173+0000 7fe3e31da700 1 --2- 172.21.15.19:0/1609644315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3dc10b2e0 0x7fe3dc07e470 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:26.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:26.173+0000 7fe3e31da700 1 -- 172.21.15.19:0/1609644315 >> 172.21.15.19:0/1609644315 conn(0x7fe3dc077c80 msgr2=0x7fe3dc07b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:26.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:26.173+0000 7fe3e31da700 1 -- 172.21.15.19:0/1609644315 shutdown_connections 2024-05-09T02:15:26.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:26.173+0000 7fe3e31da700 1 -- 172.21.15.19:0/1609644315 wait complete. 2024-05-09T02:15:26.177 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 38 2024-05-09T02:15:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:26 smithi196 ceph-mon[34718]: pgmap v127: 97 pgs: 97 active+clean; 2.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 42 MiB/s wr, 1.05k op/s 2024-05-09T02:15:26.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:26 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1609644315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:15:26.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:26 smithi019 ceph-mon[30151]: pgmap v127: 97 pgs: 97 active+clean; 2.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 42 MiB/s wr, 1.05k op/s 2024-05-09T02:15:26.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:26 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1609644315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:15:26.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:26 smithi059 ceph-mon[34781]: pgmap v127: 97 pgs: 97 active+clean; 2.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 42 MiB/s wr, 1.05k op/s 2024-05-09T02:15:26.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:26 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1609644315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:15:27.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:27 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:15:27.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:27 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:15:27.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 225.0 s, 3224.6 tps, lat 30.854 ms stddev 37.618 2024-05-09T02:15:27.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:27 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:15:27.737 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T02:15:28.153 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:15:28.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:28 smithi196 ceph-mon[34718]: pgmap v128: 97 pgs: 97 active+clean; 2.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 646 op/s 2024-05-09T02:15:28.563 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:28 smithi019 ceph-mon[30151]: pgmap v128: 97 pgs: 97 active+clean; 2.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 646 op/s 2024-05-09T02:15:28.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:28 smithi059 ceph-mon[34781]: pgmap v128: 97 pgs: 97 active+clean; 2.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 646 op/s 2024-05-09T02:15:30.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.380+0000 7f79f0eae700 1 -- 172.21.15.19:0/3292640364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79ec10b280 msgr2=0x7f79ec11d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:30.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.380+0000 7f79f0eae700 1 --2- 172.21.15.19:0/3292640364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79ec10b280 0x7f79ec11d3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009f90 tx=0x7f79e400b330 comp rx=0 tx=0).stop 2024-05-09T02:15:30.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.381+0000 7f79f0eae700 1 -- 172.21.15.19:0/3292640364 shutdown_connections 2024-05-09T02:15:30.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.381+0000 7f79f0eae700 1 --2- 172.21.15.19:0/3292640364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec11d930 0x7f79ec11fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.381+0000 7f79f0eae700 1 --2- 172.21.15.19:0/3292640364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79ec10a930 0x7f79ec10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.381+0000 7f79f0eae700 1 --2- 172.21.15.19:0/3292640364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79ec10b280 0x7f79ec11d3f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.381+0000 7f79f0eae700 1 -- 172.21.15.19:0/3292640364 >> 172.21.15.19:0/3292640364 conn(0x7f79ec06f4d0 msgr2=0x7f79ec06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:30.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.382+0000 7f79f0eae700 1 -- 172.21.15.19:0/3292640364 shutdown_connections 2024-05-09T02:15:30.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.382+0000 7f79f0eae700 1 -- 172.21.15.19:0/3292640364 wait complete. 2024-05-09T02:15:30.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.383+0000 7f79f0eae700 1 Processor -- start 2024-05-09T02:15:30.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.383+0000 7f79f0eae700 1 -- start start 2024-05-09T02:15:30.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.384+0000 7f79f0eae700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79ec10a930 0x7f79ec1af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.384+0000 7f79f0eae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79ec10b280 0x7f79ec1aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.384+0000 7f79f0eae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec11d930 0x7f79ec1a9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.384+0000 7f79f0eae700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1232f0 con 0x7f79ec11d930 2024-05-09T02:15:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.384+0000 7f79f0eae700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec123170 con 0x7f79ec10b280 2024-05-09T02:15:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.384+0000 7f79f0eae700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec123470 con 0x7f79ec10a930 2024-05-09T02:15:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.384+0000 7f79ea59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79ec10a930 0x7f79ec1af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.384+0000 7f79e9d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79ec10b280 0x7f79ec1aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.384+0000 7f79ea59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79ec10a930 0x7f79ec1af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:59496/0 (socket says 172.21.15.19:59496) 2024-05-09T02:15:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.384+0000 7f79ea59c700 1 -- 172.21.15.19:0/145489679 learned_addr learned my addr 172.21.15.19:0/145489679 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:30.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.385+0000 7f79e9d9b700 1 -- 172.21.15.19:0/145489679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79ec10a930 msgr2=0x7f79ec1af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:30.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.385+0000 7f79e9d9b700 1 --2- 172.21.15.19:0/145489679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79ec10a930 0x7f79ec1af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.385+0000 7f79e9d9b700 1 -- 172.21.15.19:0/145489679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec11d930 msgr2=0x7f79ec1a9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:30.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.385+0000 7f79ead9d700 1 --2- 172.21.15.19:0/145489679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec11d930 0x7f79ec1a9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.385+0000 7f79e9d9b700 1 --2- 172.21.15.19:0/145489679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec11d930 0x7f79ec1a9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.385+0000 7f79e9d9b700 1 -- 172.21.15.19:0/145489679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ec1aa280 con 0x7f79ec10b280 2024-05-09T02:15:30.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.385+0000 7f79ea59c700 1 --2- 172.21.15.19:0/145489679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79ec10a930 0x7f79ec1af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:15:30.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.385+0000 7f79ead9d700 1 --2- 172.21.15.19:0/145489679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec11d930 0x7f79ec1a9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:15:30.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.386+0000 7f79e9d9b700 1 --2- 172.21.15.19:0/145489679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79ec10b280 0x7f79ec1aff00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009f90 tx=0x7f79e4004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:30.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.386+0000 7f79db7fe700 1 -- 172.21.15.19:0/145489679 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4021070 con 0x7f79ec10b280 2024-05-09T02:15:30.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.386+0000 7f79db7fe700 1 -- 172.21.15.19:0/145489679 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e4007b80 con 0x7f79ec10b280 2024-05-09T02:15:30.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.386+0000 7f79db7fe700 1 -- 172.21.15.19:0/145489679 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e40052a0 con 0x7f79ec10b280 2024-05-09T02:15:30.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.387+0000 7f79f0eae700 1 -- 172.21.15.19:0/145489679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ec1aa510 con 0x7f79ec10b280 2024-05-09T02:15:30.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.387+0000 7f79f0eae700 1 -- 172.21.15.19:0/145489679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ec1b6770 con 0x7f79ec10b280 2024-05-09T02:15:30.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.391+0000 7f79db7fe700 1 -- 172.21.15.19:0/145489679 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f79e4024070 con 0x7f79ec10b280 2024-05-09T02:15:30.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.392+0000 7f79db7fe700 1 --2- 172.21.15.19:0/145489679 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79d40b2bc0 0x7f79d40b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.393+0000 7f79ea59c700 1 --2- 172.21.15.19:0/145489679 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79d40b2bc0 0x7f79d40b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.393+0000 7f79ff8c4b80 1 Processor -- start 2024-05-09T02:15:30.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.393+0000 7f79db7fe700 1 -- 172.21.15.19:0/145489679 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f79e400f020 con 0x7f79ec10b280 2024-05-09T02:15:30.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.393+0000 7f79ff8c4b80 1 -- start start 2024-05-09T02:15:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.393+0000 7f79ea59c700 1 --2- 172.21.15.19:0/145489679 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79d40b2bc0 0x7f79d40b50d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f79dc005950 tx=0x7f79dc00b500 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.393+0000 7f79ff8c4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6083d160 0x563e608d64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.393+0000 7f79ff8c4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e607534a0 0x563e608d6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.393+0000 7f79ff8c4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e608d06a0 0x563e608d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.393+0000 7f79ff8c4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563e607f2ff0 con 0x563e608d06a0 2024-05-09T02:15:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.393+0000 7f79ff8c4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563e607917f0 con 0x563e607534a0 2024-05-09T02:15:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.393+0000 7f79ff8c4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e607eefb0 con 0x563e6083d160 2024-05-09T02:15:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.394+0000 7f79ea59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6083d160 0x563e608d64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.394+0000 7f79e9d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e607534a0 0x563e608d6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.394+0000 7f79ea59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6083d160 0x563e608d64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:59502/0 (socket says 172.21.15.19:59502) 2024-05-09T02:15:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.394+0000 7f79ea59c700 1 -- 172.21.15.19:0/1377641659 learned_addr learned my addr 172.21.15.19:0/1377641659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.394+0000 7f79ead9d700 1 --2- 172.21.15.19:0/1377641659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e608d06a0 0x563e608d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.394+0000 7f79e9d9b700 1 -- 172.21.15.19:0/1377641659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6083d160 msgr2=0x563e608d64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.394+0000 7f79e9d9b700 1 --2- 172.21.15.19:0/1377641659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6083d160 0x563e608d64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.394+0000 7f79e9d9b700 1 -- 172.21.15.19:0/1377641659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e608d06a0 msgr2=0x563e608d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:30.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.394+0000 7f79e9d9b700 1 --2- 172.21.15.19:0/1377641659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e608d06a0 0x563e608d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.394+0000 7f79e9d9b700 1 -- 172.21.15.19:0/1377641659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e60789370 con 0x563e607534a0 2024-05-09T02:15:30.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.394+0000 7f79ea59c700 1 --2- 172.21.15.19:0/1377641659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6083d160 0x563e608d64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:15:30.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.394+0000 7f79e9d9b700 1 --2- 172.21.15.19:0/1377641659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e607534a0 0x563e608d6a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f79e40e1350 tx=0x7f79e400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d59b8ec8592b5a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:30.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.394+0000 7f79d8ff9700 1 -- 172.21.15.19:0/1377641659 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e40e17e0 con 0x563e607534a0 2024-05-09T02:15:30.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.395+0000 7f79d8ff9700 1 -- 172.21.15.19:0/1377641659 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e40af320 con 0x563e607534a0 2024-05-09T02:15:30.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.395+0000 7f79d8ff9700 1 -- 172.21.15.19:0/1377641659 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e40b38b0 con 0x563e607534a0 2024-05-09T02:15:30.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.396+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/1377641659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e607534a0 msgr2=0x563e608d6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:30.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.396+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/1377641659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e607534a0 0x563e608d6a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f79e40e1350 tx=0x7f79e400b330 comp rx=0 tx=0).stop 2024-05-09T02:15:30.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.396+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/1377641659 shutdown_connections 2024-05-09T02:15:30.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.396+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/1377641659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e608d06a0 0x563e608d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.396+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/1377641659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6083d160 0x563e608d64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.396+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/1377641659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e607534a0 0x563e608d6a30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.396+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/1377641659 >> 172.21.15.19:0/1377641659 conn(0x563e607b36a0 msgr2=0x563e608cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:30.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.396+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/1377641659 shutdown_connections 2024-05-09T02:15:30.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.396+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/1377641659 wait complete. 2024-05-09T02:15:30.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.397+0000 7f79ff8c4b80 1 -- start start 2024-05-09T02:15:30.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.397+0000 7f79ff8c4b80 1 Processor -- start 2024-05-09T02:15:30.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.397+0000 7f79ff8c4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e607534a0 0x563e608dfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.397+0000 7f79ff8c4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6083d160 0x563e608e0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.397+0000 7f79ff8c4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e608d06a0 0x563e608da0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.397+0000 7f79ff8c4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563e6066eba0 con 0x563e6083d160 2024-05-09T02:15:30.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.397+0000 7f79ff8c4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563e607ad150 con 0x563e607534a0 2024-05-09T02:15:30.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.397+0000 7f79ff8c4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e608d1360 con 0x563e608d06a0 2024-05-09T02:15:30.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.397+0000 7f79ff8c4b80 10 client.0 _ll_register_callbacks cb 0x563e606646c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:30.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.398+0000 7f79ea59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e607534a0 0x563e608dfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.398+0000 7f79ea59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e607534a0 0x563e608dfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:41916/0 (socket says 172.21.15.19:41916) 2024-05-09T02:15:30.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.398+0000 7f79ea59c700 1 -- 172.21.15.19:0/2328652171 learned_addr learned my addr 172.21.15.19:0/2328652171 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:30.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.398+0000 7f79ead9d700 1 --2- 172.21.15.19:0/2328652171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e608d06a0 0x563e608da0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.398+0000 7f79e9d9b700 1 --2- 172.21.15.19:0/2328652171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6083d160 0x563e608e0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.398+0000 7f79ea59c700 1 -- 172.21.15.19:0/2328652171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e608d06a0 msgr2=0x563e608da0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:30.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.398+0000 7f79ea59c700 1 --2- 172.21.15.19:0/2328652171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e608d06a0 0x563e608da0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.398+0000 7f79ea59c700 1 -- 172.21.15.19:0/2328652171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6083d160 msgr2=0x563e608e0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:30.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.398+0000 7f79ea59c700 1 --2- 172.21.15.19:0/2328652171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6083d160 0x563e608e0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.398+0000 7f79ea59c700 1 -- 172.21.15.19:0/2328652171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6066e490 con 0x563e607534a0 2024-05-09T02:15:30.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.398+0000 7f79e9d9b700 1 --2- 172.21.15.19:0/2328652171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6083d160 0x563e608e0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:15:30.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.398+0000 7f79ead9d700 1 --2- 172.21.15.19:0/2328652171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e608d06a0 0x563e608da0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:15:30.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.398+0000 7f79ea59c700 1 --2- 172.21.15.19:0/2328652171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e607534a0 0x563e608dfce0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f79dc011cf0 tx=0x7f79dc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d6b09c746f35097 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:30.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.398+0000 7f79d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:15:30.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.400+0000 7f79d1ffb700 1 -- 172.21.15.19:0/2328652171 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc01ea50 con 0x563e607534a0 2024-05-09T02:15:30.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.400+0000 7f79d1ffb700 1 -- 172.21.15.19:0/2328652171 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f79dc01ebf0 con 0x563e607534a0 2024-05-09T02:15:30.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.400+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2328652171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e40af320 con 0x563e607534a0 2024-05-09T02:15:30.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.401+0000 7f79d1ffb700 1 -- 172.21.15.19:0/2328652171 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f79dc01edf0 con 0x563e607534a0 2024-05-09T02:15:30.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.401+0000 7f79ff8c4b80 10 client.25523 fetch_fsmap learned FSMap version 39 2024-05-09T02:15:30.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.401+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2328652171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563e608e7750 con 0x563e607534a0 2024-05-09T02:15:30.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.402+0000 7f79d1ffb700 1 -- 172.21.15.19:0/2328652171 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e40af320 con 0x563e607534a0 2024-05-09T02:15:30.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.402+0000 7f79d1ffb700 1 -- 172.21.15.19:0/2328652171 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f79dc028430 con 0x563e607534a0 2024-05-09T02:15:30.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.402+0000 7f79ff8c4b80 10 client.25523 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T02:15:30.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.402+0000 7f79ff8c4b80 10 client.25523 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:15:30.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.402+0000 7f79ff8c4b80 20 client.25523 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:30.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.402+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/2328652171 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563e608e9d30 0x563e608ec150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.402+0000 7f79ff8c4b80 4 client.25523 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:15:30.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.402+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2328652171 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e608009d0 con 0x563e608e9d30 2024-05-09T02:15:30.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.402+0000 7f79e9d9b700 1 --2- 172.21.15.19:0/2328652171 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563e608e9d30 0x563e608ec150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.403+0000 7f79e9d9b700 1 --2- 172.21.15.19:0/2328652171 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563e608e9d30 0x563e608ec150 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=67f7f37def478f50 server_cookie=d81c94ad185aef2c in_seq=0 out_seq=0 2024-05-09T02:15:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.403+0000 7f79d1ffb700 10 client.25523 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:15:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.404+0000 7f79d1ffb700 1 -- 172.21.15.19:0/2328652171 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563e608009d0 con 0x563e608e9d30 2024-05-09T02:15:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.404+0000 7f79d1ffb700 10 client.25523 handle_command_reply: tid=0 2024-05-09T02:15:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.404+0000 7f79ff8c4b80 1 client.25523 shutdown 2024-05-09T02:15:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.404+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2328652171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e607534a0 msgr2=0x563e608dfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.404+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/2328652171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e607534a0 0x563e608dfce0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f79dc011cf0 tx=0x7f79dc0047c0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.404+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2328652171 shutdown_connections 2024-05-09T02:15:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.404+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/2328652171 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563e608e9d30 0x563e608ec150 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.404+0000 7f79d1ffb700 0 client.25523 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:15:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.404+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/2328652171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e608d06a0 0x563e608da0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.404+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/2328652171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e607534a0 0x563e608dfce0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.404+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/2328652171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6083d160 0x563e608e0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.404+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2328652171 >> 172.21.15.19:0/2328652171 conn(0x563e607b36a0 msgr2=0x563e608cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.404+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2328652171 shutdown_connections 2024-05-09T02:15:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.404+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2328652171 wait complete. 2024-05-09T02:15:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.404+0000 7f79ff8c4b80 20 client.25523 trim_cache size 0 max 16384 2024-05-09T02:15:30.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.427+0000 7f79ff8c4b80 1 Processor -- start 2024-05-09T02:15:30.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.428+0000 7f79ff8c4b80 1 -- start start 2024-05-09T02:15:30.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.428+0000 7f79ff8c4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e607534a0 0x563e608d4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.428+0000 7f79ff8c4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6083d160 0x563e608d58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.428+0000 7f79ff8c4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e608d06a0 0x563e608d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.428+0000 7f79ff8c4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1bdd20 con 0x563e6083d160 2024-05-09T02:15:30.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.428+0000 7f79ff8c4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1bf8f0 con 0x563e608d06a0 2024-05-09T02:15:30.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.428+0000 7f79ff8c4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1be3f0 con 0x563e607534a0 2024-05-09T02:15:30.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.429+0000 7f79ead9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e608d06a0 0x563e608d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.429+0000 7f79ead9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e608d06a0 0x563e608d2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:41924/0 (socket says 172.21.15.19:41924) 2024-05-09T02:15:30.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.429+0000 7f79ead9d700 1 -- 172.21.15.19:0/843027757 learned_addr learned my addr 172.21.15.19:0/843027757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:30.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.430+0000 7f79ea59c700 1 --2- 172.21.15.19:0/843027757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e607534a0 0x563e608d4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.430+0000 7f79ead9d700 1 -- 172.21.15.19:0/843027757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e607534a0 msgr2=0x563e608d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:30.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.430+0000 7f79ead9d700 1 --2- 172.21.15.19:0/843027757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e607534a0 0x563e608d4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.430+0000 7f79ead9d700 1 -- 172.21.15.19:0/843027757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6083d160 msgr2=0x563e608d58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:30.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.430+0000 7f79ead9d700 1 --2- 172.21.15.19:0/843027757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6083d160 0x563e608d58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.430+0000 7f79ead9d700 1 -- 172.21.15.19:0/843027757 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e608cced0 con 0x563e608d06a0 2024-05-09T02:15:30.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.430+0000 7f79ea59c700 1 --2- 172.21.15.19:0/843027757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e607534a0 0x563e608d4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:15:30.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.430+0000 7f79ead9d700 1 --2- 172.21.15.19:0/843027757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e608d06a0 0x563e608d2e00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009dc0 tx=0x7f79e0012730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a64f70bd39aa056a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:30.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.430+0000 7f79d3fff700 1 -- 172.21.15.19:0/843027757 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0004030 con 0x563e608d06a0 2024-05-09T02:15:30.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.430+0000 7f79d3fff700 1 -- 172.21.15.19:0/843027757 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e0003680 con 0x563e608d06a0 2024-05-09T02:15:30.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.431+0000 7f79d3fff700 1 -- 172.21.15.19:0/843027757 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e001c6c0 con 0x563e608d06a0 2024-05-09T02:15:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.431+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/843027757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e608d06a0 msgr2=0x563e608d2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.432+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/843027757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e608d06a0 0x563e608d2e00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009dc0 tx=0x7f79e0012730 comp rx=0 tx=0).stop 2024-05-09T02:15:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.432+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/843027757 shutdown_connections 2024-05-09T02:15:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.432+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/843027757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e608d06a0 0x563e608d2e00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.432+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/843027757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e607534a0 0x563e608d4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.432+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/843027757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e6083d160 0x563e608d58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.432+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/843027757 >> 172.21.15.19:0/843027757 conn(0x563e607b36a0 msgr2=0x563e608d51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:30.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.432+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/843027757 shutdown_connections 2024-05-09T02:15:30.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.432+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/843027757 wait complete. 2024-05-09T02:15:30.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.432+0000 7f79ff8c4b80 1 -- start start 2024-05-09T02:15:30.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.433+0000 7f79ff8c4b80 1 Processor -- start 2024-05-09T02:15:30.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.433+0000 7f79ff8c4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e607534a0 0x563e608e14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.433+0000 7f79ff8c4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6083d160 0x563e608e19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.433+0000 7f79ea59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e607534a0 0x563e608e14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.433+0000 7f79ff8c4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e608d06a0 0x563e609524d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.433+0000 7f79ea59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e607534a0 0x563e608e14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37156/0 (socket says 172.21.15.19:37156) 2024-05-09T02:15:30.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.434+0000 7f79ea59c700 1 -- 172.21.15.19:0/2066016882 learned_addr learned my addr 172.21.15.19:0/2066016882 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:30.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.434+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2066016882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1bf220 con 0x563e607534a0 2024-05-09T02:15:30.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.434+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2066016882 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1beac0 con 0x563e608d06a0 2024-05-09T02:15:30.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.434+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2066016882 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1bbcc0 con 0x563e6083d160 2024-05-09T02:15:30.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.434+0000 7f79e9d9b700 1 --2- 172.21.15.19:0/2066016882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6083d160 0x563e608e19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.434+0000 7f79ff8c4b80 10 client.0 _ll_register_callbacks cb 0x563e606646c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:30.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.434+0000 7f79ead9d700 1 --2- 172.21.15.19:0/2066016882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e608d06a0 0x563e609524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.434+0000 7f79e9d9b700 1 -- 172.21.15.19:0/2066016882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e608d06a0 msgr2=0x563e609524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:30.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.434+0000 7f79e9d9b700 1 --2- 172.21.15.19:0/2066016882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e608d06a0 0x563e609524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.434+0000 7f79e9d9b700 1 -- 172.21.15.19:0/2066016882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e607534a0 msgr2=0x563e608e14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:30.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.434+0000 7f79e9d9b700 1 --2- 172.21.15.19:0/2066016882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e607534a0 0x563e608e14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.434+0000 7f79e9d9b700 1 -- 172.21.15.19:0/2066016882 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e60952a40 con 0x563e6083d160 2024-05-09T02:15:30.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.434+0000 7f79ead9d700 1 --2- 172.21.15.19:0/2066016882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e608d06a0 0x563e609524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:15:30.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.434+0000 7f79e9d9b700 1 --2- 172.21.15.19:0/2066016882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6083d160 0x563e608e19f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f79e40e1380 tx=0x7f79e40aae70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=231a3f5013fce46e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:30.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.434+0000 7f79d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:15:30.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.435+0000 7f79d1ffb700 1 -- 172.21.15.19:0/2066016882 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e40c9030 con 0x563e6083d160 2024-05-09T02:15:30.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.435+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2066016882 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e0003680 con 0x563e6083d160 2024-05-09T02:15:30.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.435+0000 7f79d1ffb700 1 -- 172.21.15.19:0/2066016882 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f79e4007d30 con 0x563e6083d160 2024-05-09T02:15:30.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.435+0000 7f79d1ffb700 1 -- 172.21.15.19:0/2066016882 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f79e40af320 con 0x563e6083d160 2024-05-09T02:15:30.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.435+0000 7f79ff8c4b80 10 client.25582 fetch_fsmap learned FSMap version 39 2024-05-09T02:15:30.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.435+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2066016882 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563e60953c40 con 0x563e6083d160 2024-05-09T02:15:30.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.435+0000 7f79d1ffb700 1 -- 172.21.15.19:0/2066016882 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e0003680 con 0x563e6083d160 2024-05-09T02:15:30.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.435+0000 7f79d1ffb700 1 -- 172.21.15.19:0/2066016882 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f79e4011070 con 0x563e6083d160 2024-05-09T02:15:30.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.435+0000 7f79ff8c4b80 10 client.25582 fetch_fsmap finished waiting for FSMap version 39 2024-05-09T02:15:30.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.435+0000 7f79ff8c4b80 10 client.25582 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:15:30.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.435+0000 7f79ff8c4b80 20 client.25582 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:30.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.435+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/2066016882 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563e609541e0 0x563e60956600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:30.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.435+0000 7f79ff8c4b80 4 client.25582 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:15:30.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.435+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2066016882 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e607faa70 con 0x563e609541e0 2024-05-09T02:15:30.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.436+0000 7f79ea59c700 1 --2- 172.21.15.19:0/2066016882 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563e609541e0 0x563e60956600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:30.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.438+0000 7f79ea59c700 1 --2- 172.21.15.19:0/2066016882 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563e609541e0 0x563e60956600 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3b22a73f539a96d0 server_cookie=9e03e023a8d3a409 in_seq=0 out_seq=0 2024-05-09T02:15:30.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.438+0000 7f79d1ffb700 10 client.25582 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:15:30.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:30 smithi019 ceph-mon[30151]: pgmap v129: 97 pgs: 97 active+clean; 3.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 36 MiB/s wr, 966 op/s 2024-05-09T02:15:30.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.438+0000 7f79d1ffb700 1 -- 172.21.15.19:0/2066016882 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e607faa70 con 0x563e609541e0 2024-05-09T02:15:30.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.438+0000 7f79d1ffb700 10 client.25582 handle_command_reply: tid=0 2024-05-09T02:15:30.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.438+0000 7f79ff8c4b80 1 client.25582 shutdown 2024-05-09T02:15:30.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.439+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2066016882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6083d160 msgr2=0x563e608e19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:30.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.439+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/2066016882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6083d160 0x563e608e19f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f79e40e1380 tx=0x7f79e40aae70 comp rx=0 tx=0).stop 2024-05-09T02:15:30.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.439+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2066016882 shutdown_connections 2024-05-09T02:15:30.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.439+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/2066016882 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563e609541e0 0x563e60956600 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.439+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/2066016882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e608d06a0 0x563e609524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.439+0000 7f79d1ffb700 0 client.25582 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:15:30.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.439+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/2066016882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e607534a0 0x563e608e14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.439+0000 7f79ff8c4b80 1 --2- 172.21.15.19:0/2066016882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e6083d160 0x563e608e19f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.439+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2066016882 >> 172.21.15.19:0/2066016882 conn(0x563e607b36a0 msgr2=0x563e608d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:30.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.439+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2066016882 shutdown_connections 2024-05-09T02:15:30.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.439+0000 7f79ff8c4b80 1 -- 172.21.15.19:0/2066016882 wait complete. 2024-05-09T02:15:30.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.440+0000 7f79ff8c4b80 20 client.25582 trim_cache size 0 max 16384 2024-05-09T02:15:30.444 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:15:30.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.440+0000 7f79f0eae700 1 -- 172.21.15.19:0/145489679 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79d40b2bc0 msgr2=0x7f79d40b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:30.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.440+0000 7f79f0eae700 1 --2- 172.21.15.19:0/145489679 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79d40b2bc0 0x7f79d40b50d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f79dc005950 tx=0x7f79dc00b500 comp rx=0 tx=0).stop 2024-05-09T02:15:30.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.440+0000 7f79f0eae700 1 -- 172.21.15.19:0/145489679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79ec10b280 msgr2=0x7f79ec1aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:30.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.440+0000 7f79f0eae700 1 --2- 172.21.15.19:0/145489679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79ec10b280 0x7f79ec1aff00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009f90 tx=0x7f79e4004d50 comp rx=0 tx=0).stop 2024-05-09T02:15:30.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.441+0000 7f79f0eae700 1 -- 172.21.15.19:0/145489679 shutdown_connections 2024-05-09T02:15:30.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.441+0000 7f79f0eae700 1 --2- 172.21.15.19:0/145489679 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79d40b2bc0 0x7f79d40b50d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.441+0000 7f79f0eae700 1 --2- 172.21.15.19:0/145489679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec11d930 0x7f79ec1a9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.441+0000 7f79f0eae700 1 --2- 172.21.15.19:0/145489679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79ec10a930 0x7f79ec1af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.441+0000 7f79f0eae700 1 --2- 172.21.15.19:0/145489679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79ec10b280 0x7f79ec1aff00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:30.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.441+0000 7f79f0eae700 1 -- 172.21.15.19:0/145489679 >> 172.21.15.19:0/145489679 conn(0x7f79ec06f4d0 msgr2=0x7f79ec11b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:30.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.441+0000 7f79f0eae700 1 -- 172.21.15.19:0/145489679 shutdown_connections 2024-05-09T02:15:30.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:30.441+0000 7f79f0eae700 1 -- 172.21.15.19:0/145489679 wait complete. 2024-05-09T02:15:30.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:30 smithi059 ceph-mon[34781]: pgmap v129: 97 pgs: 97 active+clean; 3.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 36 MiB/s wr, 966 op/s 2024-05-09T02:15:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:30 smithi196 ceph-mon[34718]: pgmap v129: 97 pgs: 97 active+clean; 3.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 36 MiB/s wr, 966 op/s 2024-05-09T02:15:31.249 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T02:15:31.675 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:15:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 230.0 s, 3470.4 tps, lat 29.022 ms stddev 35.871 2024-05-09T02:15:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:32 smithi059 ceph-mon[34781]: pgmap v130: 97 pgs: 97 active+clean; 3.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 22 MiB/s wr, 551 op/s 2024-05-09T02:15:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:32 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:15:32.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:32 smithi196 ceph-mon[34718]: pgmap v130: 97 pgs: 97 active+clean; 3.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 22 MiB/s wr, 551 op/s 2024-05-09T02:15:32.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:32 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:15:32.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:32 smithi019 ceph-mon[30151]: pgmap v130: 97 pgs: 97 active+clean; 3.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 22 MiB/s wr, 551 op/s 2024-05-09T02:15:32.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:32 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:15:33.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:33 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:15:33.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:33 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:15:33.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:33 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:15:34.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.200+0000 7f82f6671700 1 -- 172.21.15.19:0/696948466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f0109870 msgr2=0x7f82f01111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:34.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.200+0000 7f82f6671700 1 --2- 172.21.15.19:0/696948466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f0109870 0x7f82f01111d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009fc0 tx=0x7f82e800b330 comp rx=0 tx=0).stop 2024-05-09T02:15:34.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.200+0000 7f82f6671700 1 -- 172.21.15.19:0/696948466 shutdown_connections 2024-05-09T02:15:34.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.200+0000 7f82f6671700 1 --2- 172.21.15.19:0/696948466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f0109870 0x7f82f01111d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.200+0000 7f82f6671700 1 --2- 172.21.15.19:0/696948466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f01045f0 0x7f82f0104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.200+0000 7f82f6671700 1 --2- 172.21.15.19:0/696948466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f0108f20 0x7f82f0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.200+0000 7f82f6671700 1 -- 172.21.15.19:0/696948466 >> 172.21.15.19:0/696948466 conn(0x7f82f0100040 msgr2=0x7f82f0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.200+0000 7f82f6671700 1 -- 172.21.15.19:0/696948466 shutdown_connections 2024-05-09T02:15:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.200+0000 7f82f6671700 1 -- 172.21.15.19:0/696948466 wait complete. 2024-05-09T02:15:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.201+0000 7f82f6671700 1 Processor -- start 2024-05-09T02:15:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.201+0000 7f82f6671700 1 -- start start 2024-05-09T02:15:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.202+0000 7f82f6671700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f01045f0 0x7f82f0075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.202+0000 7f82f6671700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f0108f20 0x7f82f006d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.202+0000 7f82f6671700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f0109870 0x7f82f006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.202+0000 7f82f6671700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01144f0 con 0x7f82f0109870 2024-05-09T02:15:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.202+0000 7f82f6671700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f82f0114370 con 0x7f82f01045f0 2024-05-09T02:15:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.202+0000 7f82f6671700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82f0114670 con 0x7f82f0108f20 2024-05-09T02:15:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.202+0000 7f82effff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f01045f0 0x7f82f0075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.202+0000 7f82effff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f01045f0 0x7f82f0075260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:41942/0 (socket says 172.21.15.19:41942) 2024-05-09T02:15:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.202+0000 7f82effff700 1 -- 172.21.15.19:0/4130674205 learned_addr learned my addr 172.21.15.19:0/4130674205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.202+0000 7f82e7fff700 1 --2- 172.21.15.19:0/4130674205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f0108f20 0x7f82f006d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.203+0000 7f82effff700 1 -- 172.21.15.19:0/4130674205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f0108f20 msgr2=0x7f82f006d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.203+0000 7f82effff700 1 --2- 172.21.15.19:0/4130674205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f0108f20 0x7f82f006d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.203+0000 7f82effff700 1 -- 172.21.15.19:0/4130674205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f0109870 msgr2=0x7f82f006d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.203+0000 7f82effff700 1 --2- 172.21.15.19:0/4130674205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f0109870 0x7f82f006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.203+0000 7f82effff700 1 -- 172.21.15.19:0/4130674205 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f006df50 con 0x7f82f01045f0 2024-05-09T02:15:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.203+0000 7f82e7fff700 1 --2- 172.21.15.19:0/4130674205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f0108f20 0x7f82f006d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:15:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.204+0000 7f82effff700 1 --2- 172.21.15.19:0/4130674205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f01045f0 0x7f82f0075260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f82e0030c20 tx=0x7f82e0030f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.204+0000 7f82edffb700 1 -- 172.21.15.19:0/4130674205 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e002ed50 con 0x7f82f01045f0 2024-05-09T02:15:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.204+0000 7f82edffb700 1 -- 172.21.15.19:0/4130674205 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82e00044d0 con 0x7f82f01045f0 2024-05-09T02:15:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.204+0000 7f82edffb700 1 -- 172.21.15.19:0/4130674205 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e00326a0 con 0x7f82f01045f0 2024-05-09T02:15:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.204+0000 7f82f6671700 1 -- 172.21.15.19:0/4130674205 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f019ab80 con 0x7f82f01045f0 2024-05-09T02:15:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.204+0000 7f82f6671700 1 -- 172.21.15.19:0/4130674205 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f019b070 con 0x7f82f01045f0 2024-05-09T02:15:34.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.208+0000 7f82edffb700 1 -- 172.21.15.19:0/4130674205 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f82e00037a0 con 0x7f82f01045f0 2024-05-09T02:15:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.209+0000 7f82edffb700 1 --2- 172.21.15.19:0/4130674205 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f82dc0b2a70 0x7f82dc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.210+0000 7f82edffb700 1 -- 172.21.15.19:0/4130674205 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f82e0104970 con 0x7f82f01045f0 2024-05-09T02:15:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.210+0000 7f82e7fff700 1 --2- 172.21.15.19:0/4130674205 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f82dc0b2a70 0x7f82dc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.210+0000 7f8305087b80 1 Processor -- start 2024-05-09T02:15:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.210+0000 7f8305087b80 1 -- start start 2024-05-09T02:15:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.210+0000 7f8305087b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139bfe160 0x55e139c9fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.210+0000 7f8305087b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139b144a0 0x55e139ca0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.210+0000 7f8305087b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139c99eb0 0x55e139c9a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.210+0000 7f8305087b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e139bb3ff0 con 0x55e139b144a0 2024-05-09T02:15:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.210+0000 7f8305087b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e139b527f0 con 0x55e139bfe160 2024-05-09T02:15:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.210+0000 7f8305087b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e139baffb0 con 0x55e139c99eb0 2024-05-09T02:15:34.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.210+0000 7f82e7fff700 1 --2- 172.21.15.19:0/4130674205 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f82dc0b2a70 0x7f82dc0b4f80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f82f0107880 tx=0x7f82d8009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:34.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.210+0000 7f82effff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139bfe160 0x55e139c9fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.210+0000 7f82effff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139bfe160 0x55e139c9fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:41954/0 (socket says 172.21.15.19:41954) 2024-05-09T02:15:34.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.210+0000 7f82effff700 1 -- 172.21.15.19:0/2557996731 learned_addr learned my addr 172.21.15.19:0/2557996731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:34.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.211+0000 7f82f4c0e700 1 --2- 172.21.15.19:0/2557996731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139c99eb0 0x55e139c9a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.211+0000 7f82f4c0e700 1 -- 172.21.15.19:0/2557996731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139bfe160 msgr2=0x55e139c9fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:34.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.211+0000 7f82f4c0e700 1 --2- 172.21.15.19:0/2557996731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139bfe160 0x55e139c9fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.211+0000 7f82f4c0e700 1 -- 172.21.15.19:0/2557996731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139b144a0 msgr2=0x55e139ca0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:34.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.211+0000 7f82f4c0e700 1 --2- 172.21.15.19:0/2557996731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139b144a0 0x55e139ca0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.211+0000 7f82f4c0e700 1 -- 172.21.15.19:0/2557996731 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e139b4a370 con 0x55e139c99eb0 2024-05-09T02:15:34.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.211+0000 7f82f4c0e700 1 --2- 172.21.15.19:0/2557996731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139c99eb0 0x55e139c9a2e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009470 tx=0x7f82e8009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3de3549d3803d0cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:34.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.212+0000 7f82e67fc700 1 -- 172.21.15.19:0/2557996731 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8021070 con 0x55e139c99eb0 2024-05-09T02:15:34.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.212+0000 7f82e67fc700 1 -- 172.21.15.19:0/2557996731 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82e800b9f0 con 0x55e139c99eb0 2024-05-09T02:15:34.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.212+0000 7f82e67fc700 1 -- 172.21.15.19:0/2557996731 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8007af0 con 0x55e139c99eb0 2024-05-09T02:15:34.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.214+0000 7f8305087b80 1 -- 172.21.15.19:0/2557996731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139c99eb0 msgr2=0x55e139c9a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:34.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.214+0000 7f8305087b80 1 --2- 172.21.15.19:0/2557996731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139c99eb0 0x55e139c9a2e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009470 tx=0x7f82e8009800 comp rx=0 tx=0).stop 2024-05-09T02:15:34.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.215+0000 7f8305087b80 1 -- 172.21.15.19:0/2557996731 shutdown_connections 2024-05-09T02:15:34.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.215+0000 7f8305087b80 1 --2- 172.21.15.19:0/2557996731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139c99eb0 0x55e139c9a2e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.215+0000 7f8305087b80 1 --2- 172.21.15.19:0/2557996731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139bfe160 0x55e139c9fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.215+0000 7f8305087b80 1 --2- 172.21.15.19:0/2557996731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139b144a0 0x55e139ca0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.215+0000 7f8305087b80 1 -- 172.21.15.19:0/2557996731 >> 172.21.15.19:0/2557996731 conn(0x55e139b746a0 msgr2=0x55e139c8c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:34.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.215+0000 7f8305087b80 1 -- 172.21.15.19:0/2557996731 shutdown_connections 2024-05-09T02:15:34.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.215+0000 7f8305087b80 1 -- 172.21.15.19:0/2557996731 wait complete. 2024-05-09T02:15:34.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.215+0000 7f8305087b80 1 -- start start 2024-05-09T02:15:34.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.216+0000 7f8305087b80 1 Processor -- start 2024-05-09T02:15:34.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.216+0000 7f8305087b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139b144a0 0x55e139ca3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.216+0000 7f8305087b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139bfe160 0x55e139ca3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.216+0000 7f8305087b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139c99eb0 0x55e139ca4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.216+0000 7f82effff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139b144a0 0x55e139ca3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.216+0000 7f82effff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139b144a0 0x55e139ca3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:41958/0 (socket says 172.21.15.19:41958) 2024-05-09T02:15:34.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.216+0000 7f82effff700 1 -- 172.21.15.19:0/1187903822 learned_addr learned my addr 172.21.15.19:0/1187903822 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:34.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.216+0000 7f82e7fff700 1 --2- 172.21.15.19:0/1187903822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139bfe160 0x55e139ca3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.216+0000 7f8305087b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e139a2fba0 con 0x55e139c99eb0 2024-05-09T02:15:34.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.217+0000 7f8305087b80 1 -- 172.21.15.19:0/1187903822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e139b6e150 con 0x55e139b144a0 2024-05-09T02:15:34.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.217+0000 7f82f4c0e700 1 --2- 172.21.15.19:0/1187903822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139c99eb0 0x55e139ca4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.217+0000 7f8305087b80 1 -- 172.21.15.19:0/1187903822 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e139ca69e0 con 0x55e139bfe160 2024-05-09T02:15:34.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.217+0000 7f8305087b80 10 client.0 _ll_register_callbacks cb 0x55e139a256c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.218+0000 7f82e7fff700 1 -- 172.21.15.19:0/1187903822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139b144a0 msgr2=0x55e139ca3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.218+0000 7f82e7fff700 1 --2- 172.21.15.19:0/1187903822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139b144a0 0x55e139ca3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.218+0000 7f82e7fff700 1 -- 172.21.15.19:0/1187903822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139c99eb0 msgr2=0x55e139ca4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.218+0000 7f82e7fff700 1 --2- 172.21.15.19:0/1187903822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139c99eb0 0x55e139ca4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.218+0000 7f82e7fff700 1 -- 172.21.15.19:0/1187903822 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e139a2f490 con 0x55e139bfe160 2024-05-09T02:15:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.218+0000 7f82f4c0e700 1 --2- 172.21.15.19:0/1187903822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139c99eb0 0x55e139ca4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:15:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.218+0000 7f82e7fff700 1 --2- 172.21.15.19:0/1187903822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139bfe160 0x55e139ca3ff0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55e139c8ded0 tx=0x7f82d8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c38013c90d28ec20 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.218+0000 7f82cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:15:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.219+0000 7f82cf7fe700 1 -- 172.21.15.19:0/1187903822 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d8022070 con 0x55e139bfe160 2024-05-09T02:15:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.219+0000 7f8305087b80 1 -- 172.21.15.19:0/1187903822 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82e800b9f0 con 0x55e139bfe160 2024-05-09T02:15:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.219+0000 7f82cf7fe700 1 -- 172.21.15.19:0/1187903822 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f82d8004460 con 0x55e139bfe160 2024-05-09T02:15:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.219+0000 7f82cf7fe700 1 -- 172.21.15.19:0/1187903822 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f82d802e050 con 0x55e139bfe160 2024-05-09T02:15:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.219+0000 7f8305087b80 10 client.25594 fetch_fsmap learned FSMap version 40 2024-05-09T02:15:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.219+0000 7f8305087b80 1 -- 172.21.15.19:0/1187903822 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e139cb1530 con 0x55e139bfe160 2024-05-09T02:15:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.219+0000 7f82cf7fe700 1 -- 172.21.15.19:0/1187903822 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82d800fc50 con 0x55e139bfe160 2024-05-09T02:15:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.219+0000 7f82cf7fe700 1 -- 172.21.15.19:0/1187903822 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f82d8003c50 con 0x55e139bfe160 2024-05-09T02:15:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.220+0000 7f8305087b80 10 client.25594 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T02:15:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.220+0000 7f8305087b80 10 client.25594 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:15:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.220+0000 7f8305087b80 20 client.25594 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.220+0000 7f8305087b80 1 --2- 172.21.15.19:0/1187903822 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55e139cb3ac0 0x55e139cb5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.220+0000 7f8305087b80 4 client.25594 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:15:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.220+0000 7f8305087b80 1 -- 172.21.15.19:0/1187903822 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f82e800b9f0 con 0x55e139cb3ac0 2024-05-09T02:15:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.220+0000 7f82effff700 1 --2- 172.21.15.19:0/1187903822 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55e139cb3ac0 0x55e139cb5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.220+0000 7f82cf7fe700 10 client.25594 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:15:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.221+0000 7f82effff700 1 --2- 172.21.15.19:0/1187903822 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55e139cb3ac0 0x55e139cb5ee0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7471c124deb98362 server_cookie=94700ad83df45d2 in_seq=0 out_seq=0 2024-05-09T02:15:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.222+0000 7f82cf7fe700 1 -- 172.21.15.19:0/1187903822 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f82e800b9f0 con 0x55e139cb3ac0 2024-05-09T02:15:34.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.222+0000 7f82cf7fe700 10 client.25594 handle_command_reply: tid=0 2024-05-09T02:15:34.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.222+0000 7f8305087b80 1 client.25594 shutdown 2024-05-09T02:15:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.223+0000 7f8305087b80 1 -- 172.21.15.19:0/1187903822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139bfe160 msgr2=0x55e139ca3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.223+0000 7f8305087b80 1 --2- 172.21.15.19:0/1187903822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139bfe160 0x55e139ca3ff0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55e139c8ded0 tx=0x7f82d8004750 comp rx=0 tx=0).stop 2024-05-09T02:15:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.223+0000 7f8305087b80 1 -- 172.21.15.19:0/1187903822 shutdown_connections 2024-05-09T02:15:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.223+0000 7f8305087b80 1 --2- 172.21.15.19:0/1187903822 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55e139cb3ac0 0x55e139cb5ee0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.223+0000 7f8305087b80 1 --2- 172.21.15.19:0/1187903822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139c99eb0 0x55e139ca4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.223+0000 7f8305087b80 1 --2- 172.21.15.19:0/1187903822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139b144a0 0x55e139ca3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.223+0000 7f8305087b80 1 --2- 172.21.15.19:0/1187903822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139bfe160 0x55e139ca3ff0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.223+0000 7f82cf7fe700 0 client.25594 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:15:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.223+0000 7f8305087b80 1 -- 172.21.15.19:0/1187903822 >> 172.21.15.19:0/1187903822 conn(0x55e139b746a0 msgr2=0x55e139c8c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.224+0000 7f8305087b80 1 -- 172.21.15.19:0/1187903822 shutdown_connections 2024-05-09T02:15:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.224+0000 7f8305087b80 1 -- 172.21.15.19:0/1187903822 wait complete. 2024-05-09T02:15:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.224+0000 7f8305087b80 20 client.25594 trim_cache size 0 max 16384 2024-05-09T02:15:34.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.250+0000 7f8305087b80 1 Processor -- start 2024-05-09T02:15:34.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.250+0000 7f8305087b80 1 -- start start 2024-05-09T02:15:34.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.250+0000 7f8305087b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139b144a0 0x55e139ca2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.250+0000 7f8305087b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139bfe160 0x55e139ca4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.250+0000 7f8305087b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139c99eb0 0x55e139c995b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.250+0000 7f8305087b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01a0490 con 0x55e139bfe160 2024-05-09T02:15:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.250+0000 7f8305087b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01a2090 con 0x55e139c99eb0 2024-05-09T02:15:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.250+0000 7f8305087b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01a0b60 con 0x55e139b144a0 2024-05-09T02:15:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.251+0000 7f82e7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139bfe160 0x55e139ca4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.251+0000 7f82f4c0e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139c99eb0 0x55e139c995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.251+0000 7f82e7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139bfe160 0x55e139ca4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37214/0 (socket says 172.21.15.19:37214) 2024-05-09T02:15:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.251+0000 7f82e7fff700 1 -- 172.21.15.19:0/3919032176 learned_addr learned my addr 172.21.15.19:0/3919032176 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:34.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.251+0000 7f82effff700 1 --2- 172.21.15.19:0/3919032176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139b144a0 0x55e139ca2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.251+0000 7f82f4c0e700 1 -- 172.21.15.19:0/3919032176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139b144a0 msgr2=0x55e139ca2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:34.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.251+0000 7f82f4c0e700 1 --2- 172.21.15.19:0/3919032176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139b144a0 0x55e139ca2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.251+0000 7f82f4c0e700 1 -- 172.21.15.19:0/3919032176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139bfe160 msgr2=0x55e139ca4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:34.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.251+0000 7f82f4c0e700 1 --2- 172.21.15.19:0/3919032176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139bfe160 0x55e139ca4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.251+0000 7f82f4c0e700 1 -- 172.21.15.19:0/3919032176 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e139ca68b0 con 0x55e139c99eb0 2024-05-09T02:15:34.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.251+0000 7f82effff700 1 --2- 172.21.15.19:0/3919032176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139b144a0 0x55e139ca2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:15:34.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.251+0000 7f82f4c0e700 1 --2- 172.21.15.19:0/3919032176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139c99eb0 0x55e139c995b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009960 tx=0x7f82e8007500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8610416512e65a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:34.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.252+0000 7f82e5ffb700 1 -- 172.21.15.19:0/3919032176 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8021040 con 0x55e139c99eb0 2024-05-09T02:15:34.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.252+0000 7f82e5ffb700 1 -- 172.21.15.19:0/3919032176 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82e80076a0 con 0x55e139c99eb0 2024-05-09T02:15:34.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.252+0000 7f82e5ffb700 1 -- 172.21.15.19:0/3919032176 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e80053d0 con 0x55e139c99eb0 2024-05-09T02:15:34.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.253+0000 7f8305087b80 1 -- 172.21.15.19:0/3919032176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139c99eb0 msgr2=0x55e139c995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:34.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.253+0000 7f8305087b80 1 --2- 172.21.15.19:0/3919032176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139c99eb0 0x55e139c995b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009960 tx=0x7f82e8007500 comp rx=0 tx=0).stop 2024-05-09T02:15:34.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.253+0000 7f8305087b80 1 -- 172.21.15.19:0/3919032176 shutdown_connections 2024-05-09T02:15:34.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.253+0000 7f8305087b80 1 --2- 172.21.15.19:0/3919032176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139c99eb0 0x55e139c995b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.253+0000 7f8305087b80 1 --2- 172.21.15.19:0/3919032176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139b144a0 0x55e139ca2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.253+0000 7f8305087b80 1 --2- 172.21.15.19:0/3919032176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139bfe160 0x55e139ca4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.253+0000 7f8305087b80 1 -- 172.21.15.19:0/3919032176 >> 172.21.15.19:0/3919032176 conn(0x55e139b746a0 msgr2=0x55e139c98ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:34.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.253+0000 7f8305087b80 1 -- 172.21.15.19:0/3919032176 shutdown_connections 2024-05-09T02:15:34.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.253+0000 7f8305087b80 1 -- 172.21.15.19:0/3919032176 wait complete. 2024-05-09T02:15:34.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.254+0000 7f8305087b80 1 -- start start 2024-05-09T02:15:34.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.254+0000 7f8305087b80 1 Processor -- start 2024-05-09T02:15:34.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f8305087b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139b144a0 0x55e139cabd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f8305087b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139bfe160 0x55e139cae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f8305087b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139c99eb0 0x55e139caecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f8305087b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01a19c0 con 0x55e139b144a0 2024-05-09T02:15:34.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f8305087b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01a1230 con 0x55e139c99eb0 2024-05-09T02:15:34.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f8305087b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82f019e450 con 0x55e139bfe160 2024-05-09T02:15:34.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f82effff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139b144a0 0x55e139cabd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f82e7fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139bfe160 0x55e139cae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f82effff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139b144a0 0x55e139cabd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37230/0 (socket says 172.21.15.19:37230) 2024-05-09T02:15:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f82e7fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139bfe160 0x55e139cae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:59580/0 (socket says 172.21.15.19:59580) 2024-05-09T02:15:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f82effff700 1 -- 172.21.15.19:0/2183477357 learned_addr learned my addr 172.21.15.19:0/2183477357 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f82f4c0e700 1 --2- 172.21.15.19:0/2183477357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139c99eb0 0x55e139caecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f8305087b80 10 client.0 _ll_register_callbacks cb 0x55e139a256c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f82e7fff700 1 -- 172.21.15.19:0/2183477357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139c99eb0 msgr2=0x55e139caecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f82e7fff700 1 --2- 172.21.15.19:0/2183477357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139c99eb0 0x55e139caecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f82e7fff700 1 -- 172.21.15.19:0/2183477357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139b144a0 msgr2=0x55e139cabd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f82e7fff700 1 --2- 172.21.15.19:0/2183477357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139b144a0 0x55e139cabd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f82e7fff700 1 -- 172.21.15.19:0/2183477357 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e139caf230 con 0x55e139bfe160 2024-05-09T02:15:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.255+0000 7f82effff700 1 --2- 172.21.15.19:0/2183477357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139b144a0 0x55e139cabd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:15:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.256+0000 7f82e7fff700 1 --2- 172.21.15.19:0/2183477357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139bfe160 0x55e139cae7b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82d8013010 tx=0x7f82d801ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36a17f646ee1443 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.256+0000 7f82cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:15:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.256+0000 7f82cf7fe700 1 -- 172.21.15.19:0/2183477357 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d8022040 con 0x55e139bfe160 2024-05-09T02:15:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.256+0000 7f82cf7fe700 1 -- 172.21.15.19:0/2183477357 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f82d8005030 con 0x55e139bfe160 2024-05-09T02:15:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.256+0000 7f8305087b80 1 -- 172.21.15.19:0/2183477357 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82e80076a0 con 0x55e139bfe160 2024-05-09T02:15:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.256+0000 7f82cf7fe700 1 -- 172.21.15.19:0/2183477357 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f82d8005230 con 0x55e139bfe160 2024-05-09T02:15:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.256+0000 7f8305087b80 10 client.25600 fetch_fsmap learned FSMap version 40 2024-05-09T02:15:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.256+0000 7f8305087b80 1 -- 172.21.15.19:0/2183477357 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e139d1d200 con 0x55e139bfe160 2024-05-09T02:15:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.256+0000 7f82cf7fe700 1 -- 172.21.15.19:0/2183477357 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82e80076a0 con 0x55e139bfe160 2024-05-09T02:15:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.257+0000 7f82cf7fe700 1 -- 172.21.15.19:0/2183477357 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f82d8005440 con 0x55e139bfe160 2024-05-09T02:15:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.257+0000 7f8305087b80 10 client.25600 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T02:15:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.257+0000 7f8305087b80 10 client.25600 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:15:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.257+0000 7f8305087b80 20 client.25600 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.257+0000 7f8305087b80 1 --2- 172.21.15.19:0/2183477357 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55e139cb3ac0 0x55e139d1f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.257+0000 7f8305087b80 4 client.25600 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:15:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.257+0000 7f8305087b80 1 -- 172.21.15.19:0/2183477357 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e139bc19d0 con 0x55e139cb3ac0 2024-05-09T02:15:34.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.258+0000 7f82effff700 1 --2- 172.21.15.19:0/2183477357 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55e139cb3ac0 0x55e139d1f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.258+0000 7f82effff700 1 --2- 172.21.15.19:0/2183477357 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55e139cb3ac0 0x55e139d1f7d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88d3f68c84718202 server_cookie=15090c09b5c6f01f in_seq=0 out_seq=0 2024-05-09T02:15:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.258+0000 7f82cf7fe700 10 client.25600 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:15:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.258+0000 7f82cf7fe700 1 -- 172.21.15.19:0/2183477357 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e139bc19d0 con 0x55e139cb3ac0 2024-05-09T02:15:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.258+0000 7f82cf7fe700 10 client.25600 handle_command_reply: tid=0 2024-05-09T02:15:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.258+0000 7f8305087b80 1 client.25600 shutdown 2024-05-09T02:15:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.259+0000 7f8305087b80 1 -- 172.21.15.19:0/2183477357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139bfe160 msgr2=0x55e139cae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.259+0000 7f8305087b80 1 --2- 172.21.15.19:0/2183477357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139bfe160 0x55e139cae7b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82d8013010 tx=0x7f82d801ed50 comp rx=0 tx=0).stop 2024-05-09T02:15:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.259+0000 7f8305087b80 1 -- 172.21.15.19:0/2183477357 shutdown_connections 2024-05-09T02:15:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.259+0000 7f8305087b80 1 --2- 172.21.15.19:0/2183477357 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55e139cb3ac0 0x55e139d1f7d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.259+0000 7f8305087b80 1 --2- 172.21.15.19:0/2183477357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e139c99eb0 0x55e139caecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.259+0000 7f8305087b80 1 --2- 172.21.15.19:0/2183477357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e139b144a0 0x55e139cabd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.259+0000 7f8305087b80 1 --2- 172.21.15.19:0/2183477357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e139bfe160 0x55e139cae7b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.259+0000 7f82cf7fe700 0 client.25600 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:15:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.259+0000 7f8305087b80 1 -- 172.21.15.19:0/2183477357 >> 172.21.15.19:0/2183477357 conn(0x55e139b746a0 msgr2=0x55e139c9f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:34.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.259+0000 7f8305087b80 1 -- 172.21.15.19:0/2183477357 shutdown_connections 2024-05-09T02:15:34.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.260+0000 7f8305087b80 1 -- 172.21.15.19:0/2183477357 wait complete. 2024-05-09T02:15:34.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.260+0000 7f8305087b80 20 client.25600 trim_cache size 0 max 16384 2024-05-09T02:15:34.265 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:15:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.262+0000 7f82f6671700 1 -- 172.21.15.19:0/4130674205 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f82dc0b2a70 msgr2=0x7f82dc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.262+0000 7f82f6671700 1 --2- 172.21.15.19:0/4130674205 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f82dc0b2a70 0x7f82dc0b4f80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f82f0107880 tx=0x7f82d8009260 comp rx=0 tx=0).stop 2024-05-09T02:15:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.262+0000 7f82f6671700 1 -- 172.21.15.19:0/4130674205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f01045f0 msgr2=0x7f82f0075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.262+0000 7f82f6671700 1 --2- 172.21.15.19:0/4130674205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f01045f0 0x7f82f0075260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f82e0030c20 tx=0x7f82e0030f30 comp rx=0 tx=0).stop 2024-05-09T02:15:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.263+0000 7f82f6671700 1 -- 172.21.15.19:0/4130674205 shutdown_connections 2024-05-09T02:15:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.263+0000 7f82f6671700 1 --2- 172.21.15.19:0/4130674205 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f82dc0b2a70 0x7f82dc0b4f80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.263+0000 7f82f6671700 1 --2- 172.21.15.19:0/4130674205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f0109870 0x7f82f006d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.263+0000 7f82f6671700 1 --2- 172.21.15.19:0/4130674205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f01045f0 0x7f82f0075260 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.263+0000 7f82f6671700 1 --2- 172.21.15.19:0/4130674205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f0108f20 0x7f82f006d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.263+0000 7f82f6671700 1 -- 172.21.15.19:0/4130674205 >> 172.21.15.19:0/4130674205 conn(0x7f82f0100040 msgr2=0x7f82f0108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:34.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.263+0000 7f82f6671700 1 -- 172.21.15.19:0/4130674205 shutdown_connections 2024-05-09T02:15:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:34.263+0000 7f82f6671700 1 -- 172.21.15.19:0/4130674205 wait complete. 2024-05-09T02:15:34.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:34 smithi019 ceph-mon[30151]: pgmap v131: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 MiB/s wr, 570 op/s 2024-05-09T02:15:34.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:34 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:15:34.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:34 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:15:34.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:34 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:15:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:34 smithi059 ceph-mon[34781]: pgmap v131: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 MiB/s wr, 570 op/s 2024-05-09T02:15:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:34 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:15:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:34 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:15:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:34 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:15:34.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:34 smithi196 ceph-mon[34718]: pgmap v131: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 MiB/s wr, 570 op/s 2024-05-09T02:15:34.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:34 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:15:34.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:34 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:15:34.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:34 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:15:35.418 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T02:15:35.830 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:15:36.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:36 smithi019 ceph-mon[30151]: pgmap v132: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 33 MiB/s wr, 888 op/s 2024-05-09T02:15:36.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:36 smithi059 ceph-mon[34781]: pgmap v132: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 33 MiB/s wr, 888 op/s 2024-05-09T02:15:36.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:36 smithi196 ceph-mon[34718]: pgmap v132: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 33 MiB/s wr, 888 op/s 2024-05-09T02:15:37.620 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:37 smithi019 ceph-mon[30151]: pgmap v133: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 26 MiB/s wr, 660 op/s 2024-05-09T02:15:37.656 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 235.0 s, 2350.7 tps, lat 36.065 ms stddev 80.283 2024-05-09T02:15:37.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.735+0000 7f40c0a26700 1 -- 172.21.15.19:0/618766671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40bc1045f0 msgr2=0x7f40bc104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:37.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.735+0000 7f40c0a26700 1 --2- 172.21.15.19:0/618766671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40bc1045f0 0x7f40bc104a00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00b0d0 tx=0x7f40ac00b3e0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.736+0000 7f40c0a26700 1 -- 172.21.15.19:0/618766671 shutdown_connections 2024-05-09T02:15:37.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.736+0000 7f40c0a26700 1 --2- 172.21.15.19:0/618766671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40bc109870 0x7f40bc1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.736+0000 7f40c0a26700 1 --2- 172.21.15.19:0/618766671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40bc1045f0 0x7f40bc104a00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.736+0000 7f40c0a26700 1 --2- 172.21.15.19:0/618766671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40bc108f20 0x7f40bc109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.736+0000 7f40c0a26700 1 -- 172.21.15.19:0/618766671 >> 172.21.15.19:0/618766671 conn(0x7f40bc100040 msgr2=0x7f40bc102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:37.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.736+0000 7f40c0a26700 1 -- 172.21.15.19:0/618766671 shutdown_connections 2024-05-09T02:15:37.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.736+0000 7f40c0a26700 1 -- 172.21.15.19:0/618766671 wait complete. 2024-05-09T02:15:37.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40c0a26700 1 Processor -- start 2024-05-09T02:15:37.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40c0a26700 1 -- start start 2024-05-09T02:15:37.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40c0a26700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40bc1045f0 0x7f40bc19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40c0a26700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40bc108f20 0x7f40bc19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40c0a26700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40bc109870 0x7f40bc19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40c0a26700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc1144a0 con 0x7f40bc108f20 2024-05-09T02:15:37.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40c0a26700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc114320 con 0x7f40bc1045f0 2024-05-09T02:15:37.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40c0a26700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc114620 con 0x7f40bc109870 2024-05-09T02:15:37.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40b9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40bc108f20 0x7f40bc19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40b9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40bc108f20 0x7f40bc19b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34740/0 (socket says 172.21.15.19:34740) 2024-05-09T02:15:37.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40b9d9b700 1 -- 172.21.15.19:0/2681025753 learned_addr learned my addr 172.21.15.19:0/2681025753 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:37.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40ba59c700 1 --2- 172.21.15.19:0/2681025753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40bc1045f0 0x7f40bc19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40bad9d700 1 --2- 172.21.15.19:0/2681025753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40bc109870 0x7f40bc19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40b9d9b700 1 -- 172.21.15.19:0/2681025753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40bc109870 msgr2=0x7f40bc19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:37.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40b9d9b700 1 --2- 172.21.15.19:0/2681025753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40bc109870 0x7f40bc19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40b9d9b700 1 -- 172.21.15.19:0/2681025753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40bc1045f0 msgr2=0x7f40bc19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:37.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40b9d9b700 1 --2- 172.21.15.19:0/2681025753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40bc1045f0 0x7f40bc19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.737+0000 7f40b9d9b700 1 -- 172.21.15.19:0/2681025753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40bc19fd20 con 0x7f40bc108f20 2024-05-09T02:15:37.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.738+0000 7f40b9d9b700 1 --2- 172.21.15.19:0/2681025753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40bc108f20 0x7f40bc19b3f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f40a400d980 tx=0x7f40a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:37.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.738+0000 7f40b37fe700 1 -- 172.21.15.19:0/2681025753 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a4017070 con 0x7f40bc108f20 2024-05-09T02:15:37.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.738+0000 7f40c0a26700 1 -- 172.21.15.19:0/2681025753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40bc1a0010 con 0x7f40bc108f20 2024-05-09T02:15:37.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.738+0000 7f40b37fe700 1 -- 172.21.15.19:0/2681025753 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a4009de0 con 0x7f40bc108f20 2024-05-09T02:15:37.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.738+0000 7f40b37fe700 1 -- 172.21.15.19:0/2681025753 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a400f3c0 con 0x7f40bc108f20 2024-05-09T02:15:37.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.738+0000 7f40c0a26700 1 -- 172.21.15.19:0/2681025753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40bc1a7930 con 0x7f40bc108f20 2024-05-09T02:15:37.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.741+0000 7f40b37fe700 1 -- 172.21.15.19:0/2681025753 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f40a400f560 con 0x7f40bc108f20 2024-05-09T02:15:37.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.741+0000 7f40b37fe700 1 --2- 172.21.15.19:0/2681025753 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f40a80b29f0 0x7f40a80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.741+0000 7f40ba59c700 1 --2- 172.21.15.19:0/2681025753 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f40a80b29f0 0x7f40a80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.742+0000 7f40b37fe700 1 -- 172.21.15.19:0/2681025753 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f40a40e20c0 con 0x7f40bc108f20 2024-05-09T02:15:37.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.742+0000 7f40cf43cb80 1 Processor -- start 2024-05-09T02:15:37.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.742+0000 7f40ba59c700 1 --2- 172.21.15.19:0/2681025753 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f40a80b29f0 0x7f40a80b4f00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009900 tx=0x7f40ac00b040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:37.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.742+0000 7f40cf43cb80 1 -- start start 2024-05-09T02:15:37.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.742+0000 7f40cf43cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ece94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.742+0000 7f40cf43cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9eb664a0 0x556c9ece9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.742+0000 7f40cf43cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9ece36a0 0x556c9ece3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.742+0000 7f40cf43cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556c9ec05ff0 con 0x556c9ec50160 2024-05-09T02:15:37.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.742+0000 7f40cf43cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556c9eba47f0 con 0x556c9ece36a0 2024-05-09T02:15:37.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.742+0000 7f40cf43cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556c9ec01fb0 con 0x556c9eb664a0 2024-05-09T02:15:37.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.743+0000 7f40ba59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ece94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.743+0000 7f40ba59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ece94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34744/0 (socket says 172.21.15.19:34744) 2024-05-09T02:15:37.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.743+0000 7f40ba59c700 1 -- 172.21.15.19:0/3519265695 learned_addr learned my addr 172.21.15.19:0/3519265695 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.743+0000 7f40b9d9b700 1 --2- 172.21.15.19:0/3519265695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9eb664a0 0x556c9ece9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.743+0000 7f40bad9d700 1 --2- 172.21.15.19:0/3519265695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9ece36a0 0x556c9ece3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.743+0000 7f40ba59c700 1 -- 172.21.15.19:0/3519265695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9eb664a0 msgr2=0x556c9ece9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.743+0000 7f40ba59c700 1 --2- 172.21.15.19:0/3519265695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9eb664a0 0x556c9ece9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.743+0000 7f40ba59c700 1 -- 172.21.15.19:0/3519265695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9ece36a0 msgr2=0x556c9ece3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:37.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.743+0000 7f40ba59c700 1 --2- 172.21.15.19:0/3519265695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9ece36a0 0x556c9ece3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.743+0000 7f40ba59c700 1 -- 172.21.15.19:0/3519265695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c9eb9c370 con 0x556c9ec50160 2024-05-09T02:15:37.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.743+0000 7f40bad9d700 1 --2- 172.21.15.19:0/3519265695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9ece36a0 0x556c9ece3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:15:37.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.743+0000 7f40ba59c700 1 --2- 172.21.15.19:0/3519265695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ece94f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f40ac015000 tx=0x7f40ac01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa91190f2dc80023 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:37.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.743+0000 7f40b0ff9700 1 -- 172.21.15.19:0/3519265695 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac02c030 con 0x556c9ec50160 2024-05-09T02:15:37.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.744+0000 7f40b0ff9700 1 -- 172.21.15.19:0/3519265695 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40ac007a70 con 0x556c9ec50160 2024-05-09T02:15:37.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.744+0000 7f40b0ff9700 1 -- 172.21.15.19:0/3519265695 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0054c0 con 0x556c9ec50160 2024-05-09T02:15:37.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.744+0000 7f40cf43cb80 1 -- 172.21.15.19:0/3519265695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 msgr2=0x556c9ece94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:37.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.744+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/3519265695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ece94f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f40ac015000 tx=0x7f40ac01d000 comp rx=0 tx=0).stop 2024-05-09T02:15:37.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.744+0000 7f40cf43cb80 1 -- 172.21.15.19:0/3519265695 shutdown_connections 2024-05-09T02:15:37.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.744+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/3519265695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9ece36a0 0x556c9ece3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.744+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/3519265695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ece94f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.744+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/3519265695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9eb664a0 0x556c9ece9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.744+0000 7f40cf43cb80 1 -- 172.21.15.19:0/3519265695 >> 172.21.15.19:0/3519265695 conn(0x556c9ebc66a0 msgr2=0x556c9ecde400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:37.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.745+0000 7f40cf43cb80 1 -- 172.21.15.19:0/3519265695 shutdown_connections 2024-05-09T02:15:37.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.745+0000 7f40cf43cb80 1 -- 172.21.15.19:0/3519265695 wait complete. 2024-05-09T02:15:37.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.745+0000 7f40cf43cb80 1 -- start start 2024-05-09T02:15:37.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.745+0000 7f40cf43cb80 1 Processor -- start 2024-05-09T02:15:37.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.745+0000 7f40cf43cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9eb664a0 0x556c9eced1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.745+0000 7f40cf43cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9eced730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.745+0000 7f40cf43cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 0x556c9ecedc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.745+0000 7f40cf43cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556c9ea81ba0 con 0x556c9ec50160 2024-05-09T02:15:37.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.745+0000 7f40cf43cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556c9ebc0150 con 0x556c9eb664a0 2024-05-09T02:15:37.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.745+0000 7f40cf43cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556c9ecf01d0 con 0x556c9ece36a0 2024-05-09T02:15:37.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40cf43cb80 10 client.0 _ll_register_callbacks cb 0x556c9ea776c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:37.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40b9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9eced730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40b9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9eced730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34750/0 (socket says 172.21.15.19:34750) 2024-05-09T02:15:37.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40b9d9b700 1 -- 172.21.15.19:0/152826282 learned_addr learned my addr 172.21.15.19:0/152826282 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:37.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40ba59c700 1 --2- 172.21.15.19:0/152826282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9eb664a0 0x556c9eced1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40bad9d700 1 --2- 172.21.15.19:0/152826282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 0x556c9ecedc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40b9d9b700 1 -- 172.21.15.19:0/152826282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 msgr2=0x556c9ecedc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:37.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40b9d9b700 1 --2- 172.21.15.19:0/152826282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 0x556c9ecedc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40b9d9b700 1 -- 172.21.15.19:0/152826282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9eb664a0 msgr2=0x556c9eced1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:37.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40b9d9b700 1 --2- 172.21.15.19:0/152826282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9eb664a0 0x556c9eced1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40b9d9b700 1 -- 172.21.15.19:0/152826282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c9ea81490 con 0x556c9ec50160 2024-05-09T02:15:37.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40b9d9b700 1 --2- 172.21.15.19:0/152826282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9eced730 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f40a40a90a0 tx=0x7f40a40dfd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6cdc5fb83de50b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:37.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:15:37.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40a1ffb700 1 -- 172.21.15.19:0/152826282 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a40c3060 con 0x556c9ec50160 2024-05-09T02:15:37.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40a1ffb700 1 -- 172.21.15.19:0/152826282 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f40a40b1f80 con 0x556c9ec50160 2024-05-09T02:15:37.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40cf43cb80 1 -- 172.21.15.19:0/152826282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40ac007a70 con 0x556c9ec50160 2024-05-09T02:15:37.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40a1ffb700 1 -- 172.21.15.19:0/152826282 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f40a4020070 con 0x556c9ec50160 2024-05-09T02:15:37.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40cf43cb80 10 client.15861 fetch_fsmap learned FSMap version 40 2024-05-09T02:15:37.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.746+0000 7f40cf43cb80 1 -- 172.21.15.19:0/152826282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556c9ecfab50 con 0x556c9ec50160 2024-05-09T02:15:37.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.747+0000 7f40a1ffb700 1 -- 172.21.15.19:0/152826282 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40ac007a70 con 0x556c9ec50160 2024-05-09T02:15:37.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.747+0000 7f40a1ffb700 1 -- 172.21.15.19:0/152826282 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f40a40a8600 con 0x556c9ec50160 2024-05-09T02:15:37.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.747+0000 7f40cf43cb80 10 client.15861 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T02:15:37.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.747+0000 7f40cf43cb80 10 client.15861 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:15:37.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.747+0000 7f40cf43cb80 20 client.15861 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.747+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/152826282 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556c9ecfd0e0 0x556c9ecff500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.747+0000 7f40cf43cb80 4 client.15861 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:15:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.747+0000 7f40cf43cb80 1 -- 172.21.15.19:0/152826282 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c9ec139d0 con 0x556c9ecfd0e0 2024-05-09T02:15:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.747+0000 7f40ba59c700 1 --2- 172.21.15.19:0/152826282 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556c9ecfd0e0 0x556c9ecff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.748+0000 7f40a1ffb700 10 client.15861 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:15:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.748+0000 7f40ba59c700 1 --2- 172.21.15.19:0/152826282 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556c9ecfd0e0 0x556c9ecff500 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa12a2f3b2c87087 server_cookie=41d92eefabcd69df in_seq=0 out_seq=0 2024-05-09T02:15:37.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.748+0000 7f40a1ffb700 1 -- 172.21.15.19:0/152826282 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c9ec139d0 con 0x556c9ecfd0e0 2024-05-09T02:15:37.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.748+0000 7f40a1ffb700 10 client.15861 handle_command_reply: tid=0 2024-05-09T02:15:37.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.749+0000 7f40cf43cb80 1 client.15861 shutdown 2024-05-09T02:15:37.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.749+0000 7f40cf43cb80 1 -- 172.21.15.19:0/152826282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 msgr2=0x556c9eced730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:37.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.749+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/152826282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9eced730 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f40a40a90a0 tx=0x7f40a40dfd10 comp rx=0 tx=0).stop 2024-05-09T02:15:37.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.749+0000 7f40cf43cb80 1 -- 172.21.15.19:0/152826282 shutdown_connections 2024-05-09T02:15:37.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.749+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/152826282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 0x556c9ecedc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.749+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/152826282 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556c9ecfd0e0 0x556c9ecff500 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.749+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/152826282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9eb664a0 0x556c9eced1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.749+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/152826282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9eced730 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.749+0000 7f40a1ffb700 0 client.15861 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:15:37.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.749+0000 7f40cf43cb80 1 -- 172.21.15.19:0/152826282 >> 172.21.15.19:0/152826282 conn(0x556c9ebc66a0 msgr2=0x556c9ecde1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:37.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.749+0000 7f40cf43cb80 1 -- 172.21.15.19:0/152826282 shutdown_connections 2024-05-09T02:15:37.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.749+0000 7f40cf43cb80 1 -- 172.21.15.19:0/152826282 wait complete. 2024-05-09T02:15:37.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.749+0000 7f40cf43cb80 20 client.15861 trim_cache size 0 max 16384 2024-05-09T02:15:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.773+0000 7f40cf43cb80 1 Processor -- start 2024-05-09T02:15:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.775+0000 7f40cf43cb80 1 -- start start 2024-05-09T02:15:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.776+0000 7f40cf43cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9eb664a0 0x556c9ecec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.776+0000 7f40cf43cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ece6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.776+0000 7f40cf43cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 0x556c9ece7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.776+0000 7f40cf43cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc1aeec0 con 0x556c9ec50160 2024-05-09T02:15:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.776+0000 7f40cf43cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc1b0ac0 con 0x556c9eb664a0 2024-05-09T02:15:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.776+0000 7f40cf43cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc1af590 con 0x556c9ece36a0 2024-05-09T02:15:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.778+0000 7f40b9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ece6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.778+0000 7f40b9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ece6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34752/0 (socket says 172.21.15.19:34752) 2024-05-09T02:15:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.778+0000 7f40b9d9b700 1 -- 172.21.15.19:0/4012594035 learned_addr learned my addr 172.21.15.19:0/4012594035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:37.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.779+0000 7f40bad9d700 1 --2- 172.21.15.19:0/4012594035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 0x556c9ece7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.779+0000 7f40b9d9b700 1 -- 172.21.15.19:0/4012594035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 msgr2=0x556c9ece7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:37.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.779+0000 7f40b9d9b700 1 --2- 172.21.15.19:0/4012594035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 0x556c9ece7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.779+0000 7f40b9d9b700 1 -- 172.21.15.19:0/4012594035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9eb664a0 msgr2=0x556c9ecec1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:15:37.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.779+0000 7f40b9d9b700 1 --2- 172.21.15.19:0/4012594035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9eb664a0 0x556c9ecec1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.779+0000 7f40b9d9b700 1 -- 172.21.15.19:0/4012594035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c9ececfc0 con 0x556c9ec50160 2024-05-09T02:15:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.779+0000 7f40b9d9b700 1 --2- 172.21.15.19:0/4012594035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ece6e00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f40a40a6610 tx=0x7f40a4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4bffad9bcfbb11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.780+0000 7f40a3fff700 1 -- 172.21.15.19:0/4012594035 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a40a93a0 con 0x556c9ec50160 2024-05-09T02:15:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.780+0000 7f40a3fff700 1 -- 172.21.15.19:0/4012594035 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a40ad750 con 0x556c9ec50160 2024-05-09T02:15:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.780+0000 7f40a3fff700 1 -- 172.21.15.19:0/4012594035 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a40b0ad0 con 0x556c9ec50160 2024-05-09T02:15:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.782+0000 7f40cf43cb80 1 -- 172.21.15.19:0/4012594035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 msgr2=0x556c9ece6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.782+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/4012594035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ece6e00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f40a40a6610 tx=0x7f40a4009f90 comp rx=0 tx=0).stop 2024-05-09T02:15:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.784+0000 7f40cf43cb80 1 -- 172.21.15.19:0/4012594035 shutdown_connections 2024-05-09T02:15:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.784+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/4012594035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 0x556c9ece7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.784+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/4012594035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9eb664a0 0x556c9ecec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.784+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/4012594035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ece6e00 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.784+0000 7f40cf43cb80 1 -- 172.21.15.19:0/4012594035 >> 172.21.15.19:0/4012594035 conn(0x556c9ebc66a0 msgr2=0x556c9ecebae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:37.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.788+0000 7f40cf43cb80 1 -- 172.21.15.19:0/4012594035 shutdown_connections 2024-05-09T02:15:37.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.789+0000 7f40cf43cb80 1 -- 172.21.15.19:0/4012594035 wait complete. 2024-05-09T02:15:37.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.790+0000 7f40cf43cb80 1 -- start start 2024-05-09T02:15:37.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.790+0000 7f40cf43cb80 1 Processor -- start 2024-05-09T02:15:37.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.790+0000 7f40cf43cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9eb664a0 0x556c9ecf5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.790+0000 7f40cf43cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ecea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.790+0000 7f40cf43cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 0x556c9ecead50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.790+0000 7f40cf43cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc1b03f0 con 0x556c9ec50160 2024-05-09T02:15:37.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.790+0000 7f40cf43cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc1afc60 con 0x556c9eb664a0 2024-05-09T02:15:37.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.790+0000 7f40cf43cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc1ace60 con 0x556c9ece36a0 2024-05-09T02:15:37.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.790+0000 7f40cf43cb80 10 client.0 _ll_register_callbacks cb 0x556c9ea776c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:37.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:37 smithi196 ceph-mon[34718]: pgmap v133: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 26 MiB/s wr, 660 op/s 2024-05-09T02:15:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.791+0000 7f40bad9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 0x556c9ecead50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.791+0000 7f40bad9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 0x556c9ecead50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:46522/0 (socket says 172.21.15.19:46522) 2024-05-09T02:15:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.791+0000 7f40bad9d700 1 -- 172.21.15.19:0/1260099539 learned_addr learned my addr 172.21.15.19:0/1260099539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.792+0000 7f40b9d9b700 1 --2- 172.21.15.19:0/1260099539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ecea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.793+0000 7f40bad9d700 1 -- 172.21.15.19:0/1260099539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9eb664a0 msgr2=0x556c9ecf5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.793+0000 7f40bad9d700 1 --2- 172.21.15.19:0/1260099539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9eb664a0 0x556c9ecf5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.793+0000 7f40bad9d700 1 -- 172.21.15.19:0/1260099539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 msgr2=0x556c9ecea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:37.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.793+0000 7f40bad9d700 1 --2- 172.21.15.19:0/1260099539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ecea810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.793+0000 7f40bad9d700 1 -- 172.21.15.19:0/1260099539 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c9ecf7f50 con 0x556c9ece36a0 2024-05-09T02:15:37.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.793+0000 7f40bad9d700 1 --2- 172.21.15.19:0/1260099539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 0x556c9ecead50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40b4007e60 tx=0x7f40b40126d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49c15c80c43bd0e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:37.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.793+0000 7f40a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:15:37.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.794+0000 7f40a1ffb700 1 -- 172.21.15.19:0/1260099539 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4004260 con 0x556c9ece36a0 2024-05-09T02:15:37.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.794+0000 7f40a1ffb700 1 -- 172.21.15.19:0/1260099539 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f40b4004400 con 0x556c9ece36a0 2024-05-09T02:15:37.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.795+0000 7f40cf43cb80 1 -- 172.21.15.19:0/1260099539 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a40ad750 con 0x556c9ece36a0 2024-05-09T02:15:37.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.796+0000 7f40a1ffb700 1 -- 172.21.15.19:0/1260099539 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f40b4003680 con 0x556c9ece36a0 2024-05-09T02:15:37.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.798+0000 7f40cf43cb80 10 client.25615 fetch_fsmap learned FSMap version 40 2024-05-09T02:15:37.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.798+0000 7f40cf43cb80 1 -- 172.21.15.19:0/1260099539 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556c9ed802b0 con 0x556c9ece36a0 2024-05-09T02:15:37.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.798+0000 7f40a1ffb700 1 -- 172.21.15.19:0/1260099539 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a40ad750 con 0x556c9ece36a0 2024-05-09T02:15:37.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.798+0000 7f40a1ffb700 1 -- 172.21.15.19:0/1260099539 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f40b40269d0 con 0x556c9ece36a0 2024-05-09T02:15:37.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.799+0000 7f40cf43cb80 10 client.25615 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T02:15:37.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.799+0000 7f40cf43cb80 10 client.25615 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:15:37.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.799+0000 7f40cf43cb80 20 client.25615 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:37.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.799+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/1260099539 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556c9ed80960 0x556c9ed82d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:37.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.799+0000 7f40cf43cb80 4 client.25615 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:15:37.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.799+0000 7f40cf43cb80 1 -- 172.21.15.19:0/1260099539 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c9ec0da70 con 0x556c9ed80960 2024-05-09T02:15:37.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.800+0000 7f40ba59c700 1 --2- 172.21.15.19:0/1260099539 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556c9ed80960 0x556c9ed82d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:37.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.801+0000 7f40a1ffb700 10 client.25615 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:15:37.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.801+0000 7f40ba59c700 1 --2- 172.21.15.19:0/1260099539 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556c9ed80960 0x556c9ed82d80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=beb8f44ec67ddca9 server_cookie=4d06ed54cf746f2e in_seq=0 out_seq=0 2024-05-09T02:15:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.801+0000 7f40a1ffb700 1 -- 172.21.15.19:0/1260099539 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c9ec0da70 con 0x556c9ed80960 2024-05-09T02:15:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.802+0000 7f40a1ffb700 10 client.25615 handle_command_reply: tid=0 2024-05-09T02:15:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.802+0000 7f40cf43cb80 1 client.25615 shutdown 2024-05-09T02:15:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.802+0000 7f40cf43cb80 1 -- 172.21.15.19:0/1260099539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 msgr2=0x556c9ecead50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.802+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/1260099539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 0x556c9ecead50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40b4007e60 tx=0x7f40b40126d0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.802+0000 7f40cf43cb80 1 -- 172.21.15.19:0/1260099539 shutdown_connections 2024-05-09T02:15:37.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.802+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/1260099539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556c9ece36a0 0x556c9ecead50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.802+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/1260099539 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556c9ed80960 0x556c9ed82d80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.802+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/1260099539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556c9eb664a0 0x556c9ecf5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.802+0000 7f40cf43cb80 1 --2- 172.21.15.19:0/1260099539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c9ec50160 0x556c9ecea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.802+0000 7f40cf43cb80 1 -- 172.21.15.19:0/1260099539 >> 172.21.15.19:0/1260099539 conn(0x556c9ebc66a0 msgr2=0x556c9ecee800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:37.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.803+0000 7f40a1ffb700 0 client.25615 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:15:37.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.803+0000 7f40cf43cb80 1 -- 172.21.15.19:0/1260099539 shutdown_connections 2024-05-09T02:15:37.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.803+0000 7f40cf43cb80 1 -- 172.21.15.19:0/1260099539 wait complete. 2024-05-09T02:15:37.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.803+0000 7f40cf43cb80 20 client.25615 trim_cache size 0 max 16384 2024-05-09T02:15:37.811 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:15:37.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.804+0000 7f40c0a26700 1 -- 172.21.15.19:0/2681025753 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f40a80b29f0 msgr2=0x7f40a80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.804+0000 7f40c0a26700 1 --2- 172.21.15.19:0/2681025753 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f40a80b29f0 0x7f40a80b4f00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009900 tx=0x7f40ac00b040 comp rx=0 tx=0).stop 2024-05-09T02:15:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.804+0000 7f40c0a26700 1 -- 172.21.15.19:0/2681025753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40bc108f20 msgr2=0x7f40bc19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.804+0000 7f40c0a26700 1 --2- 172.21.15.19:0/2681025753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40bc108f20 0x7f40bc19b3f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f40a400d980 tx=0x7f40a400dc90 comp rx=0 tx=0).stop 2024-05-09T02:15:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.804+0000 7f40c0a26700 1 -- 172.21.15.19:0/2681025753 shutdown_connections 2024-05-09T02:15:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.804+0000 7f40c0a26700 1 --2- 172.21.15.19:0/2681025753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40bc109870 0x7f40bc19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.804+0000 7f40c0a26700 1 --2- 172.21.15.19:0/2681025753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40bc1045f0 0x7f40bc19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.804+0000 7f40c0a26700 1 --2- 172.21.15.19:0/2681025753 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f40a80b29f0 0x7f40a80b4f00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.804+0000 7f40c0a26700 1 --2- 172.21.15.19:0/2681025753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40bc108f20 0x7f40bc19b3f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.804+0000 7f40c0a26700 1 -- 172.21.15.19:0/2681025753 >> 172.21.15.19:0/2681025753 conn(0x7f40bc100040 msgr2=0x7f40bc1086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.804+0000 7f40c0a26700 1 -- 172.21.15.19:0/2681025753 shutdown_connections 2024-05-09T02:15:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:37.804+0000 7f40c0a26700 1 -- 172.21.15.19:0/2681025753 wait complete. 2024-05-09T02:15:37.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:37 smithi059 ceph-mon[34781]: pgmap v133: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 26 MiB/s wr, 660 op/s 2024-05-09T02:15:39.315 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T02:15:39.579 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:15:40.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:40 smithi019 ceph-mon[30151]: pgmap v134: 97 pgs: 97 active+clean; 3.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 54 MiB/s wr, 2.00k op/s 2024-05-09T02:15:40.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:40 smithi196 ceph-mon[34718]: pgmap v134: 97 pgs: 97 active+clean; 3.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 54 MiB/s wr, 2.00k op/s 2024-05-09T02:15:40.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:40 smithi059 ceph-mon[34781]: pgmap v134: 97 pgs: 97 active+clean; 3.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 54 MiB/s wr, 2.00k op/s 2024-05-09T02:15:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.374+0000 7f393cdce700 1 -- 172.21.15.19:0/1758903430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3938108740 msgr2=0x7f39381100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.374+0000 7f393cdce700 1 --2- 172.21.15.19:0/1758903430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3938108740 0x7f39381100a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f393000b0d0 tx=0x7f393000b3e0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.374+0000 7f393cdce700 1 -- 172.21.15.19:0/1758903430 shutdown_connections 2024-05-09T02:15:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.374+0000 7f393cdce700 1 --2- 172.21.15.19:0/1758903430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3938108740 0x7f39381100a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.374+0000 7f393cdce700 1 --2- 172.21.15.19:0/1758903430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39381034c0 0x7f39381038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.374+0000 7f393cdce700 1 --2- 172.21.15.19:0/1758903430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3938107df0 0x7f3938108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.374+0000 7f393cdce700 1 -- 172.21.15.19:0/1758903430 >> 172.21.15.19:0/1758903430 conn(0x7f39380fedf0 msgr2=0x7f3938101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.374+0000 7f393cdce700 1 -- 172.21.15.19:0/1758903430 shutdown_connections 2024-05-09T02:15:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.374+0000 7f393cdce700 1 -- 172.21.15.19:0/1758903430 wait complete. 2024-05-09T02:15:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.375+0000 7f393cdce700 1 Processor -- start 2024-05-09T02:15:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.375+0000 7f393cdce700 1 -- start start 2024-05-09T02:15:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.375+0000 7f393cdce700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39381034c0 0x7f393819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.375+0000 7f393cdce700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3938107df0 0x7f393819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.375+0000 7f393cdce700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3938108740 0x7f393819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.375+0000 7f393cdce700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39381144c0 con 0x7f3938107df0 2024-05-09T02:15:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.375+0000 7f393cdce700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3938114340 con 0x7f3938108740 2024-05-09T02:15:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.375+0000 7f393cdce700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3938114640 con 0x7f39381034c0 2024-05-09T02:15:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.376+0000 7f3935d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3938107df0 0x7f393819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.376+0000 7f3935d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3938107df0 0x7f393819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34786/0 (socket says 172.21.15.19:34786) 2024-05-09T02:15:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.376+0000 7f3935d9b700 1 -- 172.21.15.19:0/1435127044 learned_addr learned my addr 172.21.15.19:0/1435127044 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.376+0000 7f3936d9d700 1 --2- 172.21.15.19:0/1435127044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3938108740 0x7f393819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.376+0000 7f393659c700 1 --2- 172.21.15.19:0/1435127044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39381034c0 0x7f393819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.376+0000 7f3935d9b700 1 -- 172.21.15.19:0/1435127044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39381034c0 msgr2=0x7f393819adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.376+0000 7f3935d9b700 1 --2- 172.21.15.19:0/1435127044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39381034c0 0x7f393819adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.376+0000 7f3935d9b700 1 -- 172.21.15.19:0/1435127044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3938108740 msgr2=0x7f393819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.376+0000 7f3935d9b700 1 --2- 172.21.15.19:0/1435127044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3938108740 0x7f393819f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.376+0000 7f3935d9b700 1 -- 172.21.15.19:0/1435127044 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393819fc80 con 0x7f3938107df0 2024-05-09T02:15:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.376+0000 7f3935d9b700 1 --2- 172.21.15.19:0/1435127044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3938107df0 0x7f393819b350 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f392c00e970 tx=0x7f392c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.376+0000 7f39277fe700 1 -- 172.21.15.19:0/1435127044 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c004a30 con 0x7f3938107df0 2024-05-09T02:15:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.376+0000 7f393cdce700 1 -- 172.21.15.19:0/1435127044 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393819ff70 con 0x7f3938107df0 2024-05-09T02:15:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.376+0000 7f39277fe700 1 -- 172.21.15.19:0/1435127044 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392c00cc90 con 0x7f3938107df0 2024-05-09T02:15:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.376+0000 7f39277fe700 1 -- 172.21.15.19:0/1435127044 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c005610 con 0x7f3938107df0 2024-05-09T02:15:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.377+0000 7f393cdce700 1 -- 172.21.15.19:0/1435127044 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39381a7890 con 0x7f3938107df0 2024-05-09T02:15:41.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.380+0000 7f39277fe700 1 -- 172.21.15.19:0/1435127044 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f392c021440 con 0x7f3938107df0 2024-05-09T02:15:41.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.381+0000 7f39277fe700 1 --2- 172.21.15.19:0/1435127044 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f39200b29f0 0x7f39200b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.381+0000 7f394b7e4b80 1 Processor -- start 2024-05-09T02:15:41.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.381+0000 7f393659c700 1 --2- 172.21.15.19:0/1435127044 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f39200b29f0 0x7f39200b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.381+0000 7f394b7e4b80 1 -- start start 2024-05-09T02:15:41.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.381+0000 7f39277fe700 1 -- 172.21.15.19:0/1435127044 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f392c0e27c0 con 0x7f3938107df0 2024-05-09T02:15:41.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.381+0000 7f394b7e4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ec35160 0x56483ecce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.381+0000 7f394b7e4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483eb4b4a0 0x56483eccea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.381+0000 7f394b7e4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483ecc86a0 0x56483ecc8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.381+0000 7f393659c700 1 --2- 172.21.15.19:0/1435127044 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f39200b29f0 0x7f39200b4f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3928005950 tx=0x7f392800b410 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:41.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f393659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ec35160 0x56483ecce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f393659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ec35160 0x56483ecce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:53434/0 (socket says 172.21.15.19:53434) 2024-05-09T02:15:41.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f3936d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483ecc86a0 0x56483ecc8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f3935d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483eb4b4a0 0x56483eccea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f3936d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483ecc86a0 0x56483ecc8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34798/0 (socket says 172.21.15.19:34798) 2024-05-09T02:15:41.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f3935d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483eb4b4a0 0x56483eccea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:46542/0 (socket says 172.21.15.19:46542) 2024-05-09T02:15:41.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f393659c700 1 -- 172.21.15.19:0/2675666330 learned_addr learned my addr 172.21.15.19:0/2675666330 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:41.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2675666330 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56483ebeaff0 con 0x56483ecc86a0 2024-05-09T02:15:41.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2675666330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56483eb897f0 con 0x56483ec35160 2024-05-09T02:15:41.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2675666330 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56483ebe6fb0 con 0x56483eb4b4a0 2024-05-09T02:15:41.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f3935d9b700 1 -- 172.21.15.19:0/2675666330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ec35160 msgr2=0x56483ecce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:41.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f3935d9b700 1 --2- 172.21.15.19:0/2675666330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ec35160 0x56483ecce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f3935d9b700 1 -- 172.21.15.19:0/2675666330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483ecc86a0 msgr2=0x56483ecc8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:41.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f3935d9b700 1 --2- 172.21.15.19:0/2675666330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483ecc86a0 0x56483ecc8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f3935d9b700 1 -- 172.21.15.19:0/2675666330 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56483eb81370 con 0x56483eb4b4a0 2024-05-09T02:15:41.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f3936d9d700 1 --2- 172.21.15.19:0/2675666330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483ecc86a0 0x56483ecc8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:15:41.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.382+0000 7f3935d9b700 1 --2- 172.21.15.19:0/2675666330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483eb4b4a0 0x56483eccea30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f392c0dfeb0 tx=0x7f392c0a91b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72449b0c232db2ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:41.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.383+0000 7f3924ff9700 1 -- 172.21.15.19:0/2675666330 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c0b2050 con 0x56483eb4b4a0 2024-05-09T02:15:41.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.383+0000 7f3924ff9700 1 -- 172.21.15.19:0/2675666330 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392c0b21f0 con 0x56483eb4b4a0 2024-05-09T02:15:41.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.383+0000 7f3924ff9700 1 -- 172.21.15.19:0/2675666330 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c0b28a0 con 0x56483eb4b4a0 2024-05-09T02:15:41.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.383+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2675666330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483eb4b4a0 msgr2=0x56483eccea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:41.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.383+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/2675666330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483eb4b4a0 0x56483eccea30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f392c0dfeb0 tx=0x7f392c0a91b0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.384+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2675666330 shutdown_connections 2024-05-09T02:15:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.384+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/2675666330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483ecc86a0 0x56483ecc8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.384+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/2675666330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ec35160 0x56483ecce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.384+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/2675666330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483eb4b4a0 0x56483eccea30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.384+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2675666330 >> 172.21.15.19:0/2675666330 conn(0x56483ebab6a0 msgr2=0x56483ecc3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.384+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2675666330 shutdown_connections 2024-05-09T02:15:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.384+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2675666330 wait complete. 2024-05-09T02:15:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.384+0000 7f394b7e4b80 1 -- start start 2024-05-09T02:15:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.384+0000 7f394b7e4b80 1 Processor -- start 2024-05-09T02:15:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.384+0000 7f394b7e4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483eb4b4a0 0x56483ecd21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.384+0000 7f394b7e4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483ec35160 0x56483ecd2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.384+0000 7f394b7e4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ecc86a0 0x56483ecd2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.384+0000 7f394b7e4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56483ea66ba0 con 0x56483eb4b4a0 2024-05-09T02:15:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f394b7e4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56483eba5150 con 0x56483ecc86a0 2024-05-09T02:15:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f394b7e4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56483ecd51d0 con 0x56483ec35160 2024-05-09T02:15:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f394b7e4b80 10 client.0 _ll_register_callbacks cb 0x56483ea5c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f393659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483eb4b4a0 0x56483ecd21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f393659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483eb4b4a0 0x56483ecd21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34808/0 (socket says 172.21.15.19:34808) 2024-05-09T02:15:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f393659c700 1 -- 172.21.15.19:0/68659929 learned_addr learned my addr 172.21.15.19:0/68659929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f3936d9d700 1 --2- 172.21.15.19:0/68659929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ecc86a0 0x56483ecd2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f3935d9b700 1 --2- 172.21.15.19:0/68659929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483ec35160 0x56483ecd2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f393659c700 1 -- 172.21.15.19:0/68659929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483ec35160 msgr2=0x56483ecd2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f393659c700 1 --2- 172.21.15.19:0/68659929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483ec35160 0x56483ecd2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f393659c700 1 -- 172.21.15.19:0/68659929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ecc86a0 msgr2=0x56483ecd2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f393659c700 1 --2- 172.21.15.19:0/68659929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ecc86a0 0x56483ecd2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f393659c700 1 -- 172.21.15.19:0/68659929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56483ea66490 con 0x56483eb4b4a0 2024-05-09T02:15:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f393659c700 1 --2- 172.21.15.19:0/68659929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483eb4b4a0 0x56483ecd21f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f39280116f0 tx=0x7f39280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828333b5dd652461 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f391dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:15:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f391dffb700 1 -- 172.21.15.19:0/68659929 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392801e830 con 0x56483eb4b4a0 2024-05-09T02:15:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f394b7e4b80 1 -- 172.21.15.19:0/68659929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f392c0b21f0 con 0x56483eb4b4a0 2024-05-09T02:15:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.385+0000 7f391dffb700 1 -- 172.21.15.19:0/68659929 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f392801e9d0 con 0x56483eb4b4a0 2024-05-09T02:15:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.386+0000 7f391dffb700 1 -- 172.21.15.19:0/68659929 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f392801ec10 con 0x56483eb4b4a0 2024-05-09T02:15:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.386+0000 7f394b7e4b80 10 client.15879 fetch_fsmap learned FSMap version 40 2024-05-09T02:15:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.386+0000 7f394b7e4b80 1 -- 172.21.15.19:0/68659929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56483ecdfc20 con 0x56483eb4b4a0 2024-05-09T02:15:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.386+0000 7f391dffb700 1 -- 172.21.15.19:0/68659929 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392c0b21f0 con 0x56483eb4b4a0 2024-05-09T02:15:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.386+0000 7f391dffb700 1 -- 172.21.15.19:0/68659929 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f3928026020 con 0x56483eb4b4a0 2024-05-09T02:15:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.386+0000 7f394b7e4b80 10 client.15879 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T02:15:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.386+0000 7f394b7e4b80 10 client.15879 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:15:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.386+0000 7f394b7e4b80 20 client.15879 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.386+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/68659929 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x56483ece2200 0x56483ece46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.386+0000 7f394b7e4b80 4 client.15879 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:15:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.386+0000 7f394b7e4b80 1 -- 172.21.15.19:0/68659929 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56483ebf89d0 con 0x56483ece2200 2024-05-09T02:15:41.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.386+0000 7f3935d9b700 1 --2- 172.21.15.19:0/68659929 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x56483ece2200 0x56483ece46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.387+0000 7f391dffb700 10 client.15879 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:15:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.387+0000 7f3935d9b700 1 --2- 172.21.15.19:0/68659929 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x56483ece2200 0x56483ece46b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d20a19f2e236412e server_cookie=fa1dc41d07f95a13 in_seq=0 out_seq=0 2024-05-09T02:15:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.388+0000 7f391dffb700 1 -- 172.21.15.19:0/68659929 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56483ebf89d0 con 0x56483ece2200 2024-05-09T02:15:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.388+0000 7f391dffb700 10 client.15879 handle_command_reply: tid=0 2024-05-09T02:15:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.388+0000 7f394b7e4b80 1 client.15879 shutdown 2024-05-09T02:15:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.388+0000 7f394b7e4b80 1 -- 172.21.15.19:0/68659929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483eb4b4a0 msgr2=0x56483ecd21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.388+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/68659929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483eb4b4a0 0x56483ecd21f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f39280116f0 tx=0x7f39280047c0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.388+0000 7f394b7e4b80 1 -- 172.21.15.19:0/68659929 shutdown_connections 2024-05-09T02:15:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.388+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/68659929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ecc86a0 0x56483ecd2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.388+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/68659929 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x56483ece2200 0x56483ece46b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.388+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/68659929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483eb4b4a0 0x56483ecd21f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.388+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/68659929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483ec35160 0x56483ecd2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.388+0000 7f394b7e4b80 1 -- 172.21.15.19:0/68659929 >> 172.21.15.19:0/68659929 conn(0x56483ebab6a0 msgr2=0x56483ecc31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.388+0000 7f391dffb700 0 client.15879 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:15:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.388+0000 7f394b7e4b80 1 -- 172.21.15.19:0/68659929 shutdown_connections 2024-05-09T02:15:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.388+0000 7f394b7e4b80 1 -- 172.21.15.19:0/68659929 wait complete. 2024-05-09T02:15:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.388+0000 7f394b7e4b80 20 client.15879 trim_cache size 0 max 16384 2024-05-09T02:15:41.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.403+0000 7f394b7e4b80 1 Processor -- start 2024-05-09T02:15:41.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.403+0000 7f394b7e4b80 1 -- start start 2024-05-09T02:15:41.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f394b7e4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483eb4b4a0 0x56483eccd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f394b7e4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483ec35160 0x56483ecc7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f394b7e4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ecc86a0 0x56483ecc7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f394b7e4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39381aef60 con 0x56483eb4b4a0 2024-05-09T02:15:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f394b7e4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39381b0b60 con 0x56483ecc86a0 2024-05-09T02:15:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f394b7e4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39381af630 con 0x56483ec35160 2024-05-09T02:15:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f393659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483eb4b4a0 0x56483eccd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f393659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483eb4b4a0 0x56483eccd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34822/0 (socket says 172.21.15.19:34822) 2024-05-09T02:15:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f393659c700 1 -- 172.21.15.19:0/2615377141 learned_addr learned my addr 172.21.15.19:0/2615377141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f3935d9b700 1 --2- 172.21.15.19:0/2615377141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483ec35160 0x56483ecc7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f393659c700 1 -- 172.21.15.19:0/2615377141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483ec35160 msgr2=0x56483ecc7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f393659c700 1 --2- 172.21.15.19:0/2615377141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483ec35160 0x56483ecc7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f3936d9d700 1 --2- 172.21.15.19:0/2615377141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ecc86a0 0x56483ecc7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f393659c700 1 -- 172.21.15.19:0/2615377141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ecc86a0 msgr2=0x56483ecc7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f393659c700 1 --2- 172.21.15.19:0/2615377141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ecc86a0 0x56483ecc7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f393659c700 1 -- 172.21.15.19:0/2615377141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56483ece4c40 con 0x56483eb4b4a0 2024-05-09T02:15:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f3936d9d700 1 --2- 172.21.15.19:0/2615377141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ecc86a0 0x56483ecc7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:15:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.404+0000 7f393659c700 1 --2- 172.21.15.19:0/2615377141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483eb4b4a0 0x56483eccd9f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3928000c00 tx=0x7f3928010fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e158238b3db2ffd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.405+0000 7f391ffff700 1 -- 172.21.15.19:0/2615377141 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928030680 con 0x56483eb4b4a0 2024-05-09T02:15:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.405+0000 7f391ffff700 1 -- 172.21.15.19:0/2615377141 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3928030920 con 0x56483eb4b4a0 2024-05-09T02:15:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.405+0000 7f391ffff700 1 -- 172.21.15.19:0/2615377141 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928031860 con 0x56483eb4b4a0 2024-05-09T02:15:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.405+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2615377141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483eb4b4a0 msgr2=0x56483eccd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.405+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/2615377141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483eb4b4a0 0x56483eccd9f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3928000c00 tx=0x7f3928010fc0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.405+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2615377141 shutdown_connections 2024-05-09T02:15:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.405+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/2615377141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483ecc86a0 0x56483ecc7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.405+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/2615377141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483eb4b4a0 0x56483eccd9f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.405+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/2615377141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483ec35160 0x56483ecc7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.405+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2615377141 >> 172.21.15.19:0/2615377141 conn(0x56483ebab6a0 msgr2=0x56483eccd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.405+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2615377141 shutdown_connections 2024-05-09T02:15:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.405+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2615377141 wait complete. 2024-05-09T02:15:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.406+0000 7f394b7e4b80 1 -- start start 2024-05-09T02:15:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.406+0000 7f394b7e4b80 1 Processor -- start 2024-05-09T02:15:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.406+0000 7f394b7e4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483eb4b4a0 0x56483ecda4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.406+0000 7f394b7e4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483ec35160 0x56483ecdaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.406+0000 7f394b7e4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483ecc86a0 0x56483ece9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.406+0000 7f394b7e4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39381b0490 con 0x56483ecc86a0 2024-05-09T02:15:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.406+0000 7f394b7e4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39381afd00 con 0x56483eb4b4a0 2024-05-09T02:15:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.406+0000 7f394b7e4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39381acf00 con 0x56483ec35160 2024-05-09T02:15:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.406+0000 7f394b7e4b80 10 client.0 _ll_register_callbacks cb 0x56483ea5c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.406+0000 7f3936d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483ecc86a0 0x56483ece9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f3936d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483ecc86a0 0x56483ece9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34838/0 (socket says 172.21.15.19:34838) 2024-05-09T02:15:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f3936d9d700 1 -- 172.21.15.19:0/2142313782 learned_addr learned my addr 172.21.15.19:0/2142313782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f3935d9b700 1 --2- 172.21.15.19:0/2142313782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483ec35160 0x56483ecdaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f393659c700 1 --2- 172.21.15.19:0/2142313782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483eb4b4a0 0x56483ecda4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f3936d9d700 1 -- 172.21.15.19:0/2142313782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483ec35160 msgr2=0x56483ecdaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f3936d9d700 1 --2- 172.21.15.19:0/2142313782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483ec35160 0x56483ecdaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f3936d9d700 1 -- 172.21.15.19:0/2142313782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483eb4b4a0 msgr2=0x56483ecda4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f3936d9d700 1 --2- 172.21.15.19:0/2142313782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483eb4b4a0 0x56483ecda4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f3936d9d700 1 -- 172.21.15.19:0/2142313782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56483ece98a0 con 0x56483ecc86a0 2024-05-09T02:15:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f3936d9d700 1 --2- 172.21.15.19:0/2142313782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483ecc86a0 0x56483ece9120 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3930015000 tx=0x7f393000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5555e94621359e5d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f391dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:15:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f391dffb700 1 -- 172.21.15.19:0/2142313782 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930003e10 con 0x56483ecc86a0 2024-05-09T02:15:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f391dffb700 1 -- 172.21.15.19:0/2142313782 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f3930005070 con 0x56483ecc86a0 2024-05-09T02:15:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2142313782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3928030920 con 0x56483ecc86a0 2024-05-09T02:15:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f391dffb700 1 -- 172.21.15.19:0/2142313782 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3930033050 con 0x56483ecc86a0 2024-05-09T02:15:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f394b7e4b80 10 client.15885 fetch_fsmap learned FSMap version 40 2024-05-09T02:15:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.407+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2142313782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56483ed4c4b0 con 0x56483ecc86a0 2024-05-09T02:15:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.408+0000 7f391dffb700 1 -- 172.21.15.19:0/2142313782 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3928030920 con 0x56483ecc86a0 2024-05-09T02:15:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.408+0000 7f391dffb700 1 -- 172.21.15.19:0/2142313782 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f393002e030 con 0x56483ecc86a0 2024-05-09T02:15:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.408+0000 7f394b7e4b80 10 client.15885 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T02:15:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.408+0000 7f394b7e4b80 10 client.15885 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:15:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.408+0000 7f394b7e4b80 20 client.15885 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.408+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/2142313782 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x56483ed4cd00 0x56483ed4f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.408+0000 7f394b7e4b80 4 client.15885 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:15:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.408+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2142313782 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56483ebf2a70 con 0x56483ed4cd00 2024-05-09T02:15:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.408+0000 7f393659c700 1 --2- 172.21.15.19:0/2142313782 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x56483ed4cd00 0x56483ed4f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f391dffb700 10 client.15885 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:15:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f393659c700 1 --2- 172.21.15.19:0/2142313782 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x56483ed4cd00 0x56483ed4f120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=efa204cf2d4f18c6 server_cookie=e4a92aa129ba7239 in_seq=0 out_seq=0 2024-05-09T02:15:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f391dffb700 1 -- 172.21.15.19:0/2142313782 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56483ebf2a70 con 0x56483ed4cd00 2024-05-09T02:15:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f391dffb700 10 client.15885 handle_command_reply: tid=0 2024-05-09T02:15:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f394b7e4b80 1 client.15885 shutdown 2024-05-09T02:15:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2142313782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483ecc86a0 msgr2=0x56483ece9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/2142313782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483ecc86a0 0x56483ece9120 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3930015000 tx=0x7f393000be10 comp rx=0 tx=0).stop 2024-05-09T02:15:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2142313782 shutdown_connections 2024-05-09T02:15:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/2142313782 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x56483ed4cd00 0x56483ed4f120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/2142313782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56483ecc86a0 0x56483ece9120 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/2142313782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56483eb4b4a0 0x56483ecda4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f394b7e4b80 1 --2- 172.21.15.19:0/2142313782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56483ec35160 0x56483ecdaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2142313782 >> 172.21.15.19:0/2142313782 conn(0x56483ebab6a0 msgr2=0x56483ecc7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:41.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f391dffb700 0 client.15885 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:15:41.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2142313782 shutdown_connections 2024-05-09T02:15:41.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f394b7e4b80 1 -- 172.21.15.19:0/2142313782 wait complete. 2024-05-09T02:15:41.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.409+0000 7f394b7e4b80 20 client.15885 trim_cache size 0 max 16384 2024-05-09T02:15:41.417 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:15:41.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.410+0000 7f393cdce700 1 -- 172.21.15.19:0/1435127044 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f39200b29f0 msgr2=0x7f39200b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:41.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.410+0000 7f393cdce700 1 --2- 172.21.15.19:0/1435127044 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f39200b29f0 0x7f39200b4f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3928005950 tx=0x7f392800b410 comp rx=0 tx=0).stop 2024-05-09T02:15:41.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.410+0000 7f393cdce700 1 -- 172.21.15.19:0/1435127044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3938107df0 msgr2=0x7f393819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:41.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.410+0000 7f393cdce700 1 --2- 172.21.15.19:0/1435127044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3938107df0 0x7f393819b350 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f392c00e970 tx=0x7f392c00ec80 comp rx=0 tx=0).stop 2024-05-09T02:15:41.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.410+0000 7f393cdce700 1 -- 172.21.15.19:0/1435127044 shutdown_connections 2024-05-09T02:15:41.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.410+0000 7f393cdce700 1 --2- 172.21.15.19:0/1435127044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3938108740 0x7f393819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.410+0000 7f393cdce700 1 --2- 172.21.15.19:0/1435127044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39381034c0 0x7f393819adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.410+0000 7f393cdce700 1 --2- 172.21.15.19:0/1435127044 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f39200b29f0 0x7f39200b4f00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.410+0000 7f393cdce700 1 --2- 172.21.15.19:0/1435127044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3938107df0 0x7f393819b350 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:41.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.410+0000 7f393cdce700 1 -- 172.21.15.19:0/1435127044 >> 172.21.15.19:0/1435127044 conn(0x7f39380fedf0 msgr2=0x7f3938107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:41.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.410+0000 7f393cdce700 1 -- 172.21.15.19:0/1435127044 shutdown_connections 2024-05-09T02:15:41.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:41.410+0000 7f393cdce700 1 -- 172.21.15.19:0/1435127044 wait complete. 2024-05-09T02:15:42.284 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T02:15:42.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:42 smithi196 ceph-mon[34718]: pgmap v135: 97 pgs: 97 active+clean; 3.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 39 MiB/s wr, 1.68k op/s 2024-05-09T02:15:42.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:42 smithi019 ceph-mon[30151]: pgmap v135: 97 pgs: 97 active+clean; 3.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 39 MiB/s wr, 1.68k op/s 2024-05-09T02:15:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 240.0 s, 424.7 tps, lat 269.146 ms stddev 1068.217 2024-05-09T02:15:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:15:42.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:42 smithi059 ceph-mon[34781]: pgmap v135: 97 pgs: 97 active+clean; 3.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 39 MiB/s wr, 1.68k op/s 2024-05-09T02:15:43.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:43 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:15:43.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:43 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:15:43.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:43 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:15:44.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:44 smithi019 ceph-mon[30151]: pgmap v136: 97 pgs: 97 active+clean; 3.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 40 MiB/s wr, 1.69k op/s 2024-05-09T02:15:44.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:44 smithi196 ceph-mon[34718]: pgmap v136: 97 pgs: 97 active+clean; 3.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 40 MiB/s wr, 1.69k op/s 2024-05-09T02:15:44.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:44 smithi059 ceph-mon[34781]: pgmap v136: 97 pgs: 97 active+clean; 3.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 40 MiB/s wr, 1.69k op/s 2024-05-09T02:15:44.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.957+0000 7f2534a77700 1 -- 172.21.15.19:0/593240726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530103250 msgr2=0x7f2530103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:44.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.957+0000 7f2534a77700 1 --2- 172.21.15.19:0/593240726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530103250 0x7f2530103660 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f252801c0d0 tx=0x7f252801c3e0 comp rx=0 tx=0).stop 2024-05-09T02:15:44.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.958+0000 7f2534a77700 1 -- 172.21.15.19:0/593240726 shutdown_connections 2024-05-09T02:15:44.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.958+0000 7f2534a77700 1 --2- 172.21.15.19:0/593240726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25301086a0 0x7f2530108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:44.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.958+0000 7f2534a77700 1 --2- 172.21.15.19:0/593240726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530103250 0x7f2530103660 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:44.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.958+0000 7f2534a77700 1 --2- 172.21.15.19:0/593240726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530103ba0 0x7f25301080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:44.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.958+0000 7f2534a77700 1 -- 172.21.15.19:0/593240726 >> 172.21.15.19:0/593240726 conn(0x7f25300fedc0 msgr2=0x7f25301011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:44.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.960+0000 7f2534a77700 1 -- 172.21.15.19:0/593240726 shutdown_connections 2024-05-09T02:15:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.962+0000 7f2534a77700 1 -- 172.21.15.19:0/593240726 wait complete. 2024-05-09T02:15:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.963+0000 7f2534a77700 1 Processor -- start 2024-05-09T02:15:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.964+0000 7f2534a77700 1 -- start start 2024-05-09T02:15:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.964+0000 7f2534a77700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530103250 0x7f2530110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.964+0000 7f2534a77700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2530103ba0 0x7f25301105b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.964+0000 7f2534a77700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25301086a0 0x7f2530074510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.964+0000 7f2534a77700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f253010bf80 con 0x7f2530103250 2024-05-09T02:15:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.964+0000 7f2534a77700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f253010be00 con 0x7f25301086a0 2024-05-09T02:15:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.964+0000 7f2534a77700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f253010c100 con 0x7f2530103ba0 2024-05-09T02:15:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.965+0000 7f252e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530103250 0x7f2530110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.965+0000 7f252e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530103250 0x7f2530110050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34854/0 (socket says 172.21.15.19:34854) 2024-05-09T02:15:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.965+0000 7f252e59c700 1 -- 172.21.15.19:0/1329739430 learned_addr learned my addr 172.21.15.19:0/1329739430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.966+0000 7f252e59c700 1 -- 172.21.15.19:0/1329739430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2530103ba0 msgr2=0x7f25301105b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T02:15:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.966+0000 7f252e59c700 1 --2- 172.21.15.19:0/1329739430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2530103ba0 0x7f25301105b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.966+0000 7f252e59c700 1 -- 172.21.15.19:0/1329739430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25301086a0 msgr2=0x7f2530074510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T02:15:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.966+0000 7f252e59c700 1 --2- 172.21.15.19:0/1329739430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25301086a0 0x7f2530074510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.966+0000 7f252e59c700 1 -- 172.21.15.19:0/1329739430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2530074be0 con 0x7f2530103250 2024-05-09T02:15:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.970+0000 7f252e59c700 1 --2- 172.21.15.19:0/1329739430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530103250 0x7f2530110050 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f252801c860 tx=0x7f2528019a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:44.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.971+0000 7f251f7fe700 1 -- 172.21.15.19:0/1329739430 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528030070 con 0x7f2530103250 2024-05-09T02:15:44.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.971+0000 7f251f7fe700 1 -- 172.21.15.19:0/1329739430 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2528005770 con 0x7f2530103250 2024-05-09T02:15:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.971+0000 7f2534a77700 1 -- 172.21.15.19:0/1329739430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2530074e70 con 0x7f2530103250 2024-05-09T02:15:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.972+0000 7f251f7fe700 1 -- 172.21.15.19:0/1329739430 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528030070 con 0x7f2530103250 2024-05-09T02:15:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.972+0000 7f2534a77700 1 -- 172.21.15.19:0/1329739430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2530075350 con 0x7f2530103250 2024-05-09T02:15:44.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.977+0000 7f254348db80 1 Processor -- start 2024-05-09T02:15:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.978+0000 7f254348db80 1 -- start start 2024-05-09T02:15:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.978+0000 7f254348db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4786d00 0x55dee488f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.978+0000 7f254348db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 0x55dee488f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.979+0000 7f254348db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4889400 0x55dee4889810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.979+0000 7f254348db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dee47ac0a0 con 0x55dee4889400 2024-05-09T02:15:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.979+0000 7f254348db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dee47519a0 con 0x55dee470c4a0 2024-05-09T02:15:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.979+0000 7f254348db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dee47a8060 con 0x55dee4786d00 2024-05-09T02:15:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.979+0000 7f252ed9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4786d00 0x55dee488f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:44.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.979+0000 7f252ed9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4786d00 0x55dee488f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:46592/0 (socket says 172.21.15.19:46592) 2024-05-09T02:15:44.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.979+0000 7f252ed9d700 1 -- 172.21.15.19:0/2146922298 learned_addr learned my addr 172.21.15.19:0/2146922298 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.983+0000 7f252dd9b700 1 --2- 172.21.15.19:0/2146922298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4889400 0x55dee4889810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.984+0000 7f252ed9d700 1 -- 172.21.15.19:0/2146922298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 msgr2=0x55dee488f780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:15:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.984+0000 7f252ed9d700 1 --2- 172.21.15.19:0/2146922298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 0x55dee488f780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.984+0000 7f252ed9d700 1 -- 172.21.15.19:0/2146922298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4889400 msgr2=0x55dee4889810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.984+0000 7f252ed9d700 1 --2- 172.21.15.19:0/2146922298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4889400 0x55dee4889810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.984+0000 7f252ed9d700 1 -- 172.21.15.19:0/2146922298 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee4742500 con 0x55dee4786d00 2024-05-09T02:15:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.984+0000 7f252ed9d700 1 --2- 172.21.15.19:0/2146922298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4786d00 0x55dee488f240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2520009780 tx=0x7f2520009a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71ef397fd994dc1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:44.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.986+0000 7f251cff9700 1 -- 172.21.15.19:0/2146922298 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2520015020 con 0x55dee4786d00 2024-05-09T02:15:44.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.986+0000 7f251cff9700 1 -- 172.21.15.19:0/2146922298 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2520011ce0 con 0x55dee4786d00 2024-05-09T02:15:44.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.988+0000 7f254348db80 1 -- 172.21.15.19:0/2146922298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4786d00 msgr2=0x55dee488f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:44.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.988+0000 7f254348db80 1 --2- 172.21.15.19:0/2146922298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4786d00 0x55dee488f240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2520009780 tx=0x7f2520009a90 comp rx=0 tx=0).stop 2024-05-09T02:15:44.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.988+0000 7f251f7fe700 1 -- 172.21.15.19:0/1329739430 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2528033070 con 0x7f2530103250 2024-05-09T02:15:44.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.989+0000 7f251f7fe700 1 --2- 172.21.15.19:0/1329739430 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f25180b2ec0 0x7f25180b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:44.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.990+0000 7f251f7fe700 1 -- 172.21.15.19:0/1329739430 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f2528020030 con 0x7f2530103250 2024-05-09T02:15:44.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.991+0000 7f252ed9d700 1 --2- 172.21.15.19:0/1329739430 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f25180b2ec0 0x7f25180b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:44.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.992+0000 7f254348db80 1 -- 172.21.15.19:0/2146922298 shutdown_connections 2024-05-09T02:15:44.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.992+0000 7f254348db80 1 --2- 172.21.15.19:0/2146922298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4889400 0x55dee4889810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:44.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.992+0000 7f254348db80 1 --2- 172.21.15.19:0/2146922298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4786d00 0x55dee488f240 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:44.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.992+0000 7f254348db80 1 --2- 172.21.15.19:0/2146922298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 0x55dee488f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:44.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.992+0000 7f254348db80 1 -- 172.21.15.19:0/2146922298 >> 172.21.15.19:0/2146922298 conn(0x55dee470ff70 msgr2=0x55dee4884220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:44.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.992+0000 7f252ed9d700 1 --2- 172.21.15.19:0/1329739430 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f25180b2ec0 0x7f25180b53d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2520009780 tx=0x7f2520019410 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:44.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.994+0000 7f254348db80 1 -- 172.21.15.19:0/2146922298 shutdown_connections 2024-05-09T02:15:44.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.994+0000 7f254348db80 1 -- 172.21.15.19:0/2146922298 wait complete. 2024-05-09T02:15:44.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.995+0000 7f254348db80 1 -- start start 2024-05-09T02:15:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.995+0000 7f254348db80 1 Processor -- start 2024-05-09T02:15:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.996+0000 7f254348db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee470c4a0 0x55dee4892f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.996+0000 7f254348db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee4786d00 0x55dee4893440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.996+0000 7f252dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee4786d00 0x55dee4893440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.996+0000 7f252dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee4786d00 0x55dee4893440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:53480/0 (socket says 172.21.15.19:53480) 2024-05-09T02:15:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.996+0000 7f252dd9b700 1 -- 172.21.15.19:0/3094900128 learned_addr learned my addr 172.21.15.19:0/3094900128 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.997+0000 7f254348db80 1 --2- 172.21.15.19:0/3094900128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4889400 0x55dee4893980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.997+0000 7f254348db80 1 -- 172.21.15.19:0/3094900128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dee488a1b0 con 0x55dee4889400 2024-05-09T02:15:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.997+0000 7f254348db80 1 -- 172.21.15.19:0/3094900128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dee4746b20 con 0x55dee4786d00 2024-05-09T02:15:45.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.997+0000 7f254348db80 1 -- 172.21.15.19:0/3094900128 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dee4896020 con 0x55dee470c4a0 2024-05-09T02:15:45.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.998+0000 7f254348db80 10 client.0 _ll_register_callbacks cb 0x55dee461d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:45.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.998+0000 7f252e59c700 1 --2- 172.21.15.19:0/3094900128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee470c4a0 0x55dee4892f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:45.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:44.999+0000 7f252ed9d700 1 --2- 172.21.15.19:0/3094900128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4889400 0x55dee4893980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:45.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.000+0000 7f252e59c700 1 -- 172.21.15.19:0/3094900128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee4786d00 msgr2=0x55dee4893440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.000+0000 7f252e59c700 1 --2- 172.21.15.19:0/3094900128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee4786d00 0x55dee4893440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.000+0000 7f252e59c700 1 -- 172.21.15.19:0/3094900128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4889400 msgr2=0x55dee4893980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.000+0000 7f252e59c700 1 --2- 172.21.15.19:0/3094900128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4889400 0x55dee4893980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.000+0000 7f252e59c700 1 -- 172.21.15.19:0/3094900128 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee4627490 con 0x55dee470c4a0 2024-05-09T02:15:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.001+0000 7f252e59c700 1 --2- 172.21.15.19:0/3094900128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee470c4a0 0x55dee4892f00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f25280f1680 tx=0x7f2528005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd48c7da75516402 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.001+0000 7f2515ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:15:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.001+0000 7f2515ffb700 1 -- 172.21.15.19:0/3094900128 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528030070 con 0x55dee470c4a0 2024-05-09T02:15:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.001+0000 7f2515ffb700 1 -- 172.21.15.19:0/3094900128 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f25280b7890 con 0x55dee470c4a0 2024-05-09T02:15:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.001+0000 7f254348db80 1 -- 172.21.15.19:0/3094900128 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2520011ce0 con 0x55dee470c4a0 2024-05-09T02:15:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.002+0000 7f2515ffb700 1 -- 172.21.15.19:0/3094900128 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f252802a530 con 0x55dee470c4a0 2024-05-09T02:15:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.002+0000 7f254348db80 10 client.25639 fetch_fsmap learned FSMap version 41 2024-05-09T02:15:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.002+0000 7f254348db80 1 -- 172.21.15.19:0/3094900128 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55dee48a09e0 con 0x55dee470c4a0 2024-05-09T02:15:45.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.002+0000 7f2515ffb700 1 -- 172.21.15.19:0/3094900128 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2520011ce0 con 0x55dee470c4a0 2024-05-09T02:15:45.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.003+0000 7f2515ffb700 1 -- 172.21.15.19:0/3094900128 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f2528022070 con 0x55dee470c4a0 2024-05-09T02:15:45.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.003+0000 7f254348db80 10 client.25639 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:15:45.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.003+0000 7f254348db80 10 client.25639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:15:45.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.003+0000 7f254348db80 20 client.25639 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:45.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.003+0000 7f254348db80 1 --2- 172.21.15.19:0/3094900128 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dee48a30f0 0x55dee48a5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:45.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.003+0000 7f254348db80 4 client.25639 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:15:45.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.003+0000 7f254348db80 1 -- 172.21.15.19:0/3094900128 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dee47b9a80 con 0x55dee48a30f0 2024-05-09T02:15:45.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.005+0000 7f252dd9b700 1 --2- 172.21.15.19:0/3094900128 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dee48a30f0 0x55dee48a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.007+0000 7f2515ffb700 10 client.25639 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.009+0000 7f252dd9b700 1 --2- 172.21.15.19:0/3094900128 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dee48a30f0 0x55dee48a5580 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f942f157cedd9d41 server_cookie=66dc5ba37e98d984 in_seq=0 out_seq=0 2024-05-09T02:15:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.010+0000 7f2515ffb700 1 -- 172.21.15.19:0/3094900128 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dee47b9a80 con 0x55dee48a30f0 2024-05-09T02:15:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.010+0000 7f2515ffb700 10 client.25639 handle_command_reply: tid=0 2024-05-09T02:15:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.011+0000 7f254348db80 1 client.25639 shutdown 2024-05-09T02:15:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.011+0000 7f254348db80 1 -- 172.21.15.19:0/3094900128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee470c4a0 msgr2=0x55dee4892f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.011+0000 7f254348db80 1 --2- 172.21.15.19:0/3094900128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee470c4a0 0x55dee4892f00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f25280f1680 tx=0x7f2528005b50 comp rx=0 tx=0).stop 2024-05-09T02:15:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.012+0000 7f254348db80 1 -- 172.21.15.19:0/3094900128 shutdown_connections 2024-05-09T02:15:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.012+0000 7f254348db80 1 --2- 172.21.15.19:0/3094900128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4889400 0x55dee4893980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.012+0000 7f254348db80 1 --2- 172.21.15.19:0/3094900128 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dee48a30f0 0x55dee48a5580 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.012+0000 7f254348db80 1 --2- 172.21.15.19:0/3094900128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee470c4a0 0x55dee4892f00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.012+0000 7f254348db80 1 --2- 172.21.15.19:0/3094900128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee4786d00 0x55dee4893440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.012+0000 7f254348db80 1 -- 172.21.15.19:0/3094900128 >> 172.21.15.19:0/3094900128 conn(0x55dee470ff70 msgr2=0x55dee487dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.012+0000 7f2515ffb700 0 client.25639 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.012+0000 7f254348db80 1 -- 172.21.15.19:0/3094900128 shutdown_connections 2024-05-09T02:15:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.012+0000 7f254348db80 1 -- 172.21.15.19:0/3094900128 wait complete. 2024-05-09T02:15:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.012+0000 7f254348db80 20 client.25639 trim_cache size 0 max 16384 2024-05-09T02:15:45.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.042+0000 7f254348db80 1 Processor -- start 2024-05-09T02:15:45.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.042+0000 7f254348db80 1 -- start start 2024-05-09T02:15:45.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.042+0000 7f254348db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 0x55dee4891b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:45.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.042+0000 7f254348db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4786d00 0x55dee488d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:45.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.042+0000 7f254348db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4889400 0x55dee489eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:45.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.042+0000 7f254348db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f251000d720 con 0x55dee4889400 2024-05-09T02:15:45.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.042+0000 7f254348db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f251000f320 con 0x55dee470c4a0 2024-05-09T02:15:45.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.042+0000 7f254348db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f251000ddf0 con 0x55dee4786d00 2024-05-09T02:15:45.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.043+0000 7f252e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 0x55dee4891b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:45.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.043+0000 7f252e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 0x55dee4891b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:53496/0 (socket says 172.21.15.19:53496) 2024-05-09T02:15:45.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.043+0000 7f252e59c700 1 -- 172.21.15.19:0/1767027616 learned_addr learned my addr 172.21.15.19:0/1767027616 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:45.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.043+0000 7f252e59c700 1 -- 172.21.15.19:0/1767027616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4786d00 msgr2=0x55dee488d310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:15:45.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.043+0000 7f252e59c700 1 --2- 172.21.15.19:0/1767027616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4786d00 0x55dee488d310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.043+0000 7f252e59c700 1 -- 172.21.15.19:0/1767027616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4889400 msgr2=0x55dee489eef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:15:45.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.043+0000 7f252e59c700 1 --2- 172.21.15.19:0/1767027616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4889400 0x55dee489eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.043+0000 7f252e59c700 1 -- 172.21.15.19:0/1767027616 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee4891470 con 0x55dee470c4a0 2024-05-09T02:15:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.044+0000 7f252e59c700 1 --2- 172.21.15.19:0/1767027616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 0x55dee4891b60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f25280bb230 tx=0x7f25280bad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89b8ad79d029aa6e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.044+0000 7f2517fff700 1 -- 172.21.15.19:0/1767027616 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528030070 con 0x55dee470c4a0 2024-05-09T02:15:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.044+0000 7f2517fff700 1 -- 172.21.15.19:0/1767027616 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25280bb9a0 con 0x55dee470c4a0 2024-05-09T02:15:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.044+0000 7f2517fff700 1 -- 172.21.15.19:0/1767027616 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528005e80 con 0x55dee470c4a0 2024-05-09T02:15:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.045+0000 7f254348db80 1 -- 172.21.15.19:0/1767027616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 msgr2=0x55dee4891b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.045+0000 7f254348db80 1 --2- 172.21.15.19:0/1767027616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 0x55dee4891b60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f25280bb230 tx=0x7f25280bad20 comp rx=0 tx=0).stop 2024-05-09T02:15:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.046+0000 7f254348db80 1 -- 172.21.15.19:0/1767027616 shutdown_connections 2024-05-09T02:15:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.046+0000 7f254348db80 1 --2- 172.21.15.19:0/1767027616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4889400 0x55dee489eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.046+0000 7f254348db80 1 --2- 172.21.15.19:0/1767027616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 0x55dee4891b60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.046+0000 7f254348db80 1 --2- 172.21.15.19:0/1767027616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4786d00 0x55dee488d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.046+0000 7f254348db80 1 -- 172.21.15.19:0/1767027616 >> 172.21.15.19:0/1767027616 conn(0x55dee470ff70 msgr2=0x55dee488cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.046+0000 7f254348db80 1 -- 172.21.15.19:0/1767027616 shutdown_connections 2024-05-09T02:15:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.046+0000 7f254348db80 1 -- 172.21.15.19:0/1767027616 wait complete. 2024-05-09T02:15:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.046+0000 7f254348db80 1 -- start start 2024-05-09T02:15:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.047+0000 7f254348db80 1 Processor -- start 2024-05-09T02:15:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.047+0000 7f254348db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 0x55dee489b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.048+0000 7f252e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 0x55dee489b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.048+0000 7f252e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 0x55dee489b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:53506/0 (socket says 172.21.15.19:53506) 2024-05-09T02:15:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.048+0000 7f254348db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4786d00 0x55dee489b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.048+0000 7f252e59c700 1 -- 172.21.15.19:0/3444544038 learned_addr learned my addr 172.21.15.19:0/3444544038 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.048+0000 7f254348db80 1 --2- 172.21.15.19:0/3444544038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4889400 0x55dee48a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.048+0000 7f252dd9b700 1 --2- 172.21.15.19:0/3444544038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4786d00 0x55dee489b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.048+0000 7f254348db80 1 -- 172.21.15.19:0/3444544038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f251000ec50 con 0x55dee4786d00 2024-05-09T02:15:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.048+0000 7f254348db80 1 -- 172.21.15.19:0/3444544038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f251000e4c0 con 0x55dee470c4a0 2024-05-09T02:15:45.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.048+0000 7f254348db80 1 -- 172.21.15.19:0/3444544038 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f251000b6c0 con 0x55dee4889400 2024-05-09T02:15:45.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.048+0000 7f252ed9d700 1 --2- 172.21.15.19:0/3444544038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4889400 0x55dee48a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:45.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.048+0000 7f254348db80 10 client.0 _ll_register_callbacks cb 0x55dee461d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:45.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.049+0000 7f252e59c700 1 -- 172.21.15.19:0/3444544038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4889400 msgr2=0x55dee48a9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:45.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.049+0000 7f252e59c700 1 --2- 172.21.15.19:0/3444544038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4889400 0x55dee48a9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.049+0000 7f252e59c700 1 -- 172.21.15.19:0/3444544038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4786d00 msgr2=0x55dee489b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:45.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.049+0000 7f252e59c700 1 --2- 172.21.15.19:0/3444544038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4786d00 0x55dee489b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.049+0000 7f252e59c700 1 -- 172.21.15.19:0/3444544038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee48aa270 con 0x55dee470c4a0 2024-05-09T02:15:45.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.050+0000 7f252e59c700 1 --2- 172.21.15.19:0/3444544038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 0x55dee489b270 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f252802c6c0 tx=0x7f2528005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b719c1a41b8bae69 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:45.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.050+0000 7f2515ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:15:45.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.050+0000 7f2515ffb700 1 -- 172.21.15.19:0/3444544038 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25280d9020 con 0x55dee470c4a0 2024-05-09T02:15:45.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.050+0000 7f2515ffb700 1 -- 172.21.15.19:0/3444544038 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f25280c3df0 con 0x55dee470c4a0 2024-05-09T02:15:45.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.050+0000 7f254348db80 1 -- 172.21.15.19:0/3444544038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25280bb9a0 con 0x55dee470c4a0 2024-05-09T02:15:45.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.050+0000 7f2515ffb700 1 -- 172.21.15.19:0/3444544038 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2528030070 con 0x55dee470c4a0 2024-05-09T02:15:45.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.050+0000 7f254348db80 10 client.25580 fetch_fsmap learned FSMap version 41 2024-05-09T02:15:45.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.051+0000 7f254348db80 1 -- 172.21.15.19:0/3444544038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55dee48aa270 con 0x55dee470c4a0 2024-05-09T02:15:45.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.051+0000 7f2515ffb700 1 -- 172.21.15.19:0/3444544038 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25280bb9a0 con 0x55dee470c4a0 2024-05-09T02:15:45.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.051+0000 7f2515ffb700 1 -- 172.21.15.19:0/3444544038 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f25280d5b00 con 0x55dee470c4a0 2024-05-09T02:15:45.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.051+0000 7f254348db80 10 client.25580 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:15:45.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.051+0000 7f254348db80 10 client.25580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:15:45.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.051+0000 7f254348db80 20 client.25580 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:45.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.051+0000 7f254348db80 1 --2- 172.21.15.19:0/3444544038 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dee490d180 0x55dee490f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:45.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.051+0000 7f254348db80 4 client.25580 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:15:45.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.051+0000 7f254348db80 1 -- 172.21.15.19:0/3444544038 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dee47b3b20 con 0x55dee490d180 2024-05-09T02:15:45.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.053+0000 7f252dd9b700 1 --2- 172.21.15.19:0/3444544038 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dee490d180 0x55dee490f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:45.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.054+0000 7f2515ffb700 10 client.25580 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:45.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.054+0000 7f252dd9b700 1 --2- 172.21.15.19:0/3444544038 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dee490d180 0x55dee490f5a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5c243a552c7b3e9 server_cookie=fc01663d8fbf6b7 in_seq=0 out_seq=0 2024-05-09T02:15:45.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.055+0000 7f2515ffb700 1 -- 172.21.15.19:0/3444544038 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dee47b3b20 con 0x55dee490d180 2024-05-09T02:15:45.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.055+0000 7f2515ffb700 10 client.25580 handle_command_reply: tid=0 2024-05-09T02:15:45.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.055+0000 7f254348db80 1 client.25580 shutdown 2024-05-09T02:15:45.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.055+0000 7f254348db80 1 -- 172.21.15.19:0/3444544038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 msgr2=0x55dee489b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:45.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.055+0000 7f254348db80 1 --2- 172.21.15.19:0/3444544038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 0x55dee489b270 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f252802c6c0 tx=0x7f2528005e70 comp rx=0 tx=0).stop 2024-05-09T02:15:45.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.056+0000 7f254348db80 1 -- 172.21.15.19:0/3444544038 shutdown_connections 2024-05-09T02:15:45.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.056+0000 7f254348db80 1 --2- 172.21.15.19:0/3444544038 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dee490d180 0x55dee490f5a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.056+0000 7f254348db80 1 --2- 172.21.15.19:0/3444544038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee4889400 0x55dee48a9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.056+0000 7f254348db80 1 --2- 172.21.15.19:0/3444544038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dee470c4a0 0x55dee489b270 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.056+0000 7f254348db80 1 --2- 172.21.15.19:0/3444544038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee4786d00 0x55dee489b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.056+0000 7f2515ffb700 0 client.25580 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:45.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.056+0000 7f254348db80 1 -- 172.21.15.19:0/3444544038 >> 172.21.15.19:0/3444544038 conn(0x55dee470ff70 msgr2=0x55dee488dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.056+0000 7f254348db80 1 -- 172.21.15.19:0/3444544038 shutdown_connections 2024-05-09T02:15:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.056+0000 7f254348db80 1 -- 172.21.15.19:0/3444544038 wait complete. 2024-05-09T02:15:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.056+0000 7f254348db80 20 client.25580 trim_cache size 0 max 16384 2024-05-09T02:15:45.060 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:15:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.057+0000 7f2534a77700 1 -- 172.21.15.19:0/1329739430 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f25180b2ec0 msgr2=0x7f25180b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.057+0000 7f2534a77700 1 --2- 172.21.15.19:0/1329739430 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f25180b2ec0 0x7f25180b53d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2520009780 tx=0x7f2520019410 comp rx=0 tx=0).stop 2024-05-09T02:15:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.057+0000 7f2534a77700 1 -- 172.21.15.19:0/1329739430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530103250 msgr2=0x7f2530110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.057+0000 7f2534a77700 1 --2- 172.21.15.19:0/1329739430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530103250 0x7f2530110050 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f252801c860 tx=0x7f2528019a70 comp rx=0 tx=0).stop 2024-05-09T02:15:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.058+0000 7f2534a77700 1 -- 172.21.15.19:0/1329739430 shutdown_connections 2024-05-09T02:15:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.058+0000 7f2534a77700 1 --2- 172.21.15.19:0/1329739430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25301086a0 0x7f2530074510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.058+0000 7f2534a77700 1 --2- 172.21.15.19:0/1329739430 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f25180b2ec0 0x7f25180b53d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.058+0000 7f2534a77700 1 --2- 172.21.15.19:0/1329739430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530103250 0x7f2530110050 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.058+0000 7f2534a77700 1 --2- 172.21.15.19:0/1329739430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2530103ba0 0x7f25301105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.058+0000 7f2534a77700 1 -- 172.21.15.19:0/1329739430 >> 172.21.15.19:0/1329739430 conn(0x7f25300fedc0 msgr2=0x7f25301060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.058+0000 7f2534a77700 1 -- 172.21.15.19:0/1329739430 shutdown_connections 2024-05-09T02:15:45.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:45.058+0000 7f2534a77700 1 -- 172.21.15.19:0/1329739430 wait complete. 2024-05-09T02:15:46.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:46 smithi196 ceph-mon[34718]: pgmap v137: 97 pgs: 97 active+clean; 3.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 43 MiB/s wr, 1.82k op/s 2024-05-09T02:15:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:46 smithi019 ceph-mon[30151]: pgmap v137: 97 pgs: 97 active+clean; 3.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 43 MiB/s wr, 1.82k op/s 2024-05-09T02:15:46.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:46 smithi059 ceph-mon[34781]: pgmap v137: 97 pgs: 97 active+clean; 3.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 43 MiB/s wr, 1.82k op/s 2024-05-09T02:15:47.257 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T02:15:47.258 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T02:15:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 245.0 s, 3163.4 tps, lat 31.765 ms stddev 37.388 2024-05-09T02:15:47.656 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:15:48.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:48 smithi019 ceph-mon[30151]: pgmap v138: 97 pgs: 97 active+clean; 3.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 31 MiB/s wr, 1.50k op/s 2024-05-09T02:15:48.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:48 smithi196 ceph-mon[34718]: pgmap v138: 97 pgs: 97 active+clean; 3.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 31 MiB/s wr, 1.50k op/s 2024-05-09T02:15:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:48 smithi059 ceph-mon[34781]: pgmap v138: 97 pgs: 97 active+clean; 3.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 31 MiB/s wr, 1.50k op/s 2024-05-09T02:15:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.972+0000 7f6a88090700 1 -- 172.21.15.19:0/4221698741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a801034c0 msgr2=0x7f6a801038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.972+0000 7f6a88090700 1 --2- 172.21.15.19:0/4221698741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a801034c0 0x7f6a801038d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0099c0 tx=0x7f6a7c009cd0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.972+0000 7f6a88090700 1 -- 172.21.15.19:0/4221698741 shutdown_connections 2024-05-09T02:15:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.972+0000 7f6a88090700 1 --2- 172.21.15.19:0/4221698741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a80108740 0x7f6a801100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.972+0000 7f6a88090700 1 --2- 172.21.15.19:0/4221698741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a801034c0 0x7f6a801038d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.972+0000 7f6a88090700 1 --2- 172.21.15.19:0/4221698741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a80107df0 0x7f6a80108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.972+0000 7f6a88090700 1 -- 172.21.15.19:0/4221698741 >> 172.21.15.19:0/4221698741 conn(0x7f6a800fedf0 msgr2=0x7f6a80101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:49.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.972+0000 7f6a88090700 1 -- 172.21.15.19:0/4221698741 shutdown_connections 2024-05-09T02:15:49.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.972+0000 7f6a88090700 1 -- 172.21.15.19:0/4221698741 wait complete. 2024-05-09T02:15:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.973+0000 7f6a88090700 1 Processor -- start 2024-05-09T02:15:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.973+0000 7f6a88090700 1 -- start start 2024-05-09T02:15:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.973+0000 7f6a88090700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a801034c0 0x7f6a80118e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.973+0000 7f6a88090700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a80107df0 0x7f6a80112fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.973+0000 7f6a88090700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a80108740 0x7f6a80113620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.973+0000 7f6a88090700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a80111540 con 0x7f6a80107df0 2024-05-09T02:15:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.973+0000 7f6a88090700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a801113c0 con 0x7f6a801034c0 2024-05-09T02:15:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.973+0000 7f6a88090700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a801116c0 con 0x7f6a80108740 2024-05-09T02:15:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.973+0000 7f6a8562b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a80107df0 0x7f6a80112fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.973+0000 7f6a8662d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a80108740 0x7f6a80113620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.973+0000 7f6a85e2c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a801034c0 0x7f6a80118e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.973+0000 7f6a85e2c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a801034c0 0x7f6a80118e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:47782/0 (socket says 172.21.15.19:47782) 2024-05-09T02:15:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.973+0000 7f6a8562b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a80107df0 0x7f6a80112fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33268/0 (socket says 172.21.15.19:33268) 2024-05-09T02:15:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.973+0000 7f6a8662d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a80108740 0x7f6a80113620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:57272/0 (socket says 172.21.15.19:57272) 2024-05-09T02:15:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.973+0000 7f6a85e2c700 1 -- 172.21.15.19:0/3368464007 learned_addr learned my addr 172.21.15.19:0/3368464007 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.974+0000 7f6a85e2c700 1 -- 172.21.15.19:0/3368464007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a80108740 msgr2=0x7f6a80113620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.974+0000 7f6a85e2c700 1 --2- 172.21.15.19:0/3368464007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a80108740 0x7f6a80113620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.974+0000 7f6a85e2c700 1 -- 172.21.15.19:0/3368464007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a80107df0 msgr2=0x7f6a80112fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.974+0000 7f6a85e2c700 1 --2- 172.21.15.19:0/3368464007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a80107df0 0x7f6a80112fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.974+0000 7f6a85e2c700 1 -- 172.21.15.19:0/3368464007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a801b5f50 con 0x7f6a801034c0 2024-05-09T02:15:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.974+0000 7f6a85e2c700 1 --2- 172.21.15.19:0/3368464007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a801034c0 0x7f6a80118e80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c005910 tx=0x7f6a7c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.974+0000 7f6a76ffd700 1 -- 172.21.15.19:0/3368464007 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c009e60 con 0x7f6a801034c0 2024-05-09T02:15:49.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.974+0000 7f6a76ffd700 1 -- 172.21.15.19:0/3368464007 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a7c0045f0 con 0x7f6a801034c0 2024-05-09T02:15:49.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.974+0000 7f6a76ffd700 1 -- 172.21.15.19:0/3368464007 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00fb70 con 0x7f6a801034c0 2024-05-09T02:15:49.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.974+0000 7f6a88090700 1 -- 172.21.15.19:0/3368464007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a801b6120 con 0x7f6a801034c0 2024-05-09T02:15:49.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.974+0000 7f6a88090700 1 -- 172.21.15.19:0/3368464007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a801b64c0 con 0x7f6a801034c0 2024-05-09T02:15:49.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.980+0000 7f6a96aa6b80 1 Processor -- start 2024-05-09T02:15:49.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.980+0000 7f6a96aa6b80 1 -- start start 2024-05-09T02:15:49.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.980+0000 7f6a96aa6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498dd2d80 0x564498edb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:49.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.980+0000 7f6a96aa6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498edb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:49.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.980+0000 7f6a96aa6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498ed5480 0x564498ed5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:49.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.980+0000 7f6a96aa6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564498df8120 con 0x564498d58520 2024-05-09T02:15:49.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.980+0000 7f6a96aa6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564498d9da20 con 0x564498dd2d80 2024-05-09T02:15:49.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.981+0000 7f6a96aa6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564498df40e0 con 0x564498ed5480 2024-05-09T02:15:49.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.981+0000 7f6a8662d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498dd2d80 0x564498edb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:49.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.981+0000 7f6a8662d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498dd2d80 0x564498edb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:47784/0 (socket says 172.21.15.19:47784) 2024-05-09T02:15:49.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.981+0000 7f6a8662d700 1 -- 172.21.15.19:0/541905332 learned_addr learned my addr 172.21.15.19:0/541905332 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.981+0000 7f6a8562b700 1 --2- 172.21.15.19:0/541905332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498ed5480 0x564498ed5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.981+0000 7f6a76ffd700 1 -- 172.21.15.19:0/3368464007 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6a7c02b430 con 0x7f6a801034c0 2024-05-09T02:15:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.981+0000 7f6a8662d700 1 -- 172.21.15.19:0/541905332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498ed5480 msgr2=0x564498ed5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.981+0000 7f6a8662d700 1 --2- 172.21.15.19:0/541905332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498ed5480 0x564498ed5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.981+0000 7f6a8662d700 1 -- 172.21.15.19:0/541905332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 msgr2=0x564498edb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.981+0000 7f6a8662d700 1 --2- 172.21.15.19:0/541905332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498edb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.981+0000 7f6a8662d700 1 -- 172.21.15.19:0/541905332 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564498d8e580 con 0x564498dd2d80 2024-05-09T02:15:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.981+0000 7f6a8562b700 1 --2- 172.21.15.19:0/541905332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498ed5480 0x564498ed5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:15:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.981+0000 7f6a8662d700 1 --2- 172.21.15.19:0/541905332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498dd2d80 0x564498edb2c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a78011680 tx=0x7f6a78009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55d5183cdb019746 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.982+0000 7f6a6bfff700 1 -- 172.21.15.19:0/541905332 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78019020 con 0x564498dd2d80 2024-05-09T02:15:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.982+0000 7f6a6bfff700 1 -- 172.21.15.19:0/541905332 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a780079b0 con 0x564498dd2d80 2024-05-09T02:15:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.982+0000 7f6a6bfff700 1 -- 172.21.15.19:0/541905332 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78015aa0 con 0x564498dd2d80 2024-05-09T02:15:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.982+0000 7f6a76ffd700 1 --2- 172.21.15.19:0/3368464007 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6a6c0b2a20 0x7f6a6c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:49.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.982+0000 7f6a85e2c700 1 --2- 172.21.15.19:0/3368464007 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6a6c0b2a20 0x7f6a6c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:49.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.982+0000 7f6a76ffd700 1 -- 172.21.15.19:0/3368464007 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f6a7c0e3600 con 0x7f6a801034c0 2024-05-09T02:15:49.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.983+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/541905332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498dd2d80 msgr2=0x564498edb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:49.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.983+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/541905332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498dd2d80 0x564498edb2c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a78011680 tx=0x7f6a78009920 comp rx=0 tx=0).stop 2024-05-09T02:15:49.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.983+0000 7f6a85e2c700 1 --2- 172.21.15.19:0/3368464007 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6a6c0b2a20 0x7f6a6c0b4f30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x564498ed1d10 tx=0x7f6a7c092040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:49.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.983+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/541905332 shutdown_connections 2024-05-09T02:15:49.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.983+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/541905332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498ed5480 0x564498ed5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.983+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/541905332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498dd2d80 0x564498edb2c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.983+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/541905332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498edb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.983+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/541905332 >> 172.21.15.19:0/541905332 conn(0x564498d5bff0 msgr2=0x564498ed02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:49.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.983+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/541905332 shutdown_connections 2024-05-09T02:15:49.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.983+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/541905332 wait complete. 2024-05-09T02:15:49.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.983+0000 7f6a96aa6b80 1 -- start start 2024-05-09T02:15:49.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.984+0000 7f6a96aa6b80 1 Processor -- start 2024-05-09T02:15:49.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.984+0000 7f6a96aa6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498edf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:49.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.984+0000 7f6a96aa6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498dd2d80 0x564498edf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:49.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.984+0000 7f6a96aa6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498ed5480 0x564498edfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:49.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.984+0000 7f6a96aa6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564498ed61e0 con 0x564498d58520 2024-05-09T02:15:49.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.984+0000 7f6a96aa6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564498d92ba0 con 0x564498ed5480 2024-05-09T02:15:49.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.984+0000 7f6a96aa6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564498ee20a0 con 0x564498dd2d80 2024-05-09T02:15:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.984+0000 7f6a96aa6b80 10 client.0 _ll_register_callbacks cb 0x564498c69740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.985+0000 7f6a8662d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498ed5480 0x564498edfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.985+0000 7f6a8662d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498edf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.985+0000 7f6a8662d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498edf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33288/0 (socket says 172.21.15.19:33288) 2024-05-09T02:15:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.985+0000 7f6a8662d700 1 -- 172.21.15.19:0/969448248 learned_addr learned my addr 172.21.15.19:0/969448248 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.986+0000 7f6a8562b700 1 --2- 172.21.15.19:0/969448248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498dd2d80 0x564498edf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.986+0000 7f6a8662d700 1 -- 172.21.15.19:0/969448248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498dd2d80 msgr2=0x564498edf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.986+0000 7f6a8662d700 1 --2- 172.21.15.19:0/969448248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498dd2d80 0x564498edf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.986+0000 7f6a8662d700 1 -- 172.21.15.19:0/969448248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 msgr2=0x564498edf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.986+0000 7f6a8662d700 1 --2- 172.21.15.19:0/969448248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498edf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.986+0000 7f6a8662d700 1 -- 172.21.15.19:0/969448248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564498c73510 con 0x564498ed5480 2024-05-09T02:15:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.986+0000 7f6a8662d700 1 --2- 172.21.15.19:0/969448248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498ed5480 0x564498edfb90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009020 tx=0x7f6a78015640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24087a32c0fe214 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.986+0000 7f6a697fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:15:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.987+0000 7f6a697fa700 1 -- 172.21.15.19:0/969448248 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78019040 con 0x564498ed5480 2024-05-09T02:15:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.987+0000 7f6a697fa700 1 -- 172.21.15.19:0/969448248 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f6a78015bd0 con 0x564498ed5480 2024-05-09T02:15:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.988+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/969448248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a780079b0 con 0x564498ed5480 2024-05-09T02:15:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.988+0000 7f6a697fa700 1 -- 172.21.15.19:0/969448248 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6a78039040 con 0x564498ed5480 2024-05-09T02:15:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.988+0000 7f6a96aa6b80 10 client.25592 fetch_fsmap learned FSMap version 41 2024-05-09T02:15:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.988+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/969448248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564498eecae0 con 0x564498ed5480 2024-05-09T02:15:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.988+0000 7f6a697fa700 1 -- 172.21.15.19:0/969448248 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a7801e8d0 con 0x564498ed5480 2024-05-09T02:15:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.989+0000 7f6a697fa700 1 -- 172.21.15.19:0/969448248 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f6a7801d3d0 con 0x564498ed5480 2024-05-09T02:15:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.989+0000 7f6a96aa6b80 10 client.25592 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:15:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.989+0000 7f6a96aa6b80 10 client.25592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:15:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.989+0000 7f6a96aa6b80 20 client.25592 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.989+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/969448248 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564498eef010 0x564498ef1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.989+0000 7f6a96aa6b80 4 client.25592 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:15:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.989+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/969448248 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6a780079b0 con 0x564498eef010 2024-05-09T02:15:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.989+0000 7f6a8562b700 1 --2- 172.21.15.19:0/969448248 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564498eef010 0x564498ef1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.989+0000 7f6a8562b700 1 --2- 172.21.15.19:0/969448248 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564498eef010 0x564498ef1430 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a2116bdfc352ff8 server_cookie=a69cabdbd53db80a in_seq=0 out_seq=0 2024-05-09T02:15:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.990+0000 7f6a697fa700 10 client.25592 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.990+0000 7f6a697fa700 1 -- 172.21.15.19:0/969448248 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f6a780079b0 con 0x564498eef010 2024-05-09T02:15:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.990+0000 7f6a697fa700 10 client.25592 handle_command_reply: tid=0 2024-05-09T02:15:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.990+0000 7f6a96aa6b80 1 client.25592 shutdown 2024-05-09T02:15:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.990+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/969448248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498ed5480 msgr2=0x564498edfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.990+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/969448248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498ed5480 0x564498edfb90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009020 tx=0x7f6a78015640 comp rx=0 tx=0).stop 2024-05-09T02:15:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.991+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/969448248 shutdown_connections 2024-05-09T02:15:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.991+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/969448248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498ed5480 0x564498edfb90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.991+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/969448248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498edf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.991+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/969448248 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564498eef010 0x564498ef1430 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.991+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/969448248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498dd2d80 0x564498edf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.991+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/969448248 >> 172.21.15.19:0/969448248 conn(0x564498d5bff0 msgr2=0x564498eca070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.991+0000 7f6a697fa700 0 client.25592 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.991+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/969448248 shutdown_connections 2024-05-09T02:15:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.991+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/969448248 wait complete. 2024-05-09T02:15:49.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:49.991+0000 7f6a96aa6b80 20 client.25592 trim_cache size 0 max 16384 2024-05-09T02:15:50.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.020+0000 7f6a96aa6b80 1 Processor -- start 2024-05-09T02:15:50.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.020+0000 7f6a96aa6b80 1 -- start start 2024-05-09T02:15:50.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.021+0000 7f6a96aa6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498ed4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:50.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.021+0000 7f6a96aa6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498dd2d80 0x564498ee1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:50.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.021+0000 7f6a96aa6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498ed5480 0x564498eeb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:50.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.021+0000 7f6a96aa6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a80074780 con 0x564498d58520 2024-05-09T02:15:50.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.021+0000 7f6a96aa6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a801b7150 con 0x564498dd2d80 2024-05-09T02:15:50.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.021+0000 7f6a96aa6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a80074e50 con 0x564498ed5480 2024-05-09T02:15:50.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.021+0000 7f6a8562b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498dd2d80 0x564498ee1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:50.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.021+0000 7f6a8662d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498ed4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:50.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.021+0000 7f6a8562b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498dd2d80 0x564498ee1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:47808/0 (socket says 172.21.15.19:47808) 2024-05-09T02:15:50.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.021+0000 7f6a8562b700 1 -- 172.21.15.19:0/2679669919 learned_addr learned my addr 172.21.15.19:0/2679669919 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:50.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.021+0000 7f6a8662d700 1 --2- 172.21.15.19:0/2679669919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498ed5480 0x564498eeb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:50.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.021+0000 7f6a8562b700 1 -- 172.21.15.19:0/2679669919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498ed5480 msgr2=0x564498eeb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:50.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.022+0000 7f6a8562b700 1 --2- 172.21.15.19:0/2679669919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498ed5480 0x564498eeb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:50.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.022+0000 7f6a8562b700 1 -- 172.21.15.19:0/2679669919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 msgr2=0x564498ed4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:50.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.022+0000 7f6a8562b700 1 --2- 172.21.15.19:0/2679669919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498ed4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:50.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.022+0000 7f6a8562b700 1 -- 172.21.15.19:0/2679669919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564498ed6660 con 0x564498dd2d80 2024-05-09T02:15:50.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.022+0000 7f6a8562b700 1 --2- 172.21.15.19:0/2679669919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498dd2d80 0x564498ee1510 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009da0 tx=0x7f6a70009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad90d400485c0739 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:50.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.022+0000 7f6a6b7fe700 1 -- 172.21.15.19:0/2679669919 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70022070 con 0x564498dd2d80 2024-05-09T02:15:50.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.023+0000 7f6a6b7fe700 1 -- 172.21.15.19:0/2679669919 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a70003820 con 0x564498dd2d80 2024-05-09T02:15:50.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.023+0000 7f6a6b7fe700 1 -- 172.21.15.19:0/2679669919 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70005370 con 0x564498dd2d80 2024-05-09T02:15:50.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.024+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/2679669919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498dd2d80 msgr2=0x564498ee1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:50.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.024+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/2679669919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498dd2d80 0x564498ee1510 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009da0 tx=0x7f6a70009260 comp rx=0 tx=0).stop 2024-05-09T02:15:50.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.024+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/2679669919 shutdown_connections 2024-05-09T02:15:50.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.024+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/2679669919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498ed5480 0x564498eeb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:50.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.024+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/2679669919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498ed4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:50.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.024+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/2679669919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498dd2d80 0x564498ee1510 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009da0 tx=0x7f6a70009260 comp rx=0 tx=0).stop 2024-05-09T02:15:50.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.024+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/2679669919 >> 172.21.15.19:0/2679669919 conn(0x564498d5bff0 msgr2=0x564498eca6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:50.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.030+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/2679669919 shutdown_connections 2024-05-09T02:15:50.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.030+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/2679669919 wait complete. 2024-05-09T02:15:50.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.031+0000 7f6a96aa6b80 1 -- start start 2024-05-09T02:15:50.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.031+0000 7f6a96aa6b80 1 Processor -- start 2024-05-09T02:15:50.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.032+0000 7f6a96aa6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498f4d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:50.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.032+0000 7f6a96aa6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498ed5480 0x564498f4dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:50.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.032+0000 7f6a96aa6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498f56560 0x564498f58980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:50.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.032+0000 7f6a96aa6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a801b6a80 con 0x564498d58520 2024-05-09T02:15:50.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.032+0000 7f6a96aa6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a80075520 con 0x564498ed5480 2024-05-09T02:15:50.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.032+0000 7f6a96aa6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a80072720 con 0x564498f56560 2024-05-09T02:15:50.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.033+0000 7f6a96aa6b80 10 client.0 _ll_register_callbacks cb 0x564498c69740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:50.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.035+0000 7f6a8562b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498ed5480 0x564498f4dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:50.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.035+0000 7f6a8662d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498f56560 0x564498f58980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:50.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.035+0000 7f6a8662d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498f4d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:50.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.035+0000 7f6a8662d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498f4d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33306/0 (socket says 172.21.15.19:33306) 2024-05-09T02:15:50.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.035+0000 7f6a8662d700 1 -- 172.21.15.19:0/695397196 learned_addr learned my addr 172.21.15.19:0/695397196 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.046+0000 7f6a8562b700 1 -- 172.21.15.19:0/695397196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498f56560 msgr2=0x564498f58980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.046+0000 7f6a8562b700 1 --2- 172.21.15.19:0/695397196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498f56560 0x564498f58980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.046+0000 7f6a8562b700 1 -- 172.21.15.19:0/695397196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 msgr2=0x564498f4d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.046+0000 7f6a8562b700 1 --2- 172.21.15.19:0/695397196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498f4d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.046+0000 7f6a8562b700 1 -- 172.21.15.19:0/695397196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564498f58ec0 con 0x564498ed5480 2024-05-09T02:15:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.046+0000 7f6a8662d700 1 --2- 172.21.15.19:0/695397196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498f56560 0x564498f58980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:15:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.046+0000 7f6a8562b700 1 --2- 172.21.15.19:0/695397196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498ed5480 0x564498f4dc30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x564498ed12a0 tx=0x7f6a70003b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbca04ed28eec990 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.046+0000 7f6a697fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:15:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.047+0000 7f6a697fa700 1 -- 172.21.15.19:0/695397196 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70022040 con 0x564498ed5480 2024-05-09T02:15:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.047+0000 7f6a697fa700 1 -- 172.21.15.19:0/695397196 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f6a700049c0 con 0x564498ed5480 2024-05-09T02:15:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.047+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/695397196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a70003820 con 0x564498ed5480 2024-05-09T02:15:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.048+0000 7f6a697fa700 1 -- 172.21.15.19:0/695397196 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6a70004bc0 con 0x564498ed5480 2024-05-09T02:15:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.048+0000 7f6a96aa6b80 10 client.25598 fetch_fsmap learned FSMap version 41 2024-05-09T02:15:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.048+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/695397196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564498f5a1a0 con 0x564498ed5480 2024-05-09T02:15:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.049+0000 7f6a697fa700 1 -- 172.21.15.19:0/695397196 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a70003820 con 0x564498ed5480 2024-05-09T02:15:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.049+0000 7f6a697fa700 1 -- 172.21.15.19:0/695397196 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f6a700104a0 con 0x564498ed5480 2024-05-09T02:15:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.049+0000 7f6a96aa6b80 10 client.25598 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:15:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.049+0000 7f6a96aa6b80 10 client.25598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:15:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.049+0000 7f6a96aa6b80 20 client.25598 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.049+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/695397196 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564498f5a7c0 0x564498f5cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.049+0000 7f6a96aa6b80 4 client.25598 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T02:15:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.050+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/695397196 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564498e05b00 con 0x564498f5a7c0 2024-05-09T02:15:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.052+0000 7f6a8662d700 1 --2- 172.21.15.19:0/695397196 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564498f5a7c0 0x564498f5cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:50.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.054+0000 7f6a8662d700 1 --2- 172.21.15.19:0/695397196 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564498f5a7c0 0x564498f5cbe0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c92e873376a5ce3d server_cookie=68f0f053a0a1e686 in_seq=0 out_seq=0 2024-05-09T02:15:50.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.055+0000 7f6a697fa700 10 client.25598 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:50.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.056+0000 7f6a697fa700 1 -- 172.21.15.19:0/695397196 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564498e05b00 con 0x564498f5a7c0 2024-05-09T02:15:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.056+0000 7f6a697fa700 10 client.25598 handle_command_reply: tid=0 2024-05-09T02:15:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.056+0000 7f6a96aa6b80 1 client.25598 shutdown 2024-05-09T02:15:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.056+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/695397196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498ed5480 msgr2=0x564498f4dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.056+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/695397196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498ed5480 0x564498f4dc30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x564498ed12a0 tx=0x7f6a70003b50 comp rx=0 tx=0).stop 2024-05-09T02:15:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.056+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/695397196 shutdown_connections 2024-05-09T02:15:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.056+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/695397196 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564498f5a7c0 0x564498f5cbe0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.057+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/695397196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564498f56560 0x564498f58980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:50.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.057+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/695397196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564498d58520 0x564498f4d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:50.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.057+0000 7f6a697fa700 0 client.25598 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:50.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.057+0000 7f6a96aa6b80 1 --2- 172.21.15.19:0/695397196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564498ed5480 0x564498f4dc30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:50.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.057+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/695397196 >> 172.21.15.19:0/695397196 conn(0x564498d5bff0 msgr2=0x564498ed8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:50.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.057+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/695397196 shutdown_connections 2024-05-09T02:15:50.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.057+0000 7f6a96aa6b80 1 -- 172.21.15.19:0/695397196 wait complete. 2024-05-09T02:15:50.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.057+0000 7f6a96aa6b80 20 client.25598 trim_cache size 0 max 16384 2024-05-09T02:15:50.061 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:15:50.061 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T02:15:50.061 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "d30e3e2e-84ab-4c85-8450-4e0cee8c1fad", 2024-05-09T02:15:50.061 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T02:15:50.061 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:15:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.058+0000 7f6a88090700 1 -- 172.21.15.19:0/3368464007 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6a6c0b2a20 msgr2=0x7f6a6c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.058+0000 7f6a88090700 1 --2- 172.21.15.19:0/3368464007 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6a6c0b2a20 0x7f6a6c0b4f30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x564498ed1d10 tx=0x7f6a7c092040 comp rx=0 tx=0).stop 2024-05-09T02:15:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.058+0000 7f6a88090700 1 -- 172.21.15.19:0/3368464007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a801034c0 msgr2=0x7f6a80118e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.058+0000 7f6a88090700 1 --2- 172.21.15.19:0/3368464007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a801034c0 0x7f6a80118e80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c005910 tx=0x7f6a7c004270 comp rx=0 tx=0).stop 2024-05-09T02:15:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.058+0000 7f6a88090700 1 -- 172.21.15.19:0/3368464007 shutdown_connections 2024-05-09T02:15:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.058+0000 7f6a88090700 1 --2- 172.21.15.19:0/3368464007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a80108740 0x7f6a80113620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.058+0000 7f6a88090700 1 --2- 172.21.15.19:0/3368464007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a801034c0 0x7f6a80118e80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.058+0000 7f6a88090700 1 --2- 172.21.15.19:0/3368464007 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6a6c0b2a20 0x7f6a6c0b4f30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.058+0000 7f6a88090700 1 --2- 172.21.15.19:0/3368464007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a80107df0 0x7f6a80112fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.058+0000 7f6a88090700 1 -- 172.21.15.19:0/3368464007 >> 172.21.15.19:0/3368464007 conn(0x7f6a800fedf0 msgr2=0x7f6a80101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.059+0000 7f6a88090700 1 -- 172.21.15.19:0/3368464007 shutdown_connections 2024-05-09T02:15:50.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:50.059+0000 7f6a88090700 1 -- 172.21.15.19:0/3368464007 wait complete. 2024-05-09T02:15:50.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:50 smithi196 ceph-mon[34718]: pgmap v139: 97 pgs: 97 active+clean; 3.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 37 MiB/s wr, 1.90k op/s 2024-05-09T02:15:50.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:50 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:15:50.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:50 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:50.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:50 smithi019 ceph-mon[30151]: pgmap v139: 97 pgs: 97 active+clean; 3.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 37 MiB/s wr, 1.90k op/s 2024-05-09T02:15:50.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:50 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:15:50.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:50 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:50.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:50 smithi059 ceph-mon[34781]: pgmap v139: 97 pgs: 97 active+clean; 3.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 37 MiB/s wr, 1.90k op/s 2024-05-09T02:15:50.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:50 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:15:50.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:50 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.369 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:15:51.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub queued for path: / 2024-05-09T02:15:51.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:15:51.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:15:51.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:15:51.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:15:51.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:15:51.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:15:51.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:15:51.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:15:51.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:51 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub queued for path: / 2024-05-09T02:15:51.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:15:51.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:15:51.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:15:51.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:15:51.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:15:51.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:15:51.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:15:51.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:15:51.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:51 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.784 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:15:51.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub queued for path: / 2024-05-09T02:15:51.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:15:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:15:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:15:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:15:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:15:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:15:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:15:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:15:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:51 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:52.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:52 smithi019 ceph-mon[30151]: pgmap v140: 97 pgs: 97 active+clean; 3.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 9.2 MiB/s wr, 559 op/s 2024-05-09T02:15:52.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:52 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:15:52.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:52 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 250.0 s, 3172.6 tps, lat 31.672 ms stddev 38.534 2024-05-09T02:15:52.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:52 smithi059 ceph-mon[34781]: pgmap v140: 97 pgs: 97 active+clean; 3.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 9.2 MiB/s wr, 559 op/s 2024-05-09T02:15:52.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:52 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:15:52.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:52 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:52.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:52 smithi196 ceph-mon[34718]: pgmap v140: 97 pgs: 97 active+clean; 3.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 9.2 MiB/s wr, 559 op/s 2024-05-09T02:15:52.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:52 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:15:52.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:52 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:53.566 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:53 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:53.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:53 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:53.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.732+0000 7f56672b1700 1 -- 172.21.15.19:0/704119716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5660106ed0 msgr2=0x7f5660113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:53.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.732+0000 7f56672b1700 1 --2- 172.21.15.19:0/704119716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5660106ed0 0x7f5660113360 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f565800b0a0 tx=0x7f565800b3b0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.732+0000 7f56672b1700 1 -- 172.21.15.19:0/704119716 shutdown_connections 2024-05-09T02:15:53.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.732+0000 7f56672b1700 1 --2- 172.21.15.19:0/704119716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5660106ed0 0x7f5660113360 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.732+0000 7f56672b1700 1 --2- 172.21.15.19:0/704119716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f566010c5c0 0x7f566010c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.732+0000 7f56672b1700 1 --2- 172.21.15.19:0/704119716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5660106580 0x7f5660106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.732+0000 7f56672b1700 1 -- 172.21.15.19:0/704119716 >> 172.21.15.19:0/704119716 conn(0x7f5660102080 msgr2=0x7f56601044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.732+0000 7f56672b1700 1 -- 172.21.15.19:0/704119716 shutdown_connections 2024-05-09T02:15:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.732+0000 7f56672b1700 1 -- 172.21.15.19:0/704119716 wait complete. 2024-05-09T02:15:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.733+0000 7f56672b1700 1 Processor -- start 2024-05-09T02:15:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.733+0000 7f56672b1700 1 -- start start 2024-05-09T02:15:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f56672b1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5660106580 0x7f566019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f56672b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5660106ed0 0x7f566019d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f56672b1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f566010c5c0 0x7f56601a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f56672b1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f56601166f0 con 0x7f5660106ed0 2024-05-09T02:15:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f566504d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5660106580 0x7f566019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f56672b1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5660116570 con 0x7f5660106580 2024-05-09T02:15:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f566504d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5660106580 0x7f566019cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:47840/0 (socket says 172.21.15.19:47840) 2024-05-09T02:15:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f566504d700 1 -- 172.21.15.19:0/3681181548 learned_addr learned my addr 172.21.15.19:0/3681181548 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f56672b1700 1 -- 172.21.15.19:0/3681181548 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5660116870 con 0x7f566010c5c0 2024-05-09T02:15:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f566484c700 1 --2- 172.21.15.19:0/3681181548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5660106ed0 0x7f566019d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f566584e700 1 --2- 172.21.15.19:0/3681181548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f566010c5c0 0x7f56601a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f566504d700 1 -- 172.21.15.19:0/3681181548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f566010c5c0 msgr2=0x7f56601a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f566504d700 1 --2- 172.21.15.19:0/3681181548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f566010c5c0 0x7f56601a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f566504d700 1 -- 172.21.15.19:0/3681181548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5660106ed0 msgr2=0x7f566019d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f566504d700 1 --2- 172.21.15.19:0/3681181548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5660106ed0 0x7f566019d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.734+0000 7f566504d700 1 -- 172.21.15.19:0/3681181548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56601a1e90 con 0x7f5660106580 2024-05-09T02:15:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.735+0000 7f566504d700 1 --2- 172.21.15.19:0/3681181548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5660106580 0x7f566019cfc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f565000d980 tx=0x7f565000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.736+0000 7f56567fc700 1 -- 172.21.15.19:0/3681181548 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650017070 con 0x7f5660106580 2024-05-09T02:15:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.736+0000 7f56567fc700 1 -- 172.21.15.19:0/3681181548 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5650009de0 con 0x7f5660106580 2024-05-09T02:15:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.736+0000 7f56567fc700 1 -- 172.21.15.19:0/3681181548 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565000f3c0 con 0x7f5660106580 2024-05-09T02:15:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.736+0000 7f56672b1700 1 -- 172.21.15.19:0/3681181548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56601a2180 con 0x7f5660106580 2024-05-09T02:15:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.736+0000 7f56672b1700 1 -- 172.21.15.19:0/3681181548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56601a9aa0 con 0x7f5660106580 2024-05-09T02:15:53.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.740+0000 7f56567fc700 1 -- 172.21.15.19:0/3681181548 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f565000f560 con 0x7f5660106580 2024-05-09T02:15:53.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.741+0000 7f56567fc700 1 --2- 172.21.15.19:0/3681181548 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f564c0b2b40 0x7f564c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.741+0000 7f56567fc700 1 -- 172.21.15.19:0/3681181548 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f56500e1fd0 con 0x7f5660106580 2024-05-09T02:15:53.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.741+0000 7f566484c700 1 --2- 172.21.15.19:0/3681181548 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f564c0b2b40 0x7f564c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.742+0000 7f5675cc7b80 1 Processor -- start 2024-05-09T02:15:53.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.742+0000 7f5675cc7b80 1 -- start start 2024-05-09T02:15:53.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.742+0000 7f566484c700 1 --2- 172.21.15.19:0/3681181548 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f564c0b2b40 0x7f564c0b5050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5660105040 tx=0x7f565c009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:53.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.742+0000 7f5675cc7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cc03fd00 0x5586cc148240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.742+0000 7f5675cc7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc54a0 0x5586cc148780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.742+0000 7f566504d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cc03fd00 0x5586cc148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.742+0000 7f5675cc7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc142400 0x5586cc142810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.742+0000 7f566504d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cc03fd00 0x5586cc148240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33334/0 (socket says 172.21.15.19:33334) 2024-05-09T02:15:53.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.743+0000 7f566484c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc54a0 0x5586cc148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.743+0000 7f566484c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc54a0 0x5586cc148780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:47846/0 (socket says 172.21.15.19:47846) 2024-05-09T02:15:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.743+0000 7f566504d700 1 -- 172.21.15.19:0/3252373688 learned_addr learned my addr 172.21.15.19:0/3252373688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.743+0000 7f5675cc7b80 1 -- 172.21.15.19:0/3252373688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5586cc0650a0 con 0x5586cc03fd00 2024-05-09T02:15:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.743+0000 7f5675cc7b80 1 -- 172.21.15.19:0/3252373688 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5586cc00a9a0 con 0x5586cbfc54a0 2024-05-09T02:15:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.743+0000 7f5675cc7b80 1 -- 172.21.15.19:0/3252373688 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5586cc061060 con 0x5586cc142400 2024-05-09T02:15:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.743+0000 7f566584e700 1 --2- 172.21.15.19:0/3252373688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc142400 0x5586cc142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.743+0000 7f566504d700 1 -- 172.21.15.19:0/3252373688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc142400 msgr2=0x5586cc142810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:53.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.743+0000 7f566504d700 1 --2- 172.21.15.19:0/3252373688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc142400 0x5586cc142810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.743+0000 7f566504d700 1 -- 172.21.15.19:0/3252373688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc54a0 msgr2=0x5586cc148780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:53.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.743+0000 7f566504d700 1 --2- 172.21.15.19:0/3252373688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc54a0 0x5586cc148780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.743+0000 7f566504d700 1 -- 172.21.15.19:0/3252373688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586cbffb500 con 0x5586cc03fd00 2024-05-09T02:15:53.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.743+0000 7f566504d700 1 --2- 172.21.15.19:0/3252373688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cc03fd00 0x5586cc148240 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f56500a9020 tx=0x7f56500ad6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe17ee15b1fea4a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:53.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.743+0000 7f564b7fe700 1 -- 172.21.15.19:0/3252373688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650020040 con 0x5586cc03fd00 2024-05-09T02:15:53.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.744+0000 7f564b7fe700 1 -- 172.21.15.19:0/3252373688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56500a8580 con 0x5586cc03fd00 2024-05-09T02:15:53.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.744+0000 7f564b7fe700 1 -- 172.21.15.19:0/3252373688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500a4ef0 con 0x5586cc03fd00 2024-05-09T02:15:53.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.745+0000 7f5675cc7b80 1 -- 172.21.15.19:0/3252373688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cc03fd00 msgr2=0x5586cc148240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.745+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/3252373688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cc03fd00 0x5586cc148240 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f56500a9020 tx=0x7f56500ad6e0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.745+0000 7f5675cc7b80 1 -- 172.21.15.19:0/3252373688 shutdown_connections 2024-05-09T02:15:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.745+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/3252373688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc142400 0x5586cc142810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.745+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/3252373688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cc03fd00 0x5586cc148240 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.745+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/3252373688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc54a0 0x5586cc148780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.745+0000 7f5675cc7b80 1 -- 172.21.15.19:0/3252373688 >> 172.21.15.19:0/3252373688 conn(0x5586cbfc8f70 msgr2=0x5586cc13d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.745+0000 7f5675cc7b80 1 -- 172.21.15.19:0/3252373688 shutdown_connections 2024-05-09T02:15:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.745+0000 7f5675cc7b80 1 -- 172.21.15.19:0/3252373688 wait complete. 2024-05-09T02:15:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.746+0000 7f5675cc7b80 1 -- start start 2024-05-09T02:15:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.746+0000 7f5675cc7b80 1 Processor -- start 2024-05-09T02:15:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.746+0000 7f5675cc7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cbfc54a0 0x5586cc14bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.746+0000 7f5675cc7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cc03fd00 0x5586cc14c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f5675cc7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc142400 0x5586cc14ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f566484c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cc03fd00 0x5586cc14c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f566484c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cc03fd00 0x5586cc14c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:47848/0 (socket says 172.21.15.19:47848) 2024-05-09T02:15:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f566584e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc142400 0x5586cc14ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f566504d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cbfc54a0 0x5586cc14bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f566584e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc142400 0x5586cc14ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:57330/0 (socket says 172.21.15.19:57330) 2024-05-09T02:15:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f566504d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cbfc54a0 0x5586cc14bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33340/0 (socket says 172.21.15.19:33340) 2024-05-09T02:15:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f566484c700 1 -- 172.21.15.19:0/232422006 learned_addr learned my addr 172.21.15.19:0/232422006 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f5675cc7b80 1 -- 172.21.15.19:0/232422006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5586cc143100 con 0x5586cbfc54a0 2024-05-09T02:15:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f5675cc7b80 1 -- 172.21.15.19:0/232422006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5586cbfffb20 con 0x5586cc03fd00 2024-05-09T02:15:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f5675cc7b80 1 -- 172.21.15.19:0/232422006 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5586cc158230 con 0x5586cc142400 2024-05-09T02:15:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f566484c700 1 -- 172.21.15.19:0/232422006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc142400 msgr2=0x5586cc14ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f566484c700 1 --2- 172.21.15.19:0/232422006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc142400 0x5586cc14ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f566484c700 1 -- 172.21.15.19:0/232422006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cbfc54a0 msgr2=0x5586cc14bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f566484c700 1 --2- 172.21.15.19:0/232422006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cbfc54a0 0x5586cc14bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.747+0000 7f566484c700 1 -- 172.21.15.19:0/232422006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586cbee0490 con 0x5586cc03fd00 2024-05-09T02:15:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.748+0000 7f5675cc7b80 10 client.0 _ll_register_callbacks cb 0x5586cbed66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.748+0000 7f566484c700 1 --2- 172.21.15.19:0/232422006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cc03fd00 0x5586cc14c4d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f566006d090 tx=0x7f565c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59a5388bf5b35f05 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:53.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.748+0000 7f5648ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:15:53.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.748+0000 7f5675cc7b80 1 -- 172.21.15.19:0/232422006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56500a8580 con 0x5586cc03fd00 2024-05-09T02:15:53.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.748+0000 7f5648ff9700 1 -- 172.21.15.19:0/232422006 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c01e830 con 0x5586cc03fd00 2024-05-09T02:15:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.748+0000 7f5648ff9700 1 -- 172.21.15.19:0/232422006 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f565c01e9d0 con 0x5586cc03fd00 2024-05-09T02:15:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.748+0000 7f5648ff9700 1 -- 172.21.15.19:0/232422006 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f565c01ec30 con 0x5586cc03fd00 2024-05-09T02:15:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.749+0000 7f5675cc7b80 10 client.25610 fetch_fsmap learned FSMap version 41 2024-05-09T02:15:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.749+0000 7f5675cc7b80 1 -- 172.21.15.19:0/232422006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=41}) -- 0x5586cc1599b0 con 0x5586cc03fd00 2024-05-09T02:15:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.749+0000 7f5648ff9700 1 -- 172.21.15.19:0/232422006 <== mon.2 v2:172.21.15.59:3300/0 4 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f565c026020 con 0x5586cc03fd00 2024-05-09T02:15:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.749+0000 7f5675cc7b80 10 client.25610 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:15:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.749+0000 7f5675cc7b80 10 client.25610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:15:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.749+0000 7f5675cc7b80 20 client.25610 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.749+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/232422006 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5586cc15bee0 0x5586cc15e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.749+0000 7f5675cc7b80 4 client.25610 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:15:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.749+0000 7f5675cc7b80 1 -- 172.21.15.19:0/232422006 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f56500a8580 con 0x5586cc15bee0 2024-05-09T02:15:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.750+0000 7f566504d700 1 --2- 172.21.15.19:0/232422006 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5586cc15bee0 0x5586cc15e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.750+0000 7f566504d700 1 --2- 172.21.15.19:0/232422006 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5586cc15bee0 0x5586cc15e300 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe112e423bd3590 server_cookie=9686480439ad84bf in_seq=0 out_seq=0 2024-05-09T02:15:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.750+0000 7f5648ff9700 10 client.25610 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.751+0000 7f5648ff9700 1 -- 172.21.15.19:0/232422006 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f56500a8580 con 0x5586cc15bee0 2024-05-09T02:15:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.751+0000 7f5648ff9700 10 client.25610 handle_command_reply: tid=0 2024-05-09T02:15:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.751+0000 7f5675cc7b80 1 client.25610 shutdown 2024-05-09T02:15:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.751+0000 7f5675cc7b80 1 -- 172.21.15.19:0/232422006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cc03fd00 msgr2=0x5586cc14c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.751+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/232422006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cc03fd00 0x5586cc14c4d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f566006d090 tx=0x7f565c0047c0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.751+0000 7f5675cc7b80 1 -- 172.21.15.19:0/232422006 shutdown_connections 2024-05-09T02:15:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.751+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/232422006 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5586cc15bee0 0x5586cc15e300 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.751+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/232422006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc142400 0x5586cc14ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.751+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/232422006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cbfc54a0 0x5586cc14bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.751+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/232422006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cc03fd00 0x5586cc14c4d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.751+0000 7f5675cc7b80 1 -- 172.21.15.19:0/232422006 >> 172.21.15.19:0/232422006 conn(0x5586cbfc8f70 msgr2=0x5586cc136ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.752+0000 7f5648ff9700 0 client.25610 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.752+0000 7f5675cc7b80 1 -- 172.21.15.19:0/232422006 shutdown_connections 2024-05-09T02:15:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.752+0000 7f5675cc7b80 1 -- 172.21.15.19:0/232422006 wait complete. 2024-05-09T02:15:53.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.752+0000 7f5675cc7b80 20 client.25610 trim_cache size 0 max 16384 2024-05-09T02:15:53.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.781+0000 7f5675cc7b80 1 Processor -- start 2024-05-09T02:15:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.781+0000 7f5675cc7b80 1 -- start start 2024-05-09T02:15:53.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:53 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:15:53.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.781+0000 7f5675cc7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc8f70 0x5586cc147460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.781+0000 7f5675cc7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cbfc54a0 0x5586cc137a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.781+0000 7f5675cc7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc03fd00 0x5586cc1463b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.781+0000 7f5675cc7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f564400d6d0 con 0x5586cbfc54a0 2024-05-09T02:15:53.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.781+0000 7f5675cc7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f564400f2d0 con 0x5586cbfc8f70 2024-05-09T02:15:53.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.781+0000 7f5675cc7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f564400dda0 con 0x5586cc03fd00 2024-05-09T02:15:53.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.781+0000 7f566504d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc8f70 0x5586cc147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.782+0000 7f566584e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc03fd00 0x5586cc1463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.782+0000 7f566584e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc03fd00 0x5586cc1463b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:57334/0 (socket says 172.21.15.19:57334) 2024-05-09T02:15:53.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.782+0000 7f566584e700 1 -- 172.21.15.19:0/2041868987 learned_addr learned my addr 172.21.15.19:0/2041868987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:53.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.782+0000 7f566484c700 1 --2- 172.21.15.19:0/2041868987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cbfc54a0 0x5586cc137a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.782+0000 7f566584e700 1 -- 172.21.15.19:0/2041868987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc8f70 msgr2=0x5586cc147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:53.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.782+0000 7f566584e700 1 --2- 172.21.15.19:0/2041868987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc8f70 0x5586cc147460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.782+0000 7f566584e700 1 -- 172.21.15.19:0/2041868987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cbfc54a0 msgr2=0x5586cc137a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:53.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.782+0000 7f566584e700 1 --2- 172.21.15.19:0/2041868987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cbfc54a0 0x5586cc137a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.782+0000 7f566584e700 1 -- 172.21.15.19:0/2041868987 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586cc147c80 con 0x5586cc03fd00 2024-05-09T02:15:53.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.782+0000 7f566504d700 1 --2- 172.21.15.19:0/2041868987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc8f70 0x5586cc147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:15:53.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.782+0000 7f566584e700 1 --2- 172.21.15.19:0/2041868987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc03fd00 0x5586cc1463b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f565800b0a0 tx=0x7f5658009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4c49ee2de26dd3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:53.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.783+0000 7f564affd700 1 -- 172.21.15.19:0/2041868987 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565800bab0 con 0x5586cc03fd00 2024-05-09T02:15:53.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.783+0000 7f564affd700 1 -- 172.21.15.19:0/2041868987 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565800bc50 con 0x5586cc03fd00 2024-05-09T02:15:53.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.783+0000 7f564affd700 1 -- 172.21.15.19:0/2041868987 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658007da0 con 0x5586cc03fd00 2024-05-09T02:15:53.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.784+0000 7f5675cc7b80 1 -- 172.21.15.19:0/2041868987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc03fd00 msgr2=0x5586cc1463b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:53.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.784+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/2041868987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc03fd00 0x5586cc1463b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f565800b0a0 tx=0x7f5658009520 comp rx=0 tx=0).stop 2024-05-09T02:15:53.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.784+0000 7f5675cc7b80 1 -- 172.21.15.19:0/2041868987 shutdown_connections 2024-05-09T02:15:53.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.784+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/2041868987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cc03fd00 0x5586cc1463b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.784+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/2041868987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc8f70 0x5586cc147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.784+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/2041868987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cbfc54a0 0x5586cc137a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.784+0000 7f5675cc7b80 1 -- 172.21.15.19:0/2041868987 >> 172.21.15.19:0/2041868987 conn(0x5586cc13d520 msgr2=0x5586cc145ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:53.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.784+0000 7f5675cc7b80 1 -- 172.21.15.19:0/2041868987 shutdown_connections 2024-05-09T02:15:53.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.784+0000 7f5675cc7b80 1 -- 172.21.15.19:0/2041868987 wait complete. 2024-05-09T02:15:53.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.784+0000 7f5675cc7b80 1 -- start start 2024-05-09T02:15:53.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.785+0000 7f5675cc7b80 1 Processor -- start 2024-05-09T02:15:53.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.785+0000 7f5675cc7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc54a0 0x5586cc154190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.785+0000 7f5675cc7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cbfc8f70 0x5586cc1546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.785+0000 7f5675cc7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cc03fd00 0x5586cc151160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.785+0000 7f5675cc7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f564400ec00 con 0x5586cc03fd00 2024-05-09T02:15:53.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.785+0000 7f5675cc7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f564400e470 con 0x5586cbfc54a0 2024-05-09T02:15:53.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.785+0000 7f5675cc7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f564400b670 con 0x5586cbfc8f70 2024-05-09T02:15:53.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.785+0000 7f5675cc7b80 10 client.0 _ll_register_callbacks cb 0x5586cbed66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:15:53.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.785+0000 7f566584e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cc03fd00 0x5586cc151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.785+0000 7f566484c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cbfc8f70 0x5586cc1546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.785+0000 7f566584e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cc03fd00 0x5586cc151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33368/0 (socket says 172.21.15.19:33368) 2024-05-09T02:15:53.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.785+0000 7f566484c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cbfc8f70 0x5586cc1546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:57346/0 (socket says 172.21.15.19:57346) 2024-05-09T02:15:53.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.785+0000 7f566584e700 1 -- 172.21.15.19:0/4086303072 learned_addr learned my addr 172.21.15.19:0/4086303072 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:15:53.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.785+0000 7f566504d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc54a0 0x5586cc154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.786+0000 7f566484c700 1 -- 172.21.15.19:0/4086303072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc54a0 msgr2=0x5586cc154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:53.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.786+0000 7f566484c700 1 --2- 172.21.15.19:0/4086303072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc54a0 0x5586cc154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.786+0000 7f566484c700 1 -- 172.21.15.19:0/4086303072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cc03fd00 msgr2=0x5586cc151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:15:53.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.786+0000 7f566484c700 1 --2- 172.21.15.19:0/4086303072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cc03fd00 0x5586cc151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.786+0000 7f566484c700 1 -- 172.21.15.19:0/4086303072 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586cc14bd60 con 0x5586cbfc8f70 2024-05-09T02:15:53.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.786+0000 7f566584e700 1 --2- 172.21.15.19:0/4086303072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cc03fd00 0x5586cc151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:15:53.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.786+0000 7f566504d700 1 --2- 172.21.15.19:0/4086303072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc54a0 0x5586cc154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:15:53.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.786+0000 7f566484c700 1 --2- 172.21.15.19:0/4086303072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cbfc8f70 0x5586cc1546d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f565c0097c0 tx=0x7f565c018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b4f31eb8b70a04c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:15:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.786+0000 7f5648ff9700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:15:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.786+0000 7f5648ff9700 1 -- 172.21.15.19:0/4086303072 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c00fbb0 con 0x5586cbfc8f70 2024-05-09T02:15:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.786+0000 7f5648ff9700 1 -- 172.21.15.19:0/4086303072 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f565c028440 con 0x5586cbfc8f70 2024-05-09T02:15:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.786+0000 7f5675cc7b80 1 -- 172.21.15.19:0/4086303072 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f565800bc50 con 0x5586cbfc8f70 2024-05-09T02:15:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.786+0000 7f5648ff9700 1 -- 172.21.15.19:0/4086303072 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f565c01ea50 con 0x5586cbfc8f70 2024-05-09T02:15:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.786+0000 7f5675cc7b80 10 client.25678 fetch_fsmap learned FSMap version 41 2024-05-09T02:15:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.786+0000 7f5675cc7b80 1 -- 172.21.15.19:0/4086303072 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5586cc1c6db0 con 0x5586cbfc8f70 2024-05-09T02:15:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.787+0000 7f5648ff9700 1 -- 172.21.15.19:0/4086303072 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565800bc50 con 0x5586cbfc8f70 2024-05-09T02:15:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.787+0000 7f5648ff9700 1 -- 172.21.15.19:0/4086303072 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f565c03b070 con 0x5586cbfc8f70 2024-05-09T02:15:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.787+0000 7f5675cc7b80 10 client.25678 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:15:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.787+0000 7f5675cc7b80 10 client.25678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:15:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.787+0000 7f5675cc7b80 20 client.25678 populate_metadata read hostname 'smithi019' 2024-05-09T02:15:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.788+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/4086303072 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5586cc142400 0x5586cc1c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:15:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.788+0000 7f5675cc7b80 4 client.25678 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:15:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.788+0000 7f5675cc7b80 1 -- 172.21.15.19:0/4086303072 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586cc072a80 con 0x5586cc142400 2024-05-09T02:15:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.788+0000 7f566504d700 1 --2- 172.21.15.19:0/4086303072 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5586cc142400 0x5586cc1c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:15:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.790+0000 7f5648ff9700 10 client.25678 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.790+0000 7f566504d700 1 --2- 172.21.15.19:0/4086303072 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5586cc142400 0x5586cc1c9550 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63ace225c7b43dd3 server_cookie=ffad2405817f5422 in_seq=0 out_seq=0 2024-05-09T02:15:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.790+0000 7f5648ff9700 1 -- 172.21.15.19:0/4086303072 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5586cc072a80 con 0x5586cc142400 2024-05-09T02:15:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.790+0000 7f5648ff9700 10 client.25678 handle_command_reply: tid=0 2024-05-09T02:15:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.790+0000 7f5675cc7b80 1 client.25678 shutdown 2024-05-09T02:15:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.790+0000 7f5675cc7b80 1 -- 172.21.15.19:0/4086303072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cbfc8f70 msgr2=0x5586cc1546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.790+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/4086303072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cbfc8f70 0x5586cc1546d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f565c0097c0 tx=0x7f565c018000 comp rx=0 tx=0).stop 2024-05-09T02:15:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.790+0000 7f5675cc7b80 1 -- 172.21.15.19:0/4086303072 shutdown_connections 2024-05-09T02:15:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.790+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/4086303072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586cc03fd00 0x5586cc151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.790+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/4086303072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586cbfc54a0 0x5586cc154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.790+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/4086303072 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5586cc142400 0x5586cc1c9550 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.790+0000 7f5675cc7b80 1 --2- 172.21.15.19:0/4086303072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586cbfc8f70 0x5586cc1546d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.790+0000 7f5648ff9700 0 client.25678 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:15:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.790+0000 7f5675cc7b80 1 -- 172.21.15.19:0/4086303072 >> 172.21.15.19:0/4086303072 conn(0x5586cc13d520 msgr2=0x5586cc137780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:53.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.791+0000 7f5675cc7b80 1 -- 172.21.15.19:0/4086303072 shutdown_connections 2024-05-09T02:15:53.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.791+0000 7f5675cc7b80 1 -- 172.21.15.19:0/4086303072 wait complete. 2024-05-09T02:15:53.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.791+0000 7f5675cc7b80 20 client.25678 trim_cache size 0 max 16384 2024-05-09T02:15:53.808 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:15:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T02:15:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T02:15:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: "d30e3e2e-84ab-4c85-8450-4e0cee8c1fad": { 2024-05-09T02:15:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T02:15:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "d30e3e2e-84ab-4c85-8450-4e0cee8c1fad", 2024-05-09T02:15:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T02:15:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:15:53.809 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:15:53.809 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:15:53.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.791+0000 7f56672b1700 1 -- 172.21.15.19:0/3681181548 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f564c0b2b40 msgr2=0x7f564c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:53.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.791+0000 7f56672b1700 1 --2- 172.21.15.19:0/3681181548 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f564c0b2b40 0x7f564c0b5050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5660105040 tx=0x7f565c009260 comp rx=0 tx=0).stop 2024-05-09T02:15:53.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.791+0000 7f56672b1700 1 -- 172.21.15.19:0/3681181548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5660106580 msgr2=0x7f566019cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:15:53.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.791+0000 7f56672b1700 1 --2- 172.21.15.19:0/3681181548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5660106580 0x7f566019cfc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f565000d980 tx=0x7f565000dc90 comp rx=0 tx=0).stop 2024-05-09T02:15:53.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.791+0000 7f56672b1700 1 -- 172.21.15.19:0/3681181548 shutdown_connections 2024-05-09T02:15:53.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.791+0000 7f56672b1700 1 --2- 172.21.15.19:0/3681181548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f566010c5c0 0x7f56601a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.791+0000 7f56672b1700 1 --2- 172.21.15.19:0/3681181548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5660106580 0x7f566019cfc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.791+0000 7f56672b1700 1 --2- 172.21.15.19:0/3681181548 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f564c0b2b40 0x7f564c0b5050 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.791+0000 7f56672b1700 1 --2- 172.21.15.19:0/3681181548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5660106ed0 0x7f566019d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:15:53.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.791+0000 7f56672b1700 1 -- 172.21.15.19:0/3681181548 >> 172.21.15.19:0/3681181548 conn(0x7f5660102080 msgr2=0x7f566010d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:15:53.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.792+0000 7f56672b1700 1 -- 172.21.15.19:0/3681181548 shutdown_connections 2024-05-09T02:15:53.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:15:53.792+0000 7f56672b1700 1 -- 172.21.15.19:0/3681181548 wait complete. 2024-05-09T02:15:54.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:54 smithi019 ceph-mon[30151]: pgmap v141: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.6 MiB/s wr, 573 op/s 2024-05-09T02:15:54.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:54 smithi059 ceph-mon[34781]: pgmap v141: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.6 MiB/s wr, 573 op/s 2024-05-09T02:15:54.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:54 smithi196 ceph-mon[34718]: pgmap v141: 97 pgs: 97 active+clean; 3.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.6 MiB/s wr, 573 op/s 2024-05-09T02:15:55.014 INFO:tasks.cephfs.filesystem:scrub status for tag:d30e3e2e-84ab-4c85-8450-4e0cee8c1fad - {'path': '/', 'tag': 'd30e3e2e-84ab-4c85-8450-4e0cee8c1fad', 'options': 'recursive,force'} 2024-05-09T02:15:56.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:56 smithi019 ceph-mon[30151]: pgmap v142: 97 pgs: 97 active+clean; 3.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 14 MiB/s wr, 865 op/s 2024-05-09T02:15:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:56 smithi059 ceph-mon[34781]: pgmap v142: 97 pgs: 97 active+clean; 3.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 14 MiB/s wr, 865 op/s 2024-05-09T02:15:56.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:56 smithi196 ceph-mon[34718]: pgmap v142: 97 pgs: 97 active+clean; 3.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 14 MiB/s wr, 865 op/s 2024-05-09T02:15:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 255.0 s, 2943.1 tps, lat 33.643 ms stddev 41.627 2024-05-09T02:15:58.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:15:58 smithi019 ceph-mon[30151]: pgmap v143: 97 pgs: 97 active+clean; 3.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 714 op/s 2024-05-09T02:15:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:15:58 smithi059 ceph-mon[34781]: pgmap v143: 97 pgs: 97 active+clean; 3.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 714 op/s 2024-05-09T02:15:58.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:15:58 smithi196 ceph-mon[34718]: pgmap v143: 97 pgs: 97 active+clean; 3.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 714 op/s 2024-05-09T02:16:00.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:00 smithi019 ceph-mon[30151]: pgmap v144: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-05-09T02:16:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:00 smithi059 ceph-mon[34781]: pgmap v144: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-05-09T02:16:00.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:00 smithi196 ceph-mon[34718]: pgmap v144: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-05-09T02:16:01.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:01 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:16:01.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:01 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:16:01.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:01 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:16:01.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:01 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:16:01.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:01 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:16:01.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:01 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:16:01.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:01 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:16:01.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:01 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:16:01.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:01 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:16:01.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:01 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:16:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:01 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:16:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:01 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:16:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:01 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:16:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:01 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:16:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:01 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:16:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 260.0 s, 1559.8 tps, lat 63.695 ms stddev 121.363 2024-05-09T02:16:02.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:02 smithi059 ceph-mon[34781]: pgmap v145: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 684 op/s 2024-05-09T02:16:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:02 smithi196 ceph-mon[34718]: pgmap v145: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 684 op/s 2024-05-09T02:16:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:02 smithi019 ceph-mon[30151]: pgmap v145: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 684 op/s 2024-05-09T02:16:04.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:04 smithi019 ceph-mon[30151]: pgmap v146: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 708 op/s 2024-05-09T02:16:04.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:04 smithi059 ceph-mon[34781]: pgmap v146: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 708 op/s 2024-05-09T02:16:04.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:04 smithi196 ceph-mon[34718]: pgmap v146: 97 pgs: 97 active+clean; 3.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 708 op/s 2024-05-09T02:16:06.595 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:06 smithi196 ceph-mon[34718]: pgmap v147: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 31 MiB/s wr, 2.06k op/s 2024-05-09T02:16:06.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:06 smithi059 ceph-mon[34781]: pgmap v147: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 31 MiB/s wr, 2.06k op/s 2024-05-09T02:16:06.781 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:06 smithi019 ceph-mon[30151]: pgmap v147: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 31 MiB/s wr, 2.06k op/s 2024-05-09T02:16:07.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 265.0 s, 2489.2 tps, lat 40.809 ms stddev 47.783 2024-05-09T02:16:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:08 smithi059 ceph-mon[34781]: pgmap v148: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 MiB/s wr, 1.76k op/s 2024-05-09T02:16:08.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:08 smithi196 ceph-mon[34718]: pgmap v148: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 MiB/s wr, 1.76k op/s 2024-05-09T02:16:08.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:08 smithi019 ceph-mon[30151]: pgmap v148: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 MiB/s wr, 1.76k op/s 2024-05-09T02:16:10.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:10 smithi059 ceph-mon[34781]: pgmap v149: 97 pgs: 97 active+clean; 3.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 30 MiB/s wr, 2.05k op/s 2024-05-09T02:16:10.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:10 smithi196 ceph-mon[34718]: pgmap v149: 97 pgs: 97 active+clean; 3.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 30 MiB/s wr, 2.05k op/s 2024-05-09T02:16:10.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:10 smithi019 ceph-mon[30151]: pgmap v149: 97 pgs: 97 active+clean; 3.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 30 MiB/s wr, 2.05k op/s 2024-05-09T02:16:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 270.0 s, 2819.6 tps, lat 35.330 ms stddev 39.609 2024-05-09T02:16:12.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:12 smithi059 ceph-mon[34781]: pgmap v150: 97 pgs: 97 active+clean; 3.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.68k op/s 2024-05-09T02:16:12.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:12 smithi196 ceph-mon[34718]: pgmap v150: 97 pgs: 97 active+clean; 3.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.68k op/s 2024-05-09T02:16:12.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:12 smithi019 ceph-mon[30151]: pgmap v150: 97 pgs: 97 active+clean; 3.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.68k op/s 2024-05-09T02:16:14.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:14 smithi059 ceph-mon[34781]: pgmap v151: 97 pgs: 97 active+clean; 3.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.68k op/s 2024-05-09T02:16:14.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:14 smithi196 ceph-mon[34718]: pgmap v151: 97 pgs: 97 active+clean; 3.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.68k op/s 2024-05-09T02:16:14.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:14 smithi019 ceph-mon[30151]: pgmap v151: 97 pgs: 97 active+clean; 3.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.68k op/s 2024-05-09T02:16:16.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:16 smithi059 ceph-mon[34781]: pgmap v152: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 28 MiB/s wr, 1.93k op/s 2024-05-09T02:16:16.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:16 smithi196 ceph-mon[34718]: pgmap v152: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 28 MiB/s wr, 1.93k op/s 2024-05-09T02:16:16.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:16 smithi019 ceph-mon[30151]: pgmap v152: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 28 MiB/s wr, 1.93k op/s 2024-05-09T02:16:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 275.0 s, 1964.2 tps, lat 51.032 ms stddev 199.772 2024-05-09T02:16:18.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:18 smithi059 ceph-mon[34781]: pgmap v153: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 573 op/s 2024-05-09T02:16:18.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:18 smithi196 ceph-mon[34718]: pgmap v153: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 573 op/s 2024-05-09T02:16:18.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:18 smithi019 ceph-mon[30151]: pgmap v153: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 573 op/s 2024-05-09T02:16:20.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:20 smithi059 ceph-mon[34781]: pgmap v154: 97 pgs: 97 active+clean; 3.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 891 op/s 2024-05-09T02:16:20.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:20 smithi196 ceph-mon[34718]: pgmap v154: 97 pgs: 97 active+clean; 3.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 891 op/s 2024-05-09T02:16:20.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:20 smithi019 ceph-mon[30151]: pgmap v154: 97 pgs: 97 active+clean; 3.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 891 op/s 2024-05-09T02:16:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 280.0 s, 3374.8 tps, lat 29.445 ms stddev 31.814 2024-05-09T02:16:22.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:22 smithi059 ceph-mon[34781]: pgmap v155: 97 pgs: 97 active+clean; 3.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 601 op/s 2024-05-09T02:16:22.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:22 smithi196 ceph-mon[34718]: pgmap v155: 97 pgs: 97 active+clean; 3.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 601 op/s 2024-05-09T02:16:22.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:22 smithi019 ceph-mon[30151]: pgmap v155: 97 pgs: 97 active+clean; 3.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 601 op/s 2024-05-09T02:16:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:24 smithi059 ceph-mon[34781]: pgmap v156: 97 pgs: 97 active+clean; 3.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.8 MiB/s wr, 615 op/s 2024-05-09T02:16:24.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:24 smithi196 ceph-mon[34718]: pgmap v156: 97 pgs: 97 active+clean; 3.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.8 MiB/s wr, 615 op/s 2024-05-09T02:16:24.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:24 smithi019 ceph-mon[30151]: pgmap v156: 97 pgs: 97 active+clean; 3.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.8 MiB/s wr, 615 op/s 2024-05-09T02:16:25.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:16:25.428 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:16:26.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:26 smithi059 ceph-mon[34781]: pgmap v157: 97 pgs: 97 active+clean; 3.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 907 op/s 2024-05-09T02:16:26.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:26 smithi019 ceph-mon[30151]: pgmap v157: 97 pgs: 97 active+clean; 3.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 907 op/s 2024-05-09T02:16:26.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:26 smithi196 ceph-mon[34718]: pgmap v157: 97 pgs: 97 active+clean; 3.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 907 op/s 2024-05-09T02:16:27.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.503+0000 7f56f4a19700 1 -- 172.21.15.19:0/1396432669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56f010a7c0 msgr2=0x7f56f010b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.503+0000 7f56f4a19700 1 --2- 172.21.15.19:0/1396432669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56f010a7c0 0x7f56f010b010 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f56e40098f0 tx=0x7f56e4009c00 comp rx=0 tx=0).stop 2024-05-09T02:16:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.503+0000 7f56f4a19700 1 -- 172.21.15.19:0/1396432669 shutdown_connections 2024-05-09T02:16:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.503+0000 7f56f4a19700 1 --2- 172.21.15.19:0/1396432669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56f010fe80 0x7f56f0110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.503+0000 7f56f4a19700 1 --2- 172.21.15.19:0/1396432669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56f010a7c0 0x7f56f010b010 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.503+0000 7f56f4a19700 1 --2- 172.21.15.19:0/1396432669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f010b5e0 0x7f56f010f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.503+0000 7f56f4a19700 1 -- 172.21.15.19:0/1396432669 >> 172.21.15.19:0/1396432669 conn(0x7f56f006f4d0 msgr2=0x7f56f006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.503+0000 7f56f4a19700 1 -- 172.21.15.19:0/1396432669 shutdown_connections 2024-05-09T02:16:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.503+0000 7f56f4a19700 1 -- 172.21.15.19:0/1396432669 wait complete. 2024-05-09T02:16:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.505+0000 7f56f4a19700 1 Processor -- start 2024-05-09T02:16:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.505+0000 7f56f4a19700 1 -- start start 2024-05-09T02:16:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.505+0000 7f56f4a19700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56f010a7c0 0x7f56f01ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.505+0000 7f56f4a19700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f010b5e0 0x7f56f01abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.505+0000 7f56f4a19700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56f010fe80 0x7f56f01a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.505+0000 7f56f4a19700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0116b40 con 0x7f56f010a7c0 2024-05-09T02:16:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.505+0000 7f56f4a19700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01169c0 con 0x7f56f010b5e0 2024-05-09T02:16:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.505+0000 7f56f4a19700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0116cc0 con 0x7f56f010fe80 2024-05-09T02:16:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.505+0000 7f56edd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f010b5e0 0x7f56f01abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.505+0000 7f56edd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f010b5e0 0x7f56f01abb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:55408/0 (socket says 172.21.15.19:55408) 2024-05-09T02:16:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.505+0000 7f56edd9b700 1 -- 172.21.15.19:0/2110906183 learned_addr learned my addr 172.21.15.19:0/2110906183 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.505+0000 7f56ee59c700 1 --2- 172.21.15.19:0/2110906183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56f010a7c0 0x7f56f01ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.506+0000 7f56eed9d700 1 --2- 172.21.15.19:0/2110906183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56f010fe80 0x7f56f01a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.506+0000 7f56edd9b700 1 -- 172.21.15.19:0/2110906183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56f010fe80 msgr2=0x7f56f01a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.506+0000 7f56edd9b700 1 --2- 172.21.15.19:0/2110906183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56f010fe80 0x7f56f01a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.506+0000 7f56edd9b700 1 -- 172.21.15.19:0/2110906183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56f010a7c0 msgr2=0x7f56f01ab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.506+0000 7f56edd9b700 1 --2- 172.21.15.19:0/2110906183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56f010a7c0 0x7f56f01ab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.506+0000 7f56edd9b700 1 -- 172.21.15.19:0/2110906183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f01a5e80 con 0x7f56f010b5e0 2024-05-09T02:16:27.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.506+0000 7f56edd9b700 1 --2- 172.21.15.19:0/2110906183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f010b5e0 0x7f56f01abb00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f56e800ef70 tx=0x7f56e800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:27.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.507+0000 7f56df7fe700 1 -- 172.21.15.19:0/2110906183 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e800c9c0 con 0x7f56f010b5e0 2024-05-09T02:16:27.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.507+0000 7f56df7fe700 1 -- 172.21.15.19:0/2110906183 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e8019070 con 0x7f56f010b5e0 2024-05-09T02:16:27.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.507+0000 7f56df7fe700 1 -- 172.21.15.19:0/2110906183 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8014480 con 0x7f56f010b5e0 2024-05-09T02:16:27.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.507+0000 7f56f4a19700 1 -- 172.21.15.19:0/2110906183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f01a6110 con 0x7f56f010b5e0 2024-05-09T02:16:27.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.507+0000 7f56f4a19700 1 -- 172.21.15.19:0/2110906183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f01a6540 con 0x7f56f010b5e0 2024-05-09T02:16:27.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.511+0000 7f56df7fe700 1 -- 172.21.15.19:0/2110906183 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f56e8012050 con 0x7f56f010b5e0 2024-05-09T02:16:27.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.512+0000 7f56df7fe700 1 --2- 172.21.15.19:0/2110906183 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f56d80b2aa0 0x7f56d80b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.512+0000 7f56df7fe700 1 -- 172.21.15.19:0/2110906183 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f56e80e2640 con 0x7f56f010b5e0 2024-05-09T02:16:27.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.513+0000 7f56ee59c700 1 --2- 172.21.15.19:0/2110906183 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f56d80b2aa0 0x7f56d80b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.513+0000 7f56ee59c700 1 --2- 172.21.15.19:0/2110906183 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f56d80b2aa0 0x7f56d80b4fb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009810 tx=0x7f56e4019040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:27.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.515+0000 7f570342fb80 1 Processor -- start 2024-05-09T02:16:27.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.515+0000 7f570342fb80 1 -- start start 2024-05-09T02:16:27.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.515+0000 7f570342fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562654511160 0x5626545aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.515+0000 7f570342fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626544274a0 0x5626545aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.516+0000 7f570342fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626545a46a0 0x5626545a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.516+0000 7f570342fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5626544c6ff0 con 0x5626544274a0 2024-05-09T02:16:27.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.516+0000 7f570342fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5626544657f0 con 0x562654511160 2024-05-09T02:16:27.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.516+0000 7f570342fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5626544c2fb0 con 0x5626545a46a0 2024-05-09T02:16:27.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.516+0000 7f56edd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626544274a0 0x5626545aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.516+0000 7f56edd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626544274a0 0x5626545aaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48012/0 (socket says 172.21.15.19:48012) 2024-05-09T02:16:27.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.516+0000 7f56edd9b700 1 -- 172.21.15.19:0/837236399 learned_addr learned my addr 172.21.15.19:0/837236399 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:27.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.516+0000 7f56ee59c700 1 --2- 172.21.15.19:0/837236399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562654511160 0x5626545aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.517+0000 7f56edd9b700 1 -- 172.21.15.19:0/837236399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626545a46a0 msgr2=0x5626545a4ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:16:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.517+0000 7f56edd9b700 1 --2- 172.21.15.19:0/837236399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626545a46a0 0x5626545a4ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.517+0000 7f56edd9b700 1 -- 172.21.15.19:0/837236399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562654511160 msgr2=0x5626545aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.517+0000 7f56edd9b700 1 --2- 172.21.15.19:0/837236399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562654511160 0x5626545aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.517+0000 7f56edd9b700 1 -- 172.21.15.19:0/837236399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56265445d370 con 0x5626544274a0 2024-05-09T02:16:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.517+0000 7f56edd9b700 1 --2- 172.21.15.19:0/837236399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626544274a0 0x5626545aaa30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f56e80090a0 tx=0x7f56e80a9030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845c03c5d8897ae6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.518+0000 7f56dcff9700 1 -- 172.21.15.19:0/837236399 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e80a93d0 con 0x5626544274a0 2024-05-09T02:16:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.518+0000 7f56dcff9700 1 -- 172.21.15.19:0/837236399 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e80a9fc0 con 0x5626544274a0 2024-05-09T02:16:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.518+0000 7f56dcff9700 1 -- 172.21.15.19:0/837236399 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e80a9b20 con 0x5626544274a0 2024-05-09T02:16:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.519+0000 7f570342fb80 1 -- 172.21.15.19:0/837236399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626544274a0 msgr2=0x5626545aaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.519+0000 7f570342fb80 1 --2- 172.21.15.19:0/837236399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626544274a0 0x5626545aaa30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f56e80090a0 tx=0x7f56e80a9030 comp rx=0 tx=0).stop 2024-05-09T02:16:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.519+0000 7f570342fb80 1 -- 172.21.15.19:0/837236399 shutdown_connections 2024-05-09T02:16:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.519+0000 7f570342fb80 1 --2- 172.21.15.19:0/837236399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626545a46a0 0x5626545a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.519+0000 7f570342fb80 1 --2- 172.21.15.19:0/837236399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562654511160 0x5626545aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.519+0000 7f570342fb80 1 --2- 172.21.15.19:0/837236399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626544274a0 0x5626545aaa30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.519+0000 7f570342fb80 1 -- 172.21.15.19:0/837236399 >> 172.21.15.19:0/837236399 conn(0x5626544876a0 msgr2=0x56265459f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.521+0000 7f570342fb80 1 -- 172.21.15.19:0/837236399 shutdown_connections 2024-05-09T02:16:27.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.521+0000 7f570342fb80 1 -- 172.21.15.19:0/837236399 wait complete. 2024-05-09T02:16:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.521+0000 7f570342fb80 1 -- start start 2024-05-09T02:16:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.522+0000 7f570342fb80 1 Processor -- start 2024-05-09T02:16:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.522+0000 7f570342fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626544274a0 0x5626545ae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.522+0000 7f570342fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562654511160 0x5626545ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.522+0000 7f570342fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545a46a0 0x5626545aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.522+0000 7f570342fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562654342ba0 con 0x5626544274a0 2024-05-09T02:16:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.522+0000 7f570342fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562654481150 con 0x5626545a46a0 2024-05-09T02:16:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.522+0000 7f570342fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5626545a54e0 con 0x562654511160 2024-05-09T02:16:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.522+0000 7f56ee59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626544274a0 0x5626545ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.522+0000 7f570342fb80 10 client.0 _ll_register_callbacks cb 0x5626543386c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.522+0000 7f56ee59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626544274a0 0x5626545ae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48020/0 (socket says 172.21.15.19:48020) 2024-05-09T02:16:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.522+0000 7f56ee59c700 1 -- 172.21.15.19:0/3143381709 learned_addr learned my addr 172.21.15.19:0/3143381709 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.522+0000 7f56edd9b700 1 --2- 172.21.15.19:0/3143381709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562654511160 0x5626545ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.523+0000 7f56eed9d700 1 --2- 172.21.15.19:0/3143381709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545a46a0 0x5626545aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.523+0000 7f56edd9b700 1 -- 172.21.15.19:0/3143381709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545a46a0 msgr2=0x5626545aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.523+0000 7f56edd9b700 1 --2- 172.21.15.19:0/3143381709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545a46a0 0x5626545aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.523+0000 7f56edd9b700 1 -- 172.21.15.19:0/3143381709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626544274a0 msgr2=0x5626545ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.523+0000 7f56edd9b700 1 --2- 172.21.15.19:0/3143381709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626544274a0 0x5626545ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.523+0000 7f56edd9b700 1 -- 172.21.15.19:0/3143381709 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562654342490 con 0x562654511160 2024-05-09T02:16:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.523+0000 7f56edd9b700 1 --2- 172.21.15.19:0/3143381709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562654511160 0x5626545ae770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f56e80a9a80 tx=0x7f56e800cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17376d5192f6a786 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.523+0000 7f56d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:16:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.524+0000 7f56d5ffb700 1 -- 172.21.15.19:0/3143381709 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e80c2060 con 0x562654511160 2024-05-09T02:16:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.524+0000 7f56d5ffb700 1 -- 172.21.15.19:0/3143381709 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f56e80b24a0 con 0x562654511160 2024-05-09T02:16:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.524+0000 7f570342fb80 1 -- 172.21.15.19:0/3143381709 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56e80a9fc0 con 0x562654511160 2024-05-09T02:16:27.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.525+0000 7f56d5ffb700 1 -- 172.21.15.19:0/3143381709 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f56e80a93d0 con 0x562654511160 2024-05-09T02:16:27.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.525+0000 7f570342fb80 10 client.25684 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.525+0000 7f570342fb80 1 -- 172.21.15.19:0/3143381709 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5626545bbb50 con 0x562654511160 2024-05-09T02:16:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.525+0000 7f56d5ffb700 1 -- 172.21.15.19:0/3143381709 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e80a98c0 con 0x562654511160 2024-05-09T02:16:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.525+0000 7f56d5ffb700 1 -- 172.21.15.19:0/3143381709 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f56e80a8b20 con 0x562654511160 2024-05-09T02:16:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.525+0000 7f570342fb80 10 client.25684 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.525+0000 7f570342fb80 10 client.25684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:16:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.526+0000 7f570342fb80 20 client.25684 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.526+0000 7f570342fb80 1 --2- 172.21.15.19:0/3143381709 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5626545be020 0x5626545c0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.526+0000 7f570342fb80 4 client.25684 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:16:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.526+0000 7f570342fb80 1 -- 172.21.15.19:0/3143381709 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f56e80a9fc0 con 0x5626545be020 2024-05-09T02:16:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.526+0000 7f56ee59c700 1 --2- 172.21.15.19:0/3143381709 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5626545be020 0x5626545c0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.526+0000 7f56d5ffb700 10 client.25684 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.527+0000 7f56ee59c700 1 --2- 172.21.15.19:0/3143381709 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5626545be020 0x5626545c0440 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24d9f4bfaf3f66f4 server_cookie=e364858e2b6a006e in_seq=0 out_seq=0 2024-05-09T02:16:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.528+0000 7f56d5ffb700 1 -- 172.21.15.19:0/3143381709 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f56e80a9fc0 con 0x5626545be020 2024-05-09T02:16:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.528+0000 7f56d5ffb700 10 client.25684 handle_command_reply: tid=0 2024-05-09T02:16:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.528+0000 7f570342fb80 1 client.25684 shutdown 2024-05-09T02:16:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.528+0000 7f570342fb80 1 -- 172.21.15.19:0/3143381709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562654511160 msgr2=0x5626545ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.528+0000 7f570342fb80 1 --2- 172.21.15.19:0/3143381709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562654511160 0x5626545ae770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f56e80a9a80 tx=0x7f56e800cf10 comp rx=0 tx=0).stop 2024-05-09T02:16:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.528+0000 7f570342fb80 1 -- 172.21.15.19:0/3143381709 shutdown_connections 2024-05-09T02:16:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.528+0000 7f570342fb80 1 --2- 172.21.15.19:0/3143381709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545a46a0 0x5626545aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.528+0000 7f570342fb80 1 --2- 172.21.15.19:0/3143381709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626544274a0 0x5626545ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.528+0000 7f570342fb80 1 --2- 172.21.15.19:0/3143381709 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5626545be020 0x5626545c0440 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.528+0000 7f570342fb80 1 --2- 172.21.15.19:0/3143381709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562654511160 0x5626545ae770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.528+0000 7f56d5ffb700 0 client.25684 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:27.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.528+0000 7f570342fb80 1 -- 172.21.15.19:0/3143381709 >> 172.21.15.19:0/3143381709 conn(0x5626544876a0 msgr2=0x56265459f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:27.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.529+0000 7f570342fb80 1 -- 172.21.15.19:0/3143381709 shutdown_connections 2024-05-09T02:16:27.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.529+0000 7f570342fb80 1 -- 172.21.15.19:0/3143381709 wait complete. 2024-05-09T02:16:27.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.529+0000 7f570342fb80 20 client.25684 trim_cache size 0 max 16384 2024-05-09T02:16:27.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.559+0000 7f570342fb80 1 Processor -- start 2024-05-09T02:16:27.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.559+0000 7f570342fb80 1 -- start start 2024-05-09T02:16:27.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.559+0000 7f570342fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626544274a0 0x5626545a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.559+0000 7f570342fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562654511160 0x5626545b99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.560+0000 7f56ee59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626544274a0 0x5626545a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.560+0000 7f56ee59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626544274a0 0x5626545a40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50634/0 (socket says 172.21.15.19:50634) 2024-05-09T02:16:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.560+0000 7f56edd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562654511160 0x5626545b99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.560+0000 7f56ee59c700 1 -- 172.21.15.19:0/3220599564 learned_addr learned my addr 172.21.15.19:0/3220599564 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.560+0000 7f570342fb80 1 --2- 172.21.15.19:0/3220599564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545a46a0 0x5626545ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.560+0000 7f570342fb80 1 -- 172.21.15.19:0/3220599564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01b56f0 con 0x562654511160 2024-05-09T02:16:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.560+0000 7f570342fb80 1 -- 172.21.15.19:0/3220599564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01b72f0 con 0x5626545a46a0 2024-05-09T02:16:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.560+0000 7f570342fb80 1 -- 172.21.15.19:0/3220599564 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01b5dc0 con 0x5626544274a0 2024-05-09T02:16:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.560+0000 7f56eed9d700 1 --2- 172.21.15.19:0/3220599564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545a46a0 0x5626545ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.561+0000 7f56ee59c700 1 -- 172.21.15.19:0/3220599564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545a46a0 msgr2=0x5626545ae7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.561+0000 7f56ee59c700 1 --2- 172.21.15.19:0/3220599564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545a46a0 0x5626545ae7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.561+0000 7f56ee59c700 1 -- 172.21.15.19:0/3220599564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562654511160 msgr2=0x5626545b99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.561+0000 7f56ee59c700 1 --2- 172.21.15.19:0/3220599564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562654511160 0x5626545b99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.561+0000 7f56ee59c700 1 -- 172.21.15.19:0/3220599564 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626545a99b0 con 0x5626544274a0 2024-05-09T02:16:27.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.561+0000 7f56ee59c700 1 --2- 172.21.15.19:0/3220599564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626544274a0 0x5626545a40c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f56e4000c00 tx=0x7f56e401e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f33e7f8d4b743b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:27.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.561+0000 7f56d7fff700 1 -- 172.21.15.19:0/3220599564 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e4005300 con 0x5626544274a0 2024-05-09T02:16:27.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.561+0000 7f56d7fff700 1 -- 172.21.15.19:0/3220599564 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e40054a0 con 0x5626544274a0 2024-05-09T02:16:27.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.561+0000 7f56d7fff700 1 -- 172.21.15.19:0/3220599564 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e400b7c0 con 0x5626544274a0 2024-05-09T02:16:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.562+0000 7f570342fb80 1 -- 172.21.15.19:0/3220599564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626544274a0 msgr2=0x5626545a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.562+0000 7f570342fb80 1 --2- 172.21.15.19:0/3220599564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626544274a0 0x5626545a40c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f56e4000c00 tx=0x7f56e401e000 comp rx=0 tx=0).stop 2024-05-09T02:16:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.563+0000 7f570342fb80 1 -- 172.21.15.19:0/3220599564 shutdown_connections 2024-05-09T02:16:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.563+0000 7f570342fb80 1 --2- 172.21.15.19:0/3220599564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545a46a0 0x5626545ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.563+0000 7f570342fb80 1 --2- 172.21.15.19:0/3220599564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626544274a0 0x5626545a40c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.563+0000 7f570342fb80 1 --2- 172.21.15.19:0/3220599564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562654511160 0x5626545b99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.563+0000 7f570342fb80 1 -- 172.21.15.19:0/3220599564 >> 172.21.15.19:0/3220599564 conn(0x5626544876a0 msgr2=0x5626545b92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.563+0000 7f570342fb80 1 -- 172.21.15.19:0/3220599564 shutdown_connections 2024-05-09T02:16:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.563+0000 7f570342fb80 1 -- 172.21.15.19:0/3220599564 wait complete. 2024-05-09T02:16:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.563+0000 7f570342fb80 1 -- start start 2024-05-09T02:16:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.563+0000 7f570342fb80 1 Processor -- start 2024-05-09T02:16:27.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.565+0000 7f570342fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562654511160 0x5626545c3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.565+0000 7f570342fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626545a46a0 0x5626545c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.565+0000 7f570342fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545c1c00 0x5626545c2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.565+0000 7f570342fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01b6c20 con 0x562654511160 2024-05-09T02:16:27.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.565+0000 7f570342fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01b6490 con 0x5626545c1c00 2024-05-09T02:16:27.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.565+0000 7f570342fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01b36c0 con 0x5626545a46a0 2024-05-09T02:16:27.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.565+0000 7f570342fb80 10 client.0 _ll_register_callbacks cb 0x5626543386c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:27.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.565+0000 7f56ee59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562654511160 0x5626545c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.565+0000 7f56ee59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562654511160 0x5626545c3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48046/0 (socket says 172.21.15.19:48046) 2024-05-09T02:16:27.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.565+0000 7f56ee59c700 1 -- 172.21.15.19:0/3289103404 learned_addr learned my addr 172.21.15.19:0/3289103404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:27.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.565+0000 7f56eed9d700 1 --2- 172.21.15.19:0/3289103404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545c1c00 0x5626545c2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.565+0000 7f56edd9b700 1 --2- 172.21.15.19:0/3289103404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626545a46a0 0x5626545c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.566+0000 7f56edd9b700 1 -- 172.21.15.19:0/3289103404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545c1c00 msgr2=0x5626545c2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.566+0000 7f56edd9b700 1 --2- 172.21.15.19:0/3289103404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545c1c00 0x5626545c2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.566+0000 7f56edd9b700 1 -- 172.21.15.19:0/3289103404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562654511160 msgr2=0x5626545c3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.566+0000 7f56edd9b700 1 --2- 172.21.15.19:0/3289103404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562654511160 0x5626545c3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.566+0000 7f56edd9b700 1 -- 172.21.15.19:0/3289103404 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562654628580 con 0x5626545a46a0 2024-05-09T02:16:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.566+0000 7f56eed9d700 1 --2- 172.21.15.19:0/3289103404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545c1c00 0x5626545c2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:16:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.566+0000 7f56ee59c700 1 --2- 172.21.15.19:0/3289103404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562654511160 0x5626545c3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:16:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.566+0000 7f56edd9b700 1 --2- 172.21.15.19:0/3289103404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626545a46a0 0x5626545c16c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009f00 tx=0x7f56e80a9030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3669a3abc983fcd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.566+0000 7f56d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:16:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.566+0000 7f56d5ffb700 1 -- 172.21.15.19:0/3289103404 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e80c2070 con 0x5626545a46a0 2024-05-09T02:16:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.566+0000 7f56d5ffb700 1 -- 172.21.15.19:0/3289103404 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f56e80a8b20 con 0x5626545a46a0 2024-05-09T02:16:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.567+0000 7f570342fb80 1 -- 172.21.15.19:0/3289103404 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56e40054a0 con 0x5626545a46a0 2024-05-09T02:16:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.567+0000 7f56d5ffb700 1 -- 172.21.15.19:0/3289103404 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f56e80a9fc0 con 0x5626545a46a0 2024-05-09T02:16:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.567+0000 7f570342fb80 10 client.25690 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:27.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.567+0000 7f570342fb80 1 -- 172.21.15.19:0/3289103404 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562654629660 con 0x5626545a46a0 2024-05-09T02:16:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.568+0000 7f56d5ffb700 1 -- 172.21.15.19:0/3289103404 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e40054a0 con 0x5626545a46a0 2024-05-09T02:16:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.568+0000 7f56d5ffb700 1 -- 172.21.15.19:0/3289103404 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f56e80075c0 con 0x5626545a46a0 2024-05-09T02:16:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.568+0000 7f570342fb80 10 client.25690 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.568+0000 7f570342fb80 10 client.25690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:16:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.568+0000 7f570342fb80 20 client.25690 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.568+0000 7f570342fb80 1 --2- 172.21.15.19:0/3289103404 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x562654629b00 0x56265462bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.568+0000 7f570342fb80 4 client.25690 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:16:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.568+0000 7f570342fb80 1 -- 172.21.15.19:0/3289103404 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5626544d49d0 con 0x562654629b00 2024-05-09T02:16:27.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.568+0000 7f56ee59c700 1 --2- 172.21.15.19:0/3289103404 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x562654629b00 0x56265462bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.569+0000 7f56d5ffb700 10 client.25690 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.569+0000 7f56ee59c700 1 --2- 172.21.15.19:0/3289103404 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x562654629b00 0x56265462bf20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ffbffaeb2afdea4 server_cookie=423684fc89d0f101 in_seq=0 out_seq=0 2024-05-09T02:16:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.569+0000 7f56d5ffb700 1 -- 172.21.15.19:0/3289103404 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5626544d49d0 con 0x562654629b00 2024-05-09T02:16:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.569+0000 7f56d5ffb700 10 client.25690 handle_command_reply: tid=0 2024-05-09T02:16:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.570+0000 7f570342fb80 1 client.25690 shutdown 2024-05-09T02:16:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.570+0000 7f570342fb80 1 -- 172.21.15.19:0/3289103404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626545a46a0 msgr2=0x5626545c16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.570+0000 7f570342fb80 1 --2- 172.21.15.19:0/3289103404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626545a46a0 0x5626545c16c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009f00 tx=0x7f56e80a9030 comp rx=0 tx=0).stop 2024-05-09T02:16:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.570+0000 7f570342fb80 1 -- 172.21.15.19:0/3289103404 shutdown_connections 2024-05-09T02:16:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.570+0000 7f570342fb80 1 --2- 172.21.15.19:0/3289103404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626545c1c00 0x5626545c2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.570+0000 7f570342fb80 1 --2- 172.21.15.19:0/3289103404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562654511160 0x5626545c3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.570+0000 7f570342fb80 1 --2- 172.21.15.19:0/3289103404 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x562654629b00 0x56265462bf20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.570+0000 7f570342fb80 1 --2- 172.21.15.19:0/3289103404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626545a46a0 0x5626545c16c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.570+0000 7f570342fb80 1 -- 172.21.15.19:0/3289103404 >> 172.21.15.19:0/3289103404 conn(0x5626544876a0 msgr2=0x5626545a3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.571+0000 7f56d5ffb700 0 client.25690 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.571+0000 7f570342fb80 1 -- 172.21.15.19:0/3289103404 shutdown_connections 2024-05-09T02:16:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.571+0000 7f570342fb80 1 -- 172.21.15.19:0/3289103404 wait complete. 2024-05-09T02:16:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.571+0000 7f570342fb80 20 client.25690 trim_cache size 0 max 16384 2024-05-09T02:16:27.575 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:16:27.575 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T02:16:27.575 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T02:16:27.575 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:16:27.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.572+0000 7f56f4a19700 1 -- 172.21.15.19:0/2110906183 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f56d80b2aa0 msgr2=0x7f56d80b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:27.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.572+0000 7f56f4a19700 1 --2- 172.21.15.19:0/2110906183 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f56d80b2aa0 0x7f56d80b4fb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009810 tx=0x7f56e4019040 comp rx=0 tx=0).stop 2024-05-09T02:16:27.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.572+0000 7f56f4a19700 1 -- 172.21.15.19:0/2110906183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f010b5e0 msgr2=0x7f56f01abb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:27.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.572+0000 7f56f4a19700 1 --2- 172.21.15.19:0/2110906183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f010b5e0 0x7f56f01abb00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f56e800ef70 tx=0x7f56e800c3a0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.572+0000 7f56f4a19700 1 -- 172.21.15.19:0/2110906183 shutdown_connections 2024-05-09T02:16:27.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.572+0000 7f56f4a19700 1 --2- 172.21.15.19:0/2110906183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56f010fe80 0x7f56f01a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.572+0000 7f56f4a19700 1 --2- 172.21.15.19:0/2110906183 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f56d80b2aa0 0x7f56d80b4fb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.572+0000 7f56f4a19700 1 --2- 172.21.15.19:0/2110906183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56f010a7c0 0x7f56f01ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.572+0000 7f56f4a19700 1 --2- 172.21.15.19:0/2110906183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f010b5e0 0x7f56f01abb00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:27.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.572+0000 7f56f4a19700 1 -- 172.21.15.19:0/2110906183 >> 172.21.15.19:0/2110906183 conn(0x7f56f006f4d0 msgr2=0x7f56f010e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:27.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.573+0000 7f56f4a19700 1 -- 172.21.15.19:0/2110906183 shutdown_connections 2024-05-09T02:16:27.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:27.573+0000 7f56f4a19700 1 -- 172.21.15.19:0/2110906183 wait complete. 2024-05-09T02:16:27.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 285.0 s, 2854.2 tps, lat 35.136 ms stddev 36.848 2024-05-09T02:16:28.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:28 smithi019 ceph-mon[30151]: pgmap v158: 97 pgs: 97 active+clean; 3.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.0 MiB/s wr, 629 op/s 2024-05-09T02:16:28.625 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T02:16:28.625 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:16:28.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:28 smithi059 ceph-mon[34781]: pgmap v158: 97 pgs: 97 active+clean; 3.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.0 MiB/s wr, 629 op/s 2024-05-09T02:16:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:28 smithi196 ceph-mon[34718]: pgmap v158: 97 pgs: 97 active+clean; 3.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.0 MiB/s wr, 629 op/s 2024-05-09T02:16:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:16:30.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:30 smithi019 ceph-mon[30151]: pgmap v159: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 993 op/s 2024-05-09T02:16:30.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:30 smithi059 ceph-mon[34781]: pgmap v159: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 993 op/s 2024-05-09T02:16:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:30 smithi196 ceph-mon[34718]: pgmap v159: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 993 op/s 2024-05-09T02:16:31.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.347+0000 7f1eaacfd700 1 -- 172.21.15.19:0/1549345754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea410c5b0 msgr2=0x7f1ea410c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:31.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.347+0000 7f1eaacfd700 1 --2- 172.21.15.19:0/1549345754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea410c5b0 0x7f1ea410c9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009990 tx=0x7f1e94009ca0 comp rx=0 tx=0).stop 2024-05-09T02:16:31.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.347+0000 7f1eaacfd700 1 -- 172.21.15.19:0/1549345754 shutdown_connections 2024-05-09T02:16:31.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.347+0000 7f1eaacfd700 1 --2- 172.21.15.19:0/1549345754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea4106ec0 0x7f1ea4113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:31.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.347+0000 7f1eaacfd700 1 --2- 172.21.15.19:0/1549345754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea410c5b0 0x7f1ea410c9c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:31.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.347+0000 7f1eaacfd700 1 --2- 172.21.15.19:0/1549345754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea4106570 0x7f1ea4106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:31.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.347+0000 7f1eaacfd700 1 -- 172.21.15.19:0/1549345754 >> 172.21.15.19:0/1549345754 conn(0x7f1ea4102050 msgr2=0x7f1ea41044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:31.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.347+0000 7f1eaacfd700 1 -- 172.21.15.19:0/1549345754 shutdown_connections 2024-05-09T02:16:31.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.347+0000 7f1eaacfd700 1 -- 172.21.15.19:0/1549345754 wait complete. 2024-05-09T02:16:31.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.348+0000 7f1eaacfd700 1 Processor -- start 2024-05-09T02:16:31.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.348+0000 7f1eaacfd700 1 -- start start 2024-05-09T02:16:31.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.348+0000 7f1eaacfd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea4106570 0x7f1ea4111f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:31.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.349+0000 7f1eaacfd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea4106ec0 0x7f1ea41124a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:31.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.349+0000 7f1eaacfd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea410c5b0 0x7f1ea41a6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:31.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.349+0000 7f1eaacfd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea4078a00 con 0x7f1ea4106ec0 2024-05-09T02:16:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.349+0000 7f1ea8a99700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea4106570 0x7f1ea4111f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.349+0000 7f1eaacfd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea4078880 con 0x7f1ea410c5b0 2024-05-09T02:16:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.349+0000 7f1ea8a99700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea4106570 0x7f1ea4111f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50652/0 (socket says 172.21.15.19:50652) 2024-05-09T02:16:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.349+0000 7f1ea8a99700 1 -- 172.21.15.19:0/2906489060 learned_addr learned my addr 172.21.15.19:0/2906489060 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.349+0000 7f1eaacfd700 1 -- 172.21.15.19:0/2906489060 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea4078b80 con 0x7f1ea4106570 2024-05-09T02:16:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.349+0000 7f1ea929a700 1 --2- 172.21.15.19:0/2906489060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea410c5b0 0x7f1ea41a6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.349+0000 7f1ea8a99700 1 -- 172.21.15.19:0/2906489060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea410c5b0 msgr2=0x7f1ea41a6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.349+0000 7f1ea8a99700 1 --2- 172.21.15.19:0/2906489060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea410c5b0 0x7f1ea41a6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.349+0000 7f1ea8a99700 1 -- 172.21.15.19:0/2906489060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea4106ec0 msgr2=0x7f1ea41124a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.349+0000 7f1ea8a99700 1 --2- 172.21.15.19:0/2906489060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea4106ec0 0x7f1ea41124a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.349+0000 7f1ea8a99700 1 -- 172.21.15.19:0/2906489060 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea41a75d0 con 0x7f1ea4106570 2024-05-09T02:16:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.349+0000 7f1ea8a99700 1 --2- 172.21.15.19:0/2906489060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea4106570 0x7f1ea4111f40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1e94005b40 tx=0x7f1e94009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.350+0000 7f1e99ffb700 1 -- 172.21.15.19:0/2906489060 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e94027070 con 0x7f1ea4106570 2024-05-09T02:16:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.350+0000 7f1e99ffb700 1 -- 172.21.15.19:0/2906489060 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e94003fb0 con 0x7f1ea4106570 2024-05-09T02:16:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.350+0000 7f1e99ffb700 1 -- 172.21.15.19:0/2906489060 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e94022c00 con 0x7f1ea4106570 2024-05-09T02:16:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.350+0000 7f1eaacfd700 1 -- 172.21.15.19:0/2906489060 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea41a7860 con 0x7f1ea4106570 2024-05-09T02:16:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.350+0000 7f1eaacfd700 1 -- 172.21.15.19:0/2906489060 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea41a7c30 con 0x7f1ea4106570 2024-05-09T02:16:31.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.353+0000 7f1eaacfd700 1 -- 172.21.15.19:0/2906489060 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea40512c0 con 0x7f1ea4106570 2024-05-09T02:16:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.353+0000 7f1e99ffb700 1 -- 172.21.15.19:0/2906489060 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1e9402a070 con 0x7f1ea4106570 2024-05-09T02:16:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.354+0000 7f1e99ffb700 1 --2- 172.21.15.19:0/2906489060 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1e8c0b2a70 0x7f1e8c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.354+0000 7f1e9bfff700 1 --2- 172.21.15.19:0/2906489060 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1e8c0b2a70 0x7f1e8c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.354+0000 7f1e99ffb700 1 -- 172.21.15.19:0/2906489060 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f1e940e3540 con 0x7f1ea4106570 2024-05-09T02:16:31.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.355+0000 7f1e9bfff700 1 --2- 172.21.15.19:0/2906489060 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1e8c0b2a70 0x7f1e8c0b4f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ea41057a0 tx=0x7f1ea0006cd0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:31.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.360+0000 7f1e99ffb700 1 -- 172.21.15.19:0/2906489060 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e94013070 con 0x7f1ea4106570 2024-05-09T02:16:31.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.753+0000 7f1eaacfd700 1 -- 172.21.15.19:0/2906489060 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ea402e5f0 con 0x7f1ea4106570 2024-05-09T02:16:31.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.757+0000 7f1e99ffb700 1 -- 172.21.15.19:0/2906489060 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11925 (secure 0 0 0) 0x7f1e940aafb0 con 0x7f1ea4106570 2024-05-09T02:16:31.759 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:16:31.760 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:15:41.825211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24463,24484,24473,24455,24457]},"id":1}]} 2024-05-09T02:16:31.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.762+0000 7f1eaacfd700 1 -- 172.21.15.19:0/2906489060 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1e8c0b2a70 msgr2=0x7f1e8c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:31.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.762+0000 7f1eaacfd700 1 --2- 172.21.15.19:0/2906489060 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1e8c0b2a70 0x7f1e8c0b4f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ea41057a0 tx=0x7f1ea0006cd0 comp rx=0 tx=0).stop 2024-05-09T02:16:31.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.762+0000 7f1eaacfd700 1 -- 172.21.15.19:0/2906489060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea4106570 msgr2=0x7f1ea4111f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:31.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.762+0000 7f1eaacfd700 1 --2- 172.21.15.19:0/2906489060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea4106570 0x7f1ea4111f40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1e94005b40 tx=0x7f1e94009ee0 comp rx=0 tx=0).stop 2024-05-09T02:16:31.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.762+0000 7f1eaacfd700 1 -- 172.21.15.19:0/2906489060 shutdown_connections 2024-05-09T02:16:31.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.763+0000 7f1eaacfd700 1 --2- 172.21.15.19:0/2906489060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea410c5b0 0x7f1ea41a6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:31.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.763+0000 7f1eaacfd700 1 --2- 172.21.15.19:0/2906489060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ea4106570 0x7f1ea4111f40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:31.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.763+0000 7f1eaacfd700 1 --2- 172.21.15.19:0/2906489060 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1e8c0b2a70 0x7f1e8c0b4f80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:31.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.763+0000 7f1eaacfd700 1 --2- 172.21.15.19:0/2906489060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea4106ec0 0x7f1ea41124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:31.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.763+0000 7f1eaacfd700 1 -- 172.21.15.19:0/2906489060 >> 172.21.15.19:0/2906489060 conn(0x7f1ea4102050 msgr2=0x7f1ea41039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:31.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.763+0000 7f1eaacfd700 1 -- 172.21.15.19:0/2906489060 shutdown_connections 2024-05-09T02:16:31.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:31.763+0000 7f1eaacfd700 1 -- 172.21.15.19:0/2906489060 wait complete. 2024-05-09T02:16:31.766 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 41 2024-05-09T02:16:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 290.0 s, 3400.2 tps, lat 29.512 ms stddev 31.666 2024-05-09T02:16:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:32 smithi059 ceph-mon[34781]: pgmap v160: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.5 MiB/s wr, 675 op/s 2024-05-09T02:16:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:32 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2906489060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:16:32.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:32 smithi019 ceph-mon[30151]: pgmap v160: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.5 MiB/s wr, 675 op/s 2024-05-09T02:16:32.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:32 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2906489060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:16:32.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:32 smithi196 ceph-mon[34718]: pgmap v160: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.5 MiB/s wr, 675 op/s 2024-05-09T02:16:32.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:32 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2906489060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:16:32.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T02:16:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:16:34.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:34 smithi019 ceph-mon[30151]: pgmap v161: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.6 MiB/s wr, 679 op/s 2024-05-09T02:16:34.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:34 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:16:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:34 smithi059 ceph-mon[34781]: pgmap v161: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.6 MiB/s wr, 679 op/s 2024-05-09T02:16:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:34 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:16:34.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:34 smithi196 ceph-mon[34718]: pgmap v161: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.6 MiB/s wr, 679 op/s 2024-05-09T02:16:34.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:34 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:16:35.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.696+0000 7fe7f3e43700 1 -- 172.21.15.19:0/4111084648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec10b280 msgr2=0x7fe7ec1198b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:35.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.696+0000 7fe7f3e43700 1 --2- 172.21.15.19:0/4111084648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec10b280 0x7fe7ec1198b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009990 tx=0x7fe7dc009ca0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.696+0000 7fe7f3e43700 1 -- 172.21.15.19:0/4111084648 shutdown_connections 2024-05-09T02:16:35.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.696+0000 7fe7f3e43700 1 --2- 172.21.15.19:0/4111084648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec10b280 0x7fe7ec1198b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.696+0000 7fe7f3e43700 1 --2- 172.21.15.19:0/4111084648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7ec111bd0 0x7fe7ec111fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.696+0000 7fe7f3e43700 1 --2- 172.21.15.19:0/4111084648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec10a930 0x7fe7ec10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.696+0000 7fe7f3e43700 1 -- 172.21.15.19:0/4111084648 >> 172.21.15.19:0/4111084648 conn(0x7fe7ec06f4d0 msgr2=0x7fe7ec06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:35.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.697+0000 7fe7f3e43700 1 -- 172.21.15.19:0/4111084648 shutdown_connections 2024-05-09T02:16:35.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.697+0000 7fe7f3e43700 1 -- 172.21.15.19:0/4111084648 wait complete. 2024-05-09T02:16:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.698+0000 7fe7f3e43700 1 Processor -- start 2024-05-09T02:16:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.698+0000 7fe7f3e43700 1 -- start start 2024-05-09T02:16:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f3e43700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7ec10a930 0x7fe7ec10c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f3e43700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec10b280 0x7fe7ec10cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f3e43700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec111bd0 0x7fe7ec10d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f3e43700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec11cc60 con 0x7fe7ec111bd0 2024-05-09T02:16:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f3e43700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec11cae0 con 0x7fe7ec10b280 2024-05-09T02:16:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f3e43700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec11cde0 con 0x7fe7ec10a930 2024-05-09T02:16:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f23e0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec111bd0 0x7fe7ec10d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f1bdf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7ec10a930 0x7fe7ec10c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f13de700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec10b280 0x7fe7ec10cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f1bdf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7ec10a930 0x7fe7ec10c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50666/0 (socket says 172.21.15.19:50666) 2024-05-09T02:16:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/3627131947 learned_addr learned my addr 172.21.15.19:0/3627131947 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/3627131947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec10b280 msgr2=0x7fe7ec10cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f1bdf700 1 --2- 172.21.15.19:0/3627131947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec10b280 0x7fe7ec10cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/3627131947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec111bd0 msgr2=0x7fe7ec10d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f1bdf700 1 --2- 172.21.15.19:0/3627131947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec111bd0 0x7fe7ec10d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.699+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/3627131947 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec1b18b0 con 0x7fe7ec10a930 2024-05-09T02:16:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.700+0000 7fe7f1bdf700 1 --2- 172.21.15.19:0/3627131947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7ec10a930 0x7fe7ec10c680 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400efc0 tx=0x7fe7e400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.700+0000 7fe7e2ffd700 1 -- 172.21.15.19:0/3627131947 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40092e0 con 0x7fe7ec10a930 2024-05-09T02:16:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.700+0000 7fe7f3e43700 1 -- 172.21.15.19:0/3627131947 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec1b1b40 con 0x7fe7ec10a930 2024-05-09T02:16:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.701+0000 7fe7f3e43700 1 -- 172.21.15.19:0/3627131947 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec1b2080 con 0x7fe7ec10a930 2024-05-09T02:16:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.701+0000 7fe7e2ffd700 1 -- 172.21.15.19:0/3627131947 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e4003a00 con 0x7fe7ec10a930 2024-05-09T02:16:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.701+0000 7fe7e2ffd700 1 -- 172.21.15.19:0/3627131947 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40045f0 con 0x7fe7ec10a930 2024-05-09T02:16:35.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.706+0000 7fe7e2ffd700 1 -- 172.21.15.19:0/3627131947 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe7e401b030 con 0x7fe7ec10a930 2024-05-09T02:16:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.707+0000 7fe7e2ffd700 1 --2- 172.21.15.19:0/3627131947 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe7d80b2b40 0x7fe7d80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.707+0000 7fe7f13de700 1 --2- 172.21.15.19:0/3627131947 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe7d80b2b40 0x7fe7d80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.707+0000 7fe7e2ffd700 1 -- 172.21.15.19:0/3627131947 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fe7e40e2430 con 0x7fe7ec10a930 2024-05-09T02:16:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.709+0000 7fe802859b80 1 Processor -- start 2024-05-09T02:16:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.709+0000 7fe7f13de700 1 --2- 172.21.15.19:0/3627131947 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe7d80b2b40 0x7fe7d80b5050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec06fb00 tx=0x7fe7e8009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.711+0000 7fe802859b80 1 -- start start 2024-05-09T02:16:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.711+0000 7fe802859b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557014227160 0x5570142c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.711+0000 7fe802859b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701413d4a0 0x5570142c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.711+0000 7fe802859b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570142c2eb0 0x5570142c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.711+0000 7fe802859b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5570141dcff0 con 0x557014227160 2024-05-09T02:16:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.711+0000 7fe802859b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55701417b7f0 con 0x5570142c2eb0 2024-05-09T02:16:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.711+0000 7fe802859b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5570141d8fb0 con 0x55701413d4a0 2024-05-09T02:16:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.711+0000 7fe7f1bdf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557014227160 0x5570142c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.711+0000 7fe7f1bdf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557014227160 0x5570142c8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48098/0 (socket says 172.21.15.19:48098) 2024-05-09T02:16:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.711+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/1599212427 learned_addr learned my addr 172.21.15.19:0/1599212427 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.711+0000 7fe7f13de700 1 --2- 172.21.15.19:0/1599212427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701413d4a0 0x5570142c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.712+0000 7fe7f23e0700 1 --2- 172.21.15.19:0/1599212427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570142c2eb0 0x5570142c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.712+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/1599212427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701413d4a0 msgr2=0x5570142c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.712+0000 7fe7f1bdf700 1 --2- 172.21.15.19:0/1599212427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701413d4a0 0x5570142c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.712+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/1599212427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570142c2eb0 msgr2=0x5570142c32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.712+0000 7fe7f1bdf700 1 --2- 172.21.15.19:0/1599212427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570142c2eb0 0x5570142c32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.712+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/1599212427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557014173370 con 0x557014227160 2024-05-09T02:16:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.712+0000 7fe7f23e0700 1 --2- 172.21.15.19:0/1599212427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570142c2eb0 0x5570142c32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:16:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.712+0000 7fe7f1bdf700 1 --2- 172.21.15.19:0/1599212427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557014227160 0x5570142c8d40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40aa3f0 tx=0x7fe7e40cc1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8926eb24902079 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.712+0000 7fe7d7fff700 1 -- 172.21.15.19:0/1599212427 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40aa6f0 con 0x557014227160 2024-05-09T02:16:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.712+0000 7fe7d7fff700 1 -- 172.21.15.19:0/1599212427 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e40aeaa0 con 0x557014227160 2024-05-09T02:16:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.713+0000 7fe7d7fff700 1 -- 172.21.15.19:0/1599212427 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40aa6f0 con 0x557014227160 2024-05-09T02:16:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.713+0000 7fe802859b80 1 -- 172.21.15.19:0/1599212427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557014227160 msgr2=0x5570142c8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.713+0000 7fe802859b80 1 --2- 172.21.15.19:0/1599212427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557014227160 0x5570142c8d40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40aa3f0 tx=0x7fe7e40cc1a0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.713+0000 7fe802859b80 1 -- 172.21.15.19:0/1599212427 shutdown_connections 2024-05-09T02:16:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.713+0000 7fe802859b80 1 --2- 172.21.15.19:0/1599212427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570142c2eb0 0x5570142c32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.713+0000 7fe802859b80 1 --2- 172.21.15.19:0/1599212427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557014227160 0x5570142c8d40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.713+0000 7fe802859b80 1 --2- 172.21.15.19:0/1599212427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55701413d4a0 0x5570142c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.713+0000 7fe802859b80 1 -- 172.21.15.19:0/1599212427 >> 172.21.15.19:0/1599212427 conn(0x55701419d6a0 msgr2=0x5570142b5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.714+0000 7fe802859b80 1 -- 172.21.15.19:0/1599212427 shutdown_connections 2024-05-09T02:16:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.714+0000 7fe802859b80 1 -- 172.21.15.19:0/1599212427 wait complete. 2024-05-09T02:16:35.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.714+0000 7fe802859b80 1 -- start start 2024-05-09T02:16:35.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.714+0000 7fe802859b80 1 Processor -- start 2024-05-09T02:16:35.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.715+0000 7fe802859b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701413d4a0 0x5570142cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.715+0000 7fe802859b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557014227160 0x5570142ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.715+0000 7fe802859b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570142c2eb0 0x5570142cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.715+0000 7fe802859b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557014058ba0 con 0x5570142c2eb0 2024-05-09T02:16:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.715+0000 7fe802859b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557014197150 con 0x55701413d4a0 2024-05-09T02:16:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.715+0000 7fe802859b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5570142c3cd0 con 0x557014227160 2024-05-09T02:16:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.715+0000 7fe802859b80 10 client.0 _ll_register_callbacks cb 0x55701404e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.715+0000 7fe7f1bdf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701413d4a0 0x5570142cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.715+0000 7fe7f1bdf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701413d4a0 0x5570142cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:55500/0 (socket says 172.21.15.19:55500) 2024-05-09T02:16:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.715+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/1948971823 learned_addr learned my addr 172.21.15.19:0/1948971823 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.716+0000 7fe7f13de700 1 --2- 172.21.15.19:0/1948971823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557014227160 0x5570142ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.716+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/1948971823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557014227160 msgr2=0x5570142ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.716+0000 7fe7f1bdf700 1 --2- 172.21.15.19:0/1948971823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557014227160 0x5570142ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.716+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/1948971823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570142c2eb0 msgr2=0x5570142cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.716+0000 7fe7f1bdf700 1 --2- 172.21.15.19:0/1948971823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570142c2eb0 0x5570142cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.716+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/1948971823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557014058490 con 0x55701413d4a0 2024-05-09T02:16:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.716+0000 7fe7f1bdf700 1 --2- 172.21.15.19:0/1948971823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701413d4a0 0x5570142cca50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400ef90 tx=0x7fe7e40dfb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=904055d01e849da2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.716+0000 7fe7d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:16:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.716+0000 7fe7d57fa700 1 -- 172.21.15.19:0/1948971823 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40c9060 con 0x55701413d4a0 2024-05-09T02:16:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.716+0000 7fe7d57fa700 1 -- 172.21.15.19:0/1948971823 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fe7e40b3240 con 0x55701413d4a0 2024-05-09T02:16:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.717+0000 7fe802859b80 1 -- 172.21.15.19:0/1948971823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e40aeaa0 con 0x55701413d4a0 2024-05-09T02:16:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.717+0000 7fe7d57fa700 1 -- 172.21.15.19:0/1948971823 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe7e400cad0 con 0x55701413d4a0 2024-05-09T02:16:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.717+0000 7fe802859b80 10 client.25649 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.717+0000 7fe802859b80 1 -- 172.21.15.19:0/1948971823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5570142e2bb0 con 0x55701413d4a0 2024-05-09T02:16:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.717+0000 7fe7d57fa700 1 -- 172.21.15.19:0/1948971823 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e40aeaa0 con 0x55701413d4a0 2024-05-09T02:16:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.717+0000 7fe7d57fa700 1 -- 172.21.15.19:0/1948971823 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7fe7e40bfab0 con 0x55701413d4a0 2024-05-09T02:16:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.717+0000 7fe802859b80 10 client.25649 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.717+0000 7fe802859b80 10 client.25649 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:16:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.717+0000 7fe802859b80 20 client.25649 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:35.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.717+0000 7fe802859b80 1 --2- 172.21.15.19:0/1948971823 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5570142e5080 0x5570142e74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.717+0000 7fe802859b80 4 client.25649 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:16:35.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.717+0000 7fe802859b80 1 -- 172.21.15.19:0/1948971823 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570141ea9d0 con 0x5570142e5080 2024-05-09T02:16:35.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.717+0000 7fe7f13de700 1 --2- 172.21.15.19:0/1948971823 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5570142e5080 0x5570142e74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:35.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.718+0000 7fe7d57fa700 10 client.25649 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:16:35.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.718+0000 7fe7f13de700 1 --2- 172.21.15.19:0/1948971823 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5570142e5080 0x5570142e74a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ace1da4e8c0204b7 server_cookie=4f85b282e6fffb09 in_seq=0 out_seq=0 2024-05-09T02:16:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.719+0000 7fe7d57fa700 1 -- 172.21.15.19:0/1948971823 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5570141ea9d0 con 0x5570142e5080 2024-05-09T02:16:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.719+0000 7fe7d57fa700 10 client.25649 handle_command_reply: tid=0 2024-05-09T02:16:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.719+0000 7fe802859b80 1 client.25649 shutdown 2024-05-09T02:16:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.719+0000 7fe802859b80 1 -- 172.21.15.19:0/1948971823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701413d4a0 msgr2=0x5570142cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.719+0000 7fe802859b80 1 --2- 172.21.15.19:0/1948971823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701413d4a0 0x5570142cca50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400ef90 tx=0x7fe7e40dfb90 comp rx=0 tx=0).stop 2024-05-09T02:16:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.719+0000 7fe802859b80 1 -- 172.21.15.19:0/1948971823 shutdown_connections 2024-05-09T02:16:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.719+0000 7fe802859b80 1 --2- 172.21.15.19:0/1948971823 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5570142e5080 0x5570142e74a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.719+0000 7fe802859b80 1 --2- 172.21.15.19:0/1948971823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570142c2eb0 0x5570142cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.719+0000 7fe802859b80 1 --2- 172.21.15.19:0/1948971823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701413d4a0 0x5570142cca50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.719+0000 7fe802859b80 1 --2- 172.21.15.19:0/1948971823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557014227160 0x5570142ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.719+0000 7fe802859b80 1 -- 172.21.15.19:0/1948971823 >> 172.21.15.19:0/1948971823 conn(0x55701419d6a0 msgr2=0x5570142b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.719+0000 7fe7d57fa700 0 client.25649 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:16:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.720+0000 7fe802859b80 1 -- 172.21.15.19:0/1948971823 shutdown_connections 2024-05-09T02:16:35.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.720+0000 7fe802859b80 1 -- 172.21.15.19:0/1948971823 wait complete. 2024-05-09T02:16:35.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.720+0000 7fe802859b80 20 client.25649 trim_cache size 0 max 16384 2024-05-09T02:16:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.744+0000 7fe802859b80 1 Processor -- start 2024-05-09T02:16:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.745+0000 7fe802859b80 1 -- start start 2024-05-09T02:16:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.746+0000 7fe802859b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701413d4a0 0x5570142cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.746+0000 7fe802859b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557014227160 0x5570142c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.746+0000 7fe802859b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570142c2eb0 0x5570142c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.746+0000 7fe802859b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d000d600 con 0x55701413d4a0 2024-05-09T02:16:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.746+0000 7fe802859b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d000f200 con 0x5570142c2eb0 2024-05-09T02:16:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.746+0000 7fe802859b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d000dcd0 con 0x557014227160 2024-05-09T02:16:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.746+0000 7fe7f13de700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557014227160 0x5570142c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.746+0000 7fe7f13de700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557014227160 0x5570142c17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50706/0 (socket says 172.21.15.19:50706) 2024-05-09T02:16:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.746+0000 7fe7f13de700 1 -- 172.21.15.19:0/2147784949 learned_addr learned my addr 172.21.15.19:0/2147784949 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.747+0000 7fe7f13de700 1 -- 172.21.15.19:0/2147784949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570142c2eb0 msgr2=0x5570142c82a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:16:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.747+0000 7fe7f13de700 1 --2- 172.21.15.19:0/2147784949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570142c2eb0 0x5570142c82a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.747+0000 7fe7f13de700 1 -- 172.21.15.19:0/2147784949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701413d4a0 msgr2=0x5570142cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.747+0000 7fe7f13de700 1 --2- 172.21.15.19:0/2147784949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701413d4a0 0x5570142cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.747+0000 7fe7f13de700 1 -- 172.21.15.19:0/2147784949 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570142aef60 con 0x557014227160 2024-05-09T02:16:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.748+0000 7fe7f13de700 1 --2- 172.21.15.19:0/2147784949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557014227160 0x5570142c17a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x5570142b6ed0 tx=0x7fe7e8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e267b480837340b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.751+0000 7fe7d77fe700 1 -- 172.21.15.19:0/2147784949 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8004020 con 0x557014227160 2024-05-09T02:16:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.751+0000 7fe7d77fe700 1 -- 172.21.15.19:0/2147784949 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e8028340 con 0x557014227160 2024-05-09T02:16:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.751+0000 7fe7d77fe700 1 -- 172.21.15.19:0/2147784949 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80038c0 con 0x557014227160 2024-05-09T02:16:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.752+0000 7fe802859b80 1 -- 172.21.15.19:0/2147784949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557014227160 msgr2=0x5570142c17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.752+0000 7fe802859b80 1 --2- 172.21.15.19:0/2147784949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557014227160 0x5570142c17a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x5570142b6ed0 tx=0x7fe7e8017000 comp rx=0 tx=0).stop 2024-05-09T02:16:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.755+0000 7fe802859b80 1 -- 172.21.15.19:0/2147784949 shutdown_connections 2024-05-09T02:16:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.755+0000 7fe802859b80 1 --2- 172.21.15.19:0/2147784949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570142c2eb0 0x5570142c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.755+0000 7fe802859b80 1 --2- 172.21.15.19:0/2147784949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55701413d4a0 0x5570142cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.755+0000 7fe802859b80 1 --2- 172.21.15.19:0/2147784949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557014227160 0x5570142c17a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.755+0000 7fe802859b80 1 -- 172.21.15.19:0/2147784949 >> 172.21.15.19:0/2147784949 conn(0x55701419d6a0 msgr2=0x5570142c7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:35.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.762+0000 7fe802859b80 1 -- 172.21.15.19:0/2147784949 shutdown_connections 2024-05-09T02:16:35.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.764+0000 7fe802859b80 1 -- 172.21.15.19:0/2147784949 wait complete. 2024-05-09T02:16:35.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.764+0000 7fe802859b80 1 -- start start 2024-05-09T02:16:35.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.764+0000 7fe802859b80 1 Processor -- start 2024-05-09T02:16:35.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.767+0000 7fe802859b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701413d4a0 0x5570142c19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.767+0000 7fe802859b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557014227160 0x5570142bed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.767+0000 7fe802859b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570142c2eb0 0x5570142bf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.767+0000 7fe802859b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d000eb30 con 0x557014227160 2024-05-09T02:16:35.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.767+0000 7fe802859b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d000e3a0 con 0x55701413d4a0 2024-05-09T02:16:35.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.767+0000 7fe802859b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d000b5a0 con 0x5570142c2eb0 2024-05-09T02:16:35.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.767+0000 7fe802859b80 10 client.0 _ll_register_callbacks cb 0x55701404e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.768+0000 7fe7f1bdf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701413d4a0 0x5570142c19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.768+0000 7fe7f1bdf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701413d4a0 0x5570142c19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:55526/0 (socket says 172.21.15.19:55526) 2024-05-09T02:16:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.768+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/4044951590 learned_addr learned my addr 172.21.15.19:0/4044951590 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.769+0000 7fe7f23e0700 1 --2- 172.21.15.19:0/4044951590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570142c2eb0 0x5570142bf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.770+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/4044951590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570142c2eb0 msgr2=0x5570142bf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.770+0000 7fe7f1bdf700 1 --2- 172.21.15.19:0/4044951590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570142c2eb0 0x5570142bf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.770+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/4044951590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557014227160 msgr2=0x5570142bed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:35.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.770+0000 7fe7f1bdf700 1 --2- 172.21.15.19:0/4044951590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557014227160 0x5570142bed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.770+0000 7fe7f1bdf700 1 -- 172.21.15.19:0/4044951590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570142bfb20 con 0x55701413d4a0 2024-05-09T02:16:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.774+0000 7fe7f1bdf700 1 --2- 172.21.15.19:0/4044951590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701413d4a0 0x5570142c19b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40e0180 tx=0x7fe7e40aa130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe3f48f9e0d33a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.774+0000 7fe7d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:16:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.774+0000 7fe7d57fa700 1 -- 172.21.15.19:0/4044951590 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40c9050 con 0x55701413d4a0 2024-05-09T02:16:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.774+0000 7fe7d57fa700 1 -- 172.21.15.19:0/4044951590 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fe7e40bfab0 con 0x55701413d4a0 2024-05-09T02:16:35.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.775+0000 7fe802859b80 1 -- 172.21.15.19:0/4044951590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e8028340 con 0x55701413d4a0 2024-05-09T02:16:35.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.778+0000 7fe7d57fa700 1 -- 172.21.15.19:0/4044951590 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe7e40a65b0 con 0x55701413d4a0 2024-05-09T02:16:35.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.781+0000 7fe802859b80 10 client.25652 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:35.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.781+0000 7fe802859b80 1 -- 172.21.15.19:0/4044951590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5570142bfb20 con 0x55701413d4a0 2024-05-09T02:16:35.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.785+0000 7fe7d57fa700 1 -- 172.21.15.19:0/4044951590 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e8028340 con 0x55701413d4a0 2024-05-09T02:16:35.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.785+0000 7fe7d57fa700 1 -- 172.21.15.19:0/4044951590 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7fe7e40a6060 con 0x55701413d4a0 2024-05-09T02:16:35.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.785+0000 7fe802859b80 10 client.25652 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:35.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.785+0000 7fe802859b80 10 client.25652 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:16:35.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.785+0000 7fe802859b80 20 client.25652 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:35.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.785+0000 7fe802859b80 1 --2- 172.21.15.19:0/4044951590 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55701434f930 0x557014351d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:35.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.785+0000 7fe802859b80 4 client.25652 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:16:35.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.785+0000 7fe802859b80 1 -- 172.21.15.19:0/4044951590 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570141e4a70 con 0x55701434f930 2024-05-09T02:16:35.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.788+0000 7fe7f13de700 1 --2- 172.21.15.19:0/4044951590 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55701434f930 0x557014351d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:35.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.789+0000 7fe7d57fa700 10 client.25652 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:16:35.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.789+0000 7fe7f13de700 1 --2- 172.21.15.19:0/4044951590 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55701434f930 0x557014351d50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=def0fc5ac2cef87a server_cookie=5e54a48e14ade40d in_seq=0 out_seq=0 2024-05-09T02:16:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.794+0000 7fe7d57fa700 1 -- 172.21.15.19:0/4044951590 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570141e4a70 con 0x55701434f930 2024-05-09T02:16:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.794+0000 7fe7d57fa700 10 client.25652 handle_command_reply: tid=0 2024-05-09T02:16:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.794+0000 7fe802859b80 1 client.25652 shutdown 2024-05-09T02:16:35.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.795+0000 7fe802859b80 1 -- 172.21.15.19:0/4044951590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701413d4a0 msgr2=0x5570142c19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:35.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.795+0000 7fe802859b80 1 --2- 172.21.15.19:0/4044951590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701413d4a0 0x5570142c19b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40e0180 tx=0x7fe7e40aa130 comp rx=0 tx=0).stop 2024-05-09T02:16:35.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.798+0000 7fe802859b80 1 -- 172.21.15.19:0/4044951590 shutdown_connections 2024-05-09T02:16:35.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.799+0000 7fe802859b80 1 --2- 172.21.15.19:0/4044951590 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55701434f930 0x557014351d50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.799+0000 7fe802859b80 1 --2- 172.21.15.19:0/4044951590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570142c2eb0 0x5570142bf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.799+0000 7fe802859b80 1 --2- 172.21.15.19:0/4044951590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55701413d4a0 0x5570142c19b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.799+0000 7fe802859b80 1 --2- 172.21.15.19:0/4044951590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557014227160 0x5570142bed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.799+0000 7fe802859b80 1 -- 172.21.15.19:0/4044951590 >> 172.21.15.19:0/4044951590 conn(0x55701419d6a0 msgr2=0x5570142b9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:35.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.799+0000 7fe7d57fa700 0 client.25652 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:16:35.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.799+0000 7fe802859b80 1 -- 172.21.15.19:0/4044951590 shutdown_connections 2024-05-09T02:16:35.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.799+0000 7fe802859b80 1 -- 172.21.15.19:0/4044951590 wait complete. 2024-05-09T02:16:35.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.799+0000 7fe802859b80 20 client.25652 trim_cache size 0 max 16384 2024-05-09T02:16:35.803 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:16:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.800+0000 7fe7f3e43700 1 -- 172.21.15.19:0/3627131947 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe7d80b2b40 msgr2=0x7fe7d80b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.800+0000 7fe7f3e43700 1 --2- 172.21.15.19:0/3627131947 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe7d80b2b40 0x7fe7d80b5050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec06fb00 tx=0x7fe7e8009260 comp rx=0 tx=0).stop 2024-05-09T02:16:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.800+0000 7fe7f3e43700 1 -- 172.21.15.19:0/3627131947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7ec10a930 msgr2=0x7fe7ec10c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.800+0000 7fe7f3e43700 1 --2- 172.21.15.19:0/3627131947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7ec10a930 0x7fe7ec10c680 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400efc0 tx=0x7fe7e400c3a0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.801+0000 7fe7f3e43700 1 -- 172.21.15.19:0/3627131947 shutdown_connections 2024-05-09T02:16:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.801+0000 7fe7f3e43700 1 --2- 172.21.15.19:0/3627131947 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe7d80b2b40 0x7fe7d80b5050 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.801+0000 7fe7f3e43700 1 --2- 172.21.15.19:0/3627131947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec111bd0 0x7fe7ec10d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.801+0000 7fe7f3e43700 1 --2- 172.21.15.19:0/3627131947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7ec10a930 0x7fe7ec10c680 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.801+0000 7fe7f3e43700 1 --2- 172.21.15.19:0/3627131947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec10b280 0x7fe7ec10cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.801+0000 7fe7f3e43700 1 -- 172.21.15.19:0/3627131947 >> 172.21.15.19:0/3627131947 conn(0x7fe7ec06f4d0 msgr2=0x7fe7ec110760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.801+0000 7fe7f3e43700 1 -- 172.21.15.19:0/3627131947 shutdown_connections 2024-05-09T02:16:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:35.801+0000 7fe7f3e43700 1 -- 172.21.15.19:0/3627131947 wait complete. 2024-05-09T02:16:36.488 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:36 smithi019 ceph-mon[30151]: pgmap v162: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 981 op/s 2024-05-09T02:16:36.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:36 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:16:36.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:36 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:16:36.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:36 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:16:36.533 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T02:16:36.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:36 smithi059 ceph-mon[34781]: pgmap v162: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 981 op/s 2024-05-09T02:16:36.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:36 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:16:36.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:36 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:16:36.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:36 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:16:36.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:36 smithi196 ceph-mon[34718]: pgmap v162: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 981 op/s 2024-05-09T02:16:36.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:36 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:16:36.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:36 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:16:36.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:36 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:16:36.905 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:16:37.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 295.0 s, 3071.2 tps, lat 32.441 ms stddev 35.577 2024-05-09T02:16:38.636 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:38 smithi019 ceph-mon[30151]: pgmap v163: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.5 MiB/s wr, 684 op/s 2024-05-09T02:16:38.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:38 smithi059 ceph-mon[34781]: pgmap v163: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.5 MiB/s wr, 684 op/s 2024-05-09T02:16:38.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:38 smithi196 ceph-mon[34718]: pgmap v163: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.5 MiB/s wr, 684 op/s 2024-05-09T02:16:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.810+0000 7f4531044700 1 -- 172.21.15.19:0/1536024846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f452c109860 msgr2=0x7f452c1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.810+0000 7f4531044700 1 --2- 172.21.15.19:0/1536024846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f452c109860 0x7f452c1111c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f452400b0d0 tx=0x7f452400b3e0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.810+0000 7f4531044700 1 -- 172.21.15.19:0/1536024846 shutdown_connections 2024-05-09T02:16:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.810+0000 7f4531044700 1 --2- 172.21.15.19:0/1536024846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f452c109860 0x7f452c1111c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.810+0000 7f4531044700 1 --2- 172.21.15.19:0/1536024846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f452c1045e0 0x7f452c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.810+0000 7f4531044700 1 --2- 172.21.15.19:0/1536024846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f452c108f10 0x7f452c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.810+0000 7f4531044700 1 -- 172.21.15.19:0/1536024846 >> 172.21.15.19:0/1536024846 conn(0x7f452c0ffe70 msgr2=0x7f452c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.810+0000 7f4531044700 1 -- 172.21.15.19:0/1536024846 shutdown_connections 2024-05-09T02:16:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.810+0000 7f4531044700 1 -- 172.21.15.19:0/1536024846 wait complete. 2024-05-09T02:16:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.811+0000 7f4531044700 1 Processor -- start 2024-05-09T02:16:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.812+0000 7f4531044700 1 -- start start 2024-05-09T02:16:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.812+0000 7f4531044700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f452c1045e0 0x7f452c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.812+0000 7f4531044700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f452c108f10 0x7f452c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.812+0000 7f4531044700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f452c109860 0x7f452c1a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.812+0000 7f4531044700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f452c114490 con 0x7f452c108f10 2024-05-09T02:16:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.812+0000 7f4531044700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f452c114310 con 0x7f452c1045e0 2024-05-09T02:16:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.812+0000 7f4531044700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f452c114610 con 0x7f452c109860 2024-05-09T02:16:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.813+0000 7f452b59e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f452c109860 0x7f452c1a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.813+0000 7f452b59e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f452c109860 0x7f452c1a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36084/0 (socket says 172.21.15.19:36084) 2024-05-09T02:16:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.813+0000 7f452b59e700 1 -- 172.21.15.19:0/1897043432 learned_addr learned my addr 172.21.15.19:0/1897043432 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.813+0000 7f452ad9d700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f452c1045e0 0x7f452c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.813+0000 7f452a59c700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f452c108f10 0x7f452c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.813+0000 7f452ad9d700 1 -- 172.21.15.19:0/1897043432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f452c109860 msgr2=0x7f452c1a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.813+0000 7f452ad9d700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f452c109860 0x7f452c1a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.813+0000 7f452ad9d700 1 -- 172.21.15.19:0/1897043432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f452c108f10 msgr2=0x7f452c199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.813+0000 7f452ad9d700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f452c108f10 0x7f452c199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.813+0000 7f452ad9d700 1 -- 172.21.15.19:0/1897043432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c19a750 con 0x7f452c1045e0 2024-05-09T02:16:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.813+0000 7f452b59e700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f452c109860 0x7f452c1a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:16:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.813+0000 7f452a59c700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f452c108f10 0x7f452c199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:16:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.813+0000 7f452ad9d700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f452c1045e0 0x7f452c199960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f451c00e9f0 tx=0x7f451c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.814+0000 7f4513fff700 1 -- 172.21.15.19:0/1897043432 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c00cb70 con 0x7f452c1045e0 2024-05-09T02:16:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.814+0000 7f4513fff700 1 -- 172.21.15.19:0/1897043432 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f451c0044d0 con 0x7f452c1045e0 2024-05-09T02:16:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.814+0000 7f4513fff700 1 -- 172.21.15.19:0/1897043432 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c003e00 con 0x7f452c1045e0 2024-05-09T02:16:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.814+0000 7f4531044700 1 -- 172.21.15.19:0/1897043432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452c1a1020 con 0x7f452c1045e0 2024-05-09T02:16:38.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.814+0000 7f4531044700 1 -- 172.21.15.19:0/1897043432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452c1a1510 con 0x7f452c1045e0 2024-05-09T02:16:38.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.818+0000 7f4513fff700 1 -- 172.21.15.19:0/1897043432 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f451c003fa0 con 0x7f452c1045e0 2024-05-09T02:16:38.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.819+0000 7f453fa5ab80 1 Processor -- start 2024-05-09T02:16:38.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.819+0000 7f453fa5ab80 1 -- start start 2024-05-09T02:16:38.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.820+0000 7f453fa5ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636736a6160 0x56367373f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.820+0000 7f453fa5ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636735bc4a0 0x56367373fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.820+0000 7f453fa5ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636737396a0 0x563673739ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.821+0000 7f452ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636735bc4a0 0x56367373fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.821+0000 7f452ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636735bc4a0 0x56367373fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51540/0 (socket says 172.21.15.19:51540) 2024-05-09T02:16:38.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.821+0000 7f452ad9d700 1 -- 172.21.15.19:0/1597441539 learned_addr learned my addr 172.21.15.19:0/1597441539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:38.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.821+0000 7f4513fff700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45140b2a70 0x7f45140b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.821+0000 7f452b59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636736a6160 0x56367373f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.821+0000 7f452a59c700 1 --2- 172.21.15.19:0/1597441539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636737396a0 0x563673739ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.821+0000 7f4513fff700 1 -- 172.21.15.19:0/1897043432 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f451c0e2720 con 0x7f452c1045e0 2024-05-09T02:16:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.821+0000 7f453fa5ab80 1 -- 172.21.15.19:0/1597441539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56367365bff0 con 0x5636735bc4a0 2024-05-09T02:16:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.821+0000 7f453fa5ab80 1 -- 172.21.15.19:0/1597441539 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5636735fa7f0 con 0x5636736a6160 2024-05-09T02:16:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.821+0000 7f453fa5ab80 1 -- 172.21.15.19:0/1597441539 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563673657fb0 con 0x5636737396a0 2024-05-09T02:16:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.821+0000 7f452b59e700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45140b2a70 0x7f45140b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.822+0000 7f452b59e700 1 -- 172.21.15.19:0/1597441539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636737396a0 msgr2=0x563673739ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.822+0000 7f452b59e700 1 --2- 172.21.15.19:0/1597441539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636737396a0 0x563673739ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.822+0000 7f452b59e700 1 -- 172.21.15.19:0/1597441539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636735bc4a0 msgr2=0x56367373fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.822+0000 7f452b59e700 1 --2- 172.21.15.19:0/1597441539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636735bc4a0 0x56367373fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.822+0000 7f452b59e700 1 -- 172.21.15.19:0/1597441539 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636735f2370 con 0x5636736a6160 2024-05-09T02:16:38.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.822+0000 7f452b59e700 1 --2- 172.21.15.19:0/1597441539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636736a6160 0x56367373f4f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4524006070 tx=0x7f4524004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48019e2c24698d2d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:38.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.822+0000 7f452b59e700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45140b2a70 0x7f45140b4f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4524027000 tx=0x7f4524029000 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:38.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.822+0000 7f45117fa700 1 -- 172.21.15.19:0/1597441539 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45240077c0 con 0x5636736a6160 2024-05-09T02:16:38.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.822+0000 7f45117fa700 1 -- 172.21.15.19:0/1597441539 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45240060a0 con 0x5636736a6160 2024-05-09T02:16:38.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.822+0000 7f45117fa700 1 -- 172.21.15.19:0/1597441539 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452401dd80 con 0x5636736a6160 2024-05-09T02:16:38.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.823+0000 7f453fa5ab80 1 -- 172.21.15.19:0/1597441539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636736a6160 msgr2=0x56367373f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:38.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.823+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/1597441539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636736a6160 0x56367373f4f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4524006070 tx=0x7f4524004a30 comp rx=0 tx=0).stop 2024-05-09T02:16:38.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.823+0000 7f453fa5ab80 1 -- 172.21.15.19:0/1597441539 shutdown_connections 2024-05-09T02:16:38.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.823+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/1597441539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636737396a0 0x563673739ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.823+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/1597441539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636736a6160 0x56367373f4f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.823+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/1597441539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636735bc4a0 0x56367373fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.824+0000 7f453fa5ab80 1 -- 172.21.15.19:0/1597441539 >> 172.21.15.19:0/1597441539 conn(0x56367361c6a0 msgr2=0x563673734400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:38.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.824+0000 7f453fa5ab80 1 -- 172.21.15.19:0/1597441539 shutdown_connections 2024-05-09T02:16:38.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.825+0000 7f453fa5ab80 1 -- 172.21.15.19:0/1597441539 wait complete. 2024-05-09T02:16:38.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.825+0000 7f453fa5ab80 1 -- start start 2024-05-09T02:16:38.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.825+0000 7f453fa5ab80 1 Processor -- start 2024-05-09T02:16:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f453fa5ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 0x563673743230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f453fa5ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636736a6160 0x563673743770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f453fa5ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636737396a0 0x563673743cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f453fa5ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5636734d7ba0 con 0x5636737396a0 2024-05-09T02:16:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f453fa5ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563673616150 con 0x5636735bc4a0 2024-05-09T02:16:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f453fa5ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56367373a4e0 con 0x5636736a6160 2024-05-09T02:16:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f453fa5ab80 10 client.0 _ll_register_callbacks cb 0x5636734cd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f452a59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636736a6160 0x563673743770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f452b59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636737396a0 0x563673743cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f452a59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636736a6160 0x563673743770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36102/0 (socket says 172.21.15.19:36102) 2024-05-09T02:16:38.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f452b59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636737396a0 0x563673743cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51556/0 (socket says 172.21.15.19:51556) 2024-05-09T02:16:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f452a59c700 1 -- 172.21.15.19:0/3214801861 learned_addr learned my addr 172.21.15.19:0/3214801861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f452ad9d700 1 --2- 172.21.15.19:0/3214801861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 0x563673743230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f452a59c700 1 -- 172.21.15.19:0/3214801861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 msgr2=0x563673743230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f452a59c700 1 --2- 172.21.15.19:0/3214801861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 0x563673743230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f452a59c700 1 -- 172.21.15.19:0/3214801861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636737396a0 msgr2=0x563673743cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f452a59c700 1 --2- 172.21.15.19:0/3214801861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636737396a0 0x563673743cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.826+0000 7f452a59c700 1 -- 172.21.15.19:0/3214801861 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636734d7490 con 0x5636736a6160 2024-05-09T02:16:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.827+0000 7f452a59c700 1 --2- 172.21.15.19:0/3214801861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636736a6160 0x563673743770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4520009eb0 tx=0x7f4520012730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=489f9b422a96f82b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.827+0000 7f450eff5700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:16:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.827+0000 7f450eff5700 1 -- 172.21.15.19:0/3214801861 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520004220 con 0x5636736a6160 2024-05-09T02:16:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.827+0000 7f450eff5700 1 -- 172.21.15.19:0/3214801861 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f45200043c0 con 0x5636736a6160 2024-05-09T02:16:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.827+0000 7f453fa5ab80 1 -- 172.21.15.19:0/3214801861 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45240060a0 con 0x5636736a6160 2024-05-09T02:16:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.829+0000 7f450eff5700 1 -- 172.21.15.19:0/3214801861 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4520004830 con 0x5636736a6160 2024-05-09T02:16:38.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.829+0000 7f453fa5ab80 10 client.25723 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:38.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.829+0000 7f453fa5ab80 1 -- 172.21.15.19:0/3214801861 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563673750c00 con 0x5636736a6160 2024-05-09T02:16:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.830+0000 7f450eff5700 1 -- 172.21.15.19:0/3214801861 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45240060a0 con 0x5636736a6160 2024-05-09T02:16:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.830+0000 7f450eff5700 1 -- 172.21.15.19:0/3214801861 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f4520019030 con 0x5636736a6160 2024-05-09T02:16:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.830+0000 7f453fa5ab80 10 client.25723 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.830+0000 7f453fa5ab80 10 client.25723 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:16:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.830+0000 7f453fa5ab80 20 client.25723 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.830+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/3214801861 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563673753290 0x5636737556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.830+0000 7f453fa5ab80 4 client.25723 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:16:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.831+0000 7f453fa5ab80 1 -- 172.21.15.19:0/3214801861 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636736699d0 con 0x563673753290 2024-05-09T02:16:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.831+0000 7f452ad9d700 1 --2- 172.21.15.19:0/3214801861 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563673753290 0x5636737556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.831+0000 7f450eff5700 10 client.25723 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:16:38.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.831+0000 7f452ad9d700 1 --2- 172.21.15.19:0/3214801861 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563673753290 0x5636737556e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15028b095fe26473 server_cookie=4428bb430b9ab6e1 in_seq=0 out_seq=0 2024-05-09T02:16:38.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.832+0000 7f450eff5700 1 -- 172.21.15.19:0/3214801861 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5636736699d0 con 0x563673753290 2024-05-09T02:16:38.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.832+0000 7f450eff5700 10 client.25723 handle_command_reply: tid=0 2024-05-09T02:16:38.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.832+0000 7f453fa5ab80 1 client.25723 shutdown 2024-05-09T02:16:38.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.832+0000 7f453fa5ab80 1 -- 172.21.15.19:0/3214801861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636736a6160 msgr2=0x563673743770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:38.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.832+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/3214801861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636736a6160 0x563673743770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4520009eb0 tx=0x7f4520012730 comp rx=0 tx=0).stop 2024-05-09T02:16:38.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.832+0000 7f453fa5ab80 1 -- 172.21.15.19:0/3214801861 shutdown_connections 2024-05-09T02:16:38.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.832+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/3214801861 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563673753290 0x5636737556e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.832+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/3214801861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636737396a0 0x563673743cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.832+0000 7f450eff5700 0 client.25723 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:16:38.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.832+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/3214801861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 0x563673743230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.832+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/3214801861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636736a6160 0x563673743770 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.832+0000 7f453fa5ab80 1 -- 172.21.15.19:0/3214801861 >> 172.21.15.19:0/3214801861 conn(0x56367361c6a0 msgr2=0x5636737341c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:38.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.832+0000 7f453fa5ab80 1 -- 172.21.15.19:0/3214801861 shutdown_connections 2024-05-09T02:16:38.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.832+0000 7f453fa5ab80 1 -- 172.21.15.19:0/3214801861 wait complete. 2024-05-09T02:16:38.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.832+0000 7f453fa5ab80 20 client.25723 trim_cache size 0 max 16384 2024-05-09T02:16:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.854+0000 7f453fa5ab80 1 Processor -- start 2024-05-09T02:16:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.854+0000 7f453fa5ab80 1 -- start start 2024-05-09T02:16:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.854+0000 7f453fa5ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 0x563673743af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.854+0000 7f453fa5ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636736a6160 0x56367374ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.854+0000 7f453fa5ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636737396a0 0x56367374f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.854+0000 7f452a59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636736a6160 0x56367374ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.854+0000 7f453fa5ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1bdc60 con 0x5636737396a0 2024-05-09T02:16:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.854+0000 7f453fa5ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1bf860 con 0x5636735bc4a0 2024-05-09T02:16:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.854+0000 7f452b59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636737396a0 0x56367374f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.854+0000 7f453fa5ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1be330 con 0x5636736a6160 2024-05-09T02:16:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.854+0000 7f452a59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636736a6160 0x56367374ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36106/0 (socket says 172.21.15.19:36106) 2024-05-09T02:16:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.854+0000 7f452a59c700 1 -- 172.21.15.19:0/4143043244 learned_addr learned my addr 172.21.15.19:0/4143043244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.854+0000 7f452ad9d700 1 --2- 172.21.15.19:0/4143043244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 0x563673743af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.855+0000 7f452a59c700 1 -- 172.21.15.19:0/4143043244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 msgr2=0x563673743af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.855+0000 7f452a59c700 1 --2- 172.21.15.19:0/4143043244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 0x563673743af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.855+0000 7f452a59c700 1 -- 172.21.15.19:0/4143043244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636737396a0 msgr2=0x56367374f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.855+0000 7f452a59c700 1 --2- 172.21.15.19:0/4143043244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636737396a0 0x56367374f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.855+0000 7f452a59c700 1 -- 172.21.15.19:0/4143043244 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56367373f050 con 0x5636736a6160 2024-05-09T02:16:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.855+0000 7f452a59c700 1 --2- 172.21.15.19:0/4143043244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636736a6160 0x56367374ef20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4520006010 tx=0x7f4520012b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca6802ad3b72ad1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.855+0000 7f4510ff9700 1 -- 172.21.15.19:0/4143043244 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520031070 con 0x5636736a6160 2024-05-09T02:16:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.855+0000 7f4510ff9700 1 -- 172.21.15.19:0/4143043244 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4520010c90 con 0x5636736a6160 2024-05-09T02:16:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.855+0000 7f4510ff9700 1 -- 172.21.15.19:0/4143043244 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520028040 con 0x5636736a6160 2024-05-09T02:16:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.856+0000 7f453fa5ab80 1 -- 172.21.15.19:0/4143043244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636736a6160 msgr2=0x56367374ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.856+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/4143043244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636736a6160 0x56367374ef20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4520006010 tx=0x7f4520012b80 comp rx=0 tx=0).stop 2024-05-09T02:16:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.856+0000 7f453fa5ab80 1 -- 172.21.15.19:0/4143043244 shutdown_connections 2024-05-09T02:16:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.856+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/4143043244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636737396a0 0x56367374f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.856+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/4143043244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 0x563673743af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.856+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/4143043244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636736a6160 0x56367374ef20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.856+0000 7f453fa5ab80 1 -- 172.21.15.19:0/4143043244 >> 172.21.15.19:0/4143043244 conn(0x56367361c6a0 msgr2=0x563673741b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.856+0000 7f453fa5ab80 1 -- 172.21.15.19:0/4143043244 shutdown_connections 2024-05-09T02:16:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.856+0000 7f453fa5ab80 1 -- 172.21.15.19:0/4143043244 wait complete. 2024-05-09T02:16:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.856+0000 7f453fa5ab80 1 -- start start 2024-05-09T02:16:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.857+0000 7f453fa5ab80 1 Processor -- start 2024-05-09T02:16:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.857+0000 7f453fa5ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 0x56367374b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.857+0000 7f453fa5ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636736a6160 0x56367374b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.857+0000 7f453fa5ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636737396a0 0x56367373f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.857+0000 7f453fa5ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1bf190 con 0x5636736a6160 2024-05-09T02:16:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.857+0000 7f452a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636736a6160 0x56367374b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.857+0000 7f452a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636736a6160 0x56367374b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51572/0 (socket says 172.21.15.19:51572) 2024-05-09T02:16:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.857+0000 7f452a59c700 1 -- 172.21.15.19:0/2016309172 learned_addr learned my addr 172.21.15.19:0/2016309172 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.857+0000 7f453fa5ab80 1 -- 172.21.15.19:0/2016309172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1bea00 con 0x5636735bc4a0 2024-05-09T02:16:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.857+0000 7f453fa5ab80 1 -- 172.21.15.19:0/2016309172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1bbc00 con 0x5636737396a0 2024-05-09T02:16:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.857+0000 7f452ad9d700 1 --2- 172.21.15.19:0/2016309172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 0x56367374b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.857+0000 7f453fa5ab80 10 client.0 _ll_register_callbacks cb 0x5636734cd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:38.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.857+0000 7f452b59e700 1 --2- 172.21.15.19:0/2016309172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636737396a0 0x56367373f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.858+0000 7f452ad9d700 1 -- 172.21.15.19:0/2016309172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636737396a0 msgr2=0x56367373f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:38.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.858+0000 7f452ad9d700 1 --2- 172.21.15.19:0/2016309172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636737396a0 0x56367373f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.858+0000 7f452ad9d700 1 -- 172.21.15.19:0/2016309172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636736a6160 msgr2=0x56367374b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:38.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.858+0000 7f452ad9d700 1 --2- 172.21.15.19:0/2016309172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636736a6160 0x56367374b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.858+0000 7f452ad9d700 1 -- 172.21.15.19:0/2016309172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56367373fc70 con 0x5636735bc4a0 2024-05-09T02:16:38.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.858+0000 7f452ad9d700 1 --2- 172.21.15.19:0/2016309172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 0x56367374b2a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f451c0dfed0 tx=0x7f451c003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=799e9b83d16f4527 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:38.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.858+0000 7f450eff5700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:16:38.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.858+0000 7f450eff5700 1 -- 172.21.15.19:0/2016309172 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c0a9510 con 0x5636735bc4a0 2024-05-09T02:16:38.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.858+0000 7f450eff5700 1 -- 172.21.15.19:0/2016309172 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f451c0a5c90 con 0x5636735bc4a0 2024-05-09T02:16:38.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.859+0000 7f453fa5ab80 1 -- 172.21.15.19:0/2016309172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4520010c90 con 0x5636735bc4a0 2024-05-09T02:16:38.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.859+0000 7f450eff5700 1 -- 172.21.15.19:0/2016309172 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f451c0aa100 con 0x5636735bc4a0 2024-05-09T02:16:38.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.859+0000 7f453fa5ab80 10 client.25670 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:38.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.859+0000 7f453fa5ab80 1 -- 172.21.15.19:0/2016309172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5636737bd270 con 0x5636735bc4a0 2024-05-09T02:16:38.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.860+0000 7f450eff5700 1 -- 172.21.15.19:0/2016309172 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4520010c90 con 0x5636735bc4a0 2024-05-09T02:16:38.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.860+0000 7f450eff5700 1 -- 172.21.15.19:0/2016309172 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f451c0c5080 con 0x5636735bc4a0 2024-05-09T02:16:38.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.860+0000 7f453fa5ab80 10 client.25670 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:38.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.860+0000 7f453fa5ab80 10 client.25670 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:16:38.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.860+0000 7f453fa5ab80 20 client.25670 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:38.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.860+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/2016309172 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563673753290 0x5636737bf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:38.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.860+0000 7f453fa5ab80 4 client.25670 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:16:38.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.860+0000 7f453fa5ab80 1 -- 172.21.15.19:0/2016309172 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563673663a70 con 0x563673753290 2024-05-09T02:16:38.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.860+0000 7f452a59c700 1 --2- 172.21.15.19:0/2016309172 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563673753290 0x5636737bf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:38.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.861+0000 7f450eff5700 10 client.25670 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:16:38.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.861+0000 7f452a59c700 1 --2- 172.21.15.19:0/2016309172 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563673753290 0x5636737bf9c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a80b4677d8543e9a server_cookie=162c74362851d3a7 in_seq=0 out_seq=0 2024-05-09T02:16:38.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.862+0000 7f450eff5700 1 -- 172.21.15.19:0/2016309172 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563673663a70 con 0x563673753290 2024-05-09T02:16:38.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.862+0000 7f450eff5700 10 client.25670 handle_command_reply: tid=0 2024-05-09T02:16:38.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.862+0000 7f453fa5ab80 1 client.25670 shutdown 2024-05-09T02:16:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.862+0000 7f453fa5ab80 1 -- 172.21.15.19:0/2016309172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 msgr2=0x56367374b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.862+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/2016309172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 0x56367374b2a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f451c0dfed0 tx=0x7f451c003680 comp rx=0 tx=0).stop 2024-05-09T02:16:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.862+0000 7f453fa5ab80 1 -- 172.21.15.19:0/2016309172 shutdown_connections 2024-05-09T02:16:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.862+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/2016309172 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563673753290 0x5636737bf9c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.862+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/2016309172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636737396a0 0x56367373f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.862+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/2016309172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5636735bc4a0 0x56367374b2a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.862+0000 7f453fa5ab80 1 --2- 172.21.15.19:0/2016309172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636736a6160 0x56367374b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.862+0000 7f453fa5ab80 1 -- 172.21.15.19:0/2016309172 >> 172.21.15.19:0/2016309172 conn(0x56367361c6a0 msgr2=0x563673741ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.862+0000 7f450eff5700 0 client.25670 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:16:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.862+0000 7f453fa5ab80 1 -- 172.21.15.19:0/2016309172 shutdown_connections 2024-05-09T02:16:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.862+0000 7f453fa5ab80 1 -- 172.21.15.19:0/2016309172 wait complete. 2024-05-09T02:16:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.862+0000 7f453fa5ab80 20 client.25670 trim_cache size 0 max 16384 2024-05-09T02:16:38.868 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:16:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.863+0000 7f4531044700 1 -- 172.21.15.19:0/1897043432 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45140b2a70 msgr2=0x7f45140b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.863+0000 7f4531044700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45140b2a70 0x7f45140b4f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4524027000 tx=0x7f4524029000 comp rx=0 tx=0).stop 2024-05-09T02:16:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.863+0000 7f4531044700 1 -- 172.21.15.19:0/1897043432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f452c1045e0 msgr2=0x7f452c199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.863+0000 7f4531044700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f452c1045e0 0x7f452c199960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f451c00e9f0 tx=0x7f451c00ed00 comp rx=0 tx=0).stop 2024-05-09T02:16:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.863+0000 7f4531044700 1 -- 172.21.15.19:0/1897043432 shutdown_connections 2024-05-09T02:16:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.863+0000 7f4531044700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f452c109860 0x7f452c1a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.863+0000 7f4531044700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f452c1045e0 0x7f452c199960 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.863+0000 7f4531044700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45140b2a70 0x7f45140b4f80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.863+0000 7f4531044700 1 --2- 172.21.15.19:0/1897043432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f452c108f10 0x7f452c199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.863+0000 7f4531044700 1 -- 172.21.15.19:0/1897043432 >> 172.21.15.19:0/1897043432 conn(0x7f452c0ffe70 msgr2=0x7f452c110e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.863+0000 7f4531044700 1 -- 172.21.15.19:0/1897043432 shutdown_connections 2024-05-09T02:16:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:38.863+0000 7f4531044700 1 -- 172.21.15.19:0/1897043432 wait complete. 2024-05-09T02:16:39.621 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T02:16:40.020 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:16:40.412 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:40 smithi019 ceph-mon[30151]: pgmap v164: 97 pgs: 97 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 979 op/s 2024-05-09T02:16:40.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:40 smithi059 ceph-mon[34781]: pgmap v164: 97 pgs: 97 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 979 op/s 2024-05-09T02:16:40.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:40 smithi196 ceph-mon[34718]: pgmap v164: 97 pgs: 97 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 979 op/s 2024-05-09T02:16:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.865+0000 7fe71b46e700 1 -- 172.21.15.19:0/1761312291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe714108760 msgr2=0x7fe7141100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.865+0000 7fe71b46e700 1 --2- 172.21.15.19:0/1761312291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe714108760 0x7fe7141100c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009f90 tx=0x7fe70c00b330 comp rx=0 tx=0).stop 2024-05-09T02:16:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.865+0000 7fe71b46e700 1 -- 172.21.15.19:0/1761312291 shutdown_connections 2024-05-09T02:16:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.865+0000 7fe71b46e700 1 --2- 172.21.15.19:0/1761312291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe714108760 0x7fe7141100c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.865+0000 7fe71b46e700 1 --2- 172.21.15.19:0/1761312291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7141034e0 0x7fe7141038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.865+0000 7fe71b46e700 1 --2- 172.21.15.19:0/1761312291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe714107e10 0x7fe714108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.865+0000 7fe71b46e700 1 -- 172.21.15.19:0/1761312291 >> 172.21.15.19:0/1761312291 conn(0x7fe7140fef30 msgr2=0x7fe714101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:41.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.866+0000 7fe71b46e700 1 -- 172.21.15.19:0/1761312291 shutdown_connections 2024-05-09T02:16:41.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.866+0000 7fe71b46e700 1 -- 172.21.15.19:0/1761312291 wait complete. 2024-05-09T02:16:41.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.866+0000 7fe71b46e700 1 Processor -- start 2024-05-09T02:16:41.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.866+0000 7fe71b46e700 1 -- start start 2024-05-09T02:16:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.867+0000 7fe71b46e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7141034e0 0x7fe7141a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.867+0000 7fe71b46e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe714107e10 0x7fe7141a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.867+0000 7fe71920a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7141034e0 0x7fe7141a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.867+0000 7fe71920a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7141034e0 0x7fe7141a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51582/0 (socket says 172.21.15.19:51582) 2024-05-09T02:16:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.867+0000 7fe71920a700 1 -- 172.21.15.19:0/1751205442 learned_addr learned my addr 172.21.15.19:0/1751205442 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.867+0000 7fe71b46e700 1 --2- 172.21.15.19:0/1751205442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe714108760 0x7fe71419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.867+0000 7fe71b46e700 1 -- 172.21.15.19:0/1751205442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141144f0 con 0x7fe7141034e0 2024-05-09T02:16:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.867+0000 7fe71b46e700 1 -- 172.21.15.19:0/1751205442 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe714114370 con 0x7fe714107e10 2024-05-09T02:16:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.867+0000 7fe71b46e700 1 -- 172.21.15.19:0/1751205442 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe714114670 con 0x7fe714108760 2024-05-09T02:16:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.867+0000 7fe718a09700 1 --2- 172.21.15.19:0/1751205442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe714107e10 0x7fe7141a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.867+0000 7fe719a0b700 1 --2- 172.21.15.19:0/1751205442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe714108760 0x7fe71419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.868+0000 7fe71920a700 1 -- 172.21.15.19:0/1751205442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe714108760 msgr2=0x7fe71419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.868+0000 7fe71920a700 1 --2- 172.21.15.19:0/1751205442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe714108760 0x7fe71419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.868+0000 7fe71920a700 1 -- 172.21.15.19:0/1751205442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe714107e10 msgr2=0x7fe7141a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.868+0000 7fe71920a700 1 --2- 172.21.15.19:0/1751205442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe714107e10 0x7fe7141a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.868+0000 7fe71920a700 1 -- 172.21.15.19:0/1751205442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71419b0c0 con 0x7fe7141034e0 2024-05-09T02:16:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.868+0000 7fe71920a700 1 --2- 172.21.15.19:0/1751205442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7141034e0 0x7fe7141a07f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe71000e9c0 tx=0x7fe71000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.868+0000 7fe70a7fc700 1 -- 172.21.15.19:0/1751205442 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710004a30 con 0x7fe7141034e0 2024-05-09T02:16:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.868+0000 7fe70a7fc700 1 -- 172.21.15.19:0/1751205442 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe71000cc70 con 0x7fe7141034e0 2024-05-09T02:16:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.868+0000 7fe71b46e700 1 -- 172.21.15.19:0/1751205442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71419b3b0 con 0x7fe7141034e0 2024-05-09T02:16:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.868+0000 7fe70a7fc700 1 -- 172.21.15.19:0/1751205442 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7100054a0 con 0x7fe7141034e0 2024-05-09T02:16:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.869+0000 7fe71b46e700 1 -- 172.21.15.19:0/1751205442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71419b800 con 0x7fe7141034e0 2024-05-09T02:16:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.874+0000 7fe70a7fc700 1 -- 172.21.15.19:0/1751205442 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe710021440 con 0x7fe7141034e0 2024-05-09T02:16:41.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.875+0000 7fe70a7fc700 1 --2- 172.21.15.19:0/1751205442 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe7000b29f0 0x7fe7000b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.876+0000 7fe70a7fc700 1 -- 172.21.15.19:0/1751205442 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fe7100e26d0 con 0x7fe7141034e0 2024-05-09T02:16:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.876+0000 7fe729e84b80 1 Processor -- start 2024-05-09T02:16:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.876+0000 7fe718a09700 1 --2- 172.21.15.19:0/1751205442 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe7000b29f0 0x7fe7000b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.879+0000 7fe729e84b80 1 -- start start 2024-05-09T02:16:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.879+0000 7fe729e84b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f344a4d00 0x564f345b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.879+0000 7fe729e84b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 0x564f345b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.879+0000 7fe729e84b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f345afc10 0x564f345b0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.879+0000 7fe729e84b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564f344ca0a0 con 0x564f345afc10 2024-05-09T02:16:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.879+0000 7fe729e84b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564f3446f9a0 con 0x564f344a4d00 2024-05-09T02:16:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.879+0000 7fe729e84b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564f344c6060 con 0x564f3442a4a0 2024-05-09T02:16:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.879+0000 7fe719a0b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f345afc10 0x564f345b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.879+0000 7fe719a0b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f345afc10 0x564f345b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51588/0 (socket says 172.21.15.19:51588) 2024-05-09T02:16:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.879+0000 7fe719a0b700 1 -- 172.21.15.19:0/2899232702 learned_addr learned my addr 172.21.15.19:0/2899232702 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.880+0000 7fe719a0b700 1 -- 172.21.15.19:0/2899232702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 msgr2=0x564f345b5fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:16:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.880+0000 7fe719a0b700 1 --2- 172.21.15.19:0/2899232702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 0x564f345b5fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.880+0000 7fe719a0b700 1 -- 172.21.15.19:0/2899232702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f344a4d00 msgr2=0x564f345b5a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:16:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.880+0000 7fe719a0b700 1 --2- 172.21.15.19:0/2899232702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f344a4d00 0x564f345b5a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.880+0000 7fe719a0b700 1 -- 172.21.15.19:0/2899232702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f34460500 con 0x564f345afc10 2024-05-09T02:16:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.880+0000 7fe719a0b700 1 --2- 172.21.15.19:0/2899232702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f345afc10 0x564f345b0020 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0094f0 tx=0x7fe70c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686d9eee48056e28 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:41.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.883+0000 7fe6ff7fe700 1 -- 172.21.15.19:0/2899232702 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c020070 con 0x564f345afc10 2024-05-09T02:16:41.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.883+0000 7fe6ff7fe700 1 -- 172.21.15.19:0/2899232702 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe70c003c90 con 0x564f345afc10 2024-05-09T02:16:41.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.883+0000 7fe6ff7fe700 1 -- 172.21.15.19:0/2899232702 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c005040 con 0x564f345afc10 2024-05-09T02:16:41.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.888+0000 7fe729e84b80 1 -- 172.21.15.19:0/2899232702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f345afc10 msgr2=0x564f345b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:41.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.888+0000 7fe729e84b80 1 --2- 172.21.15.19:0/2899232702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f345afc10 0x564f345b0020 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0094f0 tx=0x7fe70c009b20 comp rx=0 tx=0).stop 2024-05-09T02:16:41.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.889+0000 7fe718a09700 1 --2- 172.21.15.19:0/1751205442 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe7000b29f0 0x7fe7000b4f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x564f345a3a00 tx=0x7fe704009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:41.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.890+0000 7fe729e84b80 1 -- 172.21.15.19:0/2899232702 shutdown_connections 2024-05-09T02:16:41.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.890+0000 7fe729e84b80 1 --2- 172.21.15.19:0/2899232702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f345afc10 0x564f345b0020 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.890+0000 7fe729e84b80 1 --2- 172.21.15.19:0/2899232702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f344a4d00 0x564f345b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.890+0000 7fe729e84b80 1 --2- 172.21.15.19:0/2899232702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 0x564f345b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.890+0000 7fe729e84b80 1 -- 172.21.15.19:0/2899232702 >> 172.21.15.19:0/2899232702 conn(0x564f3442df70 msgr2=0x564f345a2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.897+0000 7fe729e84b80 1 -- 172.21.15.19:0/2899232702 shutdown_connections 2024-05-09T02:16:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.899+0000 7fe729e84b80 1 -- 172.21.15.19:0/2899232702 wait complete. 2024-05-09T02:16:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.900+0000 7fe729e84b80 1 -- start start 2024-05-09T02:16:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.900+0000 7fe729e84b80 1 Processor -- start 2024-05-09T02:16:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.901+0000 7fe729e84b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f3442a4a0 0x564f345b9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.901+0000 7fe729e84b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f344a4d00 0x564f345b9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.901+0000 7fe729e84b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f345afc10 0x564f345ba410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.901+0000 7fe729e84b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564f345b0910 con 0x564f3442a4a0 2024-05-09T02:16:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.901+0000 7fe729e84b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564f34464b20 con 0x564f344a4d00 2024-05-09T02:16:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.901+0000 7fe729e84b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564f345ae6a0 con 0x564f345afc10 2024-05-09T02:16:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.901+0000 7fe729e84b80 10 client.0 _ll_register_callbacks cb 0x564f3433b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.901+0000 7fe719a0b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f345afc10 0x564f345ba410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.901+0000 7fe719a0b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f345afc10 0x564f345ba410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36170/0 (socket says 172.21.15.19:36170) 2024-05-09T02:16:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.901+0000 7fe719a0b700 1 -- 172.21.15.19:0/718574335 learned_addr learned my addr 172.21.15.19:0/718574335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.903+0000 7fe71920a700 1 --2- 172.21.15.19:0/718574335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f3442a4a0 0x564f345b9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.903+0000 7fe719a0b700 1 -- 172.21.15.19:0/718574335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f344a4d00 msgr2=0x564f345b9ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:16:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.903+0000 7fe719a0b700 1 --2- 172.21.15.19:0/718574335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f344a4d00 0x564f345b9ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.903+0000 7fe719a0b700 1 -- 172.21.15.19:0/718574335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f3442a4a0 msgr2=0x564f345b9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.903+0000 7fe719a0b700 1 --2- 172.21.15.19:0/718574335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f3442a4a0 0x564f345b9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.903+0000 7fe719a0b700 1 -- 172.21.15.19:0/718574335 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f34345490 con 0x564f345afc10 2024-05-09T02:16:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.903+0000 7fe71920a700 1 --2- 172.21.15.19:0/718574335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f3442a4a0 0x564f345b9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:16:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.904+0000 7fe719a0b700 1 --2- 172.21.15.19:0/718574335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f345afc10 0x564f345ba410 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00b370 tx=0x7fe70c007e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd7d8b43a4912f48 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.904+0000 7fe6fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:16:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.905+0000 7fe6fcff9700 1 -- 172.21.15.19:0/718574335 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c02a030 con 0x564f345afc10 2024-05-09T02:16:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.905+0000 7fe6fcff9700 1 -- 172.21.15.19:0/718574335 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fe70c00bcc0 con 0x564f345afc10 2024-05-09T02:16:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.905+0000 7fe729e84b80 1 -- 172.21.15.19:0/718574335 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe70c003c90 con 0x564f345afc10 2024-05-09T02:16:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.905+0000 7fe6fcff9700 1 -- 172.21.15.19:0/718574335 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe70c020070 con 0x564f345afc10 2024-05-09T02:16:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.905+0000 7fe729e84b80 10 client.25738 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.905+0000 7fe729e84b80 1 -- 172.21.15.19:0/718574335 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564f345bcd10 con 0x564f345afc10 2024-05-09T02:16:41.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.907+0000 7fe6fcff9700 1 -- 172.21.15.19:0/718574335 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe70c003c90 con 0x564f345afc10 2024-05-09T02:16:41.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.907+0000 7fe6fcff9700 1 -- 172.21.15.19:0/718574335 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7fe70c011070 con 0x564f345afc10 2024-05-09T02:16:41.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.908+0000 7fe729e84b80 10 client.25738 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:41.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.908+0000 7fe729e84b80 10 client.25738 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:16:41.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.908+0000 7fe729e84b80 20 client.25738 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:41.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.908+0000 7fe729e84b80 1 --2- 172.21.15.19:0/718574335 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x564f345bf2a0 0x564f345c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.908+0000 7fe729e84b80 4 client.25738 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:16:41.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.908+0000 7fe729e84b80 1 -- 172.21.15.19:0/718574335 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f344d7a80 con 0x564f345bf2a0 2024-05-09T02:16:41.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.909+0000 7fe71920a700 1 --2- 172.21.15.19:0/718574335 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x564f345bf2a0 0x564f345c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:41.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.914+0000 7fe6fcff9700 10 client.25738 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:16:41.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.914+0000 7fe71920a700 1 --2- 172.21.15.19:0/718574335 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x564f345bf2a0 0x564f345c16c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=194a22d3bf40e8e0 server_cookie=83de505561610a60 in_seq=0 out_seq=0 2024-05-09T02:16:41.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.916+0000 7fe6fcff9700 1 -- 172.21.15.19:0/718574335 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564f344d7a80 con 0x564f345bf2a0 2024-05-09T02:16:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.916+0000 7fe6fcff9700 10 client.25738 handle_command_reply: tid=0 2024-05-09T02:16:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.916+0000 7fe729e84b80 1 client.25738 shutdown 2024-05-09T02:16:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.917+0000 7fe729e84b80 1 -- 172.21.15.19:0/718574335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f345afc10 msgr2=0x564f345ba410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.917+0000 7fe729e84b80 1 --2- 172.21.15.19:0/718574335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f345afc10 0x564f345ba410 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00b370 tx=0x7fe70c007e20 comp rx=0 tx=0).stop 2024-05-09T02:16:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.918+0000 7fe729e84b80 1 -- 172.21.15.19:0/718574335 shutdown_connections 2024-05-09T02:16:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.918+0000 7fe729e84b80 1 --2- 172.21.15.19:0/718574335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f345afc10 0x564f345ba410 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.918+0000 7fe729e84b80 1 --2- 172.21.15.19:0/718574335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f3442a4a0 0x564f345b9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.918+0000 7fe729e84b80 1 --2- 172.21.15.19:0/718574335 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x564f345bf2a0 0x564f345c16c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.918+0000 7fe729e84b80 1 --2- 172.21.15.19:0/718574335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f344a4d00 0x564f345b9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.918+0000 7fe729e84b80 1 -- 172.21.15.19:0/718574335 >> 172.21.15.19:0/718574335 conn(0x564f3442df70 msgr2=0x564f345a63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.919+0000 7fe6fcff9700 0 client.25738 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:16:41.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.921+0000 7fe729e84b80 1 -- 172.21.15.19:0/718574335 shutdown_connections 2024-05-09T02:16:41.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.921+0000 7fe729e84b80 1 -- 172.21.15.19:0/718574335 wait complete. 2024-05-09T02:16:41.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.921+0000 7fe729e84b80 20 client.25738 trim_cache size 0 max 16384 2024-05-09T02:16:41.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.951+0000 7fe729e84b80 1 Processor -- start 2024-05-09T02:16:41.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.951+0000 7fe729e84b80 1 -- start start 2024-05-09T02:16:41.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.951+0000 7fe729e84b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 0x564f3459ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.951+0000 7fe729e84b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f344a4d00 0x564f345c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.951+0000 7fe71920a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 0x564f3459ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:41.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.951+0000 7fe71920a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 0x564f3459ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36182/0 (socket says 172.21.15.19:36182) 2024-05-09T02:16:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.951+0000 7fe718a09700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f344a4d00 0x564f345c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.952+0000 7fe71920a700 1 -- 172.21.15.19:0/730750539 learned_addr learned my addr 172.21.15.19:0/730750539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.952+0000 7fe729e84b80 1 --2- 172.21.15.19:0/730750539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f345afc10 0x564f345c7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.952+0000 7fe729e84b80 1 -- 172.21.15.19:0/730750539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141acaf0 con 0x564f344a4d00 2024-05-09T02:16:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.952+0000 7fe729e84b80 1 -- 172.21.15.19:0/730750539 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141ae6f0 con 0x564f345afc10 2024-05-09T02:16:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.952+0000 7fe729e84b80 1 -- 172.21.15.19:0/730750539 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141ad1c0 con 0x564f3442a4a0 2024-05-09T02:16:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.952+0000 7fe719a0b700 1 --2- 172.21.15.19:0/730750539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f345afc10 0x564f345c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.952+0000 7fe71920a700 1 -- 172.21.15.19:0/730750539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f345afc10 msgr2=0x564f345c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.952+0000 7fe71920a700 1 --2- 172.21.15.19:0/730750539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f345afc10 0x564f345c7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.952+0000 7fe71920a700 1 -- 172.21.15.19:0/730750539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f344a4d00 msgr2=0x564f345c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.952+0000 7fe71920a700 1 --2- 172.21.15.19:0/730750539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f344a4d00 0x564f345c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.952+0000 7fe71920a700 1 -- 172.21.15.19:0/730750539 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f345a2cf0 con 0x564f3442a4a0 2024-05-09T02:16:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.952+0000 7fe71920a700 1 --2- 172.21.15.19:0/730750539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 0x564f3459ca20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe7100dfe80 tx=0x7fe7100e2040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3818bd546009b4c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.953+0000 7fe6feffd700 1 -- 172.21.15.19:0/730750539 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7100c2020 con 0x564f3442a4a0 2024-05-09T02:16:41.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.953+0000 7fe6feffd700 1 -- 172.21.15.19:0/730750539 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7100aa0b0 con 0x564f3442a4a0 2024-05-09T02:16:41.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.953+0000 7fe6feffd700 1 -- 172.21.15.19:0/730750539 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710005640 con 0x564f3442a4a0 2024-05-09T02:16:41.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.954+0000 7fe729e84b80 1 -- 172.21.15.19:0/730750539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 msgr2=0x564f3459ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:41.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.954+0000 7fe729e84b80 1 --2- 172.21.15.19:0/730750539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 0x564f3459ca20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe7100dfe80 tx=0x7fe7100e2040 comp rx=0 tx=0).stop 2024-05-09T02:16:41.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.954+0000 7fe729e84b80 1 -- 172.21.15.19:0/730750539 shutdown_connections 2024-05-09T02:16:41.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.954+0000 7fe729e84b80 1 --2- 172.21.15.19:0/730750539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f345afc10 0x564f345c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.954+0000 7fe729e84b80 1 --2- 172.21.15.19:0/730750539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 0x564f3459ca20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.954+0000 7fe729e84b80 1 --2- 172.21.15.19:0/730750539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f344a4d00 0x564f345c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.954+0000 7fe729e84b80 1 -- 172.21.15.19:0/730750539 >> 172.21.15.19:0/730750539 conn(0x564f3442df70 msgr2=0x564f345a21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:41.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.954+0000 7fe729e84b80 1 -- 172.21.15.19:0/730750539 shutdown_connections 2024-05-09T02:16:41.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.954+0000 7fe729e84b80 1 -- 172.21.15.19:0/730750539 wait complete. 2024-05-09T02:16:41.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.955+0000 7fe729e84b80 1 -- start start 2024-05-09T02:16:41.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.955+0000 7fe729e84b80 1 Processor -- start 2024-05-09T02:16:41.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.956+0000 7fe729e84b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 0x564f345aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.956+0000 7fe729e84b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f344a4d00 0x564f345aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.956+0000 7fe71920a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 0x564f345aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:41.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.956+0000 7fe71920a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 0x564f345aa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36198/0 (socket says 172.21.15.19:36198) 2024-05-09T02:16:41.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.956+0000 7fe71920a700 1 -- 172.21.15.19:0/1379586465 learned_addr learned my addr 172.21.15.19:0/1379586465 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:41.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.956+0000 7fe729e84b80 1 --2- 172.21.15.19:0/1379586465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f345afc10 0x564f345aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.956+0000 7fe729e84b80 1 -- 172.21.15.19:0/1379586465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141ae020 con 0x564f344a4d00 2024-05-09T02:16:41.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.956+0000 7fe729e84b80 1 -- 172.21.15.19:0/1379586465 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141ad890 con 0x564f345afc10 2024-05-09T02:16:41.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.957+0000 7fe719a0b700 1 --2- 172.21.15.19:0/1379586465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f345afc10 0x564f345aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:41.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.957+0000 7fe729e84b80 1 -- 172.21.15.19:0/1379586465 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141aaab0 con 0x564f3442a4a0 2024-05-09T02:16:41.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.957+0000 7fe71920a700 1 -- 172.21.15.19:0/1379586465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f345afc10 msgr2=0x564f345aaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:41.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.957+0000 7fe71920a700 1 --2- 172.21.15.19:0/1379586465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f345afc10 0x564f345aaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.957+0000 7fe71920a700 1 -- 172.21.15.19:0/1379586465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f344a4d00 msgr2=0x564f345aa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:41.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.957+0000 7fe71920a700 1 --2- 172.21.15.19:0/1379586465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f344a4d00 0x564f345aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.957+0000 7fe71920a700 1 -- 172.21.15.19:0/1379586465 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f3464bb30 con 0x564f3442a4a0 2024-05-09T02:16:41.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.958+0000 7fe729e84b80 10 client.0 _ll_register_callbacks cb 0x564f3433b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:41.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.958+0000 7fe729e84b80 1 -- 172.21.15.19:0/1379586465 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7100aa0b0 con 0x564f3442a4a0 2024-05-09T02:16:41.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.958+0000 7fe71920a700 1 --2- 172.21.15.19:0/1379586465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 0x564f345aa4a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe7100a6c20 tx=0x7fe7100a9300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c45c2e11ee74d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:41.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.958+0000 7fe6fcff9700 10 client.25744 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:16:41.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.958+0000 7fe6fcff9700 1 -- 172.21.15.19:0/1379586465 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7100c2030 con 0x564f3442a4a0 2024-05-09T02:16:41.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.958+0000 7fe6fcff9700 1 -- 172.21.15.19:0/1379586465 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fe7100a5fa0 con 0x564f3442a4a0 2024-05-09T02:16:41.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.958+0000 7fe6fcff9700 1 -- 172.21.15.19:0/1379586465 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe7100b2810 con 0x564f3442a4a0 2024-05-09T02:16:41.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.958+0000 7fe729e84b80 10 client.25744 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:41.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.958+0000 7fe729e84b80 1 -- 172.21.15.19:0/1379586465 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=41}) -- 0x564f3464cbf0 con 0x564f3442a4a0 2024-05-09T02:16:41.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.959+0000 7fe6fcff9700 1 -- 172.21.15.19:0/1379586465 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7fe7100c6060 con 0x564f3442a4a0 2024-05-09T02:16:41.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.959+0000 7fe729e84b80 10 client.25744 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:41.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.959+0000 7fe729e84b80 10 client.25744 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:16:41.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.959+0000 7fe729e84b80 20 client.25744 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:41.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.959+0000 7fe729e84b80 1 --2- 172.21.15.19:0/1379586465 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x564f3464d120 0x564f3464f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:41.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.959+0000 7fe729e84b80 4 client.25744 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:16:41.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.959+0000 7fe729e84b80 1 -- 172.21.15.19:0/1379586465 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f344d1b20 con 0x564f3464d120 2024-05-09T02:16:41.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.960+0000 7fe718a09700 1 --2- 172.21.15.19:0/1379586465 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x564f3464d120 0x564f3464f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:41.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.960+0000 7fe718a09700 1 --2- 172.21.15.19:0/1379586465 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x564f3464d120 0x564f3464f5f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b2f6b01c77704280 server_cookie=2855072fab420ee6 in_seq=0 out_seq=0 2024-05-09T02:16:41.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.960+0000 7fe6fcff9700 10 client.25744 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:16:41.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.961+0000 7fe6fcff9700 1 -- 172.21.15.19:0/1379586465 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f344d1b20 con 0x564f3464d120 2024-05-09T02:16:41.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.961+0000 7fe6fcff9700 10 client.25744 handle_command_reply: tid=0 2024-05-09T02:16:41.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.961+0000 7fe729e84b80 1 client.25744 shutdown 2024-05-09T02:16:41.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.961+0000 7fe729e84b80 1 -- 172.21.15.19:0/1379586465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 msgr2=0x564f345aa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:41.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.961+0000 7fe729e84b80 1 --2- 172.21.15.19:0/1379586465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 0x564f345aa4a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe7100a6c20 tx=0x7fe7100a9300 comp rx=0 tx=0).stop 2024-05-09T02:16:41.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.962+0000 7fe729e84b80 1 -- 172.21.15.19:0/1379586465 shutdown_connections 2024-05-09T02:16:41.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.962+0000 7fe729e84b80 1 --2- 172.21.15.19:0/1379586465 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x564f3464d120 0x564f3464f5f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.962+0000 7fe729e84b80 1 --2- 172.21.15.19:0/1379586465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f345afc10 0x564f345aaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.962+0000 7fe729e84b80 1 --2- 172.21.15.19:0/1379586465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f3442a4a0 0x564f345aa4a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.962+0000 7fe729e84b80 1 --2- 172.21.15.19:0/1379586465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f344a4d00 0x564f345aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.962+0000 7fe6fcff9700 0 client.25744 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:16:41.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.962+0000 7fe729e84b80 1 -- 172.21.15.19:0/1379586465 >> 172.21.15.19:0/1379586465 conn(0x564f3442df70 msgr2=0x564f345a21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:41.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.962+0000 7fe729e84b80 1 -- 172.21.15.19:0/1379586465 shutdown_connections 2024-05-09T02:16:41.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.962+0000 7fe729e84b80 1 -- 172.21.15.19:0/1379586465 wait complete. 2024-05-09T02:16:41.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.962+0000 7fe729e84b80 20 client.25744 trim_cache size 0 max 16384 2024-05-09T02:16:41.967 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:16:41.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.963+0000 7fe71b46e700 1 -- 172.21.15.19:0/1751205442 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe7000b29f0 msgr2=0x7fe7000b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:41.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.963+0000 7fe71b46e700 1 --2- 172.21.15.19:0/1751205442 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe7000b29f0 0x7fe7000b4f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x564f345a3a00 tx=0x7fe704009260 comp rx=0 tx=0).stop 2024-05-09T02:16:41.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.963+0000 7fe71b46e700 1 -- 172.21.15.19:0/1751205442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7141034e0 msgr2=0x7fe7141a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:41.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.963+0000 7fe71b46e700 1 --2- 172.21.15.19:0/1751205442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7141034e0 0x7fe7141a07f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe71000e9c0 tx=0x7fe71000ecd0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.963+0000 7fe71b46e700 1 -- 172.21.15.19:0/1751205442 shutdown_connections 2024-05-09T02:16:41.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.963+0000 7fe71b46e700 1 --2- 172.21.15.19:0/1751205442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe714108760 0x7fe71419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.963+0000 7fe71b46e700 1 --2- 172.21.15.19:0/1751205442 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe7000b29f0 0x7fe7000b4f00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.963+0000 7fe71b46e700 1 --2- 172.21.15.19:0/1751205442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7141034e0 0x7fe7141a07f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.963+0000 7fe71b46e700 1 --2- 172.21.15.19:0/1751205442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe714107e10 0x7fe7141a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:41.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.963+0000 7fe71b46e700 1 -- 172.21.15.19:0/1751205442 >> 172.21.15.19:0/1751205442 conn(0x7fe7140fef30 msgr2=0x7fe714105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:41.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.964+0000 7fe71b46e700 1 -- 172.21.15.19:0/1751205442 shutdown_connections 2024-05-09T02:16:41.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:41.964+0000 7fe71b46e700 1 -- 172.21.15.19:0/1751205442 wait complete. 2024-05-09T02:16:42.620 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:42 smithi019 ceph-mon[30151]: pgmap v165: 97 pgs: 97 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.5 MiB/s wr, 616 op/s 2024-05-09T02:16:42.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T02:16:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 300.0 s, 2544.6 tps, lat 39.046 ms stddev 42.192 2024-05-09T02:16:42.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:42 smithi059 ceph-mon[34781]: pgmap v165: 97 pgs: 97 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.5 MiB/s wr, 616 op/s 2024-05-09T02:16:42.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:42 smithi196 ceph-mon[34718]: pgmap v165: 97 pgs: 97 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.5 MiB/s wr, 616 op/s 2024-05-09T02:16:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:16:44.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:44 smithi019 ceph-mon[30151]: pgmap v166: 97 pgs: 97 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.6 MiB/s wr, 627 op/s 2024-05-09T02:16:44.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:44 smithi059 ceph-mon[34781]: pgmap v166: 97 pgs: 97 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.6 MiB/s wr, 627 op/s 2024-05-09T02:16:44.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:44 smithi196 ceph-mon[34718]: pgmap v166: 97 pgs: 97 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.6 MiB/s wr, 627 op/s 2024-05-09T02:16:45.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.375+0000 7f8a5b1b5700 1 -- 172.21.15.19:0/1390945230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a54110fb0 msgr2=0x7f8a541188f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:45.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.375+0000 7f8a5b1b5700 1 --2- 172.21.15.19:0/1390945230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a54110fb0 0x7f8a541188f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009990 tx=0x7f8a4c009ca0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.375+0000 7f8a5b1b5700 1 -- 172.21.15.19:0/1390945230 shutdown_connections 2024-05-09T02:16:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.375+0000 7f8a5b1b5700 1 --2- 172.21.15.19:0/1390945230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a54110fb0 0x7f8a541188f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.375+0000 7f8a5b1b5700 1 --2- 172.21.15.19:0/1390945230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a5410c840 0x7f8a5410cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.375+0000 7f8a5b1b5700 1 --2- 172.21.15.19:0/1390945230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a54110660 0x7f8a54110a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.375+0000 7f8a5b1b5700 1 -- 172.21.15.19:0/1390945230 >> 172.21.15.19:0/1390945230 conn(0x7f8a5406f4d0 msgr2=0x7f8a5406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.375+0000 7f8a5b1b5700 1 -- 172.21.15.19:0/1390945230 shutdown_connections 2024-05-09T02:16:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.375+0000 7f8a5b1b5700 1 -- 172.21.15.19:0/1390945230 wait complete. 2024-05-09T02:16:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.376+0000 7f8a5b1b5700 1 Processor -- start 2024-05-09T02:16:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.376+0000 7f8a5b1b5700 1 -- start start 2024-05-09T02:16:45.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.376+0000 7f8a5b1b5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a5410c840 0x7f8a54114530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.376+0000 7f8a5b1b5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a54110660 0x7f8a54112990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.376+0000 7f8a5b1b5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a54110fb0 0x7f8a54112f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.377+0000 7f8a5b1b5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5411bbc0 con 0x7f8a54110660 2024-05-09T02:16:45.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.377+0000 7f8a5b1b5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5411ba40 con 0x7f8a5410c840 2024-05-09T02:16:45.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.377+0000 7f8a5b1b5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5411bd40 con 0x7f8a54110fb0 2024-05-09T02:16:45.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.377+0000 7f8a58f51700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a5410c840 0x7f8a54114530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.377+0000 7f8a58f51700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a5410c840 0x7f8a54114530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:49686/0 (socket says 172.21.15.19:49686) 2024-05-09T02:16:45.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.377+0000 7f8a58f51700 1 -- 172.21.15.19:0/3966242123 learned_addr learned my addr 172.21.15.19:0/3966242123 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:45.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.377+0000 7f8a59752700 1 --2- 172.21.15.19:0/3966242123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a54110fb0 0x7f8a54112f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.377+0000 7f8a58f51700 1 -- 172.21.15.19:0/3966242123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a54110fb0 msgr2=0x7f8a54112f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:45.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.377+0000 7f8a58f51700 1 --2- 172.21.15.19:0/3966242123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a54110fb0 0x7f8a54112f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.377+0000 7f8a58f51700 1 -- 172.21.15.19:0/3966242123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a54110660 msgr2=0x7f8a54112990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:45.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.377+0000 7f8a58f51700 1 --2- 172.21.15.19:0/3966242123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a54110660 0x7f8a54112990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.377+0000 7f8a58f51700 1 -- 172.21.15.19:0/3966242123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a541137a0 con 0x7f8a5410c840 2024-05-09T02:16:45.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.378+0000 7f8a58f51700 1 --2- 172.21.15.19:0/3966242123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a5410c840 0x7f8a54114530 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00e970 tx=0x7f8a3c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:45.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.378+0000 7f8a51ffb700 1 -- 172.21.15.19:0/3966242123 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c00cbd0 con 0x7f8a5410c840 2024-05-09T02:16:45.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.379+0000 7f8a51ffb700 1 -- 172.21.15.19:0/3966242123 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a3c00cd70 con 0x7f8a5410c840 2024-05-09T02:16:45.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.379+0000 7f8a51ffb700 1 -- 172.21.15.19:0/3966242123 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c005680 con 0x7f8a5410c840 2024-05-09T02:16:45.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.379+0000 7f8a5b1b5700 1 -- 172.21.15.19:0/3966242123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a541b60d0 con 0x7f8a5410c840 2024-05-09T02:16:45.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.379+0000 7f8a5b1b5700 1 -- 172.21.15.19:0/3966242123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a541b6590 con 0x7f8a5410c840 2024-05-09T02:16:45.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.383+0000 7f8a51ffb700 1 -- 172.21.15.19:0/3966242123 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8a3c004750 con 0x7f8a5410c840 2024-05-09T02:16:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.385+0000 7f8a69bcbb80 1 Processor -- start 2024-05-09T02:16:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.385+0000 7f8a51ffb700 1 --2- 172.21.15.19:0/3966242123 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8a400b2af0 0x7f8a400b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.385+0000 7f8a51ffb700 1 -- 172.21.15.19:0/3966242123 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f8a3c0e29d0 con 0x7f8a5410c840 2024-05-09T02:16:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.386+0000 7f8a69bcbb80 1 -- start start 2024-05-09T02:16:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.386+0000 7f8a59752700 1 --2- 172.21.15.19:0/3966242123 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8a400b2af0 0x7f8a400b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.386+0000 7f8a69bcbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52ead00 0x55dcf53f3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.386+0000 7f8a69bcbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52704a0 0x55dcf53f3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.386+0000 7f8a69bcbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf53ed400 0x55dcf53ed810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.386+0000 7f8a69bcbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dcf53100a0 con 0x55dcf52704a0 2024-05-09T02:16:45.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.386+0000 7f8a69bcbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dcf52b59a0 con 0x55dcf53ed400 2024-05-09T02:16:45.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.386+0000 7f8a69bcbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dcf530c060 con 0x55dcf52ead00 2024-05-09T02:16:45.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.387+0000 7f8a58f51700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52ead00 0x55dcf53f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.387+0000 7f8a58f51700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52ead00 0x55dcf53f3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36228/0 (socket says 172.21.15.19:36228) 2024-05-09T02:16:45.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.387+0000 7f8a58f51700 1 -- 172.21.15.19:0/997030703 learned_addr learned my addr 172.21.15.19:0/997030703 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:45.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.387+0000 7f8a59752700 1 --2- 172.21.15.19:0/3966242123 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8a400b2af0 0x7f8a400b5000 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00b2e0 tx=0x7f8a4c005960 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:45.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.387+0000 7f8a53fff700 1 --2- 172.21.15.19:0/997030703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52704a0 0x55dcf53f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.387+0000 7f8a59752700 1 --2- 172.21.15.19:0/997030703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf53ed400 0x55dcf53ed810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.387+0000 7f8a58f51700 1 -- 172.21.15.19:0/997030703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf53ed400 msgr2=0x55dcf53ed810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:45.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.387+0000 7f8a58f51700 1 --2- 172.21.15.19:0/997030703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf53ed400 0x55dcf53ed810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.387+0000 7f8a58f51700 1 -- 172.21.15.19:0/997030703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52704a0 msgr2=0x55dcf53f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:45.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.387+0000 7f8a58f51700 1 --2- 172.21.15.19:0/997030703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52704a0 0x55dcf53f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.387+0000 7f8a58f51700 1 -- 172.21.15.19:0/997030703 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf52a6500 con 0x55dcf52ead00 2024-05-09T02:16:45.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.387+0000 7f8a53fff700 1 --2- 172.21.15.19:0/997030703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52704a0 0x55dcf53f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:16:45.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.388+0000 7f8a58f51700 1 --2- 172.21.15.19:0/997030703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52ead00 0x55dcf53f3240 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00cfd0 tx=0x7f8a3c0a93c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46f37ce59b4cda37 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:45.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.389+0000 7f8a4affd700 1 -- 172.21.15.19:0/997030703 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c091040 con 0x55dcf52ead00 2024-05-09T02:16:45.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.389+0000 7f8a4affd700 1 -- 172.21.15.19:0/997030703 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a3c0b2400 con 0x55dcf52ead00 2024-05-09T02:16:45.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.389+0000 7f8a4affd700 1 -- 172.21.15.19:0/997030703 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0aa110 con 0x55dcf52ead00 2024-05-09T02:16:45.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.390+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/997030703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52ead00 msgr2=0x55dcf53f3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:45.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.390+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/997030703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52ead00 0x55dcf53f3240 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00cfd0 tx=0x7f8a3c0a93c0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.390+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/997030703 shutdown_connections 2024-05-09T02:16:45.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.390+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/997030703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf53ed400 0x55dcf53ed810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.390+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/997030703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52ead00 0x55dcf53f3240 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.390+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/997030703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52704a0 0x55dcf53f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.390+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/997030703 >> 172.21.15.19:0/997030703 conn(0x55dcf5273f70 msgr2=0x55dcf53e8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:45.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.390+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/997030703 shutdown_connections 2024-05-09T02:16:45.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.390+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/997030703 wait complete. 2024-05-09T02:16:45.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.391+0000 7f8a69bcbb80 1 -- start start 2024-05-09T02:16:45.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.391+0000 7f8a69bcbb80 1 Processor -- start 2024-05-09T02:16:45.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.391+0000 7f8a69bcbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52704a0 0x55dcf53f7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.391+0000 7f8a69bcbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52ead00 0x55dcf53f7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a69bcbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf53ed400 0x55dcf53f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a69bcbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dcf53ee160 con 0x55dcf52ead00 2024-05-09T02:16:45.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a58f51700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52704a0 0x55dcf53f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a69bcbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dcf52aab20 con 0x55dcf53ed400 2024-05-09T02:16:45.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a69bcbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dcf53fa020 con 0x55dcf52704a0 2024-05-09T02:16:45.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a58f51700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52704a0 0x55dcf53f7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36238/0 (socket says 172.21.15.19:36238) 2024-05-09T02:16:45.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a53fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52ead00 0x55dcf53f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a58f51700 1 -- 172.21.15.19:0/3818671711 learned_addr learned my addr 172.21.15.19:0/3818671711 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a59752700 1 --2- 172.21.15.19:0/3818671711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf53ed400 0x55dcf53f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a69bcbb80 10 client.0 _ll_register_callbacks cb 0x55dcf51816c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a58f51700 1 -- 172.21.15.19:0/3818671711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf53ed400 msgr2=0x55dcf53f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a58f51700 1 --2- 172.21.15.19:0/3818671711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf53ed400 0x55dcf53f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a58f51700 1 -- 172.21.15.19:0/3818671711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52ead00 msgr2=0x55dcf53f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a58f51700 1 --2- 172.21.15.19:0/3818671711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52ead00 0x55dcf53f7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a58f51700 1 -- 172.21.15.19:0/3818671711 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf518b490 con 0x55dcf52704a0 2024-05-09T02:16:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a58f51700 1 --2- 172.21.15.19:0/3818671711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52704a0 0x55dcf53f7110 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0a9be0 tx=0x7f8a3c0a9890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=958cc7d944f87651 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a33fff700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:16:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a33fff700 1 -- 172.21.15.19:0/3818671711 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0c8070 con 0x55dcf52704a0 2024-05-09T02:16:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a33fff700 1 -- 172.21.15.19:0/3818671711 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f8a3c0a90c0 con 0x55dcf52704a0 2024-05-09T02:16:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.392+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3818671711 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a3c0b2400 con 0x55dcf52704a0 2024-05-09T02:16:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.393+0000 7f8a33fff700 1 -- 172.21.15.19:0/3818671711 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8a3c0e0550 con 0x55dcf52704a0 2024-05-09T02:16:45.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.393+0000 7f8a69bcbb80 10 client.25756 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:45.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.393+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3818671711 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55dcf5404a80 con 0x55dcf52704a0 2024-05-09T02:16:45.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.393+0000 7f8a33fff700 1 -- 172.21.15.19:0/3818671711 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a3c0b2400 con 0x55dcf52704a0 2024-05-09T02:16:45.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.393+0000 7f8a33fff700 1 -- 172.21.15.19:0/3818671711 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f8a3c004a40 con 0x55dcf52704a0 2024-05-09T02:16:45.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.393+0000 7f8a69bcbb80 10 client.25756 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:45.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.393+0000 7f8a69bcbb80 10 client.25756 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:16:45.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.393+0000 7f8a69bcbb80 20 client.25756 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:45.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.393+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/3818671711 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dcf5406f50 0x55dcf5409370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.393+0000 7f8a69bcbb80 4 client.25756 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:16:45.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.393+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3818671711 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcf531da80 con 0x55dcf5406f50 2024-05-09T02:16:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.394+0000 7f8a53fff700 1 --2- 172.21.15.19:0/3818671711 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dcf5406f50 0x55dcf5409370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.394+0000 7f8a33fff700 10 client.25756 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:16:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.394+0000 7f8a53fff700 1 --2- 172.21.15.19:0/3818671711 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dcf5406f50 0x55dcf5409370 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9efd9b06bba5d3a7 server_cookie=15d983c21f4cb424 in_seq=0 out_seq=0 2024-05-09T02:16:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.395+0000 7f8a33fff700 1 -- 172.21.15.19:0/3818671711 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dcf531da80 con 0x55dcf5406f50 2024-05-09T02:16:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.395+0000 7f8a33fff700 10 client.25756 handle_command_reply: tid=0 2024-05-09T02:16:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.395+0000 7f8a69bcbb80 1 client.25756 shutdown 2024-05-09T02:16:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.396+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3818671711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52704a0 msgr2=0x55dcf53f7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.396+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/3818671711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52704a0 0x55dcf53f7110 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0a9be0 tx=0x7f8a3c0a9890 comp rx=0 tx=0).stop 2024-05-09T02:16:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.396+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3818671711 shutdown_connections 2024-05-09T02:16:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.396+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/3818671711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf53ed400 0x55dcf53f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.396+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/3818671711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52704a0 0x55dcf53f7110 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.396+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/3818671711 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dcf5406f50 0x55dcf5409370 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.396+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/3818671711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52ead00 0x55dcf53f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.396+0000 7f8a33fff700 0 client.25756 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:16:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.396+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3818671711 >> 172.21.15.19:0/3818671711 conn(0x55dcf5273f70 msgr2=0x55dcf53e1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.396+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3818671711 shutdown_connections 2024-05-09T02:16:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.396+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3818671711 wait complete. 2024-05-09T02:16:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.396+0000 7f8a69bcbb80 20 client.25756 trim_cache size 0 max 16384 2024-05-09T02:16:45.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.421+0000 7f8a69bcbb80 1 Processor -- start 2024-05-09T02:16:45.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.421+0000 7f8a69bcbb80 1 -- start start 2024-05-09T02:16:45.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.421+0000 7f8a69bcbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52704a0 0x55dcf53f6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.421+0000 7f8a69bcbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf52ead00 0x55dcf53e2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.421+0000 7f8a58f51700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52704a0 0x55dcf53f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.421+0000 7f8a58f51700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52704a0 0x55dcf53f6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51644/0 (socket says 172.21.15.19:51644) 2024-05-09T02:16:45.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.421+0000 7f8a58f51700 1 -- 172.21.15.19:0/3226317663 learned_addr learned my addr 172.21.15.19:0/3226317663 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.421+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/3226317663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf53ed400 0x55dcf53f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.421+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3226317663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541bdde0 con 0x55dcf52704a0 2024-05-09T02:16:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.421+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3226317663 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541bf9e0 con 0x55dcf52ead00 2024-05-09T02:16:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.421+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3226317663 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541be4b0 con 0x55dcf53ed400 2024-05-09T02:16:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.421+0000 7f8a53fff700 1 --2- 172.21.15.19:0/3226317663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf52ead00 0x55dcf53e2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.421+0000 7f8a59752700 1 --2- 172.21.15.19:0/3226317663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf53ed400 0x55dcf53f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.422+0000 7f8a58f51700 1 -- 172.21.15.19:0/3226317663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf53ed400 msgr2=0x55dcf53f70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.422+0000 7f8a58f51700 1 --2- 172.21.15.19:0/3226317663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf53ed400 0x55dcf53f70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.422+0000 7f8a58f51700 1 -- 172.21.15.19:0/3226317663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf52ead00 msgr2=0x55dcf53e2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.422+0000 7f8a58f51700 1 --2- 172.21.15.19:0/3226317663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf52ead00 0x55dcf53e2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.422+0000 7f8a58f51700 1 -- 172.21.15.19:0/3226317663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf53e49d0 con 0x55dcf52704a0 2024-05-09T02:16:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.422+0000 7f8a58f51700 1 --2- 172.21.15.19:0/3226317663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52704a0 0x55dcf53f6070 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0a9950 tx=0x7f8a3c0a5b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741b2493f3920e1f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.422+0000 7f8a4a7fc700 1 -- 172.21.15.19:0/3226317663 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c091030 con 0x55dcf52704a0 2024-05-09T02:16:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.422+0000 7f8a4a7fc700 1 -- 172.21.15.19:0/3226317663 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a3c0b2400 con 0x55dcf52704a0 2024-05-09T02:16:45.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.422+0000 7f8a4a7fc700 1 -- 172.21.15.19:0/3226317663 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0c8070 con 0x55dcf52704a0 2024-05-09T02:16:45.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.423+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3226317663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52704a0 msgr2=0x55dcf53f6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:45.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.423+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/3226317663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52704a0 0x55dcf53f6070 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0a9950 tx=0x7f8a3c0a5b10 comp rx=0 tx=0).stop 2024-05-09T02:16:45.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.423+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3226317663 shutdown_connections 2024-05-09T02:16:45.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.423+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/3226317663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf53ed400 0x55dcf53f70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.423+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/3226317663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf52704a0 0x55dcf53f6070 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.423+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/3226317663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf52ead00 0x55dcf53e2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.423+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3226317663 >> 172.21.15.19:0/3226317663 conn(0x55dcf5273f70 msgr2=0x55dcf53e1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:45.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.423+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3226317663 shutdown_connections 2024-05-09T02:16:45.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.423+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/3226317663 wait complete. 2024-05-09T02:16:45.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.423+0000 7f8a69bcbb80 1 -- start start 2024-05-09T02:16:45.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.424+0000 7f8a69bcbb80 1 Processor -- start 2024-05-09T02:16:45.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.424+0000 7f8a69bcbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52704a0 0x55dcf540af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.424+0000 7f8a69bcbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf52ead00 0x55dcf540b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.424+0000 7f8a69bcbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf53ed400 0x55dcf540b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.424+0000 7f8a58f51700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52704a0 0x55dcf540af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.424+0000 7f8a58f51700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52704a0 0x55dcf540af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36270/0 (socket says 172.21.15.19:36270) 2024-05-09T02:16:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.424+0000 7f8a58f51700 1 -- 172.21.15.19:0/2025040766 learned_addr learned my addr 172.21.15.19:0/2025040766 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.424+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/2025040766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541bf310 con 0x55dcf53ed400 2024-05-09T02:16:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.424+0000 7f8a59752700 1 --2- 172.21.15.19:0/2025040766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf53ed400 0x55dcf540b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.424+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/2025040766 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541beb80 con 0x55dcf52ead00 2024-05-09T02:16:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.424+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/2025040766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541bbd80 con 0x55dcf52704a0 2024-05-09T02:16:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.425+0000 7f8a53fff700 1 --2- 172.21.15.19:0/2025040766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf52ead00 0x55dcf540b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.425+0000 7f8a69bcbb80 10 client.0 _ll_register_callbacks cb 0x55dcf51816c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.425+0000 7f8a59752700 1 -- 172.21.15.19:0/2025040766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52704a0 msgr2=0x55dcf540af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.425+0000 7f8a59752700 1 --2- 172.21.15.19:0/2025040766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52704a0 0x55dcf540af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.425+0000 7f8a59752700 1 -- 172.21.15.19:0/2025040766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf52ead00 msgr2=0x55dcf540b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.425+0000 7f8a59752700 1 --2- 172.21.15.19:0/2025040766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf52ead00 0x55dcf540b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.425+0000 7f8a59752700 1 -- 172.21.15.19:0/2025040766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf546f4a0 con 0x55dcf53ed400 2024-05-09T02:16:45.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.426+0000 7f8a59752700 1 --2- 172.21.15.19:0/2025040766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf53ed400 0x55dcf540b9b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c006010 tx=0x7f8a4c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d054c4bb192ff53d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.426+0000 7f8a33fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:16:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.426+0000 7f8a33fff700 1 -- 172.21.15.19:0/2025040766 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c034060 con 0x55dcf53ed400 2024-05-09T02:16:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.426+0000 7f8a33fff700 1 -- 172.21.15.19:0/2025040766 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f8a4c00f660 con 0x55dcf53ed400 2024-05-09T02:16:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.427+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/2025040766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a3c0b2400 con 0x55dcf53ed400 2024-05-09T02:16:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.427+0000 7f8a33fff700 1 -- 172.21.15.19:0/2025040766 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8a4c00f8a0 con 0x55dcf53ed400 2024-05-09T02:16:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.427+0000 7f8a69bcbb80 10 client.16002 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.427+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/2025040766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55dcf5470570 con 0x55dcf53ed400 2024-05-09T02:16:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.428+0000 7f8a33fff700 1 -- 172.21.15.19:0/2025040766 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a4c023400 con 0x55dcf53ed400 2024-05-09T02:16:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.428+0000 7f8a33fff700 1 -- 172.21.15.19:0/2025040766 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f8a4c022070 con 0x55dcf53ed400 2024-05-09T02:16:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.428+0000 7f8a69bcbb80 10 client.16002 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.428+0000 7f8a69bcbb80 10 client.16002 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:16:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.428+0000 7f8a69bcbb80 20 client.16002 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.428+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/2025040766 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dcf5470cf0 0x55dcf5473110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:45.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.429+0000 7f8a69bcbb80 4 client.16002 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:16:45.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.429+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/2025040766 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8a3c0b2400 con 0x55dcf5470cf0 2024-05-09T02:16:45.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.429+0000 7f8a58f51700 1 --2- 172.21.15.19:0/2025040766 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dcf5470cf0 0x55dcf5473110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:45.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.430+0000 7f8a33fff700 10 client.16002 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:16:45.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.431+0000 7f8a58f51700 1 --2- 172.21.15.19:0/2025040766 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dcf5470cf0 0x55dcf5473110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d4fe64a9bf2bb77 server_cookie=da735b252cfff9e8 in_seq=0 out_seq=0 2024-05-09T02:16:45.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.432+0000 7f8a33fff700 1 -- 172.21.15.19:0/2025040766 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8a3c0b2400 con 0x55dcf5470cf0 2024-05-09T02:16:45.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.432+0000 7f8a33fff700 10 client.16002 handle_command_reply: tid=0 2024-05-09T02:16:45.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.432+0000 7f8a69bcbb80 1 client.16002 shutdown 2024-05-09T02:16:45.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.433+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/2025040766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf53ed400 msgr2=0x55dcf540b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:45.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.433+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/2025040766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf53ed400 0x55dcf540b9b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c006010 tx=0x7f8a4c01a050 comp rx=0 tx=0).stop 2024-05-09T02:16:45.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.433+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/2025040766 shutdown_connections 2024-05-09T02:16:45.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.433+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/2025040766 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55dcf5470cf0 0x55dcf5473110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.433+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/2025040766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcf53ed400 0x55dcf540b9b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.433+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/2025040766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcf52704a0 0x55dcf540af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.433+0000 7f8a69bcbb80 1 --2- 172.21.15.19:0/2025040766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcf52ead00 0x55dcf540b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.433+0000 7f8a33fff700 0 client.16002 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:16:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.433+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/2025040766 >> 172.21.15.19:0/2025040766 conn(0x55dcf5273f70 msgr2=0x55dcf54021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.433+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/2025040766 shutdown_connections 2024-05-09T02:16:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.433+0000 7f8a69bcbb80 1 -- 172.21.15.19:0/2025040766 wait complete. 2024-05-09T02:16:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.433+0000 7f8a69bcbb80 20 client.16002 trim_cache size 0 max 16384 2024-05-09T02:16:45.437 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:16:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.434+0000 7f8a5b1b5700 1 -- 172.21.15.19:0/3966242123 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8a400b2af0 msgr2=0x7f8a400b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.434+0000 7f8a5b1b5700 1 --2- 172.21.15.19:0/3966242123 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8a400b2af0 0x7f8a400b5000 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00b2e0 tx=0x7f8a4c005960 comp rx=0 tx=0).stop 2024-05-09T02:16:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.434+0000 7f8a5b1b5700 1 -- 172.21.15.19:0/3966242123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a5410c840 msgr2=0x7f8a54114530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.434+0000 7f8a5b1b5700 1 --2- 172.21.15.19:0/3966242123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a5410c840 0x7f8a54114530 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00e970 tx=0x7f8a3c00ec80 comp rx=0 tx=0).stop 2024-05-09T02:16:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.434+0000 7f8a5b1b5700 1 -- 172.21.15.19:0/3966242123 shutdown_connections 2024-05-09T02:16:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.434+0000 7f8a5b1b5700 1 --2- 172.21.15.19:0/3966242123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a54110fb0 0x7f8a54112f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.434+0000 7f8a5b1b5700 1 --2- 172.21.15.19:0/3966242123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a5410c840 0x7f8a54114530 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.434+0000 7f8a5b1b5700 1 --2- 172.21.15.19:0/3966242123 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8a400b2af0 0x7f8a400b5000 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.434+0000 7f8a5b1b5700 1 --2- 172.21.15.19:0/3966242123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a54110660 0x7f8a54112990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.434+0000 7f8a5b1b5700 1 -- 172.21.15.19:0/3966242123 >> 172.21.15.19:0/3966242123 conn(0x7f8a5406f4d0 msgr2=0x7f8a540747a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:45.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.436+0000 7f8a5b1b5700 1 -- 172.21.15.19:0/3966242123 shutdown_connections 2024-05-09T02:16:45.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:45.436+0000 7f8a5b1b5700 1 -- 172.21.15.19:0/3966242123 wait complete. 2024-05-09T02:16:46.230 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T02:16:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:46 smithi019 ceph-mon[30151]: pgmap v167: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.8 MiB/s wr, 900 op/s 2024-05-09T02:16:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:16:46.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:46 smithi059 ceph-mon[34781]: pgmap v167: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.8 MiB/s wr, 900 op/s 2024-05-09T02:16:46.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:46 smithi196 ceph-mon[34718]: pgmap v167: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.8 MiB/s wr, 900 op/s 2024-05-09T02:16:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 305.0 s, 2814.4 tps, lat 35.513 ms stddev 40.387 2024-05-09T02:16:48.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:48 smithi019 ceph-mon[30151]: pgmap v168: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 MiB/s wr, 584 op/s 2024-05-09T02:16:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:48 smithi059 ceph-mon[34781]: pgmap v168: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 MiB/s wr, 584 op/s 2024-05-09T02:16:48.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.763+0000 7efc71813700 1 -- 172.21.15.19:0/2186243899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc6c109860 msgr2=0x7efc6c1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:48.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.763+0000 7efc71813700 1 --2- 172.21.15.19:0/2186243899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc6c109860 0x7efc6c1111c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc6400b0a0 tx=0x7efc6400b3b0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.769+0000 7efc71813700 1 -- 172.21.15.19:0/2186243899 shutdown_connections 2024-05-09T02:16:48.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.769+0000 7efc71813700 1 --2- 172.21.15.19:0/2186243899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc6c109860 0x7efc6c1111c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.769+0000 7efc71813700 1 --2- 172.21.15.19:0/2186243899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c1045e0 0x7efc6c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.769+0000 7efc71813700 1 --2- 172.21.15.19:0/2186243899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc6c108f10 0x7efc6c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.769+0000 7efc71813700 1 -- 172.21.15.19:0/2186243899 >> 172.21.15.19:0/2186243899 conn(0x7efc6c0ffe70 msgr2=0x7efc6c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:48.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.771+0000 7efc71813700 1 -- 172.21.15.19:0/2186243899 shutdown_connections 2024-05-09T02:16:48.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.773+0000 7efc71813700 1 -- 172.21.15.19:0/2186243899 wait complete. 2024-05-09T02:16:48.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.774+0000 7efc71813700 1 Processor -- start 2024-05-09T02:16:48.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.775+0000 7efc71813700 1 -- start start 2024-05-09T02:16:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.776+0000 7efc71813700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc6c1045e0 0x7efc6c19fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.777+0000 7efc71813700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c108f10 0x7efc6c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.777+0000 7efc71813700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc6c109860 0x7efc6c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.777+0000 7efc71813700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c1123f0 con 0x7efc6c108f10 2024-05-09T02:16:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.777+0000 7efc71813700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c112270 con 0x7efc6c1045e0 2024-05-09T02:16:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.777+0000 7efc71813700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c112570 con 0x7efc6c109860 2024-05-09T02:16:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.777+0000 7efc6a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c108f10 0x7efc6c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.777+0000 7efc6a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c108f10 0x7efc6c198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34732/0 (socket says 172.21.15.19:34732) 2024-05-09T02:16:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.777+0000 7efc6a7fc700 1 -- 172.21.15.19:0/4124667947 learned_addr learned my addr 172.21.15.19:0/4124667947 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.777+0000 7efc6affd700 1 --2- 172.21.15.19:0/4124667947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc6c1045e0 0x7efc6c19fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.780+0000 7efc6a7fc700 1 -- 172.21.15.19:0/4124667947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc6c109860 msgr2=0x7efc6c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.780+0000 7efc6a7fc700 1 --2- 172.21.15.19:0/4124667947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc6c109860 0x7efc6c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.780+0000 7efc6a7fc700 1 -- 172.21.15.19:0/4124667947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc6c1045e0 msgr2=0x7efc6c19fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.780+0000 7efc6a7fc700 1 --2- 172.21.15.19:0/4124667947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc6c1045e0 0x7efc6c19fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.780+0000 7efc6a7fc700 1 -- 172.21.15.19:0/4124667947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6c19d5e0 con 0x7efc6c108f10 2024-05-09T02:16:48.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.784+0000 7efc6a7fc700 1 --2- 172.21.15.19:0/4124667947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c108f10 0x7efc6c198be0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00b550 tx=0x7efc5c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:48.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.785+0000 7efc70811700 1 -- 172.21.15.19:0/4124667947 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c015020 con 0x7efc6c108f10 2024-05-09T02:16:48.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.785+0000 7efc70811700 1 -- 172.21.15.19:0/4124667947 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc5c011ce0 con 0x7efc6c108f10 2024-05-09T02:16:48.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.785+0000 7efc71813700 1 -- 172.21.15.19:0/4124667947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6c19d8d0 con 0x7efc6c108f10 2024-05-09T02:16:48.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.785+0000 7efc71813700 1 -- 172.21.15.19:0/4124667947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6c19de10 con 0x7efc6c108f10 2024-05-09T02:16:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.785+0000 7efc70811700 1 -- 172.21.15.19:0/4124667947 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c010cf0 con 0x7efc6c108f10 2024-05-09T02:16:48.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:48 smithi196 ceph-mon[34718]: pgmap v168: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 MiB/s wr, 584 op/s 2024-05-09T02:16:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.795+0000 7efc80229b80 1 Processor -- start 2024-05-09T02:16:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.795+0000 7efc80229b80 1 -- start start 2024-05-09T02:16:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.796+0000 7efc70811700 1 -- 172.21.15.19:0/4124667947 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efc5c018020 con 0x7efc6c108f10 2024-05-09T02:16:48.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.797+0000 7efc70811700 1 --2- 172.21.15.19:0/4124667947 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc540b2b80 0x7efc540b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.798+0000 7efc70811700 1 -- 172.21.15.19:0/4124667947 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7efc5c0e1b50 con 0x7efc6c108f10 2024-05-09T02:16:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.798+0000 7efc80229b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d06160 0x55adc0d9f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.798+0000 7efc80229b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0c1c4a0 0x55adc0d9fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.798+0000 7efc80229b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0d99ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.798+0000 7efc80229b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55adc0cbbff0 con 0x55adc0d06160 2024-05-09T02:16:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.798+0000 7efc80229b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55adc0c5a7f0 con 0x55adc0d996a0 2024-05-09T02:16:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.799+0000 7efc80229b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55adc0cb7fb0 con 0x55adc0c1c4a0 2024-05-09T02:16:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.799+0000 7efc6b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0d99ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.799+0000 7efc6b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0d99ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:36520/0 (socket says 172.21.15.19:36520) 2024-05-09T02:16:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.799+0000 7efc6b7fe700 1 -- 172.21.15.19:0/2883119593 learned_addr learned my addr 172.21.15.19:0/2883119593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.800+0000 7efc6b7fe700 1 --2- 172.21.15.19:0/4124667947 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc540b2b80 0x7efc540b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.800+0000 7efc6b7fe700 1 -- 172.21.15.19:0/2883119593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0c1c4a0 msgr2=0x55adc0d9fa30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:16:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.800+0000 7efc6affd700 1 --2- 172.21.15.19:0/2883119593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d06160 0x55adc0d9f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.800+0000 7efc6b7fe700 1 --2- 172.21.15.19:0/2883119593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0c1c4a0 0x55adc0d9fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.800+0000 7efc6b7fe700 1 -- 172.21.15.19:0/2883119593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d06160 msgr2=0x55adc0d9f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.800+0000 7efc6b7fe700 1 --2- 172.21.15.19:0/2883119593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d06160 0x55adc0d9f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.800+0000 7efc6b7fe700 1 -- 172.21.15.19:0/2883119593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc0c52370 con 0x55adc0d996a0 2024-05-09T02:16:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.800+0000 7efc6affd700 1 --2- 172.21.15.19:0/2883119593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d06160 0x55adc0d9f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:16:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.801+0000 7efc6b7fe700 1 --2- 172.21.15.19:0/2883119593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0d99ad0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efc6400b070 tx=0x7efc640095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ee4344f14bde9f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.801+0000 7efc51ffb700 1 -- 172.21.15.19:0/2883119593 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64008b60 con 0x55adc0d996a0 2024-05-09T02:16:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.801+0000 7efc6b7fe700 1 --2- 172.21.15.19:0/4124667947 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc540b2b80 0x7efc540b5090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efc64025000 tx=0x7efc64023000 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.801+0000 7efc51ffb700 1 -- 172.21.15.19:0/2883119593 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc64008d00 con 0x55adc0d996a0 2024-05-09T02:16:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.801+0000 7efc51ffb700 1 -- 172.21.15.19:0/2883119593 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6401e430 con 0x55adc0d996a0 2024-05-09T02:16:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.803+0000 7efc80229b80 1 -- 172.21.15.19:0/2883119593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 msgr2=0x55adc0d99ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.803+0000 7efc80229b80 1 --2- 172.21.15.19:0/2883119593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0d99ad0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efc6400b070 tx=0x7efc640095a0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.803+0000 7efc80229b80 1 -- 172.21.15.19:0/2883119593 shutdown_connections 2024-05-09T02:16:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.803+0000 7efc80229b80 1 --2- 172.21.15.19:0/2883119593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0d99ad0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.803+0000 7efc80229b80 1 --2- 172.21.15.19:0/2883119593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d06160 0x55adc0d9f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.803+0000 7efc80229b80 1 --2- 172.21.15.19:0/2883119593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0c1c4a0 0x55adc0d9fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.803+0000 7efc80229b80 1 -- 172.21.15.19:0/2883119593 >> 172.21.15.19:0/2883119593 conn(0x55adc0c7c6a0 msgr2=0x55adc0d94400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.806+0000 7efc80229b80 1 -- 172.21.15.19:0/2883119593 shutdown_connections 2024-05-09T02:16:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.806+0000 7efc80229b80 1 -- 172.21.15.19:0/2883119593 wait complete. 2024-05-09T02:16:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.806+0000 7efc80229b80 1 -- start start 2024-05-09T02:16:48.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.807+0000 7efc80229b80 1 Processor -- start 2024-05-09T02:16:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.808+0000 7efc80229b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0c1c4a0 0x55adc0da3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.808+0000 7efc80229b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0d06160 0x55adc0da37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.808+0000 7efc6affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0c1c4a0 0x55adc0da3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.808+0000 7efc6affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0c1c4a0 0x55adc0da3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:36534/0 (socket says 172.21.15.19:36534) 2024-05-09T02:16:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.808+0000 7efc6affd700 1 -- 172.21.15.19:0/3616189637 learned_addr learned my addr 172.21.15.19:0/3616189637 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.808+0000 7efc80229b80 1 --2- 172.21.15.19:0/3616189637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d996a0 0x55adc0da3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.808+0000 7efc6a7fc700 1 --2- 172.21.15.19:0/3616189637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0d06160 0x55adc0da37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.809+0000 7efc80229b80 1 -- 172.21.15.19:0/3616189637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55adc0b37ba0 con 0x55adc0d996a0 2024-05-09T02:16:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.809+0000 7efc6b7fe700 1 --2- 172.21.15.19:0/3616189637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d996a0 0x55adc0da3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.809+0000 7efc80229b80 1 -- 172.21.15.19:0/3616189637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55adc0c76150 con 0x55adc0c1c4a0 2024-05-09T02:16:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.809+0000 7efc80229b80 1 -- 172.21.15.19:0/3616189637 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55adc0da61d0 con 0x55adc0d06160 2024-05-09T02:16:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.809+0000 7efc80229b80 10 client.0 _ll_register_callbacks cb 0x55adc0b2d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.809+0000 7efc6affd700 1 -- 172.21.15.19:0/3616189637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0d06160 msgr2=0x55adc0da37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.809+0000 7efc6affd700 1 --2- 172.21.15.19:0/3616189637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0d06160 0x55adc0da37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.809+0000 7efc6affd700 1 -- 172.21.15.19:0/3616189637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d996a0 msgr2=0x55adc0da3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.809+0000 7efc6affd700 1 --2- 172.21.15.19:0/3616189637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d996a0 0x55adc0da3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.809+0000 7efc6affd700 1 -- 172.21.15.19:0/3616189637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc0b37490 con 0x55adc0c1c4a0 2024-05-09T02:16:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.810+0000 7efc6affd700 1 --2- 172.21.15.19:0/3616189637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0c1c4a0 0x55adc0da3260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efc6000e270 tx=0x7efc6000e580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c21c035071d5ee3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.810+0000 7efc80229b80 1 -- 172.21.15.19:0/3616189637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc64008d00 con 0x55adc0c1c4a0 2024-05-09T02:16:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.811+0000 7efc4affd700 10 client.25712 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:16:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.811+0000 7efc4affd700 1 -- 172.21.15.19:0/3616189637 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60017020 con 0x55adc0c1c4a0 2024-05-09T02:16:48.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.811+0000 7efc4affd700 1 -- 172.21.15.19:0/3616189637 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7efc60013410 con 0x55adc0c1c4a0 2024-05-09T02:16:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.811+0000 7efc4affd700 1 -- 172.21.15.19:0/3616189637 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7efc60022050 con 0x55adc0c1c4a0 2024-05-09T02:16:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.812+0000 7efc80229b80 10 client.25712 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.812+0000 7efc80229b80 1 -- 172.21.15.19:0/3616189637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=41}) -- 0x55adc0db0be0 con 0x55adc0c1c4a0 2024-05-09T02:16:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.812+0000 7efc4affd700 1 -- 172.21.15.19:0/3616189637 <== mon.2 v2:172.21.15.59:3300/0 4 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7efc60012050 con 0x55adc0c1c4a0 2024-05-09T02:16:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.812+0000 7efc80229b80 10 client.25712 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.812+0000 7efc80229b80 10 client.25712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:16:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.812+0000 7efc80229b80 20 client.25712 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.812+0000 7efc80229b80 1 --2- 172.21.15.19:0/3616189637 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55adc0db30b0 0x55adc0db54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.812+0000 7efc80229b80 4 client.25712 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:16:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.812+0000 7efc80229b80 1 -- 172.21.15.19:0/3616189637 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adc0cc99d0 con 0x55adc0db30b0 2024-05-09T02:16:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.813+0000 7efc6a7fc700 1 --2- 172.21.15.19:0/3616189637 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55adc0db30b0 0x55adc0db54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.813+0000 7efc4affd700 10 client.25712 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:48.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.813+0000 7efc6a7fc700 1 --2- 172.21.15.19:0/3616189637 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55adc0db30b0 0x55adc0db54d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=908e84e4b735b75f server_cookie=758720d21edcb380 in_seq=0 out_seq=0 2024-05-09T02:16:48.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.814+0000 7efc4affd700 1 -- 172.21.15.19:0/3616189637 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55adc0cc99d0 con 0x55adc0db30b0 2024-05-09T02:16:48.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.814+0000 7efc4affd700 10 client.25712 handle_command_reply: tid=0 2024-05-09T02:16:48.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.814+0000 7efc80229b80 1 client.25712 shutdown 2024-05-09T02:16:48.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.814+0000 7efc80229b80 1 -- 172.21.15.19:0/3616189637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0c1c4a0 msgr2=0x55adc0da3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:48.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.814+0000 7efc80229b80 1 --2- 172.21.15.19:0/3616189637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0c1c4a0 0x55adc0da3260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efc6000e270 tx=0x7efc6000e580 comp rx=0 tx=0).stop 2024-05-09T02:16:48.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.815+0000 7efc80229b80 1 -- 172.21.15.19:0/3616189637 shutdown_connections 2024-05-09T02:16:48.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.815+0000 7efc80229b80 1 --2- 172.21.15.19:0/3616189637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d996a0 0x55adc0da3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.815+0000 7efc80229b80 1 --2- 172.21.15.19:0/3616189637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0c1c4a0 0x55adc0da3260 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.815+0000 7efc80229b80 1 --2- 172.21.15.19:0/3616189637 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55adc0db30b0 0x55adc0db54d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.815+0000 7efc80229b80 1 --2- 172.21.15.19:0/3616189637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0d06160 0x55adc0da37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.815+0000 7efc80229b80 1 -- 172.21.15.19:0/3616189637 >> 172.21.15.19:0/3616189637 conn(0x55adc0c7c6a0 msgr2=0x55adc0d941c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:48.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.815+0000 7efc4affd700 0 client.25712 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:48.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.815+0000 7efc80229b80 1 -- 172.21.15.19:0/3616189637 shutdown_connections 2024-05-09T02:16:48.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.815+0000 7efc80229b80 1 -- 172.21.15.19:0/3616189637 wait complete. 2024-05-09T02:16:48.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.815+0000 7efc80229b80 20 client.25712 trim_cache size 0 max 16384 2024-05-09T02:16:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.835+0000 7efc80229b80 1 Processor -- start 2024-05-09T02:16:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.835+0000 7efc80229b80 1 -- start start 2024-05-09T02:16:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.835+0000 7efc80229b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0c1c4a0 0x55adc0d9ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.835+0000 7efc80229b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0d06160 0x55adc0da4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.835+0000 7efc80229b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0da4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.835+0000 7efc80229b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc5800d6d0 con 0x55adc0c1c4a0 2024-05-09T02:16:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.835+0000 7efc80229b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efc5800f2d0 con 0x55adc0d996a0 2024-05-09T02:16:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.835+0000 7efc80229b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efc5800dda0 con 0x55adc0d06160 2024-05-09T02:16:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.835+0000 7efc6b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0da4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.835+0000 7efc6a7fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0d06160 0x55adc0da4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.835+0000 7efc6a7fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0d06160 0x55adc0da4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:38814/0 (socket says 172.21.15.19:38814) 2024-05-09T02:16:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.835+0000 7efc6b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0da4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:36540/0 (socket says 172.21.15.19:36540) 2024-05-09T02:16:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.835+0000 7efc6a7fc700 1 -- 172.21.15.19:0/1571820772 learned_addr learned my addr 172.21.15.19:0/1571820772 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.835+0000 7efc6affd700 1 --2- 172.21.15.19:0/1571820772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0c1c4a0 0x55adc0d9ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.836+0000 7efc6b7fe700 1 -- 172.21.15.19:0/1571820772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0d06160 msgr2=0x55adc0da4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.836+0000 7efc6b7fe700 1 --2- 172.21.15.19:0/1571820772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0d06160 0x55adc0da4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.836+0000 7efc6b7fe700 1 -- 172.21.15.19:0/1571820772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0c1c4a0 msgr2=0x55adc0d9ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.836+0000 7efc6b7fe700 1 --2- 172.21.15.19:0/1571820772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0c1c4a0 0x55adc0d9ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.836+0000 7efc6b7fe700 1 -- 172.21.15.19:0/1571820772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc0da5b00 con 0x55adc0d996a0 2024-05-09T02:16:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.836+0000 7efc6b7fe700 1 --2- 172.21.15.19:0/1571820772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0da4e50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc64005bd0 tx=0x7efc64003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdb5243727e80064 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.836+0000 7efc517fa700 1 -- 172.21.15.19:0/1571820772 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64008b60 con 0x55adc0d996a0 2024-05-09T02:16:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.836+0000 7efc517fa700 1 -- 172.21.15.19:0/1571820772 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc64009cc0 con 0x55adc0d996a0 2024-05-09T02:16:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.836+0000 7efc517fa700 1 -- 172.21.15.19:0/1571820772 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6401e430 con 0x55adc0d996a0 2024-05-09T02:16:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.837+0000 7efc80229b80 1 -- 172.21.15.19:0/1571820772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 msgr2=0x55adc0da4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.837+0000 7efc80229b80 1 --2- 172.21.15.19:0/1571820772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0da4e50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc64005bd0 tx=0x7efc64003b20 comp rx=0 tx=0).stop 2024-05-09T02:16:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.837+0000 7efc80229b80 1 -- 172.21.15.19:0/1571820772 shutdown_connections 2024-05-09T02:16:48.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.838+0000 7efc80229b80 1 --2- 172.21.15.19:0/1571820772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0da4e50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.838+0000 7efc80229b80 1 --2- 172.21.15.19:0/1571820772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0c1c4a0 0x55adc0d9ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.838+0000 7efc80229b80 1 --2- 172.21.15.19:0/1571820772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0d06160 0x55adc0da4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.838+0000 7efc80229b80 1 -- 172.21.15.19:0/1571820772 >> 172.21.15.19:0/1571820772 conn(0x55adc0c7c6a0 msgr2=0x55adc0d9e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:48.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.838+0000 7efc80229b80 1 -- 172.21.15.19:0/1571820772 shutdown_connections 2024-05-09T02:16:48.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.838+0000 7efc80229b80 1 -- 172.21.15.19:0/1571820772 wait complete. 2024-05-09T02:16:48.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.838+0000 7efc80229b80 1 -- start start 2024-05-09T02:16:48.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.839+0000 7efc80229b80 1 Processor -- start 2024-05-09T02:16:48.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.839+0000 7efc80229b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0c1c4a0 0x55adc0dab420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.839+0000 7efc80229b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d06160 0x55adc0dab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.840+0000 7efc80229b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0db9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.840+0000 7efc6a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d06160 0x55adc0dab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.840+0000 7efc6a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d06160 0x55adc0dab960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34778/0 (socket says 172.21.15.19:34778) 2024-05-09T02:16:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.840+0000 7efc6a7fc700 1 -- 172.21.15.19:0/2896300930 learned_addr learned my addr 172.21.15.19:0/2896300930 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.840+0000 7efc6affd700 1 --2- 172.21.15.19:0/2896300930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0c1c4a0 0x55adc0dab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.840+0000 7efc6b7fe700 1 --2- 172.21.15.19:0/2896300930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0db9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.840+0000 7efc80229b80 1 -- 172.21.15.19:0/2896300930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc5800ec00 con 0x55adc0d06160 2024-05-09T02:16:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.840+0000 7efc80229b80 1 -- 172.21.15.19:0/2896300930 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efc5800e470 con 0x55adc0d996a0 2024-05-09T02:16:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.840+0000 7efc80229b80 1 -- 172.21.15.19:0/2896300930 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efc5800b670 con 0x55adc0c1c4a0 2024-05-09T02:16:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.841+0000 7efc80229b80 10 client.0 _ll_register_callbacks cb 0x55adc0b2d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.841+0000 7efc6a7fc700 1 -- 172.21.15.19:0/2896300930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0c1c4a0 msgr2=0x55adc0dab420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.841+0000 7efc6a7fc700 1 --2- 172.21.15.19:0/2896300930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0c1c4a0 0x55adc0dab420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.841+0000 7efc6a7fc700 1 -- 172.21.15.19:0/2896300930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 msgr2=0x55adc0db9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.841+0000 7efc6a7fc700 1 --2- 172.21.15.19:0/2896300930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0db9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.841+0000 7efc6a7fc700 1 -- 172.21.15.19:0/2896300930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc0dba180 con 0x55adc0d06160 2024-05-09T02:16:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.841+0000 7efc6b7fe700 1 --2- 172.21.15.19:0/2896300930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0db9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:16:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.841+0000 7efc6affd700 1 --2- 172.21.15.19:0/2896300930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0c1c4a0 0x55adc0dab420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:16:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.841+0000 7efc6a7fc700 1 --2- 172.21.15.19:0/2896300930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d06160 0x55adc0dab960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7efc5c010860 tx=0x7efc5c010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1dc5ef88e791d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.841+0000 7efc4affd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:16:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.841+0000 7efc4affd700 1 -- 172.21.15.19:0/2896300930 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c0ad290 con 0x55adc0d06160 2024-05-09T02:16:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.841+0000 7efc80229b80 1 -- 172.21.15.19:0/2896300930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc64009cc0 con 0x55adc0d06160 2024-05-09T02:16:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.842+0000 7efc4affd700 1 -- 172.21.15.19:0/2896300930 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7efc5c0a9350 con 0x55adc0d06160 2024-05-09T02:16:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.842+0000 7efc4affd700 1 -- 172.21.15.19:0/2896300930 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7efc5c0b1d40 con 0x55adc0d06160 2024-05-09T02:16:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.842+0000 7efc80229b80 10 client.16014 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.842+0000 7efc80229b80 1 -- 172.21.15.19:0/2896300930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55adc0e1cf30 con 0x55adc0d06160 2024-05-09T02:16:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.842+0000 7efc4affd700 1 -- 172.21.15.19:0/2896300930 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc64009cc0 con 0x55adc0d06160 2024-05-09T02:16:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.842+0000 7efc4affd700 1 -- 172.21.15.19:0/2896300930 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7efc5c0a8250 con 0x55adc0d06160 2024-05-09T02:16:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.842+0000 7efc80229b80 10 client.16014 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.842+0000 7efc80229b80 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:16:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.842+0000 7efc80229b80 20 client.16014 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.842+0000 7efc80229b80 1 --2- 172.21.15.19:0/2896300930 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55adc0e1d5c0 0x55adc0e1f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.842+0000 7efc80229b80 4 client.16014 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:16:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.842+0000 7efc80229b80 1 -- 172.21.15.19:0/2896300930 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55adc0cc3a70 con 0x55adc0e1d5c0 2024-05-09T02:16:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.843+0000 7efc6affd700 1 --2- 172.21.15.19:0/2896300930 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55adc0e1d5c0 0x55adc0e1f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.843+0000 7efc4affd700 10 client.16014 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.844+0000 7efc6affd700 1 --2- 172.21.15.19:0/2896300930 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55adc0e1d5c0 0x55adc0e1f9e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f45bbd9d949034d0 server_cookie=2a4f18a5347a3d64 in_seq=0 out_seq=0 2024-05-09T02:16:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.844+0000 7efc4affd700 1 -- 172.21.15.19:0/2896300930 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55adc0cc3a70 con 0x55adc0e1d5c0 2024-05-09T02:16:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.844+0000 7efc4affd700 10 client.16014 handle_command_reply: tid=0 2024-05-09T02:16:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.844+0000 7efc80229b80 1 client.16014 shutdown 2024-05-09T02:16:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.845+0000 7efc80229b80 1 -- 172.21.15.19:0/2896300930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d06160 msgr2=0x55adc0dab960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.845+0000 7efc80229b80 1 --2- 172.21.15.19:0/2896300930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d06160 0x55adc0dab960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7efc5c010860 tx=0x7efc5c010ee0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.845+0000 7efc80229b80 1 -- 172.21.15.19:0/2896300930 shutdown_connections 2024-05-09T02:16:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.845+0000 7efc80229b80 1 --2- 172.21.15.19:0/2896300930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adc0d996a0 0x55adc0db9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.845+0000 7efc80229b80 1 --2- 172.21.15.19:0/2896300930 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55adc0e1d5c0 0x55adc0e1f9e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.845+0000 7efc80229b80 1 --2- 172.21.15.19:0/2896300930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc0c1c4a0 0x55adc0dab420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.845+0000 7efc80229b80 1 --2- 172.21.15.19:0/2896300930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adc0d06160 0x55adc0dab960 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.845+0000 7efc80229b80 1 -- 172.21.15.19:0/2896300930 >> 172.21.15.19:0/2896300930 conn(0x55adc0c7c6a0 msgr2=0x55adc0d98690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.845+0000 7efc4affd700 0 client.16014 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.845+0000 7efc80229b80 1 -- 172.21.15.19:0/2896300930 shutdown_connections 2024-05-09T02:16:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.845+0000 7efc80229b80 1 -- 172.21.15.19:0/2896300930 wait complete. 2024-05-09T02:16:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.845+0000 7efc80229b80 20 client.16014 trim_cache size 0 max 16384 2024-05-09T02:16:48.849 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:16:48.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.846+0000 7efc71813700 1 -- 172.21.15.19:0/4124667947 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc540b2b80 msgr2=0x7efc540b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:48.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.846+0000 7efc71813700 1 --2- 172.21.15.19:0/4124667947 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc540b2b80 0x7efc540b5090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efc64025000 tx=0x7efc64023000 comp rx=0 tx=0).stop 2024-05-09T02:16:48.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.846+0000 7efc71813700 1 -- 172.21.15.19:0/4124667947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c108f10 msgr2=0x7efc6c198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:48.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.846+0000 7efc71813700 1 --2- 172.21.15.19:0/4124667947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c108f10 0x7efc6c198be0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00b550 tx=0x7efc5c00b910 comp rx=0 tx=0).stop 2024-05-09T02:16:48.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.846+0000 7efc71813700 1 -- 172.21.15.19:0/4124667947 shutdown_connections 2024-05-09T02:16:48.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.846+0000 7efc71813700 1 --2- 172.21.15.19:0/4124667947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc6c109860 0x7efc6c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.846+0000 7efc71813700 1 --2- 172.21.15.19:0/4124667947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc6c1045e0 0x7efc6c19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.846+0000 7efc71813700 1 --2- 172.21.15.19:0/4124667947 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efc540b2b80 0x7efc540b5090 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.846+0000 7efc71813700 1 --2- 172.21.15.19:0/4124667947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c108f10 0x7efc6c198be0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:48.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.846+0000 7efc71813700 1 -- 172.21.15.19:0/4124667947 >> 172.21.15.19:0/4124667947 conn(0x7efc6c0ffe70 msgr2=0x7efc6c107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:48.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.846+0000 7efc71813700 1 -- 172.21.15.19:0/4124667947 shutdown_connections 2024-05-09T02:16:48.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:48.846+0000 7efc71813700 1 -- 172.21.15.19:0/4124667947 wait complete. 2024-05-09T02:16:50.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:50 smithi059 ceph-mon[34781]: pgmap v169: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 886 op/s 2024-05-09T02:16:50.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:50 smithi196 ceph-mon[34718]: pgmap v169: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 886 op/s 2024-05-09T02:16:50.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:50 smithi019 ceph-mon[30151]: pgmap v169: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 886 op/s 2024-05-09T02:16:51.050 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T02:16:51.051 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T02:16:51.448 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:16:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 310.0 s, 2248.0 tps, lat 44.689 ms stddev 50.057 2024-05-09T02:16:52.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:52 smithi059 ceph-mon[34781]: pgmap v170: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.5 MiB/s wr, 590 op/s 2024-05-09T02:16:52.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:52 smithi196 ceph-mon[34718]: pgmap v170: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.5 MiB/s wr, 590 op/s 2024-05-09T02:16:52.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:52 smithi019 ceph-mon[30151]: pgmap v170: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.5 MiB/s wr, 590 op/s 2024-05-09T02:16:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.318+0000 7ff7a6d06700 1 -- 172.21.15.19:0/4151278567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7a01034e0 msgr2=0x7ff7a01038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.318+0000 7ff7a6d06700 1 --2- 172.21.15.19:0/4151278567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7a01034e0 0x7ff7a01038f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff7940099c0 tx=0x7ff794009cd0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.318+0000 7ff7a6d06700 1 -- 172.21.15.19:0/4151278567 shutdown_connections 2024-05-09T02:16:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.318+0000 7ff7a6d06700 1 --2- 172.21.15.19:0/4151278567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7a0108760 0x7ff7a01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.318+0000 7ff7a6d06700 1 --2- 172.21.15.19:0/4151278567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7a01034e0 0x7ff7a01038f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.318+0000 7ff7a6d06700 1 --2- 172.21.15.19:0/4151278567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7a0107e10 0x7ff7a0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.318+0000 7ff7a6d06700 1 -- 172.21.15.19:0/4151278567 >> 172.21.15.19:0/4151278567 conn(0x7ff7a00fef30 msgr2=0x7ff7a0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:53.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.318+0000 7ff7a6d06700 1 -- 172.21.15.19:0/4151278567 shutdown_connections 2024-05-09T02:16:53.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.318+0000 7ff7a6d06700 1 -- 172.21.15.19:0/4151278567 wait complete. 2024-05-09T02:16:53.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.320+0000 7ff7a6d06700 1 Processor -- start 2024-05-09T02:16:53.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.320+0000 7ff7a6d06700 1 -- start start 2024-05-09T02:16:53.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.321+0000 7ff7a6d06700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7a01034e0 0x7ff7a0075460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.321+0000 7ff7a6d06700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7a0107e10 0x7ff7a006e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.321+0000 7ff7a6d06700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7a0108760 0x7ff7a006e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.321+0000 7ff7a6d06700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01133b0 con 0x7ff7a0107e10 2024-05-09T02:16:53.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.321+0000 7ff7a6d06700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a0113230 con 0x7ff7a0108760 2024-05-09T02:16:53.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.321+0000 7ff7a6d06700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a0113530 con 0x7ff7a01034e0 2024-05-09T02:16:53.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.321+0000 7ff7a52a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7a0108760 0x7ff7a006e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.321+0000 7ff7a52a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7a0108760 0x7ff7a006e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:36574/0 (socket says 172.21.15.19:36574) 2024-05-09T02:16:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.321+0000 7ff7a52a3700 1 -- 172.21.15.19:0/3450578308 learned_addr learned my addr 172.21.15.19:0/3450578308 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.321+0000 7ff7a4aa2700 1 --2- 172.21.15.19:0/3450578308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7a01034e0 0x7ff7a0075460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.322+0000 7ff7a52a3700 1 -- 172.21.15.19:0/3450578308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7a01034e0 msgr2=0x7ff7a0075460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.322+0000 7ff7a52a3700 1 --2- 172.21.15.19:0/3450578308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7a01034e0 0x7ff7a0075460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.322+0000 7ff7a52a3700 1 -- 172.21.15.19:0/3450578308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7a0107e10 msgr2=0x7ff7a006e400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:16:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.322+0000 7ff7a52a3700 1 --2- 172.21.15.19:0/3450578308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7a0107e10 0x7ff7a006e400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.322+0000 7ff7a52a3700 1 -- 172.21.15.19:0/3450578308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a006f1b0 con 0x7ff7a0108760 2024-05-09T02:16:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.323+0000 7ff7a52a3700 1 --2- 172.21.15.19:0/3450578308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7a0108760 0x7ff7a006e980 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff79800ef30 tx=0x7ff79800c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.323+0000 7ff79dffb700 1 -- 172.21.15.19:0/3450578308 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980092e0 con 0x7ff7a0108760 2024-05-09T02:16:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.323+0000 7ff79dffb700 1 -- 172.21.15.19:0/3450578308 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff798004bb0 con 0x7ff7a0108760 2024-05-09T02:16:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.323+0000 7ff7a6d06700 1 -- 172.21.15.19:0/3450578308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a01b6120 con 0x7ff7a0108760 2024-05-09T02:16:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.323+0000 7ff79dffb700 1 -- 172.21.15.19:0/3450578308 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798004280 con 0x7ff7a0108760 2024-05-09T02:16:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.323+0000 7ff7a6d06700 1 -- 172.21.15.19:0/3450578308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a01b65c0 con 0x7ff7a0108760 2024-05-09T02:16:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.326+0000 7ff79dffb700 1 -- 172.21.15.19:0/3450578308 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff79801b030 con 0x7ff7a0108760 2024-05-09T02:16:53.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.326+0000 7ff79dffb700 1 --2- 172.21.15.19:0/3450578308 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff7900b2af0 0x7ff7900b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.327+0000 7ff79dffb700 1 -- 172.21.15.19:0/3450578308 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7ff7980e21b0 con 0x7ff7a0108760 2024-05-09T02:16:53.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.327+0000 7ff7a4aa2700 1 --2- 172.21.15.19:0/3450578308 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff7900b2af0 0x7ff7900b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.327+0000 7ff7b571cb80 1 Processor -- start 2024-05-09T02:16:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.327+0000 7ff7a4aa2700 1 --2- 172.21.15.19:0/3450578308 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff7900b2af0 0x7ff7900b5000 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff794003760 tx=0x7ff79401a040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.327+0000 7ff7b571cb80 1 -- start start 2024-05-09T02:16:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7b571cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2c03d80 0x5578b2d0c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7b571cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2b89520 0x5578b2d0c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7b571cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2d06480 0x5578b2d06890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7b571cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5578b2c29120 con 0x5578b2c03d80 2024-05-09T02:16:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7b571cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5578b2bcea20 con 0x5578b2d06480 2024-05-09T02:16:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7b571cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5578b2c250e0 con 0x5578b2b89520 2024-05-09T02:16:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7a4aa2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2c03d80 0x5578b2d0c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7a52a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2d06480 0x5578b2d06890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7a52a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2d06480 0x5578b2d06890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:36582/0 (socket says 172.21.15.19:36582) 2024-05-09T02:16:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7a4aa2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2c03d80 0x5578b2d0c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34784/0 (socket says 172.21.15.19:34784) 2024-05-09T02:16:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7a52a3700 1 -- 172.21.15.19:0/4227693197 learned_addr learned my addr 172.21.15.19:0/4227693197 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff79ffff700 1 --2- 172.21.15.19:0/4227693197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2b89520 0x5578b2d0c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7a52a3700 1 -- 172.21.15.19:0/4227693197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2b89520 msgr2=0x5578b2d0c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7a52a3700 1 --2- 172.21.15.19:0/4227693197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2b89520 0x5578b2d0c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7a52a3700 1 -- 172.21.15.19:0/4227693197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2c03d80 msgr2=0x5578b2d0c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7a52a3700 1 --2- 172.21.15.19:0/4227693197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2c03d80 0x5578b2d0c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7a52a3700 1 -- 172.21.15.19:0/4227693197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b2bbf580 con 0x5578b2d06480 2024-05-09T02:16:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.328+0000 7ff7a52a3700 1 --2- 172.21.15.19:0/4227693197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2d06480 0x5578b2d06890 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff7980df840 tx=0x7ff7980a93c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94d084bab834733f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:53.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.329+0000 7ff78affd700 1 -- 172.21.15.19:0/4227693197 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980a94c0 con 0x5578b2d06480 2024-05-09T02:16:53.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.329+0000 7ff78affd700 1 -- 172.21.15.19:0/4227693197 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7980ad870 con 0x5578b2d06480 2024-05-09T02:16:53.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.329+0000 7ff78affd700 1 -- 172.21.15.19:0/4227693197 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980a9720 con 0x5578b2d06480 2024-05-09T02:16:53.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.329+0000 7ff7b571cb80 1 -- 172.21.15.19:0/4227693197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2d06480 msgr2=0x5578b2d06890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:53.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.329+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/4227693197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2d06480 0x5578b2d06890 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff7980df840 tx=0x7ff7980a93c0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.330+0000 7ff7b571cb80 1 -- 172.21.15.19:0/4227693197 shutdown_connections 2024-05-09T02:16:53.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.330+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/4227693197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2d06480 0x5578b2d06890 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.330+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/4227693197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2c03d80 0x5578b2d0c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.330+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/4227693197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2b89520 0x5578b2d0c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.330+0000 7ff7b571cb80 1 -- 172.21.15.19:0/4227693197 >> 172.21.15.19:0/4227693197 conn(0x5578b2b8cff0 msgr2=0x5578b2d012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:53.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.330+0000 7ff7b571cb80 1 -- 172.21.15.19:0/4227693197 shutdown_connections 2024-05-09T02:16:53.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.330+0000 7ff7b571cb80 1 -- 172.21.15.19:0/4227693197 wait complete. 2024-05-09T02:16:53.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.330+0000 7ff7b571cb80 1 -- start start 2024-05-09T02:16:53.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.330+0000 7ff7b571cb80 1 Processor -- start 2024-05-09T02:16:53.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.331+0000 7ff7b571cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2b89520 0x5578b2d10110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.331+0000 7ff7b571cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2c03d80 0x5578b2d10650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.332+0000 7ff7b571cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2d06480 0x5578b2d10b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.332+0000 7ff7b571cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5578b2d071e0 con 0x5578b2b89520 2024-05-09T02:16:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.332+0000 7ff7b571cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5578b2bc3ba0 con 0x5578b2d06480 2024-05-09T02:16:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.332+0000 7ff7b571cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5578b2d130a0 con 0x5578b2c03d80 2024-05-09T02:16:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.331+0000 7ff7a4aa2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2b89520 0x5578b2d10110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.333+0000 7ff7a4aa2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2b89520 0x5578b2d10110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34788/0 (socket says 172.21.15.19:34788) 2024-05-09T02:16:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.333+0000 7ff7a4aa2700 1 -- 172.21.15.19:0/430362474 learned_addr learned my addr 172.21.15.19:0/430362474 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.333+0000 7ff7b571cb80 10 client.0 _ll_register_callbacks cb 0x5578b2a9a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.331+0000 7ff79ffff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2c03d80 0x5578b2d10650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.333+0000 7ff7a52a3700 1 --2- 172.21.15.19:0/430362474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2d06480 0x5578b2d10b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.333+0000 7ff79ffff700 1 -- 172.21.15.19:0/430362474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2d06480 msgr2=0x5578b2d10b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:53.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.333+0000 7ff79ffff700 1 --2- 172.21.15.19:0/430362474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2d06480 0x5578b2d10b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.333+0000 7ff79ffff700 1 -- 172.21.15.19:0/430362474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2b89520 msgr2=0x5578b2d10110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:53.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.333+0000 7ff79ffff700 1 --2- 172.21.15.19:0/430362474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2b89520 0x5578b2d10110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.333+0000 7ff79ffff700 1 -- 172.21.15.19:0/430362474 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578b2aa4510 con 0x5578b2c03d80 2024-05-09T02:16:53.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.334+0000 7ff79ffff700 1 --2- 172.21.15.19:0/430362474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2c03d80 0x5578b2d10650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x5578b2d02d10 tx=0x7ff78c0119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66540a7595a6d7ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:53.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.334+0000 7ff77ffff700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:16:53.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.335+0000 7ff77ffff700 1 -- 172.21.15.19:0/430362474 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c004500 con 0x5578b2c03d80 2024-05-09T02:16:53.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.335+0000 7ff77ffff700 1 -- 172.21.15.19:0/430362474 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7ff78c004830 con 0x5578b2c03d80 2024-05-09T02:16:53.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.335+0000 7ff7b571cb80 1 -- 172.21.15.19:0/430362474 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7980ad870 con 0x5578b2c03d80 2024-05-09T02:16:53.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.335+0000 7ff77ffff700 1 -- 172.21.15.19:0/430362474 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff78c004a30 con 0x5578b2c03d80 2024-05-09T02:16:53.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.335+0000 7ff7b571cb80 10 client.25783 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:53.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.335+0000 7ff7b571cb80 1 -- 172.21.15.19:0/430362474 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5578b2d1db20 con 0x5578b2c03d80 2024-05-09T02:16:53.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.335+0000 7ff77ffff700 1 -- 172.21.15.19:0/430362474 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7980ad870 con 0x5578b2c03d80 2024-05-09T02:16:53.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.335+0000 7ff77ffff700 1 -- 172.21.15.19:0/430362474 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7ff78c018070 con 0x5578b2c03d80 2024-05-09T02:16:53.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.336+0000 7ff7b571cb80 10 client.25783 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:53.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.336+0000 7ff7b571cb80 10 client.25783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:16:53.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.336+0000 7ff7b571cb80 20 client.25783 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:53.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.336+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/430362474 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5578b2d20160 0x5578b2d22580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.336+0000 7ff7b571cb80 4 client.25783 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:16:53.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.336+0000 7ff7b571cb80 1 -- 172.21.15.19:0/430362474 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578b2c36b00 con 0x5578b2d20160 2024-05-09T02:16:53.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.336+0000 7ff7a4aa2700 1 --2- 172.21.15.19:0/430362474 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5578b2d20160 0x5578b2d22580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.336+0000 7ff77ffff700 10 client.25783 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.336+0000 7ff7a4aa2700 1 --2- 172.21.15.19:0/430362474 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5578b2d20160 0x5578b2d22580 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d510c23e2a414af6 server_cookie=3d756984d92ee5 in_seq=0 out_seq=0 2024-05-09T02:16:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.337+0000 7ff77ffff700 1 -- 172.21.15.19:0/430362474 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5578b2c36b00 con 0x5578b2d20160 2024-05-09T02:16:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.337+0000 7ff77ffff700 10 client.25783 handle_command_reply: tid=0 2024-05-09T02:16:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.337+0000 7ff7b571cb80 1 client.25783 shutdown 2024-05-09T02:16:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.337+0000 7ff7b571cb80 1 -- 172.21.15.19:0/430362474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2c03d80 msgr2=0x5578b2d10650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.337+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/430362474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2c03d80 0x5578b2d10650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x5578b2d02d10 tx=0x7ff78c0119a0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.338+0000 7ff7b571cb80 1 -- 172.21.15.19:0/430362474 shutdown_connections 2024-05-09T02:16:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.338+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/430362474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2d06480 0x5578b2d10b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.338+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/430362474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2b89520 0x5578b2d10110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.338+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/430362474 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5578b2d20160 0x5578b2d22580 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.338+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/430362474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2c03d80 0x5578b2d10650 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.338+0000 7ff7b571cb80 1 -- 172.21.15.19:0/430362474 >> 172.21.15.19:0/430362474 conn(0x5578b2b8cff0 msgr2=0x5578b2cfb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.338+0000 7ff77ffff700 0 client.25783 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.338+0000 7ff7b571cb80 1 -- 172.21.15.19:0/430362474 shutdown_connections 2024-05-09T02:16:53.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.338+0000 7ff7b571cb80 1 -- 172.21.15.19:0/430362474 wait complete. 2024-05-09T02:16:53.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.338+0000 7ff7b571cb80 20 client.25783 trim_cache size 0 max 16384 2024-05-09T02:16:53.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.366+0000 7ff7b571cb80 1 Processor -- start 2024-05-09T02:16:53.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.368+0000 7ff7b571cb80 1 -- start start 2024-05-09T02:16:53.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.368+0000 7ff7b571cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2b89520 0x5578b2d05ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.368+0000 7ff7b571cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2c03d80 0x5578b2d10680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.368+0000 7ff7b571cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2d06480 0x5578b2d1bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.369+0000 7ff7b571cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01bdf30 con 0x5578b2d06480 2024-05-09T02:16:53.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.369+0000 7ff7b571cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01bfb30 con 0x5578b2b89520 2024-05-09T02:16:53.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.369+0000 7ff7b571cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01be600 con 0x5578b2c03d80 2024-05-09T02:16:53.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.369+0000 7ff7a4aa2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2b89520 0x5578b2d05ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.369+0000 7ff7a4aa2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2b89520 0x5578b2d05ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:36604/0 (socket says 172.21.15.19:36604) 2024-05-09T02:16:53.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.369+0000 7ff7a4aa2700 1 -- 172.21.15.19:0/3077070100 learned_addr learned my addr 172.21.15.19:0/3077070100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:53.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.369+0000 7ff7a52a3700 1 --2- 172.21.15.19:0/3077070100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2d06480 0x5578b2d1bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.369+0000 7ff79ffff700 1 --2- 172.21.15.19:0/3077070100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2c03d80 0x5578b2d10680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.369+0000 7ff79ffff700 1 -- 172.21.15.19:0/3077070100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2b89520 msgr2=0x5578b2d05ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:53.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.369+0000 7ff79ffff700 1 --2- 172.21.15.19:0/3077070100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2b89520 0x5578b2d05ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.369+0000 7ff79ffff700 1 -- 172.21.15.19:0/3077070100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2d06480 msgr2=0x5578b2d1bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:53.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.369+0000 7ff79ffff700 1 --2- 172.21.15.19:0/3077070100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2d06480 0x5578b2d1bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.369+0000 7ff79ffff700 1 -- 172.21.15.19:0/3077070100 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b2cfda50 con 0x5578b2c03d80 2024-05-09T02:16:53.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.369+0000 7ff7a52a3700 1 --2- 172.21.15.19:0/3077070100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2d06480 0x5578b2d1bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:16:53.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.370+0000 7ff79ffff700 1 --2- 172.21.15.19:0/3077070100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2c03d80 0x5578b2d10680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x5578b2d02330 tx=0x7ff78c021c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7ca3adc9c96a2f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:53.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.370+0000 7ff78a7fc700 1 -- 172.21.15.19:0/3077070100 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c004300 con 0x5578b2c03d80 2024-05-09T02:16:53.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.370+0000 7ff78a7fc700 1 -- 172.21.15.19:0/3077070100 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff78c011e50 con 0x5578b2c03d80 2024-05-09T02:16:53.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.370+0000 7ff78a7fc700 1 -- 172.21.15.19:0/3077070100 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c003da0 con 0x5578b2c03d80 2024-05-09T02:16:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.371+0000 7ff7b571cb80 1 -- 172.21.15.19:0/3077070100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2c03d80 msgr2=0x5578b2d10680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.371+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/3077070100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2c03d80 0x5578b2d10680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x5578b2d02330 tx=0x7ff78c021c70 comp rx=0 tx=0).stop 2024-05-09T02:16:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.371+0000 7ff7b571cb80 1 -- 172.21.15.19:0/3077070100 shutdown_connections 2024-05-09T02:16:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.371+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/3077070100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2d06480 0x5578b2d1bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.371+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/3077070100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2b89520 0x5578b2d05ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.371+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/3077070100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2c03d80 0x5578b2d10680 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.371+0000 7ff7b571cb80 1 -- 172.21.15.19:0/3077070100 >> 172.21.15.19:0/3077070100 conn(0x5578b2b8cff0 msgr2=0x5578b2cfb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.371+0000 7ff7b571cb80 1 -- 172.21.15.19:0/3077070100 shutdown_connections 2024-05-09T02:16:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.371+0000 7ff7b571cb80 1 -- 172.21.15.19:0/3077070100 wait complete. 2024-05-09T02:16:53.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.371+0000 7ff7b571cb80 1 -- start start 2024-05-09T02:16:53.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.372+0000 7ff7b571cb80 1 Processor -- start 2024-05-09T02:16:53.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.372+0000 7ff7b571cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2b89520 0x5578b2d18220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.372+0000 7ff7b571cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2c03d80 0x5578b2d18760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.372+0000 7ff7b571cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2d06480 0x5578b2d151f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.372+0000 7ff7b571cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01bf460 con 0x5578b2d06480 2024-05-09T02:16:53.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.372+0000 7ff7b571cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01becd0 con 0x5578b2c03d80 2024-05-09T02:16:53.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.372+0000 7ff7b571cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01bbed0 con 0x5578b2b89520 2024-05-09T02:16:53.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.372+0000 7ff7b571cb80 10 client.0 _ll_register_callbacks cb 0x5578b2a9a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:53.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.372+0000 7ff7a4aa2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2b89520 0x5578b2d18220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.372+0000 7ff7a52a3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2d06480 0x5578b2d151f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.372+0000 7ff7a4aa2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2b89520 0x5578b2d18220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:38876/0 (socket says 172.21.15.19:38876) 2024-05-09T02:16:53.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.372+0000 7ff7a4aa2700 1 -- 172.21.15.19:0/4244096610 learned_addr learned my addr 172.21.15.19:0/4244096610 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:53.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.372+0000 7ff7a52a3700 1 --2- 172.21.15.19:0/4244096610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2d06480 0x5578b2d151f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34804/0 (socket says 172.21.15.19:34804) 2024-05-09T02:16:53.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.373+0000 7ff79ffff700 1 --2- 172.21.15.19:0/4244096610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2c03d80 0x5578b2d18760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.373+0000 7ff79ffff700 1 -- 172.21.15.19:0/4244096610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2b89520 msgr2=0x5578b2d18220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.373+0000 7ff79ffff700 1 --2- 172.21.15.19:0/4244096610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2b89520 0x5578b2d18220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.373+0000 7ff79ffff700 1 -- 172.21.15.19:0/4244096610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2d06480 msgr2=0x5578b2d151f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.373+0000 7ff79ffff700 1 --2- 172.21.15.19:0/4244096610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2d06480 0x5578b2d151f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.373+0000 7ff79ffff700 1 -- 172.21.15.19:0/4244096610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578b2d15730 con 0x5578b2c03d80 2024-05-09T02:16:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.373+0000 7ff7a52a3700 1 --2- 172.21.15.19:0/4244096610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2d06480 0x5578b2d151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:16:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.373+0000 7ff79ffff700 1 --2- 172.21.15.19:0/4244096610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2c03d80 0x5578b2d18760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff78c014040 tx=0x7ff78c0119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3a240dcd41b50de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.373+0000 7ff77ffff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:16:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.373+0000 7ff77ffff700 1 -- 172.21.15.19:0/4244096610 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c003cb0 con 0x5578b2c03d80 2024-05-09T02:16:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.373+0000 7ff77ffff700 1 -- 172.21.15.19:0/4244096610 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7ff78c004300 con 0x5578b2c03d80 2024-05-09T02:16:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.374+0000 7ff7b571cb80 1 -- 172.21.15.19:0/4244096610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff78c011e50 con 0x5578b2c03d80 2024-05-09T02:16:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.374+0000 7ff77ffff700 1 -- 172.21.15.19:0/4244096610 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff78c003e50 con 0x5578b2c03d80 2024-05-09T02:16:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.374+0000 7ff7b571cb80 10 client.25736 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.374+0000 7ff7b571cb80 1 -- 172.21.15.19:0/4244096610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5578b2d8af50 con 0x5578b2c03d80 2024-05-09T02:16:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.374+0000 7ff77ffff700 1 -- 172.21.15.19:0/4244096610 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff78c011e50 con 0x5578b2c03d80 2024-05-09T02:16:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.374+0000 7ff77ffff700 1 -- 172.21.15.19:0/4244096610 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7ff78c028070 con 0x5578b2c03d80 2024-05-09T02:16:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.374+0000 7ff7b571cb80 10 client.25736 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.374+0000 7ff7b571cb80 10 client.25736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:16:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.374+0000 7ff7b571cb80 20 client.25736 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.374+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/4244096610 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5578b2d8b590 0x5578b2d8d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.374+0000 7ff7b571cb80 4 client.25736 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T02:16:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.374+0000 7ff7b571cb80 1 -- 172.21.15.19:0/4244096610 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5578b2d22af0 con 0x5578b2d8b590 2024-05-09T02:16:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.375+0000 7ff7a4aa2700 1 --2- 172.21.15.19:0/4244096610 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5578b2d8b590 0x5578b2d8d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.375+0000 7ff77ffff700 10 client.25736 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.375+0000 7ff7a4aa2700 1 --2- 172.21.15.19:0/4244096610 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5578b2d8b590 0x5578b2d8d9b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f62338932c358a0 server_cookie=edf02f132f881ab1 in_seq=0 out_seq=0 2024-05-09T02:16:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.376+0000 7ff77ffff700 1 -- 172.21.15.19:0/4244096610 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5578b2d22af0 con 0x5578b2d8b590 2024-05-09T02:16:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.376+0000 7ff77ffff700 10 client.25736 handle_command_reply: tid=0 2024-05-09T02:16:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.376+0000 7ff7b571cb80 1 client.25736 shutdown 2024-05-09T02:16:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.376+0000 7ff7b571cb80 1 -- 172.21.15.19:0/4244096610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2c03d80 msgr2=0x5578b2d18760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.376+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/4244096610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2c03d80 0x5578b2d18760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff78c014040 tx=0x7ff78c0119a0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.376+0000 7ff7b571cb80 1 -- 172.21.15.19:0/4244096610 shutdown_connections 2024-05-09T02:16:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.376+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/4244096610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578b2d06480 0x5578b2d151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.376+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/4244096610 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5578b2d8b590 0x5578b2d8d9b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.376+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/4244096610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578b2b89520 0x5578b2d18220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.376+0000 7ff7b571cb80 1 --2- 172.21.15.19:0/4244096610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578b2c03d80 0x5578b2d18760 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.376+0000 7ff7b571cb80 1 -- 172.21.15.19:0/4244096610 >> 172.21.15.19:0/4244096610 conn(0x5578b2b8cff0 msgr2=0x5578b2d09a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.376+0000 7ff77ffff700 0 client.25736 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.379+0000 7ff7b571cb80 1 -- 172.21.15.19:0/4244096610 shutdown_connections 2024-05-09T02:16:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.379+0000 7ff7b571cb80 1 -- 172.21.15.19:0/4244096610 wait complete. 2024-05-09T02:16:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.379+0000 7ff7b571cb80 20 client.25736 trim_cache size 0 max 16384 2024-05-09T02:16:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:16:53.384 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T02:16:53.384 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "608bd60b-e64f-4d82-8150-c3106b9505b2", 2024-05-09T02:16:53.384 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T02:16:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:16:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.380+0000 7ff7a6d06700 1 -- 172.21.15.19:0/3450578308 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff7900b2af0 msgr2=0x7ff7900b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.380+0000 7ff7a6d06700 1 --2- 172.21.15.19:0/3450578308 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff7900b2af0 0x7ff7900b5000 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff794003760 tx=0x7ff79401a040 comp rx=0 tx=0).stop 2024-05-09T02:16:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.381+0000 7ff7a6d06700 1 -- 172.21.15.19:0/3450578308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7a0108760 msgr2=0x7ff7a006e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.381+0000 7ff7a6d06700 1 --2- 172.21.15.19:0/3450578308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7a0108760 0x7ff7a006e980 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff79800ef30 tx=0x7ff79800c450 comp rx=0 tx=0).stop 2024-05-09T02:16:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.381+0000 7ff7a6d06700 1 -- 172.21.15.19:0/3450578308 shutdown_connections 2024-05-09T02:16:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.381+0000 7ff7a6d06700 1 --2- 172.21.15.19:0/3450578308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7a0108760 0x7ff7a006e980 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.381+0000 7ff7a6d06700 1 --2- 172.21.15.19:0/3450578308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7a01034e0 0x7ff7a0075460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.381+0000 7ff7a6d06700 1 --2- 172.21.15.19:0/3450578308 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff7900b2af0 0x7ff7900b5000 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.381+0000 7ff7a6d06700 1 --2- 172.21.15.19:0/3450578308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7a0107e10 0x7ff7a006e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.381+0000 7ff7a6d06700 1 -- 172.21.15.19:0/3450578308 >> 172.21.15.19:0/3450578308 conn(0x7ff7a00fef30 msgr2=0x7ff7a01096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.381+0000 7ff7a6d06700 1 -- 172.21.15.19:0/3450578308 shutdown_connections 2024-05-09T02:16:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:53.381+0000 7ff7a6d06700 1 -- 172.21.15.19:0/3450578308 wait complete. 2024-05-09T02:16:54.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:16:54.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: pgmap v171: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.6 MiB/s wr, 594 op/s 2024-05-09T02:16:54.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub queued for path: / 2024-05-09T02:16:54.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:16:54.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:16:54.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:16:54.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:16:54.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:16:54.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:16:54.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:16:54.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:16:54.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:16:54.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:54 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: pgmap v171: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.6 MiB/s wr, 594 op/s 2024-05-09T02:16:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub queued for path: / 2024-05-09T02:16:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:16:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:16:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:16:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:16:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:16:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:16:54.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:16:54.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:16:54.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:16:54.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:54 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: pgmap v171: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.6 MiB/s wr, 594 op/s 2024-05-09T02:16:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub queued for path: / 2024-05-09T02:16:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:16:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:16:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:16:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:16:54.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:16:54.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:16:54.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:16:54.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:16:54.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:16:54.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:54 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:16:54.829 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:16:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:56 smithi059 ceph-mon[34781]: pgmap v172: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 854 op/s 2024-05-09T02:16:56.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:56 smithi196 ceph-mon[34718]: pgmap v172: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 854 op/s 2024-05-09T02:16:56.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:56 smithi019 ceph-mon[30151]: pgmap v172: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 854 op/s 2024-05-09T02:16:57.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.546+0000 7fce950c2700 1 -- 172.21.15.19:0/290411093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce90075d30 msgr2=0x7fce90076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:57.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.546+0000 7fce950c2700 1 --2- 172.21.15.19:0/290411093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce90075d30 0x7fce90076140 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fce800098f0 tx=0x7fce80009c00 comp rx=0 tx=0).stop 2024-05-09T02:16:57.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.547+0000 7fce950c2700 1 -- 172.21.15.19:0/290411093 shutdown_connections 2024-05-09T02:16:57.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.547+0000 7fce950c2700 1 --2- 172.21.15.19:0/290411093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce90110ba0 0x7fce90113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.547+0000 7fce950c2700 1 --2- 172.21.15.19:0/290411093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce90075d30 0x7fce90076140 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.547+0000 7fce950c2700 1 --2- 172.21.15.19:0/290411093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce90076680 0x7fce90110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.547+0000 7fce950c2700 1 -- 172.21.15.19:0/290411093 >> 172.21.15.19:0/290411093 conn(0x7fce90100040 msgr2=0x7fce90102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:57.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.547+0000 7fce950c2700 1 -- 172.21.15.19:0/290411093 shutdown_connections 2024-05-09T02:16:57.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.547+0000 7fce950c2700 1 -- 172.21.15.19:0/290411093 wait complete. 2024-05-09T02:16:57.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.548+0000 7fce950c2700 1 Processor -- start 2024-05-09T02:16:57.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.549+0000 7fce950c2700 1 -- start start 2024-05-09T02:16:57.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.550+0000 7fce950c2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce90075d30 0x7fce9010f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.550+0000 7fce950c2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce90076680 0x7fce9010fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.550+0000 7fce8ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce90075d30 0x7fce9010f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:57.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.550+0000 7fce8ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce90075d30 0x7fce9010f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:45710/0 (socket says 172.21.15.19:45710) 2024-05-09T02:16:57.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.550+0000 7fce8ed9d700 1 -- 172.21.15.19:0/4212600200 learned_addr learned my addr 172.21.15.19:0/4212600200 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:57.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.550+0000 7fce950c2700 1 --2- 172.21.15.19:0/4212600200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce90110ba0 0x7fce901a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.550+0000 7fce950c2700 1 -- 172.21.15.19:0/4212600200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce901052a0 con 0x7fce90076680 2024-05-09T02:16:57.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.550+0000 7fce950c2700 1 -- 172.21.15.19:0/4212600200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fce90105120 con 0x7fce90075d30 2024-05-09T02:16:57.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.550+0000 7fce950c2700 1 -- 172.21.15.19:0/4212600200 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce90105420 con 0x7fce90110ba0 2024-05-09T02:16:57.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.551+0000 7fce8ed9d700 1 -- 172.21.15.19:0/4212600200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce90110ba0 msgr2=0x7fce901a5d40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T02:16:57.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.551+0000 7fce8ed9d700 1 --2- 172.21.15.19:0/4212600200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce90110ba0 0x7fce901a5d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.551+0000 7fce8ed9d700 1 -- 172.21.15.19:0/4212600200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce90076680 msgr2=0x7fce9010fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:57.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.551+0000 7fce8ed9d700 1 --2- 172.21.15.19:0/4212600200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce90076680 0x7fce9010fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.551+0000 7fce8ed9d700 1 -- 172.21.15.19:0/4212600200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce901a6380 con 0x7fce90075d30 2024-05-09T02:16:57.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.551+0000 7fce8ed9d700 1 --2- 172.21.15.19:0/4212600200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce90075d30 0x7fce9010f500 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fce80009fd0 tx=0x7fce8000fb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:57.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.552+0000 7fce77fff700 1 -- 172.21.15.19:0/4212600200 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80027070 con 0x7fce90075d30 2024-05-09T02:16:57.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.552+0000 7fce77fff700 1 -- 172.21.15.19:0/4212600200 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce8000b560 con 0x7fce90075d30 2024-05-09T02:16:57.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.552+0000 7fce77fff700 1 -- 172.21.15.19:0/4212600200 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce800049c0 con 0x7fce90075d30 2024-05-09T02:16:57.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.552+0000 7fce950c2700 1 -- 172.21.15.19:0/4212600200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce901a6610 con 0x7fce90075d30 2024-05-09T02:16:57.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.552+0000 7fce950c2700 1 -- 172.21.15.19:0/4212600200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce901a6aa0 con 0x7fce90075d30 2024-05-09T02:16:57.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.558+0000 7fce77fff700 1 -- 172.21.15.19:0/4212600200 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fce8002a070 con 0x7fce90075d30 2024-05-09T02:16:57.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.559+0000 7fce77fff700 1 --2- 172.21.15.19:0/4212600200 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fce780b2bc0 0x7fce780b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.559+0000 7fcea3ad8b80 1 Processor -- start 2024-05-09T02:16:57.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.559+0000 7fcea3ad8b80 1 -- start start 2024-05-09T02:16:57.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.559+0000 7fce77fff700 1 -- 172.21.15.19:0/4212600200 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fce800e3270 con 0x7fce90075d30 2024-05-09T02:16:57.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.559+0000 7fce8e59c700 1 --2- 172.21.15.19:0/4212600200 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fce780b2bc0 0x7fce780b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:57.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.560+0000 7fcea3ad8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558035fdc160 0x5580360754f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.560+0000 7fcea3ad8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036075a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.560+0000 7fcea3ad8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803606f6a0 0x55803606fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.560+0000 7fcea3ad8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558035f91ff0 con 0x558035fdc160 2024-05-09T02:16:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.560+0000 7fcea3ad8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558035f307f0 con 0x558035ef24a0 2024-05-09T02:16:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.560+0000 7fcea3ad8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558035f8dfb0 con 0x55803606f6a0 2024-05-09T02:16:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.560+0000 7fce8e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036075a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.560+0000 7fce8e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036075a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:45726/0 (socket says 172.21.15.19:45726) 2024-05-09T02:16:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.560+0000 7fce8e59c700 1 -- 172.21.15.19:0/21925347 learned_addr learned my addr 172.21.15.19:0/21925347 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.560+0000 7fce8f59e700 1 --2- 172.21.15.19:0/21925347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803606f6a0 0x55803606fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.561+0000 7fce8e59c700 1 -- 172.21.15.19:0/21925347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803606f6a0 msgr2=0x55803606fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.561+0000 7fce8e59c700 1 --2- 172.21.15.19:0/21925347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803606f6a0 0x55803606fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.561+0000 7fce8e59c700 1 -- 172.21.15.19:0/21925347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558035fdc160 msgr2=0x5580360754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:57.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.561+0000 7fce8e59c700 1 --2- 172.21.15.19:0/21925347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558035fdc160 0x5580360754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.561+0000 7fce8e59c700 1 -- 172.21.15.19:0/21925347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558035f28370 con 0x558035ef24a0 2024-05-09T02:16:57.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.561+0000 7fce8f59e700 1 --2- 172.21.15.19:0/21925347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803606f6a0 0x55803606fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:16:57.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.561+0000 7fce8e59c700 1 --2- 172.21.15.19:0/21925347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036075a30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fce840129e0 tx=0x7fce84012cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=462ed0da3e693b9a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:57.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.562+0000 7fce8e59c700 1 --2- 172.21.15.19:0/4212600200 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fce780b2bc0 0x7fce780b50d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fce84020000 tx=0x7fce8401b000 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:57.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.562+0000 7fce757fa700 1 -- 172.21.15.19:0/21925347 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84023070 con 0x558035ef24a0 2024-05-09T02:16:57.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.562+0000 7fce757fa700 1 -- 172.21.15.19:0/21925347 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce84005050 con 0x558035ef24a0 2024-05-09T02:16:57.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.562+0000 7fce757fa700 1 -- 172.21.15.19:0/21925347 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8401d660 con 0x558035ef24a0 2024-05-09T02:16:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.563+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/21925347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 msgr2=0x558036075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.563+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/21925347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036075a30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fce840129e0 tx=0x7fce84012cf0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.563+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/21925347 shutdown_connections 2024-05-09T02:16:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.563+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/21925347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803606f6a0 0x55803606fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.563+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/21925347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558035fdc160 0x5580360754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.563+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/21925347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036075a30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.563+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/21925347 >> 172.21.15.19:0/21925347 conn(0x558035f526a0 msgr2=0x55803606a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.564+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/21925347 shutdown_connections 2024-05-09T02:16:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.564+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/21925347 wait complete. 2024-05-09T02:16:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.564+0000 7fcea3ad8b80 1 -- start start 2024-05-09T02:16:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.564+0000 7fcea3ad8b80 1 Processor -- start 2024-05-09T02:16:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.564+0000 7fcea3ad8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036079240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.564+0000 7fcea3ad8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558035fdc160 0x558036079780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.564+0000 7fcea3ad8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803606f6a0 0x558036079cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.565+0000 7fcea3ad8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558035e0dba0 con 0x55803606f6a0 2024-05-09T02:16:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.565+0000 7fcea3ad8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558035f4c150 con 0x558035ef24a0 2024-05-09T02:16:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.565+0000 7fcea3ad8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55803607c1d0 con 0x558035fdc160 2024-05-09T02:16:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.565+0000 7fcea3ad8b80 10 client.0 _ll_register_callbacks cb 0x558035e036c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.565+0000 7fce8ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036079240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.565+0000 7fce8ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036079240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:45738/0 (socket says 172.21.15.19:45738) 2024-05-09T02:16:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.565+0000 7fce8ed9d700 1 -- 172.21.15.19:0/131648742 learned_addr learned my addr 172.21.15.19:0/131648742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.565+0000 7fce8ed9d700 1 -- 172.21.15.19:0/131648742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558035fdc160 msgr2=0x558036079780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.565+0000 7fce8ed9d700 1 --2- 172.21.15.19:0/131648742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558035fdc160 0x558036079780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.565+0000 7fce8ed9d700 1 -- 172.21.15.19:0/131648742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803606f6a0 msgr2=0x558036079cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.565+0000 7fce8ed9d700 1 --2- 172.21.15.19:0/131648742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803606f6a0 0x558036079cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.565+0000 7fce8ed9d700 1 -- 172.21.15.19:0/131648742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558035e0d490 con 0x558035ef24a0 2024-05-09T02:16:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.567+0000 7fce8ed9d700 1 --2- 172.21.15.19:0/131648742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036079240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fce800e09c0 tx=0x7fce800095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c32c4922943d2429 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.567+0000 7fce72ff5700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:16:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.567+0000 7fce72ff5700 1 -- 172.21.15.19:0/131648742 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce800aabe0 con 0x558035ef24a0 2024-05-09T02:16:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.567+0000 7fce72ff5700 1 -- 172.21.15.19:0/131648742 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fce800a7880 con 0x558035ef24a0 2024-05-09T02:16:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.568+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/131648742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce84005050 con 0x558035ef24a0 2024-05-09T02:16:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.568+0000 7fce72ff5700 1 -- 172.21.15.19:0/131648742 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fce800aaa00 con 0x558035ef24a0 2024-05-09T02:16:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.568+0000 7fcea3ad8b80 10 client.25748 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.568+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/131648742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558036086c50 con 0x558035ef24a0 2024-05-09T02:16:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.569+0000 7fce72ff5700 1 -- 172.21.15.19:0/131648742 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce84005050 con 0x558035ef24a0 2024-05-09T02:16:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.569+0000 7fce72ff5700 1 -- 172.21.15.19:0/131648742 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7fce800aa4f0 con 0x558035ef24a0 2024-05-09T02:16:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.569+0000 7fcea3ad8b80 10 client.25748 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.569+0000 7fcea3ad8b80 10 client.25748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:16:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.569+0000 7fcea3ad8b80 20 client.25748 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.569+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/131648742 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5580360891c0 0x55803608b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.569+0000 7fcea3ad8b80 4 client.25748 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:16:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.569+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/131648742 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558035f9f9d0 con 0x5580360891c0 2024-05-09T02:16:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.570+0000 7fce8e59c700 1 --2- 172.21.15.19:0/131648742 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5580360891c0 0x55803608b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.570+0000 7fce8e59c700 1 --2- 172.21.15.19:0/131648742 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5580360891c0 0x55803608b630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=744eacd2612f6be5 server_cookie=de8973c2a9fbb420 in_seq=0 out_seq=0 2024-05-09T02:16:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.570+0000 7fce72ff5700 10 client.25748 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.571+0000 7fce72ff5700 1 -- 172.21.15.19:0/131648742 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558035f9f9d0 con 0x5580360891c0 2024-05-09T02:16:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.571+0000 7fce72ff5700 10 client.25748 handle_command_reply: tid=0 2024-05-09T02:16:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.571+0000 7fcea3ad8b80 1 client.25748 shutdown 2024-05-09T02:16:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.572+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/131648742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 msgr2=0x558036079240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.572+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/131648742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036079240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fce800e09c0 tx=0x7fce800095b0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.572+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/131648742 shutdown_connections 2024-05-09T02:16:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.572+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/131648742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803606f6a0 0x558036079cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.572+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/131648742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036079240 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.572+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/131648742 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5580360891c0 0x55803608b630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.572+0000 7fce72ff5700 0 client.25748 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.572+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/131648742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558035fdc160 0x558036079780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.572+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/131648742 >> 172.21.15.19:0/131648742 conn(0x558035f526a0 msgr2=0x55803606a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.572+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/131648742 shutdown_connections 2024-05-09T02:16:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.572+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/131648742 wait complete. 2024-05-09T02:16:57.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.572+0000 7fcea3ad8b80 20 client.25748 trim_cache size 0 max 16384 2024-05-09T02:16:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.603+0000 7fcea3ad8b80 1 Processor -- start 2024-05-09T02:16:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.603+0000 7fcea3ad8b80 1 -- start start 2024-05-09T02:16:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.603+0000 7fcea3ad8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036079c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fcea3ad8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558035fdc160 0x55803606e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fcea3ad8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803606f6a0 0x55803606ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fcea3ad8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce901b54a0 con 0x55803606f6a0 2024-05-09T02:16:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fcea3ad8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fce901b70a0 con 0x558035ef24a0 2024-05-09T02:16:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fcea3ad8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce901b5b70 con 0x558035fdc160 2024-05-09T02:16:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fce8ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036079c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fce8f59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803606f6a0 0x55803606ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fce8ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036079c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:45740/0 (socket says 172.21.15.19:45740) 2024-05-09T02:16:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fce8ed9d700 1 -- 172.21.15.19:0/2775862310 learned_addr learned my addr 172.21.15.19:0/2775862310 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fce8e59c700 1 --2- 172.21.15.19:0/2775862310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558035fdc160 0x55803606e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fce8ed9d700 1 -- 172.21.15.19:0/2775862310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558035fdc160 msgr2=0x55803606e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fce8ed9d700 1 --2- 172.21.15.19:0/2775862310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558035fdc160 0x55803606e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fce8ed9d700 1 -- 172.21.15.19:0/2775862310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803606f6a0 msgr2=0x55803606ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fce8ed9d700 1 --2- 172.21.15.19:0/2775862310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803606f6a0 0x55803606ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fce8ed9d700 1 -- 172.21.15.19:0/2775862310 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55803607bae0 con 0x558035ef24a0 2024-05-09T02:16:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fce8ed9d700 1 --2- 172.21.15.19:0/2775862310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036079c30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fce800a8500 tx=0x7fce8001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae57764aa6d3110e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.604+0000 7fce74ff9700 1 -- 172.21.15.19:0/2775862310 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce800aabe0 con 0x558035ef24a0 2024-05-09T02:16:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.605+0000 7fce74ff9700 1 -- 172.21.15.19:0/2775862310 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce800c4e50 con 0x558035ef24a0 2024-05-09T02:16:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.605+0000 7fce74ff9700 1 -- 172.21.15.19:0/2775862310 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce800ab290 con 0x558035ef24a0 2024-05-09T02:16:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.605+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/2775862310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 msgr2=0x558036079c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.605+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/2775862310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036079c30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fce800a8500 tx=0x7fce8001c000 comp rx=0 tx=0).stop 2024-05-09T02:16:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.606+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/2775862310 shutdown_connections 2024-05-09T02:16:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.606+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/2775862310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803606f6a0 0x55803606ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.606+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/2775862310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036079c30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.606+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/2775862310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558035fdc160 0x55803606e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.606+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/2775862310 >> 172.21.15.19:0/2775862310 conn(0x558035f526a0 msgr2=0x558036077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.606+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/2775862310 shutdown_connections 2024-05-09T02:16:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.606+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/2775862310 wait complete. 2024-05-09T02:16:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.607+0000 7fcea3ad8b80 1 -- start start 2024-05-09T02:16:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.607+0000 7fcea3ad8b80 1 Processor -- start 2024-05-09T02:16:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.607+0000 7fcea3ad8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036081420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.607+0000 7fcea3ad8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558035fdc160 0x558036081960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fcea3ad8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803606f6a0 0x55803607e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fcea3ad8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce901b69d0 con 0x55803606f6a0 2024-05-09T02:16:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fcea3ad8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fce901b6240 con 0x558035ef24a0 2024-05-09T02:16:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fcea3ad8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce901b3440 con 0x558035fdc160 2024-05-09T02:16:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fcea3ad8b80 10 client.0 _ll_register_callbacks cb 0x558035e036c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:16:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fce8ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036081420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fce8ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036081420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:45756/0 (socket says 172.21.15.19:45756) 2024-05-09T02:16:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fce8ed9d700 1 -- 172.21.15.19:0/1251912688 learned_addr learned my addr 172.21.15.19:0/1251912688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:16:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fce8e59c700 1 --2- 172.21.15.19:0/1251912688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558035fdc160 0x558036081960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fce8f59e700 1 --2- 172.21.15.19:0/1251912688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803606f6a0 0x55803607e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fce8ed9d700 1 -- 172.21.15.19:0/1251912688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558035fdc160 msgr2=0x558036081960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fce8ed9d700 1 --2- 172.21.15.19:0/1251912688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558035fdc160 0x558036081960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fce8ed9d700 1 -- 172.21.15.19:0/1251912688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803606f6a0 msgr2=0x55803607e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:16:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fce8ed9d700 1 --2- 172.21.15.19:0/1251912688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803606f6a0 0x55803607e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fce8ed9d700 1 -- 172.21.15.19:0/1251912688 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55803607e960 con 0x558035ef24a0 2024-05-09T02:16:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fce8ed9d700 1 --2- 172.21.15.19:0/1251912688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036081420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce800aae20 tx=0x7fce800a6e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f116be44d3f41861 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:16:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fce72ff5700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:16:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.608+0000 7fce72ff5700 1 -- 172.21.15.19:0/1251912688 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80004e40 con 0x558035ef24a0 2024-05-09T02:16:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.609+0000 7fce72ff5700 1 -- 172.21.15.19:0/1251912688 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fce800a7880 con 0x558035ef24a0 2024-05-09T02:16:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.609+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/1251912688 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce800c4e50 con 0x558035ef24a0 2024-05-09T02:16:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.609+0000 7fce72ff5700 1 -- 172.21.15.19:0/1251912688 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fce800a6f50 con 0x558035ef24a0 2024-05-09T02:16:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.609+0000 7fcea3ad8b80 10 client.25754 fetch_fsmap learned FSMap version 41 2024-05-09T02:16:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.609+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/1251912688 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5580360f41a0 con 0x558035ef24a0 2024-05-09T02:16:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.609+0000 7fce72ff5700 1 -- 172.21.15.19:0/1251912688 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce800c4e50 con 0x558035ef24a0 2024-05-09T02:16:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.609+0000 7fce72ff5700 1 -- 172.21.15.19:0/1251912688 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7fce800859d0 con 0x558035ef24a0 2024-05-09T02:16:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.609+0000 7fcea3ad8b80 10 client.25754 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:16:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.609+0000 7fcea3ad8b80 10 client.25754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:16:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.609+0000 7fcea3ad8b80 20 client.25754 populate_metadata read hostname 'smithi019' 2024-05-09T02:16:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.609+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/1251912688 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5580360f4650 0x5580360f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:16:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.609+0000 7fcea3ad8b80 4 client.25754 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:16:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.609+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/1251912688 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558035f99a70 con 0x5580360f4650 2024-05-09T02:16:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.609+0000 7fce8e59c700 1 --2- 172.21.15.19:0/1251912688 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5580360f4650 0x5580360f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:16:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.610+0000 7fce72ff5700 10 client.25754 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.610+0000 7fce8e59c700 1 --2- 172.21.15.19:0/1251912688 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5580360f4650 0x5580360f6a70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a0f24414828d3b server_cookie=286bf0636b371406 in_seq=0 out_seq=0 2024-05-09T02:16:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.610+0000 7fce72ff5700 1 -- 172.21.15.19:0/1251912688 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558035f99a70 con 0x5580360f4650 2024-05-09T02:16:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.610+0000 7fce72ff5700 10 client.25754 handle_command_reply: tid=0 2024-05-09T02:16:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.610+0000 7fcea3ad8b80 1 client.25754 shutdown 2024-05-09T02:16:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.610+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/1251912688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 msgr2=0x558036081420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.610+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/1251912688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036081420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce800aae20 tx=0x7fce800a6e10 comp rx=0 tx=0).stop 2024-05-09T02:16:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.611+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/1251912688 shutdown_connections 2024-05-09T02:16:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.611+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/1251912688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55803606f6a0 0x55803607e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.611+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/1251912688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558035ef24a0 0x558036081420 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.611+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/1251912688 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5580360f4650 0x5580360f6a70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.611+0000 7fcea3ad8b80 1 --2- 172.21.15.19:0/1251912688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558035fdc160 0x558036081960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.611+0000 7fce72ff5700 0 client.25754 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:16:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.611+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/1251912688 >> 172.21.15.19:0/1251912688 conn(0x558035f526a0 msgr2=0x55803606e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.612+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/1251912688 shutdown_connections 2024-05-09T02:16:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.612+0000 7fcea3ad8b80 1 -- 172.21.15.19:0/1251912688 wait complete. 2024-05-09T02:16:57.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.612+0000 7fcea3ad8b80 20 client.25754 trim_cache size 0 max 16384 2024-05-09T02:16:57.618 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:16:57.618 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T02:16:57.618 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T02:16:57.618 INFO:teuthology.orchestra.run.smithi019.stdout: "608bd60b-e64f-4d82-8150-c3106b9505b2": { 2024-05-09T02:16:57.618 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T02:16:57.618 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "608bd60b-e64f-4d82-8150-c3106b9505b2", 2024-05-09T02:16:57.618 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T02:16:57.619 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:16:57.619 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:16:57.619 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:16:57.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.614+0000 7fce950c2700 1 -- 172.21.15.19:0/4212600200 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fce780b2bc0 msgr2=0x7fce780b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:57.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.614+0000 7fce950c2700 1 --2- 172.21.15.19:0/4212600200 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fce780b2bc0 0x7fce780b50d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fce84020000 tx=0x7fce8401b000 comp rx=0 tx=0).stop 2024-05-09T02:16:57.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.615+0000 7fce950c2700 1 -- 172.21.15.19:0/4212600200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce90075d30 msgr2=0x7fce9010f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:16:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.615+0000 7fce950c2700 1 --2- 172.21.15.19:0/4212600200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce90075d30 0x7fce9010f500 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fce80009fd0 tx=0x7fce8000fb20 comp rx=0 tx=0).stop 2024-05-09T02:16:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.615+0000 7fce950c2700 1 -- 172.21.15.19:0/4212600200 shutdown_connections 2024-05-09T02:16:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.615+0000 7fce950c2700 1 --2- 172.21.15.19:0/4212600200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce90110ba0 0x7fce901a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.615+0000 7fce950c2700 1 --2- 172.21.15.19:0/4212600200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce90075d30 0x7fce9010f500 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.615+0000 7fce950c2700 1 --2- 172.21.15.19:0/4212600200 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fce780b2bc0 0x7fce780b50d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.615+0000 7fce950c2700 1 --2- 172.21.15.19:0/4212600200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce90076680 0x7fce9010fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:16:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.615+0000 7fce950c2700 1 -- 172.21.15.19:0/4212600200 >> 172.21.15.19:0/4212600200 conn(0x7fce90100040 msgr2=0x7fce90100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:16:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.615+0000 7fce950c2700 1 -- 172.21.15.19:0/4212600200 shutdown_connections 2024-05-09T02:16:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:16:57.615+0000 7fce950c2700 1 -- 172.21.15.19:0/4212600200 wait complete. 2024-05-09T02:16:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 315.0 s, 2804.9 tps, lat 35.826 ms stddev 40.859 2024-05-09T02:16:58.374 INFO:tasks.cephfs.filesystem:scrub status for tag:608bd60b-e64f-4d82-8150-c3106b9505b2 - {'path': '/', 'tag': '608bd60b-e64f-4d82-8150-c3106b9505b2', 'options': 'recursive,force'} 2024-05-09T02:16:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:16:58 smithi059 ceph-mon[34781]: pgmap v173: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.8 MiB/s wr, 577 op/s 2024-05-09T02:16:58.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:16:58 smithi196 ceph-mon[34718]: pgmap v173: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.8 MiB/s wr, 577 op/s 2024-05-09T02:16:58.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:16:58 smithi019 ceph-mon[30151]: pgmap v173: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.8 MiB/s wr, 577 op/s 2024-05-09T02:17:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:00 smithi059 ceph-mon[34781]: pgmap v174: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 896 op/s 2024-05-09T02:17:00.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:00 smithi196 ceph-mon[34718]: pgmap v174: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 896 op/s 2024-05-09T02:17:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:00 smithi019 ceph-mon[30151]: pgmap v174: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 896 op/s 2024-05-09T02:17:01.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:01 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:17:01.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:01 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:17:01.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:01 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:17:01.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:01 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:17:01.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:01 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:17:01.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:01 smithi059 ceph-mon[34781]: pgmap v175: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 594 op/s 2024-05-09T02:17:01.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:01 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:17:01.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:01 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:17:01.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:01 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:17:01.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:01 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:17:01.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:01 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:17:01.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:01 smithi196 ceph-mon[34718]: pgmap v175: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 594 op/s 2024-05-09T02:17:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:01 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:17:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:01 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:17:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:01 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:17:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:01 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:17:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:01 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:17:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:01 smithi019 ceph-mon[30151]: pgmap v175: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 594 op/s 2024-05-09T02:17:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 320.0 s, 1793.2 tps, lat 54.722 ms stddev 148.385 2024-05-09T02:17:04.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:04 smithi196 ceph-mon[34718]: pgmap v176: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 611 op/s 2024-05-09T02:17:04.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:04 smithi019 ceph-mon[30151]: pgmap v176: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 611 op/s 2024-05-09T02:17:04.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:04 smithi059 ceph-mon[34781]: pgmap v176: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 611 op/s 2024-05-09T02:17:06.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:06 smithi196 ceph-mon[34718]: pgmap v177: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 MiB/s wr, 1.62k op/s 2024-05-09T02:17:06.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:06 smithi019 ceph-mon[30151]: pgmap v177: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 MiB/s wr, 1.62k op/s 2024-05-09T02:17:06.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:06 smithi059 ceph-mon[34781]: pgmap v177: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 MiB/s wr, 1.62k op/s 2024-05-09T02:17:07.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 325.0 s, 2533.2 tps, lat 40.098 ms stddev 43.244 2024-05-09T02:17:08.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:08 smithi196 ceph-mon[34718]: pgmap v178: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.35k op/s 2024-05-09T02:17:08.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:08 smithi019 ceph-mon[30151]: pgmap v178: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.35k op/s 2024-05-09T02:17:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:08 smithi059 ceph-mon[34781]: pgmap v178: 97 pgs: 97 active+clean; 4.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.35k op/s 2024-05-09T02:17:10.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:10 smithi196 ceph-mon[34718]: pgmap v179: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 MiB/s wr, 1.63k op/s 2024-05-09T02:17:10.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:10 smithi019 ceph-mon[30151]: pgmap v179: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 MiB/s wr, 1.63k op/s 2024-05-09T02:17:10.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:10 smithi059 ceph-mon[34781]: pgmap v179: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 MiB/s wr, 1.63k op/s 2024-05-09T02:17:12.544 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:12 smithi196 ceph-mon[34718]: pgmap v180: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 MiB/s wr, 1.31k op/s 2024-05-09T02:17:12.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:12 smithi019 ceph-mon[30151]: pgmap v180: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 MiB/s wr, 1.31k op/s 2024-05-09T02:17:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 330.0 s, 3001.2 tps, lat 33.297 ms stddev 38.863 2024-05-09T02:17:12.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:12 smithi059 ceph-mon[34781]: pgmap v180: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 MiB/s wr, 1.31k op/s 2024-05-09T02:17:14.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:14 smithi019 ceph-mon[30151]: pgmap v181: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 MiB/s wr, 1.37k op/s 2024-05-09T02:17:14.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:14 smithi059 ceph-mon[34781]: pgmap v181: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 MiB/s wr, 1.37k op/s 2024-05-09T02:17:14.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:14 smithi196 ceph-mon[34718]: pgmap v181: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 MiB/s wr, 1.37k op/s 2024-05-09T02:17:16.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:16 smithi019 ceph-mon[30151]: pgmap v182: 97 pgs: 97 active+clean; 4.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 38 MiB/s wr, 1.59k op/s 2024-05-09T02:17:16.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:16 smithi059 ceph-mon[34781]: pgmap v182: 97 pgs: 97 active+clean; 4.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 38 MiB/s wr, 1.59k op/s 2024-05-09T02:17:16.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:16 smithi196 ceph-mon[34718]: pgmap v182: 97 pgs: 97 active+clean; 4.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 38 MiB/s wr, 1.59k op/s 2024-05-09T02:17:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 335.0 s, 2031.0 tps, lat 48.811 ms stddev 52.430 2024-05-09T02:17:18.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:18 smithi019 ceph-mon[30151]: pgmap v183: 97 pgs: 97 active+clean; 4.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.6 MiB/s wr, 582 op/s 2024-05-09T02:17:18.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:18 smithi059 ceph-mon[34781]: pgmap v183: 97 pgs: 97 active+clean; 4.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.6 MiB/s wr, 582 op/s 2024-05-09T02:17:18.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:18 smithi196 ceph-mon[34718]: pgmap v183: 97 pgs: 97 active+clean; 4.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.6 MiB/s wr, 582 op/s 2024-05-09T02:17:20.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:20 smithi019 ceph-mon[30151]: pgmap v184: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.7 MiB/s wr, 894 op/s 2024-05-09T02:17:20.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:20 smithi059 ceph-mon[34781]: pgmap v184: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.7 MiB/s wr, 894 op/s 2024-05-09T02:17:20.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:20 smithi196 ceph-mon[34718]: pgmap v184: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.7 MiB/s wr, 894 op/s 2024-05-09T02:17:22.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:22 smithi019 ceph-mon[30151]: pgmap v185: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.9 MiB/s wr, 610 op/s 2024-05-09T02:17:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 340.0 s, 2601.2 tps, lat 38.829 ms stddev 43.975 2024-05-09T02:17:22.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:22 smithi059 ceph-mon[34781]: pgmap v185: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.9 MiB/s wr, 610 op/s 2024-05-09T02:17:22.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:22 smithi196 ceph-mon[34718]: pgmap v185: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.9 MiB/s wr, 610 op/s 2024-05-09T02:17:24.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:24 smithi019 ceph-mon[30151]: pgmap v186: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 614 op/s 2024-05-09T02:17:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:24 smithi059 ceph-mon[34781]: pgmap v186: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 614 op/s 2024-05-09T02:17:24.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:24 smithi196 ceph-mon[34718]: pgmap v186: 97 pgs: 97 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 614 op/s 2024-05-09T02:17:26.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:26 smithi019 ceph-mon[30151]: pgmap v187: 97 pgs: 97 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.0 MiB/s wr, 852 op/s 2024-05-09T02:17:26.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:26 smithi059 ceph-mon[34781]: pgmap v187: 97 pgs: 97 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.0 MiB/s wr, 852 op/s 2024-05-09T02:17:26.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:26 smithi196 ceph-mon[34718]: pgmap v187: 97 pgs: 97 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.0 MiB/s wr, 852 op/s 2024-05-09T02:17:27.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 345.0 s, 1921.4 tps, lat 52.002 ms stddev 58.280 2024-05-09T02:17:28.375 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:17:28.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:28 smithi059 ceph-mon[34781]: pgmap v188: 97 pgs: 97 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.9 MiB/s wr, 611 op/s 2024-05-09T02:17:28.736 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:17:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:28 smithi196 ceph-mon[34718]: pgmap v188: 97 pgs: 97 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.9 MiB/s wr, 611 op/s 2024-05-09T02:17:28.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:28 smithi019 ceph-mon[30151]: pgmap v188: 97 pgs: 97 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.9 MiB/s wr, 611 op/s 2024-05-09T02:17:30.462 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:30 smithi019 ceph-mon[30151]: pgmap v189: 97 pgs: 97 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.4 MiB/s wr, 883 op/s 2024-05-09T02:17:30.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.634+0000 7fe48cd89700 1 -- 172.21.15.19:0/4056121580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4881054f0 msgr2=0x7fe488105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:30.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.634+0000 7fe48cd89700 1 --2- 172.21.15.19:0/4056121580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4881054f0 0x7fe488105900 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe478009990 tx=0x7fe478009ca0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.635+0000 7fe48cd89700 1 -- 172.21.15.19:0/4056121580 shutdown_connections 2024-05-09T02:17:30.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.635+0000 7fe48cd89700 1 --2- 172.21.15.19:0/4056121580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe48810e920 0x7fe488110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.635+0000 7fe48cd89700 1 --2- 172.21.15.19:0/4056121580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4881054f0 0x7fe488105900 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.635+0000 7fe48cd89700 1 --2- 172.21.15.19:0/4056121580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe488105e40 0x7fe48810e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.635+0000 7fe48cd89700 1 -- 172.21.15.19:0/4056121580 >> 172.21.15.19:0/4056121580 conn(0x7fe488078e00 msgr2=0x7fe488079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:30.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.635+0000 7fe48cd89700 1 -- 172.21.15.19:0/4056121580 shutdown_connections 2024-05-09T02:17:30.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.635+0000 7fe48cd89700 1 -- 172.21.15.19:0/4056121580 wait complete. 2024-05-09T02:17:30.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.636+0000 7fe48cd89700 1 Processor -- start 2024-05-09T02:17:30.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.636+0000 7fe48cd89700 1 -- start start 2024-05-09T02:17:30.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe48cd89700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4881054f0 0x7fe48819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe48cd89700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe488105e40 0x7fe48819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe48cd89700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe48810e920 0x7fe48819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe48cd89700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe488114470 con 0x7fe4881054f0 2024-05-09T02:17:30.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe48cd89700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881142f0 con 0x7fe48810e920 2024-05-09T02:17:30.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe48cd89700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881145f0 con 0x7fe488105e40 2024-05-09T02:17:30.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe48659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4881054f0 0x7fe48819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe486d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe48810e920 0x7fe48819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe48659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4881054f0 0x7fe48819ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57918/0 (socket says 172.21.15.19:57918) 2024-05-09T02:17:30.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe48659c700 1 -- 172.21.15.19:0/1732840211 learned_addr learned my addr 172.21.15.19:0/1732840211 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:30.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe485d9b700 1 --2- 172.21.15.19:0/1732840211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe488105e40 0x7fe48819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe486d9d700 1 -- 172.21.15.19:0/1732840211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe488105e40 msgr2=0x7fe48819b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:30.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe486d9d700 1 --2- 172.21.15.19:0/1732840211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe488105e40 0x7fe48819b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe486d9d700 1 -- 172.21.15.19:0/1732840211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4881054f0 msgr2=0x7fe48819ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:30.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe486d9d700 1 --2- 172.21.15.19:0/1732840211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4881054f0 0x7fe48819ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.637+0000 7fe486d9d700 1 -- 172.21.15.19:0/1732840211 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48819fd50 con 0x7fe48810e920 2024-05-09T02:17:30.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.638+0000 7fe486d9d700 1 --2- 172.21.15.19:0/1732840211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe48810e920 0x7fe48819f6b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe48000efb0 tx=0x7fe48000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:30.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.638+0000 7fe4777fe700 1 -- 172.21.15.19:0/1732840211 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480003a00 con 0x7fe48810e920 2024-05-09T02:17:30.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.638+0000 7fe48cd89700 1 -- 172.21.15.19:0/1732840211 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4881a0040 con 0x7fe48810e920 2024-05-09T02:17:30.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.638+0000 7fe4777fe700 1 -- 172.21.15.19:0/1732840211 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe48000ce50 con 0x7fe48810e920 2024-05-09T02:17:30.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.638+0000 7fe48cd89700 1 -- 172.21.15.19:0/1732840211 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4881a7960 con 0x7fe48810e920 2024-05-09T02:17:30.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.638+0000 7fe4777fe700 1 -- 172.21.15.19:0/1732840211 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4800047c0 con 0x7fe48810e920 2024-05-09T02:17:30.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.643+0000 7fe4777fe700 1 -- 172.21.15.19:0/1732840211 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe480012050 con 0x7fe48810e920 2024-05-09T02:17:30.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.644+0000 7fe4777fe700 1 --2- 172.21.15.19:0/1732840211 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe4700b2a70 0x7fe4700b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.644+0000 7fe49b79fb80 1 Processor -- start 2024-05-09T02:17:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.644+0000 7fe48659c700 1 --2- 172.21.15.19:0/1732840211 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe4700b2a70 0x7fe4700b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.644+0000 7fe49b79fb80 1 -- start start 2024-05-09T02:17:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.644+0000 7fe4777fe700 1 -- 172.21.15.19:0/1732840211 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fe4800e1e00 con 0x7fe48810e920 2024-05-09T02:17:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.644+0000 7fe49b79fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748d5b160 0x558748df44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.644+0000 7fe49b79fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748df4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.644+0000 7fe49b79fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748dee6a0 0x558748deead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.644+0000 7fe49b79fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558748d10ff0 con 0x558748c714a0 2024-05-09T02:17:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.644+0000 7fe49b79fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558748caf7f0 con 0x558748dee6a0 2024-05-09T02:17:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.644+0000 7fe49b79fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558748d0cfb0 con 0x558748d5b160 2024-05-09T02:17:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.645+0000 7fe485d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748df4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.645+0000 7fe485d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748df4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57920/0 (socket says 172.21.15.19:57920) 2024-05-09T02:17:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.645+0000 7fe48659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748d5b160 0x558748df44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.645+0000 7fe485d9b700 1 -- 172.21.15.19:0/4174681228 learned_addr learned my addr 172.21.15.19:0/4174681228 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:30.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.645+0000 7fe486d9d700 1 --2- 172.21.15.19:0/4174681228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748dee6a0 0x558748deead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.645+0000 7fe48659c700 1 --2- 172.21.15.19:0/1732840211 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe4700b2a70 0x7fe4700b4f80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe47800b2e0 tx=0x7fe478009590 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:30.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.645+0000 7fe485d9b700 1 -- 172.21.15.19:0/4174681228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748d5b160 msgr2=0x558748df44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:30.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.645+0000 7fe485d9b700 1 --2- 172.21.15.19:0/4174681228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748d5b160 0x558748df44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.645+0000 7fe485d9b700 1 -- 172.21.15.19:0/4174681228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748dee6a0 msgr2=0x558748deead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:30.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.645+0000 7fe485d9b700 1 --2- 172.21.15.19:0/4174681228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748dee6a0 0x558748deead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.645+0000 7fe485d9b700 1 -- 172.21.15.19:0/4174681228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558748ca7370 con 0x558748c714a0 2024-05-09T02:17:30.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.645+0000 7fe485d9b700 1 --2- 172.21.15.19:0/4174681228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748df4a30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00bd30 tx=0x7fe47c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f275f59b72cb5c4a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:30.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.646+0000 7fe474ff9700 1 -- 172.21.15.19:0/4174681228 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c015850 con 0x558748c714a0 2024-05-09T02:17:30.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.646+0000 7fe474ff9700 1 -- 172.21.15.19:0/4174681228 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe47c020480 con 0x558748c714a0 2024-05-09T02:17:30.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.646+0000 7fe474ff9700 1 -- 172.21.15.19:0/4174681228 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c01e5e0 con 0x558748c714a0 2024-05-09T02:17:30.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.647+0000 7fe49b79fb80 1 -- 172.21.15.19:0/4174681228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 msgr2=0x558748df4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:30.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.647+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/4174681228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748df4a30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00bd30 tx=0x7fe47c0106b0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.648+0000 7fe49b79fb80 1 -- 172.21.15.19:0/4174681228 shutdown_connections 2024-05-09T02:17:30.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.648+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/4174681228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748dee6a0 0x558748deead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.648+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/4174681228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748d5b160 0x558748df44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.648+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/4174681228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748df4a30 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.648+0000 7fe49b79fb80 1 -- 172.21.15.19:0/4174681228 >> 172.21.15.19:0/4174681228 conn(0x558748cd16a0 msgr2=0x558748de9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:30.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.648+0000 7fe49b79fb80 1 -- 172.21.15.19:0/4174681228 shutdown_connections 2024-05-09T02:17:30.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.648+0000 7fe49b79fb80 1 -- 172.21.15.19:0/4174681228 wait complete. 2024-05-09T02:17:30.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.648+0000 7fe49b79fb80 1 -- start start 2024-05-09T02:17:30.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.649+0000 7fe49b79fb80 1 Processor -- start 2024-05-09T02:17:30.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.649+0000 7fe49b79fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748c714a0 0x558748df8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.649+0000 7fe49b79fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748d5b160 0x558748df8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.649+0000 7fe49b79fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748dee6a0 0x558748df8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.649+0000 7fe49b79fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558748b8cba0 con 0x558748d5b160 2024-05-09T02:17:30.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.649+0000 7fe49b79fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558748ccb150 con 0x558748c714a0 2024-05-09T02:17:30.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.649+0000 7fe49b79fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558748dfb1d0 con 0x558748dee6a0 2024-05-09T02:17:30.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.649+0000 7fe49b79fb80 10 client.0 _ll_register_callbacks cb 0x558748b826c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:17:30.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.649+0000 7fe485d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748d5b160 0x558748df8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.649+0000 7fe48659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748c714a0 0x558748df8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.650+0000 7fe48659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748c714a0 0x558748df8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:60198/0 (socket says 172.21.15.19:60198) 2024-05-09T02:17:30.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.650+0000 7fe48659c700 1 -- 172.21.15.19:0/2742837207 learned_addr learned my addr 172.21.15.19:0/2742837207 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:30.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.650+0000 7fe48659c700 1 -- 172.21.15.19:0/2742837207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748dee6a0 msgr2=0x558748df8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:30.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.650+0000 7fe486d9d700 1 --2- 172.21.15.19:0/2742837207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748dee6a0 0x558748df8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.650+0000 7fe48659c700 1 --2- 172.21.15.19:0/2742837207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748dee6a0 0x558748df8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.650+0000 7fe48659c700 1 -- 172.21.15.19:0/2742837207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748d5b160 msgr2=0x558748df8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.650+0000 7fe48659c700 1 --2- 172.21.15.19:0/2742837207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748d5b160 0x558748df8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.650+0000 7fe48659c700 1 -- 172.21.15.19:0/2742837207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558748b8c490 con 0x558748c714a0 2024-05-09T02:17:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.650+0000 7fe485d9b700 1 --2- 172.21.15.19:0/2742837207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748d5b160 0x558748df8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:17:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.650+0000 7fe48659c700 1 --2- 172.21.15.19:0/2742837207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748c714a0 0x558748df8150 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe47800be50 tx=0x7fe478009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8e7f6988641129f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.651+0000 7fe46dffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:17:30.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.651+0000 7fe46dffb700 1 -- 172.21.15.19:0/2742837207 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478032060 con 0x558748c714a0 2024-05-09T02:17:30.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.651+0000 7fe49b79fb80 1 -- 172.21.15.19:0/2742837207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe47c020480 con 0x558748c714a0 2024-05-09T02:17:30.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.651+0000 7fe46dffb700 1 -- 172.21.15.19:0/2742837207 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fe47800f460 con 0x558748c714a0 2024-05-09T02:17:30.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.651+0000 7fe46dffb700 1 -- 172.21.15.19:0/2742837207 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe478029040 con 0x558748c714a0 2024-05-09T02:17:30.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.651+0000 7fe49b79fb80 10 client.25766 fetch_fsmap learned FSMap version 41 2024-05-09T02:17:30.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.651+0000 7fe49b79fb80 1 -- 172.21.15.19:0/2742837207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558748e05960 con 0x558748c714a0 2024-05-09T02:17:30.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.652+0000 7fe46dffb700 1 -- 172.21.15.19:0/2742837207 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe47800fb10 con 0x558748c714a0 2024-05-09T02:17:30.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.652+0000 7fe46dffb700 1 -- 172.21.15.19:0/2742837207 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7fe47802c070 con 0x558748c714a0 2024-05-09T02:17:30.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.652+0000 7fe49b79fb80 10 client.25766 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:17:30.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.652+0000 7fe49b79fb80 10 client.25766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:17:30.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.652+0000 7fe49b79fb80 20 client.25766 populate_metadata read hostname 'smithi019' 2024-05-09T02:17:30.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.652+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/2742837207 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558748e08380 0x558748e0a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.652+0000 7fe49b79fb80 4 client.25766 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:17:30.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.652+0000 7fe49b79fb80 1 -- 172.21.15.19:0/2742837207 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe47c020480 con 0x558748e08380 2024-05-09T02:17:30.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.652+0000 7fe485d9b700 1 --2- 172.21.15.19:0/2742837207 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558748e08380 0x558748e0a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.653+0000 7fe485d9b700 1 --2- 172.21.15.19:0/2742837207 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558748e08380 0x558748e0a810 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7aa9fdf674eaeb server_cookie=2f64c0624192763d in_seq=0 out_seq=0 2024-05-09T02:17:30.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.653+0000 7fe46dffb700 10 client.25766 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:17:30.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.654+0000 7fe46dffb700 1 -- 172.21.15.19:0/2742837207 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fe47c020480 con 0x558748e08380 2024-05-09T02:17:30.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.654+0000 7fe46dffb700 10 client.25766 handle_command_reply: tid=0 2024-05-09T02:17:30.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.654+0000 7fe49b79fb80 1 client.25766 shutdown 2024-05-09T02:17:30.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.654+0000 7fe49b79fb80 1 -- 172.21.15.19:0/2742837207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748c714a0 msgr2=0x558748df8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:30.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.654+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/2742837207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748c714a0 0x558748df8150 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe47800be50 tx=0x7fe478009ca0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.654+0000 7fe49b79fb80 1 -- 172.21.15.19:0/2742837207 shutdown_connections 2024-05-09T02:17:30.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.654+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/2742837207 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558748e08380 0x558748e0a810 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.654+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/2742837207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748dee6a0 0x558748df8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.654+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/2742837207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748c714a0 0x558748df8150 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.654+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/2742837207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748d5b160 0x558748df8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.654+0000 7fe49b79fb80 1 -- 172.21.15.19:0/2742837207 >> 172.21.15.19:0/2742837207 conn(0x558748cd16a0 msgr2=0x558748de91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:30.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.654+0000 7fe46dffb700 0 client.25766 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:17:30.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.654+0000 7fe49b79fb80 1 -- 172.21.15.19:0/2742837207 shutdown_connections 2024-05-09T02:17:30.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.654+0000 7fe49b79fb80 1 -- 172.21.15.19:0/2742837207 wait complete. 2024-05-09T02:17:30.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.654+0000 7fe49b79fb80 20 client.25766 trim_cache size 0 max 16384 2024-05-09T02:17:30.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.683+0000 7fe49b79fb80 1 Processor -- start 2024-05-09T02:17:30.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.683+0000 7fe49b79fb80 1 -- start start 2024-05-09T02:17:30.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe49b79fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748df7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe49b79fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748d5b160 0x558748e040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe49b79fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748dee6a0 0x558748e10010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe49b79fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881b55e0 con 0x558748c714a0 2024-05-09T02:17:30.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe49b79fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881b71e0 con 0x558748d5b160 2024-05-09T02:17:30.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe49b79fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881b5cb0 con 0x558748dee6a0 2024-05-09T02:17:30.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe48659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748df7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe48659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748df7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57938/0 (socket says 172.21.15.19:57938) 2024-05-09T02:17:30.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe48659c700 1 -- 172.21.15.19:0/2651117509 learned_addr learned my addr 172.21.15.19:0/2651117509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:30.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe485d9b700 1 --2- 172.21.15.19:0/2651117509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748d5b160 0x558748e040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe486d9d700 1 --2- 172.21.15.19:0/2651117509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748dee6a0 0x558748e10010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe48659c700 1 -- 172.21.15.19:0/2651117509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748dee6a0 msgr2=0x558748e10010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:30.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe48659c700 1 --2- 172.21.15.19:0/2651117509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748dee6a0 0x558748e10010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe48659c700 1 -- 172.21.15.19:0/2651117509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748d5b160 msgr2=0x558748e040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:30.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe48659c700 1 --2- 172.21.15.19:0/2651117509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748d5b160 0x558748e040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe48659c700 1 -- 172.21.15.19:0/2651117509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558748dead00 con 0x558748c714a0 2024-05-09T02:17:30.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.684+0000 7fe485d9b700 1 --2- 172.21.15.19:0/2651117509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748d5b160 0x558748e040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:17:30.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.685+0000 7fe48659c700 1 --2- 172.21.15.19:0/2651117509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748df7150 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe4780098e0 tx=0x7fe478003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7689bdd23ff5582f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:30.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.685+0000 7fe46ffff700 1 -- 172.21.15.19:0/2651117509 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478032070 con 0x558748c714a0 2024-05-09T02:17:30.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.685+0000 7fe46ffff700 1 -- 172.21.15.19:0/2651117509 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe478018400 con 0x558748c714a0 2024-05-09T02:17:30.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.685+0000 7fe46ffff700 1 -- 172.21.15.19:0/2651117509 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47800fda0 con 0x558748c714a0 2024-05-09T02:17:30.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.686+0000 7fe49b79fb80 1 -- 172.21.15.19:0/2651117509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 msgr2=0x558748df7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:30.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.686+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/2651117509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748df7150 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe4780098e0 tx=0x7fe478003e60 comp rx=0 tx=0).stop 2024-05-09T02:17:30.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.686+0000 7fe49b79fb80 1 -- 172.21.15.19:0/2651117509 shutdown_connections 2024-05-09T02:17:30.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.686+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/2651117509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748dee6a0 0x558748e10010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.686+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/2651117509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748df7150 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.686+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/2651117509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748d5b160 0x558748e040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.686+0000 7fe49b79fb80 1 -- 172.21.15.19:0/2651117509 >> 172.21.15.19:0/2651117509 conn(0x558748cd16a0 msgr2=0x558748df6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:30.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.686+0000 7fe49b79fb80 1 -- 172.21.15.19:0/2651117509 shutdown_connections 2024-05-09T02:17:30.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.686+0000 7fe49b79fb80 1 -- 172.21.15.19:0/2651117509 wait complete. 2024-05-09T02:17:30.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.686+0000 7fe49b79fb80 1 -- start start 2024-05-09T02:17:30.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.686+0000 7fe49b79fb80 1 Processor -- start 2024-05-09T02:17:30.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe49b79fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748e0f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe49b79fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748d5b160 0x558748e0fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe49b79fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748dee6a0 0x558748e101d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe49b79fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881b6b10 con 0x558748c714a0 2024-05-09T02:17:30.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe49b79fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881b6380 con 0x558748d5b160 2024-05-09T02:17:30.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe49b79fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881b3580 con 0x558748dee6a0 2024-05-09T02:17:30.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe48659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748e0f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe49b79fb80 10 client.0 _ll_register_callbacks cb 0x558748b826c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:17:30.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe48659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748e0f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57946/0 (socket says 172.21.15.19:57946) 2024-05-09T02:17:30.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe48659c700 1 -- 172.21.15.19:0/3312316615 learned_addr learned my addr 172.21.15.19:0/3312316615 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:30.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe486d9d700 1 --2- 172.21.15.19:0/3312316615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748dee6a0 0x558748e101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe485d9b700 1 --2- 172.21.15.19:0/3312316615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748d5b160 0x558748e0fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe48659c700 1 -- 172.21.15.19:0/3312316615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748dee6a0 msgr2=0x558748e101d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:30.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe48659c700 1 --2- 172.21.15.19:0/3312316615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748dee6a0 0x558748e101d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe48659c700 1 -- 172.21.15.19:0/3312316615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748d5b160 msgr2=0x558748e0fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:30.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe48659c700 1 --2- 172.21.15.19:0/3312316615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748d5b160 0x558748e0fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe48659c700 1 -- 172.21.15.19:0/3312316615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558748e71bc0 con 0x558748c714a0 2024-05-09T02:17:30.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.687+0000 7fe485d9b700 1 --2- 172.21.15.19:0/3312316615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748d5b160 0x558748e0fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:17:30.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.689+0000 7fe48659c700 1 --2- 172.21.15.19:0/3312316615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748e0f750 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe47800be50 tx=0x7fe478018cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40362afd47a92c98 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.689+0000 7fe46dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:17:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.689+0000 7fe46dffb700 1 -- 172.21.15.19:0/3312316615 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478032040 con 0x558748c714a0 2024-05-09T02:17:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.689+0000 7fe46dffb700 1 -- 172.21.15.19:0/3312316615 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fe47800f670 con 0x558748c714a0 2024-05-09T02:17:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.689+0000 7fe49b79fb80 1 -- 172.21.15.19:0/3312316615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe478018400 con 0x558748c714a0 2024-05-09T02:17:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.690+0000 7fe46dffb700 1 -- 172.21.15.19:0/3312316615 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe47800fa20 con 0x558748c714a0 2024-05-09T02:17:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.690+0000 7fe49b79fb80 10 client.16053 fetch_fsmap learned FSMap version 41 2024-05-09T02:17:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.690+0000 7fe49b79fb80 1 -- 172.21.15.19:0/3312316615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558748e72cd0 con 0x558748c714a0 2024-05-09T02:17:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.690+0000 7fe46dffb700 1 -- 172.21.15.19:0/3312316615 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe478018400 con 0x558748c714a0 2024-05-09T02:17:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.691+0000 7fe46dffb700 1 -- 172.21.15.19:0/3312316615 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7fe47802c070 con 0x558748c714a0 2024-05-09T02:17:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.691+0000 7fe49b79fb80 10 client.16053 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T02:17:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.691+0000 7fe49b79fb80 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:17:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.691+0000 7fe49b79fb80 20 client.16053 populate_metadata read hostname 'smithi019' 2024-05-09T02:17:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.691+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/3312316615 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558748e73350 0x558748e75770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.691+0000 7fe49b79fb80 4 client.16053 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:17:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.691+0000 7fe49b79fb80 1 -- 172.21.15.19:0/3312316615 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558748d1e9d0 con 0x558748e73350 2024-05-09T02:17:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.691+0000 7fe485d9b700 1 --2- 172.21.15.19:0/3312316615 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558748e73350 0x558748e75770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.691+0000 7fe485d9b700 1 --2- 172.21.15.19:0/3312316615 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558748e73350 0x558748e75770 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89b364ee1a834f96 server_cookie=ee365fe2aecc658 in_seq=0 out_seq=0 2024-05-09T02:17:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.691+0000 7fe46dffb700 10 client.16053 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:17:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.692+0000 7fe46dffb700 1 -- 172.21.15.19:0/3312316615 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558748d1e9d0 con 0x558748e73350 2024-05-09T02:17:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.692+0000 7fe46dffb700 10 client.16053 handle_command_reply: tid=0 2024-05-09T02:17:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.692+0000 7fe49b79fb80 1 client.16053 shutdown 2024-05-09T02:17:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.692+0000 7fe49b79fb80 1 -- 172.21.15.19:0/3312316615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 msgr2=0x558748e0f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.692+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/3312316615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748e0f750 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe47800be50 tx=0x7fe478018cf0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.692+0000 7fe49b79fb80 1 -- 172.21.15.19:0/3312316615 shutdown_connections 2024-05-09T02:17:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.692+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/3312316615 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558748e73350 0x558748e75770 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.692+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/3312316615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558748dee6a0 0x558748e101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.692+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/3312316615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558748c714a0 0x558748e0f750 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.692+0000 7fe49b79fb80 1 --2- 172.21.15.19:0/3312316615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558748d5b160 0x558748e0fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.692+0000 7fe49b79fb80 1 -- 172.21.15.19:0/3312316615 >> 172.21.15.19:0/3312316615 conn(0x558748cd16a0 msgr2=0x558748df1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:30.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.692+0000 7fe46dffb700 0 client.16053 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:17:30.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.692+0000 7fe49b79fb80 1 -- 172.21.15.19:0/3312316615 shutdown_connections 2024-05-09T02:17:30.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.692+0000 7fe49b79fb80 1 -- 172.21.15.19:0/3312316615 wait complete. 2024-05-09T02:17:30.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.692+0000 7fe49b79fb80 20 client.16053 trim_cache size 0 max 16384 2024-05-09T02:17:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:17:30.698 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T02:17:30.698 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T02:17:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:17:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.693+0000 7fe48cd89700 1 -- 172.21.15.19:0/1732840211 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe4700b2a70 msgr2=0x7fe4700b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.693+0000 7fe48cd89700 1 --2- 172.21.15.19:0/1732840211 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe4700b2a70 0x7fe4700b4f80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe47800b2e0 tx=0x7fe478009590 comp rx=0 tx=0).stop 2024-05-09T02:17:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.693+0000 7fe48cd89700 1 -- 172.21.15.19:0/1732840211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe48810e920 msgr2=0x7fe48819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.693+0000 7fe48cd89700 1 --2- 172.21.15.19:0/1732840211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe48810e920 0x7fe48819f6b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe48000efb0 tx=0x7fe48000c3a0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.693+0000 7fe48cd89700 1 -- 172.21.15.19:0/1732840211 shutdown_connections 2024-05-09T02:17:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.693+0000 7fe48cd89700 1 --2- 172.21.15.19:0/1732840211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe48810e920 0x7fe48819f6b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.693+0000 7fe48cd89700 1 --2- 172.21.15.19:0/1732840211 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe4700b2a70 0x7fe4700b4f80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.693+0000 7fe48cd89700 1 --2- 172.21.15.19:0/1732840211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4881054f0 0x7fe48819ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.693+0000 7fe48cd89700 1 --2- 172.21.15.19:0/1732840211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe488105e40 0x7fe48819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:30.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.693+0000 7fe48cd89700 1 -- 172.21.15.19:0/1732840211 >> 172.21.15.19:0/1732840211 conn(0x7fe488078e00 msgr2=0x7fe4881011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:30.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.693+0000 7fe48cd89700 1 -- 172.21.15.19:0/1732840211 shutdown_connections 2024-05-09T02:17:30.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:30.693+0000 7fe48cd89700 1 -- 172.21.15.19:0/1732840211 wait complete. 2024-05-09T02:17:30.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:30 smithi059 ceph-mon[34781]: pgmap v189: 97 pgs: 97 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.4 MiB/s wr, 883 op/s 2024-05-09T02:17:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:30 smithi196 ceph-mon[34718]: pgmap v189: 97 pgs: 97 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.4 MiB/s wr, 883 op/s 2024-05-09T02:17:31.546 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T02:17:31.546 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:17:31.946 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:17:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 350.0 s, 2497.0 tps, lat 39.980 ms stddev 42.440 2024-05-09T02:17:32.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:32 smithi019 ceph-mon[30151]: pgmap v190: 97 pgs: 97 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 570 op/s 2024-05-09T02:17:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:32 smithi059 ceph-mon[34781]: pgmap v190: 97 pgs: 97 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 570 op/s 2024-05-09T02:17:32.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:32 smithi196 ceph-mon[34718]: pgmap v190: 97 pgs: 97 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 570 op/s 2024-05-09T02:17:33.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:33 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:33.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:33 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:33.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:33 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:34 smithi059 ceph-mon[34781]: pgmap v191: 97 pgs: 97 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 574 op/s 2024-05-09T02:17:34.722 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:34 smithi019 ceph-mon[30151]: pgmap v191: 97 pgs: 97 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 574 op/s 2024-05-09T02:17:34.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:34 smithi196 ceph-mon[34718]: pgmap v191: 97 pgs: 97 active+clean; 4.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 574 op/s 2024-05-09T02:17:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.698+0000 7f1940c80700 1 -- 172.21.15.19:0/4274273548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193c10b010 msgr2=0x7f193c10b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.698+0000 7f1940c80700 1 --2- 172.21.15.19:0/4274273548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193c10b010 0x7f193c10b420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f192c008570 tx=0x7f192c00af90 comp rx=0 tx=0).stop 2024-05-09T02:17:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.700+0000 7f1940c80700 1 -- 172.21.15.19:0/4274273548 shutdown_connections 2024-05-09T02:17:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.700+0000 7f1940c80700 1 --2- 172.21.15.19:0/4274273548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f193c110290 0x7f193c117bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.700+0000 7f1940c80700 1 --2- 172.21.15.19:0/4274273548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193c10b010 0x7f193c10b420 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.700+0000 7f1940c80700 1 --2- 172.21.15.19:0/4274273548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f193c10f940 0x7f193c10fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.700+0000 7f1940c80700 1 -- 172.21.15.19:0/4274273548 >> 172.21.15.19:0/4274273548 conn(0x7f193c06f4d0 msgr2=0x7f193c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.700+0000 7f1940c80700 1 -- 172.21.15.19:0/4274273548 shutdown_connections 2024-05-09T02:17:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.701+0000 7f1940c80700 1 -- 172.21.15.19:0/4274273548 wait complete. 2024-05-09T02:17:35.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.704+0000 7f1940c80700 1 Processor -- start 2024-05-09T02:17:35.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.704+0000 7f1940c80700 1 -- start start 2024-05-09T02:17:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.705+0000 7f1940c80700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f193c10b010 0x7f193c1b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.705+0000 7f1940c80700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f193c10f940 0x7f193c1b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.705+0000 7f1940c80700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193c110290 0x7f193c1b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.705+0000 7f193a59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f193c10b010 0x7f193c1b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.705+0000 7f193a59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f193c10b010 0x7f193c1b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:60560/0 (socket says 172.21.15.19:60560) 2024-05-09T02:17:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.705+0000 7f193a59c700 1 -- 172.21.15.19:0/160179256 learned_addr learned my addr 172.21.15.19:0/160179256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.705+0000 7f1940c80700 1 -- 172.21.15.19:0/160179256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f193c12bb30 con 0x7f193c10f940 2024-05-09T02:17:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.705+0000 7f1931d9b700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f193c10f940 0x7f193c1b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.705+0000 7f1940c80700 1 -- 172.21.15.19:0/160179256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f193c12b9b0 con 0x7f193c110290 2024-05-09T02:17:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.705+0000 7f1940c80700 1 -- 172.21.15.19:0/160179256 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f193c12bcb0 con 0x7f193c10b010 2024-05-09T02:17:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.706+0000 7f193ad9d700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193c110290 0x7f193c1b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.706+0000 7f193a59c700 1 -- 172.21.15.19:0/160179256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193c110290 msgr2=0x7f193c1b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.706+0000 7f193a59c700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193c110290 0x7f193c1b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.706+0000 7f193a59c700 1 -- 172.21.15.19:0/160179256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f193c10f940 msgr2=0x7f193c1b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.706+0000 7f193a59c700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f193c10f940 0x7f193c1b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.706+0000 7f193a59c700 1 -- 172.21.15.19:0/160179256 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193c1b7120 con 0x7f193c10b010 2024-05-09T02:17:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.706+0000 7f1931d9b700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f193c10f940 0x7f193c1b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:17:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.706+0000 7f193ad9d700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193c110290 0x7f193c1b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:17:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.707+0000 7f193a59c700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f193c10b010 0x7f193c1b2250 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f192c003ed0 tx=0x7f192c003f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.707+0000 7f1933fff700 1 -- 172.21.15.19:0/160179256 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c008c60 con 0x7f193c10b010 2024-05-09T02:17:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.707+0000 7f1933fff700 1 -- 172.21.15.19:0/160179256 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f192c00f250 con 0x7f193c10b010 2024-05-09T02:17:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.707+0000 7f1933fff700 1 -- 172.21.15.19:0/160179256 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c004ad0 con 0x7f193c10b010 2024-05-09T02:17:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.707+0000 7f1940c80700 1 -- 172.21.15.19:0/160179256 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193c1b73b0 con 0x7f193c10b010 2024-05-09T02:17:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.707+0000 7f1940c80700 1 -- 172.21.15.19:0/160179256 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193c1b78a0 con 0x7f193c10b010 2024-05-09T02:17:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.710+0000 7f1940c80700 1 -- 172.21.15.19:0/160179256 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f193c0512c0 con 0x7f193c10b010 2024-05-09T02:17:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.713+0000 7f1933fff700 1 -- 172.21.15.19:0/160179256 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f192c004c70 con 0x7f193c10b010 2024-05-09T02:17:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.714+0000 7f1933fff700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f19280b2ab0 0x7f19280b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.714+0000 7f1933fff700 1 -- 172.21.15.19:0/160179256 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f192c0e2c60 con 0x7f193c10b010 2024-05-09T02:17:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.714+0000 7f1931d9b700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f19280b2ab0 0x7f19280b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.716+0000 7f1931d9b700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f19280b2ab0 0x7f19280b4fc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f193c10f6e0 tx=0x7f1924009190 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:35.720+0000 7f1933fff700 1 -- 172.21.15.19:0/160179256 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f192c0aa6d0 con 0x7f193c10b010 2024-05-09T02:17:36.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:36.135+0000 7f1940c80700 1 -- 172.21.15.19:0/160179256 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f193c1bed30 con 0x7f193c10b010 2024-05-09T02:17:36.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:36.136+0000 7f1933fff700 1 -- 172.21.15.19:0/160179256 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11924 (secure 0 0 0) 0x7f192c0a9fd0 con 0x7f193c10b010 2024-05-09T02:17:36.139 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:17:36.139 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:17:31.829058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24457,24455,24473,24463,24484]},"id":1}]} 2024-05-09T02:17:36.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:36.139+0000 7f1940c80700 1 -- 172.21.15.19:0/160179256 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f19280b2ab0 msgr2=0x7f19280b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:36.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:36.139+0000 7f1940c80700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f19280b2ab0 0x7f19280b4fc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f193c10f6e0 tx=0x7f1924009190 comp rx=0 tx=0).stop 2024-05-09T02:17:36.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:36.139+0000 7f1940c80700 1 -- 172.21.15.19:0/160179256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f193c10b010 msgr2=0x7f193c1b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:36.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:36.139+0000 7f1940c80700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f193c10b010 0x7f193c1b2250 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f192c003ed0 tx=0x7f192c003f00 comp rx=0 tx=0).stop 2024-05-09T02:17:36.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:36.140+0000 7f1940c80700 1 -- 172.21.15.19:0/160179256 shutdown_connections 2024-05-09T02:17:36.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:36.140+0000 7f1940c80700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193c110290 0x7f193c1b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:36.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:36.140+0000 7f1940c80700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f193c10b010 0x7f193c1b2250 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:36.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:36.140+0000 7f1940c80700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f19280b2ab0 0x7f19280b4fc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:36.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:36.140+0000 7f1940c80700 1 --2- 172.21.15.19:0/160179256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f193c10f940 0x7f193c1b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:36.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:36.140+0000 7f1940c80700 1 -- 172.21.15.19:0/160179256 >> 172.21.15.19:0/160179256 conn(0x7f193c06f4d0 msgr2=0x7f193c10d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:36.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:36.140+0000 7f1940c80700 1 -- 172.21.15.19:0/160179256 shutdown_connections 2024-05-09T02:17:36.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:36.140+0000 7f1940c80700 1 -- 172.21.15.19:0/160179256 wait complete. 2024-05-09T02:17:36.143 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 42 2024-05-09T02:17:36.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:36 smithi059 ceph-mon[34781]: pgmap v192: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.7 MiB/s wr, 877 op/s 2024-05-09T02:17:36.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:36 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/160179256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:17:36.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:36 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:17:36.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:36 smithi196 ceph-mon[34718]: pgmap v192: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.7 MiB/s wr, 877 op/s 2024-05-09T02:17:36.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:36 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/160179256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:17:36.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:36 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:17:36.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:36 smithi019 ceph-mon[30151]: pgmap v192: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.7 MiB/s wr, 877 op/s 2024-05-09T02:17:36.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:36 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/160179256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:17:36.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:36 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:17:36.994 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T02:17:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:17:37.532 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:37 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:37.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:37 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:17:37.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:37 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:17:37.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:37 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:17:37.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 355.0 s, 1738.4 tps, lat 57.588 ms stddev 63.173 2024-05-09T02:17:37.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:37 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:37.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:37 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:17:37.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:37 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:17:37.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:37 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:17:37.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:37 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:37.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:37 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:17:37.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:37 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:17:37.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:37 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:17:38.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:38 smithi059 ceph-mon[34781]: pgmap v193: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 582 op/s 2024-05-09T02:17:38.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:38 smithi196 ceph-mon[34718]: pgmap v193: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 582 op/s 2024-05-09T02:17:38.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:38 smithi019 ceph-mon[30151]: pgmap v193: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 582 op/s 2024-05-09T02:17:39.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.526+0000 7f4b51b8e700 1 -- 172.21.15.19:0/469670579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b4c107df0 msgr2=0x7f4b4c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:39.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.526+0000 7f4b51b8e700 1 --2- 172.21.15.19:0/469670579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b4c107df0 0x7f4b4c108200 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4b400099c0 tx=0x7f4b40009cd0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.526+0000 7f4b51b8e700 1 -- 172.21.15.19:0/469670579 shutdown_connections 2024-05-09T02:17:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.526+0000 7f4b51b8e700 1 --2- 172.21.15.19:0/469670579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b4c108740 0x7f4b4c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.527+0000 7f4b51b8e700 1 --2- 172.21.15.19:0/469670579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b4c1034c0 0x7f4b4c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.527+0000 7f4b51b8e700 1 --2- 172.21.15.19:0/469670579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b4c107df0 0x7f4b4c108200 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.527+0000 7f4b51b8e700 1 -- 172.21.15.19:0/469670579 >> 172.21.15.19:0/469670579 conn(0x7f4b4c0fedc0 msgr2=0x7f4b4c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.527+0000 7f4b51b8e700 1 -- 172.21.15.19:0/469670579 shutdown_connections 2024-05-09T02:17:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.527+0000 7f4b51b8e700 1 -- 172.21.15.19:0/469670579 wait complete. 2024-05-09T02:17:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.528+0000 7f4b51b8e700 1 Processor -- start 2024-05-09T02:17:39.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.529+0000 7f4b51b8e700 1 -- start start 2024-05-09T02:17:39.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.529+0000 7f4b51b8e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b4c1034c0 0x7f4b4c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.529+0000 7f4b51b8e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b4c107df0 0x7f4b4c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.529+0000 7f4b51b8e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b4c108740 0x7f4b4c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.529+0000 7f4b51b8e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c114460 con 0x7f4b4c108740 2024-05-09T02:17:39.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.529+0000 7f4b51b8e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1142e0 con 0x7f4b4c1034c0 2024-05-09T02:17:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.529+0000 7f4b51b8e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1145e0 con 0x7f4b4c107df0 2024-05-09T02:17:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.529+0000 7f4b4b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b4c1034c0 0x7f4b4c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.529+0000 7f4b4b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b4c1034c0 0x7f4b4c19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:45254/0 (socket says 172.21.15.19:45254) 2024-05-09T02:17:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.529+0000 7f4b4b7fe700 1 -- 172.21.15.19:0/1368356988 learned_addr learned my addr 172.21.15.19:0/1368356988 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.529+0000 7f4b4affd700 1 --2- 172.21.15.19:0/1368356988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b4c107df0 0x7f4b4c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.529+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/1368356988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b4c108740 0x7f4b4c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.530+0000 7f4b4b7fe700 1 -- 172.21.15.19:0/1368356988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b4c107df0 msgr2=0x7f4b4c19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.530+0000 7f4b4b7fe700 1 --2- 172.21.15.19:0/1368356988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b4c107df0 0x7f4b4c19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.530+0000 7f4b4b7fe700 1 -- 172.21.15.19:0/1368356988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b4c108740 msgr2=0x7f4b4c19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.530+0000 7f4b4b7fe700 1 --2- 172.21.15.19:0/1368356988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b4c108740 0x7f4b4c19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.530+0000 7f4b4b7fe700 1 -- 172.21.15.19:0/1368356988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c19fcc0 con 0x7f4b4c1034c0 2024-05-09T02:17:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.530+0000 7f4b4b7fe700 1 --2- 172.21.15.19:0/1368356988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b4c1034c0 0x7f4b4c19adf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00e9a0 tx=0x7f4b3c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.530+0000 7f4b48ff9700 1 -- 172.21.15.19:0/1368356988 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c00ca10 con 0x7f4b4c1034c0 2024-05-09T02:17:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.530+0000 7f4b48ff9700 1 -- 172.21.15.19:0/1368356988 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b3c00cbb0 con 0x7f4b4c1034c0 2024-05-09T02:17:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.530+0000 7f4b51b8e700 1 -- 172.21.15.19:0/1368356988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4c19ffb0 con 0x7f4b4c1034c0 2024-05-09T02:17:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.530+0000 7f4b48ff9700 1 -- 172.21.15.19:0/1368356988 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c018680 con 0x7f4b4c1034c0 2024-05-09T02:17:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.530+0000 7f4b51b8e700 1 -- 172.21.15.19:0/1368356988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4c1a78d0 con 0x7f4b4c1034c0 2024-05-09T02:17:39.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.536+0000 7f4b605a4b80 1 Processor -- start 2024-05-09T02:17:39.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.536+0000 7f4b605a4b80 1 -- start start 2024-05-09T02:17:39.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.536+0000 7f4b605a4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 0x561d794c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.536+0000 7f4b605a4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 0x561d794c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.536+0000 7f4b605a4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794c5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.536+0000 7f4b605a4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561d793e6ff0 con 0x561d793474a0 2024-05-09T02:17:39.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.536+0000 7f4b605a4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561d793857f0 con 0x561d794c4c80 2024-05-09T02:17:39.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.536+0000 7f4b605a4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561d793e2fb0 con 0x561d79431160 2024-05-09T02:17:39.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.537+0000 7f4b4bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 0x561d794c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.537+0000 7f4b4bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 0x561d794c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:39242/0 (socket says 172.21.15.19:39242) 2024-05-09T02:17:39.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.537+0000 7f4b4bfff700 1 -- 172.21.15.19:0/649266344 learned_addr learned my addr 172.21.15.19:0/649266344 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:39.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.537+0000 7f4b4affd700 1 --2- 172.21.15.19:0/649266344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.537+0000 7f4b4bfff700 1 -- 172.21.15.19:0/649266344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 msgr2=0x561d794c5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.537+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/649266344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794c5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.537+0000 7f4b4bfff700 1 -- 172.21.15.19:0/649266344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 msgr2=0x561d794c46a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:17:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.537+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/649266344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 0x561d794c46a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.537+0000 7f4b4bfff700 1 -- 172.21.15.19:0/649266344 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7937d370 con 0x561d79431160 2024-05-09T02:17:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.537+0000 7f4b48ff9700 1 -- 172.21.15.19:0/1368356988 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4b3c010430 con 0x7f4b4c1034c0 2024-05-09T02:17:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.537+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/649266344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 0x561d794c84f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4b44011680 tx=0x7f4b44009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=233a62ab0f5eaddd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.538+0000 7f4b31ffb700 1 -- 172.21.15.19:0/649266344 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44019030 con 0x561d79431160 2024-05-09T02:17:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.538+0000 7f4b31ffb700 1 -- 172.21.15.19:0/649266344 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b440092e0 con 0x561d79431160 2024-05-09T02:17:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.538+0000 7f4b31ffb700 1 -- 172.21.15.19:0/649266344 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44007500 con 0x561d79431160 2024-05-09T02:17:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.538+0000 7f4b48ff9700 1 --2- 172.21.15.19:0/1368356988 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f4b340b2a70 0x7f4b340b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.539+0000 7f4b48ff9700 1 -- 172.21.15.19:0/1368356988 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f4b3c0e2750 con 0x7f4b4c1034c0 2024-05-09T02:17:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.539+0000 7f4b4b7fe700 1 --2- 172.21.15.19:0/1368356988 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f4b340b2a70 0x7f4b340b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.539+0000 7f4b605a4b80 1 -- 172.21.15.19:0/649266344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 msgr2=0x561d794c84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.539+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/649266344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 0x561d794c84f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4b44011680 tx=0x7f4b44009920 comp rx=0 tx=0).stop 2024-05-09T02:17:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.539+0000 7f4b605a4b80 1 -- 172.21.15.19:0/649266344 shutdown_connections 2024-05-09T02:17:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.539+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/649266344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794c5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.539+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/649266344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 0x561d794c84f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.539+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/649266344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 0x561d794c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.539+0000 7f4b605a4b80 1 -- 172.21.15.19:0/649266344 >> 172.21.15.19:0/649266344 conn(0x561d793a76a0 msgr2=0x561d794bf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.539+0000 7f4b605a4b80 1 -- 172.21.15.19:0/649266344 shutdown_connections 2024-05-09T02:17:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.540+0000 7f4b605a4b80 1 -- 172.21.15.19:0/649266344 wait complete. 2024-05-09T02:17:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.540+0000 7f4b4b7fe700 1 --2- 172.21.15.19:0/1368356988 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f4b340b2a70 0x7f4b340b4f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x561d794c0c40 tx=0x7f4b3c009790 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.540+0000 7f4b605a4b80 1 -- start start 2024-05-09T02:17:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.540+0000 7f4b605a4b80 1 Processor -- start 2024-05-09T02:17:39.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.541+0000 7f4b605a4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d793474a0 0x561d794d6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.541+0000 7f4b605a4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d79431160 0x561d794d6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.541+0000 7f4b605a4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794d6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.541+0000 7f4b605a4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561d79262ba0 con 0x561d79431160 2024-05-09T02:17:39.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.541+0000 7f4b605a4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561d793a1150 con 0x561d794c4c80 2024-05-09T02:17:39.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.541+0000 7f4b4bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d793474a0 0x561d794d6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.541+0000 7f4b4bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d793474a0 0x561d794d6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:39246/0 (socket says 172.21.15.19:39246) 2024-05-09T02:17:39.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.541+0000 7f4b4bfff700 1 -- 172.21.15.19:0/1009396273 learned_addr learned my addr 172.21.15.19:0/1009396273 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:39.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.541+0000 7f4b605a4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561d794d9850 con 0x561d793474a0 2024-05-09T02:17:39.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.541+0000 7f4b605a4b80 10 client.0 _ll_register_callbacks cb 0x561d792586c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:17:39.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.542+0000 7f4b4affd700 1 --2- 172.21.15.19:0/1009396273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d79431160 0x561d794d6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.542+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/1009396273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794d6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.542+0000 7f4b4affd700 1 -- 172.21.15.19:0/1009396273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d793474a0 msgr2=0x561d794d6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:39.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.542+0000 7f4b4affd700 1 --2- 172.21.15.19:0/1009396273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d793474a0 0x561d794d6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.542+0000 7f4b4affd700 1 -- 172.21.15.19:0/1009396273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 msgr2=0x561d794d6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:39.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.543+0000 7f4b4affd700 1 --2- 172.21.15.19:0/1009396273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794d6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.543+0000 7f4b4affd700 1 -- 172.21.15.19:0/1009396273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d79262490 con 0x561d79431160 2024-05-09T02:17:39.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.543+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/1009396273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794d6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:17:39.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.543+0000 7f4b2affd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:17:39.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.543+0000 7f4b4affd700 1 --2- 172.21.15.19:0/1009396273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d79431160 0x561d794d6a90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000b580 tx=0x7f4b40005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10f6da473945665 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:39.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.544+0000 7f4b2affd700 1 -- 172.21.15.19:0/1009396273 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b400039c0 con 0x561d79431160 2024-05-09T02:17:39.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.544+0000 7f4b2affd700 1 -- 172.21.15.19:0/1009396273 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f4b400037c0 con 0x561d79431160 2024-05-09T02:17:39.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.544+0000 7f4b605a4b80 1 -- 172.21.15.19:0/1009396273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b440092e0 con 0x561d79431160 2024-05-09T02:17:39.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.544+0000 7f4b2affd700 1 -- 172.21.15.19:0/1009396273 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4b40031050 con 0x561d79431160 2024-05-09T02:17:39.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.544+0000 7f4b605a4b80 10 client.16065 fetch_fsmap learned FSMap version 43 2024-05-09T02:17:39.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.544+0000 7f4b605a4b80 1 -- 172.21.15.19:0/1009396273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561d794f5290 con 0x561d79431160 2024-05-09T02:17:39.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.545+0000 7f4b2affd700 1 -- 172.21.15.19:0/1009396273 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b440092e0 con 0x561d79431160 2024-05-09T02:17:39.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.545+0000 7f4b2affd700 1 -- 172.21.15.19:0/1009396273 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 43) ==== 8366+0+0 (secure 0 0 0) 0x7f4b4002c030 con 0x561d79431160 2024-05-09T02:17:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.545+0000 7f4b605a4b80 10 client.16065 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T02:17:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.545+0000 7f4b605a4b80 10 client.16065 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:17:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.545+0000 7f4b605a4b80 20 client.16065 populate_metadata read hostname 'smithi019' 2024-05-09T02:17:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.545+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/1009396273 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x561d794f7970 0x561d794f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.545+0000 7f4b605a4b80 4 client.16065 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:17:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.545+0000 7f4b605a4b80 1 -- 172.21.15.19:0/1009396273 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d793f49d0 con 0x561d794f7970 2024-05-09T02:17:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.546+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/1009396273 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x561d794f7970 0x561d794f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.546+0000 7f4b2affd700 10 client.16065 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:17:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.546+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/1009396273 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x561d794f7970 0x561d794f9db0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ba5b8ad2b682e657 server_cookie=aee2ec492c8468ae in_seq=0 out_seq=0 2024-05-09T02:17:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.547+0000 7f4b2affd700 1 -- 172.21.15.19:0/1009396273 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561d793f49d0 con 0x561d794f7970 2024-05-09T02:17:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.547+0000 7f4b2affd700 10 client.16065 handle_command_reply: tid=0 2024-05-09T02:17:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.547+0000 7f4b605a4b80 1 client.16065 shutdown 2024-05-09T02:17:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.547+0000 7f4b605a4b80 1 -- 172.21.15.19:0/1009396273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d79431160 msgr2=0x561d794d6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.547+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/1009396273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d79431160 0x561d794d6a90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000b580 tx=0x7f4b40005ad0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.548+0000 7f4b605a4b80 1 -- 172.21.15.19:0/1009396273 shutdown_connections 2024-05-09T02:17:39.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.548+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/1009396273 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x561d794f7970 0x561d794f9db0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.548+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/1009396273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794d6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.548+0000 7f4b2affd700 0 client.16065 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:17:39.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.548+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/1009396273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d793474a0 0x561d794d6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.548+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/1009396273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d79431160 0x561d794d6a90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.548+0000 7f4b605a4b80 1 -- 172.21.15.19:0/1009396273 >> 172.21.15.19:0/1009396273 conn(0x561d793a76a0 msgr2=0x561d794bf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:39.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.548+0000 7f4b605a4b80 1 -- 172.21.15.19:0/1009396273 shutdown_connections 2024-05-09T02:17:39.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.548+0000 7f4b605a4b80 1 -- 172.21.15.19:0/1009396273 wait complete. 2024-05-09T02:17:39.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.548+0000 7f4b605a4b80 20 client.16065 trim_cache size 0 max 16384 2024-05-09T02:17:39.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.573+0000 7f4b605a4b80 1 Processor -- start 2024-05-09T02:17:39.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.573+0000 7f4b605a4b80 1 -- start start 2024-05-09T02:17:39.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.573+0000 7f4b605a4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 0x561d794c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.573+0000 7f4b605a4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 0x561d794d6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.573+0000 7f4b605a4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794ff570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.573+0000 7f4b605a4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c072330 con 0x561d793474a0 2024-05-09T02:17:39.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.573+0000 7f4b605a4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c073f00 con 0x561d794c4c80 2024-05-09T02:17:39.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.573+0000 7f4b605a4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c072a00 con 0x561d79431160 2024-05-09T02:17:39.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.574+0000 7f4b4bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 0x561d794c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.574+0000 7f4b4bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 0x561d794c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53676/0 (socket says 172.21.15.19:53676) 2024-05-09T02:17:39.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.574+0000 7f4b4bfff700 1 -- 172.21.15.19:0/2849765345 learned_addr learned my addr 172.21.15.19:0/2849765345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:39.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.574+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/2849765345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.574+0000 7f4b4affd700 1 --2- 172.21.15.19:0/2849765345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 0x561d794d6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.574+0000 7f4b4bfff700 1 -- 172.21.15.19:0/2849765345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 msgr2=0x561d794d6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:39.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.574+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/2849765345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 0x561d794d6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.574+0000 7f4b4bfff700 1 -- 172.21.15.19:0/2849765345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 msgr2=0x561d794ff570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:39.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.574+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/2849765345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794ff570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.574+0000 7f4b4bfff700 1 -- 172.21.15.19:0/2849765345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d794d8df0 con 0x561d793474a0 2024-05-09T02:17:39.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.574+0000 7f4b4affd700 1 --2- 172.21.15.19:0/2849765345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 0x561d794d6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:17:39.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.575+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/2849765345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 0x561d794c40c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009fd0 tx=0x7f4b4400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171f5f263cc28e00 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:39.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.575+0000 7f4b317fa700 1 -- 172.21.15.19:0/2849765345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44019070 con 0x561d793474a0 2024-05-09T02:17:39.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.575+0000 7f4b317fa700 1 -- 172.21.15.19:0/2849765345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b44003eb0 con 0x561d793474a0 2024-05-09T02:17:39.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.575+0000 7f4b317fa700 1 -- 172.21.15.19:0/2849765345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440288b0 con 0x561d793474a0 2024-05-09T02:17:39.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.576+0000 7f4b605a4b80 1 -- 172.21.15.19:0/2849765345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 msgr2=0x561d794c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:39.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.577+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/2849765345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 0x561d794c40c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009fd0 tx=0x7f4b4400c040 comp rx=0 tx=0).stop 2024-05-09T02:17:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.578+0000 7f4b605a4b80 1 -- 172.21.15.19:0/2849765345 shutdown_connections 2024-05-09T02:17:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.578+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/2849765345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.578+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/2849765345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 0x561d794c40c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.578+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/2849765345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 0x561d794d6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.578+0000 7f4b605a4b80 1 -- 172.21.15.19:0/2849765345 >> 172.21.15.19:0/2849765345 conn(0x561d793a76a0 msgr2=0x561d794d54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.578+0000 7f4b605a4b80 1 -- 172.21.15.19:0/2849765345 shutdown_connections 2024-05-09T02:17:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.578+0000 7f4b605a4b80 1 -- 172.21.15.19:0/2849765345 wait complete. 2024-05-09T02:17:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.579+0000 7f4b605a4b80 1 -- start start 2024-05-09T02:17:39.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.580+0000 7f4b605a4b80 1 Processor -- start 2024-05-09T02:17:39.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.580+0000 7f4b605a4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 0x561d794ded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.580+0000 7f4b605a4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 0x561d794da0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.580+0000 7f4b605a4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794da620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.580+0000 7f4b4bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 0x561d794ded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.580+0000 7f4b4bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 0x561d794ded30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53688/0 (socket says 172.21.15.19:53688) 2024-05-09T02:17:39.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.580+0000 7f4b4bfff700 1 -- 172.21.15.19:0/858408856 learned_addr learned my addr 172.21.15.19:0/858408856 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:39.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.580+0000 7f4b605a4b80 1 -- 172.21.15.19:0/858408856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c073830 con 0x561d793474a0 2024-05-09T02:17:39.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.580+0000 7f4b605a4b80 1 -- 172.21.15.19:0/858408856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c0730d0 con 0x561d794c4c80 2024-05-09T02:17:39.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.580+0000 7f4b605a4b80 1 -- 172.21.15.19:0/858408856 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c0702f0 con 0x561d79431160 2024-05-09T02:17:39.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.581+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/858408856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794da620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.581+0000 7f4b605a4b80 10 client.0 _ll_register_callbacks cb 0x561d792586c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:17:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.581+0000 7f4b4affd700 1 --2- 172.21.15.19:0/858408856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 0x561d794da0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.581+0000 7f4b4bfff700 1 -- 172.21.15.19:0/858408856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 msgr2=0x561d794da0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.581+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/858408856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 0x561d794da0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.581+0000 7f4b4bfff700 1 -- 172.21.15.19:0/858408856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 msgr2=0x561d794da620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.581+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/858408856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794da620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.581+0000 7f4b4bfff700 1 -- 172.21.15.19:0/858408856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d794dab60 con 0x561d793474a0 2024-05-09T02:17:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.582+0000 7f4b4bfff700 1 --2- 172.21.15.19:0/858408856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 0x561d794ded30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4b44005c30 tx=0x7f4b44009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83a64632242fb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.582+0000 7f4b2affd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:17:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.582+0000 7f4b2affd700 1 -- 172.21.15.19:0/858408856 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44019040 con 0x561d793474a0 2024-05-09T02:17:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.582+0000 7f4b2affd700 1 -- 172.21.15.19:0/858408856 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f4b44007bc0 con 0x561d793474a0 2024-05-09T02:17:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.582+0000 7f4b605a4b80 1 -- 172.21.15.19:0/858408856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b44003eb0 con 0x561d793474a0 2024-05-09T02:17:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.582+0000 7f4b2affd700 1 -- 172.21.15.19:0/858408856 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4b44007de0 con 0x561d793474a0 2024-05-09T02:17:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.582+0000 7f4b605a4b80 10 client.16071 fetch_fsmap learned FSMap version 43 2024-05-09T02:17:39.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.582+0000 7f4b605a4b80 1 -- 172.21.15.19:0/858408856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561d794c8ab0 con 0x561d793474a0 2024-05-09T02:17:39.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.583+0000 7f4b2affd700 1 -- 172.21.15.19:0/858408856 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b4401a040 con 0x561d793474a0 2024-05-09T02:17:39.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.583+0000 7f4b2affd700 1 -- 172.21.15.19:0/858408856 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 43) ==== 8366+0+0 (secure 0 0 0) 0x7f4b44004ca0 con 0x561d793474a0 2024-05-09T02:17:39.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.583+0000 7f4b605a4b80 10 client.16071 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T02:17:39.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.583+0000 7f4b605a4b80 10 client.16071 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:17:39.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.583+0000 7f4b605a4b80 20 client.16071 populate_metadata read hostname 'smithi019' 2024-05-09T02:17:39.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.583+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/858408856 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x561d794f7970 0x561d794cae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:39.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.583+0000 7f4b605a4b80 4 client.16071 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:17:39.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.583+0000 7f4b605a4b80 1 -- 172.21.15.19:0/858408856 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4b44003eb0 con 0x561d794f7970 2024-05-09T02:17:39.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.583+0000 7f4b4affd700 1 --2- 172.21.15.19:0/858408856 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x561d794f7970 0x561d794cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.583+0000 7f4b2affd700 10 client.16071 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:17:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.584+0000 7f4b4affd700 1 --2- 172.21.15.19:0/858408856 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x561d794f7970 0x561d794cae90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1bbe8aea862a8565 server_cookie=d2108535acf449ed in_seq=0 out_seq=0 2024-05-09T02:17:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.584+0000 7f4b2affd700 1 -- 172.21.15.19:0/858408856 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4b44003eb0 con 0x561d794f7970 2024-05-09T02:17:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.584+0000 7f4b2affd700 10 client.16071 handle_command_reply: tid=0 2024-05-09T02:17:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.584+0000 7f4b605a4b80 1 client.16071 shutdown 2024-05-09T02:17:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.585+0000 7f4b605a4b80 1 -- 172.21.15.19:0/858408856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 msgr2=0x561d794ded30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.585+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/858408856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 0x561d794ded30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4b44005c30 tx=0x7f4b44009ac0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.585+0000 7f4b605a4b80 1 -- 172.21.15.19:0/858408856 shutdown_connections 2024-05-09T02:17:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.585+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/858408856 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x561d794f7970 0x561d794cae90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.585+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/858408856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d794c4c80 0x561d794da620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.585+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/858408856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561d793474a0 0x561d794ded30 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.585+0000 7f4b605a4b80 1 --2- 172.21.15.19:0/858408856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d79431160 0x561d794da0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.585+0000 7f4b605a4b80 1 -- 172.21.15.19:0/858408856 >> 172.21.15.19:0/858408856 conn(0x561d793a76a0 msgr2=0x561d794d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.585+0000 7f4b2affd700 0 client.16071 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:17:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.585+0000 7f4b605a4b80 1 -- 172.21.15.19:0/858408856 shutdown_connections 2024-05-09T02:17:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.585+0000 7f4b605a4b80 1 -- 172.21.15.19:0/858408856 wait complete. 2024-05-09T02:17:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.585+0000 7f4b605a4b80 20 client.16071 trim_cache size 0 max 16384 2024-05-09T02:17:39.590 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:17:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.588+0000 7f4b51b8e700 1 -- 172.21.15.19:0/1368356988 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f4b340b2a70 msgr2=0x7f4b340b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.588+0000 7f4b51b8e700 1 --2- 172.21.15.19:0/1368356988 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f4b340b2a70 0x7f4b340b4f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x561d794c0c40 tx=0x7f4b3c009790 comp rx=0 tx=0).stop 2024-05-09T02:17:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.588+0000 7f4b51b8e700 1 -- 172.21.15.19:0/1368356988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b4c1034c0 msgr2=0x7f4b4c19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.588+0000 7f4b51b8e700 1 --2- 172.21.15.19:0/1368356988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b4c1034c0 0x7f4b4c19adf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00e9a0 tx=0x7f4b3c00ecb0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.588+0000 7f4b51b8e700 1 -- 172.21.15.19:0/1368356988 shutdown_connections 2024-05-09T02:17:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.588+0000 7f4b51b8e700 1 --2- 172.21.15.19:0/1368356988 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f4b340b2a70 0x7f4b340b4f80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.588+0000 7f4b51b8e700 1 --2- 172.21.15.19:0/1368356988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b4c108740 0x7f4b4c19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.588+0000 7f4b51b8e700 1 --2- 172.21.15.19:0/1368356988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b4c1034c0 0x7f4b4c19adf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.588+0000 7f4b51b8e700 1 --2- 172.21.15.19:0/1368356988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b4c107df0 0x7f4b4c19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.588+0000 7f4b51b8e700 1 -- 172.21.15.19:0/1368356988 >> 172.21.15.19:0/1368356988 conn(0x7f4b4c0fedc0 msgr2=0x7f4b4c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:39.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.589+0000 7f4b51b8e700 1 -- 172.21.15.19:0/1368356988 shutdown_connections 2024-05-09T02:17:39.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:39.589+0000 7f4b51b8e700 1 -- 172.21.15.19:0/1368356988 wait complete. 2024-05-09T02:17:40.433 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T02:17:40.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:40 smithi059 ceph-mon[34781]: pgmap v194: 97 pgs: 97 active+clean; 4.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.2 MiB/s wr, 846 op/s 2024-05-09T02:17:40.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:40 smithi196 ceph-mon[34718]: pgmap v194: 97 pgs: 97 active+clean; 4.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.2 MiB/s wr, 846 op/s 2024-05-09T02:17:40.815 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:17:40.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:40 smithi019 ceph-mon[30151]: pgmap v194: 97 pgs: 97 active+clean; 4.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.2 MiB/s wr, 846 op/s 2024-05-09T02:17:42.520 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:42 smithi019 ceph-mon[30151]: pgmap v195: 97 pgs: 97 active+clean; 4.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.7 MiB/s wr, 575 op/s 2024-05-09T02:17:42.520 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:42 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 360.0 s, 2250.4 tps, lat 44.375 ms stddev 50.423 2024-05-09T02:17:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.681+0000 7f2130e3a700 1 -- 172.21.15.19:0/1256491314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c108740 msgr2=0x7f212c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.681+0000 7f2130e3a700 1 --2- 172.21.15.19:0/1256491314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c108740 0x7f212c1100a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2124009f90 tx=0x7f212400b330 comp rx=0 tx=0).stop 2024-05-09T02:17:42.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.681+0000 7f2130e3a700 1 -- 172.21.15.19:0/1256491314 shutdown_connections 2024-05-09T02:17:42.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.681+0000 7f2130e3a700 1 --2- 172.21.15.19:0/1256491314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c108740 0x7f212c1100a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.681+0000 7f2130e3a700 1 --2- 172.21.15.19:0/1256491314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f212c1034c0 0x7f212c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.681+0000 7f2130e3a700 1 --2- 172.21.15.19:0/1256491314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f212c107df0 0x7f212c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.681+0000 7f2130e3a700 1 -- 172.21.15.19:0/1256491314 >> 172.21.15.19:0/1256491314 conn(0x7f212c0fedc0 msgr2=0x7f212c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:42.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.682+0000 7f2130e3a700 1 -- 172.21.15.19:0/1256491314 shutdown_connections 2024-05-09T02:17:42.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.682+0000 7f2130e3a700 1 -- 172.21.15.19:0/1256491314 wait complete. 2024-05-09T02:17:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.683+0000 7f2130e3a700 1 Processor -- start 2024-05-09T02:17:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.683+0000 7f2130e3a700 1 -- start start 2024-05-09T02:17:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.683+0000 7f2130e3a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c1034c0 0x7f212c1a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.683+0000 7f2130e3a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f212c107df0 0x7f212c1a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.683+0000 7f2130e3a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f212c108740 0x7f212c1a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.683+0000 7f2130e3a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f212c112320 con 0x7f212c107df0 2024-05-09T02:17:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.683+0000 7f2130e3a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f212c1121a0 con 0x7f212c1034c0 2024-05-09T02:17:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.683+0000 7f2130e3a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f212c1124a0 con 0x7f212c108740 2024-05-09T02:17:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.684+0000 7f212a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c1034c0 0x7f212c1a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.684+0000 7f212ad9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f212c108740 0x7f212c1a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.684+0000 7f212a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c1034c0 0x7f212c1a1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:45312/0 (socket says 172.21.15.19:45312) 2024-05-09T02:17:42.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.684+0000 7f212a59c700 1 -- 172.21.15.19:0/3471619344 learned_addr learned my addr 172.21.15.19:0/3471619344 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:42.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.684+0000 7f2123fff700 1 --2- 172.21.15.19:0/3471619344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f212c107df0 0x7f212c1a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.684+0000 7f212a59c700 1 -- 172.21.15.19:0/3471619344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f212c108740 msgr2=0x7f212c1a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:42.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.684+0000 7f212a59c700 1 --2- 172.21.15.19:0/3471619344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f212c108740 0x7f212c1a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.684+0000 7f212a59c700 1 -- 172.21.15.19:0/3471619344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f212c107df0 msgr2=0x7f212c1a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:42.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.684+0000 7f212a59c700 1 --2- 172.21.15.19:0/3471619344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f212c107df0 0x7f212c1a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.684+0000 7f212a59c700 1 -- 172.21.15.19:0/3471619344 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212c1a62d0 con 0x7f212c1034c0 2024-05-09T02:17:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.684+0000 7f212a59c700 1 --2- 172.21.15.19:0/3471619344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c1034c0 0x7f212c1a1400 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f211c00e9c0 tx=0x7f211c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.685+0000 7f21237fe700 1 -- 172.21.15.19:0/3471619344 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c004a30 con 0x7f212c1034c0 2024-05-09T02:17:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.685+0000 7f21237fe700 1 -- 172.21.15.19:0/3471619344 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f211c00cc70 con 0x7f212c1034c0 2024-05-09T02:17:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.685+0000 7f21237fe700 1 -- 172.21.15.19:0/3471619344 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c005490 con 0x7f212c1034c0 2024-05-09T02:17:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.686+0000 7f2130e3a700 1 -- 172.21.15.19:0/3471619344 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212c1a65c0 con 0x7f212c1034c0 2024-05-09T02:17:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.686+0000 7f2130e3a700 1 -- 172.21.15.19:0/3471619344 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f212c1adee0 con 0x7f212c1034c0 2024-05-09T02:17:42.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.690+0000 7f21237fe700 1 -- 172.21.15.19:0/3471619344 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f211c004bd0 con 0x7f212c1034c0 2024-05-09T02:17:42.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.691+0000 7f213f850b80 1 Processor -- start 2024-05-09T02:17:42.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.691+0000 7f213f850b80 1 -- start start 2024-05-09T02:17:42.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.691+0000 7f21237fe700 1 --2- 172.21.15.19:0/3471619344 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f21180b2c00 0x7f21180b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.691+0000 7f213f850b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2cd6bd00 0x563a2ce74240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f213f850b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2ccf14a0 0x563a2ce74780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f213f850b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ce6e400 0x563a2ce6e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f212ad9d700 1 --2- 172.21.15.19:0/3471619344 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f21180b2c00 0x7f21180b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f213f850b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563a2cd910a0 con 0x563a2ccf14a0 2024-05-09T02:17:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f213f850b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563a2cd369a0 con 0x563a2ce6e400 2024-05-09T02:17:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f21237fe700 1 -- 172.21.15.19:0/3471619344 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f211c0e26f0 con 0x7f212c1034c0 2024-05-09T02:17:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f213f850b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563a2cd8d060 con 0x563a2cd6bd00 2024-05-09T02:17:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f212ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ce6e400 0x563a2ce6e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f2123fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2ccf14a0 0x563a2ce74780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f212a59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2cd6bd00 0x563a2ce74240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f212a59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2cd6bd00 0x563a2ce74240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:39306/0 (socket says 172.21.15.19:39306) 2024-05-09T02:17:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f212a59c700 1 -- 172.21.15.19:0/554527818 learned_addr learned my addr 172.21.15.19:0/554527818 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f2123fff700 1 --2- 172.21.15.19:0/554527818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2ccf14a0 0x563a2ce74780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53730/0 (socket says 172.21.15.19:53730) 2024-05-09T02:17:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f212ad9d700 1 --2- 172.21.15.19:0/3471619344 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f21180b2c00 0x7f21180b5110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2124009da0 tx=0x7f2124008fc0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f212a59c700 1 -- 172.21.15.19:0/554527818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ce6e400 msgr2=0x563a2ce6e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f212a59c700 1 --2- 172.21.15.19:0/554527818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ce6e400 0x563a2ce6e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f212a59c700 1 -- 172.21.15.19:0/554527818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2ccf14a0 msgr2=0x563a2ce74780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f212a59c700 1 --2- 172.21.15.19:0/554527818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2ccf14a0 0x563a2ce74780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.692+0000 7f212a59c700 1 -- 172.21.15.19:0/554527818 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a2cd27500 con 0x563a2cd6bd00 2024-05-09T02:17:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.693+0000 7f212a59c700 1 --2- 172.21.15.19:0/554527818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2cd6bd00 0x563a2ce74240 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f211c00bfd0 tx=0x7f211c0a92c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0978649f5871fa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.693+0000 7f2120ff9700 1 -- 172.21.15.19:0/554527818 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c04aed0 con 0x563a2cd6bd00 2024-05-09T02:17:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.693+0000 7f2120ff9700 1 -- 172.21.15.19:0/554527818 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f211c0b2120 con 0x563a2cd6bd00 2024-05-09T02:17:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.693+0000 7f2120ff9700 1 -- 172.21.15.19:0/554527818 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c0a5100 con 0x563a2cd6bd00 2024-05-09T02:17:42.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.694+0000 7f213f850b80 1 -- 172.21.15.19:0/554527818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2cd6bd00 msgr2=0x563a2ce74240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:42.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.694+0000 7f213f850b80 1 --2- 172.21.15.19:0/554527818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2cd6bd00 0x563a2ce74240 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f211c00bfd0 tx=0x7f211c0a92c0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.694+0000 7f213f850b80 1 -- 172.21.15.19:0/554527818 shutdown_connections 2024-05-09T02:17:42.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.694+0000 7f213f850b80 1 --2- 172.21.15.19:0/554527818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ce6e400 0x563a2ce6e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.694+0000 7f213f850b80 1 --2- 172.21.15.19:0/554527818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2cd6bd00 0x563a2ce74240 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.694+0000 7f213f850b80 1 --2- 172.21.15.19:0/554527818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2ccf14a0 0x563a2ce74780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.694+0000 7f213f850b80 1 -- 172.21.15.19:0/554527818 >> 172.21.15.19:0/554527818 conn(0x563a2ccf4f70 msgr2=0x563a2ce69220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:42.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.694+0000 7f213f850b80 1 -- 172.21.15.19:0/554527818 shutdown_connections 2024-05-09T02:17:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.694+0000 7f213f850b80 1 -- 172.21.15.19:0/554527818 wait complete. 2024-05-09T02:17:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.695+0000 7f213f850b80 1 -- start start 2024-05-09T02:17:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.695+0000 7f213f850b80 1 Processor -- start 2024-05-09T02:17:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.696+0000 7f213f850b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 0x563a2ce77f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.696+0000 7f213f850b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cd6bd00 0x563a2ce784b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.696+0000 7f213f850b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2ce6e400 0x563a2ce789f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.696+0000 7f213f850b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563a2ce6f160 con 0x563a2cd6bd00 2024-05-09T02:17:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.696+0000 7f213f850b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563a2cd2bb20 con 0x563a2ccf14a0 2024-05-09T02:17:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.696+0000 7f213f850b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563a2ce7b020 con 0x563a2ce6e400 2024-05-09T02:17:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.696+0000 7f213f850b80 10 client.0 _ll_register_callbacks cb 0x563a2cc026c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:17:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.696+0000 7f212a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 0x563a2ce77f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.696+0000 7f2123fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cd6bd00 0x563a2ce784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.696+0000 7f212a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 0x563a2ce77f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:45326/0 (socket says 172.21.15.19:45326) 2024-05-09T02:17:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.696+0000 7f212a59c700 1 -- 172.21.15.19:0/2869866642 learned_addr learned my addr 172.21.15.19:0/2869866642 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.696+0000 7f212ad9d700 1 --2- 172.21.15.19:0/2869866642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2ce6e400 0x563a2ce789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.697+0000 7f212a59c700 1 -- 172.21.15.19:0/2869866642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2ce6e400 msgr2=0x563a2ce789f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.697+0000 7f212a59c700 1 --2- 172.21.15.19:0/2869866642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2ce6e400 0x563a2ce789f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.697+0000 7f212a59c700 1 -- 172.21.15.19:0/2869866642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cd6bd00 msgr2=0x563a2ce784b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.697+0000 7f212a59c700 1 --2- 172.21.15.19:0/2869866642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cd6bd00 0x563a2ce784b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.697+0000 7f212a59c700 1 -- 172.21.15.19:0/2869866642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a2cc0c490 con 0x563a2ccf14a0 2024-05-09T02:17:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.697+0000 7f212ad9d700 1 --2- 172.21.15.19:0/2869866642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2ce6e400 0x563a2ce789f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:17:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.697+0000 7f2123fff700 1 --2- 172.21.15.19:0/2869866642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cd6bd00 0x563a2ce784b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:17:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.697+0000 7f212a59c700 1 --2- 172.21.15.19:0/2869866642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 0x563a2ce77f70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f211c0a59f0 tx=0x7f211c0cc360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=368a441084d02f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.697+0000 7f2111ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:17:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.697+0000 7f2111ffb700 1 -- 172.21.15.19:0/2869866642 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c0b6020 con 0x563a2ccf14a0 2024-05-09T02:17:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.697+0000 7f2111ffb700 1 -- 172.21.15.19:0/2869866642 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f211c0b2550 con 0x563a2ccf14a0 2024-05-09T02:17:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.698+0000 7f213f850b80 1 -- 172.21.15.19:0/2869866642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f211c0b2120 con 0x563a2ccf14a0 2024-05-09T02:17:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.698+0000 7f2111ffb700 1 -- 172.21.15.19:0/2869866642 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f211c04aed0 con 0x563a2ccf14a0 2024-05-09T02:17:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.698+0000 7f213f850b80 10 client.25805 fetch_fsmap learned FSMap version 44 2024-05-09T02:17:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.698+0000 7f213f850b80 1 -- 172.21.15.19:0/2869866642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563a2ce85980 con 0x563a2ccf14a0 2024-05-09T02:17:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.698+0000 7f2111ffb700 1 -- 172.21.15.19:0/2869866642 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f211c00ce20 con 0x563a2ccf14a0 2024-05-09T02:17:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.698+0000 7f2111ffb700 1 -- 172.21.15.19:0/2869866642 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f211c0a9bd0 con 0x563a2ccf14a0 2024-05-09T02:17:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.699+0000 7f213f850b80 10 client.25805 fetch_fsmap finished waiting for FSMap version 44 2024-05-09T02:17:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.699+0000 7f213f850b80 10 client.25805 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:17:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.699+0000 7f213f850b80 20 client.25805 populate_metadata read hostname 'smithi019' 2024-05-09T02:17:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.699+0000 7f213f850b80 1 --2- 172.21.15.19:0/2869866642 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563a2ce87e50 0x563a2ce8a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.699+0000 7f213f850b80 4 client.25805 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:17:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.699+0000 7f213f850b80 1 -- 172.21.15.19:0/2869866642 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f211c0b2120 con 0x563a2ce87e50 2024-05-09T02:17:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.699+0000 7f2123fff700 1 --2- 172.21.15.19:0/2869866642 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563a2ce87e50 0x563a2ce8a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.699+0000 7f2111ffb700 10 client.25805 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:17:42.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.700+0000 7f2123fff700 1 --2- 172.21.15.19:0/2869866642 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563a2ce87e50 0x563a2ce8a270 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb6519bfe76a8581 server_cookie=2dd59f469041d031 in_seq=0 out_seq=0 2024-05-09T02:17:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.703+0000 7f2111ffb700 1 -- 172.21.15.19:0/2869866642 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f211c0b2120 con 0x563a2ce87e50 2024-05-09T02:17:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.703+0000 7f2111ffb700 10 client.25805 handle_command_reply: tid=0 2024-05-09T02:17:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.703+0000 7f213f850b80 1 client.25805 shutdown 2024-05-09T02:17:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.703+0000 7f213f850b80 1 -- 172.21.15.19:0/2869866642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 msgr2=0x563a2ce77f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.703+0000 7f213f850b80 1 --2- 172.21.15.19:0/2869866642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 0x563a2ce77f70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f211c0a59f0 tx=0x7f211c0cc360 comp rx=0 tx=0).stop 2024-05-09T02:17:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.704+0000 7f213f850b80 1 -- 172.21.15.19:0/2869866642 shutdown_connections 2024-05-09T02:17:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.704+0000 7f213f850b80 1 --2- 172.21.15.19:0/2869866642 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563a2ce87e50 0x563a2ce8a270 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.704+0000 7f213f850b80 1 --2- 172.21.15.19:0/2869866642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2ce6e400 0x563a2ce789f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.704+0000 7f213f850b80 1 --2- 172.21.15.19:0/2869866642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 0x563a2ce77f70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.704+0000 7f213f850b80 1 --2- 172.21.15.19:0/2869866642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cd6bd00 0x563a2ce784b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.704+0000 7f2111ffb700 0 client.25805 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:17:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.704+0000 7f213f850b80 1 -- 172.21.15.19:0/2869866642 >> 172.21.15.19:0/2869866642 conn(0x563a2ccf4f70 msgr2=0x563a2ce62ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.704+0000 7f213f850b80 1 -- 172.21.15.19:0/2869866642 shutdown_connections 2024-05-09T02:17:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.704+0000 7f213f850b80 1 -- 172.21.15.19:0/2869866642 wait complete. 2024-05-09T02:17:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.704+0000 7f213f850b80 20 client.25805 trim_cache size 0 max 16384 2024-05-09T02:17:42.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:42 smithi059 ceph-mon[34781]: pgmap v195: 97 pgs: 97 active+clean; 4.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.7 MiB/s wr, 575 op/s 2024-05-09T02:17:42.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:42 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.726+0000 7f213f850b80 1 Processor -- start 2024-05-09T02:17:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.726+0000 7f213f850b80 1 -- start start 2024-05-09T02:17:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.727+0000 7f213f850b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 0x563a2ce83fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.727+0000 7f213f850b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2cd6bd00 0x563a2ce63700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.727+0000 7f213f850b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2ce6e400 0x563a2ce8ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.727+0000 7f213f850b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f212c1b54c0 con 0x563a2ce6e400 2024-05-09T02:17:42.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.727+0000 7f213f850b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f212c1b7060 con 0x563a2ccf14a0 2024-05-09T02:17:42.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.727+0000 7f213f850b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f212c1b5b90 con 0x563a2cd6bd00 2024-05-09T02:17:42.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.727+0000 7f2123fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2cd6bd00 0x563a2ce63700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.727+0000 7f2123fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2cd6bd00 0x563a2ce63700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:39332/0 (socket says 172.21.15.19:39332) 2024-05-09T02:17:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.727+0000 7f2123fff700 1 -- 172.21.15.19:0/1587504189 learned_addr learned my addr 172.21.15.19:0/1587504189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.727+0000 7f212ad9d700 1 --2- 172.21.15.19:0/1587504189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2ce6e400 0x563a2ce8ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.728+0000 7f2123fff700 1 -- 172.21.15.19:0/1587504189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 msgr2=0x563a2ce83fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:17:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.728+0000 7f2123fff700 1 --2- 172.21.15.19:0/1587504189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 0x563a2ce83fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.728+0000 7f2123fff700 1 -- 172.21.15.19:0/1587504189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2ce6e400 msgr2=0x563a2ce8ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.728+0000 7f2123fff700 1 --2- 172.21.15.19:0/1587504189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2ce6e400 0x563a2ce8ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.728+0000 7f2123fff700 1 -- 172.21.15.19:0/1587504189 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a2ce77d40 con 0x563a2cd6bd00 2024-05-09T02:17:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.728+0000 7f212ad9d700 1 --2- 172.21.15.19:0/1587504189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2ce6e400 0x563a2ce8ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:17:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.728+0000 7f2123fff700 1 --2- 172.21.15.19:0/1587504189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2cd6bd00 0x563a2ce63700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2114005950 tx=0x7f2114004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a9ae55a94255141 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.729+0000 7f2113fff700 1 -- 172.21.15.19:0/1587504189 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114003bc0 con 0x563a2cd6bd00 2024-05-09T02:17:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.729+0000 7f2113fff700 1 -- 172.21.15.19:0/1587504189 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2114003d60 con 0x563a2cd6bd00 2024-05-09T02:17:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.729+0000 7f2113fff700 1 -- 172.21.15.19:0/1587504189 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211401d450 con 0x563a2cd6bd00 2024-05-09T02:17:42.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.730+0000 7f213f850b80 1 -- 172.21.15.19:0/1587504189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2cd6bd00 msgr2=0x563a2ce63700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:42.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.730+0000 7f213f850b80 1 --2- 172.21.15.19:0/1587504189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2cd6bd00 0x563a2ce63700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2114005950 tx=0x7f2114004750 comp rx=0 tx=0).stop 2024-05-09T02:17:42.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.730+0000 7f213f850b80 1 -- 172.21.15.19:0/1587504189 shutdown_connections 2024-05-09T02:17:42.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.730+0000 7f213f850b80 1 --2- 172.21.15.19:0/1587504189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2ce6e400 0x563a2ce8ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.730+0000 7f213f850b80 1 --2- 172.21.15.19:0/1587504189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 0x563a2ce83fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.730+0000 7f213f850b80 1 --2- 172.21.15.19:0/1587504189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2cd6bd00 0x563a2ce63700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.730+0000 7f213f850b80 1 -- 172.21.15.19:0/1587504189 >> 172.21.15.19:0/1587504189 conn(0x563a2ccf4f70 msgr2=0x563a2ce62ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.730+0000 7f213f850b80 1 -- 172.21.15.19:0/1587504189 shutdown_connections 2024-05-09T02:17:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.730+0000 7f213f850b80 1 -- 172.21.15.19:0/1587504189 wait complete. 2024-05-09T02:17:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.730+0000 7f213f850b80 1 -- start start 2024-05-09T02:17:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.731+0000 7f213f850b80 1 Processor -- start 2024-05-09T02:17:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.731+0000 7f213f850b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 0x563a2ce89e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.731+0000 7f213f850b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cd6bd00 0x563a2ce8a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.731+0000 7f213f850b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2ce6e400 0x563a2ce8a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.731+0000 7f213f850b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f212c1b6990 con 0x563a2cd6bd00 2024-05-09T02:17:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.731+0000 7f213f850b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f212c1b6260 con 0x563a2ccf14a0 2024-05-09T02:17:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.731+0000 7f213f850b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f212c1b3460 con 0x563a2ce6e400 2024-05-09T02:17:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.731+0000 7f213f850b80 10 client.0 _ll_register_callbacks cb 0x563a2cc026c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:17:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.731+0000 7f2123fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cd6bd00 0x563a2ce8a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.731+0000 7f212a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 0x563a2ce89e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.731+0000 7f2123fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cd6bd00 0x563a2ce8a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53754/0 (socket says 172.21.15.19:53754) 2024-05-09T02:17:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.731+0000 7f2123fff700 1 -- 172.21.15.19:0/1940576561 learned_addr learned my addr 172.21.15.19:0/1940576561 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.731+0000 7f212ad9d700 1 --2- 172.21.15.19:0/1940576561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2ce6e400 0x563a2ce8a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.732+0000 7f2123fff700 1 -- 172.21.15.19:0/1940576561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2ce6e400 msgr2=0x563a2ce8a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.732+0000 7f2123fff700 1 --2- 172.21.15.19:0/1940576561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2ce6e400 0x563a2ce8a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.732+0000 7f2123fff700 1 -- 172.21.15.19:0/1940576561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 msgr2=0x563a2ce89e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.732+0000 7f2123fff700 1 --2- 172.21.15.19:0/1940576561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 0x563a2ce89e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.732+0000 7f2123fff700 1 -- 172.21.15.19:0/1940576561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a2cef0390 con 0x563a2cd6bd00 2024-05-09T02:17:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.732+0000 7f212ad9d700 1 --2- 172.21.15.19:0/1940576561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2ce6e400 0x563a2ce8a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:17:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.732+0000 7f212a59c700 1 --2- 172.21.15.19:0/1940576561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 0x563a2ce89e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:17:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.732+0000 7f2123fff700 1 --2- 172.21.15.19:0/1940576561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cd6bd00 0x563a2ce8a3a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f211400bc70 tx=0x7f211400bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d9fa46c8181752 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.732+0000 7f2111ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:17:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.732+0000 7f2111ffb700 1 -- 172.21.15.19:0/1940576561 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114005400 con 0x563a2cd6bd00 2024-05-09T02:17:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.732+0000 7f213f850b80 1 -- 172.21.15.19:0/1940576561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2114003d60 con 0x563a2cd6bd00 2024-05-09T02:17:42.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.732+0000 7f2111ffb700 1 -- 172.21.15.19:0/1940576561 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f21140055a0 con 0x563a2cd6bd00 2024-05-09T02:17:42.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.732+0000 7f2111ffb700 1 -- 172.21.15.19:0/1940576561 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2114038050 con 0x563a2cd6bd00 2024-05-09T02:17:42.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.734+0000 7f213f850b80 10 client.16089 fetch_fsmap learned FSMap version 44 2024-05-09T02:17:42.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.734+0000 7f213f850b80 1 -- 172.21.15.19:0/1940576561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563a2cef1570 con 0x563a2cd6bd00 2024-05-09T02:17:42.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.734+0000 7f2111ffb700 1 -- 172.21.15.19:0/1940576561 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2114003d60 con 0x563a2cd6bd00 2024-05-09T02:17:42.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.735+0000 7f2111ffb700 1 -- 172.21.15.19:0/1940576561 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f211402a590 con 0x563a2cd6bd00 2024-05-09T02:17:42.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.735+0000 7f213f850b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 44 2024-05-09T02:17:42.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.735+0000 7f213f850b80 10 client.16089 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:17:42.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.735+0000 7f213f850b80 20 client.16089 populate_metadata read hostname 'smithi019' 2024-05-09T02:17:42.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.735+0000 7f213f850b80 1 --2- 172.21.15.19:0/1940576561 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563a2cef1b20 0x563a2cef3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:42.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.735+0000 7f213f850b80 4 client.16089 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:17:42.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.735+0000 7f213f850b80 1 -- 172.21.15.19:0/1940576561 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a2cd9ea80 con 0x563a2cef1b20 2024-05-09T02:17:42.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.735+0000 7f212a59c700 1 --2- 172.21.15.19:0/1940576561 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563a2cef1b20 0x563a2cef3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:42.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.735+0000 7f212a59c700 1 --2- 172.21.15.19:0/1940576561 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563a2cef1b20 0x563a2cef3f40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=89e82cd12c0e406a server_cookie=efd68a589cd30f3b in_seq=0 out_seq=0 2024-05-09T02:17:42.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.735+0000 7f2111ffb700 10 client.16089 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:17:42.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.736+0000 7f2111ffb700 1 -- 172.21.15.19:0/1940576561 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a2cd9ea80 con 0x563a2cef1b20 2024-05-09T02:17:42.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.736+0000 7f2111ffb700 10 client.16089 handle_command_reply: tid=0 2024-05-09T02:17:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.736+0000 7f213f850b80 1 client.16089 shutdown 2024-05-09T02:17:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.736+0000 7f213f850b80 1 -- 172.21.15.19:0/1940576561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cd6bd00 msgr2=0x563a2ce8a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.736+0000 7f213f850b80 1 --2- 172.21.15.19:0/1940576561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cd6bd00 0x563a2ce8a3a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f211400bc70 tx=0x7f211400bca0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.736+0000 7f213f850b80 1 -- 172.21.15.19:0/1940576561 shutdown_connections 2024-05-09T02:17:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.736+0000 7f213f850b80 1 --2- 172.21.15.19:0/1940576561 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x563a2cef1b20 0x563a2cef3f40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.736+0000 7f213f850b80 1 --2- 172.21.15.19:0/1940576561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a2ce6e400 0x563a2ce8a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.736+0000 7f213f850b80 1 --2- 172.21.15.19:0/1940576561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a2ccf14a0 0x563a2ce89e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.736+0000 7f213f850b80 1 --2- 172.21.15.19:0/1940576561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cd6bd00 0x563a2ce8a3a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.736+0000 7f213f850b80 1 -- 172.21.15.19:0/1940576561 >> 172.21.15.19:0/1940576561 conn(0x563a2ccf4f70 msgr2=0x563a2ce77f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.736+0000 7f2111ffb700 0 client.16089 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:17:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.737+0000 7f213f850b80 1 -- 172.21.15.19:0/1940576561 shutdown_connections 2024-05-09T02:17:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.737+0000 7f213f850b80 1 -- 172.21.15.19:0/1940576561 wait complete. 2024-05-09T02:17:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.737+0000 7f213f850b80 20 client.16089 trim_cache size 0 max 16384 2024-05-09T02:17:42.743 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:17:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.738+0000 7f2130e3a700 1 -- 172.21.15.19:0/3471619344 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f21180b2c00 msgr2=0x7f21180b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.738+0000 7f2130e3a700 1 --2- 172.21.15.19:0/3471619344 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f21180b2c00 0x7f21180b5110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2124009da0 tx=0x7f2124008fc0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.738+0000 7f2130e3a700 1 -- 172.21.15.19:0/3471619344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c1034c0 msgr2=0x7f212c1a1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.738+0000 7f2130e3a700 1 --2- 172.21.15.19:0/3471619344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c1034c0 0x7f212c1a1400 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f211c00e9c0 tx=0x7f211c00ecd0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.738+0000 7f2130e3a700 1 -- 172.21.15.19:0/3471619344 shutdown_connections 2024-05-09T02:17:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.738+0000 7f2130e3a700 1 --2- 172.21.15.19:0/3471619344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f212c108740 0x7f212c1a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.738+0000 7f2130e3a700 1 --2- 172.21.15.19:0/3471619344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c1034c0 0x7f212c1a1400 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.738+0000 7f2130e3a700 1 --2- 172.21.15.19:0/3471619344 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f21180b2c00 0x7f21180b5110 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.738+0000 7f2130e3a700 1 --2- 172.21.15.19:0/3471619344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f212c107df0 0x7f212c1a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.738+0000 7f2130e3a700 1 -- 172.21.15.19:0/3471619344 >> 172.21.15.19:0/3471619344 conn(0x7f212c0fedc0 msgr2=0x7f212c109700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.738+0000 7f2130e3a700 1 -- 172.21.15.19:0/3471619344 shutdown_connections 2024-05-09T02:17:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:42.738+0000 7f2130e3a700 1 -- 172.21.15.19:0/3471619344 wait complete. 2024-05-09T02:17:42.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:42 smithi196 ceph-mon[34718]: pgmap v195: 97 pgs: 97 active+clean; 4.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.7 MiB/s wr, 575 op/s 2024-05-09T02:17:42.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:42 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:43.506 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T02:17:43.884 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:17:44.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:44 smithi019 ceph-mon[30151]: pgmap v196: 97 pgs: 97 active+clean; 4.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.9 MiB/s wr, 592 op/s 2024-05-09T02:17:44.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:44 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:44.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:44 smithi059 ceph-mon[34781]: pgmap v196: 97 pgs: 97 active+clean; 4.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.9 MiB/s wr, 592 op/s 2024-05-09T02:17:44.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:44 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:44.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:44 smithi196 ceph-mon[34718]: pgmap v196: 97 pgs: 97 active+clean; 4.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.9 MiB/s wr, 592 op/s 2024-05-09T02:17:44.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:44 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:46.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.236+0000 7f7e9270a700 1 -- 172.21.15.19:0/3493615062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8c107bf0 msgr2=0x7f7e8c108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:46.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.236+0000 7f7e9270a700 1 --2- 172.21.15.19:0/3493615062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8c107bf0 0x7f7e8c108000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0099c0 tx=0x7f7e7c009cd0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.236+0000 7f7e9270a700 1 -- 172.21.15.19:0/3493615062 shutdown_connections 2024-05-09T02:17:46.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.236+0000 7f7e9270a700 1 --2- 172.21.15.19:0/3493615062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e8c108540 0x7f7e8c10fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.236+0000 7f7e9270a700 1 --2- 172.21.15.19:0/3493615062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e8c1032c0 0x7f7e8c1036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.236+0000 7f7e9270a700 1 --2- 172.21.15.19:0/3493615062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8c107bf0 0x7f7e8c108000 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.236+0000 7f7e9270a700 1 -- 172.21.15.19:0/3493615062 >> 172.21.15.19:0/3493615062 conn(0x7f7e8c078bf0 msgr2=0x7f7e8c079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:46.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.237+0000 7f7e9270a700 1 -- 172.21.15.19:0/3493615062 shutdown_connections 2024-05-09T02:17:46.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.237+0000 7f7e9270a700 1 -- 172.21.15.19:0/3493615062 wait complete. 2024-05-09T02:17:46.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.238+0000 7f7e9270a700 1 Processor -- start 2024-05-09T02:17:46.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.238+0000 7f7e9270a700 1 -- start start 2024-05-09T02:17:46.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.239+0000 7f7e9270a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8c1032c0 0x7f7e8c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.239+0000 7f7e9270a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e8c107bf0 0x7f7e8c1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.239+0000 7f7e9270a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e8c108540 0x7f7e8c1a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.239+0000 7f7e9270a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c113150 con 0x7f7e8c107bf0 2024-05-09T02:17:46.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.239+0000 7f7e9270a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c112fd0 con 0x7f7e8c108540 2024-05-09T02:17:46.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.239+0000 7f7e9270a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1132d0 con 0x7f7e8c1032c0 2024-05-09T02:17:46.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.239+0000 7f7e91708700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8c1032c0 0x7f7e8c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.239+0000 7f7e91708700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8c1032c0 0x7f7e8c1a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:39356/0 (socket says 172.21.15.19:39356) 2024-05-09T02:17:46.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.239+0000 7f7e91708700 1 -- 172.21.15.19:0/650354003 learned_addr learned my addr 172.21.15.19:0/650354003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:46.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.240+0000 7f7e91708700 1 -- 172.21.15.19:0/650354003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e8c108540 msgr2=0x7f7e8c1a5ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T02:17:46.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.240+0000 7f7e91708700 1 --2- 172.21.15.19:0/650354003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e8c108540 0x7f7e8c1a5ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.240+0000 7f7e91708700 1 -- 172.21.15.19:0/650354003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e8c107bf0 msgr2=0x7f7e8c1a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:46.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.240+0000 7f7e91708700 1 --2- 172.21.15.19:0/650354003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e8c107bf0 0x7f7e8c1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.240+0000 7f7e91708700 1 -- 172.21.15.19:0/650354003 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c1a6180 con 0x7f7e8c1032c0 2024-05-09T02:17:46.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.240+0000 7f7e91708700 1 --2- 172.21.15.19:0/650354003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8c1032c0 0x7f7e8c1a13c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800d950 tx=0x7f7e8800dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:46.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.240+0000 7f7e827fc700 1 -- 172.21.15.19:0/650354003 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88016070 con 0x7f7e8c1032c0 2024-05-09T02:17:46.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.240+0000 7f7e827fc700 1 -- 172.21.15.19:0/650354003 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e88010460 con 0x7f7e8c1032c0 2024-05-09T02:17:46.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.240+0000 7f7e9270a700 1 -- 172.21.15.19:0/650354003 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8c1a6470 con 0x7f7e8c1032c0 2024-05-09T02:17:46.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.240+0000 7f7e827fc700 1 -- 172.21.15.19:0/650354003 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8800f620 con 0x7f7e8c1032c0 2024-05-09T02:17:46.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.241+0000 7f7e9270a700 1 -- 172.21.15.19:0/650354003 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e8c1a68f0 con 0x7f7e8c1032c0 2024-05-09T02:17:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.244+0000 7f7e827fc700 1 -- 172.21.15.19:0/650354003 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7e88019070 con 0x7f7e8c1032c0 2024-05-09T02:17:46.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.245+0000 7f7e827fc700 1 --2- 172.21.15.19:0/650354003 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7e780b2b40 0x7f7e780b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.245+0000 7f7e827fc700 1 -- 172.21.15.19:0/650354003 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f7e880e2220 con 0x7f7e8c1032c0 2024-05-09T02:17:46.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.245+0000 7f7e90f07700 1 --2- 172.21.15.19:0/650354003 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7e780b2b40 0x7f7e780b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.245+0000 7f7ea2382b80 1 Processor -- start 2024-05-09T02:17:46.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.245+0000 7f7e90f07700 1 --2- 172.21.15.19:0/650354003 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7e780b2b40 0x7f7e780b5050 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0098e0 tx=0x7f7e7c000bc0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:46.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.246+0000 7f7ea2382b80 1 -- start start 2024-05-09T02:17:46.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.246+0000 7f7ea2382b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8fad00 0x5598cd9fda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.246+0000 7f7ea2382b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cd8804a0 0x5598cd9fdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.246+0000 7f7ea2382b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cda07540 0x5598cda09960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.246+0000 7f7ea2382b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5598cd9200a0 con 0x5598cda07540 2024-05-09T02:17:46.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.246+0000 7f7ea2382b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5598cd8c59a0 con 0x5598cd8fad00 2024-05-09T02:17:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.246+0000 7f7ea2382b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5598cd91c060 con 0x5598cd8804a0 2024-05-09T02:17:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.246+0000 7f7e91f09700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cda07540 0x5598cda09960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.246+0000 7f7e91f09700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cda07540 0x5598cda09960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53768/0 (socket says 172.21.15.19:53768) 2024-05-09T02:17:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.246+0000 7f7e91f09700 1 -- 172.21.15.19:0/2832267313 learned_addr learned my addr 172.21.15.19:0/2832267313 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.246+0000 7f7e90f07700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cd8804a0 0x5598cd9fdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.246+0000 7f7e91708700 1 --2- 172.21.15.19:0/2832267313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8fad00 0x5598cd9fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.247+0000 7f7e91f09700 1 -- 172.21.15.19:0/2832267313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cd8804a0 msgr2=0x5598cd9fdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.247+0000 7f7e91f09700 1 --2- 172.21.15.19:0/2832267313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cd8804a0 0x5598cd9fdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.247+0000 7f7e91f09700 1 -- 172.21.15.19:0/2832267313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8fad00 msgr2=0x5598cd9fda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.247+0000 7f7e91f09700 1 --2- 172.21.15.19:0/2832267313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8fad00 0x5598cd9fda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.247+0000 7f7e91f09700 1 -- 172.21.15.19:0/2832267313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598cd8b6500 con 0x5598cda07540 2024-05-09T02:17:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.247+0000 7f7e91f09700 1 --2- 172.21.15.19:0/2832267313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cda07540 0x5598cda09960 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400e380 tx=0x7f7e8400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6a0f1df26578ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:46.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.247+0000 7f7e777fe700 1 -- 172.21.15.19:0/2832267313 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8400ee30 con 0x5598cda07540 2024-05-09T02:17:46.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.247+0000 7f7e777fe700 1 -- 172.21.15.19:0/2832267313 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e84004bb0 con 0x5598cda07540 2024-05-09T02:17:46.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.247+0000 7f7e777fe700 1 -- 172.21.15.19:0/2832267313 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84004280 con 0x5598cda07540 2024-05-09T02:17:46.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.248+0000 7f7ea2382b80 1 -- 172.21.15.19:0/2832267313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cda07540 msgr2=0x5598cda09960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:46.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.248+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/2832267313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cda07540 0x5598cda09960 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400e380 tx=0x7f7e8400b040 comp rx=0 tx=0).stop 2024-05-09T02:17:46.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.249+0000 7f7ea2382b80 1 -- 172.21.15.19:0/2832267313 shutdown_connections 2024-05-09T02:17:46.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.249+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/2832267313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cda07540 0x5598cda09960 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.249+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/2832267313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8fad00 0x5598cd9fda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.249+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/2832267313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cd8804a0 0x5598cd9fdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.249+0000 7f7ea2382b80 1 -- 172.21.15.19:0/2832267313 >> 172.21.15.19:0/2832267313 conn(0x5598cd883f70 msgr2=0x5598cd9f8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:46.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.250+0000 7f7ea2382b80 1 -- 172.21.15.19:0/2832267313 shutdown_connections 2024-05-09T02:17:46.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.250+0000 7f7ea2382b80 1 -- 172.21.15.19:0/2832267313 wait complete. 2024-05-09T02:17:46.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.250+0000 7f7ea2382b80 1 -- start start 2024-05-09T02:17:46.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.251+0000 7f7ea2382b80 1 Processor -- start 2024-05-09T02:17:46.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.251+0000 7f7ea2382b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda04550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.251+0000 7f7ea2382b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cd8fad00 0x5598cda04a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.251+0000 7f7ea2382b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cda07540 0x5598cda04fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.251+0000 7f7ea2382b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5598cda09f00 con 0x5598cd8fad00 2024-05-09T02:17:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7ea2382b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5598cd8bab20 con 0x5598cd8804a0 2024-05-09T02:17:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7ea2382b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5598cda0a080 con 0x5598cda07540 2024-05-09T02:17:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7ea2382b80 10 client.0 _ll_register_callbacks cb 0x5598cd7916c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:17:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7e91708700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda04550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7e91f09700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cda07540 0x5598cda04fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7e91f09700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cda07540 0x5598cda04fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:39368/0 (socket says 172.21.15.19:39368) 2024-05-09T02:17:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7e91708700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda04550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:45380/0 (socket says 172.21.15.19:45380) 2024-05-09T02:17:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7e91f09700 1 -- 172.21.15.19:0/3252497468 learned_addr learned my addr 172.21.15.19:0/3252497468 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7e90f07700 1 --2- 172.21.15.19:0/3252497468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cd8fad00 0x5598cda04a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7e91f09700 1 -- 172.21.15.19:0/3252497468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 msgr2=0x5598cda04550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7e91f09700 1 --2- 172.21.15.19:0/3252497468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda04550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7e91f09700 1 -- 172.21.15.19:0/3252497468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cd8fad00 msgr2=0x5598cda04a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7e91f09700 1 --2- 172.21.15.19:0/3252497468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cd8fad00 0x5598cda04a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7e91f09700 1 -- 172.21.15.19:0/3252497468 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598cd79b490 con 0x5598cda07540 2024-05-09T02:17:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7e91f09700 1 --2- 172.21.15.19:0/3252497468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cda07540 0x5598cda04fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7e84003da0 tx=0x7f7e84003ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d787c43dc6ade17e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.252+0000 7f7e74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:17:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.253+0000 7f7e74ff9700 1 -- 172.21.15.19:0/3252497468 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84021020 con 0x5598cda07540 2024-05-09T02:17:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.253+0000 7f7e74ff9700 1 -- 172.21.15.19:0/3252497468 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f7e840092e0 con 0x5598cda07540 2024-05-09T02:17:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.253+0000 7f7ea2382b80 1 -- 172.21.15.19:0/3252497468 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e84004bb0 con 0x5598cda07540 2024-05-09T02:17:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.253+0000 7f7e74ff9700 1 -- 172.21.15.19:0/3252497468 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7e84004020 con 0x5598cda07540 2024-05-09T02:17:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.253+0000 7f7ea2382b80 10 client.25867 fetch_fsmap learned FSMap version 45 2024-05-09T02:17:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.253+0000 7f7ea2382b80 1 -- 172.21.15.19:0/3252497468 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5598cda14970 con 0x5598cda07540 2024-05-09T02:17:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.254+0000 7f7e74ff9700 1 -- 172.21.15.19:0/3252497468 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e84004bb0 con 0x5598cda07540 2024-05-09T02:17:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.254+0000 7f7e74ff9700 1 -- 172.21.15.19:0/3252497468 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 45) ==== 8366+0+0 (secure 0 0 0) 0x7f7e840079c0 con 0x5598cda07540 2024-05-09T02:17:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.254+0000 7f7ea2382b80 10 client.25867 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T02:17:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.254+0000 7f7ea2382b80 10 client.25867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:17:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.254+0000 7f7ea2382b80 20 client.25867 populate_metadata read hostname 'smithi019' 2024-05-09T02:17:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.254+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/3252497468 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5598cda16e40 0x5598cda19260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.254+0000 7f7ea2382b80 4 client.25867 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:17:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.254+0000 7f7ea2382b80 1 -- 172.21.15.19:0/3252497468 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598cd92da80 con 0x5598cda16e40 2024-05-09T02:17:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.254+0000 7f7e91708700 1 --2- 172.21.15.19:0/3252497468 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5598cda16e40 0x5598cda19260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.255+0000 7f7e91708700 1 --2- 172.21.15.19:0/3252497468 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5598cda16e40 0x5598cda19260 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f2fec4149c0938c server_cookie=30cd387f1f3c072 in_seq=0 out_seq=0 2024-05-09T02:17:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.255+0000 7f7e74ff9700 10 client.25867 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:17:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.256+0000 7f7e74ff9700 1 -- 172.21.15.19:0/3252497468 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5598cd92da80 con 0x5598cda16e40 2024-05-09T02:17:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.256+0000 7f7e74ff9700 10 client.25867 handle_command_reply: tid=0 2024-05-09T02:17:46.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.256+0000 7f7ea2382b80 1 client.25867 shutdown 2024-05-09T02:17:46.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.256+0000 7f7ea2382b80 1 -- 172.21.15.19:0/3252497468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cda07540 msgr2=0x5598cda04fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:46.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.256+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/3252497468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cda07540 0x5598cda04fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7e84003da0 tx=0x7f7e84003ef0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.256+0000 7f7ea2382b80 1 -- 172.21.15.19:0/3252497468 shutdown_connections 2024-05-09T02:17:46.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.256+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/3252497468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cda07540 0x5598cda04fd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.256+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/3252497468 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5598cda16e40 0x5598cda19260 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.256+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/3252497468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda04550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.256+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/3252497468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cd8fad00 0x5598cda04a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.256+0000 7f7ea2382b80 1 -- 172.21.15.19:0/3252497468 >> 172.21.15.19:0/3252497468 conn(0x5598cd883f70 msgr2=0x5598cd9f8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:46.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.256+0000 7f7e74ff9700 0 client.25867 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:17:46.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.259+0000 7f7ea2382b80 1 -- 172.21.15.19:0/3252497468 shutdown_connections 2024-05-09T02:17:46.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.259+0000 7f7ea2382b80 1 -- 172.21.15.19:0/3252497468 wait complete. 2024-05-09T02:17:46.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.259+0000 7f7ea2382b80 20 client.25867 trim_cache size 0 max 16384 2024-05-09T02:17:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.280+0000 7f7ea2382b80 1 Processor -- start 2024-05-09T02:17:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.281+0000 7f7ea2382b80 1 -- start start 2024-05-09T02:17:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.281+0000 7f7ea2382b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.281+0000 7f7ea2382b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cd8fad00 0x5598cda13320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.281+0000 7f7ea2382b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cda07540 0x5598cda04e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.281+0000 7f7ea2382b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1b5430 con 0x5598cda07540 2024-05-09T02:17:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.281+0000 7f7ea2382b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1b7000 con 0x5598cd8804a0 2024-05-09T02:17:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.281+0000 7f7ea2382b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1b5b30 con 0x5598cd8fad00 2024-05-09T02:17:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.281+0000 7f7e91708700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.281+0000 7f7e90f07700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cd8fad00 0x5598cda13320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.281+0000 7f7e91708700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:45390/0 (socket says 172.21.15.19:45390) 2024-05-09T02:17:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.281+0000 7f7e91708700 1 -- 172.21.15.19:0/834861954 learned_addr learned my addr 172.21.15.19:0/834861954 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.281+0000 7f7e91f09700 1 --2- 172.21.15.19:0/834861954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cda07540 0x5598cda04e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.282+0000 7f7e91708700 1 -- 172.21.15.19:0/834861954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cd8fad00 msgr2=0x5598cda13320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.282+0000 7f7e91708700 1 --2- 172.21.15.19:0/834861954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cd8fad00 0x5598cda13320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.282+0000 7f7e91708700 1 -- 172.21.15.19:0/834861954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cda07540 msgr2=0x5598cda04e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.282+0000 7f7e91708700 1 --2- 172.21.15.19:0/834861954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cda07540 0x5598cda04e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.282+0000 7f7e91708700 1 -- 172.21.15.19:0/834861954 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598cd9f4970 con 0x5598cd8804a0 2024-05-09T02:17:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.282+0000 7f7e90f07700 1 --2- 172.21.15.19:0/834861954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cd8fad00 0x5598cda13320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:17:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.282+0000 7f7e91708700 1 --2- 172.21.15.19:0/834861954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda034b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009520 tx=0x7f7e8801d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a0612dd816b25a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.282+0000 7f7e76ffd700 1 -- 172.21.15.19:0/834861954 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88020040 con 0x5598cd8804a0 2024-05-09T02:17:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.282+0000 7f7e76ffd700 1 -- 172.21.15.19:0/834861954 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e880c8070 con 0x5598cd8804a0 2024-05-09T02:17:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.282+0000 7f7e76ffd700 1 -- 172.21.15.19:0/834861954 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e880ca570 con 0x5598cd8804a0 2024-05-09T02:17:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.283+0000 7f7ea2382b80 1 -- 172.21.15.19:0/834861954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 msgr2=0x5598cda034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.283+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/834861954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda034b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009520 tx=0x7f7e8801d040 comp rx=0 tx=0).stop 2024-05-09T02:17:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.283+0000 7f7ea2382b80 1 -- 172.21.15.19:0/834861954 shutdown_connections 2024-05-09T02:17:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.283+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/834861954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cda07540 0x5598cda04e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.283+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/834861954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda034b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.283+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/834861954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cd8fad00 0x5598cda13320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.283+0000 7f7ea2382b80 1 -- 172.21.15.19:0/834861954 >> 172.21.15.19:0/834861954 conn(0x5598cd883f70 msgr2=0x5598cd9ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.283+0000 7f7ea2382b80 1 -- 172.21.15.19:0/834861954 shutdown_connections 2024-05-09T02:17:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.283+0000 7f7ea2382b80 1 -- 172.21.15.19:0/834861954 wait complete. 2024-05-09T02:17:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.284+0000 7f7ea2382b80 1 -- start start 2024-05-09T02:17:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.284+0000 7f7ea2382b80 1 Processor -- start 2024-05-09T02:17:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.285+0000 7f7ea2382b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda797c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.285+0000 7f7ea2382b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cd8fad00 0x5598cda79d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.285+0000 7f7e91708700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.285+0000 7f7e91708700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda797c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:45392/0 (socket says 172.21.15.19:45392) 2024-05-09T02:17:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.285+0000 7f7e91708700 1 -- 172.21.15.19:0/4166326823 learned_addr learned my addr 172.21.15.19:0/4166326823 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.285+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/4166326823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cda07540 0x5598cda196a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.285+0000 7f7ea2382b80 1 -- 172.21.15.19:0/4166326823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1b6930 con 0x5598cd8fad00 2024-05-09T02:17:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.285+0000 7f7ea2382b80 1 -- 172.21.15.19:0/4166326823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1b6200 con 0x5598cd8804a0 2024-05-09T02:17:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.285+0000 7f7ea2382b80 1 -- 172.21.15.19:0/4166326823 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1b33d0 con 0x5598cda07540 2024-05-09T02:17:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.285+0000 7f7ea2382b80 10 client.0 _ll_register_callbacks cb 0x5598cd7916c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:17:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.285+0000 7f7e90f07700 1 --2- 172.21.15.19:0/4166326823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cd8fad00 0x5598cda79d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.285+0000 7f7e91f09700 1 --2- 172.21.15.19:0/4166326823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cda07540 0x5598cda196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.286+0000 7f7e91708700 1 -- 172.21.15.19:0/4166326823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cda07540 msgr2=0x5598cda196a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.286+0000 7f7e91708700 1 --2- 172.21.15.19:0/4166326823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cda07540 0x5598cda196a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.286+0000 7f7e91708700 1 -- 172.21.15.19:0/4166326823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cd8fad00 msgr2=0x5598cda79d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.286+0000 7f7e91708700 1 --2- 172.21.15.19:0/4166326823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cd8fad00 0x5598cda79d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.286+0000 7f7e91708700 1 -- 172.21.15.19:0/4166326823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598cda19ce0 con 0x5598cd8804a0 2024-05-09T02:17:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.286+0000 7f7e91f09700 1 --2- 172.21.15.19:0/4166326823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cda07540 0x5598cda196a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:17:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.286+0000 7f7e91708700 1 --2- 172.21.15.19:0/4166326823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda797c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800b830 tx=0x7f7e880a9500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c45b3a237142e41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.286+0000 7f7e74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:17:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.286+0000 7f7e74ff9700 1 -- 172.21.15.19:0/4166326823 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e880b6070 con 0x5598cd8804a0 2024-05-09T02:17:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.286+0000 7f7e74ff9700 1 -- 172.21.15.19:0/4166326823 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7f7e880a5700 con 0x5598cd8804a0 2024-05-09T02:17:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.286+0000 7f7ea2382b80 1 -- 172.21.15.19:0/4166326823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e880c8070 con 0x5598cd8804a0 2024-05-09T02:17:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.286+0000 7f7e74ff9700 1 -- 172.21.15.19:0/4166326823 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7e88020070 con 0x5598cd8804a0 2024-05-09T02:17:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.286+0000 7f7ea2382b80 10 client.25823 fetch_fsmap learned FSMap version 45 2024-05-09T02:17:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.286+0000 7f7ea2382b80 1 -- 172.21.15.19:0/4166326823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5598cda80dc0 con 0x5598cd8804a0 2024-05-09T02:17:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.286+0000 7f7e74ff9700 1 -- 172.21.15.19:0/4166326823 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e880c8070 con 0x5598cd8804a0 2024-05-09T02:17:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.287+0000 7f7e74ff9700 1 -- 172.21.15.19:0/4166326823 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 45) ==== 8366+0+0 (secure 0 0 0) 0x7f7e880b1ed0 con 0x5598cd8804a0 2024-05-09T02:17:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.287+0000 7f7ea2382b80 10 client.25823 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T02:17:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.287+0000 7f7ea2382b80 10 client.25823 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:17:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.287+0000 7f7ea2382b80 20 client.25823 populate_metadata read hostname 'smithi019' 2024-05-09T02:17:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.287+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/4166326823 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5598cda81460 0x5598cda83880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.287+0000 7f7ea2382b80 4 client.25823 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:17:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.287+0000 7f7ea2382b80 1 -- 172.21.15.19:0/4166326823 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598cd927b20 con 0x5598cda81460 2024-05-09T02:17:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.287+0000 7f7e90f07700 1 --2- 172.21.15.19:0/4166326823 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5598cda81460 0x5598cda83880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.287+0000 7f7e74ff9700 10 client.25823 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:17:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7e90f07700 1 --2- 172.21.15.19:0/4166326823 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5598cda81460 0x5598cda83880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=edf50287a921d7e3 server_cookie=ac84ad5dc313b3ad in_seq=0 out_seq=0 2024-05-09T02:17:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7e74ff9700 1 -- 172.21.15.19:0/4166326823 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598cd927b20 con 0x5598cda81460 2024-05-09T02:17:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7e74ff9700 10 client.25823 handle_command_reply: tid=0 2024-05-09T02:17:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7ea2382b80 1 client.25823 shutdown 2024-05-09T02:17:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7ea2382b80 1 -- 172.21.15.19:0/4166326823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 msgr2=0x5598cda797c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/4166326823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda797c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800b830 tx=0x7f7e880a9500 comp rx=0 tx=0).stop 2024-05-09T02:17:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7ea2382b80 1 -- 172.21.15.19:0/4166326823 shutdown_connections 2024-05-09T02:17:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/4166326823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598cda07540 0x5598cda196a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/4166326823 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5598cda81460 0x5598cda83880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/4166326823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5598cd8804a0 0x5598cda797c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7ea2382b80 1 --2- 172.21.15.19:0/4166326823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cd8fad00 0x5598cda79d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7ea2382b80 1 -- 172.21.15.19:0/4166326823 >> 172.21.15.19:0/4166326823 conn(0x5598cd883f70 msgr2=0x5598cda05b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7e74ff9700 0 client.25823 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:17:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7ea2382b80 1 -- 172.21.15.19:0/4166326823 shutdown_connections 2024-05-09T02:17:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7ea2382b80 1 -- 172.21.15.19:0/4166326823 wait complete. 2024-05-09T02:17:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.288+0000 7f7ea2382b80 20 client.25823 trim_cache size 0 max 16384 2024-05-09T02:17:46.295 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:17:46.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.289+0000 7f7e9270a700 1 -- 172.21.15.19:0/650354003 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7e780b2b40 msgr2=0x7f7e780b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:46.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.289+0000 7f7e9270a700 1 --2- 172.21.15.19:0/650354003 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7e780b2b40 0x7f7e780b5050 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0098e0 tx=0x7f7e7c000bc0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.289+0000 7f7e9270a700 1 -- 172.21.15.19:0/650354003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8c1032c0 msgr2=0x7f7e8c1a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:46.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.289+0000 7f7e9270a700 1 --2- 172.21.15.19:0/650354003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8c1032c0 0x7f7e8c1a13c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800d950 tx=0x7f7e8800dd10 comp rx=0 tx=0).stop 2024-05-09T02:17:46.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.289+0000 7f7e9270a700 1 -- 172.21.15.19:0/650354003 shutdown_connections 2024-05-09T02:17:46.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.289+0000 7f7e9270a700 1 --2- 172.21.15.19:0/650354003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e8c108540 0x7f7e8c1a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.289+0000 7f7e9270a700 1 --2- 172.21.15.19:0/650354003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8c1032c0 0x7f7e8c1a13c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.289+0000 7f7e9270a700 1 --2- 172.21.15.19:0/650354003 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7e780b2b40 0x7f7e780b5050 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.289+0000 7f7e9270a700 1 --2- 172.21.15.19:0/650354003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e8c107bf0 0x7f7e8c1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:46.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.289+0000 7f7e9270a700 1 -- 172.21.15.19:0/650354003 >> 172.21.15.19:0/650354003 conn(0x7f7e8c078bf0 msgr2=0x7f7e8c06b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:46.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.289+0000 7f7e9270a700 1 -- 172.21.15.19:0/650354003 shutdown_connections 2024-05-09T02:17:46.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:46.289+0000 7f7e9270a700 1 -- 172.21.15.19:0/650354003 wait complete. 2024-05-09T02:17:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:46 smithi019 ceph-mon[30151]: pgmap v197: 97 pgs: 97 active+clean; 4.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 8.3 MiB/s wr, 852 op/s 2024-05-09T02:17:46.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:46 smithi059 ceph-mon[34781]: pgmap v197: 97 pgs: 97 active+clean; 4.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 8.3 MiB/s wr, 852 op/s 2024-05-09T02:17:46.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:46 smithi196 ceph-mon[34718]: pgmap v197: 97 pgs: 97 active+clean; 4.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 8.3 MiB/s wr, 852 op/s 2024-05-09T02:17:47.065 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T02:17:47.447 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:17:47.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:47 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 365.0 s, 2556.6 tps, lat 39.028 ms stddev 46.292 2024-05-09T02:17:47.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:47 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:47.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:47 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:48 smithi059 ceph-mon[34781]: pgmap v198: 97 pgs: 97 active+clean; 4.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 5.0 MiB/s wr, 545 op/s 2024-05-09T02:17:48.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:48 smithi196 ceph-mon[34718]: pgmap v198: 97 pgs: 97 active+clean; 4.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 5.0 MiB/s wr, 545 op/s 2024-05-09T02:17:48.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:48 smithi019 ceph-mon[30151]: pgmap v198: 97 pgs: 97 active+clean; 4.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 5.0 MiB/s wr, 545 op/s 2024-05-09T02:17:49.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.400+0000 7fefacac5700 1 -- 172.21.15.19:0/2464020234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa810aa50 msgr2=0x7fefa81111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:49.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.400+0000 7fefacac5700 1 --2- 172.21.15.19:0/2464020234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa810aa50 0x7fefa81111d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009920 tx=0x7fef9c009c30 comp rx=0 tx=0).stop 2024-05-09T02:17:49.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.401+0000 7fefacac5700 1 -- 172.21.15.19:0/2464020234 shutdown_connections 2024-05-09T02:17:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.401+0000 7fefacac5700 1 --2- 172.21.15.19:0/2464020234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa810aa50 0x7fefa81111d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.401+0000 7fefacac5700 1 --2- 172.21.15.19:0/2464020234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefa81026c0 0x7fefa8102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.401+0000 7fefacac5700 1 --2- 172.21.15.19:0/2464020234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa810a100 0x7fefa810a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.401+0000 7fefacac5700 1 -- 172.21.15.19:0/2464020234 >> 172.21.15.19:0/2464020234 conn(0x7fefa80fe0d0 msgr2=0x7fefa81004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.401+0000 7fefacac5700 1 -- 172.21.15.19:0/2464020234 shutdown_connections 2024-05-09T02:17:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.401+0000 7fefacac5700 1 -- 172.21.15.19:0/2464020234 wait complete. 2024-05-09T02:17:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.402+0000 7fefacac5700 1 Processor -- start 2024-05-09T02:17:49.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.402+0000 7fefacac5700 1 -- start start 2024-05-09T02:17:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.402+0000 7fefacac5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefa81026c0 0x7fefa819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.402+0000 7fefacac5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa810a100 0x7fefa819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.402+0000 7fefacac5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa810aa50 0x7fefa819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.402+0000 7fefacac5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81144d0 con 0x7fefa810a100 2024-05-09T02:17:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.402+0000 7fefacac5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8114350 con 0x7fefa810aa50 2024-05-09T02:17:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.402+0000 7fefacac5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8114650 con 0x7fefa81026c0 2024-05-09T02:17:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.402+0000 7fefa5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa810a100 0x7fefa819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.402+0000 7fefa5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa810a100 0x7fefa819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59676/0 (socket says 172.21.15.19:59676) 2024-05-09T02:17:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.402+0000 7fefa5d9b700 1 -- 172.21.15.19:0/2685638649 learned_addr learned my addr 172.21.15.19:0/2685638649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.402+0000 7fefa659c700 1 --2- 172.21.15.19:0/2685638649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefa81026c0 0x7fefa819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.402+0000 7fefa6d9d700 1 --2- 172.21.15.19:0/2685638649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa810aa50 0x7fefa819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.403+0000 7fefa5d9b700 1 -- 172.21.15.19:0/2685638649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefa81026c0 msgr2=0x7fefa819aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.403+0000 7fefa5d9b700 1 --2- 172.21.15.19:0/2685638649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefa81026c0 0x7fefa819aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.403+0000 7fefa5d9b700 1 -- 172.21.15.19:0/2685638649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa810aa50 msgr2=0x7fefa819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:49.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.403+0000 7fefa5d9b700 1 --2- 172.21.15.19:0/2685638649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa810aa50 0x7fefa819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.403+0000 7fefa5d9b700 1 -- 172.21.15.19:0/2685638649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa819fd70 con 0x7fefa810a100 2024-05-09T02:17:49.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.403+0000 7fefa5d9b700 1 --2- 172.21.15.19:0/2685638649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa810a100 0x7fefa819b440 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fef9800e970 tx=0x7fef9800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:49.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.403+0000 7fef977fe700 1 -- 172.21.15.19:0/2685638649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9800cbd0 con 0x7fefa810a100 2024-05-09T02:17:49.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.403+0000 7fefacac5700 1 -- 172.21.15.19:0/2685638649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa81a0060 con 0x7fefa810a100 2024-05-09T02:17:49.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.403+0000 7fef977fe700 1 -- 172.21.15.19:0/2685638649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef9800cd70 con 0x7fefa810a100 2024-05-09T02:17:49.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.403+0000 7fefacac5700 1 -- 172.21.15.19:0/2685638649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa81a7980 con 0x7fefa810a100 2024-05-09T02:17:49.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.403+0000 7fef977fe700 1 -- 172.21.15.19:0/2685638649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9800cbd0 con 0x7fefa810a100 2024-05-09T02:17:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.406+0000 7fef977fe700 1 -- 172.21.15.19:0/2685638649 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fef98005400 con 0x7fefa810a100 2024-05-09T02:17:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.407+0000 7fef977fe700 1 --2- 172.21.15.19:0/2685638649 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fef900b2bf0 0x7fef900b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.407+0000 7fef977fe700 1 -- 172.21.15.19:0/2685638649 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fef980e2490 con 0x7fefa810a100 2024-05-09T02:17:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.408+0000 7fefa659c700 1 --2- 172.21.15.19:0/2685638649 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fef900b2bf0 0x7fef900b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.408+0000 7fefbb4dbb80 1 Processor -- start 2024-05-09T02:17:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.408+0000 7fefa659c700 1 --2- 172.21.15.19:0/2685638649 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fef900b2bf0 0x7fef900b5100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fefa806ab90 tx=0x7fefa000b040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.408+0000 7fefbb4dbb80 1 -- start start 2024-05-09T02:17:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.408+0000 7fefbb4dbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c09605160 0x561c0969cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.408+0000 7fefbb4dbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c0951b4a0 0x561c0969d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.408+0000 7fefbb4dbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c096a1400 0x561c096a1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.408+0000 7fefbb4dbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561c095baff0 con 0x561c0951b4a0 2024-05-09T02:17:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.408+0000 7fefbb4dbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561c095597f0 con 0x561c096a1400 2024-05-09T02:17:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.408+0000 7fefbb4dbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c095b6fb0 con 0x561c09605160 2024-05-09T02:17:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.409+0000 7fefa5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c0951b4a0 0x561c0969d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.409+0000 7fefa5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c0951b4a0 0x561c0969d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59686/0 (socket says 172.21.15.19:59686) 2024-05-09T02:17:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.409+0000 7fefa659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c09605160 0x561c0969cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.409+0000 7fefa5d9b700 1 -- 172.21.15.19:0/1363195708 learned_addr learned my addr 172.21.15.19:0/1363195708 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.409+0000 7fefa6d9d700 1 --2- 172.21.15.19:0/1363195708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c096a1400 0x561c096a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.409+0000 7fefa659c700 1 -- 172.21.15.19:0/1363195708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c096a1400 msgr2=0x561c096a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.409+0000 7fefa659c700 1 --2- 172.21.15.19:0/1363195708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c096a1400 0x561c096a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.409+0000 7fefa659c700 1 -- 172.21.15.19:0/1363195708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c0951b4a0 msgr2=0x561c0969d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.409+0000 7fefa659c700 1 --2- 172.21.15.19:0/1363195708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c0951b4a0 0x561c0969d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.409+0000 7fefa659c700 1 -- 172.21.15.19:0/1363195708 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c09551370 con 0x561c09605160 2024-05-09T02:17:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.409+0000 7fefa5d9b700 1 --2- 172.21.15.19:0/1363195708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c0951b4a0 0x561c0969d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:17:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.409+0000 7fefa659c700 1 --2- 172.21.15.19:0/1363195708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c09605160 0x561c0969cd50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fefa8103460 tx=0x7fefa0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e24912ec4dd1f677 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.409+0000 7fef94ff9700 1 -- 172.21.15.19:0/1363195708 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0003c00 con 0x561c09605160 2024-05-09T02:17:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.409+0000 7fef94ff9700 1 -- 172.21.15.19:0/1363195708 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefa001b070 con 0x561c09605160 2024-05-09T02:17:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.409+0000 7fef94ff9700 1 -- 172.21.15.19:0/1363195708 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0015700 con 0x561c09605160 2024-05-09T02:17:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.410+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1363195708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c09605160 msgr2=0x561c0969cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.410+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/1363195708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c09605160 0x561c0969cd50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fefa8103460 tx=0x7fefa0014000 comp rx=0 tx=0).stop 2024-05-09T02:17:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.410+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1363195708 shutdown_connections 2024-05-09T02:17:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.410+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/1363195708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c096a1400 0x561c096a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.410+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/1363195708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c09605160 0x561c0969cd50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.410+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/1363195708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c0951b4a0 0x561c0969d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.410+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1363195708 >> 172.21.15.19:0/1363195708 conn(0x561c0957b6a0 msgr2=0x561c09693400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:49.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.411+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1363195708 shutdown_connections 2024-05-09T02:17:49.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.411+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1363195708 wait complete. 2024-05-09T02:17:49.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.411+0000 7fefbb4dbb80 1 -- start start 2024-05-09T02:17:49.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.411+0000 7fefbb4dbb80 1 Processor -- start 2024-05-09T02:17:49.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.411+0000 7fefbb4dbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0951b4a0 0x561c096a6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.411+0000 7fefbb4dbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c09605160 0x561c096a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.411+0000 7fefbb4dbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c096a1400 0x561c096a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.411+0000 7fefbb4dbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561c09436ba0 con 0x561c09605160 2024-05-09T02:17:49.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.411+0000 7fefbb4dbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561c09575150 con 0x561c0951b4a0 2024-05-09T02:17:49.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.411+0000 7fefbb4dbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c096a21b0 con 0x561c096a1400 2024-05-09T02:17:49.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.411+0000 7fefbb4dbb80 10 client.0 _ll_register_callbacks cb 0x561c0942c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:17:49.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.412+0000 7fefa5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c09605160 0x561c096a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.412+0000 7fefa5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c09605160 0x561c096a6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59700/0 (socket says 172.21.15.19:59700) 2024-05-09T02:17:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.412+0000 7fefa5d9b700 1 -- 172.21.15.19:0/3536939490 learned_addr learned my addr 172.21.15.19:0/3536939490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.412+0000 7fefa659c700 1 --2- 172.21.15.19:0/3536939490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0951b4a0 0x561c096a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.412+0000 7fefa6d9d700 1 --2- 172.21.15.19:0/3536939490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c096a1400 0x561c096a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.412+0000 7fefa5d9b700 1 -- 172.21.15.19:0/3536939490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c096a1400 msgr2=0x561c096a70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.412+0000 7fefa5d9b700 1 --2- 172.21.15.19:0/3536939490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c096a1400 0x561c096a70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.412+0000 7fefa5d9b700 1 -- 172.21.15.19:0/3536939490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0951b4a0 msgr2=0x561c096a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.412+0000 7fefa5d9b700 1 --2- 172.21.15.19:0/3536939490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0951b4a0 0x561c096a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.412+0000 7fefa5d9b700 1 -- 172.21.15.19:0/3536939490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c09436490 con 0x561c09605160 2024-05-09T02:17:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.412+0000 7fefa5d9b700 1 --2- 172.21.15.19:0/3536939490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c09605160 0x561c096a6b70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fef980dfc90 tx=0x7fef980aa210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2070b85b11bc95 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.412+0000 7fef8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:17:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.412+0000 7fef8dffb700 1 -- 172.21.15.19:0/3536939490 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef980ae9e0 con 0x561c09605160 2024-05-09T02:17:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.412+0000 7fef8dffb700 1 -- 172.21.15.19:0/3536939490 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fef980a6c20 con 0x561c09605160 2024-05-09T02:17:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.413+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/3536939490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa001b070 con 0x561c09605160 2024-05-09T02:17:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.414+0000 7fef8dffb700 1 -- 172.21.15.19:0/3536939490 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fef980b3490 con 0x561c09605160 2024-05-09T02:17:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.414+0000 7fefbb4dbb80 10 client.16116 fetch_fsmap learned FSMap version 47 2024-05-09T02:17:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.414+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/3536939490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561c096b3f50 con 0x561c09605160 2024-05-09T02:17:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.415+0000 7fef8dffb700 1 -- 172.21.15.19:0/3536939490 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefa001b070 con 0x561c09605160 2024-05-09T02:17:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.415+0000 7fef8dffb700 1 -- 172.21.15.19:0/3536939490 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fef980b3000 con 0x561c09605160 2024-05-09T02:17:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.415+0000 7fefbb4dbb80 10 client.16116 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T02:17:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.415+0000 7fefbb4dbb80 10 client.16116 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:17:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.415+0000 7fefbb4dbb80 20 client.16116 populate_metadata read hostname 'smithi019' 2024-05-09T02:17:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.415+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/3536939490 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c096b6590 0x561c096b89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.415+0000 7fefbb4dbb80 4 client.16116 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:17:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.415+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/3536939490 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c095c89d0 con 0x561c096b6590 2024-05-09T02:17:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.415+0000 7fefa659c700 1 --2- 172.21.15.19:0/3536939490 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c096b6590 0x561c096b89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.416+0000 7fef8dffb700 10 client.16116 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:17:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.416+0000 7fefa659c700 1 --2- 172.21.15.19:0/3536939490 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c096b6590 0x561c096b89b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=906d8dff4f572a0c server_cookie=2d7a51503cf82312 in_seq=0 out_seq=0 2024-05-09T02:17:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.417+0000 7fef8dffb700 1 -- 172.21.15.19:0/3536939490 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561c095c89d0 con 0x561c096b6590 2024-05-09T02:17:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.417+0000 7fef8dffb700 10 client.16116 handle_command_reply: tid=0 2024-05-09T02:17:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.417+0000 7fefbb4dbb80 1 client.16116 shutdown 2024-05-09T02:17:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.417+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/3536939490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c09605160 msgr2=0x561c096a6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.417+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/3536939490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c09605160 0x561c096a6b70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fef980dfc90 tx=0x7fef980aa210 comp rx=0 tx=0).stop 2024-05-09T02:17:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.417+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/3536939490 shutdown_connections 2024-05-09T02:17:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.417+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/3536939490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c096a1400 0x561c096a70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.417+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/3536939490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0951b4a0 0x561c096a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.417+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/3536939490 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c096b6590 0x561c096b89b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.417+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/3536939490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c09605160 0x561c096a6b70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.417+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/3536939490 >> 172.21.15.19:0/3536939490 conn(0x561c0957b6a0 msgr2=0x561c096931c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.417+0000 7fef8dffb700 0 client.16116 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:17:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.417+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/3536939490 shutdown_connections 2024-05-09T02:17:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.417+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/3536939490 wait complete. 2024-05-09T02:17:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.417+0000 7fefbb4dbb80 20 client.16116 trim_cache size 0 max 16384 2024-05-09T02:17:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.441+0000 7fefbb4dbb80 1 Processor -- start 2024-05-09T02:17:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.441+0000 7fefbb4dbb80 1 -- start start 2024-05-09T02:17:49.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.442+0000 7fefbb4dbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0951b4a0 0x561c096b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.442+0000 7fefbb4dbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c09605160 0x561c096be150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.442+0000 7fefbb4dbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c096a1400 0x561c096be690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.442+0000 7fefbb4dbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81acd80 con 0x561c09605160 2024-05-09T02:17:49.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.442+0000 7fefbb4dbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81ae980 con 0x561c0951b4a0 2024-05-09T02:17:49.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.442+0000 7fefbb4dbb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81ad4e0 con 0x561c096a1400 2024-05-09T02:17:49.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.442+0000 7fefa5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c09605160 0x561c096be150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.442+0000 7fefa5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c09605160 0x561c096be150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59716/0 (socket says 172.21.15.19:59716) 2024-05-09T02:17:49.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.442+0000 7fefa5d9b700 1 -- 172.21.15.19:0/3418164056 learned_addr learned my addr 172.21.15.19:0/3418164056 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:49.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.442+0000 7fefa659c700 1 --2- 172.21.15.19:0/3418164056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0951b4a0 0x561c096b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.443+0000 7fefa6d9d700 1 --2- 172.21.15.19:0/3418164056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c096a1400 0x561c096be690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.443+0000 7fefa5d9b700 1 -- 172.21.15.19:0/3418164056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c096a1400 msgr2=0x561c096be690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:49.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.443+0000 7fefa5d9b700 1 --2- 172.21.15.19:0/3418164056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c096a1400 0x561c096be690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.443+0000 7fefa5d9b700 1 -- 172.21.15.19:0/3418164056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0951b4a0 msgr2=0x561c096b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.443+0000 7fefa5d9b700 1 --2- 172.21.15.19:0/3418164056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0951b4a0 0x561c096b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.443+0000 7fefa5d9b700 1 -- 172.21.15.19:0/3418164056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c096a4ba0 con 0x561c09605160 2024-05-09T02:17:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.443+0000 7fefa659c700 1 --2- 172.21.15.19:0/3418164056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0951b4a0 0x561c096b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:17:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.443+0000 7fefa5d9b700 1 --2- 172.21.15.19:0/3418164056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c09605160 0x561c096be150 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fef980a78a0 tx=0x7fef980aa7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8173e8a2bb8eb6cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.443+0000 7fef8ffff700 1 -- 172.21.15.19:0/3418164056 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef980056f0 con 0x561c09605160 2024-05-09T02:17:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.443+0000 7fef8ffff700 1 -- 172.21.15.19:0/3418164056 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef980a6c20 con 0x561c09605160 2024-05-09T02:17:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.443+0000 7fef8ffff700 1 -- 172.21.15.19:0/3418164056 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98016e40 con 0x561c09605160 2024-05-09T02:17:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.445+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/3418164056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c09605160 msgr2=0x561c096be150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.445+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/3418164056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c09605160 0x561c096be150 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fef980a78a0 tx=0x7fef980aa7b0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.445+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/3418164056 shutdown_connections 2024-05-09T02:17:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.445+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/3418164056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c096a1400 0x561c096be690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.445+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/3418164056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0951b4a0 0x561c096b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.445+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/3418164056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c09605160 0x561c096be150 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.445+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/3418164056 >> 172.21.15.19:0/3418164056 conn(0x561c0957b6a0 msgr2=0x561c096a4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.445+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/3418164056 shutdown_connections 2024-05-09T02:17:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.445+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/3418164056 wait complete. 2024-05-09T02:17:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.446+0000 7fefbb4dbb80 1 -- start start 2024-05-09T02:17:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.446+0000 7fefbb4dbb80 1 Processor -- start 2024-05-09T02:17:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.446+0000 7fefbb4dbb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0951b4a0 0x561c096a3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefbb4dbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c09605160 0x561c096a3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefa659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0951b4a0 0x561c096a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefa659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0951b4a0 0x561c096a3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:49058/0 (socket says 172.21.15.19:49058) 2024-05-09T02:17:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefa659c700 1 -- 172.21.15.19:0/1154708518 learned_addr learned my addr 172.21.15.19:0/1154708518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c096a1400 0x561c096ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1154708518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81ae2b0 con 0x561c096a1400 2024-05-09T02:17:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1154708518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81adbb0 con 0x561c09605160 2024-05-09T02:17:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefa5d9b700 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c09605160 0x561c096a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1154708518 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81aad40 con 0x561c0951b4a0 2024-05-09T02:17:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefbb4dbb80 10 client.0 _ll_register_callbacks cb 0x561c0942c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:17:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefa6d9d700 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c096a1400 0x561c096ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefa659c700 1 -- 172.21.15.19:0/1154708518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c09605160 msgr2=0x561c096a3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefa659c700 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c09605160 0x561c096a3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefa659c700 1 -- 172.21.15.19:0/1154708518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c096a1400 msgr2=0x561c096ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefa659c700 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c096a1400 0x561c096ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefa659c700 1 -- 172.21.15.19:0/1154708518 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c096aecb0 con 0x561c0951b4a0 2024-05-09T02:17:49.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefa5d9b700 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c09605160 0x561c096a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:17:49.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefa6d9d700 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c096a1400 0x561c096ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:17:49.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fefa659c700 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0951b4a0 0x561c096a3410 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fefa0017fd0 tx=0x7fefa0029690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ab3b8802f5e7a48 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:49.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.447+0000 7fef8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:17:49.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.448+0000 7fef8dffb700 1 -- 172.21.15.19:0/1154708518 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa003a070 con 0x561c0951b4a0 2024-05-09T02:17:49.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.448+0000 7fef8dffb700 1 -- 172.21.15.19:0/1154708518 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fefa0029d40 con 0x561c0951b4a0 2024-05-09T02:17:49.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.448+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1154708518 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef980a6c20 con 0x561c0951b4a0 2024-05-09T02:17:49.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.448+0000 7fef8dffb700 1 -- 172.21.15.19:0/1154708518 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fefa0046050 con 0x561c0951b4a0 2024-05-09T02:17:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.448+0000 7fefbb4dbb80 10 client.25891 fetch_fsmap learned FSMap version 47 2024-05-09T02:17:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.448+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1154708518 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561c09723fa0 con 0x561c0951b4a0 2024-05-09T02:17:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.448+0000 7fef8dffb700 1 -- 172.21.15.19:0/1154708518 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef980a6c20 con 0x561c0951b4a0 2024-05-09T02:17:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.449+0000 7fef8dffb700 1 -- 172.21.15.19:0/1154708518 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fefa0033070 con 0x561c0951b4a0 2024-05-09T02:17:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.449+0000 7fefbb4dbb80 10 client.25891 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T02:17:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.449+0000 7fefbb4dbb80 10 client.25891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:17:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.449+0000 7fefbb4dbb80 20 client.25891 populate_metadata read hostname 'smithi019' 2024-05-09T02:17:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.449+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c09724380 0x561c097267a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.449+0000 7fefbb4dbb80 4 client.25891 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:17:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.449+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1154708518 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c095c2a70 con 0x561c09724380 2024-05-09T02:17:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.450+0000 7fefa5d9b700 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c09724380 0x561c097267a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:49.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.450+0000 7fefa5d9b700 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c09724380 0x561c097267a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=acd7350f965f0376 server_cookie=5e9558ecc3ae6320 in_seq=0 out_seq=0 2024-05-09T02:17:49.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.450+0000 7fef8dffb700 10 client.25891 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:17:49.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.450+0000 7fef8dffb700 1 -- 172.21.15.19:0/1154708518 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c095c2a70 con 0x561c09724380 2024-05-09T02:17:49.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.450+0000 7fef8dffb700 10 client.25891 handle_command_reply: tid=0 2024-05-09T02:17:49.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.450+0000 7fefbb4dbb80 1 client.25891 shutdown 2024-05-09T02:17:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.450+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1154708518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0951b4a0 msgr2=0x561c096a3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.450+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0951b4a0 0x561c096a3410 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fefa0017fd0 tx=0x7fefa0029690 comp rx=0 tx=0).stop 2024-05-09T02:17:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.451+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1154708518 shutdown_connections 2024-05-09T02:17:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.451+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c09724380 0x561c097267a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.451+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c096a1400 0x561c096ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.451+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0951b4a0 0x561c096a3410 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.451+0000 7fef8dffb700 0 client.25891 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:17:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.451+0000 7fefbb4dbb80 1 --2- 172.21.15.19:0/1154708518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c09605160 0x561c096a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.451+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1154708518 >> 172.21.15.19:0/1154708518 conn(0x561c0957b6a0 msgr2=0x561c096a0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:49.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.451+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1154708518 shutdown_connections 2024-05-09T02:17:49.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.451+0000 7fefbb4dbb80 1 -- 172.21.15.19:0/1154708518 wait complete. 2024-05-09T02:17:49.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.451+0000 7fefbb4dbb80 20 client.25891 trim_cache size 0 max 16384 2024-05-09T02:17:49.460 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:17:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.451+0000 7fefacac5700 1 -- 172.21.15.19:0/2685638649 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fef900b2bf0 msgr2=0x7fef900b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.451+0000 7fefacac5700 1 --2- 172.21.15.19:0/2685638649 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fef900b2bf0 0x7fef900b5100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fefa806ab90 tx=0x7fefa000b040 comp rx=0 tx=0).stop 2024-05-09T02:17:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.451+0000 7fefacac5700 1 -- 172.21.15.19:0/2685638649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa810a100 msgr2=0x7fefa819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.451+0000 7fefacac5700 1 --2- 172.21.15.19:0/2685638649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa810a100 0x7fefa819b440 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fef9800e970 tx=0x7fef9800ec80 comp rx=0 tx=0).stop 2024-05-09T02:17:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.452+0000 7fefacac5700 1 -- 172.21.15.19:0/2685638649 shutdown_connections 2024-05-09T02:17:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.452+0000 7fefacac5700 1 --2- 172.21.15.19:0/2685638649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa810aa50 0x7fefa819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.452+0000 7fefacac5700 1 --2- 172.21.15.19:0/2685638649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefa81026c0 0x7fefa819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.452+0000 7fefacac5700 1 --2- 172.21.15.19:0/2685638649 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fef900b2bf0 0x7fef900b5100 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.452+0000 7fefacac5700 1 --2- 172.21.15.19:0/2685638649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefa810a100 0x7fefa819b440 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.452+0000 7fefacac5700 1 -- 172.21.15.19:0/2685638649 >> 172.21.15.19:0/2685638649 conn(0x7fefa80fe0d0 msgr2=0x7fefa8100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:49.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.452+0000 7fefacac5700 1 -- 172.21.15.19:0/2685638649 shutdown_connections 2024-05-09T02:17:49.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:49.452+0000 7fefacac5700 1 -- 172.21.15.19:0/2685638649 wait complete. 2024-05-09T02:17:49.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:49 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:49.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:49 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:49.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:49 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:17:50.421 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T02:17:50.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:50 smithi059 ceph-mon[34781]: pgmap v199: 97 pgs: 97 active+clean; 4.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 7.7 MiB/s wr, 833 op/s 2024-05-09T02:17:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:17:50.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:50 smithi196 ceph-mon[34718]: pgmap v199: 97 pgs: 97 active+clean; 4.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 7.7 MiB/s wr, 833 op/s 2024-05-09T02:17:50.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:50 smithi019 ceph-mon[30151]: pgmap v199: 97 pgs: 97 active+clean; 4.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 7.7 MiB/s wr, 833 op/s 2024-05-09T02:17:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.650+0000 7fcc52195700 1 -- 172.21.15.19:0/1339968008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc4c1034e0 msgr2=0x7fcc4c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.650+0000 7fcc52195700 1 --2- 172.21.15.19:0/1339968008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc4c1034e0 0x7fcc4c1038f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009990 tx=0x7fcc3c009ca0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.650+0000 7fcc52195700 1 -- 172.21.15.19:0/1339968008 shutdown_connections 2024-05-09T02:17:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.650+0000 7fcc52195700 1 --2- 172.21.15.19:0/1339968008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c108760 0x7fcc4c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.650+0000 7fcc52195700 1 --2- 172.21.15.19:0/1339968008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc4c1034e0 0x7fcc4c1038f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.650+0000 7fcc52195700 1 --2- 172.21.15.19:0/1339968008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc4c107e10 0x7fcc4c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.650+0000 7fcc52195700 1 -- 172.21.15.19:0/1339968008 >> 172.21.15.19:0/1339968008 conn(0x7fcc4c0fef30 msgr2=0x7fcc4c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.650+0000 7fcc52195700 1 -- 172.21.15.19:0/1339968008 shutdown_connections 2024-05-09T02:17:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.650+0000 7fcc52195700 1 -- 172.21.15.19:0/1339968008 wait complete. 2024-05-09T02:17:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.651+0000 7fcc52195700 1 Processor -- start 2024-05-09T02:17:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.651+0000 7fcc52195700 1 -- start start 2024-05-09T02:17:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.652+0000 7fcc52195700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c1034e0 0x7fcc4c19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.652+0000 7fcc52195700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc4c107e10 0x7fcc4c19b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.652+0000 7fcc52195700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc4c108760 0x7fcc4c19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.652+0000 7fcc52195700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c114620 con 0x7fcc4c1034e0 2024-05-09T02:17:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.652+0000 7fcc52195700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c1144a0 con 0x7fcc4c107e10 2024-05-09T02:17:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.652+0000 7fcc4affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc4c107e10 0x7fcc4c19b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:52.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.652+0000 7fcc52195700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c1147a0 con 0x7fcc4c108760 2024-05-09T02:17:52.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.652+0000 7fcc4affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc4c107e10 0x7fcc4c19b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:35496/0 (socket says 172.21.15.19:35496) 2024-05-09T02:17:52.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.652+0000 7fcc4affd700 1 -- 172.21.15.19:0/1798307855 learned_addr learned my addr 172.21.15.19:0/1798307855 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:52.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.652+0000 7fcc4bfff700 1 --2- 172.21.15.19:0/1798307855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc4c108760 0x7fcc4c19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 370.0 s, 1730.4 tps, lat 57.781 ms stddev 67.230 2024-05-09T02:17:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.652+0000 7fcc4b7fe700 1 --2- 172.21.15.19:0/1798307855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c1034e0 0x7fcc4c19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.653+0000 7fcc4affd700 1 -- 172.21.15.19:0/1798307855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc4c108760 msgr2=0x7fcc4c19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.653+0000 7fcc4affd700 1 --2- 172.21.15.19:0/1798307855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc4c108760 0x7fcc4c19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.653+0000 7fcc4affd700 1 -- 172.21.15.19:0/1798307855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c1034e0 msgr2=0x7fcc4c19af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.653+0000 7fcc4affd700 1 --2- 172.21.15.19:0/1798307855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c1034e0 0x7fcc4c19af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.653+0000 7fcc4affd700 1 -- 172.21.15.19:0/1798307855 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c19fe40 con 0x7fcc4c107e10 2024-05-09T02:17:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.656+0000 7fcc4affd700 1 --2- 172.21.15.19:0/1798307855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc4c107e10 0x7fcc4c19b510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000b500 tx=0x7fcc4000b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:52.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.658+0000 7fcc48ff9700 1 -- 172.21.15.19:0/1798307855 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40015020 con 0x7fcc4c107e10 2024-05-09T02:17:52.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.658+0000 7fcc48ff9700 1 -- 172.21.15.19:0/1798307855 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc40011ce0 con 0x7fcc4c107e10 2024-05-09T02:17:52.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.660+0000 7fcc48ff9700 1 -- 172.21.15.19:0/1798307855 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40010bf0 con 0x7fcc4c107e10 2024-05-09T02:17:52.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.661+0000 7fcc52195700 1 -- 172.21.15.19:0/1798307855 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4c1a0130 con 0x7fcc4c107e10 2024-05-09T02:17:52.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.662+0000 7fcc52195700 1 -- 172.21.15.19:0/1798307855 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc4c1107d0 con 0x7fcc4c107e10 2024-05-09T02:17:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.667+0000 7fcc60babb80 1 Processor -- start 2024-05-09T02:17:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.670+0000 7fcc60babb80 1 -- start start 2024-05-09T02:17:52.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.671+0000 7fcc60babb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7b76d00 0x55dcd7c7f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.671+0000 7fcc48ff9700 1 -- 172.21.15.19:0/1798307855 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcc40018020 con 0x7fcc4c107e10 2024-05-09T02:17:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.672+0000 7fcc60babb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c7f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.672+0000 7fcc60babb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7c79400 0x55dcd7c79810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.672+0000 7fcc60babb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dcd7b9c0a0 con 0x55dcd7c79400 2024-05-09T02:17:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.673+0000 7fcc60babb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dcd7b419a0 con 0x55dcd7afc4a0 2024-05-09T02:17:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.673+0000 7fcc60babb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dcd7b98060 con 0x55dcd7b76d00 2024-05-09T02:17:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.674+0000 7fcc4affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7c79400 0x55dcd7c79810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.674+0000 7fcc4bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7b76d00 0x55dcd7c7f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.674+0000 7fcc4bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7b76d00 0x55dcd7c7f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:49078/0 (socket says 172.21.15.19:49078) 2024-05-09T02:17:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.674+0000 7fcc4bfff700 1 -- 172.21.15.19:0/1408053681 learned_addr learned my addr 172.21.15.19:0/1408053681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.676+0000 7fcc48ff9700 1 --2- 172.21.15.19:0/1798307855 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcc340b2a50 0x7fcc340b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.677+0000 7fcc4bfff700 1 -- 172.21.15.19:0/1408053681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 msgr2=0x55dcd7c7f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.677+0000 7fcc4bfff700 1 --2- 172.21.15.19:0/1408053681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c7f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.677+0000 7fcc4bfff700 1 -- 172.21.15.19:0/1408053681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7c79400 msgr2=0x55dcd7c79810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.677+0000 7fcc4bfff700 1 --2- 172.21.15.19:0/1408053681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7c79400 0x55dcd7c79810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.677+0000 7fcc4bfff700 1 -- 172.21.15.19:0/1408053681 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcd7b32500 con 0x55dcd7b76d00 2024-05-09T02:17:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.677+0000 7fcc4bfff700 1 --2- 172.21.15.19:0/1798307855 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcc340b2a50 0x7fcc340b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.678+0000 7fcc4bfff700 1 --2- 172.21.15.19:0/1408053681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7b76d00 0x55dcd7c7f240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400a500 tx=0x7fcc4400a810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c326c90d423f044 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.678+0000 7fcc48ff9700 1 -- 172.21.15.19:0/1798307855 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fcc400e1990 con 0x7fcc4c107e10 2024-05-09T02:17:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.679+0000 7fcc31ffb700 1 -- 172.21.15.19:0/1408053681 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44023440 con 0x55dcd7b76d00 2024-05-09T02:17:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.679+0000 7fcc31ffb700 1 -- 172.21.15.19:0/1408053681 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc44023a90 con 0x55dcd7b76d00 2024-05-09T02:17:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.679+0000 7fcc31ffb700 1 -- 172.21.15.19:0/1408053681 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44017880 con 0x55dcd7b76d00 2024-05-09T02:17:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.680+0000 7fcc4bfff700 1 --2- 172.21.15.19:0/1798307855 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcc340b2a50 0x7fcc340b4f60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcc44020000 tx=0x7fcc44021000 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.682+0000 7fcc60babb80 1 -- 172.21.15.19:0/1408053681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7b76d00 msgr2=0x55dcd7c7f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.682+0000 7fcc60babb80 1 --2- 172.21.15.19:0/1408053681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7b76d00 0x55dcd7c7f240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400a500 tx=0x7fcc4400a810 comp rx=0 tx=0).stop 2024-05-09T02:17:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.683+0000 7fcc60babb80 1 -- 172.21.15.19:0/1408053681 shutdown_connections 2024-05-09T02:17:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.683+0000 7fcc60babb80 1 --2- 172.21.15.19:0/1408053681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7c79400 0x55dcd7c79810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.683+0000 7fcc60babb80 1 --2- 172.21.15.19:0/1408053681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7b76d00 0x55dcd7c7f240 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.683+0000 7fcc60babb80 1 --2- 172.21.15.19:0/1408053681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c7f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.683+0000 7fcc60babb80 1 -- 172.21.15.19:0/1408053681 >> 172.21.15.19:0/1408053681 conn(0x55dcd7afff70 msgr2=0x55dcd7c74220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.685+0000 7fcc60babb80 1 -- 172.21.15.19:0/1408053681 shutdown_connections 2024-05-09T02:17:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.687+0000 7fcc60babb80 1 -- 172.21.15.19:0/1408053681 wait complete. 2024-05-09T02:17:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.687+0000 7fcc60babb80 1 -- start start 2024-05-09T02:17:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.688+0000 7fcc60babb80 1 Processor -- start 2024-05-09T02:17:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.689+0000 7fcc60babb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c83000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.689+0000 7fcc60babb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7b76d00 0x55dcd7c83540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.689+0000 7fcc60babb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7c79400 0x55dcd7c83a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.690+0000 7fcc60babb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dcd7c7a160 con 0x55dcd7afc4a0 2024-05-09T02:17:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.690+0000 7fcc4b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c83000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.690+0000 7fcc4b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c83000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59758/0 (socket says 172.21.15.19:59758) 2024-05-09T02:17:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.690+0000 7fcc4b7fe700 1 -- 172.21.15.19:0/1693713686 learned_addr learned my addr 172.21.15.19:0/1693713686 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.690+0000 7fcc4bfff700 1 --2- 172.21.15.19:0/1693713686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7c79400 0x55dcd7c83a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.690+0000 7fcc4affd700 1 --2- 172.21.15.19:0/1693713686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7b76d00 0x55dcd7c83540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.690+0000 7fcc60babb80 1 -- 172.21.15.19:0/1693713686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dcd7b36b20 con 0x55dcd7b76d00 2024-05-09T02:17:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.691+0000 7fcc60babb80 1 -- 172.21.15.19:0/1693713686 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dcd7c86020 con 0x55dcd7c79400 2024-05-09T02:17:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.691+0000 7fcc60babb80 10 client.0 _ll_register_callbacks cb 0x55dcd7a0d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:17:52.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.692+0000 7fcc4bfff700 1 -- 172.21.15.19:0/1693713686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7b76d00 msgr2=0x55dcd7c83540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:52.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.692+0000 7fcc4bfff700 1 --2- 172.21.15.19:0/1693713686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7b76d00 0x55dcd7c83540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.692+0000 7fcc4bfff700 1 -- 172.21.15.19:0/1693713686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7afc4a0 msgr2=0x55dcd7c83000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.692+0000 7fcc4bfff700 1 --2- 172.21.15.19:0/1693713686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c83000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.692+0000 7fcc4bfff700 1 -- 172.21.15.19:0/1693713686 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcd7a17490 con 0x55dcd7c79400 2024-05-09T02:17:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.692+0000 7fcc4bfff700 1 --2- 172.21.15.19:0/1693713686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7c79400 0x55dcd7c83a80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400bfd0 tx=0x7fcc44017d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2008d2c1c6669497 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.693+0000 7fcc2affd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:17:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.693+0000 7fcc2affd700 1 -- 172.21.15.19:0/1693713686 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44023850 con 0x55dcd7c79400 2024-05-09T02:17:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.693+0000 7fcc60babb80 1 -- 172.21.15.19:0/1693713686 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc44023a90 con 0x55dcd7c79400 2024-05-09T02:17:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.693+0000 7fcc2affd700 1 -- 172.21.15.19:0/1693713686 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fcc44030370 con 0x55dcd7c79400 2024-05-09T02:17:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.693+0000 7fcc2affd700 1 -- 172.21.15.19:0/1693713686 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fcc44030570 con 0x55dcd7c79400 2024-05-09T02:17:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.693+0000 7fcc60babb80 10 client.25900 fetch_fsmap learned FSMap version 47 2024-05-09T02:17:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.693+0000 7fcc60babb80 1 -- 172.21.15.19:0/1693713686 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55dcd7c90830 con 0x55dcd7c79400 2024-05-09T02:17:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.693+0000 7fcc2affd700 1 -- 172.21.15.19:0/1693713686 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc44023a90 con 0x55dcd7c79400 2024-05-09T02:17:52.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.693+0000 7fcc2affd700 1 -- 172.21.15.19:0/1693713686 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fcc44032470 con 0x55dcd7c79400 2024-05-09T02:17:52.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.694+0000 7fcc60babb80 10 client.25900 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T02:17:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.694+0000 7fcc60babb80 10 client.25900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:17:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.694+0000 7fcc60babb80 20 client.25900 populate_metadata read hostname 'smithi019' 2024-05-09T02:17:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.694+0000 7fcc60babb80 1 --2- 172.21.15.19:0/1693713686 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dcd7c92d00 0x55dcd7c95120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.694+0000 7fcc60babb80 4 client.25900 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:17:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.694+0000 7fcc60babb80 1 -- 172.21.15.19:0/1693713686 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcd7ba9a80 con 0x55dcd7c92d00 2024-05-09T02:17:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.694+0000 7fcc4b7fe700 1 --2- 172.21.15.19:0/1693713686 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dcd7c92d00 0x55dcd7c95120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.694+0000 7fcc4b7fe700 1 --2- 172.21.15.19:0/1693713686 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dcd7c92d00 0x55dcd7c95120 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f68236ace3a22400 server_cookie=c507cf4af358a746 in_seq=0 out_seq=0 2024-05-09T02:17:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.694+0000 7fcc2affd700 10 client.25900 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:17:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.695+0000 7fcc2affd700 1 -- 172.21.15.19:0/1693713686 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dcd7ba9a80 con 0x55dcd7c92d00 2024-05-09T02:17:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.695+0000 7fcc2affd700 10 client.25900 handle_command_reply: tid=0 2024-05-09T02:17:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.695+0000 7fcc60babb80 1 client.25900 shutdown 2024-05-09T02:17:52.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.696+0000 7fcc60babb80 1 -- 172.21.15.19:0/1693713686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7c79400 msgr2=0x55dcd7c83a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:52.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.696+0000 7fcc60babb80 1 --2- 172.21.15.19:0/1693713686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7c79400 0x55dcd7c83a80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400bfd0 tx=0x7fcc44017d10 comp rx=0 tx=0).stop 2024-05-09T02:17:52.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.696+0000 7fcc60babb80 1 -- 172.21.15.19:0/1693713686 shutdown_connections 2024-05-09T02:17:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.696+0000 7fcc60babb80 1 --2- 172.21.15.19:0/1693713686 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dcd7c92d00 0x55dcd7c95120 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.696+0000 7fcc60babb80 1 --2- 172.21.15.19:0/1693713686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7c79400 0x55dcd7c83a80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.696+0000 7fcc60babb80 1 --2- 172.21.15.19:0/1693713686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c83000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.696+0000 7fcc60babb80 1 --2- 172.21.15.19:0/1693713686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7b76d00 0x55dcd7c83540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.696+0000 7fcc2affd700 0 client.25900 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:17:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.696+0000 7fcc60babb80 1 -- 172.21.15.19:0/1693713686 >> 172.21.15.19:0/1693713686 conn(0x55dcd7afff70 msgr2=0x55dcd7c6dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.696+0000 7fcc60babb80 1 -- 172.21.15.19:0/1693713686 shutdown_connections 2024-05-09T02:17:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.696+0000 7fcc60babb80 1 -- 172.21.15.19:0/1693713686 wait complete. 2024-05-09T02:17:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.696+0000 7fcc60babb80 20 client.25900 trim_cache size 0 max 16384 2024-05-09T02:17:52.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:52 smithi059 ceph-mon[34781]: pgmap v200: 97 pgs: 97 active+clean; 4.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.3 MiB/s wr, 568 op/s 2024-05-09T02:17:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.717+0000 7fcc60babb80 1 Processor -- start 2024-05-09T02:17:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.717+0000 7fcc60babb80 1 -- start start 2024-05-09T02:17:52.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:52 smithi019 ceph-mon[30151]: pgmap v200: 97 pgs: 97 active+clean; 4.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.3 MiB/s wr, 568 op/s 2024-05-09T02:17:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.718+0000 7fcc60babb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c81f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.718+0000 7fcc60babb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7b76d00 0x55dcd7c838c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.718+0000 7fcc4b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c81f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:52.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.718+0000 7fcc4b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c81f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:35524/0 (socket says 172.21.15.19:35524) 2024-05-09T02:17:52.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.718+0000 7fcc4b7fe700 1 -- 172.21.15.19:0/3994564653 learned_addr learned my addr 172.21.15.19:0/3994564653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:52.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.718+0000 7fcc60babb80 1 --2- 172.21.15.19:0/3994564653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7c79400 0x55dcd7c6e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.718+0000 7fcc60babb80 1 -- 172.21.15.19:0/3994564653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c1acb40 con 0x55dcd7b76d00 2024-05-09T02:17:52.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.718+0000 7fcc60babb80 1 -- 172.21.15.19:0/3994564653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c1ae740 con 0x55dcd7afc4a0 2024-05-09T02:17:52.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.718+0000 7fcc60babb80 1 -- 172.21.15.19:0/3994564653 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c1ad2a0 con 0x55dcd7c79400 2024-05-09T02:17:52.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.718+0000 7fcc4b7fe700 1 -- 172.21.15.19:0/3994564653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7c79400 msgr2=0x55dcd7c6e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:52.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.719+0000 7fcc4b7fe700 1 --2- 172.21.15.19:0/3994564653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7c79400 0x55dcd7c6e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.719+0000 7fcc4b7fe700 1 -- 172.21.15.19:0/3994564653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7b76d00 msgr2=0x55dcd7c838c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:52.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.719+0000 7fcc4b7fe700 1 --2- 172.21.15.19:0/3994564653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7b76d00 0x55dcd7c838c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.719+0000 7fcc4b7fe700 1 -- 172.21.15.19:0/3994564653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcd7c75ee0 con 0x55dcd7afc4a0 2024-05-09T02:17:52.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.719+0000 7fcc4b7fe700 1 --2- 172.21.15.19:0/3994564653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c81f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009750 tx=0x7fcc3c005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc776bd516b74152 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:52.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.719+0000 7fcc317fa700 1 -- 172.21.15.19:0/3994564653 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c0052b0 con 0x55dcd7afc4a0 2024-05-09T02:17:52.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.719+0000 7fcc317fa700 1 -- 172.21.15.19:0/3994564653 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3c00bc80 con 0x55dcd7afc4a0 2024-05-09T02:17:52.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.719+0000 7fcc317fa700 1 -- 172.21.15.19:0/3994564653 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c026470 con 0x55dcd7afc4a0 2024-05-09T02:17:52.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.721+0000 7fcc60babb80 1 -- 172.21.15.19:0/3994564653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 msgr2=0x55dcd7c81f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.721+0000 7fcc60babb80 1 --2- 172.21.15.19:0/3994564653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c81f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009750 tx=0x7fcc3c005040 comp rx=0 tx=0).stop 2024-05-09T02:17:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.721+0000 7fcc60babb80 1 -- 172.21.15.19:0/3994564653 shutdown_connections 2024-05-09T02:17:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.721+0000 7fcc60babb80 1 --2- 172.21.15.19:0/3994564653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7c79400 0x55dcd7c6e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.721+0000 7fcc60babb80 1 --2- 172.21.15.19:0/3994564653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c81f80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.721+0000 7fcc60babb80 1 --2- 172.21.15.19:0/3994564653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7b76d00 0x55dcd7c838c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.721+0000 7fcc60babb80 1 -- 172.21.15.19:0/3994564653 >> 172.21.15.19:0/3994564653 conn(0x55dcd7afff70 msgr2=0x55dcd7c6dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.721+0000 7fcc60babb80 1 -- 172.21.15.19:0/3994564653 shutdown_connections 2024-05-09T02:17:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.721+0000 7fcc60babb80 1 -- 172.21.15.19:0/3994564653 wait complete. 2024-05-09T02:17:52.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.721+0000 7fcc60babb80 1 -- start start 2024-05-09T02:17:52.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.722+0000 7fcc60babb80 1 Processor -- start 2024-05-09T02:17:52.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.722+0000 7fcc60babb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c8b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.722+0000 7fcc60babb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7b76d00 0x55dcd7c8b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.722+0000 7fcc60babb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7c79400 0x55dcd7c8dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.722+0000 7fcc60babb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c1ae070 con 0x55dcd7c79400 2024-05-09T02:17:52.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.722+0000 7fcc60babb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c1ad970 con 0x55dcd7afc4a0 2024-05-09T02:17:52.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.722+0000 7fcc60babb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c1aaae0 con 0x55dcd7b76d00 2024-05-09T02:17:52.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.722+0000 7fcc60babb80 10 client.0 _ll_register_callbacks cb 0x55dcd7a0d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:17:52.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.722+0000 7fcc4bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7c79400 0x55dcd7c8dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:52.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.722+0000 7fcc4affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7b76d00 0x55dcd7c8b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:52.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.722+0000 7fcc4affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7b76d00 0x55dcd7c8b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:49100/0 (socket says 172.21.15.19:49100) 2024-05-09T02:17:52.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.722+0000 7fcc4affd700 1 -- 172.21.15.19:0/244254153 learned_addr learned my addr 172.21.15.19:0/244254153 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:52.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.722+0000 7fcc4b7fe700 1 --2- 172.21.15.19:0/244254153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c8b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:52.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.723+0000 7fcc4affd700 1 -- 172.21.15.19:0/244254153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 msgr2=0x55dcd7c8b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:52.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.723+0000 7fcc4affd700 1 --2- 172.21.15.19:0/244254153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c8b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.723+0000 7fcc4affd700 1 -- 172.21.15.19:0/244254153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7c79400 msgr2=0x55dcd7c8dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:52.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.723+0000 7fcc4affd700 1 --2- 172.21.15.19:0/244254153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7c79400 0x55dcd7c8dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.723+0000 7fcc4affd700 1 -- 172.21.15.19:0/244254153 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcd7c6dcc0 con 0x55dcd7b76d00 2024-05-09T02:17:52.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.723+0000 7fcc4bfff700 1 --2- 172.21.15.19:0/244254153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7c79400 0x55dcd7c8dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:17:52.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.723+0000 7fcc4affd700 1 --2- 172.21.15.19:0/244254153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7b76d00 0x55dcd7c8b7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000bfd0 tx=0x7fcc400a8670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b74fed3ae38310a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:52.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.723+0000 7fcc2affd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:17:52.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.723+0000 7fcc2affd700 1 -- 172.21.15.19:0/244254153 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc400bf020 con 0x55dcd7b76d00 2024-05-09T02:17:52.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.723+0000 7fcc2affd700 1 -- 172.21.15.19:0/244254153 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fcc400c33a0 con 0x55dcd7b76d00 2024-05-09T02:17:52.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.723+0000 7fcc60babb80 1 -- 172.21.15.19:0/244254153 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc3c00bc80 con 0x55dcd7b76d00 2024-05-09T02:17:52.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.723+0000 7fcc2affd700 1 -- 172.21.15.19:0/244254153 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fcc400a9420 con 0x55dcd7b76d00 2024-05-09T02:17:52.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.723+0000 7fcc60babb80 10 client.25903 fetch_fsmap learned FSMap version 47 2024-05-09T02:17:52.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.723+0000 7fcc60babb80 1 -- 172.21.15.19:0/244254153 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55dcd7c709d0 con 0x55dcd7b76d00 2024-05-09T02:17:52.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.724+0000 7fcc2affd700 1 -- 172.21.15.19:0/244254153 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3c00bc80 con 0x55dcd7b76d00 2024-05-09T02:17:52.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.724+0000 7fcc2affd700 1 -- 172.21.15.19:0/244254153 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fcc40022060 con 0x55dcd7b76d00 2024-05-09T02:17:52.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.724+0000 7fcc60babb80 10 client.25903 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T02:17:52.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.724+0000 7fcc60babb80 10 client.25903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:17:52.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.724+0000 7fcc60babb80 20 client.25903 populate_metadata read hostname 'smithi019' 2024-05-09T02:17:52.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.724+0000 7fcc60babb80 1 --2- 172.21.15.19:0/244254153 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dcd7cfd120 0x55dcd7cff540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:52.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.724+0000 7fcc60babb80 4 client.25903 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:17:52.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.724+0000 7fcc60babb80 1 -- 172.21.15.19:0/244254153 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dcd7ba3b20 con 0x55dcd7cfd120 2024-05-09T02:17:52.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.724+0000 7fcc4b7fe700 1 --2- 172.21.15.19:0/244254153 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dcd7cfd120 0x55dcd7cff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:52.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.725+0000 7fcc2affd700 10 client.25903 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:17:52.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.725+0000 7fcc4b7fe700 1 --2- 172.21.15.19:0/244254153 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dcd7cfd120 0x55dcd7cff540 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2992b41d9693d6f8 server_cookie=c9f91181d5f7b398 in_seq=0 out_seq=0 2024-05-09T02:17:52.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.725+0000 7fcc2affd700 1 -- 172.21.15.19:0/244254153 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dcd7ba3b20 con 0x55dcd7cfd120 2024-05-09T02:17:52.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.725+0000 7fcc2affd700 10 client.25903 handle_command_reply: tid=0 2024-05-09T02:17:52.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.725+0000 7fcc60babb80 1 client.25903 shutdown 2024-05-09T02:17:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.725+0000 7fcc60babb80 1 -- 172.21.15.19:0/244254153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7b76d00 msgr2=0x55dcd7c8b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.725+0000 7fcc60babb80 1 --2- 172.21.15.19:0/244254153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7b76d00 0x55dcd7c8b7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000bfd0 tx=0x7fcc400a8670 comp rx=0 tx=0).stop 2024-05-09T02:17:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.725+0000 7fcc60babb80 1 -- 172.21.15.19:0/244254153 shutdown_connections 2024-05-09T02:17:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.725+0000 7fcc60babb80 1 --2- 172.21.15.19:0/244254153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcd7c79400 0x55dcd7c8dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.725+0000 7fcc60babb80 1 --2- 172.21.15.19:0/244254153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcd7afc4a0 0x55dcd7c8b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.725+0000 7fcc60babb80 1 --2- 172.21.15.19:0/244254153 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55dcd7cfd120 0x55dcd7cff540 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.726+0000 7fcc60babb80 1 --2- 172.21.15.19:0/244254153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcd7b76d00 0x55dcd7c8b7a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.726+0000 7fcc60babb80 1 -- 172.21.15.19:0/244254153 >> 172.21.15.19:0/244254153 conn(0x55dcd7afff70 msgr2=0x55dcd7c7ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.726+0000 7fcc2affd700 0 client.25903 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:17:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.726+0000 7fcc60babb80 1 -- 172.21.15.19:0/244254153 shutdown_connections 2024-05-09T02:17:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.726+0000 7fcc60babb80 1 -- 172.21.15.19:0/244254153 wait complete. 2024-05-09T02:17:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.726+0000 7fcc60babb80 20 client.25903 trim_cache size 0 max 16384 2024-05-09T02:17:52.732 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:17:52.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.726+0000 7fcc52195700 1 -- 172.21.15.19:0/1798307855 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcc340b2a50 msgr2=0x7fcc340b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.726+0000 7fcc52195700 1 --2- 172.21.15.19:0/1798307855 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcc340b2a50 0x7fcc340b4f60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcc44020000 tx=0x7fcc44021000 comp rx=0 tx=0).stop 2024-05-09T02:17:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.726+0000 7fcc52195700 1 -- 172.21.15.19:0/1798307855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc4c107e10 msgr2=0x7fcc4c19b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.726+0000 7fcc52195700 1 --2- 172.21.15.19:0/1798307855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc4c107e10 0x7fcc4c19b510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000b500 tx=0x7fcc4000b810 comp rx=0 tx=0).stop 2024-05-09T02:17:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.727+0000 7fcc52195700 1 -- 172.21.15.19:0/1798307855 shutdown_connections 2024-05-09T02:17:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.727+0000 7fcc52195700 1 --2- 172.21.15.19:0/1798307855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc4c108760 0x7fcc4c19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.727+0000 7fcc52195700 1 --2- 172.21.15.19:0/1798307855 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcc340b2a50 0x7fcc340b4f60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.727+0000 7fcc52195700 1 --2- 172.21.15.19:0/1798307855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c1034e0 0x7fcc4c19af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.727+0000 7fcc52195700 1 --2- 172.21.15.19:0/1798307855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc4c107e10 0x7fcc4c19b510 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.727+0000 7fcc52195700 1 -- 172.21.15.19:0/1798307855 >> 172.21.15.19:0/1798307855 conn(0x7fcc4c0fef30 msgr2=0x7fcc4c107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.727+0000 7fcc52195700 1 -- 172.21.15.19:0/1798307855 shutdown_connections 2024-05-09T02:17:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:52.727+0000 7fcc52195700 1 -- 172.21.15.19:0/1798307855 wait complete. 2024-05-09T02:17:52.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:52 smithi196 ceph-mon[34718]: pgmap v200: 97 pgs: 97 active+clean; 4.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.3 MiB/s wr, 568 op/s 2024-05-09T02:17:53.734 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:53 smithi019 ceph-mon[30151]: pgmap v201: 97 pgs: 97 active+clean; 4.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.4 MiB/s wr, 572 op/s 2024-05-09T02:17:53.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:53 smithi196 ceph-mon[34718]: pgmap v201: 97 pgs: 97 active+clean; 4.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.4 MiB/s wr, 572 op/s 2024-05-09T02:17:53.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:53 smithi059 ceph-mon[34781]: pgmap v201: 97 pgs: 97 active+clean; 4.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.4 MiB/s wr, 572 op/s 2024-05-09T02:17:54.777 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T02:17:54.777 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T02:17:55.162 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:17:56.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:56 smithi019 ceph-mon[30151]: pgmap v202: 97 pgs: 97 active+clean; 4.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 7.8 MiB/s wr, 821 op/s 2024-05-09T02:17:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:56 smithi059 ceph-mon[34781]: pgmap v202: 97 pgs: 97 active+clean; 4.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 7.8 MiB/s wr, 821 op/s 2024-05-09T02:17:56.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:56 smithi196 ceph-mon[34718]: pgmap v202: 97 pgs: 97 active+clean; 4.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 7.8 MiB/s wr, 821 op/s 2024-05-09T02:17:57.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.286+0000 7ff9fc5f8700 1 -- 172.21.15.19:0/4284267383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f41086a0 msgr2=0x7ff9f4108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:57.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.286+0000 7ff9fc5f8700 1 --2- 172.21.15.19:0/4284267383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f41086a0 0x7ff9f4108b70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009990 tx=0x7ff9ec009ca0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.286+0000 7ff9fc5f8700 1 -- 172.21.15.19:0/4284267383 shutdown_connections 2024-05-09T02:17:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.286+0000 7ff9fc5f8700 1 --2- 172.21.15.19:0/4284267383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f41086a0 0x7ff9f4108b70 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.286+0000 7ff9fc5f8700 1 --2- 172.21.15.19:0/4284267383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9f4103250 0x7ff9f4103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.286+0000 7ff9fc5f8700 1 --2- 172.21.15.19:0/4284267383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9f4103ba0 0x7ff9f41080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.286+0000 7ff9fc5f8700 1 -- 172.21.15.19:0/4284267383 >> 172.21.15.19:0/4284267383 conn(0x7ff9f40fedc0 msgr2=0x7ff9f41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.287+0000 7ff9fc5f8700 1 -- 172.21.15.19:0/4284267383 shutdown_connections 2024-05-09T02:17:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.287+0000 7ff9fc5f8700 1 -- 172.21.15.19:0/4284267383 wait complete. 2024-05-09T02:17:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.287+0000 7ff9fc5f8700 1 Processor -- start 2024-05-09T02:17:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.287+0000 7ff9fc5f8700 1 -- start start 2024-05-09T02:17:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fc5f8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f4103250 0x7ff9f41a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fc5f8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9f4103ba0 0x7ff9f41a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fc5f8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9f41086a0 0x7ff9f41a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fc5f8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f411aa20 con 0x7ff9f4103250 2024-05-09T02:17:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fc5f8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f411a8a0 con 0x7ff9f4103ba0 2024-05-09T02:17:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fc5f8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f411aba0 con 0x7ff9f41086a0 2024-05-09T02:17:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fa394700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f4103250 0x7ff9f41a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fa394700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f4103250 0x7ff9f41a1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42214/0 (socket says 172.21.15.19:42214) 2024-05-09T02:17:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fa394700 1 -- 172.21.15.19:0/26090993 learned_addr learned my addr 172.21.15.19:0/26090993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9f9b93700 1 --2- 172.21.15.19:0/26090993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9f4103ba0 0x7ff9f41a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fab95700 1 --2- 172.21.15.19:0/26090993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9f41086a0 0x7ff9f41a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fa394700 1 -- 172.21.15.19:0/26090993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9f41086a0 msgr2=0x7ff9f41a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fa394700 1 --2- 172.21.15.19:0/26090993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9f41086a0 0x7ff9f41a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fa394700 1 -- 172.21.15.19:0/26090993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9f4103ba0 msgr2=0x7ff9f41a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:57.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fa394700 1 --2- 172.21.15.19:0/26090993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9f4103ba0 0x7ff9f41a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fa394700 1 -- 172.21.15.19:0/26090993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f41a6200 con 0x7ff9f4103250 2024-05-09T02:17:57.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9f9b93700 1 --2- 172.21.15.19:0/26090993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9f4103ba0 0x7ff9f41a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:17:57.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9fa394700 1 --2- 172.21.15.19:0/26090993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f4103250 0x7ff9f41a1330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000ef10 tx=0x7ff9f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:57.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9eb7fe700 1 -- 172.21.15.19:0/26090993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f000cad0 con 0x7ff9f4103250 2024-05-09T02:17:57.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9eb7fe700 1 -- 172.21.15.19:0/26090993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f0019070 con 0x7ff9f4103250 2024-05-09T02:17:57.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.288+0000 7ff9eb7fe700 1 -- 172.21.15.19:0/26090993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00146e0 con 0x7ff9f4103250 2024-05-09T02:17:57.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.289+0000 7ff9fc5f8700 1 -- 172.21.15.19:0/26090993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f41a6490 con 0x7ff9f4103250 2024-05-09T02:17:57.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.290+0000 7ff9fc5f8700 1 -- 172.21.15.19:0/26090993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f41ade10 con 0x7ff9f4103250 2024-05-09T02:17:57.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.294+0000 7ff9eb7fe700 1 -- 172.21.15.19:0/26090993 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff9f0012050 con 0x7ff9f4103250 2024-05-09T02:17:57.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.294+0000 7ff9eb7fe700 1 --2- 172.21.15.19:0/26090993 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff9e00b2a70 0x7ff9e00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.295+0000 7ff9f9b93700 1 --2- 172.21.15.19:0/26090993 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff9e00b2a70 0x7ff9e00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.295+0000 7ff9eb7fe700 1 -- 172.21.15.19:0/26090993 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7ff9f00e2660 con 0x7ff9f4103250 2024-05-09T02:17:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.295+0000 7ffa0b00eb80 1 Processor -- start 2024-05-09T02:17:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.295+0000 7ffa0b00eb80 1 -- start start 2024-05-09T02:17:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.295+0000 7ffa0b00eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565433070d80 0x5654331792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.295+0000 7ffa0b00eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565432ff6520 0x565433179800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.295+0000 7ffa0b00eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433173480 0x565433173890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.295+0000 7ff9f9b93700 1 --2- 172.21.15.19:0/26090993 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff9e00b2a70 0x7ff9e00b4f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9f41009b0 tx=0x7ff9e4009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.295+0000 7ffa0b00eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565433096120 con 0x565432ff6520 2024-05-09T02:17:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.295+0000 7ffa0b00eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56543303ba20 con 0x565433173480 2024-05-09T02:17:57.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.295+0000 7ffa0b00eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5654330920e0 con 0x565433070d80 2024-05-09T02:17:57.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.296+0000 7ff9f9b93700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565432ff6520 0x565433179800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.296+0000 7ff9f9b93700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565432ff6520 0x565433179800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42216/0 (socket says 172.21.15.19:42216) 2024-05-09T02:17:57.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.296+0000 7ff9f9b93700 1 -- 172.21.15.19:0/3894911994 learned_addr learned my addr 172.21.15.19:0/3894911994 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:57.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.296+0000 7ff9fa394700 1 --2- 172.21.15.19:0/3894911994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565433070d80 0x5654331792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.296+0000 7ff9fab95700 1 --2- 172.21.15.19:0/3894911994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433173480 0x565433173890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.296+0000 7ff9f9b93700 1 -- 172.21.15.19:0/3894911994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565433070d80 msgr2=0x5654331792c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:57.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.296+0000 7ff9f9b93700 1 --2- 172.21.15.19:0/3894911994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565433070d80 0x5654331792c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.296+0000 7ff9f9b93700 1 -- 172.21.15.19:0/3894911994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433173480 msgr2=0x565433173890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.296+0000 7ff9f9b93700 1 --2- 172.21.15.19:0/3894911994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433173480 0x565433173890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.296+0000 7ff9f9b93700 1 -- 172.21.15.19:0/3894911994 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56543302c580 con 0x565432ff6520 2024-05-09T02:17:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.296+0000 7ff9fa394700 1 --2- 172.21.15.19:0/3894911994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565433070d80 0x5654331792c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:17:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.296+0000 7ff9f9b93700 1 --2- 172.21.15.19:0/3894911994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565432ff6520 0x565433179800 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4010ec0 tx=0x7ff9e4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba7f75c122451dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:57.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.300+0000 7ff9e8ff9700 1 -- 172.21.15.19:0/3894911994 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4011db0 con 0x565432ff6520 2024-05-09T02:17:57.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.300+0000 7ff9e8ff9700 1 -- 172.21.15.19:0/3894911994 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e401e8d0 con 0x565432ff6520 2024-05-09T02:17:57.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.300+0000 7ff9e8ff9700 1 -- 172.21.15.19:0/3894911994 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e401c7d0 con 0x565432ff6520 2024-05-09T02:17:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.301+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3894911994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565432ff6520 msgr2=0x565433179800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.301+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/3894911994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565432ff6520 0x565433179800 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4010ec0 tx=0x7ff9e4013000 comp rx=0 tx=0).stop 2024-05-09T02:17:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.301+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3894911994 shutdown_connections 2024-05-09T02:17:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.301+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/3894911994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433173480 0x565433173890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.301+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/3894911994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565433070d80 0x5654331792c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.301+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/3894911994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565432ff6520 0x565433179800 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.301+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3894911994 >> 172.21.15.19:0/3894911994 conn(0x565432ff9ff0 msgr2=0x56543316e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.301+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3894911994 shutdown_connections 2024-05-09T02:17:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.301+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3894911994 wait complete. 2024-05-09T02:17:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.302+0000 7ffa0b00eb80 1 -- start start 2024-05-09T02:17:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.302+0000 7ffa0b00eb80 1 Processor -- start 2024-05-09T02:17:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.302+0000 7ffa0b00eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565432ff6520 0x56543317d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.302+0000 7ffa0b00eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433070d80 0x56543317d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.302+0000 7ffa0b00eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565433173480 0x56543317db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.302+0000 7ffa0b00eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5654331741e0 con 0x565433173480 2024-05-09T02:17:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.302+0000 7ffa0b00eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x565433030ba0 con 0x565433070d80 2024-05-09T02:17:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.302+0000 7ffa0b00eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5654331800a0 con 0x565432ff6520 2024-05-09T02:17:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.302+0000 7ffa0b00eb80 10 client.0 _ll_register_callbacks cb 0x565432f07740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:17:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.302+0000 7ff9fa394700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565432ff6520 0x56543317d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.302+0000 7ff9fab95700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565433173480 0x56543317db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.302+0000 7ff9fa394700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565432ff6520 0x56543317d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:34840/0 (socket says 172.21.15.19:34840) 2024-05-09T02:17:57.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.302+0000 7ff9fab95700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565433173480 0x56543317db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42226/0 (socket says 172.21.15.19:42226) 2024-05-09T02:17:57.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.302+0000 7ff9fa394700 1 -- 172.21.15.19:0/1843468844 learned_addr learned my addr 172.21.15.19:0/1843468844 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:57.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.303+0000 7ff9f9b93700 1 --2- 172.21.15.19:0/1843468844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433070d80 0x56543317d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.303+0000 7ff9fa394700 1 -- 172.21.15.19:0/1843468844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433070d80 msgr2=0x56543317d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:57.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.303+0000 7ff9fa394700 1 --2- 172.21.15.19:0/1843468844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433070d80 0x56543317d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.303+0000 7ff9fa394700 1 -- 172.21.15.19:0/1843468844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565433173480 msgr2=0x56543317db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:57.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.303+0000 7ff9fa394700 1 --2- 172.21.15.19:0/1843468844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565433173480 0x56543317db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.303+0000 7ff9fa394700 1 -- 172.21.15.19:0/1843468844 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565432f11510 con 0x565432ff6520 2024-05-09T02:17:57.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.303+0000 7ff9fa394700 1 --2- 172.21.15.19:0/1843468844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565432ff6520 0x56543317d0a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000eee0 tx=0x7ff9f00a9e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56cc43bd9db6ece8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:57.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.303+0000 7ff9ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:17:57.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.303+0000 7ff9ddffb700 1 -- 172.21.15.19:0/1843468844 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00c3060 con 0x565432ff6520 2024-05-09T02:17:57.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.303+0000 7ff9ddffb700 1 -- 172.21.15.19:0/1843468844 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7ff9f000cc70 con 0x565432ff6520 2024-05-09T02:17:57.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.303+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/1843468844 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9e401e8d0 con 0x565432ff6520 2024-05-09T02:17:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.304+0000 7ff9ddffb700 1 -- 172.21.15.19:0/1843468844 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff9f008fe60 con 0x565432ff6520 2024-05-09T02:17:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.304+0000 7ffa0b00eb80 10 client.25915 fetch_fsmap learned FSMap version 47 2024-05-09T02:17:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.304+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/1843468844 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56543318ab10 con 0x565432ff6520 2024-05-09T02:17:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.305+0000 7ff9ddffb700 1 -- 172.21.15.19:0/1843468844 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e401e8d0 con 0x565432ff6520 2024-05-09T02:17:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.305+0000 7ff9ddffb700 1 -- 172.21.15.19:0/1843468844 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7ff9f00a8610 con 0x565432ff6520 2024-05-09T02:17:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.305+0000 7ffa0b00eb80 10 client.25915 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T02:17:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.305+0000 7ffa0b00eb80 10 client.25915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:17:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.305+0000 7ffa0b00eb80 20 client.25915 populate_metadata read hostname 'smithi019' 2024-05-09T02:17:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.305+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/1843468844 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56543318d470 0x56543318f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.305+0000 7ffa0b00eb80 4 client.25915 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:17:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.305+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/1843468844 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654330a3b00 con 0x56543318d470 2024-05-09T02:17:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.305+0000 7ff9f9b93700 1 --2- 172.21.15.19:0/1843468844 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56543318d470 0x56543318f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.306+0000 7ff9ddffb700 10 client.25915 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:17:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.306+0000 7ff9f9b93700 1 --2- 172.21.15.19:0/1843468844 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56543318d470 0x56543318f900 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b6f25c64eacff2 server_cookie=1c5b50046c57cd8a in_seq=0 out_seq=0 2024-05-09T02:17:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.307+0000 7ff9ddffb700 1 -- 172.21.15.19:0/1843468844 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5654330a3b00 con 0x56543318d470 2024-05-09T02:17:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.307+0000 7ff9ddffb700 10 client.25915 handle_command_reply: tid=0 2024-05-09T02:17:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.307+0000 7ffa0b00eb80 1 client.25915 shutdown 2024-05-09T02:17:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.307+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/1843468844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565432ff6520 msgr2=0x56543317d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.307+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/1843468844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565432ff6520 0x56543317d0a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000eee0 tx=0x7ff9f00a9e80 comp rx=0 tx=0).stop 2024-05-09T02:17:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.307+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/1843468844 shutdown_connections 2024-05-09T02:17:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.307+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/1843468844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565433173480 0x56543317db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.307+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/1843468844 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56543318d470 0x56543318f900 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.307+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/1843468844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565432ff6520 0x56543317d0a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.307+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/1843468844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433070d80 0x56543317d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.307+0000 7ff9ddffb700 0 client.25915 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:17:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.307+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/1843468844 >> 172.21.15.19:0/1843468844 conn(0x565432ff9ff0 msgr2=0x565433168070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.307+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/1843468844 shutdown_connections 2024-05-09T02:17:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.307+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/1843468844 wait complete. 2024-05-09T02:17:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.307+0000 7ffa0b00eb80 20 client.25915 trim_cache size 0 max 16384 2024-05-09T02:17:57.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.342+0000 7ffa0b00eb80 1 Processor -- start 2024-05-09T02:17:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.346+0000 7ffa0b00eb80 1 -- start start 2024-05-09T02:17:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.346+0000 7ffa0b00eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565432ff6520 0x565433168aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.346+0000 7ffa0b00eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433070d80 0x56543317ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.346+0000 7ffa0b00eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565433173480 0x565433188d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.346+0000 7ffa0b00eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41b5540 con 0x565433173480 2024-05-09T02:17:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.346+0000 7ffa0b00eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41b7140 con 0x565433070d80 2024-05-09T02:17:57.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.346+0000 7ffa0b00eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41b5ca0 con 0x565432ff6520 2024-05-09T02:17:57.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.347+0000 7ff9f9b93700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433070d80 0x56543317ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.347+0000 7ff9f9b93700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433070d80 0x56543317ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:57798/0 (socket says 172.21.15.19:57798) 2024-05-09T02:17:57.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.347+0000 7ff9f9b93700 1 -- 172.21.15.19:0/3206291044 learned_addr learned my addr 172.21.15.19:0/3206291044 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:57.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.348+0000 7ff9fa394700 1 --2- 172.21.15.19:0/3206291044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565432ff6520 0x565433168aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.349+0000 7ff9fab95700 1 --2- 172.21.15.19:0/3206291044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565433173480 0x565433188d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.352+0000 7ff9fab95700 1 -- 172.21.15.19:0/3206291044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565432ff6520 msgr2=0x565433168aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:57.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.352+0000 7ff9fab95700 1 --2- 172.21.15.19:0/3206291044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565432ff6520 0x565433168aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.352+0000 7ff9fab95700 1 -- 172.21.15.19:0/3206291044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433070d80 msgr2=0x56543317ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:57.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.352+0000 7ff9fab95700 1 --2- 172.21.15.19:0/3206291044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433070d80 0x56543317ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.352+0000 7ff9fab95700 1 -- 172.21.15.19:0/3206291044 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56543317c070 con 0x565433173480 2024-05-09T02:17:57.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.352+0000 7ff9fab95700 1 --2- 172.21.15.19:0/3206291044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565433173480 0x565433188d40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00b2e0 tx=0x7ff9ec005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e759d24bd6972bc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:57.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.352+0000 7ff9dffff700 1 -- 172.21.15.19:0/3206291044 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec028070 con 0x565433173480 2024-05-09T02:17:57.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.352+0000 7ff9dffff700 1 -- 172.21.15.19:0/3206291044 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9ec00bbd0 con 0x565433173480 2024-05-09T02:17:57.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.352+0000 7ff9dffff700 1 -- 172.21.15.19:0/3206291044 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec023470 con 0x565433173480 2024-05-09T02:17:57.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.354+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3206291044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565433173480 msgr2=0x565433188d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:57.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.354+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/3206291044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565433173480 0x565433188d40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00b2e0 tx=0x7ff9ec005040 comp rx=0 tx=0).stop 2024-05-09T02:17:57.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.355+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3206291044 shutdown_connections 2024-05-09T02:17:57.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.355+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/3206291044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565433173480 0x565433188d40 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.355+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/3206291044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565432ff6520 0x565433168aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.355+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/3206291044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433070d80 0x56543317ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.355+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3206291044 >> 172.21.15.19:0/3206291044 conn(0x565432ff9ff0 msgr2=0x56543317e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:57.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.361+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3206291044 shutdown_connections 2024-05-09T02:17:57.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.361+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3206291044 wait complete. 2024-05-09T02:17:57.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.362+0000 7ffa0b00eb80 1 -- start start 2024-05-09T02:17:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.362+0000 7ffa0b00eb80 1 Processor -- start 2024-05-09T02:17:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.362+0000 7ffa0b00eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565432ff6520 0x565433183fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.362+0000 7ffa0b00eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565433070d80 0x565433184520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.362+0000 7ffa0b00eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433173480 0x565433184be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.362+0000 7ffa0b00eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41b6a70 con 0x565432ff6520 2024-05-09T02:17:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.362+0000 7ffa0b00eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41b6370 con 0x565433173480 2024-05-09T02:17:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.362+0000 7ffa0b00eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41b34e0 con 0x565433070d80 2024-05-09T02:17:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.362+0000 7ffa0b00eb80 10 client.0 _ll_register_callbacks cb 0x565433173e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:17:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.363+0000 7ff9fab95700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433173480 0x565433184be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.363+0000 7ff9fab95700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433173480 0x565433184be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:57806/0 (socket says 172.21.15.19:57806) 2024-05-09T02:17:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.363+0000 7ff9fab95700 1 -- 172.21.15.19:0/3906353353 learned_addr learned my addr 172.21.15.19:0/3906353353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:17:57.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.364+0000 7ff9fa394700 1 --2- 172.21.15.19:0/3906353353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565432ff6520 0x565433183fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.364+0000 7ff9fab95700 1 -- 172.21.15.19:0/3906353353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565433070d80 msgr2=0x565433184520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:17:57.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.364+0000 7ff9fab95700 1 --2- 172.21.15.19:0/3906353353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565433070d80 0x565433184520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.364+0000 7ff9fab95700 1 -- 172.21.15.19:0/3906353353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565432ff6520 msgr2=0x565433183fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:17:57.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.364+0000 7ff9fab95700 1 --2- 172.21.15.19:0/3906353353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565432ff6520 0x565433183fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.364+0000 7ff9fab95700 1 -- 172.21.15.19:0/3906353353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565433185380 con 0x565433173480 2024-05-09T02:17:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.366+0000 7ff9fab95700 1 --2- 172.21.15.19:0/3906353353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433173480 0x565433184be0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec005f20 tx=0x7ff9ec0038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7d352e6c685fc1e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:17:57.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.371+0000 7ff9ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:17:57.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.371+0000 7ff9ddffb700 1 -- 172.21.15.19:0/3906353353 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec032060 con 0x565433173480 2024-05-09T02:17:57.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.371+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3906353353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9ec00bbd0 con 0x565433173480 2024-05-09T02:17:57.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.371+0000 7ff9ddffb700 1 -- 172.21.15.19:0/3906353353 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7ff9ec00b920 con 0x565433173480 2024-05-09T02:17:57.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.372+0000 7ff9ddffb700 1 -- 172.21.15.19:0/3906353353 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff9ec028040 con 0x565433173480 2024-05-09T02:17:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.375+0000 7ffa0b00eb80 10 client.25874 fetch_fsmap learned FSMap version 47 2024-05-09T02:17:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.375+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3906353353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5654331f3e90 con 0x565433173480 2024-05-09T02:17:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.376+0000 7ff9ddffb700 1 -- 172.21.15.19:0/3906353353 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9ec00bbd0 con 0x565433173480 2024-05-09T02:17:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.378+0000 7ff9ddffb700 1 -- 172.21.15.19:0/3906353353 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7ff9ec00f460 con 0x565433173480 2024-05-09T02:17:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.378+0000 7ffa0b00eb80 10 client.25874 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T02:17:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.378+0000 7ffa0b00eb80 10 client.25874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:17:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.378+0000 7ffa0b00eb80 20 client.25874 populate_metadata read hostname 'smithi019' 2024-05-09T02:17:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.378+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/3906353353 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5654331f4450 0x5654331f6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:17:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.378+0000 7ffa0b00eb80 4 client.25874 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T02:17:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.378+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3906353353 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56543309dba0 con 0x5654331f4450 2024-05-09T02:17:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.380+0000 7ff9fa394700 1 --2- 172.21.15.19:0/3906353353 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5654331f4450 0x5654331f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:17:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.380+0000 7ff9ddffb700 10 client.25874 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:17:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.380+0000 7ff9fa394700 1 --2- 172.21.15.19:0/3906353353 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5654331f4450 0x5654331f6870 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cc45557c86f1fc9 server_cookie=8c2a5d13b1e23d74 in_seq=0 out_seq=0 2024-05-09T02:17:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.382+0000 7ff9ddffb700 1 -- 172.21.15.19:0/3906353353 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56543309dba0 con 0x5654331f4450 2024-05-09T02:17:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.382+0000 7ff9ddffb700 10 client.25874 handle_command_reply: tid=0 2024-05-09T02:17:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.382+0000 7ffa0b00eb80 1 client.25874 shutdown 2024-05-09T02:17:57.387 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:17:57.387 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T02:17:57.388 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "d259099d-158d-4bb6-b218-edd7c021115b", 2024-05-09T02:17:57.388 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T02:17:57.388 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:17:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.383+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3906353353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433173480 msgr2=0x565433184be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.383+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/3906353353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433173480 0x565433184be0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec005f20 tx=0x7ff9ec0038b0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.383+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3906353353 shutdown_connections 2024-05-09T02:17:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.383+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/3906353353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565433173480 0x565433184be0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.383+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/3906353353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565432ff6520 0x565433183fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.383+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/3906353353 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5654331f4450 0x5654331f6870 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.383+0000 7ffa0b00eb80 1 --2- 172.21.15.19:0/3906353353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565433070d80 0x565433184520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.383+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3906353353 >> 172.21.15.19:0/3906353353 conn(0x565432ff9ff0 msgr2=0x565433168800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.383+0000 7ff9ddffb700 0 client.25874 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:17:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.383+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3906353353 shutdown_connections 2024-05-09T02:17:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.383+0000 7ffa0b00eb80 1 -- 172.21.15.19:0/3906353353 wait complete. 2024-05-09T02:17:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.383+0000 7ffa0b00eb80 20 client.25874 trim_cache size 0 max 16384 2024-05-09T02:17:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.385+0000 7ff9fc5f8700 1 -- 172.21.15.19:0/26090993 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff9e00b2a70 msgr2=0x7ff9e00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.385+0000 7ff9fc5f8700 1 --2- 172.21.15.19:0/26090993 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff9e00b2a70 0x7ff9e00b4f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9f41009b0 tx=0x7ff9e4009260 comp rx=0 tx=0).stop 2024-05-09T02:17:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.385+0000 7ff9fc5f8700 1 -- 172.21.15.19:0/26090993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f4103250 msgr2=0x7ff9f41a1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:17:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.385+0000 7ff9fc5f8700 1 --2- 172.21.15.19:0/26090993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f4103250 0x7ff9f41a1330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000ef10 tx=0x7ff9f000c3a0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.386+0000 7ff9fc5f8700 1 -- 172.21.15.19:0/26090993 shutdown_connections 2024-05-09T02:17:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.386+0000 7ff9fc5f8700 1 --2- 172.21.15.19:0/26090993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9f41086a0 0x7ff9f41a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.386+0000 7ff9fc5f8700 1 --2- 172.21.15.19:0/26090993 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff9e00b2a70 0x7ff9e00b4f80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.386+0000 7ff9fc5f8700 1 --2- 172.21.15.19:0/26090993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f4103250 0x7ff9f41a1330 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.386+0000 7ff9fc5f8700 1 --2- 172.21.15.19:0/26090993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9f4103ba0 0x7ff9f41a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:17:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.386+0000 7ff9fc5f8700 1 -- 172.21.15.19:0/26090993 >> 172.21.15.19:0/26090993 conn(0x7ff9f40fedc0 msgr2=0x7ff9f4106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:17:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.386+0000 7ff9fc5f8700 1 -- 172.21.15.19:0/26090993 shutdown_connections 2024-05-09T02:17:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:17:57.386+0000 7ff9fc5f8700 1 -- 172.21.15.19:0/26090993 wait complete. 2024-05-09T02:17:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 375.0 s, 1955.0 tps, lat 51.351 ms stddev 56.742 2024-05-09T02:17:58.074 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:17:58.440 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:17:58.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: pgmap v203: 97 pgs: 97 active+clean; 4.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.4 MiB/s wr, 557 op/s 2024-05-09T02:17:58.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub queued for path: / 2024-05-09T02:17:58.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:17:58.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:17:58.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:17:58.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:17:58.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:17:58.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:17:58.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:17:58.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.574 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.574 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:17:58 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: pgmap v203: 97 pgs: 97 active+clean; 4.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.4 MiB/s wr, 557 op/s 2024-05-09T02:17:58.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub queued for path: / 2024-05-09T02:17:58.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:17:58.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:17:58.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:17:58.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:17:58.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:17:58.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:17:58.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:17:58.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:17:58 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: pgmap v203: 97 pgs: 97 active+clean; 4.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.4 MiB/s wr, 557 op/s 2024-05-09T02:17:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub queued for path: / 2024-05-09T02:17:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:17:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:17:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:17:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:17:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:17:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:17:58.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:17:58.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:17:58.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:17:58 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:18:00.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.513+0000 7fbbfae81700 1 -- 172.21.15.19:0/1287708315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf41092d0 msgr2=0x7fbbf41096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:00.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.513+0000 7fbbfae81700 1 --2- 172.21.15.19:0/1287708315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf41092d0 0x7fbbf41096e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009990 tx=0x7fbbe8009ca0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.516+0000 7fbbfae81700 1 -- 172.21.15.19:0/1287708315 shutdown_connections 2024-05-09T02:18:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.516+0000 7fbbfae81700 1 --2- 172.21.15.19:0/1287708315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbf4103230 0x7fbbf4111160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.516+0000 7fbbfae81700 1 --2- 172.21.15.19:0/1287708315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf41092d0 0x7fbbf41096e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.516+0000 7fbbfae81700 1 --2- 172.21.15.19:0/1287708315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbf41028e0 0x7fbbf4102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.516+0000 7fbbfae81700 1 -- 172.21.15.19:0/1287708315 >> 172.21.15.19:0/1287708315 conn(0x7fbbf40fe6f0 msgr2=0x7fbbf4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:00.519 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:00 smithi019 ceph-mon[30151]: pgmap v204: 97 pgs: 97 active+clean; 4.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.9 MiB/s wr, 828 op/s 2024-05-09T02:18:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.518+0000 7fbbfae81700 1 -- 172.21.15.19:0/1287708315 shutdown_connections 2024-05-09T02:18:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.522+0000 7fbbfae81700 1 -- 172.21.15.19:0/1287708315 wait complete. 2024-05-09T02:18:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.523+0000 7fbbfae81700 1 Processor -- start 2024-05-09T02:18:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.523+0000 7fbbfae81700 1 -- start start 2024-05-09T02:18:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.523+0000 7fbbfae81700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf41028e0 0x7fbbf419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.523+0000 7fbbfae81700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbf4103230 0x7fbbf419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.524+0000 7fbbfae81700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbf41092d0 0x7fbbf419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.524+0000 7fbbfae81700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf41145d0 con 0x7fbbf41028e0 2024-05-09T02:18:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.524+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbf4103230 0x7fbbf419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.524+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbf4103230 0x7fbbf419b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:34868/0 (socket says 172.21.15.19:34868) 2024-05-09T02:18:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.524+0000 7fbbf3fff700 1 -- 172.21.15.19:0/3206607600 learned_addr learned my addr 172.21.15.19:0/3206607600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:00.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.524+0000 7fbbf941e700 1 --2- 172.21.15.19:0/3206607600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbf41092d0 0x7fbbf419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:00.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.524+0000 7fbbfae81700 1 -- 172.21.15.19:0/3206607600 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf4114450 con 0x7fbbf41092d0 2024-05-09T02:18:00.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.524+0000 7fbbfae81700 1 -- 172.21.15.19:0/3206607600 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf4114750 con 0x7fbbf4103230 2024-05-09T02:18:00.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.528+0000 7fbbf3fff700 1 -- 172.21.15.19:0/3206607600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbf41092d0 msgr2=0x7fbbf419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:00.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.528+0000 7fbbf3fff700 1 --2- 172.21.15.19:0/3206607600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbf41092d0 0x7fbbf419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.528+0000 7fbbf3fff700 1 -- 172.21.15.19:0/3206607600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf41028e0 msgr2=0x7fbbf419af40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:18:00.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.528+0000 7fbbf3fff700 1 --2- 172.21.15.19:0/3206607600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf41028e0 0x7fbbf419af40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.528+0000 7fbbf3fff700 1 -- 172.21.15.19:0/3206607600 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf419fe10 con 0x7fbbf4103230 2024-05-09T02:18:00.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.528+0000 7fbbf3fff700 1 --2- 172.21.15.19:0/3206607600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbf4103230 0x7fbbf419b4e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000e970 tx=0x7fbbe000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:00.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.529+0000 7fbbf1ffb700 1 -- 172.21.15.19:0/3206607600 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0013be0 con 0x7fbbf4103230 2024-05-09T02:18:00.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.529+0000 7fbbfae81700 1 -- 172.21.15.19:0/3206607600 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf41a0100 con 0x7fbbf4103230 2024-05-09T02:18:00.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.529+0000 7fbbfae81700 1 -- 172.21.15.19:0/3206607600 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf41039f0 con 0x7fbbf4103230 2024-05-09T02:18:00.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.529+0000 7fbbf1ffb700 1 -- 172.21.15.19:0/3206607600 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe0013d80 con 0x7fbbf4103230 2024-05-09T02:18:00.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.530+0000 7fbbf1ffb700 1 -- 172.21.15.19:0/3206607600 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe000cdd0 con 0x7fbbf4103230 2024-05-09T02:18:00.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.535+0000 7fbbf1ffb700 1 -- 172.21.15.19:0/3206607600 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbbe0005020 con 0x7fbbf4103230 2024-05-09T02:18:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.537+0000 7fbc09897b80 1 Processor -- start 2024-05-09T02:18:00.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.538+0000 7fbbf1ffb700 1 --2- 172.21.15.19:0/3206607600 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fbbe40b2a70 0x7fbbe40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.539+0000 7fbbf1ffb700 1 -- 172.21.15.19:0/3206607600 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fbbe00e6770 con 0x7fbbf4103230 2024-05-09T02:18:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.539+0000 7fbc09897b80 1 -- start start 2024-05-09T02:18:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.539+0000 7fbc09897b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561d38d00 0x564561e43eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.539+0000 7fbc09897b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561cbe4a0 0x564561e44450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.539+0000 7fbc09897b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561e485b0 0x564561e489c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.539+0000 7fbc09897b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564561d5e0a0 con 0x564561d38d00 2024-05-09T02:18:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.539+0000 7fbc09897b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564561d039a0 con 0x564561cbe4a0 2024-05-09T02:18:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.539+0000 7fbc09897b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564561d5a060 con 0x564561e485b0 2024-05-09T02:18:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.540+0000 7fbbf941e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561e485b0 0x564561e489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.540+0000 7fbbf8c1d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561d38d00 0x564561e43eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.540+0000 7fbbf941e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561e485b0 0x564561e489c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:34874/0 (socket says 172.21.15.19:34874) 2024-05-09T02:18:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.540+0000 7fbbf8c1d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561d38d00 0x564561e43eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42270/0 (socket says 172.21.15.19:42270) 2024-05-09T02:18:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.540+0000 7fbbf8c1d700 1 -- 172.21.15.19:0/2844027997 learned_addr learned my addr 172.21.15.19:0/2844027997 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.542+0000 7fbbf8c1d700 1 -- 172.21.15.19:0/2844027997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561e485b0 msgr2=0x564561e489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.542+0000 7fbbf8c1d700 1 --2- 172.21.15.19:0/2844027997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561e485b0 0x564561e489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.542+0000 7fbbf8c1d700 1 -- 172.21.15.19:0/2844027997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561cbe4a0 msgr2=0x564561e44450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:18:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.542+0000 7fbbf8c1d700 1 --2- 172.21.15.19:0/2844027997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561cbe4a0 0x564561e44450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.542+0000 7fbbf8c1d700 1 -- 172.21.15.19:0/2844027997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564561cf4500 con 0x564561d38d00 2024-05-09T02:18:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.542+0000 7fbbf941e700 1 --2- 172.21.15.19:0/3206607600 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fbbe40b2a70 0x7fbbe40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.543+0000 7fbbf8c1d700 1 --2- 172.21.15.19:0/2844027997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561d38d00 0x564561e43eb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8000c00 tx=0x7fbbe800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f127865eef4679a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.543+0000 7fbbf941e700 1 --2- 172.21.15.19:0/3206607600 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fbbe40b2a70 0x7fbbe40b4f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009ec0 tx=0x7fbbec004b40 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.543+0000 7fbbdeffd700 1 -- 172.21.15.19:0/2844027997 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8026070 con 0x564561d38d00 2024-05-09T02:18:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.544+0000 7fbbdeffd700 1 -- 172.21.15.19:0/2844027997 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe8009510 con 0x564561d38d00 2024-05-09T02:18:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.544+0000 7fbbdeffd700 1 -- 172.21.15.19:0/2844027997 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe800f460 con 0x564561d38d00 2024-05-09T02:18:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.545+0000 7fbc09897b80 1 -- 172.21.15.19:0/2844027997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561d38d00 msgr2=0x564561e43eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.545+0000 7fbc09897b80 1 --2- 172.21.15.19:0/2844027997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561d38d00 0x564561e43eb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8000c00 tx=0x7fbbe800b630 comp rx=0 tx=0).stop 2024-05-09T02:18:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.545+0000 7fbc09897b80 1 -- 172.21.15.19:0/2844027997 shutdown_connections 2024-05-09T02:18:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.545+0000 7fbc09897b80 1 --2- 172.21.15.19:0/2844027997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561e485b0 0x564561e489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.545+0000 7fbc09897b80 1 --2- 172.21.15.19:0/2844027997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561d38d00 0x564561e43eb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.545+0000 7fbc09897b80 1 --2- 172.21.15.19:0/2844027997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561cbe4a0 0x564561e44450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.545+0000 7fbc09897b80 1 -- 172.21.15.19:0/2844027997 >> 172.21.15.19:0/2844027997 conn(0x564561cc1f70 msgr2=0x564561e36220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.545+0000 7fbc09897b80 1 -- 172.21.15.19:0/2844027997 shutdown_connections 2024-05-09T02:18:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.545+0000 7fbc09897b80 1 -- 172.21.15.19:0/2844027997 wait complete. 2024-05-09T02:18:00.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.545+0000 7fbc09897b80 1 -- start start 2024-05-09T02:18:00.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.545+0000 7fbc09897b80 1 Processor -- start 2024-05-09T02:18:00.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.546+0000 7fbc09897b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561cbe4a0 0x564561e4d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.546+0000 7fbc09897b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561d38d00 0x564561e4dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.546+0000 7fbc09897b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561e485b0 0x564561e4e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.546+0000 7fbc09897b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564561e49270 con 0x564561e485b0 2024-05-09T02:18:00.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.546+0000 7fbc09897b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564561cf8b20 con 0x564561d38d00 2024-05-09T02:18:00.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.546+0000 7fbc09897b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564561e493f0 con 0x564561cbe4a0 2024-05-09T02:18:00.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.546+0000 7fbc09897b80 10 client.0 _ll_register_callbacks cb 0x564561bcf6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:00.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.546+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561d38d00 0x564561e4dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:00.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.546+0000 7fbbf941e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561e485b0 0x564561e4e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:00.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.546+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561d38d00 0x564561e4dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:57832/0 (socket says 172.21.15.19:57832) 2024-05-09T02:18:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.546+0000 7fbbf3fff700 1 -- 172.21.15.19:0/915698806 learned_addr learned my addr 172.21.15.19:0/915698806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.547+0000 7fbbf941e700 1 -- 172.21.15.19:0/915698806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561cbe4a0 msgr2=0x564561e4d820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:18:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.547+0000 7fbbf941e700 1 --2- 172.21.15.19:0/915698806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561cbe4a0 0x564561e4d820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.547+0000 7fbbf941e700 1 -- 172.21.15.19:0/915698806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561d38d00 msgr2=0x564561e4dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.547+0000 7fbbf941e700 1 --2- 172.21.15.19:0/915698806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561d38d00 0x564561e4dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.547+0000 7fbbf941e700 1 -- 172.21.15.19:0/915698806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564561bd9490 con 0x564561e485b0 2024-05-09T02:18:00.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.547+0000 7fbbf941e700 1 --2- 172.21.15.19:0/915698806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561e485b0 0x564561e4e2a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbbec017e70 tx=0x7fbbec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb0430f53499ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:00.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.547+0000 7fbbd3fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:18:00.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.547+0000 7fbbd3fff700 1 -- 172.21.15.19:0/915698806 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec026020 con 0x564561e485b0 2024-05-09T02:18:00.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.547+0000 7fbbd3fff700 1 -- 172.21.15.19:0/915698806 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fbbec016bc0 con 0x564561e485b0 2024-05-09T02:18:00.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.547+0000 7fbc09897b80 1 -- 172.21.15.19:0/915698806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe8009510 con 0x564561e485b0 2024-05-09T02:18:00.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.547+0000 7fbbd3fff700 1 -- 172.21.15.19:0/915698806 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fbbec01b040 con 0x564561e485b0 2024-05-09T02:18:00.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.547+0000 7fbc09897b80 10 client.16161 fetch_fsmap learned FSMap version 47 2024-05-09T02:18:00.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.548+0000 7fbc09897b80 1 -- 172.21.15.19:0/915698806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564561e5b220 con 0x564561e485b0 2024-05-09T02:18:00.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.548+0000 7fbbd3fff700 1 -- 172.21.15.19:0/915698806 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbec007e50 con 0x564561e485b0 2024-05-09T02:18:00.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.548+0000 7fbbd3fff700 1 -- 172.21.15.19:0/915698806 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fbbec021040 con 0x564561e485b0 2024-05-09T02:18:00.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.548+0000 7fbc09897b80 10 client.16161 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T02:18:00.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.548+0000 7fbc09897b80 10 client.16161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:18:00.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.548+0000 7fbc09897b80 20 client.16161 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:00.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.549+0000 7fbc09897b80 1 --2- 172.21.15.19:0/915698806 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564561e5d860 0x564561e5fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.549+0000 7fbc09897b80 4 client.16161 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:18:00.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.549+0000 7fbc09897b80 1 -- 172.21.15.19:0/915698806 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbbe8009510 con 0x564561e5d860 2024-05-09T02:18:00.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.550+0000 7fbbf8c1d700 1 --2- 172.21.15.19:0/915698806 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564561e5d860 0x564561e5fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:00.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.551+0000 7fbbd3fff700 10 client.16161 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:00.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.551+0000 7fbbf8c1d700 1 --2- 172.21.15.19:0/915698806 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564561e5d860 0x564561e5fc80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99bc373bc0ae1495 server_cookie=9bce4423968089c5 in_seq=0 out_seq=0 2024-05-09T02:18:00.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.552+0000 7fbbd3fff700 1 -- 172.21.15.19:0/915698806 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fbbe8009510 con 0x564561e5d860 2024-05-09T02:18:00.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.552+0000 7fbbd3fff700 10 client.16161 handle_command_reply: tid=0 2024-05-09T02:18:00.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.552+0000 7fbc09897b80 1 client.16161 shutdown 2024-05-09T02:18:00.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.552+0000 7fbc09897b80 1 -- 172.21.15.19:0/915698806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561e485b0 msgr2=0x564561e4e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:00.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.552+0000 7fbc09897b80 1 --2- 172.21.15.19:0/915698806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561e485b0 0x564561e4e2a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbbec017e70 tx=0x7fbbec004ad0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.553+0000 7fbc09897b80 1 -- 172.21.15.19:0/915698806 shutdown_connections 2024-05-09T02:18:00.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.553+0000 7fbc09897b80 1 --2- 172.21.15.19:0/915698806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561e485b0 0x564561e4e2a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.553+0000 7fbc09897b80 1 --2- 172.21.15.19:0/915698806 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564561e5d860 0x564561e5fc80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.553+0000 7fbbd3fff700 0 client.16161 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.553+0000 7fbc09897b80 1 --2- 172.21.15.19:0/915698806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561cbe4a0 0x564561e4d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.553+0000 7fbc09897b80 1 --2- 172.21.15.19:0/915698806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561d38d00 0x564561e4dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.553+0000 7fbc09897b80 1 -- 172.21.15.19:0/915698806 >> 172.21.15.19:0/915698806 conn(0x564561cc1f70 msgr2=0x564561e3a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.553+0000 7fbc09897b80 1 -- 172.21.15.19:0/915698806 shutdown_connections 2024-05-09T02:18:00.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.553+0000 7fbc09897b80 1 -- 172.21.15.19:0/915698806 wait complete. 2024-05-09T02:18:00.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.553+0000 7fbc09897b80 20 client.16161 trim_cache size 0 max 16384 2024-05-09T02:18:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.584+0000 7fbc09897b80 1 Processor -- start 2024-05-09T02:18:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.584+0000 7fbc09897b80 1 -- start start 2024-05-09T02:18:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.584+0000 7fbc09897b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561cbe4a0 0x564561e30a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.584+0000 7fbc09897b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561d38d00 0x564561e59650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.584+0000 7fbc09897b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561e485b0 0x564561e653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbc09897b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd800d600 con 0x564561d38d00 2024-05-09T02:18:00.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbc09897b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd800f200 con 0x564561e485b0 2024-05-09T02:18:00.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbc09897b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd800dd60 con 0x564561cbe4a0 2024-05-09T02:18:00.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561d38d00 0x564561e59650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:00.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561d38d00 0x564561e59650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42274/0 (socket says 172.21.15.19:42274) 2024-05-09T02:18:00.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbbf3fff700 1 -- 172.21.15.19:0/2259152742 learned_addr learned my addr 172.21.15.19:0/2259152742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:00.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbbf941e700 1 --2- 172.21.15.19:0/2259152742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561e485b0 0x564561e653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:00.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbbf8c1d700 1 --2- 172.21.15.19:0/2259152742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561cbe4a0 0x564561e30a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:00.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbbf3fff700 1 -- 172.21.15.19:0/2259152742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561cbe4a0 msgr2=0x564561e30a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:00.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbbf3fff700 1 --2- 172.21.15.19:0/2259152742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561cbe4a0 0x564561e30a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbbf3fff700 1 -- 172.21.15.19:0/2259152742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561e485b0 msgr2=0x564561e653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:00.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbbf3fff700 1 --2- 172.21.15.19:0/2259152742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561e485b0 0x564561e653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbbf3fff700 1 -- 172.21.15.19:0/2259152742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564561e4e830 con 0x564561d38d00 2024-05-09T02:18:00.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbbf941e700 1 --2- 172.21.15.19:0/2259152742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561e485b0 0x564561e653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:18:00.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbbf3fff700 1 --2- 172.21.15.19:0/2259152742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561d38d00 0x564561e59650 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00a9be0 tx=0x7fbbe0009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445485f600fce7cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:00.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.585+0000 7fbbde7fc700 1 -- 172.21.15.19:0/2259152742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe00ad2d0 con 0x564561d38d00 2024-05-09T02:18:00.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.586+0000 7fbbde7fc700 1 -- 172.21.15.19:0/2259152742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe00b6300 con 0x564561d38d00 2024-05-09T02:18:00.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.586+0000 7fbbde7fc700 1 -- 172.21.15.19:0/2259152742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe00a92f0 con 0x564561d38d00 2024-05-09T02:18:00.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.587+0000 7fbc09897b80 1 -- 172.21.15.19:0/2259152742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561d38d00 msgr2=0x564561e59650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:00.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.587+0000 7fbc09897b80 1 --2- 172.21.15.19:0/2259152742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561d38d00 0x564561e59650 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00a9be0 tx=0x7fbbe0009790 comp rx=0 tx=0).stop 2024-05-09T02:18:00.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.587+0000 7fbc09897b80 1 -- 172.21.15.19:0/2259152742 shutdown_connections 2024-05-09T02:18:00.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.587+0000 7fbc09897b80 1 --2- 172.21.15.19:0/2259152742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561e485b0 0x564561e653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.587+0000 7fbc09897b80 1 --2- 172.21.15.19:0/2259152742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561cbe4a0 0x564561e30a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.587+0000 7fbc09897b80 1 --2- 172.21.15.19:0/2259152742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561d38d00 0x564561e59650 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.587+0000 7fbc09897b80 1 -- 172.21.15.19:0/2259152742 >> 172.21.15.19:0/2259152742 conn(0x564561cc1f70 msgr2=0x564561e474b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.587+0000 7fbc09897b80 1 -- 172.21.15.19:0/2259152742 shutdown_connections 2024-05-09T02:18:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.587+0000 7fbc09897b80 1 -- 172.21.15.19:0/2259152742 wait complete. 2024-05-09T02:18:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.587+0000 7fbc09897b80 1 -- start start 2024-05-09T02:18:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.588+0000 7fbc09897b80 1 Processor -- start 2024-05-09T02:18:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.588+0000 7fbc09897b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561cbe4a0 0x564561e55cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.588+0000 7fbc09897b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561d38d00 0x564561e529a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.588+0000 7fbc09897b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561e485b0 0x564561e52ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.588+0000 7fbc09897b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd800eb30 con 0x564561e485b0 2024-05-09T02:18:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.588+0000 7fbc09897b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd800e430 con 0x564561cbe4a0 2024-05-09T02:18:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.588+0000 7fbc09897b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd800b5a0 con 0x564561d38d00 2024-05-09T02:18:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.588+0000 7fbc09897b80 10 client.0 _ll_register_callbacks cb 0x564561bcf6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.588+0000 7fbbf941e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561e485b0 0x564561e52ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.588+0000 7fbbf8c1d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561cbe4a0 0x564561e55cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.588+0000 7fbbf8c1d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561cbe4a0 0x564561e55cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:57846/0 (socket says 172.21.15.19:57846) 2024-05-09T02:18:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.588+0000 7fbbf8c1d700 1 -- 172.21.15.19:0/1975244098 learned_addr learned my addr 172.21.15.19:0/1975244098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.588+0000 7fbbf941e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561e485b0 0x564561e52ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42290/0 (socket says 172.21.15.19:42290) 2024-05-09T02:18:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.588+0000 7fbbf3fff700 1 --2- 172.21.15.19:0/1975244098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561d38d00 0x564561e529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.589+0000 7fbbf8c1d700 1 -- 172.21.15.19:0/1975244098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561d38d00 msgr2=0x564561e529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.589+0000 7fbbf8c1d700 1 --2- 172.21.15.19:0/1975244098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561d38d00 0x564561e529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.589+0000 7fbbf8c1d700 1 -- 172.21.15.19:0/1975244098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561e485b0 msgr2=0x564561e52ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.589+0000 7fbbf8c1d700 1 --2- 172.21.15.19:0/1975244098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561e485b0 0x564561e52ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.589+0000 7fbbf8c1d700 1 -- 172.21.15.19:0/1975244098 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564561e329d0 con 0x564561cbe4a0 2024-05-09T02:18:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.589+0000 7fbbf8c1d700 1 --2- 172.21.15.19:0/1975244098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561cbe4a0 0x564561e55cd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800ba00 tx=0x7fbbe80219e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=791c274eea428e4f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.589+0000 7fbbd3fff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:18:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.589+0000 7fbbd3fff700 1 -- 172.21.15.19:0/1975244098 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8026040 con 0x564561cbe4a0 2024-05-09T02:18:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.589+0000 7fbbd3fff700 1 -- 172.21.15.19:0/1975244098 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8527+0+0 (secure 0 0 0) 0x7fbbe8009510 con 0x564561cbe4a0 2024-05-09T02:18:00.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.589+0000 7fbbf3fff700 1 --2- 172.21.15.19:0/1975244098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561d38d00 0x564561e529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:18:00.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.590+0000 7fbc09897b80 1 -- 172.21.15.19:0/1975244098 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564561d6ba80 con 0x564561cbe4a0 2024-05-09T02:18:00.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.590+0000 7fbbd3fff700 1 -- 172.21.15.19:0/1975244098 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fbbe8021690 con 0x564561cbe4a0 2024-05-09T02:18:00.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.591+0000 7fbc09897b80 10 client.25889 fetch_fsmap learned FSMap version 47 2024-05-09T02:18:00.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.591+0000 7fbc09897b80 1 -- 172.21.15.19:0/1975244098 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564561ec8ac0 con 0x564561cbe4a0 2024-05-09T02:18:00.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.591+0000 7fbbd3fff700 1 -- 172.21.15.19:0/1975244098 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe8021af0 con 0x564561cbe4a0 2024-05-09T02:18:00.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.591+0000 7fbbd3fff700 1 -- 172.21.15.19:0/1975244098 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fbbe802f570 con 0x564561cbe4a0 2024-05-09T02:18:00.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.591+0000 7fbc09897b80 10 client.25889 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T02:18:00.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.591+0000 7fbc09897b80 10 client.25889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:18:00.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.591+0000 7fbc09897b80 20 client.25889 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:00.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.591+0000 7fbc09897b80 1 --2- 172.21.15.19:0/1975244098 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564561e5d860 0x564561ecaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:00.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.591+0000 7fbc09897b80 4 client.25889 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:18:00.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.591+0000 7fbc09897b80 1 -- 172.21.15.19:0/1975244098 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564561d6ba80 con 0x564561e5d860 2024-05-09T02:18:00.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.591+0000 7fbbf3fff700 1 --2- 172.21.15.19:0/1975244098 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564561e5d860 0x564561ecaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:00.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.592+0000 7fbbd3fff700 10 client.25889 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.592+0000 7fbbf3fff700 1 --2- 172.21.15.19:0/1975244098 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564561e5d860 0x564561ecaca0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d91a0ce31629796 server_cookie=e59915ddb474682c in_seq=0 out_seq=0 2024-05-09T02:18:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.592+0000 7fbbd3fff700 1 -- 172.21.15.19:0/1975244098 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564561d6ba80 con 0x564561e5d860 2024-05-09T02:18:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.592+0000 7fbbd3fff700 10 client.25889 handle_command_reply: tid=0 2024-05-09T02:18:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.592+0000 7fbc09897b80 1 client.25889 shutdown 2024-05-09T02:18:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.592+0000 7fbc09897b80 1 -- 172.21.15.19:0/1975244098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561cbe4a0 msgr2=0x564561e55cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.592+0000 7fbc09897b80 1 --2- 172.21.15.19:0/1975244098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561cbe4a0 0x564561e55cd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800ba00 tx=0x7fbbe80219e0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.593+0000 7fbc09897b80 1 -- 172.21.15.19:0/1975244098 shutdown_connections 2024-05-09T02:18:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.593+0000 7fbc09897b80 1 --2- 172.21.15.19:0/1975244098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564561e485b0 0x564561e52ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.593+0000 7fbc09897b80 1 --2- 172.21.15.19:0/1975244098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564561cbe4a0 0x564561e55cd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.593+0000 7fbc09897b80 1 --2- 172.21.15.19:0/1975244098 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564561e5d860 0x564561ecaca0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.593+0000 7fbbd3fff700 0 client.25889 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.593+0000 7fbc09897b80 1 --2- 172.21.15.19:0/1975244098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564561d38d00 0x564561e529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.593+0000 7fbc09897b80 1 -- 172.21.15.19:0/1975244098 >> 172.21.15.19:0/1975244098 conn(0x564561cc1f70 msgr2=0x564561e4ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.593+0000 7fbc09897b80 1 -- 172.21.15.19:0/1975244098 shutdown_connections 2024-05-09T02:18:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.593+0000 7fbc09897b80 1 -- 172.21.15.19:0/1975244098 wait complete. 2024-05-09T02:18:00.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.593+0000 7fbc09897b80 20 client.25889 trim_cache size 0 max 16384 2024-05-09T02:18:00.602 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:18:00.602 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T02:18:00.602 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T02:18:00.602 INFO:teuthology.orchestra.run.smithi019.stdout: "d259099d-158d-4bb6-b218-edd7c021115b": { 2024-05-09T02:18:00.602 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T02:18:00.602 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "d259099d-158d-4bb6-b218-edd7c021115b", 2024-05-09T02:18:00.602 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T02:18:00.602 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:18:00.602 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:18:00.603 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:18:00.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.594+0000 7fbbfae81700 1 -- 172.21.15.19:0/3206607600 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fbbe40b2a70 msgr2=0x7fbbe40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:00.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.594+0000 7fbbfae81700 1 --2- 172.21.15.19:0/3206607600 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fbbe40b2a70 0x7fbbe40b4f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009ec0 tx=0x7fbbec004b40 comp rx=0 tx=0).stop 2024-05-09T02:18:00.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.595+0000 7fbbfae81700 1 -- 172.21.15.19:0/3206607600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbf4103230 msgr2=0x7fbbf419b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:00.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.595+0000 7fbbfae81700 1 --2- 172.21.15.19:0/3206607600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbf4103230 0x7fbbf419b4e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000e970 tx=0x7fbbe000ec80 comp rx=0 tx=0).stop 2024-05-09T02:18:00.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.595+0000 7fbbfae81700 1 -- 172.21.15.19:0/3206607600 shutdown_connections 2024-05-09T02:18:00.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.595+0000 7fbbfae81700 1 --2- 172.21.15.19:0/3206607600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbf41092d0 0x7fbbf419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.595+0000 7fbbfae81700 1 --2- 172.21.15.19:0/3206607600 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fbbe40b2a70 0x7fbbe40b4f80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.595+0000 7fbbfae81700 1 --2- 172.21.15.19:0/3206607600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf41028e0 0x7fbbf419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.595+0000 7fbbfae81700 1 --2- 172.21.15.19:0/3206607600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbf4103230 0x7fbbf419b4e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:00.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.595+0000 7fbbfae81700 1 -- 172.21.15.19:0/3206607600 >> 172.21.15.19:0/3206607600 conn(0x7fbbf40fe6f0 msgr2=0x7fbbf410a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:00.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.595+0000 7fbbfae81700 1 -- 172.21.15.19:0/3206607600 shutdown_connections 2024-05-09T02:18:00.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:00.596+0000 7fbbfae81700 1 -- 172.21.15.19:0/3206607600 wait complete. 2024-05-09T02:18:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:00 smithi059 ceph-mon[34781]: pgmap v204: 97 pgs: 97 active+clean; 4.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.9 MiB/s wr, 828 op/s 2024-05-09T02:18:00.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:00 smithi196 ceph-mon[34718]: pgmap v204: 97 pgs: 97 active+clean; 4.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.9 MiB/s wr, 828 op/s 2024-05-09T02:18:01.585 INFO:tasks.cephfs.filesystem:scrub status for tag:d259099d-158d-4bb6-b218-edd7c021115b - {'path': '/', 'tag': 'd259099d-158d-4bb6-b218-edd7c021115b', 'options': 'recursive,force'} 2024-05-09T02:18:02.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:02 smithi019 ceph-mon[30151]: pgmap v205: 97 pgs: 97 active+clean; 4.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.1 MiB/s wr, 540 op/s 2024-05-09T02:18:02.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:02 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:18:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 380.0 s, 1364.6 tps, lat 71.170 ms stddev 107.635 2024-05-09T02:18:02.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:02 smithi059 ceph-mon[34781]: pgmap v205: 97 pgs: 97 active+clean; 4.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.1 MiB/s wr, 540 op/s 2024-05-09T02:18:02.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:02 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:18:02.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:02 smithi196 ceph-mon[34718]: pgmap v205: 97 pgs: 97 active+clean; 4.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.1 MiB/s wr, 540 op/s 2024-05-09T02:18:02.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:02 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:18:04.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:04 smithi019 ceph-mon[30151]: pgmap v206: 97 pgs: 97 active+clean; 4.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.2 MiB/s wr, 544 op/s 2024-05-09T02:18:04.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:04 smithi059 ceph-mon[34781]: pgmap v206: 97 pgs: 97 active+clean; 4.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.2 MiB/s wr, 544 op/s 2024-05-09T02:18:04.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:04 smithi196 ceph-mon[34718]: pgmap v206: 97 pgs: 97 active+clean; 4.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.2 MiB/s wr, 544 op/s 2024-05-09T02:18:06.596 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:06 smithi196 ceph-mon[34718]: pgmap v207: 97 pgs: 97 active+clean; 4.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.7 MiB/s wr, 792 op/s 2024-05-09T02:18:06.596 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:06 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:18:06.596 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:06 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:18:06.596 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:06 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:18:06.597 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:06 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:18:06.597 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:06 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:18:06.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:06 smithi059 ceph-mon[34781]: pgmap v207: 97 pgs: 97 active+clean; 4.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.7 MiB/s wr, 792 op/s 2024-05-09T02:18:06.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:06 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:18:06.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:06 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:18:06.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:06 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:18:06.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:06 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:18:06.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:06 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:18:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:06 smithi019 ceph-mon[30151]: pgmap v207: 97 pgs: 97 active+clean; 4.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.7 MiB/s wr, 792 op/s 2024-05-09T02:18:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:06 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:18:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:06 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:18:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:06 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:18:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:06 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:18:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:06 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:18:07.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 385.0 s, 246.4 tps, lat 402.362 ms stddev 243.857 2024-05-09T02:18:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:07 smithi059 ceph-mon[34781]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T02:18:07.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:07 smithi196 ceph-mon[34718]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T02:18:07.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:07 smithi019 ceph-mon[30151]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T02:18:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:08 smithi059 ceph-mon[34781]: pgmap v209: 97 pgs: 97 active+clean; 4.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 631 op/s 2024-05-09T02:18:08.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:08 smithi196 ceph-mon[34718]: pgmap v209: 97 pgs: 97 active+clean; 4.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 631 op/s 2024-05-09T02:18:08.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:08 smithi019 ceph-mon[30151]: pgmap v209: 97 pgs: 97 active+clean; 4.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 631 op/s 2024-05-09T02:18:10.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:10 smithi059 ceph-mon[34781]: pgmap v210: 97 pgs: 97 active+clean; 3.8 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.1 MiB/s wr, 498 op/s 2024-05-09T02:18:10.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:10 smithi059 ceph-mon[34781]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T02:18:10.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:10 smithi196 ceph-mon[34718]: pgmap v210: 97 pgs: 97 active+clean; 3.8 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.1 MiB/s wr, 498 op/s 2024-05-09T02:18:10.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:10 smithi196 ceph-mon[34718]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T02:18:10.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:10 smithi019 ceph-mon[30151]: pgmap v210: 97 pgs: 97 active+clean; 3.8 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.1 MiB/s wr, 498 op/s 2024-05-09T02:18:10.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:10 smithi019 ceph-mon[30151]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T02:18:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 390.0 s, 597.0 tps, lat 173.634 ms stddev 335.080 2024-05-09T02:18:12.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:12 smithi059 ceph-mon[34781]: pgmap v212: 97 pgs: 97 active+clean; 3.8 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.0 MiB/s wr, 618 op/s 2024-05-09T02:18:12.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:12 smithi196 ceph-mon[34718]: pgmap v212: 97 pgs: 97 active+clean; 3.8 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.0 MiB/s wr, 618 op/s 2024-05-09T02:18:12.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:12 smithi019 ceph-mon[30151]: pgmap v212: 97 pgs: 97 active+clean; 3.8 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.0 MiB/s wr, 618 op/s 2024-05-09T02:18:14.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:14 smithi059 ceph-mon[34781]: pgmap v213: 97 pgs: 97 active+clean; 3.8 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.9 MiB/s wr, 274 op/s 2024-05-09T02:18:14.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:14 smithi196 ceph-mon[34718]: pgmap v213: 97 pgs: 97 active+clean; 3.8 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.9 MiB/s wr, 274 op/s 2024-05-09T02:18:14.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:14 smithi019 ceph-mon[30151]: pgmap v213: 97 pgs: 97 active+clean; 3.8 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.9 MiB/s wr, 274 op/s 2024-05-09T02:18:16.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:16 smithi059 ceph-mon[34781]: pgmap v214: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 MiB/s wr, 1.94k op/s 2024-05-09T02:18:16.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:16 smithi196 ceph-mon[34718]: pgmap v214: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 MiB/s wr, 1.94k op/s 2024-05-09T02:18:16.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:16 smithi019 ceph-mon[30151]: pgmap v214: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 MiB/s wr, 1.94k op/s 2024-05-09T02:18:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 395.0 s, 2415.0 tps, lat 41.138 ms stddev 44.410 2024-05-09T02:18:18.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:18 smithi059 ceph-mon[34781]: pgmap v215: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 MiB/s wr, 1.74k op/s 2024-05-09T02:18:18.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:18 smithi196 ceph-mon[34718]: pgmap v215: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 MiB/s wr, 1.74k op/s 2024-05-09T02:18:18.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:18 smithi019 ceph-mon[30151]: pgmap v215: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 MiB/s wr, 1.74k op/s 2024-05-09T02:18:20.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:20 smithi059 ceph-mon[34781]: pgmap v216: 97 pgs: 97 active+clean; 4.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 1.88k op/s 2024-05-09T02:18:20.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:20 smithi196 ceph-mon[34718]: pgmap v216: 97 pgs: 97 active+clean; 4.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 1.88k op/s 2024-05-09T02:18:20.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:20 smithi019 ceph-mon[30151]: pgmap v216: 97 pgs: 97 active+clean; 4.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 1.88k op/s 2024-05-09T02:18:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 400.0 s, 1929.2 tps, lat 51.513 ms stddev 56.551 2024-05-09T02:18:22.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:22 smithi196 ceph-mon[34718]: pgmap v217: 97 pgs: 97 active+clean; 4.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 MiB/s wr, 1.59k op/s 2024-05-09T02:18:22.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:22 smithi019 ceph-mon[30151]: pgmap v217: 97 pgs: 97 active+clean; 4.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 MiB/s wr, 1.59k op/s 2024-05-09T02:18:22.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:22 smithi059 ceph-mon[34781]: pgmap v217: 97 pgs: 97 active+clean; 4.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 MiB/s wr, 1.59k op/s 2024-05-09T02:18:24.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:24 smithi196 ceph-mon[34718]: pgmap v218: 97 pgs: 97 active+clean; 4.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 MiB/s wr, 1.57k op/s 2024-05-09T02:18:24.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:24 smithi019 ceph-mon[30151]: pgmap v218: 97 pgs: 97 active+clean; 4.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 MiB/s wr, 1.57k op/s 2024-05-09T02:18:24.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:24 smithi059 ceph-mon[34781]: pgmap v218: 97 pgs: 97 active+clean; 4.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 MiB/s wr, 1.57k op/s 2024-05-09T02:18:26.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:26 smithi196 ceph-mon[34718]: pgmap v219: 97 pgs: 97 active+clean; 4.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 MiB/s wr, 1.87k op/s 2024-05-09T02:18:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:26 smithi019 ceph-mon[30151]: pgmap v219: 97 pgs: 97 active+clean; 4.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 MiB/s wr, 1.87k op/s 2024-05-09T02:18:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:26 smithi059 ceph-mon[34781]: pgmap v219: 97 pgs: 97 active+clean; 4.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 MiB/s wr, 1.87k op/s 2024-05-09T02:18:27.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 405.0 s, 2335.6 tps, lat 43.459 ms stddev 54.387 2024-05-09T02:18:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:28 smithi196 ceph-mon[34718]: pgmap v220: 97 pgs: 97 active+clean; 4.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 606 op/s 2024-05-09T02:18:28.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:28 smithi019 ceph-mon[30151]: pgmap v220: 97 pgs: 97 active+clean; 4.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 606 op/s 2024-05-09T02:18:28.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:28 smithi059 ceph-mon[34781]: pgmap v220: 97 pgs: 97 active+clean; 4.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 606 op/s 2024-05-09T02:18:29.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:29 smithi196 ceph-mon[34718]: pgmap v221: 97 pgs: 97 active+clean; 4.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.3 MiB/s wr, 888 op/s 2024-05-09T02:18:29.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:29 smithi019 ceph-mon[30151]: pgmap v221: 97 pgs: 97 active+clean; 4.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.3 MiB/s wr, 888 op/s 2024-05-09T02:18:29.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:29 smithi059 ceph-mon[34781]: pgmap v221: 97 pgs: 97 active+clean; 4.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.3 MiB/s wr, 888 op/s 2024-05-09T02:18:31.586 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:18:31.975 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:18:32.440 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:32 smithi019 ceph-mon[30151]: pgmap v222: 97 pgs: 97 active+clean; 4.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 606 op/s 2024-05-09T02:18:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 410.0 s, 2478.8 tps, lat 40.281 ms stddev 84.205 2024-05-09T02:18:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:32 smithi059 ceph-mon[34781]: pgmap v222: 97 pgs: 97 active+clean; 4.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 606 op/s 2024-05-09T02:18:32.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:32 smithi196 ceph-mon[34718]: pgmap v222: 97 pgs: 97 active+clean; 4.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 606 op/s 2024-05-09T02:18:34.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:34 smithi019 ceph-mon[30151]: pgmap v223: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 630 op/s 2024-05-09T02:18:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.658+0000 7f4465ced700 1 -- 172.21.15.19:0/1575981995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f445809b520 msgr2=0x7f445809b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.658+0000 7f4465ced700 1 --2- 172.21.15.19:0/1575981995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f445809b520 0x7f445809b930 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4454009990 tx=0x7f4454009ca0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.659+0000 7f4465ced700 1 -- 172.21.15.19:0/1575981995 shutdown_connections 2024-05-09T02:18:34.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.659+0000 7f4465ced700 1 --2- 172.21.15.19:0/1575981995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4458095e30 0x7f44580a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.659+0000 7f4465ced700 1 --2- 172.21.15.19:0/1575981995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f445809b520 0x7f445809b930 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.659+0000 7f4465ced700 1 --2- 172.21.15.19:0/1575981995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44580954e0 0x7f44580958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.659+0000 7f4465ced700 1 -- 172.21.15.19:0/1575981995 >> 172.21.15.19:0/1575981995 conn(0x7f445800b020 msgr2=0x7f445800b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:34.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.659+0000 7f4465ced700 1 -- 172.21.15.19:0/1575981995 shutdown_connections 2024-05-09T02:18:34.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.659+0000 7f4465ced700 1 -- 172.21.15.19:0/1575981995 wait complete. 2024-05-09T02:18:34.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.659+0000 7f4465ced700 1 Processor -- start 2024-05-09T02:18:34.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.660+0000 7f4465ced700 1 -- start start 2024-05-09T02:18:34.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.660+0000 7f4465ced700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44580954e0 0x7f445812d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.660+0000 7f4465ced700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4458095e30 0x7f445812d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.660+0000 7f4465ced700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f445809b520 0x7f4458131900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.660+0000 7f4465ced700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f44580a6760 con 0x7f4458095e30 2024-05-09T02:18:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.660+0000 7f4465ced700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f44580a65e0 con 0x7f44580954e0 2024-05-09T02:18:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.660+0000 7f4465ced700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44580a68e0 con 0x7f445809b520 2024-05-09T02:18:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.661+0000 7f44654ec700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f445809b520 0x7f4458131900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.661+0000 7f44654ec700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f445809b520 0x7f4458131900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:46500/0 (socket says 172.21.15.19:46500) 2024-05-09T02:18:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.661+0000 7f44654ec700 1 -- 172.21.15.19:0/3327510930 learned_addr learned my addr 172.21.15.19:0/3327510930 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.661+0000 7f4464ceb700 1 --2- 172.21.15.19:0/3327510930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44580954e0 0x7f445812d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.662+0000 7f44654ec700 1 -- 172.21.15.19:0/3327510930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44580954e0 msgr2=0x7f445812d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.662+0000 7f44654ec700 1 --2- 172.21.15.19:0/3327510930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44580954e0 0x7f445812d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.662+0000 7f44654ec700 1 -- 172.21.15.19:0/3327510930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4458095e30 msgr2=0x7f445812d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.662+0000 7f44654ec700 1 --2- 172.21.15.19:0/3327510930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4458095e30 0x7f445812d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.662+0000 7f44654ec700 1 -- 172.21.15.19:0/3327510930 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4458131fa0 con 0x7f445809b520 2024-05-09T02:18:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.662+0000 7f4464ceb700 1 --2- 172.21.15.19:0/3327510930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44580954e0 0x7f445812d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:18:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.662+0000 7f44654ec700 1 --2- 172.21.15.19:0/3327510930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f445809b520 0x7f4458131900 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4460052d10 tx=0x7f4460075f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.663+0000 7f445dffb700 1 -- 172.21.15.19:0/3327510930 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460084070 con 0x7f445809b520 2024-05-09T02:18:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.663+0000 7f4465ced700 1 -- 172.21.15.19:0/3327510930 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4458132290 con 0x7f445809b520 2024-05-09T02:18:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.663+0000 7f445dffb700 1 -- 172.21.15.19:0/3327510930 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4460080b20 con 0x7f445809b520 2024-05-09T02:18:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.663+0000 7f445dffb700 1 -- 172.21.15.19:0/3327510930 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446007fb40 con 0x7f445809b520 2024-05-09T02:18:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.663+0000 7f4465ced700 1 -- 172.21.15.19:0/3327510930 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4458139bb0 con 0x7f445809b520 2024-05-09T02:18:34.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.670+0000 7f445dffb700 1 -- 172.21.15.19:0/3327510930 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4460087070 con 0x7f445809b520 2024-05-09T02:18:34.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.671+0000 7f445dffb700 1 --2- 172.21.15.19:0/3327510930 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f44500b2af0 0x7f44500b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.671+0000 7f445dffb700 1 -- 172.21.15.19:0/3327510930 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f4460150730 con 0x7f445809b520 2024-05-09T02:18:34.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.671+0000 7f4464ceb700 1 --2- 172.21.15.19:0/3327510930 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f44500b2af0 0x7f44500b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.671+0000 7f4464ceb700 1 --2- 172.21.15.19:0/3327510930 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f44500b2af0 0x7f44500b5000 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4454006010 tx=0x7f4454005960 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.672+0000 7f4475965b80 1 Processor -- start 2024-05-09T02:18:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.672+0000 7f4475965b80 1 -- start start 2024-05-09T02:18:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.672+0000 7f4475965b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449d93160 0x55c449e34ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.672+0000 7f4475965b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449ca94a0 0x55c449e35430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.672+0000 7f4475965b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449e26800 0x55c449e26c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.673+0000 7f4475965b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c449d48ff0 con 0x55c449e26800 2024-05-09T02:18:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.673+0000 7f4475965b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c449ce77f0 con 0x55c449ca94a0 2024-05-09T02:18:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.673+0000 7f4475965b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c449d44fb0 con 0x55c449d93160 2024-05-09T02:18:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.673+0000 7f44654ec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449e26800 0x55c449e26c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.673+0000 7f44654ec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449e26800 0x55c449e26c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34762/0 (socket says 172.21.15.19:34762) 2024-05-09T02:18:34.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.673+0000 7f44654ec700 1 -- 172.21.15.19:0/1614151960 learned_addr learned my addr 172.21.15.19:0/1614151960 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.674+0000 7f445ffff700 1 --2- 172.21.15.19:0/1614151960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449ca94a0 0x55c449e35430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.674+0000 7f44654ec700 1 -- 172.21.15.19:0/1614151960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449d93160 msgr2=0x55c449e34ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:18:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.674+0000 7f44654ec700 1 --2- 172.21.15.19:0/1614151960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449d93160 0x55c449e34ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.674+0000 7f44654ec700 1 -- 172.21.15.19:0/1614151960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449ca94a0 msgr2=0x55c449e35430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.674+0000 7f44654ec700 1 --2- 172.21.15.19:0/1614151960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449ca94a0 0x55c449e35430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.674+0000 7f44654ec700 1 -- 172.21.15.19:0/1614151960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c449cdf370 con 0x55c449e26800 2024-05-09T02:18:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.674+0000 7f445ffff700 1 --2- 172.21.15.19:0/1614151960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449ca94a0 0x55c449e35430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:18:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.674+0000 7f44654ec700 1 --2- 172.21.15.19:0/1614151960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449e26800 0x55c449e26c10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4460114bf0 tx=0x7f44600697a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f95b3901c4e9ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.675+0000 7f444affd700 1 -- 172.21.15.19:0/1614151960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446008e070 con 0x55c449e26800 2024-05-09T02:18:34.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.675+0000 7f444affd700 1 -- 172.21.15.19:0/1614151960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4460117980 con 0x55c449e26800 2024-05-09T02:18:34.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.675+0000 7f444affd700 1 -- 172.21.15.19:0/1614151960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446007fea0 con 0x55c449e26800 2024-05-09T02:18:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.676+0000 7f4475965b80 1 -- 172.21.15.19:0/1614151960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449e26800 msgr2=0x55c449e26c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.676+0000 7f4475965b80 1 --2- 172.21.15.19:0/1614151960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449e26800 0x55c449e26c10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4460114bf0 tx=0x7f44600697a0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.676+0000 7f4475965b80 1 -- 172.21.15.19:0/1614151960 shutdown_connections 2024-05-09T02:18:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.676+0000 7f4475965b80 1 --2- 172.21.15.19:0/1614151960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449e26800 0x55c449e26c10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.676+0000 7f4475965b80 1 --2- 172.21.15.19:0/1614151960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449d93160 0x55c449e34ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.676+0000 7f4475965b80 1 --2- 172.21.15.19:0/1614151960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449ca94a0 0x55c449e35430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.676+0000 7f4475965b80 1 -- 172.21.15.19:0/1614151960 >> 172.21.15.19:0/1614151960 conn(0x55c449d096a0 msgr2=0x55c449e21880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.676+0000 7f4475965b80 1 -- 172.21.15.19:0/1614151960 shutdown_connections 2024-05-09T02:18:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.676+0000 7f4475965b80 1 -- 172.21.15.19:0/1614151960 wait complete. 2024-05-09T02:18:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.677+0000 7f4475965b80 1 -- start start 2024-05-09T02:18:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.677+0000 7f4475965b80 1 Processor -- start 2024-05-09T02:18:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.678+0000 7f4475965b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449ca94a0 0x55c449e38cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.678+0000 7f4475965b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449d93160 0x55c449e39230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.678+0000 7f4475965b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449e26800 0x55c449e39770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.678+0000 7f4475965b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c449bc4ba0 con 0x55c449ca94a0 2024-05-09T02:18:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.678+0000 7f4475965b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c449d03150 con 0x55c449d93160 2024-05-09T02:18:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.678+0000 7f4475965b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c449e27630 con 0x55c449e26800 2024-05-09T02:18:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.678+0000 7f4475965b80 10 client.0 _ll_register_callbacks cb 0x55c449bba6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.678+0000 7f445ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449d93160 0x55c449e39230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.678+0000 7f445ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449d93160 0x55c449e39230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56324/0 (socket says 172.21.15.19:56324) 2024-05-09T02:18:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.678+0000 7f445ffff700 1 -- 172.21.15.19:0/3212390395 learned_addr learned my addr 172.21.15.19:0/3212390395 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.678+0000 7f44654ec700 1 --2- 172.21.15.19:0/3212390395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449e26800 0x55c449e39770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.679+0000 7f445ffff700 1 -- 172.21.15.19:0/3212390395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449e26800 msgr2=0x55c449e39770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.679+0000 7f445ffff700 1 --2- 172.21.15.19:0/3212390395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449e26800 0x55c449e39770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.679+0000 7f445ffff700 1 -- 172.21.15.19:0/3212390395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449ca94a0 msgr2=0x55c449e38cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.679+0000 7f445ffff700 1 --2- 172.21.15.19:0/3212390395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449ca94a0 0x55c449e38cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.679+0000 7f445ffff700 1 -- 172.21.15.19:0/3212390395 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c449bc4490 con 0x55c449d93160 2024-05-09T02:18:34.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.679+0000 7f443ffff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:18:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.679+0000 7f445ffff700 1 --2- 172.21.15.19:0/3212390395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449d93160 0x55c449e39230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f444c011660 tx=0x7f444c011970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d66a8b21000186f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.679+0000 7f443ffff700 1 -- 172.21.15.19:0/3212390395 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c004220 con 0x55c449d93160 2024-05-09T02:18:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.679+0000 7f4475965b80 1 -- 172.21.15.19:0/3212390395 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4460117980 con 0x55c449d93160 2024-05-09T02:18:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.680+0000 7f443ffff700 1 -- 172.21.15.19:0/3212390395 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f444c0043c0 con 0x55c449d93160 2024-05-09T02:18:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.680+0000 7f443ffff700 1 -- 172.21.15.19:0/3212390395 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f444c003680 con 0x55c449d93160 2024-05-09T02:18:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.680+0000 7f4475965b80 10 client.25901 fetch_fsmap learned FSMap version 48 2024-05-09T02:18:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.680+0000 7f4475965b80 1 -- 172.21.15.19:0/3212390395 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c449e576e0 con 0x55c449d93160 2024-05-09T02:18:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.680+0000 7f443ffff700 1 -- 172.21.15.19:0/3212390395 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f444c003cd0 con 0x55c449d93160 2024-05-09T02:18:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.680+0000 7f443ffff700 1 -- 172.21.15.19:0/3212390395 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f444c012c30 con 0x55c449d93160 2024-05-09T02:18:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.680+0000 7f4475965b80 10 client.25901 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:18:34.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.680+0000 7f4475965b80 10 client.25901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:18:34.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.680+0000 7f4475965b80 20 client.25901 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:34.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.680+0000 7f4475965b80 1 --2- 172.21.15.19:0/3212390395 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55c449e59d70 0x55c449e5c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.681+0000 7f4475965b80 4 client.25901 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:18:34.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.681+0000 7f4475965b80 1 -- 172.21.15.19:0/3212390395 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4460117980 con 0x55c449e59d70 2024-05-09T02:18:34.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.681+0000 7f4464ceb700 1 --2- 172.21.15.19:0/3212390395 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55c449e59d70 0x55c449e5c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:34.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.681+0000 7f443ffff700 10 client.25901 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:34.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.681+0000 7f4464ceb700 1 --2- 172.21.15.19:0/3212390395 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55c449e59d70 0x55c449e5c1b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e1b9ff9d6da9365 server_cookie=ec3958b7a0fd4ffc in_seq=0 out_seq=0 2024-05-09T02:18:34.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.682+0000 7f443ffff700 1 -- 172.21.15.19:0/3212390395 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f4460117980 con 0x55c449e59d70 2024-05-09T02:18:34.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.682+0000 7f443ffff700 10 client.25901 handle_command_reply: tid=0 2024-05-09T02:18:34.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.682+0000 7f4475965b80 1 client.25901 shutdown 2024-05-09T02:18:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.683+0000 7f4475965b80 1 -- 172.21.15.19:0/3212390395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449d93160 msgr2=0x55c449e39230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.683+0000 7f4475965b80 1 --2- 172.21.15.19:0/3212390395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449d93160 0x55c449e39230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f444c011660 tx=0x7f444c011970 comp rx=0 tx=0).stop 2024-05-09T02:18:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.683+0000 7f4475965b80 1 -- 172.21.15.19:0/3212390395 shutdown_connections 2024-05-09T02:18:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.683+0000 7f4475965b80 1 --2- 172.21.15.19:0/3212390395 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55c449e59d70 0x55c449e5c1b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.683+0000 7f4475965b80 1 --2- 172.21.15.19:0/3212390395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449e26800 0x55c449e39770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.683+0000 7f4475965b80 1 --2- 172.21.15.19:0/3212390395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449ca94a0 0x55c449e38cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.683+0000 7f4475965b80 1 --2- 172.21.15.19:0/3212390395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449d93160 0x55c449e39230 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.683+0000 7f4475965b80 1 -- 172.21.15.19:0/3212390395 >> 172.21.15.19:0/3212390395 conn(0x55c449d096a0 msgr2=0x55c449e34ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.684+0000 7f443ffff700 0 client.25901 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.684+0000 7f4475965b80 1 -- 172.21.15.19:0/3212390395 shutdown_connections 2024-05-09T02:18:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.684+0000 7f4475965b80 1 -- 172.21.15.19:0/3212390395 wait complete. 2024-05-09T02:18:34.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.684+0000 7f4475965b80 20 client.25901 trim_cache size 0 max 16384 2024-05-09T02:18:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.713+0000 7f4475965b80 1 Processor -- start 2024-05-09T02:18:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.713+0000 7f4475965b80 1 -- start start 2024-05-09T02:18:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.713+0000 7f4475965b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449ca94a0 0x55c449e21a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.713+0000 7f4475965b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449d93160 0x55c449e395b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.713+0000 7f4475965b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449e26800 0x55c449e33010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.713+0000 7f4475965b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f445813f0b0 con 0x55c449ca94a0 2024-05-09T02:18:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.713+0000 7f4475965b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4458140c80 con 0x55c449d93160 2024-05-09T02:18:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.713+0000 7f4475965b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f445813f810 con 0x55c449e26800 2024-05-09T02:18:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.713+0000 7f44654ec700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449e26800 0x55c449e33010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.713+0000 7f44654ec700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449e26800 0x55c449e33010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:46524/0 (socket says 172.21.15.19:46524) 2024-05-09T02:18:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.713+0000 7f44654ec700 1 -- 172.21.15.19:0/779410274 learned_addr learned my addr 172.21.15.19:0/779410274 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.713+0000 7f4464ceb700 1 --2- 172.21.15.19:0/779410274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449ca94a0 0x55c449e21a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:34.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.713+0000 7f445ffff700 1 --2- 172.21.15.19:0/779410274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449d93160 0x55c449e395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:34.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.714+0000 7f44654ec700 1 -- 172.21.15.19:0/779410274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449d93160 msgr2=0x55c449e395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:34.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.714+0000 7f44654ec700 1 --2- 172.21.15.19:0/779410274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449d93160 0x55c449e395b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.714+0000 7f44654ec700 1 -- 172.21.15.19:0/779410274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449ca94a0 msgr2=0x55c449e21a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:34.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.714+0000 7f44654ec700 1 --2- 172.21.15.19:0/779410274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449ca94a0 0x55c449e21a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.714+0000 7f44654ec700 1 -- 172.21.15.19:0/779410274 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c449e232c0 con 0x55c449e26800 2024-05-09T02:18:34.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.714+0000 7f4464ceb700 1 --2- 172.21.15.19:0/779410274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449ca94a0 0x55c449e21a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:18:34.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.714+0000 7f44654ec700 1 --2- 172.21.15.19:0/779410274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449e26800 0x55c449e33010 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4460000eb0 tx=0x7f4460080f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=858624853d78f1f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:34.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.715+0000 7f444a7fc700 1 -- 172.21.15.19:0/779410274 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446008e040 con 0x55c449e26800 2024-05-09T02:18:34.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.715+0000 7f444a7fc700 1 -- 172.21.15.19:0/779410274 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4460130070 con 0x55c449e26800 2024-05-09T02:18:34.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.715+0000 7f444a7fc700 1 -- 172.21.15.19:0/779410274 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446013a0d0 con 0x55c449e26800 2024-05-09T02:18:34.720 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:34 smithi059 ceph-mon[34781]: pgmap v223: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 630 op/s 2024-05-09T02:18:34.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.716+0000 7f4475965b80 1 -- 172.21.15.19:0/779410274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449e26800 msgr2=0x55c449e33010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:34.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.716+0000 7f4475965b80 1 --2- 172.21.15.19:0/779410274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449e26800 0x55c449e33010 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4460000eb0 tx=0x7f4460080f60 comp rx=0 tx=0).stop 2024-05-09T02:18:34.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.716+0000 7f4475965b80 1 -- 172.21.15.19:0/779410274 shutdown_connections 2024-05-09T02:18:34.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.716+0000 7f4475965b80 1 --2- 172.21.15.19:0/779410274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449e26800 0x55c449e33010 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.716+0000 7f4475965b80 1 --2- 172.21.15.19:0/779410274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449ca94a0 0x55c449e21a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.716+0000 7f4475965b80 1 --2- 172.21.15.19:0/779410274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449d93160 0x55c449e395b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.716+0000 7f4475965b80 1 -- 172.21.15.19:0/779410274 >> 172.21.15.19:0/779410274 conn(0x55c449d096a0 msgr2=0x55c449e32900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.717+0000 7f4475965b80 1 -- 172.21.15.19:0/779410274 shutdown_connections 2024-05-09T02:18:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.717+0000 7f4475965b80 1 -- 172.21.15.19:0/779410274 wait complete. 2024-05-09T02:18:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.717+0000 7f4475965b80 1 -- start start 2024-05-09T02:18:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.717+0000 7f4475965b80 1 Processor -- start 2024-05-09T02:18:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.718+0000 7f4475965b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449ca94a0 0x55c449e41010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.718+0000 7f4475965b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449d93160 0x55c449e3a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.718+0000 7f4475965b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449e26800 0x55c449e3a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.718+0000 7f4475965b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f44581405b0 con 0x55c449e26800 2024-05-09T02:18:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.718+0000 7f4475965b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f445813fee0 con 0x55c449ca94a0 2024-05-09T02:18:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.718+0000 7f4475965b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f445813d090 con 0x55c449d93160 2024-05-09T02:18:34.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.719+0000 7f4475965b80 10 client.0 _ll_register_callbacks cb 0x55c449bba6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:34.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.719+0000 7f4464ceb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449ca94a0 0x55c449e41010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:34.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.719+0000 7f4464ceb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449ca94a0 0x55c449e41010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56336/0 (socket says 172.21.15.19:56336) 2024-05-09T02:18:34.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.719+0000 7f4464ceb700 1 -- 172.21.15.19:0/3070606387 learned_addr learned my addr 172.21.15.19:0/3070606387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:34.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.719+0000 7f4464ceb700 1 -- 172.21.15.19:0/3070606387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449d93160 msgr2=0x55c449e3a1f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:18:34.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.719+0000 7f4464ceb700 1 --2- 172.21.15.19:0/3070606387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449d93160 0x55c449e3a1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.719+0000 7f4464ceb700 1 -- 172.21.15.19:0/3070606387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449e26800 msgr2=0x55c449e3a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:34.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.719+0000 7f4464ceb700 1 --2- 172.21.15.19:0/3070606387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449e26800 0x55c449e3a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.719+0000 7f4464ceb700 1 -- 172.21.15.19:0/3070606387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c449e3ac70 con 0x55c449ca94a0 2024-05-09T02:18:34.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.720+0000 7f4464ceb700 1 --2- 172.21.15.19:0/3070606387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449ca94a0 0x55c449e41010 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4454009990 tx=0x7f4454009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3054aa034b79a463 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:34.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.720+0000 7f443ffff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:18:34.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.720+0000 7f443ffff700 1 -- 172.21.15.19:0/3070606387 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454004fa0 con 0x55c449ca94a0 2024-05-09T02:18:34.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.720+0000 7f443ffff700 1 -- 172.21.15.19:0/3070606387 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f445402e2c0 con 0x55c449ca94a0 2024-05-09T02:18:34.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.720+0000 7f4475965b80 1 -- 172.21.15.19:0/3070606387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4460130070 con 0x55c449ca94a0 2024-05-09T02:18:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.721+0000 7f443ffff700 1 -- 172.21.15.19:0/3070606387 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f445403e050 con 0x55c449ca94a0 2024-05-09T02:18:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.721+0000 7f4475965b80 10 client.25907 fetch_fsmap learned FSMap version 48 2024-05-09T02:18:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.721+0000 7f4475965b80 1 -- 172.21.15.19:0/3070606387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c449ec4280 con 0x55c449ca94a0 2024-05-09T02:18:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.721+0000 7f443ffff700 1 -- 172.21.15.19:0/3070606387 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4454032070 con 0x55c449ca94a0 2024-05-09T02:18:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.722+0000 7f443ffff700 1 -- 172.21.15.19:0/3070606387 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f4454022070 con 0x55c449ca94a0 2024-05-09T02:18:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.722+0000 7f4475965b80 10 client.25907 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:18:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.722+0000 7f4475965b80 10 client.25907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:18:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.722+0000 7f4475965b80 20 client.25907 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.722+0000 7f4475965b80 1 --2- 172.21.15.19:0/3070606387 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55c449ec4920 0x55c449ec6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.722+0000 7f4475965b80 4 client.25907 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:18:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.722+0000 7f4475965b80 1 -- 172.21.15.19:0/3070606387 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f4460130070 con 0x55c449ec4920 2024-05-09T02:18:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.722+0000 7f445ffff700 1 --2- 172.21.15.19:0/3070606387 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55c449ec4920 0x55c449ec6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.723+0000 7f443ffff700 10 client.25907 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.723+0000 7f445ffff700 1 --2- 172.21.15.19:0/3070606387 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55c449ec4920 0x55c449ec6d80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb27a68603b0159c server_cookie=e19562b626b56423 in_seq=0 out_seq=0 2024-05-09T02:18:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.723+0000 7f443ffff700 1 -- 172.21.15.19:0/3070606387 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f4460130070 con 0x55c449ec4920 2024-05-09T02:18:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.723+0000 7f443ffff700 10 client.25907 handle_command_reply: tid=0 2024-05-09T02:18:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.723+0000 7f4475965b80 1 client.25907 shutdown 2024-05-09T02:18:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.723+0000 7f4475965b80 1 -- 172.21.15.19:0/3070606387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449ca94a0 msgr2=0x55c449e41010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.723+0000 7f4475965b80 1 --2- 172.21.15.19:0/3070606387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449ca94a0 0x55c449e41010 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4454009990 tx=0x7f4454009310 comp rx=0 tx=0).stop 2024-05-09T02:18:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.723+0000 7f4475965b80 1 -- 172.21.15.19:0/3070606387 shutdown_connections 2024-05-09T02:18:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.724+0000 7f4475965b80 1 --2- 172.21.15.19:0/3070606387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c449e26800 0x55c449e3a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.724+0000 7f4475965b80 1 --2- 172.21.15.19:0/3070606387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c449ca94a0 0x55c449e41010 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.724+0000 7f4475965b80 1 --2- 172.21.15.19:0/3070606387 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55c449ec4920 0x55c449ec6d80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.724+0000 7f4475965b80 1 --2- 172.21.15.19:0/3070606387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c449d93160 0x55c449e3a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.724+0000 7f4475965b80 1 -- 172.21.15.19:0/3070606387 >> 172.21.15.19:0/3070606387 conn(0x55c449d096a0 msgr2=0x55c449e25f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.725+0000 7f443ffff700 0 client.25907 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.725+0000 7f4475965b80 1 -- 172.21.15.19:0/3070606387 shutdown_connections 2024-05-09T02:18:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.725+0000 7f4475965b80 1 -- 172.21.15.19:0/3070606387 wait complete. 2024-05-09T02:18:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.725+0000 7f4475965b80 20 client.25907 trim_cache size 0 max 16384 2024-05-09T02:18:34.732 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:18:34.732 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T02:18:34.732 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T02:18:34.732 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:18:34.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.726+0000 7f4465ced700 1 -- 172.21.15.19:0/3327510930 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f44500b2af0 msgr2=0x7f44500b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:34.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.726+0000 7f4465ced700 1 --2- 172.21.15.19:0/3327510930 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f44500b2af0 0x7f44500b5000 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4454006010 tx=0x7f4454005960 comp rx=0 tx=0).stop 2024-05-09T02:18:34.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.726+0000 7f4465ced700 1 -- 172.21.15.19:0/3327510930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f445809b520 msgr2=0x7f4458131900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:34.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.726+0000 7f4465ced700 1 --2- 172.21.15.19:0/3327510930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f445809b520 0x7f4458131900 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4460052d10 tx=0x7f4460075f00 comp rx=0 tx=0).stop 2024-05-09T02:18:34.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.726+0000 7f4465ced700 1 -- 172.21.15.19:0/3327510930 shutdown_connections 2024-05-09T02:18:34.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.726+0000 7f4465ced700 1 --2- 172.21.15.19:0/3327510930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f445809b520 0x7f4458131900 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.726+0000 7f4465ced700 1 --2- 172.21.15.19:0/3327510930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44580954e0 0x7f445812d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.726+0000 7f4465ced700 1 --2- 172.21.15.19:0/3327510930 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f44500b2af0 0x7f44500b5000 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.726+0000 7f4465ced700 1 --2- 172.21.15.19:0/3327510930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4458095e30 0x7f445812d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:34.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.726+0000 7f4465ced700 1 -- 172.21.15.19:0/3327510930 >> 172.21.15.19:0/3327510930 conn(0x7f445800b020 msgr2=0x7f4458093c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:34.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.727+0000 7f4465ced700 1 -- 172.21.15.19:0/3327510930 shutdown_connections 2024-05-09T02:18:34.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:34.727+0000 7f4465ced700 1 -- 172.21.15.19:0/3327510930 wait complete. 2024-05-09T02:18:34.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:34 smithi196 ceph-mon[34718]: pgmap v223: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 630 op/s 2024-05-09T02:18:35.608 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T02:18:35.608 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:18:36.014 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:18:36.409 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:36 smithi019 ceph-mon[30151]: pgmap v224: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 2k op/s 2024-05-09T02:18:36.596 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:36 smithi196 ceph-mon[34718]: pgmap v224: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 2k op/s 2024-05-09T02:18:36.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:36 smithi059 ceph-mon[34781]: pgmap v224: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 2k op/s 2024-05-09T02:18:37.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 415.0 s, 2031.6 tps, lat 49.107 ms stddev 60.817 2024-05-09T02:18:38.411 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:38 smithi019 ceph-mon[30151]: pgmap v225: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 1.68k op/s 2024-05-09T02:18:38.411 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:38 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:18:38.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.490+0000 7fcf689a1700 1 -- 172.21.15.19:0/3500167357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf64103250 msgr2=0x7fcf64103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:38.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.490+0000 7fcf689a1700 1 --2- 172.21.15.19:0/3500167357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf64103250 0x7fcf64103660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0098f0 tx=0x7fcf4c009c00 comp rx=0 tx=0).stop 2024-05-09T02:18:38.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.491+0000 7fcf689a1700 1 -- 172.21.15.19:0/3500167357 shutdown_connections 2024-05-09T02:18:38.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.491+0000 7fcf689a1700 1 --2- 172.21.15.19:0/3500167357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf641086a0 0x7fcf64108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:38.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.491+0000 7fcf689a1700 1 --2- 172.21.15.19:0/3500167357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf64103250 0x7fcf64103660 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0098f0 tx=0x7fcf4c009c00 comp rx=0 tx=0).stop 2024-05-09T02:18:38.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.491+0000 7fcf689a1700 1 --2- 172.21.15.19:0/3500167357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf64103ba0 0x7fcf641080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:38.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.491+0000 7fcf689a1700 1 -- 172.21.15.19:0/3500167357 >> 172.21.15.19:0/3500167357 conn(0x7fcf640fedc0 msgr2=0x7fcf641011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:38.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.491+0000 7fcf689a1700 1 -- 172.21.15.19:0/3500167357 shutdown_connections 2024-05-09T02:18:38.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.491+0000 7fcf689a1700 1 -- 172.21.15.19:0/3500167357 wait complete. 2024-05-09T02:18:38.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.492+0000 7fcf689a1700 1 Processor -- start 2024-05-09T02:18:38.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.492+0000 7fcf689a1700 1 -- start start 2024-05-09T02:18:38.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.493+0000 7fcf689a1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf64103ba0 0x7fcf64113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:38.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.493+0000 7fcf689a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf641086a0 0x7fcf64114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:38.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.493+0000 7fcf689a1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf64117a60 0x7fcf641a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:38.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.493+0000 7fcf689a1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6410be70 con 0x7fcf641086a0 2024-05-09T02:18:38.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.493+0000 7fcf689a1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6410bcf0 con 0x7fcf64103ba0 2024-05-09T02:18:38.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.493+0000 7fcf689a1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6410bff0 con 0x7fcf64117a60 2024-05-09T02:18:38.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.493+0000 7fcf62d9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf64117a60 0x7fcf641a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:38.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.493+0000 7fcf62d9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf64117a60 0x7fcf641a5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:53746/0 (socket says 172.21.15.19:53746) 2024-05-09T02:18:38.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.493+0000 7fcf62d9d700 1 -- 172.21.15.19:0/1073960014 learned_addr learned my addr 172.21.15.19:0/1073960014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.494+0000 7fcf62d9d700 1 -- 172.21.15.19:0/1073960014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf64103ba0 msgr2=0x7fcf64113ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.494+0000 7fcf62d9d700 1 --2- 172.21.15.19:0/1073960014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf64103ba0 0x7fcf64113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.494+0000 7fcf62d9d700 1 -- 172.21.15.19:0/1073960014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf641086a0 msgr2=0x7fcf64114020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.494+0000 7fcf62d9d700 1 --2- 172.21.15.19:0/1073960014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf641086a0 0x7fcf64114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.494+0000 7fcf62d9d700 1 -- 172.21.15.19:0/1073960014 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf641148b0 con 0x7fcf64117a60 2024-05-09T02:18:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.494+0000 7fcf62d9d700 1 --2- 172.21.15.19:0/1073960014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf64117a60 0x7fcf641a5080 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00e940 tx=0x7fcf5c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.495+0000 7fcf5b7fe700 1 -- 172.21.15.19:0/1073960014 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c00cbd0 con 0x7fcf64117a60 2024-05-09T02:18:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.495+0000 7fcf5b7fe700 1 -- 172.21.15.19:0/1073960014 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf5c00cd70 con 0x7fcf64117a60 2024-05-09T02:18:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.495+0000 7fcf5b7fe700 1 -- 172.21.15.19:0/1073960014 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c010430 con 0x7fcf64117a60 2024-05-09T02:18:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.495+0000 7fcf689a1700 1 -- 172.21.15.19:0/1073960014 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf641a5680 con 0x7fcf64117a60 2024-05-09T02:18:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.496+0000 7fcf689a1700 1 -- 172.21.15.19:0/1073960014 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf641a5bc0 con 0x7fcf64117a60 2024-05-09T02:18:38.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.497+0000 7fcf689a1700 1 -- 172.21.15.19:0/1073960014 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf6406bef0 con 0x7fcf64117a60 2024-05-09T02:18:38.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.499+0000 7fcf5b7fe700 1 -- 172.21.15.19:0/1073960014 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcf5c004750 con 0x7fcf64117a60 2024-05-09T02:18:38.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.500+0000 7fcf5b7fe700 1 --2- 172.21.15.19:0/1073960014 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcf500b2b40 0x7fcf500b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:38.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.501+0000 7fcf5b7fe700 1 -- 172.21.15.19:0/1073960014 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fcf5c0e2880 con 0x7fcf64117a60 2024-05-09T02:18:38.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.502+0000 7fcf6259c700 1 --2- 172.21.15.19:0/1073960014 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcf500b2b40 0x7fcf500b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:38.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.502+0000 7fcf6259c700 1 --2- 172.21.15.19:0/1073960014 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcf500b2b40 0x7fcf500b5050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009680 tx=0x7fcf4c009f50 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.506+0000 7fcf5b7fe700 1 -- 172.21.15.19:0/1073960014 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf5c0aa1d0 con 0x7fcf64117a60 2024-05-09T02:18:38.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:38 smithi059 ceph-mon[34781]: pgmap v225: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 1.68k op/s 2024-05-09T02:18:38.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:38 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:18:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:38 smithi196 ceph-mon[34718]: pgmap v225: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 1.68k op/s 2024-05-09T02:18:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:38 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:18:38.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.914+0000 7fcf689a1700 1 -- 172.21.15.19:0/1073960014 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf641a5ee0 con 0x7fcf64117a60 2024-05-09T02:18:38.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.915+0000 7fcf5b7fe700 1 -- 172.21.15.19:0/1073960014 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11922 (secure 0 0 0) 0x7fcf5c0a9ad0 con 0x7fcf64117a60 2024-05-09T02:18:38.918 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:18:38.918 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:18:01.070328+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[0,1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24455,24457,24473,24484,24463]},"id":1}]} 2024-05-09T02:18:38.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.919+0000 7fcf689a1700 1 -- 172.21.15.19:0/1073960014 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcf500b2b40 msgr2=0x7fcf500b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:38.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.919+0000 7fcf689a1700 1 --2- 172.21.15.19:0/1073960014 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcf500b2b40 0x7fcf500b5050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009680 tx=0x7fcf4c009f50 comp rx=0 tx=0).stop 2024-05-09T02:18:38.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.919+0000 7fcf689a1700 1 -- 172.21.15.19:0/1073960014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf64117a60 msgr2=0x7fcf641a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:38.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.919+0000 7fcf689a1700 1 --2- 172.21.15.19:0/1073960014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf64117a60 0x7fcf641a5080 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00e940 tx=0x7fcf5c00ed00 comp rx=0 tx=0).stop 2024-05-09T02:18:38.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.919+0000 7fcf689a1700 1 -- 172.21.15.19:0/1073960014 shutdown_connections 2024-05-09T02:18:38.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.919+0000 7fcf689a1700 1 --2- 172.21.15.19:0/1073960014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf64117a60 0x7fcf641a5080 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:38.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.919+0000 7fcf689a1700 1 --2- 172.21.15.19:0/1073960014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf64103ba0 0x7fcf64113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:38.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.919+0000 7fcf689a1700 1 --2- 172.21.15.19:0/1073960014 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fcf500b2b40 0x7fcf500b5050 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009680 tx=0x7fcf4c009f50 comp rx=0 tx=0).stop 2024-05-09T02:18:38.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.919+0000 7fcf689a1700 1 --2- 172.21.15.19:0/1073960014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf641086a0 0x7fcf64114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:38.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.919+0000 7fcf689a1700 1 -- 172.21.15.19:0/1073960014 >> 172.21.15.19:0/1073960014 conn(0x7fcf640fedc0 msgr2=0x7fcf641060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:38.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.921+0000 7fcf689a1700 1 -- 172.21.15.19:0/1073960014 shutdown_connections 2024-05-09T02:18:38.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:38.921+0000 7fcf689a1700 1 -- 172.21.15.19:0/1073960014 wait complete. 2024-05-09T02:18:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 48 2024-05-09T02:18:39.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:39 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:18:39.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:39 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:18:39.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:39 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:18:39.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:39 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1073960014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:18:39.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:39 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:18:39.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:39 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:18:39.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:39 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:18:39.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:39 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1073960014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:18:39.746 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T02:18:39.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:39 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:18:39.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:39 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:18:39.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:39 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:18:39.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:39 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1073960014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:18:40.044 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:18:40.471 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:40 smithi019 ceph-mon[30151]: pgmap v226: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 24 MiB/s wr, 1.94k op/s 2024-05-09T02:18:40.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:40 smithi059 ceph-mon[34781]: pgmap v226: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 24 MiB/s wr, 1.94k op/s 2024-05-09T02:18:40.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:40 smithi196 ceph-mon[34718]: pgmap v226: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 24 MiB/s wr, 1.94k op/s 2024-05-09T02:18:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.755+0000 7fc2396d1700 1 -- 172.21.15.19:0/3234745779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc23410e940 msgr2=0x7fc234110e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.755+0000 7fc2396d1700 1 --2- 172.21.15.19:0/3234745779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc23410e940 0x7fc234110e10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009990 tx=0x7fc22c009ca0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.755+0000 7fc2396d1700 1 -- 172.21.15.19:0/3234745779 shutdown_connections 2024-05-09T02:18:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.755+0000 7fc2396d1700 1 --2- 172.21.15.19:0/3234745779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc23410e940 0x7fc234110e10 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.755+0000 7fc2396d1700 1 --2- 172.21.15.19:0/3234745779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc234105480 0x7fc234105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.755+0000 7fc2396d1700 1 --2- 172.21.15.19:0/3234745779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc234105e60 0x7fc23410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.755+0000 7fc2396d1700 1 -- 172.21.15.19:0/3234745779 >> 172.21.15.19:0/3234745779 conn(0x7fc2340fef50 msgr2=0x7fc234101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.755+0000 7fc2396d1700 1 -- 172.21.15.19:0/3234745779 shutdown_connections 2024-05-09T02:18:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.755+0000 7fc2396d1700 1 -- 172.21.15.19:0/3234745779 wait complete. 2024-05-09T02:18:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.755+0000 7fc2396d1700 1 Processor -- start 2024-05-09T02:18:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc2396d1700 1 -- start start 2024-05-09T02:18:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc2396d1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc234105480 0x7fc234103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc2396d1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc234105e60 0x7fc2341036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc2396d1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc23410e940 0x7fc234103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc2396d1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc234114560 con 0x7fc234105480 2024-05-09T02:18:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc2396d1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341143e0 con 0x7fc234105e60 2024-05-09T02:18:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc2396d1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341146e0 con 0x7fc23410e940 2024-05-09T02:18:41.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc232ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc234105480 0x7fc234103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc232ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc234105480 0x7fc234103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37350/0 (socket says 172.21.15.19:37350) 2024-05-09T02:18:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc2337fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc23410e940 0x7fc234103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc232ffd700 1 -- 172.21.15.19:0/1106976657 learned_addr learned my addr 172.21.15.19:0/1106976657 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc2327fc700 1 --2- 172.21.15.19:0/1106976657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc234105e60 0x7fc2341036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc232ffd700 1 -- 172.21.15.19:0/1106976657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc23410e940 msgr2=0x7fc234103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc232ffd700 1 --2- 172.21.15.19:0/1106976657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc23410e940 0x7fc234103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc232ffd700 1 -- 172.21.15.19:0/1106976657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc234105e60 msgr2=0x7fc2341036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc232ffd700 1 --2- 172.21.15.19:0/1106976657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc234105e60 0x7fc2341036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc232ffd700 1 -- 172.21.15.19:0/1106976657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2341a9210 con 0x7fc234105480 2024-05-09T02:18:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.756+0000 7fc2337fe700 1 --2- 172.21.15.19:0/1106976657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc23410e940 0x7fc234103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:18:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.757+0000 7fc2327fc700 1 --2- 172.21.15.19:0/1106976657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc234105e60 0x7fc2341036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:18:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.757+0000 7fc232ffd700 1 --2- 172.21.15.19:0/1106976657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc234105480 0x7fc234103170 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc22400ba50 tx=0x7fc22400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.757+0000 7fc21bfff700 1 -- 172.21.15.19:0/1106976657 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224019070 con 0x7fc234105480 2024-05-09T02:18:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.757+0000 7fc2396d1700 1 -- 172.21.15.19:0/1106976657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2341a9500 con 0x7fc234105480 2024-05-09T02:18:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.757+0000 7fc21bfff700 1 -- 172.21.15.19:0/1106976657 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc22400cde0 con 0x7fc234105480 2024-05-09T02:18:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.757+0000 7fc21bfff700 1 -- 172.21.15.19:0/1106976657 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224007370 con 0x7fc234105480 2024-05-09T02:18:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.757+0000 7fc2396d1700 1 -- 172.21.15.19:0/1106976657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2341a9a90 con 0x7fc234105480 2024-05-09T02:18:41.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.760+0000 7fc21bfff700 1 -- 172.21.15.19:0/1106976657 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc22401c070 con 0x7fc234105480 2024-05-09T02:18:41.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc2480e7b80 1 Processor -- start 2024-05-09T02:18:41.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc21bfff700 1 --2- 172.21.15.19:0/1106976657 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc21c0b29f0 0x7fc21c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc2480e7b80 1 -- start start 2024-05-09T02:18:41.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc2337fe700 1 --2- 172.21.15.19:0/1106976657 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc21c0b29f0 0x7fc21c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc21bfff700 1 -- 172.21.15.19:0/1106976657 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fc2240e1a60 con 0x7fc234105480 2024-05-09T02:18:41.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc2480e7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 0x55dc9b3fb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc2480e7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2784a0 0x55dc9b3fb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc2480e7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 0x55dc9b3f5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc2480e7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dc9b3180a0 con 0x55dc9b2784a0 2024-05-09T02:18:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc2480e7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dc9b2bd9a0 con 0x55dc9b3f5400 2024-05-09T02:18:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc2480e7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc9b314060 con 0x55dc9b2f2d00 2024-05-09T02:18:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc2327fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2784a0 0x55dc9b3fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc2327fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2784a0 0x55dc9b3fb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37360/0 (socket says 172.21.15.19:37360) 2024-05-09T02:18:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc2327fc700 1 -- 172.21.15.19:0/3915240062 learned_addr learned my addr 172.21.15.19:0/3915240062 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc232ffd700 1 --2- 172.21.15.19:0/3915240062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 0x55dc9b3fb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.761+0000 7fc2337fe700 1 --2- 172.21.15.19:0/3915240062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 0x55dc9b3f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.762+0000 7fc2327fc700 1 -- 172.21.15.19:0/3915240062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 msgr2=0x55dc9b3fb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.762+0000 7fc2327fc700 1 --2- 172.21.15.19:0/3915240062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 0x55dc9b3fb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.762+0000 7fc2327fc700 1 -- 172.21.15.19:0/3915240062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 msgr2=0x55dc9b3f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.762+0000 7fc2337fe700 1 --2- 172.21.15.19:0/1106976657 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc21c0b29f0 0x7fc21c0b4f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc22c003800 tx=0x7fc22c00b540 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.762+0000 7fc2327fc700 1 --2- 172.21.15.19:0/3915240062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 0x55dc9b3f5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.762+0000 7fc2327fc700 1 -- 172.21.15.19:0/3915240062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc9b2ae500 con 0x55dc9b2784a0 2024-05-09T02:18:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.762+0000 7fc232ffd700 1 --2- 172.21.15.19:0/3915240062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 0x55dc9b3fb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:18:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.762+0000 7fc2327fc700 1 --2- 172.21.15.19:0/3915240062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2784a0 0x55dc9b3fb780 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc228010480 tx=0x7fc228010790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d73d6d76a70eb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.762+0000 7fc2197fa700 1 -- 172.21.15.19:0/3915240062 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228004220 con 0x55dc9b2784a0 2024-05-09T02:18:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.762+0000 7fc2197fa700 1 -- 172.21.15.19:0/3915240062 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2280043c0 con 0x55dc9b2784a0 2024-05-09T02:18:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.762+0000 7fc2197fa700 1 -- 172.21.15.19:0/3915240062 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22801a770 con 0x55dc9b2784a0 2024-05-09T02:18:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.763+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3915240062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2784a0 msgr2=0x55dc9b3fb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.763+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/3915240062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2784a0 0x55dc9b3fb780 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc228010480 tx=0x7fc228010790 comp rx=0 tx=0).stop 2024-05-09T02:18:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.763+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3915240062 shutdown_connections 2024-05-09T02:18:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.763+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/3915240062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 0x55dc9b3f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.763+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/3915240062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 0x55dc9b3fb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.763+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/3915240062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2784a0 0x55dc9b3fb780 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.763+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3915240062 >> 172.21.15.19:0/3915240062 conn(0x55dc9b27bf70 msgr2=0x55dc9b3f0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.763+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3915240062 shutdown_connections 2024-05-09T02:18:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.763+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3915240062 wait complete. 2024-05-09T02:18:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.763+0000 7fc2480e7b80 1 -- start start 2024-05-09T02:18:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.763+0000 7fc2480e7b80 1 Processor -- start 2024-05-09T02:18:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc2480e7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2784a0 0x55dc9b3fefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc2480e7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 0x55dc9b3ff510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc2480e7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 0x55dc9b3ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc2480e7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dc9b3f6160 con 0x55dc9b2784a0 2024-05-09T02:18:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc2480e7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dc9b2b2b20 con 0x55dc9b3f5400 2024-05-09T02:18:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc2480e7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc9b402020 con 0x55dc9b2f2d00 2024-05-09T02:18:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc2480e7b80 10 client.0 _ll_register_callbacks cb 0x55dc9b1896c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc232ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2784a0 0x55dc9b3fefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc232ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2784a0 0x55dc9b3fefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37364/0 (socket says 172.21.15.19:37364) 2024-05-09T02:18:41.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc232ffd700 1 -- 172.21.15.19:0/3164465969 learned_addr learned my addr 172.21.15.19:0/3164465969 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:41.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc2327fc700 1 --2- 172.21.15.19:0/3164465969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 0x55dc9b3ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc2337fe700 1 --2- 172.21.15.19:0/3164465969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 0x55dc9b3ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc232ffd700 1 -- 172.21.15.19:0/3164465969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 msgr2=0x55dc9b3ff510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:41.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc232ffd700 1 --2- 172.21.15.19:0/3164465969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 0x55dc9b3ff510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc232ffd700 1 -- 172.21.15.19:0/3164465969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 msgr2=0x55dc9b3ffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:41.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc232ffd700 1 --2- 172.21.15.19:0/3164465969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 0x55dc9b3ffa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc232ffd700 1 -- 172.21.15.19:0/3164465969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc9b193490 con 0x55dc9b2784a0 2024-05-09T02:18:41.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc2327fc700 1 --2- 172.21.15.19:0/3164465969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 0x55dc9b3ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:18:41.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc2337fe700 1 --2- 172.21.15.19:0/3164465969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 0x55dc9b3ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:18:41.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc232ffd700 1 --2- 172.21.15.19:0/3164465969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2784a0 0x55dc9b3fefd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc2240a4e40 tx=0x7fc2240df650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ff82ec23ddfbe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:41.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.764+0000 7fc216ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:18:41.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.765+0000 7fc216ff5700 1 -- 172.21.15.19:0/3164465969 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2240b1c20 con 0x55dc9b2784a0 2024-05-09T02:18:41.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.765+0000 7fc216ff5700 1 -- 172.21.15.19:0/3164465969 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fc2240a9230 con 0x55dc9b2784a0 2024-05-09T02:18:41.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.765+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3164465969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2280043c0 con 0x55dc9b2784a0 2024-05-09T02:18:41.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.765+0000 7fc216ff5700 1 -- 172.21.15.19:0/3164465969 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc2240b1790 con 0x55dc9b2784a0 2024-05-09T02:18:41.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.765+0000 7fc2480e7b80 10 client.16191 fetch_fsmap learned FSMap version 48 2024-05-09T02:18:41.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.765+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3164465969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55dc9b40ca10 con 0x55dc9b2784a0 2024-05-09T02:18:41.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.765+0000 7fc216ff5700 1 -- 172.21.15.19:0/3164465969 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2280043c0 con 0x55dc9b2784a0 2024-05-09T02:18:41.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.765+0000 7fc216ff5700 1 -- 172.21.15.19:0/3164465969 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fc2240c9020 con 0x55dc9b2784a0 2024-05-09T02:18:41.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.765+0000 7fc2480e7b80 10 client.16191 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:18:41.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.765+0000 7fc2480e7b80 10 client.16191 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:18:41.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.765+0000 7fc2480e7b80 20 client.16191 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:41.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.765+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/3164465969 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55dc9b40f0a0 0x55dc9b4114c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.765+0000 7fc2480e7b80 4 client.16191 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:18:41.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.765+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3164465969 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc9b325a80 con 0x55dc9b40f0a0 2024-05-09T02:18:41.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.766+0000 7fc2327fc700 1 --2- 172.21.15.19:0/3164465969 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55dc9b40f0a0 0x55dc9b4114c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.766+0000 7fc2327fc700 1 --2- 172.21.15.19:0/3164465969 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55dc9b40f0a0 0x55dc9b4114c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f1807bd90e87a195 server_cookie=b50f8fc6cd152b70 in_seq=0 out_seq=0 2024-05-09T02:18:41.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.766+0000 7fc216ff5700 10 client.16191 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:18:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.767+0000 7fc216ff5700 1 -- 172.21.15.19:0/3164465969 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc9b325a80 con 0x55dc9b40f0a0 2024-05-09T02:18:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.767+0000 7fc216ff5700 10 client.16191 handle_command_reply: tid=0 2024-05-09T02:18:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.767+0000 7fc2480e7b80 1 client.16191 shutdown 2024-05-09T02:18:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.767+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3164465969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2784a0 msgr2=0x55dc9b3fefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.767+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/3164465969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2784a0 0x55dc9b3fefd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc2240a4e40 tx=0x7fc2240df650 comp rx=0 tx=0).stop 2024-05-09T02:18:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.767+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3164465969 shutdown_connections 2024-05-09T02:18:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.767+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/3164465969 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55dc9b40f0a0 0x55dc9b4114c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.767+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/3164465969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 0x55dc9b3ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.767+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/3164465969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2784a0 0x55dc9b3fefd0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.767+0000 7fc216ff5700 0 client.16191 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:18:41.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.767+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/3164465969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 0x55dc9b3ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.767+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3164465969 >> 172.21.15.19:0/3164465969 conn(0x55dc9b27bf70 msgr2=0x55dc9b3e9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:41.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.767+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3164465969 shutdown_connections 2024-05-09T02:18:41.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.767+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3164465969 wait complete. 2024-05-09T02:18:41.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.767+0000 7fc2480e7b80 20 client.16191 trim_cache size 0 max 16384 2024-05-09T02:18:41.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.783+0000 7fc2480e7b80 1 Processor -- start 2024-05-09T02:18:41.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.783+0000 7fc2480e7b80 1 -- start start 2024-05-09T02:18:41.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.783+0000 7fc2480e7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b2784a0 0x55dc9b3ea700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.783+0000 7fc2480e7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 0x55dc9b416c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.783+0000 7fc2480e7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b3f5400 0x55dc9b4171c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.783+0000 7fc2480e7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341b1250 con 0x55dc9b3f5400 2024-05-09T02:18:41.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.783+0000 7fc2480e7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341b2e20 con 0x55dc9b2784a0 2024-05-09T02:18:41.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.783+0000 7fc2480e7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341b19b0 con 0x55dc9b2f2d00 2024-05-09T02:18:41.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.783+0000 7fc2337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b3f5400 0x55dc9b4171c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.783+0000 7fc2337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b3f5400 0x55dc9b4171c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37380/0 (socket says 172.21.15.19:37380) 2024-05-09T02:18:41.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.783+0000 7fc2337fe700 1 -- 172.21.15.19:0/3227643266 learned_addr learned my addr 172.21.15.19:0/3227643266 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:41.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.783+0000 7fc232ffd700 1 --2- 172.21.15.19:0/3227643266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b2784a0 0x55dc9b3ea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.783+0000 7fc2327fc700 1 --2- 172.21.15.19:0/3227643266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 0x55dc9b416c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.784+0000 7fc2337fe700 1 -- 172.21.15.19:0/3227643266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 msgr2=0x55dc9b416c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:41.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.784+0000 7fc2337fe700 1 --2- 172.21.15.19:0/3227643266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 0x55dc9b416c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.784+0000 7fc2337fe700 1 -- 172.21.15.19:0/3227643266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b2784a0 msgr2=0x55dc9b3ea700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:41.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.784+0000 7fc2337fe700 1 --2- 172.21.15.19:0/3227643266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b2784a0 0x55dc9b3ea700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.784+0000 7fc2337fe700 1 -- 172.21.15.19:0/3227643266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc9b3ec9d0 con 0x55dc9b3f5400 2024-05-09T02:18:41.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.784+0000 7fc232ffd700 1 --2- 172.21.15.19:0/3227643266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b2784a0 0x55dc9b3ea700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:18:41.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.784+0000 7fc2337fe700 1 --2- 172.21.15.19:0/3227643266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b3f5400 0x55dc9b4171c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00c010 tx=0x7fc22c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185dbf0983db6fe4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:41.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.784+0000 7fc218ff9700 1 -- 172.21.15.19:0/3227643266 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c034070 con 0x55dc9b3f5400 2024-05-09T02:18:41.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.784+0000 7fc218ff9700 1 -- 172.21.15.19:0/3227643266 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc22c00f460 con 0x55dc9b3f5400 2024-05-09T02:18:41.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.784+0000 7fc218ff9700 1 -- 172.21.15.19:0/3227643266 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c02f690 con 0x55dc9b3f5400 2024-05-09T02:18:41.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.785+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3227643266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b3f5400 msgr2=0x55dc9b4171c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:41.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.785+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/3227643266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b3f5400 0x55dc9b4171c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00c010 tx=0x7fc22c01e000 comp rx=0 tx=0).stop 2024-05-09T02:18:41.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.785+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3227643266 shutdown_connections 2024-05-09T02:18:41.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.785+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/3227643266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b3f5400 0x55dc9b4171c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.785+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/3227643266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b2784a0 0x55dc9b3ea700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.785+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/3227643266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2f2d00 0x55dc9b416c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.785+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3227643266 >> 172.21.15.19:0/3227643266 conn(0x55dc9b27bf70 msgr2=0x55dc9b3e9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:41.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.785+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3227643266 shutdown_connections 2024-05-09T02:18:41.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.785+0000 7fc2480e7b80 1 -- 172.21.15.19:0/3227643266 wait complete. 2024-05-09T02:18:41.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.786+0000 7fc2480e7b80 1 -- start start 2024-05-09T02:18:41.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.786+0000 7fc2480e7b80 1 Processor -- start 2024-05-09T02:18:41.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.786+0000 7fc2480e7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2784a0 0x55dc9b471b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.786+0000 7fc2480e7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2f2d00 0x55dc9b472080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.786+0000 7fc2480e7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 0x55dc9b4026a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.786+0000 7fc2480e7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341b2750 con 0x55dc9b2f2d00 2024-05-09T02:18:41.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.786+0000 7fc2480e7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341b2080 con 0x55dc9b3f5400 2024-05-09T02:18:41.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.786+0000 7fc2480e7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341af210 con 0x55dc9b2784a0 2024-05-09T02:18:41.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.786+0000 7fc2327fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2f2d00 0x55dc9b472080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.786+0000 7fc2327fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2f2d00 0x55dc9b472080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37386/0 (socket says 172.21.15.19:37386) 2024-05-09T02:18:41.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.786+0000 7fc2327fc700 1 -- 172.21.15.19:0/1246634757 learned_addr learned my addr 172.21.15.19:0/1246634757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:41.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.786+0000 7fc2480e7b80 10 client.0 _ll_register_callbacks cb 0x55dc9b1896c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:41.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.786+0000 7fc232ffd700 1 --2- 172.21.15.19:0/1246634757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2784a0 0x55dc9b471b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.787+0000 7fc2337fe700 1 --2- 172.21.15.19:0/1246634757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 0x55dc9b4026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.787+0000 7fc2327fc700 1 -- 172.21.15.19:0/1246634757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2784a0 msgr2=0x55dc9b471b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:41.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.787+0000 7fc2327fc700 1 --2- 172.21.15.19:0/1246634757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2784a0 0x55dc9b471b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.787+0000 7fc2327fc700 1 -- 172.21.15.19:0/1246634757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 msgr2=0x55dc9b4026a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:41.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.787+0000 7fc2327fc700 1 --2- 172.21.15.19:0/1246634757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 0x55dc9b4026a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.787+0000 7fc2327fc700 1 -- 172.21.15.19:0/1246634757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc9b402ce0 con 0x55dc9b2f2d00 2024-05-09T02:18:41.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.787+0000 7fc232ffd700 1 --2- 172.21.15.19:0/1246634757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2784a0 0x55dc9b471b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:18:41.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.787+0000 7fc2327fc700 1 --2- 172.21.15.19:0/1246634757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2f2d00 0x55dc9b472080 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc228000c00 tx=0x7fc228005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429eeb0a94379b78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:41.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.787+0000 7fc216ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:18:41.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.787+0000 7fc216ff5700 1 -- 172.21.15.19:0/1246634757 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228027620 con 0x55dc9b2f2d00 2024-05-09T02:18:41.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.787+0000 7fc216ff5700 1 -- 172.21.15.19:0/1246634757 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fc2280277c0 con 0x55dc9b2f2d00 2024-05-09T02:18:41.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.787+0000 7fc2480e7b80 1 -- 172.21.15.19:0/1246634757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc22c00f460 con 0x55dc9b2f2d00 2024-05-09T02:18:41.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.787+0000 7fc216ff5700 1 -- 172.21.15.19:0/1246634757 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc228036050 con 0x55dc9b2f2d00 2024-05-09T02:18:41.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.787+0000 7fc2480e7b80 10 client.16197 fetch_fsmap learned FSMap version 48 2024-05-09T02:18:41.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.787+0000 7fc2480e7b80 1 -- 172.21.15.19:0/1246634757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55dc9b479210 con 0x55dc9b2f2d00 2024-05-09T02:18:41.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.788+0000 7fc216ff5700 1 -- 172.21.15.19:0/1246634757 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc22c00f460 con 0x55dc9b2f2d00 2024-05-09T02:18:41.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.788+0000 7fc216ff5700 1 -- 172.21.15.19:0/1246634757 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fc22801a6f0 con 0x55dc9b2f2d00 2024-05-09T02:18:41.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.788+0000 7fc2480e7b80 10 client.16197 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:18:41.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.788+0000 7fc2480e7b80 10 client.16197 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:18:41.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.788+0000 7fc2480e7b80 20 client.16197 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:41.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.788+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/1246634757 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55dc9b479730 0x55dc9b47bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:41.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.788+0000 7fc2480e7b80 4 client.16197 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:18:41.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.788+0000 7fc2480e7b80 1 -- 172.21.15.19:0/1246634757 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc9b31fb20 con 0x55dc9b479730 2024-05-09T02:18:41.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.788+0000 7fc232ffd700 1 --2- 172.21.15.19:0/1246634757 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55dc9b479730 0x55dc9b47bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:41.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc232ffd700 1 --2- 172.21.15.19:0/1246634757 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55dc9b479730 0x55dc9b47bb50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=94a24fee05d448cd server_cookie=e402b9fccdf9b481 in_seq=0 out_seq=0 2024-05-09T02:18:41.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc216ff5700 10 client.16197 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:18:41.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc216ff5700 1 -- 172.21.15.19:0/1246634757 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc9b31fb20 con 0x55dc9b479730 2024-05-09T02:18:41.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc216ff5700 10 client.16197 handle_command_reply: tid=0 2024-05-09T02:18:41.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc2480e7b80 1 client.16197 shutdown 2024-05-09T02:18:41.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc2480e7b80 1 -- 172.21.15.19:0/1246634757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2f2d00 msgr2=0x55dc9b472080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:41.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/1246634757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2f2d00 0x55dc9b472080 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc228000c00 tx=0x7fc228005430 comp rx=0 tx=0).stop 2024-05-09T02:18:41.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc2480e7b80 1 -- 172.21.15.19:0/1246634757 shutdown_connections 2024-05-09T02:18:41.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/1246634757 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55dc9b479730 0x55dc9b47bb50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/1246634757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc9b3f5400 0x55dc9b4026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/1246634757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc9b2784a0 0x55dc9b471b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc216ff5700 0 client.16197 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:18:41.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc2480e7b80 1 --2- 172.21.15.19:0/1246634757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc9b2f2d00 0x55dc9b472080 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc2480e7b80 1 -- 172.21.15.19:0/1246634757 >> 172.21.15.19:0/1246634757 conn(0x55dc9b27bf70 msgr2=0x55dc9b3f43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:41.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc2480e7b80 1 -- 172.21.15.19:0/1246634757 shutdown_connections 2024-05-09T02:18:41.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc2480e7b80 1 -- 172.21.15.19:0/1246634757 wait complete. 2024-05-09T02:18:41.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.789+0000 7fc2480e7b80 20 client.16197 trim_cache size 0 max 16384 2024-05-09T02:18:41.799 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:18:41.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.790+0000 7fc2396d1700 1 -- 172.21.15.19:0/1106976657 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc21c0b29f0 msgr2=0x7fc21c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:41.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.790+0000 7fc2396d1700 1 --2- 172.21.15.19:0/1106976657 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc21c0b29f0 0x7fc21c0b4f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc22c003800 tx=0x7fc22c00b540 comp rx=0 tx=0).stop 2024-05-09T02:18:41.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.790+0000 7fc2396d1700 1 -- 172.21.15.19:0/1106976657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc234105480 msgr2=0x7fc234103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:41.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.790+0000 7fc2396d1700 1 --2- 172.21.15.19:0/1106976657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc234105480 0x7fc234103170 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc22400ba50 tx=0x7fc22400bd60 comp rx=0 tx=0).stop 2024-05-09T02:18:41.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.790+0000 7fc2396d1700 1 -- 172.21.15.19:0/1106976657 shutdown_connections 2024-05-09T02:18:41.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.790+0000 7fc2396d1700 1 --2- 172.21.15.19:0/1106976657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc23410e940 0x7fc234103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.790+0000 7fc2396d1700 1 --2- 172.21.15.19:0/1106976657 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc21c0b29f0 0x7fc21c0b4f00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.790+0000 7fc2396d1700 1 --2- 172.21.15.19:0/1106976657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc234105480 0x7fc234103170 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.790+0000 7fc2396d1700 1 --2- 172.21.15.19:0/1106976657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc234105e60 0x7fc2341036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:41.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.790+0000 7fc2396d1700 1 -- 172.21.15.19:0/1106976657 >> 172.21.15.19:0/1106976657 conn(0x7fc2340fef50 msgr2=0x7fc2341083b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:41.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.790+0000 7fc2396d1700 1 -- 172.21.15.19:0/1106976657 shutdown_connections 2024-05-09T02:18:41.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:41.790+0000 7fc2396d1700 1 -- 172.21.15.19:0/1106976657 wait complete. 2024-05-09T02:18:42.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:42 smithi019 ceph-mon[30151]: pgmap v227: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-05-09T02:18:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 420.0 s, 967.4 tps, lat 102.625 ms stddev 369.952 2024-05-09T02:18:42.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:42 smithi059 ceph-mon[34781]: pgmap v227: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-05-09T02:18:42.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:42 smithi196 ceph-mon[34718]: pgmap v227: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-05-09T02:18:43.045 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T02:18:43.332 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:18:44.596 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:44 smithi019 ceph-mon[30151]: pgmap v228: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-05-09T02:18:44.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:44 smithi059 ceph-mon[34781]: pgmap v228: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-05-09T02:18:44.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:44 smithi196 ceph-mon[34718]: pgmap v228: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-05-09T02:18:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.010+0000 7ff2c76ff700 1 -- 172.21.15.19:0/2106368666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0107df0 msgr2=0x7ff2c0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.010+0000 7ff2c76ff700 1 --2- 172.21.15.19:0/2106368666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0107df0 0x7ff2c0108200 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40099c0 tx=0x7ff2b4009cd0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.010+0000 7ff2c76ff700 1 -- 172.21.15.19:0/2106368666 shutdown_connections 2024-05-09T02:18:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.010+0000 7ff2c76ff700 1 --2- 172.21.15.19:0/2106368666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2c0108740 0x7ff2c01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.010+0000 7ff2c76ff700 1 --2- 172.21.15.19:0/2106368666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c01034c0 0x7ff2c01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.010+0000 7ff2c76ff700 1 --2- 172.21.15.19:0/2106368666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0107df0 0x7ff2c0108200 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.010+0000 7ff2c76ff700 1 -- 172.21.15.19:0/2106368666 >> 172.21.15.19:0/2106368666 conn(0x7ff2c00fedc0 msgr2=0x7ff2c01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.010+0000 7ff2c76ff700 1 -- 172.21.15.19:0/2106368666 shutdown_connections 2024-05-09T02:18:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.010+0000 7ff2c76ff700 1 -- 172.21.15.19:0/2106368666 wait complete. 2024-05-09T02:18:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.010+0000 7ff2c76ff700 1 Processor -- start 2024-05-09T02:18:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.010+0000 7ff2c76ff700 1 -- start start 2024-05-09T02:18:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c76ff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c01034c0 0x7ff2c019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c76ff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2c0107df0 0x7ff2c019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c76ff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108740 0x7ff2c019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c76ff700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c0113350 con 0x7ff2c0108740 2024-05-09T02:18:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c76ff700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c01131d0 con 0x7ff2c0107df0 2024-05-09T02:18:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c76ff700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c01134d0 con 0x7ff2c01034c0 2024-05-09T02:18:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c5c9c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108740 0x7ff2c019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c5c9c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108740 0x7ff2c019f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37404/0 (socket says 172.21.15.19:37404) 2024-05-09T02:18:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c5c9c700 1 -- 172.21.15.19:0/2101096088 learned_addr learned my addr 172.21.15.19:0/2101096088 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c549b700 1 --2- 172.21.15.19:0/2101096088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c01034c0 0x7ff2c019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c4c9a700 1 --2- 172.21.15.19:0/2101096088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2c0107df0 0x7ff2c019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c5c9c700 1 -- 172.21.15.19:0/2101096088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c01034c0 msgr2=0x7ff2c019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c5c9c700 1 --2- 172.21.15.19:0/2101096088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c01034c0 0x7ff2c019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c5c9c700 1 -- 172.21.15.19:0/2101096088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2c0107df0 msgr2=0x7ff2c019b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c5c9c700 1 --2- 172.21.15.19:0/2101096088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2c0107df0 0x7ff2c019b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c5c9c700 1 -- 172.21.15.19:0/2101096088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c019fd20 con 0x7ff2c0108740 2024-05-09T02:18:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c4c9a700 1 --2- 172.21.15.19:0/2101096088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2c0107df0 0x7ff2c019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:18:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.011+0000 7ff2c5c9c700 1 --2- 172.21.15.19:0/2101096088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108740 0x7ff2c019f680 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800d980 tx=0x7ff2b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.012+0000 7ff2be7fc700 1 -- 172.21.15.19:0/2101096088 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8016070 con 0x7ff2c0108740 2024-05-09T02:18:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.012+0000 7ff2be7fc700 1 -- 172.21.15.19:0/2101096088 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b8004830 con 0x7ff2c0108740 2024-05-09T02:18:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.012+0000 7ff2c76ff700 1 -- 172.21.15.19:0/2101096088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c01a0010 con 0x7ff2c0108740 2024-05-09T02:18:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.012+0000 7ff2be7fc700 1 -- 172.21.15.19:0/2101096088 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80054b0 con 0x7ff2c0108740 2024-05-09T02:18:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.012+0000 7ff2c76ff700 1 -- 172.21.15.19:0/2101096088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c01a7930 con 0x7ff2c0108740 2024-05-09T02:18:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.015+0000 7ff2be7fc700 1 -- 172.21.15.19:0/2101096088 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff2b80049e0 con 0x7ff2c0108740 2024-05-09T02:18:45.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.016+0000 7ff2be7fc700 1 --2- 172.21.15.19:0/2101096088 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff2a80b2a70 0x7ff2a80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.016+0000 7ff2c549b700 1 --2- 172.21.15.19:0/2101096088 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff2a80b2a70 0x7ff2a80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.016+0000 7ff2d6115b80 1 Processor -- start 2024-05-09T02:18:45.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.016+0000 7ff2be7fc700 1 -- 172.21.15.19:0/2101096088 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7ff2b80e23e0 con 0x7ff2c0108740 2024-05-09T02:18:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.016+0000 7ff2d6115b80 1 -- start start 2024-05-09T02:18:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2c549b700 1 --2- 172.21.15.19:0/2101096088 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff2a80b2a70 0x7ff2a80b4f80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000f540 tx=0x7ff2b000b040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2d6115b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 0x56305e4b9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2d6115b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4b9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2d6115b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e4b3400 0x56305e4b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2d6115b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56305e3d60a0 con 0x56305e3364a0 2024-05-09T02:18:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2d6115b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56305e37b9a0 con 0x56305e3b0d00 2024-05-09T02:18:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2d6115b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56305e3d2060 con 0x56305e4b3400 2024-05-09T02:18:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2c4c9a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2c4c9a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4b9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37410/0 (socket says 172.21.15.19:37410) 2024-05-09T02:18:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2c4c9a700 1 -- 172.21.15.19:0/463536264 learned_addr learned my addr 172.21.15.19:0/463536264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2c549b700 1 --2- 172.21.15.19:0/463536264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 0x56305e4b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2c5c9c700 1 --2- 172.21.15.19:0/463536264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e4b3400 0x56305e4b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2c4c9a700 1 -- 172.21.15.19:0/463536264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e4b3400 msgr2=0x56305e4b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2c4c9a700 1 --2- 172.21.15.19:0/463536264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e4b3400 0x56305e4b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2c4c9a700 1 -- 172.21.15.19:0/463536264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 msgr2=0x56305e4b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2c4c9a700 1 --2- 172.21.15.19:0/463536264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 0x56305e4b9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2c4c9a700 1 -- 172.21.15.19:0/463536264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56305e36c500 con 0x56305e3364a0 2024-05-09T02:18:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2c549b700 1 --2- 172.21.15.19:0/463536264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 0x56305e4b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:18:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.017+0000 7ff2c4c9a700 1 --2- 172.21.15.19:0/463536264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4b9780 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40099c0 tx=0x7ff2b4009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36a1fe36b020b9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.018+0000 7ff2a77fe700 1 -- 172.21.15.19:0/463536264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4026070 con 0x56305e3364a0 2024-05-09T02:18:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.018+0000 7ff2a77fe700 1 -- 172.21.15.19:0/463536264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b40037c0 con 0x56305e3364a0 2024-05-09T02:18:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.018+0000 7ff2a77fe700 1 -- 172.21.15.19:0/463536264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b400ba20 con 0x56305e3364a0 2024-05-09T02:18:45.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.019+0000 7ff2d6115b80 1 -- 172.21.15.19:0/463536264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 msgr2=0x56305e4b9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:45.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.019+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/463536264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4b9780 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40099c0 tx=0x7ff2b4009510 comp rx=0 tx=0).stop 2024-05-09T02:18:45.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.019+0000 7ff2d6115b80 1 -- 172.21.15.19:0/463536264 shutdown_connections 2024-05-09T02:18:45.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.019+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/463536264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e4b3400 0x56305e4b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.019+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/463536264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 0x56305e4b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.019+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/463536264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4b9780 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.019+0000 7ff2d6115b80 1 -- 172.21.15.19:0/463536264 >> 172.21.15.19:0/463536264 conn(0x56305e339f70 msgr2=0x56305e4ae220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:45.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.019+0000 7ff2d6115b80 1 -- 172.21.15.19:0/463536264 shutdown_connections 2024-05-09T02:18:45.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.019+0000 7ff2d6115b80 1 -- 172.21.15.19:0/463536264 wait complete. 2024-05-09T02:18:45.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.019+0000 7ff2d6115b80 1 -- start start 2024-05-09T02:18:45.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.019+0000 7ff2d6115b80 1 Processor -- start 2024-05-09T02:18:45.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2d6115b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4bcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2d6115b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 0x56305e4bd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2d6115b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e4b3400 0x56305e4bda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2d6115b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56305e4b4100 con 0x56305e3364a0 2024-05-09T02:18:45.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2d6115b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56305e370b20 con 0x56305e3b0d00 2024-05-09T02:18:45.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2d6115b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56305e4c9200 con 0x56305e4b3400 2024-05-09T02:18:45.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2c549b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4bcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2d6115b80 10 client.0 _ll_register_callbacks cb 0x56305e2476c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:45.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2c549b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4bcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37414/0 (socket says 172.21.15.19:37414) 2024-05-09T02:18:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2c549b700 1 -- 172.21.15.19:0/4257927707 learned_addr learned my addr 172.21.15.19:0/4257927707 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2c4c9a700 1 --2- 172.21.15.19:0/4257927707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 0x56305e4bd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2c5c9c700 1 --2- 172.21.15.19:0/4257927707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e4b3400 0x56305e4bda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2c549b700 1 -- 172.21.15.19:0/4257927707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e4b3400 msgr2=0x56305e4bda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2c549b700 1 --2- 172.21.15.19:0/4257927707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e4b3400 0x56305e4bda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2c549b700 1 -- 172.21.15.19:0/4257927707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 msgr2=0x56305e4bd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2c549b700 1 --2- 172.21.15.19:0/4257927707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 0x56305e4bd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2c549b700 1 -- 172.21.15.19:0/4257927707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56305e251490 con 0x56305e3364a0 2024-05-09T02:18:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2c549b700 1 --2- 172.21.15.19:0/4257927707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4bcff0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00060b0 tx=0x7ff2b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f828b1850e9e3021 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.020+0000 7ff2a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:18:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.021+0000 7ff2a4ff9700 1 -- 172.21.15.19:0/4257927707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0027020 con 0x56305e3364a0 2024-05-09T02:18:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.021+0000 7ff2a4ff9700 1 -- 172.21.15.19:0/4257927707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7ff2b0015cf0 con 0x56305e3364a0 2024-05-09T02:18:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.021+0000 7ff2d6115b80 1 -- 172.21.15.19:0/4257927707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2b40037c0 con 0x56305e3364a0 2024-05-09T02:18:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.021+0000 7ff2a4ff9700 1 -- 172.21.15.19:0/4257927707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff2b0032050 con 0x56305e3364a0 2024-05-09T02:18:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.021+0000 7ff2d6115b80 10 client.16209 fetch_fsmap learned FSMap version 48 2024-05-09T02:18:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.021+0000 7ff2d6115b80 1 -- 172.21.15.19:0/4257927707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56305e4cabb0 con 0x56305e3364a0 2024-05-09T02:18:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.021+0000 7ff2a4ff9700 1 -- 172.21.15.19:0/4257927707 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b40037c0 con 0x56305e3364a0 2024-05-09T02:18:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.021+0000 7ff2a4ff9700 1 -- 172.21.15.19:0/4257927707 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7ff2b0023420 con 0x56305e3364a0 2024-05-09T02:18:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.021+0000 7ff2d6115b80 10 client.16209 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:18:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.021+0000 7ff2d6115b80 10 client.16209 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:18:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.021+0000 7ff2d6115b80 20 client.16209 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.021+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/4257927707 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x56305e4cd240 0x56305e4cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.021+0000 7ff2d6115b80 4 client.16209 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:18:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.021+0000 7ff2d6115b80 1 -- 172.21.15.19:0/4257927707 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56305e3e3a80 con 0x56305e4cd240 2024-05-09T02:18:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.022+0000 7ff2c4c9a700 1 --2- 172.21.15.19:0/4257927707 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x56305e4cd240 0x56305e4cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.022+0000 7ff2a4ff9700 10 client.16209 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:18:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.022+0000 7ff2c4c9a700 1 --2- 172.21.15.19:0/4257927707 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x56305e4cd240 0x56305e4cf660 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=25d273f2a02b45b1 server_cookie=503d820c2b55145f in_seq=0 out_seq=0 2024-05-09T02:18:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.023+0000 7ff2a4ff9700 1 -- 172.21.15.19:0/4257927707 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56305e3e3a80 con 0x56305e4cd240 2024-05-09T02:18:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.023+0000 7ff2a4ff9700 10 client.16209 handle_command_reply: tid=0 2024-05-09T02:18:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.023+0000 7ff2d6115b80 1 client.16209 shutdown 2024-05-09T02:18:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.023+0000 7ff2d6115b80 1 -- 172.21.15.19:0/4257927707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 msgr2=0x56305e4bcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.023+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/4257927707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4bcff0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00060b0 tx=0x7ff2b0004ad0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.023+0000 7ff2d6115b80 1 -- 172.21.15.19:0/4257927707 shutdown_connections 2024-05-09T02:18:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.023+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/4257927707 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x56305e4cd240 0x56305e4cf660 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.023+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/4257927707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e4b3400 0x56305e4bda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.023+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/4257927707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4bcff0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.023+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/4257927707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 0x56305e4bd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.023+0000 7ff2d6115b80 1 -- 172.21.15.19:0/4257927707 >> 172.21.15.19:0/4257927707 conn(0x56305e339f70 msgr2=0x56305e4a7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.023+0000 7ff2a4ff9700 0 client.16209 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:18:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.023+0000 7ff2d6115b80 1 -- 172.21.15.19:0/4257927707 shutdown_connections 2024-05-09T02:18:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.023+0000 7ff2d6115b80 1 -- 172.21.15.19:0/4257927707 wait complete. 2024-05-09T02:18:45.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.024+0000 7ff2d6115b80 20 client.16209 trim_cache size 0 max 16384 2024-05-09T02:18:45.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2d6115b80 1 Processor -- start 2024-05-09T02:18:45.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2d6115b80 1 -- start start 2024-05-09T02:18:45.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2d6115b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e339f70 0x56305e4bbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2d6115b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3364a0 0x56305e4a8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2d6115b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3b0d00 0x56305e4d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2d6115b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c01aee00 con 0x56305e3b0d00 2024-05-09T02:18:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2d6115b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c01b09d0 con 0x56305e3364a0 2024-05-09T02:18:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2d6115b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c01af560 con 0x56305e339f70 2024-05-09T02:18:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2c5c9c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3b0d00 0x56305e4d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2c5c9c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3b0d00 0x56305e4d4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37426/0 (socket says 172.21.15.19:37426) 2024-05-09T02:18:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2c5c9c700 1 -- 172.21.15.19:0/1902781031 learned_addr learned my addr 172.21.15.19:0/1902781031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2c549b700 1 --2- 172.21.15.19:0/1902781031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e339f70 0x56305e4bbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2c4c9a700 1 --2- 172.21.15.19:0/1902781031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3364a0 0x56305e4a8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2c5c9c700 1 -- 172.21.15.19:0/1902781031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e339f70 msgr2=0x56305e4bbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2c5c9c700 1 --2- 172.21.15.19:0/1902781031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e339f70 0x56305e4bbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2c5c9c700 1 -- 172.21.15.19:0/1902781031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3364a0 msgr2=0x56305e4a8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2c5c9c700 1 --2- 172.21.15.19:0/1902781031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3364a0 0x56305e4a8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.039+0000 7ff2c5c9c700 1 -- 172.21.15.19:0/1902781031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56305e4aa9d0 con 0x56305e3b0d00 2024-05-09T02:18:45.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.040+0000 7ff2c5c9c700 1 --2- 172.21.15.19:0/1902781031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3b0d00 0x56305e4d4e20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80a9620 tx=0x7ff2b80a9eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8aff7484e6b13a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:45.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.040+0000 7ff2a6ffd700 1 -- 172.21.15.19:0/1902781031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80b24f0 con 0x56305e3b0d00 2024-05-09T02:18:45.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.040+0000 7ff2a6ffd700 1 -- 172.21.15.19:0/1902781031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b8004d00 con 0x56305e3b0d00 2024-05-09T02:18:45.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.040+0000 7ff2a6ffd700 1 -- 172.21.15.19:0/1902781031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80a9b50 con 0x56305e3b0d00 2024-05-09T02:18:45.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.040+0000 7ff2d6115b80 1 -- 172.21.15.19:0/1902781031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3b0d00 msgr2=0x56305e4d4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.040+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/1902781031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3b0d00 0x56305e4d4e20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80a9620 tx=0x7ff2b80a9eb0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.041+0000 7ff2d6115b80 1 -- 172.21.15.19:0/1902781031 shutdown_connections 2024-05-09T02:18:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.041+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/1902781031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3b0d00 0x56305e4d4e20 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.041+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/1902781031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e339f70 0x56305e4bbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.041+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/1902781031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3364a0 0x56305e4a8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.041+0000 7ff2d6115b80 1 -- 172.21.15.19:0/1902781031 >> 172.21.15.19:0/1902781031 conn(0x56305e4b3ae0 msgr2=0x56305e4a7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.041+0000 7ff2d6115b80 1 -- 172.21.15.19:0/1902781031 shutdown_connections 2024-05-09T02:18:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.041+0000 7ff2d6115b80 1 -- 172.21.15.19:0/1902781031 wait complete. 2024-05-09T02:18:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.041+0000 7ff2d6115b80 1 -- start start 2024-05-09T02:18:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.041+0000 7ff2d6115b80 1 Processor -- start 2024-05-09T02:18:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2d6115b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2d6115b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e339f70 0x56305e4c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2d6115b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 0x56305e4c2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2d6115b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c01b0300 con 0x56305e3364a0 2024-05-09T02:18:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2d6115b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c01afc30 con 0x56305e3b0d00 2024-05-09T02:18:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2d6115b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c01acdc0 con 0x56305e339f70 2024-05-09T02:18:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2d6115b80 10 client.0 _ll_register_callbacks cb 0x56305e2476c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2c5c9c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 0x56305e4c2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2c4c9a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e339f70 0x56305e4c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2c549b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2c5c9c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 0x56305e4c2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:46638/0 (socket says 172.21.15.19:46638) 2024-05-09T02:18:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2c5c9c700 1 -- 172.21.15.19:0/895238687 learned_addr learned my addr 172.21.15.19:0/895238687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2c5c9c700 1 -- 172.21.15.19:0/895238687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e339f70 msgr2=0x56305e4c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2c5c9c700 1 --2- 172.21.15.19:0/895238687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e339f70 0x56305e4c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2c5c9c700 1 -- 172.21.15.19:0/895238687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 msgr2=0x56305e4c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2c5c9c700 1 --2- 172.21.15.19:0/895238687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2c5c9c700 1 -- 172.21.15.19:0/895238687 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56305e4c2670 con 0x56305e3b0d00 2024-05-09T02:18:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2c4c9a700 1 --2- 172.21.15.19:0/895238687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e339f70 0x56305e4c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:18:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2c549b700 1 --2- 172.21.15.19:0/895238687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:18:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2c5c9c700 1 --2- 172.21.15.19:0/895238687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 0x56305e4c2130 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800b540 tx=0x7ff2b8091280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ad5279754e262a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:18:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.042+0000 7ff2a4ff9700 1 -- 172.21.15.19:0/895238687 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80c7070 con 0x56305e3b0d00 2024-05-09T02:18:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.043+0000 7ff2a4ff9700 1 -- 172.21.15.19:0/895238687 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7ff2b80a9b00 con 0x56305e3b0d00 2024-05-09T02:18:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.043+0000 7ff2d6115b80 1 -- 172.21.15.19:0/895238687 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2b8004d00 con 0x56305e3b0d00 2024-05-09T02:18:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.043+0000 7ff2a4ff9700 1 -- 172.21.15.19:0/895238687 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff2b80b24f0 con 0x56305e3b0d00 2024-05-09T02:18:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.043+0000 7ff2d6115b80 10 client.25943 fetch_fsmap learned FSMap version 48 2024-05-09T02:18:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.043+0000 7ff2d6115b80 1 -- 172.21.15.19:0/895238687 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56305e5380e0 con 0x56305e3b0d00 2024-05-09T02:18:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.043+0000 7ff2a4ff9700 1 -- 172.21.15.19:0/895238687 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b8004d00 con 0x56305e3b0d00 2024-05-09T02:18:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.043+0000 7ff2a4ff9700 1 -- 172.21.15.19:0/895238687 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7ff2b80b9070 con 0x56305e3b0d00 2024-05-09T02:18:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.043+0000 7ff2d6115b80 10 client.25943 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:18:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.043+0000 7ff2d6115b80 10 client.25943 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:18:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.043+0000 7ff2d6115b80 20 client.25943 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.043+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/895238687 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x56305e4b3400 0x56305e53a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.043+0000 7ff2d6115b80 4 client.25943 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:18:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.043+0000 7ff2d6115b80 1 -- 172.21.15.19:0/895238687 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56305e3ddb20 con 0x56305e4b3400 2024-05-09T02:18:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.043+0000 7ff2c549b700 1 --2- 172.21.15.19:0/895238687 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x56305e4b3400 0x56305e53a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2c549b700 1 --2- 172.21.15.19:0/895238687 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x56305e4b3400 0x56305e53a7d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e14934e4953df0c6 server_cookie=8c27ebaef12708e5 in_seq=0 out_seq=0 2024-05-09T02:18:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2a4ff9700 10 client.25943 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:18:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2a4ff9700 1 -- 172.21.15.19:0/895238687 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56305e3ddb20 con 0x56305e4b3400 2024-05-09T02:18:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2a4ff9700 10 client.25943 handle_command_reply: tid=0 2024-05-09T02:18:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2d6115b80 1 client.25943 shutdown 2024-05-09T02:18:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2d6115b80 1 -- 172.21.15.19:0/895238687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 msgr2=0x56305e4c2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/895238687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 0x56305e4c2130 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800b540 tx=0x7ff2b8091280 comp rx=0 tx=0).stop 2024-05-09T02:18:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2d6115b80 1 -- 172.21.15.19:0/895238687 shutdown_connections 2024-05-09T02:18:45.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/895238687 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x56305e4b3400 0x56305e53a7d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/895238687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305e3b0d00 0x56305e4c2130 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/895238687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305e3364a0 0x56305e4c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2a4ff9700 0 client.25943 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:18:45.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2d6115b80 1 --2- 172.21.15.19:0/895238687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305e339f70 0x56305e4c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2d6115b80 1 -- 172.21.15.19:0/895238687 >> 172.21.15.19:0/895238687 conn(0x56305e4b3ae0 msgr2=0x56305e4b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:45.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2d6115b80 1 -- 172.21.15.19:0/895238687 shutdown_connections 2024-05-09T02:18:45.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2d6115b80 1 -- 172.21.15.19:0/895238687 wait complete. 2024-05-09T02:18:45.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.044+0000 7ff2d6115b80 20 client.25943 trim_cache size 0 max 16384 2024-05-09T02:18:45.053 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:18:45.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.045+0000 7ff2c76ff700 1 -- 172.21.15.19:0/2101096088 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff2a80b2a70 msgr2=0x7ff2a80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:45.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.045+0000 7ff2c76ff700 1 --2- 172.21.15.19:0/2101096088 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff2a80b2a70 0x7ff2a80b4f80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000f540 tx=0x7ff2b000b040 comp rx=0 tx=0).stop 2024-05-09T02:18:45.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.045+0000 7ff2c76ff700 1 -- 172.21.15.19:0/2101096088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108740 msgr2=0x7ff2c019f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:45.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.045+0000 7ff2c76ff700 1 --2- 172.21.15.19:0/2101096088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108740 0x7ff2c019f680 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800d980 tx=0x7ff2b800dc90 comp rx=0 tx=0).stop 2024-05-09T02:18:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.045+0000 7ff2c76ff700 1 -- 172.21.15.19:0/2101096088 shutdown_connections 2024-05-09T02:18:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.045+0000 7ff2c76ff700 1 --2- 172.21.15.19:0/2101096088 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff2a80b2a70 0x7ff2a80b4f80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.045+0000 7ff2c76ff700 1 --2- 172.21.15.19:0/2101096088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c0108740 0x7ff2c019f680 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.045+0000 7ff2c76ff700 1 --2- 172.21.15.19:0/2101096088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c01034c0 0x7ff2c019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.045+0000 7ff2c76ff700 1 --2- 172.21.15.19:0/2101096088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2c0107df0 0x7ff2c019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.045+0000 7ff2c76ff700 1 -- 172.21.15.19:0/2101096088 >> 172.21.15.19:0/2101096088 conn(0x7ff2c00fedc0 msgr2=0x7ff2c006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.045+0000 7ff2c76ff700 1 -- 172.21.15.19:0/2101096088 shutdown_connections 2024-05-09T02:18:45.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:45.045+0000 7ff2c76ff700 1 -- 172.21.15.19:0/2101096088 wait complete. 2024-05-09T02:18:46.167 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T02:18:46.435 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:18:46.558 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:46 smithi019 ceph-mon[30151]: pgmap v229: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 1.86k op/s 2024-05-09T02:18:46.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:46 smithi059 ceph-mon[34781]: pgmap v229: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 1.86k op/s 2024-05-09T02:18:46.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:46 smithi196 ceph-mon[34718]: pgmap v229: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 1.86k op/s 2024-05-09T02:18:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 425.0 s, 395.2 tps, lat 253.423 ms stddev 917.343 2024-05-09T02:18:48.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.383+0000 7f733fe0f700 1 -- 172.21.15.19:0/914676708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338107df0 msgr2=0x7f7338108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:48.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.383+0000 7f733fe0f700 1 --2- 172.21.15.19:0/914676708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338107df0 0x7f7338108200 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f73280099c0 tx=0x7f7328009cd0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.383+0000 7f733fe0f700 1 -- 172.21.15.19:0/914676708 shutdown_connections 2024-05-09T02:18:48.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.383+0000 7f733fe0f700 1 --2- 172.21.15.19:0/914676708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7338108740 0x7f73381100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.383+0000 7f733fe0f700 1 --2- 172.21.15.19:0/914676708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73381034c0 0x7f73381038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.383+0000 7f733fe0f700 1 --2- 172.21.15.19:0/914676708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338107df0 0x7f7338108200 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.383+0000 7f733fe0f700 1 -- 172.21.15.19:0/914676708 >> 172.21.15.19:0/914676708 conn(0x7f73380fedc0 msgr2=0x7f73381011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:48.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.383+0000 7f733fe0f700 1 -- 172.21.15.19:0/914676708 shutdown_connections 2024-05-09T02:18:48.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.383+0000 7f733fe0f700 1 -- 172.21.15.19:0/914676708 wait complete. 2024-05-09T02:18:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.385+0000 7f733fe0f700 1 Processor -- start 2024-05-09T02:18:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.385+0000 7f733fe0f700 1 -- start start 2024-05-09T02:18:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.385+0000 7f733fe0f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73381034c0 0x7f73381a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.385+0000 7f733fe0f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7338107df0 0x7f73381a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.385+0000 7f733fe0f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338108740 0x7f73381a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.385+0000 7f733fe0f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7338112350 con 0x7f73381034c0 2024-05-09T02:18:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.385+0000 7f733fe0f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73381121d0 con 0x7f7338107df0 2024-05-09T02:18:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.385+0000 7f733fe0f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73381124d0 con 0x7f7338108740 2024-05-09T02:18:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.385+0000 7f733e3ac700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338108740 0x7f73381a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.385+0000 7f733e3ac700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338108740 0x7f73381a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:34988/0 (socket says 172.21.15.19:34988) 2024-05-09T02:18:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.386+0000 7f733e3ac700 1 -- 172.21.15.19:0/780384986 learned_addr learned my addr 172.21.15.19:0/780384986 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.386+0000 7f733dbab700 1 --2- 172.21.15.19:0/780384986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73381034c0 0x7f73381a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.386+0000 7f733e3ac700 1 -- 172.21.15.19:0/780384986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7338107df0 msgr2=0x7f73381a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.386+0000 7f733e3ac700 1 --2- 172.21.15.19:0/780384986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7338107df0 0x7f73381a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.386+0000 7f733e3ac700 1 -- 172.21.15.19:0/780384986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73381034c0 msgr2=0x7f73381a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.386+0000 7f733e3ac700 1 --2- 172.21.15.19:0/780384986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73381034c0 0x7f73381a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.386+0000 7f733e3ac700 1 -- 172.21.15.19:0/780384986 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73381a62e0 con 0x7f7338108740 2024-05-09T02:18:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.386+0000 7f733dbab700 1 --2- 172.21.15.19:0/780384986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73381034c0 0x7f73381a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:18:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.386+0000 7f733e3ac700 1 --2- 172.21.15.19:0/780384986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338108740 0x7f73381a5c40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f733000ef30 tx=0x7f733000c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.387+0000 7f732effd700 1 -- 172.21.15.19:0/780384986 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73300092e0 con 0x7f7338108740 2024-05-09T02:18:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.387+0000 7f732effd700 1 -- 172.21.15.19:0/780384986 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7330004bb0 con 0x7f7338108740 2024-05-09T02:18:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.387+0000 7f732effd700 1 -- 172.21.15.19:0/780384986 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330004280 con 0x7f7338108740 2024-05-09T02:18:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.387+0000 7f733fe0f700 1 -- 172.21.15.19:0/780384986 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73381a6570 con 0x7f7338108740 2024-05-09T02:18:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.388+0000 7f733fe0f700 1 -- 172.21.15.19:0/780384986 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73381a6a60 con 0x7f7338108740 2024-05-09T02:18:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.390+0000 7f732effd700 1 -- 172.21.15.19:0/780384986 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f733001b030 con 0x7f7338108740 2024-05-09T02:18:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.391+0000 7f732effd700 1 --2- 172.21.15.19:0/780384986 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f73240b2bf0 0x7f73240b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.392+0000 7f732effd700 1 -- 172.21.15.19:0/780384986 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f73300e2200 con 0x7f7338108740 2024-05-09T02:18:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.392+0000 7f733dbab700 1 --2- 172.21.15.19:0/780384986 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f73240b2bf0 0x7f73240b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.392+0000 7f734e825b80 1 Processor -- start 2024-05-09T02:18:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.392+0000 7f734e825b80 1 -- start start 2024-05-09T02:18:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.392+0000 7f734e825b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b84bc160 0x5590b85554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.392+0000 7f734e825b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b83d24a0 0x5590b8555a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.393+0000 7f734e825b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 0x5590b854fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.393+0000 7f734e825b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5590b8471ff0 con 0x5590b83d24a0 2024-05-09T02:18:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.393+0000 7f734e825b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5590b84107f0 con 0x5590b854f6a0 2024-05-09T02:18:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.393+0000 7f734e825b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5590b846dfb0 con 0x5590b84bc160 2024-05-09T02:18:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.393+0000 7f733e3ac700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 0x5590b854fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.393+0000 7f733e3ac700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 0x5590b854fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:60640/0 (socket says 172.21.15.19:60640) 2024-05-09T02:18:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.393+0000 7f733e3ac700 1 -- 172.21.15.19:0/2947676078 learned_addr learned my addr 172.21.15.19:0/2947676078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.394+0000 7f733e3ac700 1 -- 172.21.15.19:0/2947676078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b84bc160 msgr2=0x5590b85554f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:18:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.394+0000 7f733e3ac700 1 --2- 172.21.15.19:0/2947676078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b84bc160 0x5590b85554f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.394+0000 7f733e3ac700 1 -- 172.21.15.19:0/2947676078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b83d24a0 msgr2=0x5590b8555a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.394+0000 7f733e3ac700 1 --2- 172.21.15.19:0/2947676078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b83d24a0 0x5590b8555a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.394+0000 7f733e3ac700 1 -- 172.21.15.19:0/2947676078 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b8408370 con 0x5590b854f6a0 2024-05-09T02:18:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.394+0000 7f733e3ac700 1 --2- 172.21.15.19:0/2947676078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 0x5590b854fad0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f73300a5560 tx=0x7f73300a8c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6afa5aeb88e6808d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.394+0000 7f7323fff700 1 -- 172.21.15.19:0/2947676078 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73300a9be0 con 0x5590b854f6a0 2024-05-09T02:18:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.394+0000 7f733dbab700 1 --2- 172.21.15.19:0/780384986 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f73240b2bf0 0x7f73240b5100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7334009cb0 tx=0x7f7334009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.394+0000 7f7323fff700 1 -- 172.21.15.19:0/2947676078 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73300a94e0 con 0x5590b854f6a0 2024-05-09T02:18:48.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.395+0000 7f7323fff700 1 -- 172.21.15.19:0/2947676078 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73300b0b80 con 0x5590b854f6a0 2024-05-09T02:18:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.396+0000 7f734e825b80 1 -- 172.21.15.19:0/2947676078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 msgr2=0x5590b854fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.396+0000 7f734e825b80 1 --2- 172.21.15.19:0/2947676078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 0x5590b854fad0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f73300a5560 tx=0x7f73300a8c50 comp rx=0 tx=0).stop 2024-05-09T02:18:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.396+0000 7f734e825b80 1 -- 172.21.15.19:0/2947676078 shutdown_connections 2024-05-09T02:18:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.396+0000 7f734e825b80 1 --2- 172.21.15.19:0/2947676078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 0x5590b854fad0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.396+0000 7f734e825b80 1 --2- 172.21.15.19:0/2947676078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b84bc160 0x5590b85554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.396+0000 7f734e825b80 1 --2- 172.21.15.19:0/2947676078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b83d24a0 0x5590b8555a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.396+0000 7f734e825b80 1 -- 172.21.15.19:0/2947676078 >> 172.21.15.19:0/2947676078 conn(0x5590b84326a0 msgr2=0x5590b854a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.396+0000 7f734e825b80 1 -- 172.21.15.19:0/2947676078 shutdown_connections 2024-05-09T02:18:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.397+0000 7f734e825b80 1 -- 172.21.15.19:0/2947676078 wait complete. 2024-05-09T02:18:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.397+0000 7f734e825b80 1 -- start start 2024-05-09T02:18:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.397+0000 7f734e825b80 1 Processor -- start 2024-05-09T02:18:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.398+0000 7f734e825b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b83d24a0 0x5590b85592c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.398+0000 7f734e825b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b84bc160 0x5590b8559800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.398+0000 7f734e825b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b854f6a0 0x5590b8559d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.398+0000 7f734e825b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5590b82edba0 con 0x5590b83d24a0 2024-05-09T02:18:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.398+0000 7f734e825b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5590b842c150 con 0x5590b84bc160 2024-05-09T02:18:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.398+0000 7f734e825b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5590b855c1d0 con 0x5590b854f6a0 2024-05-09T02:18:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.398+0000 7f733dbab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b83d24a0 0x5590b85592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.398+0000 7f734e825b80 10 client.0 _ll_register_callbacks cb 0x5590b82e36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.398+0000 7f733dbab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b83d24a0 0x5590b85592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60682/0 (socket says 172.21.15.19:60682) 2024-05-09T02:18:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.398+0000 7f733dbab700 1 -- 172.21.15.19:0/188395898 learned_addr learned my addr 172.21.15.19:0/188395898 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.399+0000 7f733d3aa700 1 --2- 172.21.15.19:0/188395898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b84bc160 0x5590b8559800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.399+0000 7f733e3ac700 1 --2- 172.21.15.19:0/188395898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b854f6a0 0x5590b8559d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.399+0000 7f733e3ac700 1 -- 172.21.15.19:0/188395898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b84bc160 msgr2=0x5590b8559800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.399+0000 7f733e3ac700 1 --2- 172.21.15.19:0/188395898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b84bc160 0x5590b8559800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.399+0000 7f733e3ac700 1 -- 172.21.15.19:0/188395898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b83d24a0 msgr2=0x5590b85592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.399+0000 7f733e3ac700 1 --2- 172.21.15.19:0/188395898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b83d24a0 0x5590b85592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.399+0000 7f733e3ac700 1 -- 172.21.15.19:0/188395898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b82ed490 con 0x5590b854f6a0 2024-05-09T02:18:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.400+0000 7f733e3ac700 1 --2- 172.21.15.19:0/188395898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b854f6a0 0x5590b8559d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73300a5560 tx=0x7f7330003a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=860634656a9d6adc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:48.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.400+0000 7f73217fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:18:48.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.400+0000 7f73217fa700 1 -- 172.21.15.19:0/188395898 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73300b22b0 con 0x5590b854f6a0 2024-05-09T02:18:48.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.400+0000 7f73217fa700 1 -- 172.21.15.19:0/188395898 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f73300a5ad0 con 0x5590b854f6a0 2024-05-09T02:18:48.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.400+0000 7f734e825b80 1 -- 172.21.15.19:0/188395898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73300a94e0 con 0x5590b854f6a0 2024-05-09T02:18:48.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.401+0000 7f73217fa700 1 -- 172.21.15.19:0/188395898 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f73300b9050 con 0x5590b854f6a0 2024-05-09T02:18:48.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.401+0000 7f734e825b80 10 client.25993 fetch_fsmap learned FSMap version 48 2024-05-09T02:18:48.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.401+0000 7f734e825b80 1 -- 172.21.15.19:0/188395898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5590b8566ce0 con 0x5590b854f6a0 2024-05-09T02:18:48.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.401+0000 7f73217fa700 1 -- 172.21.15.19:0/188395898 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73300c8070 con 0x5590b854f6a0 2024-05-09T02:18:48.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.401+0000 7f73217fa700 1 -- 172.21.15.19:0/188395898 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f73300be450 con 0x5590b854f6a0 2024-05-09T02:18:48.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.401+0000 7f734e825b80 10 client.25993 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:18:48.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.401+0000 7f734e825b80 10 client.25993 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:18:48.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.401+0000 7f734e825b80 20 client.25993 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:48.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.401+0000 7f734e825b80 1 --2- 172.21.15.19:0/188395898 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5590b85691b0 0x5590b856b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.401+0000 7f734e825b80 4 client.25993 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:18:48.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.401+0000 7f734e825b80 1 -- 172.21.15.19:0/188395898 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f73300a94e0 con 0x5590b85691b0 2024-05-09T02:18:48.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.402+0000 7f733dbab700 1 --2- 172.21.15.19:0/188395898 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5590b85691b0 0x5590b856b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:48.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.402+0000 7f73217fa700 10 client.25993 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:18:48.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.403+0000 7f733dbab700 1 --2- 172.21.15.19:0/188395898 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5590b85691b0 0x5590b856b5d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e64dc60c5ef72ff server_cookie=64020a52f3e123d2 in_seq=0 out_seq=0 2024-05-09T02:18:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.403+0000 7f73217fa700 1 -- 172.21.15.19:0/188395898 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f73300a94e0 con 0x5590b85691b0 2024-05-09T02:18:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.403+0000 7f73217fa700 10 client.25993 handle_command_reply: tid=0 2024-05-09T02:18:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.403+0000 7f734e825b80 1 client.25993 shutdown 2024-05-09T02:18:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.404+0000 7f734e825b80 1 -- 172.21.15.19:0/188395898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b854f6a0 msgr2=0x5590b8559d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.404+0000 7f734e825b80 1 --2- 172.21.15.19:0/188395898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b854f6a0 0x5590b8559d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73300a5560 tx=0x7f7330003a70 comp rx=0 tx=0).stop 2024-05-09T02:18:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.404+0000 7f734e825b80 1 -- 172.21.15.19:0/188395898 shutdown_connections 2024-05-09T02:18:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.404+0000 7f734e825b80 1 --2- 172.21.15.19:0/188395898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b854f6a0 0x5590b8559d40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.404+0000 7f734e825b80 1 --2- 172.21.15.19:0/188395898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b83d24a0 0x5590b85592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.404+0000 7f734e825b80 1 --2- 172.21.15.19:0/188395898 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5590b85691b0 0x5590b856b5d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.404+0000 7f734e825b80 1 --2- 172.21.15.19:0/188395898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b84bc160 0x5590b8559800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.404+0000 7f73217fa700 0 client.25993 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:18:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.404+0000 7f734e825b80 1 -- 172.21.15.19:0/188395898 >> 172.21.15.19:0/188395898 conn(0x5590b84326a0 msgr2=0x5590b854a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.404+0000 7f734e825b80 1 -- 172.21.15.19:0/188395898 shutdown_connections 2024-05-09T02:18:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.404+0000 7f734e825b80 1 -- 172.21.15.19:0/188395898 wait complete. 2024-05-09T02:18:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.404+0000 7f734e825b80 20 client.25993 trim_cache size 0 max 16384 2024-05-09T02:18:48.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.431+0000 7f734e825b80 1 Processor -- start 2024-05-09T02:18:48.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.431+0000 7f734e825b80 1 -- start start 2024-05-09T02:18:48.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.432+0000 7f734e825b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b83d24a0 0x5590b85582c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.432+0000 7f734e825b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b84bc160 0x5590b8559280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.432+0000 7f734e825b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 0x5590b85597c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.432+0000 7f734e825b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b5610 con 0x5590b84bc160 2024-05-09T02:18:48.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.432+0000 7f734e825b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b71e0 con 0x5590b854f6a0 2024-05-09T02:18:48.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.432+0000 7f734e825b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b5d70 con 0x5590b83d24a0 2024-05-09T02:18:48.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.432+0000 7f733dbab700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b83d24a0 0x5590b85582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:48.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.432+0000 7f733dbab700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b83d24a0 0x5590b85582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:35010/0 (socket says 172.21.15.19:35010) 2024-05-09T02:18:48.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.432+0000 7f733dbab700 1 -- 172.21.15.19:0/3023795968 learned_addr learned my addr 172.21.15.19:0/3023795968 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:48.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.433+0000 7f733e3ac700 1 --2- 172.21.15.19:0/3023795968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 0x5590b85597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:48.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.433+0000 7f733d3aa700 1 --2- 172.21.15.19:0/3023795968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b84bc160 0x5590b8559280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:48.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.433+0000 7f733dbab700 1 -- 172.21.15.19:0/3023795968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 msgr2=0x5590b85597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:48.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.433+0000 7f733dbab700 1 --2- 172.21.15.19:0/3023795968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 0x5590b85597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.433+0000 7f733dbab700 1 -- 172.21.15.19:0/3023795968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b84bc160 msgr2=0x5590b8559280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:48.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.433+0000 7f733dbab700 1 --2- 172.21.15.19:0/3023795968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b84bc160 0x5590b8559280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.433+0000 7f733dbab700 1 -- 172.21.15.19:0/3023795968 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b8555050 con 0x5590b83d24a0 2024-05-09T02:18:48.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.433+0000 7f733dbab700 1 --2- 172.21.15.19:0/3023795968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b83d24a0 0x5590b85582c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7334011a50 tx=0x7f7334014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f0bedaa141b12f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:48.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.433+0000 7f73237fe700 1 -- 172.21.15.19:0/3023795968 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733402d070 con 0x5590b83d24a0 2024-05-09T02:18:48.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.433+0000 7f73237fe700 1 -- 172.21.15.19:0/3023795968 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7334016070 con 0x5590b83d24a0 2024-05-09T02:18:48.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.433+0000 7f73237fe700 1 -- 172.21.15.19:0/3023795968 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334012450 con 0x5590b83d24a0 2024-05-09T02:18:48.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.435+0000 7f734e825b80 1 -- 172.21.15.19:0/3023795968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b83d24a0 msgr2=0x5590b85582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:48.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.435+0000 7f734e825b80 1 --2- 172.21.15.19:0/3023795968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b83d24a0 0x5590b85582c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7334011a50 tx=0x7f7334014000 comp rx=0 tx=0).stop 2024-05-09T02:18:48.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.435+0000 7f734e825b80 1 -- 172.21.15.19:0/3023795968 shutdown_connections 2024-05-09T02:18:48.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.435+0000 7f734e825b80 1 --2- 172.21.15.19:0/3023795968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 0x5590b85597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.435+0000 7f734e825b80 1 --2- 172.21.15.19:0/3023795968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b83d24a0 0x5590b85582c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.435+0000 7f734e825b80 1 --2- 172.21.15.19:0/3023795968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b84bc160 0x5590b8559280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.435+0000 7f734e825b80 1 -- 172.21.15.19:0/3023795968 >> 172.21.15.19:0/3023795968 conn(0x5590b84326a0 msgr2=0x5590b8557bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:48.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.435+0000 7f734e825b80 1 -- 172.21.15.19:0/3023795968 shutdown_connections 2024-05-09T02:18:48.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.435+0000 7f734e825b80 1 -- 172.21.15.19:0/3023795968 wait complete. 2024-05-09T02:18:48.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.435+0000 7f734e825b80 1 -- start start 2024-05-09T02:18:48.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.436+0000 7f734e825b80 1 Processor -- start 2024-05-09T02:18:48.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.436+0000 7f734e825b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b83d24a0 0x5590b85560a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.436+0000 7f734e825b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b84bc160 0x5590b85565e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.436+0000 7f734e825b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 0x5590b85d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.436+0000 7f734e825b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b6b10 con 0x5590b84bc160 2024-05-09T02:18:48.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.436+0000 7f734e825b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b6440 con 0x5590b854f6a0 2024-05-09T02:18:48.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.436+0000 7f734e825b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b35d0 con 0x5590b83d24a0 2024-05-09T02:18:48.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.436+0000 7f734e825b80 10 client.0 _ll_register_callbacks cb 0x5590b82e36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:48.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.437+0000 7f733e3ac700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 0x5590b85d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:48.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.437+0000 7f733e3ac700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 0x5590b85d16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:60672/0 (socket says 172.21.15.19:60672) 2024-05-09T02:18:48.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.437+0000 7f733e3ac700 1 -- 172.21.15.19:0/3356510995 learned_addr learned my addr 172.21.15.19:0/3356510995 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:48.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.437+0000 7f733dbab700 1 --2- 172.21.15.19:0/3356510995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b83d24a0 0x5590b85560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:48.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.438+0000 7f733dbab700 1 -- 172.21.15.19:0/3356510995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 msgr2=0x5590b85d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:48.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.438+0000 7f733dbab700 1 --2- 172.21.15.19:0/3356510995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 0x5590b85d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.438+0000 7f733dbab700 1 -- 172.21.15.19:0/3356510995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b84bc160 msgr2=0x5590b85565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:48.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.438+0000 7f733dbab700 1 --2- 172.21.15.19:0/3356510995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b84bc160 0x5590b85565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.438+0000 7f733dbab700 1 -- 172.21.15.19:0/3356510995 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b8556ec0 con 0x5590b83d24a0 2024-05-09T02:18:48.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.439+0000 7f733dbab700 1 --2- 172.21.15.19:0/3356510995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b83d24a0 0x5590b85560a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7334000c00 tx=0x7f7334005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cb05cdc6a820918 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:48.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.439+0000 7f73217fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:18:48.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.439+0000 7f73217fa700 1 -- 172.21.15.19:0/3356510995 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334035020 con 0x5590b83d24a0 2024-05-09T02:18:48.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.439+0000 7f73217fa700 1 -- 172.21.15.19:0/3356510995 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f7334027780 con 0x5590b83d24a0 2024-05-09T02:18:48.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.439+0000 7f734e825b80 1 -- 172.21.15.19:0/3356510995 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7334016070 con 0x5590b83d24a0 2024-05-09T02:18:48.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.439+0000 7f73217fa700 1 -- 172.21.15.19:0/3356510995 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f733402d040 con 0x5590b83d24a0 2024-05-09T02:18:48.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.439+0000 7f734e825b80 10 client.25999 fetch_fsmap learned FSMap version 48 2024-05-09T02:18:48.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.439+0000 7f734e825b80 1 -- 172.21.15.19:0/3356510995 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5590b85d2cc0 con 0x5590b83d24a0 2024-05-09T02:18:48.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.439+0000 7f73217fa700 1 -- 172.21.15.19:0/3356510995 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7334016070 con 0x5590b83d24a0 2024-05-09T02:18:48.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.440+0000 7f73217fa700 1 -- 172.21.15.19:0/3356510995 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f7334030030 con 0x5590b83d24a0 2024-05-09T02:18:48.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.440+0000 7f734e825b80 10 client.25999 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:18:48.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.440+0000 7f734e825b80 10 client.25999 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:18:48.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.440+0000 7f734e825b80 20 client.25999 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:48.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.440+0000 7f734e825b80 1 --2- 172.21.15.19:0/3356510995 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5590b85d32e0 0x5590b85d5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:48.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.440+0000 7f734e825b80 4 client.25999 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:18:48.448 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:18:48.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.440+0000 7f734e825b80 1 -- 172.21.15.19:0/3356510995 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590b847f9d0 con 0x5590b85d32e0 2024-05-09T02:18:48.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.440+0000 7f733d3aa700 1 --2- 172.21.15.19:0/3356510995 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5590b85d32e0 0x5590b85d5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:48.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.440+0000 7f733d3aa700 1 --2- 172.21.15.19:0/3356510995 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5590b85d32e0 0x5590b85d5700 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=deb4d1cb67a33c41 server_cookie=a35bbe2c2ad83179 in_seq=0 out_seq=0 2024-05-09T02:18:48.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.440+0000 7f73217fa700 10 client.25999 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:18:48.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.441+0000 7f73217fa700 1 -- 172.21.15.19:0/3356510995 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590b847f9d0 con 0x5590b85d32e0 2024-05-09T02:18:48.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.441+0000 7f73217fa700 10 client.25999 handle_command_reply: tid=0 2024-05-09T02:18:48.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.441+0000 7f734e825b80 1 client.25999 shutdown 2024-05-09T02:18:48.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.441+0000 7f734e825b80 1 -- 172.21.15.19:0/3356510995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b83d24a0 msgr2=0x5590b85560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:48.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.441+0000 7f734e825b80 1 --2- 172.21.15.19:0/3356510995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b83d24a0 0x5590b85560a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7334000c00 tx=0x7f7334005760 comp rx=0 tx=0).stop 2024-05-09T02:18:48.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.441+0000 7f734e825b80 1 -- 172.21.15.19:0/3356510995 shutdown_connections 2024-05-09T02:18:48.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.441+0000 7f734e825b80 1 --2- 172.21.15.19:0/3356510995 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x5590b85d32e0 0x5590b85d5700 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.441+0000 7f734e825b80 1 --2- 172.21.15.19:0/3356510995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5590b854f6a0 0x5590b85d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.441+0000 7f734e825b80 1 --2- 172.21.15.19:0/3356510995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590b83d24a0 0x5590b85560a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.441+0000 7f734e825b80 1 --2- 172.21.15.19:0/3356510995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590b84bc160 0x5590b85565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.441+0000 7f734e825b80 1 -- 172.21.15.19:0/3356510995 >> 172.21.15.19:0/3356510995 conn(0x5590b84326a0 msgr2=0x5590b8552e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:48.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.441+0000 7f73217fa700 0 client.25999 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:18:48.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.441+0000 7f734e825b80 1 -- 172.21.15.19:0/3356510995 shutdown_connections 2024-05-09T02:18:48.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.441+0000 7f734e825b80 1 -- 172.21.15.19:0/3356510995 wait complete. 2024-05-09T02:18:48.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.441+0000 7f734e825b80 20 client.25999 trim_cache size 0 max 16384 2024-05-09T02:18:48.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.442+0000 7f733fe0f700 1 -- 172.21.15.19:0/780384986 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f73240b2bf0 msgr2=0x7f73240b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:48.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.442+0000 7f733fe0f700 1 --2- 172.21.15.19:0/780384986 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f73240b2bf0 0x7f73240b5100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7334009cb0 tx=0x7f7334009260 comp rx=0 tx=0).stop 2024-05-09T02:18:48.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.442+0000 7f733fe0f700 1 -- 172.21.15.19:0/780384986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338108740 msgr2=0x7f73381a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:48.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.442+0000 7f733fe0f700 1 --2- 172.21.15.19:0/780384986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338108740 0x7f73381a5c40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f733000ef30 tx=0x7f733000c450 comp rx=0 tx=0).stop 2024-05-09T02:18:48.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.442+0000 7f733fe0f700 1 -- 172.21.15.19:0/780384986 shutdown_connections 2024-05-09T02:18:48.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.442+0000 7f733fe0f700 1 --2- 172.21.15.19:0/780384986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338108740 0x7f73381a5c40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.442+0000 7f733fe0f700 1 --2- 172.21.15.19:0/780384986 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f73240b2bf0 0x7f73240b5100 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.442+0000 7f733fe0f700 1 --2- 172.21.15.19:0/780384986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73381034c0 0x7f73381a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.442+0000 7f733fe0f700 1 --2- 172.21.15.19:0/780384986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7338107df0 0x7f73381a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:48.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.442+0000 7f733fe0f700 1 -- 172.21.15.19:0/780384986 >> 172.21.15.19:0/780384986 conn(0x7f73380fedc0 msgr2=0x7f73381011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:48.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.442+0000 7f733fe0f700 1 -- 172.21.15.19:0/780384986 shutdown_connections 2024-05-09T02:18:48.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:48.442+0000 7f733fe0f700 1 -- 172.21.15.19:0/780384986 wait complete. 2024-05-09T02:18:48.462 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:48 smithi019 ceph-mon[30151]: pgmap v230: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.4 MiB/s wr, 488 op/s 2024-05-09T02:18:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:48 smithi059 ceph-mon[34781]: pgmap v230: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.4 MiB/s wr, 488 op/s 2024-05-09T02:18:48.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:48 smithi196 ceph-mon[34718]: pgmap v230: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.4 MiB/s wr, 488 op/s 2024-05-09T02:18:49.289 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T02:18:49.696 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:18:50.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:50 smithi059 ceph-mon[34781]: pgmap v231: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 10 MiB/s wr, 714 op/s 2024-05-09T02:18:50.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:50 smithi196 ceph-mon[34718]: pgmap v231: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 10 MiB/s wr, 714 op/s 2024-05-09T02:18:50.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:50 smithi019 ceph-mon[30151]: pgmap v231: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 10 MiB/s wr, 714 op/s 2024-05-09T02:18:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.540+0000 7f43555e3700 1 -- 172.21.15.19:0/2600453771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4350103250 msgr2=0x7f4350103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.540+0000 7f43555e3700 1 --2- 172.21.15.19:0/2600453771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4350103250 0x7f4350103660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4340009990 tx=0x7f4340009ca0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.540+0000 7f43555e3700 1 -- 172.21.15.19:0/2600453771 shutdown_connections 2024-05-09T02:18:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.540+0000 7f43555e3700 1 --2- 172.21.15.19:0/2600453771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43501086a0 0x7f4350108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.540+0000 7f43555e3700 1 --2- 172.21.15.19:0/2600453771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4350103250 0x7f4350103660 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.540+0000 7f43555e3700 1 --2- 172.21.15.19:0/2600453771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350103ba0 0x7f43501080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.540+0000 7f43555e3700 1 -- 172.21.15.19:0/2600453771 >> 172.21.15.19:0/2600453771 conn(0x7f43500fedc0 msgr2=0x7f43501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.540+0000 7f43555e3700 1 -- 172.21.15.19:0/2600453771 shutdown_connections 2024-05-09T02:18:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.540+0000 7f43555e3700 1 -- 172.21.15.19:0/2600453771 wait complete. 2024-05-09T02:18:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.541+0000 7f43555e3700 1 Processor -- start 2024-05-09T02:18:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.541+0000 7f43555e3700 1 -- start start 2024-05-09T02:18:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f43555e3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4350103250 0x7f43501a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f43555e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350103ba0 0x7f43501a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f43555e3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43501086a0 0x7f43501a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f43555e3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f435011aa70 con 0x7f4350103ba0 2024-05-09T02:18:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f43555e3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f435011a8f0 con 0x7f43501086a0 2024-05-09T02:18:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f43555e3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f435011abf0 con 0x7f4350103250 2024-05-09T02:18:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f434effd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4350103250 0x7f43501a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f434e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350103ba0 0x7f43501a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f434e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350103ba0 0x7f43501a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60714/0 (socket says 172.21.15.19:60714) 2024-05-09T02:18:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f434f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43501086a0 0x7f43501a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f434effd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4350103250 0x7f43501a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:35032/0 (socket says 172.21.15.19:35032) 2024-05-09T02:18:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f434e7fc700 1 -- 172.21.15.19:0/4104420313 learned_addr learned my addr 172.21.15.19:0/4104420313 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f434f7fe700 1 -- 172.21.15.19:0/4104420313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4350103250 msgr2=0x7f43501a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f434f7fe700 1 --2- 172.21.15.19:0/4104420313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4350103250 0x7f43501a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f434f7fe700 1 -- 172.21.15.19:0/4104420313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350103ba0 msgr2=0x7f43501a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f434f7fe700 1 --2- 172.21.15.19:0/4104420313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350103ba0 0x7f43501a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f434f7fe700 1 -- 172.21.15.19:0/4104420313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43501a6310 con 0x7f43501086a0 2024-05-09T02:18:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f434effd700 1 --2- 172.21.15.19:0/4104420313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4350103250 0x7f43501a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:18:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.542+0000 7f434f7fe700 1 --2- 172.21.15.19:0/4104420313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43501086a0 0x7f43501a5c70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f434800efb0 tx=0x7f434800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.543+0000 7f4337fff700 1 -- 172.21.15.19:0/4104420313 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348003a00 con 0x7f43501086a0 2024-05-09T02:18:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.543+0000 7f4337fff700 1 -- 172.21.15.19:0/4104420313 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f434800ce50 con 0x7f43501086a0 2024-05-09T02:18:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.543+0000 7f4337fff700 1 -- 172.21.15.19:0/4104420313 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43480047c0 con 0x7f43501086a0 2024-05-09T02:18:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.544+0000 7f43555e3700 1 -- 172.21.15.19:0/4104420313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43501a6600 con 0x7f43501086a0 2024-05-09T02:18:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.544+0000 7f43555e3700 1 -- 172.21.15.19:0/4104420313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43501adf20 con 0x7f43501086a0 2024-05-09T02:18:51.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.547+0000 7f4337fff700 1 -- 172.21.15.19:0/4104420313 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4348012050 con 0x7f43501086a0 2024-05-09T02:18:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.549+0000 7f4337fff700 1 --2- 172.21.15.19:0/4104420313 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f43380b2bc0 0x7f43380b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.549+0000 7f434effd700 1 --2- 172.21.15.19:0/4104420313 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f43380b2bc0 0x7f43380b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.549+0000 7f4337fff700 1 -- 172.21.15.19:0/4104420313 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f43480e1db0 con 0x7f43501086a0 2024-05-09T02:18:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.549+0000 7f434effd700 1 --2- 172.21.15.19:0/4104420313 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f43380b2bc0 0x7f43380b50d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4340009750 tx=0x7f434000b260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.550+0000 7f4363ff9b80 1 Processor -- start 2024-05-09T02:18:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.550+0000 7f4363ff9b80 1 -- start start 2024-05-09T02:18:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.550+0000 7f4363ff9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ad72160 0x561c7ae0b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.550+0000 7f4363ff9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ac884a0 0x561c7ae0ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.550+0000 7f4363ff9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ae056a0 0x561c7ae05ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.550+0000 7f434e7fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ac884a0 0x561c7ae0ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.550+0000 7f434effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ad72160 0x561c7ae0b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.550+0000 7f4363ff9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561c7ad27ff0 con 0x561c7ad72160 2024-05-09T02:18:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.550+0000 7f434e7fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ac884a0 0x561c7ae0ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:35038/0 (socket says 172.21.15.19:35038) 2024-05-09T02:18:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.550+0000 7f434e7fc700 1 -- 172.21.15.19:0/2783018260 learned_addr learned my addr 172.21.15.19:0/2783018260 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.550+0000 7f434f7fe700 1 --2- 172.21.15.19:0/2783018260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ae056a0 0x561c7ae05ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.550+0000 7f4363ff9b80 1 -- 172.21.15.19:0/2783018260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561c7acc67f0 con 0x561c7ae056a0 2024-05-09T02:18:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.550+0000 7f4363ff9b80 1 -- 172.21.15.19:0/2783018260 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c7ad23fb0 con 0x561c7ac884a0 2024-05-09T02:18:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.551+0000 7f434e7fc700 1 -- 172.21.15.19:0/2783018260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ae056a0 msgr2=0x561c7ae05ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:51.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.551+0000 7f434e7fc700 1 --2- 172.21.15.19:0/2783018260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ae056a0 0x561c7ae05ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.551+0000 7f434e7fc700 1 -- 172.21.15.19:0/2783018260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ad72160 msgr2=0x561c7ae0b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:51.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.551+0000 7f434e7fc700 1 --2- 172.21.15.19:0/2783018260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ad72160 0x561c7ae0b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.551+0000 7f434e7fc700 1 -- 172.21.15.19:0/2783018260 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c7acbe370 con 0x561c7ac884a0 2024-05-09T02:18:51.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.551+0000 7f434e7fc700 1 --2- 172.21.15.19:0/2783018260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ac884a0 0x561c7ae0ba30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4344009df0 tx=0x7f4344011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf8b2ac2b51f4919 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:51.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.551+0000 7f43357fa700 1 -- 172.21.15.19:0/2783018260 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344004230 con 0x561c7ac884a0 2024-05-09T02:18:51.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.551+0000 7f43357fa700 1 -- 172.21.15.19:0/2783018260 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43440043d0 con 0x561c7ac884a0 2024-05-09T02:18:51.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.551+0000 7f43357fa700 1 -- 172.21.15.19:0/2783018260 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344005680 con 0x561c7ac884a0 2024-05-09T02:18:51.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.553+0000 7f4363ff9b80 1 -- 172.21.15.19:0/2783018260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ac884a0 msgr2=0x561c7ae0ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:51.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.553+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/2783018260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ac884a0 0x561c7ae0ba30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4344009df0 tx=0x7f4344011720 comp rx=0 tx=0).stop 2024-05-09T02:18:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.553+0000 7f4363ff9b80 1 -- 172.21.15.19:0/2783018260 shutdown_connections 2024-05-09T02:18:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.553+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/2783018260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ae056a0 0x561c7ae05ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.553+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/2783018260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ad72160 0x561c7ae0b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.553+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/2783018260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ac884a0 0x561c7ae0ba30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.553+0000 7f4363ff9b80 1 -- 172.21.15.19:0/2783018260 >> 172.21.15.19:0/2783018260 conn(0x561c7ace86a0 msgr2=0x561c7ae00400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.554+0000 7f4363ff9b80 1 -- 172.21.15.19:0/2783018260 shutdown_connections 2024-05-09T02:18:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.554+0000 7f4363ff9b80 1 -- 172.21.15.19:0/2783018260 wait complete. 2024-05-09T02:18:51.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.554+0000 7f4363ff9b80 1 -- start start 2024-05-09T02:18:51.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.554+0000 7f4363ff9b80 1 Processor -- start 2024-05-09T02:18:51.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.554+0000 7f4363ff9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ac884a0 0x561c7ae0f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f4363ff9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ad72160 0x561c7ae0f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f4363ff9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ae056a0 0x561c7ae0fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f4363ff9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561c7aba3ba0 con 0x561c7ad72160 2024-05-09T02:18:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f4363ff9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561c7ace2150 con 0x561c7ac884a0 2024-05-09T02:18:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f4363ff9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c7ae121d0 con 0x561c7ae056a0 2024-05-09T02:18:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f4363ff9b80 10 client.0 _ll_register_callbacks cb 0x561c7ab996c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f434effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ac884a0 0x561c7ae0f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f434e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ad72160 0x561c7ae0f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f434effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ac884a0 0x561c7ae0f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:60706/0 (socket says 172.21.15.19:60706) 2024-05-09T02:18:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f434effd700 1 -- 172.21.15.19:0/3148012464 learned_addr learned my addr 172.21.15.19:0/3148012464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f434e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ad72160 0x561c7ae0f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60730/0 (socket says 172.21.15.19:60730) 2024-05-09T02:18:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f434f7fe700 1 --2- 172.21.15.19:0/3148012464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ae056a0 0x561c7ae0fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f434effd700 1 -- 172.21.15.19:0/3148012464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ae056a0 msgr2=0x561c7ae0fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f434effd700 1 --2- 172.21.15.19:0/3148012464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ae056a0 0x561c7ae0fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f434effd700 1 -- 172.21.15.19:0/3148012464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ad72160 msgr2=0x561c7ae0f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f434effd700 1 --2- 172.21.15.19:0/3148012464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ad72160 0x561c7ae0f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.555+0000 7f434effd700 1 -- 172.21.15.19:0/3148012464 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c7aba3490 con 0x561c7ac884a0 2024-05-09T02:18:51.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.556+0000 7f434effd700 1 --2- 172.21.15.19:0/3148012464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ac884a0 0x561c7ae0f1e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4340015000 tx=0x7f434001e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53e885cabdaeea59 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:51.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.556+0000 7f4332ff5700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:18:51.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.556+0000 7f4332ff5700 1 -- 172.21.15.19:0/3148012464 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434000f660 con 0x561c7ac884a0 2024-05-09T02:18:51.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.556+0000 7f4332ff5700 1 -- 172.21.15.19:0/3148012464 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f434000f800 con 0x561c7ac884a0 2024-05-09T02:18:51.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.556+0000 7f4363ff9b80 1 -- 172.21.15.19:0/3148012464 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43440043d0 con 0x561c7ac884a0 2024-05-09T02:18:51.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.557+0000 7f4332ff5700 1 -- 172.21.15.19:0/3148012464 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f434000fa00 con 0x561c7ac884a0 2024-05-09T02:18:51.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.557+0000 7f4363ff9b80 10 client.25970 fetch_fsmap learned FSMap version 48 2024-05-09T02:18:51.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.557+0000 7f4363ff9b80 1 -- 172.21.15.19:0/3148012464 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561c7ae1cbc0 con 0x561c7ac884a0 2024-05-09T02:18:51.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.557+0000 7f4332ff5700 1 -- 172.21.15.19:0/3148012464 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43440043d0 con 0x561c7ac884a0 2024-05-09T02:18:51.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.557+0000 7f4332ff5700 1 -- 172.21.15.19:0/3148012464 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f4340022070 con 0x561c7ac884a0 2024-05-09T02:18:51.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.557+0000 7f4363ff9b80 10 client.25970 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:18:51.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.557+0000 7f4363ff9b80 10 client.25970 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:18:51.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.557+0000 7f4363ff9b80 20 client.25970 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:51.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.557+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/3148012464 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c7ae1f1f0 0x561c7ae216c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.557+0000 7f4363ff9b80 4 client.25970 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:18:51.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.557+0000 7f4363ff9b80 1 -- 172.21.15.19:0/3148012464 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c7ad359d0 con 0x561c7ae1f1f0 2024-05-09T02:18:51.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.558+0000 7f434e7fc700 1 --2- 172.21.15.19:0/3148012464 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c7ae1f1f0 0x561c7ae216c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:51.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.558+0000 7f4332ff5700 10 client.25970 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:18:51.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.558+0000 7f434e7fc700 1 --2- 172.21.15.19:0/3148012464 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c7ae1f1f0 0x561c7ae216c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6efd558fb82ef27 server_cookie=8075b731dec7981e in_seq=0 out_seq=0 2024-05-09T02:18:51.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.559+0000 7f4332ff5700 1 -- 172.21.15.19:0/3148012464 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561c7ad359d0 con 0x561c7ae1f1f0 2024-05-09T02:18:51.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.559+0000 7f4332ff5700 10 client.25970 handle_command_reply: tid=0 2024-05-09T02:18:51.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.559+0000 7f4363ff9b80 1 client.25970 shutdown 2024-05-09T02:18:51.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.560+0000 7f4363ff9b80 1 -- 172.21.15.19:0/3148012464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ac884a0 msgr2=0x561c7ae0f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:51.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.560+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/3148012464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ac884a0 0x561c7ae0f1e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4340015000 tx=0x7f434001e000 comp rx=0 tx=0).stop 2024-05-09T02:18:51.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.560+0000 7f4363ff9b80 1 -- 172.21.15.19:0/3148012464 shutdown_connections 2024-05-09T02:18:51.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.560+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/3148012464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ae056a0 0x561c7ae0fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.560+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/3148012464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ac884a0 0x561c7ae0f1e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.560+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/3148012464 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c7ae1f1f0 0x561c7ae216c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.560+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/3148012464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ad72160 0x561c7ae0f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.560+0000 7f4363ff9b80 1 -- 172.21.15.19:0/3148012464 >> 172.21.15.19:0/3148012464 conn(0x561c7ace86a0 msgr2=0x561c7ae001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:51.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.562+0000 7f4332ff5700 0 client.25970 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:18:51.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.562+0000 7f4363ff9b80 1 -- 172.21.15.19:0/3148012464 shutdown_connections 2024-05-09T02:18:51.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.562+0000 7f4363ff9b80 1 -- 172.21.15.19:0/3148012464 wait complete. 2024-05-09T02:18:51.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.562+0000 7f4363ff9b80 20 client.25970 trim_cache size 0 max 16384 2024-05-09T02:18:51.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.584+0000 7f4363ff9b80 1 Processor -- start 2024-05-09T02:18:51.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.584+0000 7f4363ff9b80 1 -- start start 2024-05-09T02:18:51.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.584+0000 7f4363ff9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ac884a0 0x561c7ae0fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.584+0000 7f4363ff9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ad72160 0x561c7ae0cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.584+0000 7f4363ff9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ae056a0 0x561c7ae0e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.584+0000 7f4363ff9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b5510 con 0x561c7ac884a0 2024-05-09T02:18:51.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.584+0000 7f4363ff9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b70e0 con 0x561c7ae056a0 2024-05-09T02:18:51.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.584+0000 7f4363ff9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b5c70 con 0x561c7ad72160 2024-05-09T02:18:51.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.585+0000 7f434effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ac884a0 0x561c7ae0fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:51.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.585+0000 7f434effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ac884a0 0x561c7ae0fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60744/0 (socket says 172.21.15.19:60744) 2024-05-09T02:18:51.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.585+0000 7f434effd700 1 -- 172.21.15.19:0/3341969741 learned_addr learned my addr 172.21.15.19:0/3341969741 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:51.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.585+0000 7f434f7fe700 1 --2- 172.21.15.19:0/3341969741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ae056a0 0x561c7ae0e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:51.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.585+0000 7f434effd700 1 -- 172.21.15.19:0/3341969741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ad72160 msgr2=0x561c7ae0cdb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:18:51.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.585+0000 7f434effd700 1 --2- 172.21.15.19:0/3341969741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ad72160 0x561c7ae0cdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.585+0000 7f434effd700 1 -- 172.21.15.19:0/3341969741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ae056a0 msgr2=0x561c7ae0e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:51.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.585+0000 7f434effd700 1 --2- 172.21.15.19:0/3341969741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ae056a0 0x561c7ae0e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.585+0000 7f434effd700 1 -- 172.21.15.19:0/3341969741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c7ae010f0 con 0x561c7ac884a0 2024-05-09T02:18:51.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.585+0000 7f434f7fe700 1 --2- 172.21.15.19:0/3341969741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ae056a0 0x561c7ae0e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:18:51.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.586+0000 7f434effd700 1 --2- 172.21.15.19:0/3341969741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ac884a0 0x561c7ae0fbd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4340000c00 tx=0x7f434000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c675bb279e74fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:51.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.586+0000 7f4334ff9700 1 -- 172.21.15.19:0/3341969741 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43400316c0 con 0x561c7ac884a0 2024-05-09T02:18:51.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.586+0000 7f4334ff9700 1 -- 172.21.15.19:0/3341969741 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f434000fda0 con 0x561c7ac884a0 2024-05-09T02:18:51.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.586+0000 7f4334ff9700 1 -- 172.21.15.19:0/3341969741 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434003b6d0 con 0x561c7ac884a0 2024-05-09T02:18:51.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.587+0000 7f4363ff9b80 1 -- 172.21.15.19:0/3341969741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ac884a0 msgr2=0x561c7ae0fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.587+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/3341969741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ac884a0 0x561c7ae0fbd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4340000c00 tx=0x7f434000b480 comp rx=0 tx=0).stop 2024-05-09T02:18:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.587+0000 7f4363ff9b80 1 -- 172.21.15.19:0/3341969741 shutdown_connections 2024-05-09T02:18:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.587+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/3341969741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ae056a0 0x561c7ae0e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.587+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/3341969741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ac884a0 0x561c7ae0fbd0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.587+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/3341969741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ad72160 0x561c7ae0cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.587+0000 7f4363ff9b80 1 -- 172.21.15.19:0/3341969741 >> 172.21.15.19:0/3341969741 conn(0x561c7ace86a0 msgr2=0x561c7ae0d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.587+0000 7f4363ff9b80 1 -- 172.21.15.19:0/3341969741 shutdown_connections 2024-05-09T02:18:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.587+0000 7f4363ff9b80 1 -- 172.21.15.19:0/3341969741 wait complete. 2024-05-09T02:18:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.587+0000 7f4363ff9b80 1 -- start start 2024-05-09T02:18:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.587+0000 7f4363ff9b80 1 Processor -- start 2024-05-09T02:18:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.588+0000 7f4363ff9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ac884a0 0x561c7ae17230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.588+0000 7f4363ff9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ad72160 0x561c7ae17770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.588+0000 7f4363ff9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ae056a0 0x561c7ae25e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.588+0000 7f4363ff9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b6a10 con 0x561c7ad72160 2024-05-09T02:18:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.588+0000 7f4363ff9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b6340 con 0x561c7ac884a0 2024-05-09T02:18:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.588+0000 7f4363ff9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b34d0 con 0x561c7ae056a0 2024-05-09T02:18:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.588+0000 7f4363ff9b80 10 client.0 _ll_register_callbacks cb 0x561c7ab996c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.588+0000 7f434f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ae056a0 0x561c7ae25e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.588+0000 7f434f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ae056a0 0x561c7ae25e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:35062/0 (socket says 172.21.15.19:35062) 2024-05-09T02:18:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.588+0000 7f434f7fe700 1 -- 172.21.15.19:0/1294284534 learned_addr learned my addr 172.21.15.19:0/1294284534 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.589+0000 7f434f7fe700 1 -- 172.21.15.19:0/1294284534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ac884a0 msgr2=0x561c7ae17230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:18:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.589+0000 7f434f7fe700 1 --2- 172.21.15.19:0/1294284534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ac884a0 0x561c7ae17230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.589+0000 7f434f7fe700 1 -- 172.21.15.19:0/1294284534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ad72160 msgr2=0x561c7ae17770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:18:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.589+0000 7f434f7fe700 1 --2- 172.21.15.19:0/1294284534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ad72160 0x561c7ae17770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.589+0000 7f434f7fe700 1 -- 172.21.15.19:0/1294284534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c7adf9f60 con 0x561c7ae056a0 2024-05-09T02:18:51.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.589+0000 7f434f7fe700 1 --2- 172.21.15.19:0/1294284534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ae056a0 0x561c7ae25e70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f43480a8d90 tx=0x7f43480a8ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70170f20b369bc1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:51.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.589+0000 7f4332ff5700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:18:51.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.590+0000 7f4332ff5700 1 -- 172.21.15.19:0/1294284534 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43480df900 con 0x561c7ae056a0 2024-05-09T02:18:51.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.590+0000 7f4332ff5700 1 -- 172.21.15.19:0/1294284534 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f4348003de0 con 0x561c7ae056a0 2024-05-09T02:18:51.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.590+0000 7f4363ff9b80 1 -- 172.21.15.19:0/1294284534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f434000fda0 con 0x561c7ae056a0 2024-05-09T02:18:51.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.591+0000 7f4332ff5700 1 -- 172.21.15.19:0/1294284534 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f43480b1e60 con 0x561c7ae056a0 2024-05-09T02:18:51.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.591+0000 7f4363ff9b80 10 client.26014 fetch_fsmap learned FSMap version 48 2024-05-09T02:18:51.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.591+0000 7f4363ff9b80 1 -- 172.21.15.19:0/1294284534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561c7ae88af0 con 0x561c7ae056a0 2024-05-09T02:18:51.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.591+0000 7f4332ff5700 1 -- 172.21.15.19:0/1294284534 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f434000fda0 con 0x561c7ae056a0 2024-05-09T02:18:51.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.591+0000 7f4332ff5700 1 -- 172.21.15.19:0/1294284534 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f43480b19d0 con 0x561c7ae056a0 2024-05-09T02:18:51.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.591+0000 7f4363ff9b80 10 client.26014 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:18:51.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.591+0000 7f4363ff9b80 10 client.26014 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:18:51.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.591+0000 7f4363ff9b80 20 client.26014 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:51.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.591+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/1294284534 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c7ae891d0 0x561c7ae8b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:51.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.591+0000 7f4363ff9b80 4 client.26014 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:18:51.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.591+0000 7f4363ff9b80 1 -- 172.21.15.19:0/1294284534 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c7ad2fa70 con 0x561c7ae891d0 2024-05-09T02:18:51.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.592+0000 7f434effd700 1 --2- 172.21.15.19:0/1294284534 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c7ae891d0 0x561c7ae8b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:51.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.592+0000 7f4332ff5700 10 client.26014 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:18:51.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.592+0000 7f434effd700 1 --2- 172.21.15.19:0/1294284534 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c7ae891d0 0x561c7ae8b630 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe4cfce7e7afaa0c server_cookie=b2d38713ef470c67 in_seq=0 out_seq=0 2024-05-09T02:18:51.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.593+0000 7f4332ff5700 1 -- 172.21.15.19:0/1294284534 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c7ad2fa70 con 0x561c7ae891d0 2024-05-09T02:18:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.593+0000 7f4332ff5700 10 client.26014 handle_command_reply: tid=0 2024-05-09T02:18:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.593+0000 7f4363ff9b80 1 client.26014 shutdown 2024-05-09T02:18:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.593+0000 7f4363ff9b80 1 -- 172.21.15.19:0/1294284534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ae056a0 msgr2=0x561c7ae25e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.593+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/1294284534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ae056a0 0x561c7ae25e70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f43480a8d90 tx=0x7f43480a8ce0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.593+0000 7f4363ff9b80 1 -- 172.21.15.19:0/1294284534 shutdown_connections 2024-05-09T02:18:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.593+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/1294284534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c7ae056a0 0x561c7ae25e70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.593+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/1294284534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c7ac884a0 0x561c7ae17230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.593+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/1294284534 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x561c7ae891d0 0x561c7ae8b630 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.593+0000 7f4363ff9b80 1 --2- 172.21.15.19:0/1294284534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561c7ad72160 0x561c7ae17770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.593+0000 7f4363ff9b80 1 -- 172.21.15.19:0/1294284534 >> 172.21.15.19:0/1294284534 conn(0x561c7ace86a0 msgr2=0x561c7ae0f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.593+0000 7f4332ff5700 0 client.26014 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:18:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.593+0000 7f4363ff9b80 1 -- 172.21.15.19:0/1294284534 shutdown_connections 2024-05-09T02:18:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.593+0000 7f4363ff9b80 1 -- 172.21.15.19:0/1294284534 wait complete. 2024-05-09T02:18:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.593+0000 7f4363ff9b80 20 client.26014 trim_cache size 0 max 16384 2024-05-09T02:18:51.599 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:18:51.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.594+0000 7f43555e3700 1 -- 172.21.15.19:0/4104420313 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f43380b2bc0 msgr2=0x7f43380b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:51.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.594+0000 7f43555e3700 1 --2- 172.21.15.19:0/4104420313 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f43380b2bc0 0x7f43380b50d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4340009750 tx=0x7f434000b260 comp rx=0 tx=0).stop 2024-05-09T02:18:51.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.594+0000 7f43555e3700 1 -- 172.21.15.19:0/4104420313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43501086a0 msgr2=0x7f43501a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:51.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.594+0000 7f43555e3700 1 --2- 172.21.15.19:0/4104420313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43501086a0 0x7f43501a5c70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f434800efb0 tx=0x7f434800c3a0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.594+0000 7f43555e3700 1 -- 172.21.15.19:0/4104420313 shutdown_connections 2024-05-09T02:18:51.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.594+0000 7f43555e3700 1 --2- 172.21.15.19:0/4104420313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43501086a0 0x7f43501a5c70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.594+0000 7f43555e3700 1 --2- 172.21.15.19:0/4104420313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4350103250 0x7f43501a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.594+0000 7f43555e3700 1 --2- 172.21.15.19:0/4104420313 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f43380b2bc0 0x7f43380b50d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.594+0000 7f43555e3700 1 --2- 172.21.15.19:0/4104420313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350103ba0 0x7f43501a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:51.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.594+0000 7f43555e3700 1 -- 172.21.15.19:0/4104420313 >> 172.21.15.19:0/4104420313 conn(0x7f43500fedc0 msgr2=0x7f43501060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:51.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.594+0000 7f43555e3700 1 -- 172.21.15.19:0/4104420313 shutdown_connections 2024-05-09T02:18:51.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:51.594+0000 7f43555e3700 1 -- 172.21.15.19:0/4104420313 wait complete. 2024-05-09T02:18:52.487 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:52 smithi019 ceph-mon[30151]: pgmap v232: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 8.0 MiB/s wr, 454 op/s 2024-05-09T02:18:52.535 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T02:18:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 430.0 s, 2011.4 tps, lat 49.991 ms stddev 54.444 2024-05-09T02:18:52.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:52 smithi059 ceph-mon[34781]: pgmap v232: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 8.0 MiB/s wr, 454 op/s 2024-05-09T02:18:52.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:52 smithi196 ceph-mon[34718]: pgmap v232: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 8.0 MiB/s wr, 454 op/s 2024-05-09T02:18:52.917 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:18:54.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:54 smithi019 ceph-mon[30151]: pgmap v233: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 8.4 MiB/s wr, 464 op/s 2024-05-09T02:18:54.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:54 smithi059 ceph-mon[34781]: pgmap v233: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 8.4 MiB/s wr, 464 op/s 2024-05-09T02:18:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:54 smithi196 ceph-mon[34718]: pgmap v233: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 8.4 MiB/s wr, 464 op/s 2024-05-09T02:18:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.966+0000 7f3e4cb59700 1 -- 172.21.15.19:0/3359959177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48102880 msgr2=0x7f3e48102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.966+0000 7f3e4cb59700 1 --2- 172.21.15.19:0/3359959177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48102880 0x7f3e48102c90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009990 tx=0x7f3e3c009ca0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.966+0000 7f3e4cb59700 1 -- 172.21.15.19:0/3359959177 shutdown_connections 2024-05-09T02:18:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.966+0000 7f3e4cb59700 1 --2- 172.21.15.19:0/3359959177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e481031d0 0x7f3e48111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.966+0000 7f3e4cb59700 1 --2- 172.21.15.19:0/3359959177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e48109300 0x7f3e48109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.966+0000 7f3e4cb59700 1 --2- 172.21.15.19:0/3359959177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48102880 0x7f3e48102c90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.966+0000 7f3e4cb59700 1 -- 172.21.15.19:0/3359959177 >> 172.21.15.19:0/3359959177 conn(0x7f3e480fe6f0 msgr2=0x7f3e48100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.966+0000 7f3e4cb59700 1 -- 172.21.15.19:0/3359959177 shutdown_connections 2024-05-09T02:18:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.966+0000 7f3e4cb59700 1 -- 172.21.15.19:0/3359959177 wait complete. 2024-05-09T02:18:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.968+0000 7f3e4cb59700 1 Processor -- start 2024-05-09T02:18:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.968+0000 7f3e4cb59700 1 -- start start 2024-05-09T02:18:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.968+0000 7f3e4cb59700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e48102880 0x7f3e48105600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.968+0000 7f3e4cb59700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e481031d0 0x7f3e48103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.968+0000 7f3e4cb59700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48109300 0x7f3e48104040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.968+0000 7f3e4cb59700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481145a0 con 0x7f3e48109300 2024-05-09T02:18:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.968+0000 7f3e4cb59700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3e48114420 con 0x7f3e481031d0 2024-05-09T02:18:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.968+0000 7f3e4cb59700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e48114720 con 0x7f3e48102880 2024-05-09T02:18:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.968+0000 7f3e45d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e481031d0 0x7f3e48103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.968+0000 7f3e4659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e48102880 0x7f3e48105600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.968+0000 7f3e45d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e481031d0 0x7f3e48103a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:60734/0 (socket says 172.21.15.19:60734) 2024-05-09T02:18:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.968+0000 7f3e45d9b700 1 -- 172.21.15.19:0/2025486077 learned_addr learned my addr 172.21.15.19:0/2025486077 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.968+0000 7f3e46d9d700 1 --2- 172.21.15.19:0/2025486077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48109300 0x7f3e48104040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.969+0000 7f3e4659c700 1 -- 172.21.15.19:0/2025486077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e481031d0 msgr2=0x7f3e48103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.969+0000 7f3e4659c700 1 --2- 172.21.15.19:0/2025486077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e481031d0 0x7f3e48103a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.969+0000 7f3e4659c700 1 -- 172.21.15.19:0/2025486077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48109300 msgr2=0x7f3e48104040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.969+0000 7f3e4659c700 1 --2- 172.21.15.19:0/2025486077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48109300 0x7f3e48104040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.969+0000 7f3e4659c700 1 -- 172.21.15.19:0/2025486077 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e48104870 con 0x7f3e48102880 2024-05-09T02:18:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.969+0000 7f3e4659c700 1 --2- 172.21.15.19:0/2025486077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e48102880 0x7f3e48105600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3e4806b050 tx=0x7f3e3800eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.969+0000 7f3e377fe700 1 -- 172.21.15.19:0/2025486077 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38004220 con 0x7f3e48102880 2024-05-09T02:18:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.969+0000 7f3e377fe700 1 -- 172.21.15.19:0/2025486077 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e380043c0 con 0x7f3e48102880 2024-05-09T02:18:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.969+0000 7f3e4cb59700 1 -- 172.21.15.19:0/2025486077 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e481ad980 con 0x7f3e48102880 2024-05-09T02:18:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.969+0000 7f3e377fe700 1 -- 172.21.15.19:0/2025486077 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38003ce0 con 0x7f3e48102880 2024-05-09T02:18:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.969+0000 7f3e4cb59700 1 -- 172.21.15.19:0/2025486077 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e481ade90 con 0x7f3e48102880 2024-05-09T02:18:54.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.974+0000 7f3e377fe700 1 -- 172.21.15.19:0/2025486077 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3e38018430 con 0x7f3e48102880 2024-05-09T02:18:54.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.975+0000 7f3e377fe700 1 --2- 172.21.15.19:0/2025486077 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3e300b2a70 0x7f3e300b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:54.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.975+0000 7f3e5b56fb80 1 Processor -- start 2024-05-09T02:18:54.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.975+0000 7f3e377fe700 1 -- 172.21.15.19:0/2025486077 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f3e380e26b0 con 0x7f3e48102880 2024-05-09T02:18:54.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e5b56fb80 1 -- start start 2024-05-09T02:18:54.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e45d9b700 1 --2- 172.21.15.19:0/2025486077 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3e300b2a70 0x7f3e300b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:54.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e5b56fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 0x55d3d65fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:54.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e5b56fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d647a4a0 0x55d3d65fda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:54.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e5b56fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d65f76a0 0x55d3d65f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:54.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e5b56fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d3d6519ff0 con 0x55d3d65f76a0 2024-05-09T02:18:54.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e5b56fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d3d64b87f0 con 0x55d3d6564160 2024-05-09T02:18:54.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e5b56fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d3d6515fb0 con 0x55d3d647a4a0 2024-05-09T02:18:54.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e4659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 0x55d3d65fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e4659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 0x55d3d65fd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:60746/0 (socket says 172.21.15.19:60746) 2024-05-09T02:18:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e4659c700 1 -- 172.21.15.19:0/3640119266 learned_addr learned my addr 172.21.15.19:0/3640119266 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e46d9d700 1 --2- 172.21.15.19:0/3640119266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d65f76a0 0x55d3d65f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e45d9b700 1 --2- 172.21.15.19:0/2025486077 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3e300b2a70 0x7f3e300b4f80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00b2e0 tx=0x7f3e3c009590 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e45d9b700 1 --2- 172.21.15.19:0/3640119266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d647a4a0 0x55d3d65fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e4659c700 1 -- 172.21.15.19:0/3640119266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d647a4a0 msgr2=0x55d3d65fda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e4659c700 1 --2- 172.21.15.19:0/3640119266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d647a4a0 0x55d3d65fda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e4659c700 1 -- 172.21.15.19:0/3640119266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d65f76a0 msgr2=0x55d3d65f7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.976+0000 7f3e4659c700 1 --2- 172.21.15.19:0/3640119266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d65f76a0 0x55d3d65f7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.977+0000 7f3e4659c700 1 -- 172.21.15.19:0/3640119266 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3d64b0370 con 0x55d3d6564160 2024-05-09T02:18:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.977+0000 7f3e45d9b700 1 --2- 172.21.15.19:0/3640119266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d647a4a0 0x55d3d65fda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:18:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.977+0000 7f3e4659c700 1 --2- 172.21.15.19:0/3640119266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 0x55d3d65fd4f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3e380a59e0 tx=0x7f3e38003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79429ecd2486d488 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.977+0000 7f3e34ff9700 1 -- 172.21.15.19:0/3640119266 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380aa060 con 0x55d3d6564160 2024-05-09T02:18:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.977+0000 7f3e34ff9700 1 -- 172.21.15.19:0/3640119266 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e380a9960 con 0x55d3d6564160 2024-05-09T02:18:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.977+0000 7f3e34ff9700 1 -- 172.21.15.19:0/3640119266 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380a9bc0 con 0x55d3d6564160 2024-05-09T02:18:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.978+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3640119266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 msgr2=0x55d3d65fd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.978+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3640119266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 0x55d3d65fd4f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3e380a59e0 tx=0x7f3e38003680 comp rx=0 tx=0).stop 2024-05-09T02:18:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.979+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3640119266 shutdown_connections 2024-05-09T02:18:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.979+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3640119266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d65f76a0 0x55d3d65f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.979+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3640119266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 0x55d3d65fd4f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.979+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3640119266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d647a4a0 0x55d3d65fda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.979+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3640119266 >> 172.21.15.19:0/3640119266 conn(0x55d3d64da6a0 msgr2=0x55d3d65f2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:54.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.980+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3640119266 shutdown_connections 2024-05-09T02:18:54.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.980+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3640119266 wait complete. 2024-05-09T02:18:54.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.980+0000 7f3e5b56fb80 1 -- start start 2024-05-09T02:18:54.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.980+0000 7f3e5b56fb80 1 Processor -- start 2024-05-09T02:18:54.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e5b56fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d647a4a0 0x55d3d6601230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:54.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e5b56fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d6564160 0x55d3d6601770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:54.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e5b56fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d65f76a0 0x55d3d6601cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:54.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e5b56fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d3d6395ba0 con 0x55d3d647a4a0 2024-05-09T02:18:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e5b56fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d3d64d4150 con 0x55d3d65f76a0 2024-05-09T02:18:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e5b56fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d3d66041d0 con 0x55d3d6564160 2024-05-09T02:18:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e5b56fb80 10 client.0 _ll_register_callbacks cb 0x55d3d638b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e4659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d647a4a0 0x55d3d6601230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e4659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d647a4a0 0x55d3d6601230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60780/0 (socket says 172.21.15.19:60780) 2024-05-09T02:18:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e4659c700 1 -- 172.21.15.19:0/3751683998 learned_addr learned my addr 172.21.15.19:0/3751683998 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e46d9d700 1 --2- 172.21.15.19:0/3751683998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d65f76a0 0x55d3d6601cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e45d9b700 1 --2- 172.21.15.19:0/3751683998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d6564160 0x55d3d6601770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e46d9d700 1 -- 172.21.15.19:0/3751683998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d6564160 msgr2=0x55d3d6601770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e46d9d700 1 --2- 172.21.15.19:0/3751683998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d6564160 0x55d3d6601770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e46d9d700 1 -- 172.21.15.19:0/3751683998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d647a4a0 msgr2=0x55d3d6601230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e46d9d700 1 --2- 172.21.15.19:0/3751683998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d647a4a0 0x55d3d6601230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e46d9d700 1 -- 172.21.15.19:0/3751683998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3d6395490 con 0x55d3d65f76a0 2024-05-09T02:18:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e45d9b700 1 --2- 172.21.15.19:0/3751683998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d6564160 0x55d3d6601770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:18:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.981+0000 7f3e4659c700 1 --2- 172.21.15.19:0/3751683998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d647a4a0 0x55d3d6601230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:18:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.982+0000 7f3e46d9d700 1 --2- 172.21.15.19:0/3751683998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d65f76a0 0x55d3d6601cb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3e40007ad0 tx=0x7f3e40007de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=946dbbeb0ee3f6b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.982+0000 7f3e2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:18:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.982+0000 7f3e2dffb700 1 -- 172.21.15.19:0/3751683998 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4001a070 con 0x55d3d65f76a0 2024-05-09T02:18:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.982+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3751683998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e380a9960 con 0x55d3d65f76a0 2024-05-09T02:18:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.982+0000 7f3e2dffb700 1 -- 172.21.15.19:0/3751683998 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f3e40004bb0 con 0x55d3d65f76a0 2024-05-09T02:18:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.982+0000 7f3e2dffb700 1 -- 172.21.15.19:0/3751683998 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f3e40025050 con 0x55d3d65f76a0 2024-05-09T02:18:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.982+0000 7f3e5b56fb80 10 client.25985 fetch_fsmap learned FSMap version 48 2024-05-09T02:18:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.982+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3751683998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d3d660ecc0 con 0x55d3d65f76a0 2024-05-09T02:18:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.982+0000 7f3e2dffb700 1 -- 172.21.15.19:0/3751683998 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e380a9960 con 0x55d3d65f76a0 2024-05-09T02:18:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.983+0000 7f3e2dffb700 1 -- 172.21.15.19:0/3751683998 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f3e40004210 con 0x55d3d65f76a0 2024-05-09T02:18:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.983+0000 7f3e5b56fb80 10 client.25985 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:18:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.983+0000 7f3e5b56fb80 10 client.25985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:18:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.983+0000 7f3e5b56fb80 20 client.25985 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.983+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3751683998 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d3d66112f0 0x55d3d6613760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.983+0000 7f3e5b56fb80 4 client.25985 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:18:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.983+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3751683998 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3d65279d0 con 0x55d3d66112f0 2024-05-09T02:18:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.983+0000 7f3e4659c700 1 --2- 172.21.15.19:0/3751683998 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d3d66112f0 0x55d3d6613760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.983+0000 7f3e2dffb700 10 client.25985 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.984+0000 7f3e4659c700 1 --2- 172.21.15.19:0/3751683998 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d3d66112f0 0x55d3d6613760 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ab6439b5d4a7747 server_cookie=6472e92cd68adba4 in_seq=0 out_seq=0 2024-05-09T02:18:54.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.984+0000 7f3e2dffb700 1 -- 172.21.15.19:0/3751683998 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d3d65279d0 con 0x55d3d66112f0 2024-05-09T02:18:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.984+0000 7f3e2dffb700 10 client.25985 handle_command_reply: tid=0 2024-05-09T02:18:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.985+0000 7f3e5b56fb80 1 client.25985 shutdown 2024-05-09T02:18:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.985+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3751683998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d65f76a0 msgr2=0x55d3d6601cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.985+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3751683998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d65f76a0 0x55d3d6601cb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3e40007ad0 tx=0x7f3e40007de0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.985+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3751683998 shutdown_connections 2024-05-09T02:18:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.985+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3751683998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d65f76a0 0x55d3d6601cb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.985+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3751683998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d647a4a0 0x55d3d6601230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.985+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3751683998 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d3d66112f0 0x55d3d6613760 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.985+0000 7f3e2dffb700 0 client.25985 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.985+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3751683998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d6564160 0x55d3d6601770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.985+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3751683998 >> 172.21.15.19:0/3751683998 conn(0x55d3d64da6a0 msgr2=0x55d3d65f21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.985+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3751683998 shutdown_connections 2024-05-09T02:18:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.985+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3751683998 wait complete. 2024-05-09T02:18:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:54.986+0000 7f3e5b56fb80 20 client.25985 trim_cache size 0 max 16384 2024-05-09T02:18:55.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.007+0000 7f3e5b56fb80 1 Processor -- start 2024-05-09T02:18:55.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.007+0000 7f3e5b56fb80 1 -- start start 2024-05-09T02:18:55.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.007+0000 7f3e5b56fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d647a4a0 0x55d3d65f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:55.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.007+0000 7f3e5b56fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 0x55d3d6618f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:55.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.007+0000 7f3e5b56fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d65f76a0 0x55d3d6619460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:55.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.007+0000 7f3e5b56fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2800d690 con 0x55d3d65f76a0 2024-05-09T02:18:55.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.007+0000 7f3e5b56fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2800f260 con 0x55d3d6564160 2024-05-09T02:18:55.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.007+0000 7f3e5b56fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2800ddf0 con 0x55d3d647a4a0 2024-05-09T02:18:55.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.007+0000 7f3e4659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d647a4a0 0x55d3d65f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:55.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.007+0000 7f3e4659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d647a4a0 0x55d3d65f70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:35102/0 (socket says 172.21.15.19:35102) 2024-05-09T02:18:55.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.007+0000 7f3e4659c700 1 -- 172.21.15.19:0/2343388019 learned_addr learned my addr 172.21.15.19:0/2343388019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:55.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.007+0000 7f3e45d9b700 1 --2- 172.21.15.19:0/2343388019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 0x55d3d6618f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:55.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.007+0000 7f3e46d9d700 1 --2- 172.21.15.19:0/2343388019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d65f76a0 0x55d3d6619460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.008+0000 7f3e4659c700 1 -- 172.21.15.19:0/2343388019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 msgr2=0x55d3d6618f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.008+0000 7f3e4659c700 1 --2- 172.21.15.19:0/2343388019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 0x55d3d6618f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.008+0000 7f3e4659c700 1 -- 172.21.15.19:0/2343388019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d65f76a0 msgr2=0x55d3d6619460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.008+0000 7f3e4659c700 1 --2- 172.21.15.19:0/2343388019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d65f76a0 0x55d3d6619460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.008+0000 7f3e4659c700 1 -- 172.21.15.19:0/2343388019 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3d65fd050 con 0x55d3d647a4a0 2024-05-09T02:18:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.008+0000 7f3e4659c700 1 --2- 172.21.15.19:0/2343388019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d647a4a0 0x55d3d65f70c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3e380e0350 tx=0x7f3e38004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec13869e3b8b87f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.009+0000 7f3e2ffff700 1 -- 172.21.15.19:0/2343388019 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380aa060 con 0x55d3d647a4a0 2024-05-09T02:18:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.009+0000 7f3e2ffff700 1 -- 172.21.15.19:0/2343388019 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e380add10 con 0x55d3d647a4a0 2024-05-09T02:18:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.009+0000 7f3e2ffff700 1 -- 172.21.15.19:0/2343388019 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380a52d0 con 0x55d3d647a4a0 2024-05-09T02:18:55.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.010+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/2343388019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d647a4a0 msgr2=0x55d3d65f70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:55.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.010+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/2343388019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d647a4a0 0x55d3d65f70c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3e380e0350 tx=0x7f3e38004570 comp rx=0 tx=0).stop 2024-05-09T02:18:55.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.010+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/2343388019 shutdown_connections 2024-05-09T02:18:55.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.010+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/2343388019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d65f76a0 0x55d3d6619460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.010+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/2343388019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d647a4a0 0x55d3d65f70c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.010+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/2343388019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 0x55d3d6618f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.010+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/2343388019 >> 172.21.15.19:0/2343388019 conn(0x55d3d64da6a0 msgr2=0x55d3d65fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:55.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.011+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/2343388019 shutdown_connections 2024-05-09T02:18:55.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.011+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/2343388019 wait complete. 2024-05-09T02:18:55.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.011+0000 7f3e5b56fb80 1 -- start start 2024-05-09T02:18:55.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.011+0000 7f3e5b56fb80 1 Processor -- start 2024-05-09T02:18:55.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.012+0000 7f3e5b56fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d647a4a0 0x55d3d6609420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:55.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.012+0000 7f3e5b56fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 0x55d3d6609960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:55.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.012+0000 7f3e5b56fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d65f76a0 0x55d3d65fd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:55.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.012+0000 7f3e5b56fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2800eb90 con 0x55d3d647a4a0 2024-05-09T02:18:55.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.012+0000 7f3e5b56fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2800e4c0 con 0x55d3d6564160 2024-05-09T02:18:55.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.012+0000 7f3e5b56fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2800b650 con 0x55d3d65f76a0 2024-05-09T02:18:55.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.012+0000 7f3e5b56fb80 10 client.0 _ll_register_callbacks cb 0x55d3d638b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:55.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.012+0000 7f3e4659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d647a4a0 0x55d3d6609420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:55.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.012+0000 7f3e4659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d647a4a0 0x55d3d6609420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60792/0 (socket says 172.21.15.19:60792) 2024-05-09T02:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.012+0000 7f3e4659c700 1 -- 172.21.15.19:0/3844845737 learned_addr learned my addr 172.21.15.19:0/3844845737 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.012+0000 7f3e45d9b700 1 --2- 172.21.15.19:0/3844845737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 0x55d3d6609960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.013+0000 7f3e46d9d700 1 --2- 172.21.15.19:0/3844845737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d65f76a0 0x55d3d65fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.013+0000 7f3e4659c700 1 -- 172.21.15.19:0/3844845737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d65f76a0 msgr2=0x55d3d65fd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.013+0000 7f3e4659c700 1 --2- 172.21.15.19:0/3844845737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d65f76a0 0x55d3d65fd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.013+0000 7f3e4659c700 1 -- 172.21.15.19:0/3844845737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 msgr2=0x55d3d6609960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.013+0000 7f3e4659c700 1 --2- 172.21.15.19:0/3844845737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 0x55d3d6609960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.013+0000 7f3e4659c700 1 -- 172.21.15.19:0/3844845737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3d65fdc00 con 0x55d3d647a4a0 2024-05-09T02:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.013+0000 7f3e45d9b700 1 --2- 172.21.15.19:0/3844845737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 0x55d3d6609960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.013+0000 7f3e4659c700 1 --2- 172.21.15.19:0/3844845737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d647a4a0 0x55d3d6609420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009ef0 tx=0x7f3e38003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e549bd7309886 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.013+0000 7f3e2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.013+0000 7f3e2dffb700 1 -- 172.21.15.19:0/3844845737 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380a9960 con 0x55d3d647a4a0 2024-05-09T02:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.013+0000 7f3e2dffb700 1 -- 172.21.15.19:0/3844845737 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f3e380b2540 con 0x55d3d647a4a0 2024-05-09T02:18:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.014+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3844845737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e380add10 con 0x55d3d647a4a0 2024-05-09T02:18:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.014+0000 7f3e2dffb700 1 -- 172.21.15.19:0/3844845737 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f3e380aa060 con 0x55d3d647a4a0 2024-05-09T02:18:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.015+0000 7f3e5b56fb80 10 client.16257 fetch_fsmap learned FSMap version 48 2024-05-09T02:18:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.015+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3844845737 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d3d667b290 con 0x55d3d647a4a0 2024-05-09T02:18:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.015+0000 7f3e2dffb700 1 -- 172.21.15.19:0/3844845737 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e380add10 con 0x55d3d647a4a0 2024-05-09T02:18:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.015+0000 7f3e2dffb700 1 -- 172.21.15.19:0/3844845737 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f3e3808b030 con 0x55d3d647a4a0 2024-05-09T02:18:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.016+0000 7f3e5b56fb80 10 client.16257 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:18:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.016+0000 7f3e5b56fb80 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:18:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.016+0000 7f3e5b56fb80 20 client.16257 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.016+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3844845737 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d3d667b510 0x55d3d667d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.016+0000 7f3e5b56fb80 4 client.16257 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:18:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.016+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3844845737 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3d6521a70 con 0x55d3d667b510 2024-05-09T02:18:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.016+0000 7f3e45d9b700 1 --2- 172.21.15.19:0/3844845737 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d3d667b510 0x55d3d667d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.016+0000 7f3e2dffb700 10 client.16257 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.017+0000 7f3e45d9b700 1 --2- 172.21.15.19:0/3844845737 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d3d667b510 0x55d3d667d930 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c5fcfa57af57e29 server_cookie=36928fa745e0f832 in_seq=0 out_seq=0 2024-05-09T02:18:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.017+0000 7f3e2dffb700 1 -- 172.21.15.19:0/3844845737 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3d6521a70 con 0x55d3d667b510 2024-05-09T02:18:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.017+0000 7f3e2dffb700 10 client.16257 handle_command_reply: tid=0 2024-05-09T02:18:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.017+0000 7f3e5b56fb80 1 client.16257 shutdown 2024-05-09T02:18:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.017+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3844845737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d647a4a0 msgr2=0x55d3d6609420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.017+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3844845737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d647a4a0 0x55d3d6609420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009ef0 tx=0x7f3e38003680 comp rx=0 tx=0).stop 2024-05-09T02:18:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.017+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3844845737 shutdown_connections 2024-05-09T02:18:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.017+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3844845737 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d3d667b510 0x55d3d667d930 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.017+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3844845737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3d65f76a0 0x55d3d65fd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.017+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3844845737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3d647a4a0 0x55d3d6609420 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.017+0000 7f3e5b56fb80 1 --2- 172.21.15.19:0/3844845737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3d6564160 0x55d3d6609960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.017+0000 7f3e2dffb700 0 client.16257 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.017+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3844845737 >> 172.21.15.19:0/3844845737 conn(0x55d3d64da6a0 msgr2=0x55d3d65ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.018+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3844845737 shutdown_connections 2024-05-09T02:18:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.018+0000 7f3e5b56fb80 1 -- 172.21.15.19:0/3844845737 wait complete. 2024-05-09T02:18:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.018+0000 7f3e5b56fb80 20 client.16257 trim_cache size 0 max 16384 2024-05-09T02:18:55.024 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:18:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.018+0000 7f3e4cb59700 1 -- 172.21.15.19:0/2025486077 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3e300b2a70 msgr2=0x7f3e300b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.018+0000 7f3e4cb59700 1 --2- 172.21.15.19:0/2025486077 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3e300b2a70 0x7f3e300b4f80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00b2e0 tx=0x7f3e3c009590 comp rx=0 tx=0).stop 2024-05-09T02:18:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.018+0000 7f3e4cb59700 1 -- 172.21.15.19:0/2025486077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e48102880 msgr2=0x7f3e48105600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.018+0000 7f3e4cb59700 1 --2- 172.21.15.19:0/2025486077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e48102880 0x7f3e48105600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3e4806b050 tx=0x7f3e3800eca0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.019+0000 7f3e4cb59700 1 -- 172.21.15.19:0/2025486077 shutdown_connections 2024-05-09T02:18:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.019+0000 7f3e4cb59700 1 --2- 172.21.15.19:0/2025486077 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3e300b2a70 0x7f3e300b4f80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.019+0000 7f3e4cb59700 1 --2- 172.21.15.19:0/2025486077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48109300 0x7f3e48104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.019+0000 7f3e4cb59700 1 --2- 172.21.15.19:0/2025486077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e48102880 0x7f3e48105600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.019+0000 7f3e4cb59700 1 --2- 172.21.15.19:0/2025486077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e481031d0 0x7f3e48103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.019+0000 7f3e4cb59700 1 -- 172.21.15.19:0/2025486077 >> 172.21.15.19:0/2025486077 conn(0x7f3e480fe6f0 msgr2=0x7f3e4810a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.019+0000 7f3e4cb59700 1 -- 172.21.15.19:0/2025486077 shutdown_connections 2024-05-09T02:18:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:55.019+0000 7f3e4cb59700 1 -- 172.21.15.19:0/2025486077 wait complete. 2024-05-09T02:18:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:56 smithi059 ceph-mon[34781]: pgmap v234: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 17 MiB/s wr, 823 op/s 2024-05-09T02:18:56.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:56 smithi196 ceph-mon[34718]: pgmap v234: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 17 MiB/s wr, 823 op/s 2024-05-09T02:18:56.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:56 smithi019 ceph-mon[30151]: pgmap v234: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 17 MiB/s wr, 823 op/s 2024-05-09T02:18:57.067 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T02:18:57.067 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T02:18:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:18:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 435.0 s, 2287.8 tps, lat 43.619 ms stddev 51.320 2024-05-09T02:18:58.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:18:58 smithi019 ceph-mon[30151]: pgmap v235: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 13 MiB/s wr, 600 op/s 2024-05-09T02:18:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:18:58 smithi059 ceph-mon[34781]: pgmap v235: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 13 MiB/s wr, 600 op/s 2024-05-09T02:18:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:18:58 smithi196 ceph-mon[34718]: pgmap v235: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 13 MiB/s wr, 600 op/s 2024-05-09T02:18:59.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.555+0000 7fc21129d700 1 -- 172.21.15.19:0/516895125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc20c108740 msgr2=0x7fc20c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:59.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.555+0000 7fc21129d700 1 --2- 172.21.15.19:0/516895125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc20c108740 0x7fc20c1100a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc204010580 tx=0x7fc204010890 comp rx=0 tx=0).stop 2024-05-09T02:18:59.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.558+0000 7fc21129d700 1 -- 172.21.15.19:0/516895125 shutdown_connections 2024-05-09T02:18:59.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.558+0000 7fc21129d700 1 --2- 172.21.15.19:0/516895125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc20c108740 0x7fc20c1100a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.558+0000 7fc21129d700 1 --2- 172.21.15.19:0/516895125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc20c1034c0 0x7fc20c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.558+0000 7fc21129d700 1 --2- 172.21.15.19:0/516895125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc20c107df0 0x7fc20c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.558+0000 7fc21129d700 1 -- 172.21.15.19:0/516895125 >> 172.21.15.19:0/516895125 conn(0x7fc20c0fedc0 msgr2=0x7fc20c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:59.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.558+0000 7fc21129d700 1 -- 172.21.15.19:0/516895125 shutdown_connections 2024-05-09T02:18:59.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.558+0000 7fc21129d700 1 -- 172.21.15.19:0/516895125 wait complete. 2024-05-09T02:18:59.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.560+0000 7fc21129d700 1 Processor -- start 2024-05-09T02:18:59.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.560+0000 7fc21129d700 1 -- start start 2024-05-09T02:18:59.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.560+0000 7fc21129d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc20c1034c0 0x7fc20c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.560+0000 7fc21129d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc20c107df0 0x7fc20c06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.560+0000 7fc21129d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc20c108740 0x7fc20c06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.560+0000 7fc21129d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c1232b0 con 0x7fc20c108740 2024-05-09T02:18:59.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.560+0000 7fc21129d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c123130 con 0x7fc20c1034c0 2024-05-09T02:18:59.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.560+0000 7fc21129d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c123430 con 0x7fc20c107df0 2024-05-09T02:18:59.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.561+0000 7fc20a7fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc20c107df0 0x7fc20c06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.561+0000 7fc20a7fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc20c107df0 0x7fc20c06e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:43980/0 (socket says 172.21.15.19:43980) 2024-05-09T02:18:59.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.561+0000 7fc20a7fc700 1 -- 172.21.15.19:0/1907378806 learned_addr learned my addr 172.21.15.19:0/1907378806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:59.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.561+0000 7fc20affd700 1 --2- 172.21.15.19:0/1907378806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc20c1034c0 0x7fc20c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.561+0000 7fc20b7fe700 1 --2- 172.21.15.19:0/1907378806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc20c108740 0x7fc20c06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.562+0000 7fc20a7fc700 1 -- 172.21.15.19:0/1907378806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc20c1034c0 msgr2=0x7fc20c075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:59.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.562+0000 7fc20a7fc700 1 --2- 172.21.15.19:0/1907378806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc20c1034c0 0x7fc20c075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.562+0000 7fc20a7fc700 1 -- 172.21.15.19:0/1907378806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc20c108740 msgr2=0x7fc20c06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:59.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.562+0000 7fc20a7fc700 1 --2- 172.21.15.19:0/1907378806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc20c108740 0x7fc20c06eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.562+0000 7fc20a7fc700 1 -- 172.21.15.19:0/1907378806 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c072eb0 con 0x7fc20c107df0 2024-05-09T02:18:59.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.562+0000 7fc20a7fc700 1 --2- 172.21.15.19:0/1907378806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc20c107df0 0x7fc20c06e4b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00ea30 tx=0x7fc1fc00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:59.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.562+0000 7fc203fff700 1 -- 172.21.15.19:0/1907378806 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc00cbd0 con 0x7fc20c107df0 2024-05-09T02:18:59.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.562+0000 7fc203fff700 1 -- 172.21.15.19:0/1907378806 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1fc0044d0 con 0x7fc20c107df0 2024-05-09T02:18:59.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.562+0000 7fc203fff700 1 -- 172.21.15.19:0/1907378806 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc003dc0 con 0x7fc20c107df0 2024-05-09T02:18:59.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.563+0000 7fc21129d700 1 -- 172.21.15.19:0/1907378806 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20c073140 con 0x7fc20c107df0 2024-05-09T02:18:59.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.563+0000 7fc21129d700 1 -- 172.21.15.19:0/1907378806 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc20c073630 con 0x7fc20c107df0 2024-05-09T02:18:59.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.568+0000 7fc21fcb3b80 1 Processor -- start 2024-05-09T02:18:59.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.568+0000 7fc21fcb3b80 1 -- start start 2024-05-09T02:18:59.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.569+0000 7fc21fcb3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048491e0 0x5595048e2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.569+0000 7fc21fcb3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048e2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.569+0000 7fc21fcb3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048dc720 0x5595048dcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.569+0000 7fc21fcb3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5595047ff070 con 0x5595048dc720 2024-05-09T02:18:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.569+0000 7fc21fcb3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55950479d870 con 0x5595048491e0 2024-05-09T02:18:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.569+0000 7fc21fcb3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5595047fb030 con 0x55950475f520 2024-05-09T02:18:59.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.571+0000 7fc20b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048491e0 0x5595048e2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.572+0000 7fc20b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048491e0 0x5595048e2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:46806/0 (socket says 172.21.15.19:46806) 2024-05-09T02:18:59.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.572+0000 7fc20b7fe700 1 -- 172.21.15.19:0/648478731 learned_addr learned my addr 172.21.15.19:0/648478731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:59.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.573+0000 7fc20a7fc700 1 --2- 172.21.15.19:0/648478731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048dc720 0x5595048dcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.573+0000 7fc20affd700 1 --2- 172.21.15.19:0/648478731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048e2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.573+0000 7fc20b7fe700 1 -- 172.21.15.19:0/648478731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 msgr2=0x5595048e2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:59.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.573+0000 7fc20b7fe700 1 --2- 172.21.15.19:0/648478731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048e2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.573+0000 7fc20b7fe700 1 -- 172.21.15.19:0/648478731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048dc720 msgr2=0x5595048dcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:59.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.573+0000 7fc20b7fe700 1 --2- 172.21.15.19:0/648478731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048dc720 0x5595048dcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.573+0000 7fc20b7fe700 1 -- 172.21.15.19:0/648478731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595047953f0 con 0x5595048491e0 2024-05-09T02:18:59.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.573+0000 7fc20b7fe700 1 --2- 172.21.15.19:0/648478731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048491e0 0x5595048e2570 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc2040094a0 tx=0x7fc20400ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b3c5f610442f519 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:59.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.574+0000 7fc2017fa700 1 -- 172.21.15.19:0/648478731 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20401e030 con 0x5595048491e0 2024-05-09T02:18:59.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.574+0000 7fc2017fa700 1 -- 172.21.15.19:0/648478731 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc204010250 con 0x5595048491e0 2024-05-09T02:18:59.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.574+0000 7fc2017fa700 1 -- 172.21.15.19:0/648478731 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204017040 con 0x5595048491e0 2024-05-09T02:18:59.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.575+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/648478731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048491e0 msgr2=0x5595048e2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:59.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.575+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/648478731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048491e0 0x5595048e2570 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc2040094a0 tx=0x7fc20400ba10 comp rx=0 tx=0).stop 2024-05-09T02:18:59.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.576+0000 7fc203fff700 1 -- 172.21.15.19:0/1907378806 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc1fc003f60 con 0x7fc20c107df0 2024-05-09T02:18:59.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.577+0000 7fc203fff700 1 --2- 172.21.15.19:0/1907378806 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc1f80b2b20 0x7fc1f80b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.577+0000 7fc203fff700 1 -- 172.21.15.19:0/1907378806 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fc1fc0e2650 con 0x7fc20c107df0 2024-05-09T02:18:59.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.578+0000 7fc20b7fe700 1 --2- 172.21.15.19:0/1907378806 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc1f80b2b20 0x7fc1f80b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.579+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/648478731 shutdown_connections 2024-05-09T02:18:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.579+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/648478731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048dc720 0x5595048dcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.579+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/648478731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048491e0 0x5595048e2570 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.579+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/648478731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048e2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.579+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/648478731 >> 172.21.15.19:0/648478731 conn(0x5595047bf720 msgr2=0x5595048d7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.579+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/648478731 shutdown_connections 2024-05-09T02:18:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.579+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/648478731 wait complete. 2024-05-09T02:18:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.581+0000 7fc21fcb3b80 1 -- start start 2024-05-09T02:18:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.581+0000 7fc21fcb3b80 1 Processor -- start 2024-05-09T02:18:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.582+0000 7fc21fcb3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048e6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.582+0000 7fc21fcb3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048491e0 0x5595048e6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.582+0000 7fc20a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048491e0 0x5595048e6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.582+0000 7fc21fcb3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048dc720 0x5595048e6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.583+0000 7fc21fcb3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55950467ac20 con 0x5595048491e0 2024-05-09T02:18:59.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.583+0000 7fc21fcb3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5595047b91d0 con 0x5595048dc720 2024-05-09T02:18:59.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.583+0000 7fc21fcb3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5595048e9250 con 0x55950475f520 2024-05-09T02:18:59.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.583+0000 7fc21fcb3b80 10 client.0 _ll_register_callbacks cb 0x559504670740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:59.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.583+0000 7fc20b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048dc720 0x5595048e6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.583+0000 7fc20b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048dc720 0x5595048e6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:46822/0 (socket says 172.21.15.19:46822) 2024-05-09T02:18:59.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.583+0000 7fc20b7fe700 1 -- 172.21.15.19:0/4118504411 learned_addr learned my addr 172.21.15.19:0/4118504411 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:59.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.583+0000 7fc20affd700 1 --2- 172.21.15.19:0/4118504411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048e6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.584+0000 7fc20b7fe700 1 -- 172.21.15.19:0/4118504411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 msgr2=0x5595048e6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:59.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.584+0000 7fc20b7fe700 1 --2- 172.21.15.19:0/4118504411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048e6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.584+0000 7fc20b7fe700 1 -- 172.21.15.19:0/4118504411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048491e0 msgr2=0x5595048e6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:59.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.584+0000 7fc20b7fe700 1 --2- 172.21.15.19:0/4118504411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048491e0 0x5595048e6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.584+0000 7fc20b7fe700 1 -- 172.21.15.19:0/4118504411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55950467a510 con 0x5595048dc720 2024-05-09T02:18:59.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.584+0000 7fc20b7fe700 1 --2- 172.21.15.19:0/1907378806 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc1f80b2b20 0x7fc1f80b5030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc204010580 tx=0x7fc204026040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:59.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.584+0000 7fc20b7fe700 1 --2- 172.21.15.19:0/4118504411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048dc720 0x5595048e6cd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc204010cd0 tx=0x7fc20400d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1a8597c00771544 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:59.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.584+0000 7fc1f27fc700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:18:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.585+0000 7fc1f27fc700 1 -- 172.21.15.19:0/4118504411 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204032020 con 0x5595048dc720 2024-05-09T02:18:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.585+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/4118504411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc204010250 con 0x5595048dc720 2024-05-09T02:18:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.586+0000 7fc1f27fc700 1 -- 172.21.15.19:0/4118504411 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fc20400aa30 con 0x5595048dc720 2024-05-09T02:18:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.586+0000 7fc1f27fc700 1 -- 172.21.15.19:0/4118504411 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc20400ac30 con 0x5595048dc720 2024-05-09T02:18:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.587+0000 7fc21fcb3b80 10 client.26003 fetch_fsmap learned FSMap version 48 2024-05-09T02:18:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.587+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/4118504411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5595048f3a10 con 0x5595048dc720 2024-05-09T02:18:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.587+0000 7fc1f27fc700 1 -- 172.21.15.19:0/4118504411 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc20400b280 con 0x5595048dc720 2024-05-09T02:18:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.587+0000 7fc1f27fc700 1 -- 172.21.15.19:0/4118504411 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fc20402d070 con 0x5595048dc720 2024-05-09T02:18:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.588+0000 7fc21fcb3b80 10 client.26003 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:18:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.588+0000 7fc21fcb3b80 10 client.26003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:18:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.588+0000 7fc21fcb3b80 20 client.26003 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.588+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/4118504411 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5595048f5ee0 0x5595048f8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.588+0000 7fc21fcb3b80 4 client.26003 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:18:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.588+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/4118504411 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc204010250 con 0x5595048f5ee0 2024-05-09T02:18:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.589+0000 7fc20affd700 1 --2- 172.21.15.19:0/4118504411 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5595048f5ee0 0x5595048f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.590+0000 7fc20affd700 1 --2- 172.21.15.19:0/4118504411 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5595048f5ee0 0x5595048f8300 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c2b13beaf90ddf2 server_cookie=d971f5b3abaf9191 in_seq=0 out_seq=0 2024-05-09T02:18:59.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.590+0000 7fc1f27fc700 10 client.26003 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.591+0000 7fc1f27fc700 1 -- 172.21.15.19:0/4118504411 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fc204010250 con 0x5595048f5ee0 2024-05-09T02:18:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.591+0000 7fc1f27fc700 10 client.26003 handle_command_reply: tid=0 2024-05-09T02:18:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.591+0000 7fc21fcb3b80 1 client.26003 shutdown 2024-05-09T02:18:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.591+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/4118504411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048dc720 msgr2=0x5595048e6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.591+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/4118504411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048dc720 0x5595048e6cd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc204010cd0 tx=0x7fc20400d040 comp rx=0 tx=0).stop 2024-05-09T02:18:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.594+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/4118504411 shutdown_connections 2024-05-09T02:18:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.594+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/4118504411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048dc720 0x5595048e6cd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.594+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/4118504411 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5595048f5ee0 0x5595048f8300 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.594+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/4118504411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048e6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.594+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/4118504411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048491e0 0x5595048e6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.594+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/4118504411 >> 172.21.15.19:0/4118504411 conn(0x5595047bf720 msgr2=0x5595048dcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:59.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.594+0000 7fc1f27fc700 0 client.26003 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:59.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.594+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/4118504411 shutdown_connections 2024-05-09T02:18:59.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.594+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/4118504411 wait complete. 2024-05-09T02:18:59.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.595+0000 7fc21fcb3b80 20 client.26003 trim_cache size 0 max 16384 2024-05-09T02:18:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.633+0000 7fc21fcb3b80 1 Processor -- start 2024-05-09T02:18:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.635+0000 7fc21fcb3b80 1 -- start start 2024-05-09T02:18:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.636+0000 7fc21fcb3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048e6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.636+0000 7fc21fcb3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048491e0 0x5595048f2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.636+0000 7fc21fcb3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048dc720 0x5595048e18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.636+0000 7fc21fcb3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec00d7d0 con 0x5595048491e0 2024-05-09T02:18:59.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.637+0000 7fc21fcb3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec00f3a0 con 0x5595048dc720 2024-05-09T02:18:59.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.637+0000 7fc20affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048e6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.637+0000 7fc20affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048e6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:44006/0 (socket says 172.21.15.19:44006) 2024-05-09T02:18:59.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.637+0000 7fc20affd700 1 -- 172.21.15.19:0/217412782 learned_addr learned my addr 172.21.15.19:0/217412782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:59.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.637+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/217412782 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec00df30 con 0x55950475f520 2024-05-09T02:18:59.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.637+0000 7fc20a7fc700 1 --2- 172.21.15.19:0/217412782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048491e0 0x5595048f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.637+0000 7fc20affd700 1 -- 172.21.15.19:0/217412782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048dc720 msgr2=0x5595048e18b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:18:59.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.638+0000 7fc20affd700 1 --2- 172.21.15.19:0/217412782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048dc720 0x5595048e18b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.638+0000 7fc20affd700 1 -- 172.21.15.19:0/217412782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048491e0 msgr2=0x5595048f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:59.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.638+0000 7fc20affd700 1 --2- 172.21.15.19:0/217412782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048491e0 0x5595048f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.638+0000 7fc20affd700 1 -- 172.21.15.19:0/217412782 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595048e9010 con 0x55950475f520 2024-05-09T02:18:59.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.638+0000 7fc20affd700 1 --2- 172.21.15.19:0/217412782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048e6c40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x5595048d8f50 tx=0x7fc1f400a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f3bda3487f84f24 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:59.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.638+0000 7fc200ff9700 1 -- 172.21.15.19:0/217412782 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4013b80 con 0x55950475f520 2024-05-09T02:18:59.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.638+0000 7fc200ff9700 1 -- 172.21.15.19:0/217412782 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f4013d20 con 0x55950475f520 2024-05-09T02:18:59.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.638+0000 7fc200ff9700 1 -- 172.21.15.19:0/217412782 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4011680 con 0x55950475f520 2024-05-09T02:18:59.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.640+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/217412782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 msgr2=0x5595048e6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:59.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.640+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/217412782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048e6c40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x5595048d8f50 tx=0x7fc1f400a300 comp rx=0 tx=0).stop 2024-05-09T02:18:59.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.640+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/217412782 shutdown_connections 2024-05-09T02:18:59.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.640+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/217412782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048dc720 0x5595048e18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.640+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/217412782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048e6c40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.640+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/217412782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048491e0 0x5595048f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.640+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/217412782 >> 172.21.15.19:0/217412782 conn(0x5595047bf720 msgr2=0x5595048db710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:59.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.641+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/217412782 shutdown_connections 2024-05-09T02:18:59.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.641+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/217412782 wait complete. 2024-05-09T02:18:59.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.641+0000 7fc21fcb3b80 1 -- start start 2024-05-09T02:18:59.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.641+0000 7fc21fcb3b80 1 Processor -- start 2024-05-09T02:18:59.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.642+0000 7fc21fcb3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.642+0000 7fc21fcb3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048491e0 0x5595048f8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.642+0000 7fc21fcb3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048dc720 0x5595048f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.642+0000 7fc21fcb3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec00ecd0 con 0x5595048dc720 2024-05-09T02:18:59.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.642+0000 7fc21fcb3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec00e600 con 0x5595048491e0 2024-05-09T02:18:59.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.642+0000 7fc21fcb3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec00b790 con 0x55950475f520 2024-05-09T02:18:59.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.642+0000 7fc21fcb3b80 10 client.0 _ll_register_callbacks cb 0x5595048d7090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:18:59.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.642+0000 7fc20affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.642+0000 7fc20b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048dc720 0x5595048f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.642+0000 7fc20a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048491e0 0x5595048f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.642+0000 7fc20affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048f7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:44012/0 (socket says 172.21.15.19:44012) 2024-05-09T02:18:59.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.642+0000 7fc20a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048491e0 0x5595048f8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:46842/0 (socket says 172.21.15.19:46842) 2024-05-09T02:18:59.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.642+0000 7fc20affd700 1 -- 172.21.15.19:0/3675347306 learned_addr learned my addr 172.21.15.19:0/3675347306 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:18:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.643+0000 7fc20affd700 1 -- 172.21.15.19:0/3675347306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048491e0 msgr2=0x5595048f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.643+0000 7fc20affd700 1 --2- 172.21.15.19:0/3675347306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048491e0 0x5595048f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.643+0000 7fc20affd700 1 -- 172.21.15.19:0/3675347306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048dc720 msgr2=0x5595048f86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:18:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.643+0000 7fc20affd700 1 --2- 172.21.15.19:0/3675347306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048dc720 0x5595048f86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.643+0000 7fc20affd700 1 -- 172.21.15.19:0/3675347306 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595048d0fe0 con 0x55950475f520 2024-05-09T02:18:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.643+0000 7fc20affd700 1 --2- 172.21.15.19:0/3675347306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048f7c30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc1f401ab60 tx=0x7fc1f4005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd62fe6a26d17ddd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:18:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.643+0000 7fc1f27fc700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:18:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.643+0000 7fc1f27fc700 1 -- 172.21.15.19:0/3675347306 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f401a800 con 0x55950475f520 2024-05-09T02:18:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.644+0000 7fc1f27fc700 1 -- 172.21.15.19:0/3675347306 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fc1f4004020 con 0x55950475f520 2024-05-09T02:18:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.644+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/3675347306 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f4013d20 con 0x55950475f520 2024-05-09T02:18:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.647+0000 7fc1f27fc700 1 -- 172.21.15.19:0/3675347306 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc1f4004220 con 0x55950475f520 2024-05-09T02:18:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.647+0000 7fc21fcb3b80 10 client.26038 fetch_fsmap learned FSMap version 48 2024-05-09T02:18:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.647+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/3675347306 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5595048d8d80 con 0x55950475f520 2024-05-09T02:18:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.647+0000 7fc1f27fc700 1 -- 172.21.15.19:0/3675347306 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f40043c0 con 0x55950475f520 2024-05-09T02:18:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.647+0000 7fc1f27fc700 1 -- 172.21.15.19:0/3675347306 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fc204003cb0 con 0x55950475f520 2024-05-09T02:18:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.647+0000 7fc21fcb3b80 10 client.26038 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:18:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.648+0000 7fc21fcb3b80 10 client.26038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:18:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.648+0000 7fc21fcb3b80 20 client.26038 populate_metadata read hostname 'smithi019' 2024-05-09T02:18:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.648+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/3675347306 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x559504961980 0x559504963da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:18:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.648+0000 7fc21fcb3b80 4 client.26038 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T02:18:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.648+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/3675347306 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fc1f4013d20 con 0x559504961980 2024-05-09T02:18:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.649+0000 7fc20a7fc700 1 --2- 172.21.15.19:0/3675347306 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x559504961980 0x559504963da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:18:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.649+0000 7fc20a7fc700 1 --2- 172.21.15.19:0/3675347306 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x559504961980 0x559504963da0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c8cf29800e39f87 server_cookie=8e3e14a762860c5e in_seq=0 out_seq=0 2024-05-09T02:18:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.649+0000 7fc1f27fc700 10 client.26038 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.650+0000 7fc1f27fc700 1 -- 172.21.15.19:0/3675347306 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fc1f4013d20 con 0x559504961980 2024-05-09T02:18:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.650+0000 7fc1f27fc700 10 client.26038 handle_command_reply: tid=0 2024-05-09T02:18:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.650+0000 7fc21fcb3b80 1 client.26038 shutdown 2024-05-09T02:18:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.651+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/3675347306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 msgr2=0x5595048f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.651+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/3675347306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048f7c30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc1f401ab60 tx=0x7fc1f4005dd0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.651+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/3675347306 shutdown_connections 2024-05-09T02:18:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.651+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/3675347306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595048dc720 0x5595048f86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.651+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/3675347306 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x559504961980 0x559504963da0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.651+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/3675347306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55950475f520 0x5595048f7c30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.651+0000 7fc21fcb3b80 1 --2- 172.21.15.19:0/3675347306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5595048491e0 0x5595048f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.651+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/3675347306 >> 172.21.15.19:0/3675347306 conn(0x5595047bf720 msgr2=0x5595048e6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.651+0000 7fc1f27fc700 0 client.26038 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:18:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.651+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/3675347306 shutdown_connections 2024-05-09T02:18:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.651+0000 7fc21fcb3b80 1 -- 172.21.15.19:0/3675347306 wait complete. 2024-05-09T02:18:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.651+0000 7fc21fcb3b80 20 client.26038 trim_cache size 0 max 16384 2024-05-09T02:18:59.654 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:18:59.654 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T02:18:59.654 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "d5778f62-c011-40ca-82ce-34c1e20e7ad4", 2024-05-09T02:18:59.654 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T02:18:59.655 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:18:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.652+0000 7fc21129d700 1 -- 172.21.15.19:0/1907378806 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc1f80b2b20 msgr2=0x7fc1f80b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.652+0000 7fc21129d700 1 --2- 172.21.15.19:0/1907378806 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc1f80b2b20 0x7fc1f80b5030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc204010580 tx=0x7fc204026040 comp rx=0 tx=0).stop 2024-05-09T02:18:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.652+0000 7fc21129d700 1 -- 172.21.15.19:0/1907378806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc20c107df0 msgr2=0x7fc20c06e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:18:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.652+0000 7fc21129d700 1 --2- 172.21.15.19:0/1907378806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc20c107df0 0x7fc20c06e4b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00ea30 tx=0x7fc1fc00ed40 comp rx=0 tx=0).stop 2024-05-09T02:18:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.652+0000 7fc21129d700 1 -- 172.21.15.19:0/1907378806 shutdown_connections 2024-05-09T02:18:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.652+0000 7fc21129d700 1 --2- 172.21.15.19:0/1907378806 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc1f80b2b20 0x7fc1f80b5030 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.652+0000 7fc21129d700 1 --2- 172.21.15.19:0/1907378806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc20c108740 0x7fc20c06eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.652+0000 7fc21129d700 1 --2- 172.21.15.19:0/1907378806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc20c1034c0 0x7fc20c075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.652+0000 7fc21129d700 1 --2- 172.21.15.19:0/1907378806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc20c107df0 0x7fc20c06e4b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:18:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.652+0000 7fc21129d700 1 -- 172.21.15.19:0/1907378806 >> 172.21.15.19:0/1907378806 conn(0x7fc20c0fedc0 msgr2=0x7fc20c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:18:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.652+0000 7fc21129d700 1 -- 172.21.15.19:0/1907378806 shutdown_connections 2024-05-09T02:18:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:18:59.652+0000 7fc21129d700 1 -- 172.21.15.19:0/1907378806 wait complete. 2024-05-09T02:19:00.340 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:19:00.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: pgmap v236: 97 pgs: 97 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 21 MiB/s wr, 888 op/s 2024-05-09T02:19:00.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub queued for path: / 2024-05-09T02:19:00.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:19:00.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:19:00.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:19:00.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:19:00.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:19:00.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:19:00.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:19:00.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:00 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: pgmap v236: 97 pgs: 97 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 21 MiB/s wr, 888 op/s 2024-05-09T02:19:00.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub queued for path: / 2024-05-09T02:19:00.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:19:00.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:19:00.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:19:00.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:19:00.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:19:00.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:19:00.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:19:00.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:00 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.781 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:19:00.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: pgmap v236: 97 pgs: 97 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 21 MiB/s wr, 888 op/s 2024-05-09T02:19:00.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub queued for path: / 2024-05-09T02:19:00.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:19:00.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:19:00.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:19:00.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:19:00.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:19:00.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:19:00.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:19:00.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:00.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:00 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:19:02.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:02 smithi019 ceph-mon[30151]: pgmap v237: 97 pgs: 97 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 MiB/s wr, 661 op/s 2024-05-09T02:19:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 440.0 s, 1725.4 tps, lat 57.216 ms stddev 89.772 2024-05-09T02:19:02.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:02 smithi059 ceph-mon[34781]: pgmap v237: 97 pgs: 97 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 MiB/s wr, 661 op/s 2024-05-09T02:19:02.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:02 smithi196 ceph-mon[34718]: pgmap v237: 97 pgs: 97 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 MiB/s wr, 661 op/s 2024-05-09T02:19:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.908+0000 7faee6a74700 1 -- 172.21.15.19:0/4197957356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faee01032e0 msgr2=0x7faee01036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.908+0000 7faee6a74700 1 --2- 172.21.15.19:0/4197957356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faee01032e0 0x7faee01036f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7faed800b0a0 tx=0x7faed800b3b0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.911+0000 7faee6a74700 1 -- 172.21.15.19:0/4197957356 shutdown_connections 2024-05-09T02:19:03.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.911+0000 7faee6a74700 1 --2- 172.21.15.19:0/4197957356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee0108560 0x7faee006b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.911+0000 7faee6a74700 1 --2- 172.21.15.19:0/4197957356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faee01032e0 0x7faee01036f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.911+0000 7faee6a74700 1 --2- 172.21.15.19:0/4197957356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee0107c10 0x7faee0108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.911+0000 7faee6a74700 1 -- 172.21.15.19:0/4197957356 >> 172.21.15.19:0/4197957356 conn(0x7faee0078ac0 msgr2=0x7faee0078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:03.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.914+0000 7faee6a74700 1 -- 172.21.15.19:0/4197957356 shutdown_connections 2024-05-09T02:19:03.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.915+0000 7faee6a74700 1 -- 172.21.15.19:0/4197957356 wait complete. 2024-05-09T02:19:03.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.916+0000 7faee6a74700 1 Processor -- start 2024-05-09T02:19:03.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.916+0000 7faee6a74700 1 -- start start 2024-05-09T02:19:03.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.917+0000 7faee6a74700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee01032e0 0x7faee0113450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:03.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.917+0000 7faee6a74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee0107c10 0x7faee0113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:03.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.917+0000 7faee5a72700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee01032e0 0x7faee0113450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:03.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.918+0000 7faee5a72700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee01032e0 0x7faee0113450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:44040/0 (socket says 172.21.15.19:44040) 2024-05-09T02:19:03.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.918+0000 7faee5a72700 1 -- 172.21.15.19:0/3165661357 learned_addr learned my addr 172.21.15.19:0/3165661357 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:03.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.918+0000 7faee5271700 1 --2- 172.21.15.19:0/3165661357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee0107c10 0x7faee0113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:03.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.918+0000 7faee6a74700 1 --2- 172.21.15.19:0/3165661357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faee0108560 0x7faee0113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:03.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.919+0000 7faee6a74700 1 -- 172.21.15.19:0/3165661357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faee0116660 con 0x7faee0108560 2024-05-09T02:19:03.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.919+0000 7faee6a74700 1 -- 172.21.15.19:0/3165661357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faee01164e0 con 0x7faee0107c10 2024-05-09T02:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.919+0000 7faee6a74700 1 -- 172.21.15.19:0/3165661357 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faee01167e0 con 0x7faee01032e0 2024-05-09T02:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.919+0000 7faee6273700 1 --2- 172.21.15.19:0/3165661357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faee0108560 0x7faee0113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:03.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.924+0000 7faee6273700 1 -- 172.21.15.19:0/3165661357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee01032e0 msgr2=0x7faee0113450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:03.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.924+0000 7faee6273700 1 --2- 172.21.15.19:0/3165661357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee01032e0 0x7faee0113450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.924+0000 7faee6273700 1 -- 172.21.15.19:0/3165661357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee0107c10 msgr2=0x7faee0113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:03.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.924+0000 7faee6273700 1 --2- 172.21.15.19:0/3165661357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee0107c10 0x7faee0113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.925+0000 7faee6273700 1 -- 172.21.15.19:0/3165661357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee01ad490 con 0x7faee0108560 2024-05-09T02:19:03.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.925+0000 7faee6273700 1 --2- 172.21.15.19:0/3165661357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faee0108560 0x7faee0113ed0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7faed000c7a0 tx=0x7faed000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:03.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.926+0000 7faed6ffd700 1 -- 172.21.15.19:0/3165661357 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0004a30 con 0x7faee0108560 2024-05-09T02:19:03.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.926+0000 7faee6a74700 1 -- 172.21.15.19:0/3165661357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee01ad780 con 0x7faee0108560 2024-05-09T02:19:03.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.926+0000 7faee6a74700 1 -- 172.21.15.19:0/3165661357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee01adc60 con 0x7faee0108560 2024-05-09T02:19:03.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.926+0000 7faed6ffd700 1 -- 172.21.15.19:0/3165661357 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed00054d0 con 0x7faee0108560 2024-05-09T02:19:03.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.926+0000 7faed6ffd700 1 -- 172.21.15.19:0/3165661357 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0007be0 con 0x7faee0108560 2024-05-09T02:19:03.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.931+0000 7faed6ffd700 1 -- 172.21.15.19:0/3165661357 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faed000f450 con 0x7faee0108560 2024-05-09T02:19:03.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.936+0000 7faef66ecb80 1 Processor -- start 2024-05-09T02:19:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.939+0000 7faef66ecb80 1 -- start start 2024-05-09T02:19:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.940+0000 7faef66ecb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839f5f160 0x555839ff8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.940+0000 7faef66ecb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 0x555839ff8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.940+0000 7faef66ecb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839ff2800 0x555839ff2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.941+0000 7faef66ecb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555839f14ff0 con 0x555839f5f160 2024-05-09T02:19:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.941+0000 7faef66ecb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555839eb37f0 con 0x555839ff2800 2024-05-09T02:19:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.941+0000 7faef66ecb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555839f10fb0 con 0x555839e754a0 2024-05-09T02:19:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.941+0000 7faee5a72700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 0x555839ff8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.941+0000 7faee5a72700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 0x555839ff8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:44042/0 (socket says 172.21.15.19:44042) 2024-05-09T02:19:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.941+0000 7faee5a72700 1 -- 172.21.15.19:0/3618928953 learned_addr learned my addr 172.21.15.19:0/3618928953 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.941+0000 7faee5271700 1 --2- 172.21.15.19:0/3618928953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839ff2800 0x555839ff2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:03.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.943+0000 7faed6ffd700 1 --2- 172.21.15.19:0/3165661357 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7faecc0b2b20 0x7faecc0b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:03.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.943+0000 7faee5a72700 1 -- 172.21.15.19:0/3618928953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839ff2800 msgr2=0x555839ff2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:03.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.943+0000 7faee5a72700 1 --2- 172.21.15.19:0/3618928953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839ff2800 0x555839ff2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.943+0000 7faee5a72700 1 -- 172.21.15.19:0/3618928953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839f5f160 msgr2=0x555839ff8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:03.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.943+0000 7faee5a72700 1 --2- 172.21.15.19:0/3618928953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839f5f160 0x555839ff8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.943+0000 7faed6ffd700 1 -- 172.21.15.19:0/3165661357 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7faed00e30a0 con 0x7faee0108560 2024-05-09T02:19:03.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.943+0000 7faee5a72700 1 -- 172.21.15.19:0/3618928953 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555839eab370 con 0x555839e754a0 2024-05-09T02:19:03.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.943+0000 7faee6273700 1 --2- 172.21.15.19:0/3165661357 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7faecc0b2b20 0x7faecc0b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:03.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.943+0000 7faee5a72700 1 --2- 172.21.15.19:0/3618928953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 0x555839ff8c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faed8009900 tx=0x7faed8009470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c70c3bbc4c1dd501 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:03.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.945+0000 7faecbfff700 1 -- 172.21.15.19:0/3618928953 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed800bd30 con 0x555839e754a0 2024-05-09T02:19:03.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.945+0000 7faecbfff700 1 -- 172.21.15.19:0/3618928953 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed80075c0 con 0x555839e754a0 2024-05-09T02:19:03.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.946+0000 7faef66ecb80 1 -- 172.21.15.19:0/3618928953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 msgr2=0x555839ff8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:03.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.946+0000 7faef66ecb80 1 --2- 172.21.15.19:0/3618928953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 0x555839ff8c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faed8009900 tx=0x7faed8009470 comp rx=0 tx=0).stop 2024-05-09T02:19:03.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.949+0000 7faef66ecb80 1 -- 172.21.15.19:0/3618928953 shutdown_connections 2024-05-09T02:19:03.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.949+0000 7faef66ecb80 1 --2- 172.21.15.19:0/3618928953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839ff2800 0x555839ff2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.949+0000 7faef66ecb80 1 --2- 172.21.15.19:0/3618928953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839f5f160 0x555839ff8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.949+0000 7faef66ecb80 1 --2- 172.21.15.19:0/3618928953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 0x555839ff8c80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.949+0000 7faef66ecb80 1 -- 172.21.15.19:0/3618928953 >> 172.21.15.19:0/3618928953 conn(0x555839ed56a0 msgr2=0x555839fed6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:03.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.949+0000 7faef66ecb80 1 -- 172.21.15.19:0/3618928953 shutdown_connections 2024-05-09T02:19:03.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.949+0000 7faee6273700 1 --2- 172.21.15.19:0/3165661357 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7faecc0b2b20 0x7faecc0b5030 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faed000cf60 tx=0x7faed0005960 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:03.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.949+0000 7faef66ecb80 1 -- 172.21.15.19:0/3618928953 wait complete. 2024-05-09T02:19:03.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.950+0000 7faef66ecb80 1 -- start start 2024-05-09T02:19:03.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.950+0000 7faef66ecb80 1 Processor -- start 2024-05-09T02:19:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.958+0000 7faef66ecb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839e754a0 0x555839ffc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.959+0000 7faef66ecb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839f5f160 0x555839ffc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.959+0000 7faef66ecb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839ff2800 0x555839ffce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.959+0000 7faef66ecb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555839d90ba0 con 0x555839e754a0 2024-05-09T02:19:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.959+0000 7faef66ecb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555839ecf150 con 0x555839f5f160 2024-05-09T02:19:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.960+0000 7faef66ecb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555839fff420 con 0x555839ff2800 2024-05-09T02:19:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.960+0000 7faee5271700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839e754a0 0x555839ffc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.960+0000 7faee5271700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839e754a0 0x555839ffc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59572/0 (socket says 172.21.15.19:59572) 2024-05-09T02:19:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.960+0000 7faee5271700 1 -- 172.21.15.19:0/2718034178 learned_addr learned my addr 172.21.15.19:0/2718034178 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.960+0000 7faef66ecb80 10 client.0 _ll_register_callbacks cb 0x555839d866c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:19:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.965+0000 7faee5271700 1 --2- 172.21.15.19:0/2718034178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839ff2800 0x555839ffce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.966+0000 7faee5271700 1 -- 172.21.15.19:0/2718034178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839ff2800 msgr2=0x555839ffce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.966+0000 7faee5271700 1 --2- 172.21.15.19:0/2718034178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839ff2800 0x555839ffce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.966+0000 7faee5271700 1 -- 172.21.15.19:0/2718034178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839f5f160 msgr2=0x555839ffc8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:19:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.966+0000 7faee5271700 1 --2- 172.21.15.19:0/2718034178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839f5f160 0x555839ffc8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.966+0000 7faee5271700 1 -- 172.21.15.19:0/2718034178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555839d90490 con 0x555839e754a0 2024-05-09T02:19:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.967+0000 7faee5271700 1 --2- 172.21.15.19:0/2718034178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839e754a0 0x555839ffc3b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7faedc011f20 tx=0x7faedc009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed09484044ab67c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.968+0000 7faec97fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:19:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.972+0000 7faec97fa700 1 -- 172.21.15.19:0/2718034178 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc010e90 con 0x555839e754a0 2024-05-09T02:19:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.972+0000 7faec97fa700 1 -- 172.21.15.19:0/2718034178 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7faedc004030 con 0x555839e754a0 2024-05-09T02:19:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.972+0000 7faef66ecb80 1 -- 172.21.15.19:0/2718034178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555839f229d0 con 0x555839e754a0 2024-05-09T02:19:03.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.974+0000 7faec97fa700 1 -- 172.21.15.19:0/2718034178 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7faedc028050 con 0x555839e754a0 2024-05-09T02:19:03.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.976+0000 7faef66ecb80 10 client.16281 fetch_fsmap learned FSMap version 48 2024-05-09T02:19:03.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.976+0000 7faef66ecb80 1 -- 172.21.15.19:0/2718034178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55583a009d30 con 0x555839e754a0 2024-05-09T02:19:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.980+0000 7faec97fa700 1 -- 172.21.15.19:0/2718034178 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555839f229d0 con 0x555839e754a0 2024-05-09T02:19:03.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.982+0000 7faec97fa700 1 -- 172.21.15.19:0/2718034178 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7faedc003ca0 con 0x555839e754a0 2024-05-09T02:19:03.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.982+0000 7faef66ecb80 10 client.16281 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:19:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.982+0000 7faef66ecb80 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:19:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.982+0000 7faef66ecb80 20 client.16281 populate_metadata read hostname 'smithi019' 2024-05-09T02:19:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.982+0000 7faef66ecb80 1 --2- 172.21.15.19:0/2718034178 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55583a00c370 0x55583a00e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.982+0000 7faef66ecb80 4 client.16281 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:19:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.982+0000 7faef66ecb80 1 -- 172.21.15.19:0/2718034178 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555839f1ca70 con 0x55583a00c370 2024-05-09T02:19:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.985+0000 7faee5a72700 1 --2- 172.21.15.19:0/2718034178 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55583a00c370 0x55583a00e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:03.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.988+0000 7faee5a72700 1 --2- 172.21.15.19:0/2718034178 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55583a00c370 0x55583a00e7f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ee21fb123955f24 server_cookie=1a3a15494daf204d in_seq=0 out_seq=0 2024-05-09T02:19:03.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.989+0000 7faec97fa700 10 client.16281 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:19:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.991+0000 7faec97fa700 1 -- 172.21.15.19:0/2718034178 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555839f1ca70 con 0x55583a00c370 2024-05-09T02:19:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.991+0000 7faec97fa700 10 client.16281 handle_command_reply: tid=0 2024-05-09T02:19:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.991+0000 7faef66ecb80 1 client.16281 shutdown 2024-05-09T02:19:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.991+0000 7faef66ecb80 1 -- 172.21.15.19:0/2718034178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839e754a0 msgr2=0x555839ffc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:03.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.991+0000 7faef66ecb80 1 --2- 172.21.15.19:0/2718034178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839e754a0 0x555839ffc3b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7faedc011f20 tx=0x7faedc009710 comp rx=0 tx=0).stop 2024-05-09T02:19:03.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.995+0000 7faef66ecb80 1 -- 172.21.15.19:0/2718034178 shutdown_connections 2024-05-09T02:19:03.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.995+0000 7faef66ecb80 1 --2- 172.21.15.19:0/2718034178 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55583a00c370 0x55583a00e7f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.995+0000 7faef66ecb80 1 --2- 172.21.15.19:0/2718034178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839ff2800 0x555839ffce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.995+0000 7faef66ecb80 1 --2- 172.21.15.19:0/2718034178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839e754a0 0x555839ffc3b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.995+0000 7faef66ecb80 1 --2- 172.21.15.19:0/2718034178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839f5f160 0x555839ffc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:03.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.995+0000 7faec97fa700 0 client.16281 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:19:03.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.995+0000 7faef66ecb80 1 -- 172.21.15.19:0/2718034178 >> 172.21.15.19:0/2718034178 conn(0x555839ed56a0 msgr2=0x555839fed3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:04.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:03.999+0000 7faef66ecb80 1 -- 172.21.15.19:0/2718034178 shutdown_connections 2024-05-09T02:19:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.000+0000 7faef66ecb80 1 -- 172.21.15.19:0/2718034178 wait complete. 2024-05-09T02:19:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.000+0000 7faef66ecb80 20 client.16281 trim_cache size 0 max 16384 2024-05-09T02:19:04.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.043+0000 7faef66ecb80 1 Processor -- start 2024-05-09T02:19:04.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.044+0000 7faef66ecb80 1 -- start start 2024-05-09T02:19:04.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.044+0000 7faef66ecb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 0x555839ffa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:04.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.044+0000 7faef66ecb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839f5f160 0x555839ff6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:04.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.045+0000 7faef66ecb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839ff2800 0x55583a0082f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:04.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.045+0000 7faee5a72700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839f5f160 0x555839ff6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:04.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.045+0000 7faef66ecb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faec400d690 con 0x555839f5f160 2024-05-09T02:19:04.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.045+0000 7faef66ecb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faec400f260 con 0x555839ff2800 2024-05-09T02:19:04.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.045+0000 7faef66ecb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faec400ddf0 con 0x555839e754a0 2024-05-09T02:19:04.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.045+0000 7faee5271700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839ff2800 0x55583a0082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:04.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.045+0000 7faee5271700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839ff2800 0x55583a0082f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:46886/0 (socket says 172.21.15.19:46886) 2024-05-09T02:19:04.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.045+0000 7faee5271700 1 -- 172.21.15.19:0/2165335234 learned_addr learned my addr 172.21.15.19:0/2165335234 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:04.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.047+0000 7faee5a72700 1 -- 172.21.15.19:0/2165335234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 msgr2=0x555839ffa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:04.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.047+0000 7faee5a72700 1 --2- 172.21.15.19:0/2165335234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 0x555839ffa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.047+0000 7faee5a72700 1 -- 172.21.15.19:0/2165335234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839ff2800 msgr2=0x55583a0082f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:04.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.047+0000 7faee5a72700 1 --2- 172.21.15.19:0/2165335234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839ff2800 0x55583a0082f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.047+0000 7faee5a72700 1 -- 172.21.15.19:0/2165335234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555839ffc180 con 0x555839f5f160 2024-05-09T02:19:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.052+0000 7faee5a72700 1 --2- 172.21.15.19:0/2165335234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839f5f160 0x555839ff6760 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x555839fe8510 tx=0x7faed80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a9e4dcd53993ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:04.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.054+0000 7faecb7fe700 1 -- 172.21.15.19:0/2165335234 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed800bcf0 con 0x555839f5f160 2024-05-09T02:19:04.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.054+0000 7faecb7fe700 1 -- 172.21.15.19:0/2165335234 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed801ccb0 con 0x555839f5f160 2024-05-09T02:19:04.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.054+0000 7faecb7fe700 1 -- 172.21.15.19:0/2165335234 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8007e30 con 0x555839f5f160 2024-05-09T02:19:04.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.055+0000 7faef66ecb80 1 -- 172.21.15.19:0/2165335234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839f5f160 msgr2=0x555839ff6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:04.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.055+0000 7faef66ecb80 1 --2- 172.21.15.19:0/2165335234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839f5f160 0x555839ff6760 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x555839fe8510 tx=0x7faed80099e0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.058+0000 7faef66ecb80 1 -- 172.21.15.19:0/2165335234 shutdown_connections 2024-05-09T02:19:04.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.058+0000 7faef66ecb80 1 --2- 172.21.15.19:0/2165335234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839ff2800 0x55583a0082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.058+0000 7faef66ecb80 1 --2- 172.21.15.19:0/2165335234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 0x555839ffa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.058+0000 7faef66ecb80 1 --2- 172.21.15.19:0/2165335234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839f5f160 0x555839ff6760 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.058+0000 7faef66ecb80 1 -- 172.21.15.19:0/2165335234 >> 172.21.15.19:0/2165335234 conn(0x555839ed56a0 msgr2=0x555839ff6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:04.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.064+0000 7faef66ecb80 1 -- 172.21.15.19:0/2165335234 shutdown_connections 2024-05-09T02:19:04.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.064+0000 7faef66ecb80 1 -- 172.21.15.19:0/2165335234 wait complete. 2024-05-09T02:19:04.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.065+0000 7faef66ecb80 1 -- start start 2024-05-09T02:19:04.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.065+0000 7faef66ecb80 1 Processor -- start 2024-05-09T02:19:04.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.066+0000 7faef66ecb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 0x55583a013680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:04.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.066+0000 7faef66ecb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839f5f160 0x55583a013bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:04.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.066+0000 7faee5271700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 0x55583a013680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:04.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.066+0000 7faee5271700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 0x55583a013680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:44076/0 (socket says 172.21.15.19:44076) 2024-05-09T02:19:04.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.066+0000 7faee5271700 1 -- 172.21.15.19:0/3464311471 learned_addr learned my addr 172.21.15.19:0/3464311471 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:04.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.066+0000 7faee5a72700 1 --2- 172.21.15.19:0/3464311471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839f5f160 0x55583a013bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:04.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.067+0000 7faef66ecb80 1 --2- 172.21.15.19:0/3464311471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839ff2800 0x55583a014100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:04.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.067+0000 7faef66ecb80 1 -- 172.21.15.19:0/3464311471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faec400eb90 con 0x555839ff2800 2024-05-09T02:19:04.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.067+0000 7faef66ecb80 1 -- 172.21.15.19:0/3464311471 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faec400e4c0 con 0x555839f5f160 2024-05-09T02:19:04.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.067+0000 7faef66ecb80 1 -- 172.21.15.19:0/3464311471 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faec400b6e0 con 0x555839e754a0 2024-05-09T02:19:04.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.067+0000 7faee5271700 1 -- 172.21.15.19:0/3464311471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839f5f160 msgr2=0x55583a013bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:04.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.067+0000 7faee5271700 1 --2- 172.21.15.19:0/3464311471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839f5f160 0x55583a013bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.067+0000 7faee5271700 1 -- 172.21.15.19:0/3464311471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839ff2800 msgr2=0x55583a014100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.067+0000 7faee5271700 1 --2- 172.21.15.19:0/3464311471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839ff2800 0x55583a014100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.067+0000 7faee5271700 1 -- 172.21.15.19:0/3464311471 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55583a005100 con 0x555839e754a0 2024-05-09T02:19:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.069+0000 7faef66ecb80 10 client.0 _ll_register_callbacks cb 0x555839d866c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:19:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.069+0000 7faee5271700 1 --2- 172.21.15.19:0/3464311471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 0x55583a013680 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x555839ff3a90 tx=0x7faedc011e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16535eb09d8e0912 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.069+0000 7faef66ecb80 1 -- 172.21.15.19:0/3464311471 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faed801ccb0 con 0x555839e754a0 2024-05-09T02:19:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.069+0000 7faec97fa700 10 client.26050 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:19:04.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.071+0000 7faec97fa700 1 -- 172.21.15.19:0/3464311471 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc0052c0 con 0x555839e754a0 2024-05-09T02:19:04.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.071+0000 7faec97fa700 1 -- 172.21.15.19:0/3464311471 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7faedc005460 con 0x555839e754a0 2024-05-09T02:19:04.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.071+0000 7faec97fa700 1 -- 172.21.15.19:0/3464311471 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7faedc0056c0 con 0x555839e754a0 2024-05-09T02:19:04.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.071+0000 7faef66ecb80 10 client.26050 fetch_fsmap learned FSMap version 48 2024-05-09T02:19:04.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.072+0000 7faef66ecb80 1 -- 172.21.15.19:0/3464311471 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=48}) -- 0x55583a076ad0 con 0x555839e754a0 2024-05-09T02:19:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.074+0000 7faec97fa700 1 -- 172.21.15.19:0/3464311471 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7faedc036020 con 0x555839e754a0 2024-05-09T02:19:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.074+0000 7faef66ecb80 10 client.26050 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T02:19:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.074+0000 7faef66ecb80 10 client.26050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:19:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.074+0000 7faef66ecb80 20 client.26050 populate_metadata read hostname 'smithi019' 2024-05-09T02:19:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.074+0000 7faef66ecb80 1 --2- 172.21.15.19:0/3464311471 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55583a0770d0 0x55583a0794f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.075+0000 7faef66ecb80 4 client.26050 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:19:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.075+0000 7faef66ecb80 1 -- 172.21.15.19:0/3464311471 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7faed801ccb0 con 0x55583a0770d0 2024-05-09T02:19:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.075+0000 7faee5a72700 1 --2- 172.21.15.19:0/3464311471 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55583a0770d0 0x55583a0794f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.078+0000 7faee5a72700 1 --2- 172.21.15.19:0/3464311471 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55583a0770d0 0x55583a0794f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed6be86ad923b59a server_cookie=d40f62612bce83b in_seq=0 out_seq=0 2024-05-09T02:19:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.078+0000 7faec97fa700 10 client.26050 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:19:04.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.082+0000 7faec97fa700 1 -- 172.21.15.19:0/3464311471 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7faed801ccb0 con 0x55583a0770d0 2024-05-09T02:19:04.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.082+0000 7faec97fa700 10 client.26050 handle_command_reply: tid=0 2024-05-09T02:19:04.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.082+0000 7faef66ecb80 1 client.26050 shutdown 2024-05-09T02:19:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.084+0000 7faef66ecb80 1 -- 172.21.15.19:0/3464311471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 msgr2=0x55583a013680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.084+0000 7faef66ecb80 1 --2- 172.21.15.19:0/3464311471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 0x55583a013680 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x555839ff3a90 tx=0x7faedc011e10 comp rx=0 tx=0).stop 2024-05-09T02:19:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.084+0000 7faef66ecb80 1 -- 172.21.15.19:0/3464311471 shutdown_connections 2024-05-09T02:19:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.084+0000 7faef66ecb80 1 --2- 172.21.15.19:0/3464311471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555839ff2800 0x55583a014100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.084+0000 7faef66ecb80 1 --2- 172.21.15.19:0/3464311471 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55583a0770d0 0x55583a0794f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.085+0000 7faef66ecb80 1 --2- 172.21.15.19:0/3464311471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555839e754a0 0x55583a013680 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.085+0000 7faec97fa700 0 client.26050 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:19:04.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.085+0000 7faef66ecb80 1 --2- 172.21.15.19:0/3464311471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555839f5f160 0x55583a013bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.085+0000 7faef66ecb80 1 -- 172.21.15.19:0/3464311471 >> 172.21.15.19:0/3464311471 conn(0x555839ed56a0 msgr2=0x555839ff17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:04.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.085+0000 7faef66ecb80 1 -- 172.21.15.19:0/3464311471 shutdown_connections 2024-05-09T02:19:04.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.085+0000 7faef66ecb80 1 -- 172.21.15.19:0/3464311471 wait complete. 2024-05-09T02:19:04.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.086+0000 7faef66ecb80 20 client.26050 trim_cache size 0 max 16384 2024-05-09T02:19:04.091 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:19:04.091 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T02:19:04.091 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T02:19:04.091 INFO:teuthology.orchestra.run.smithi019.stdout: "d5778f62-c011-40ca-82ce-34c1e20e7ad4": { 2024-05-09T02:19:04.091 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T02:19:04.091 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "d5778f62-c011-40ca-82ce-34c1e20e7ad4", 2024-05-09T02:19:04.091 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T02:19:04.091 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:19:04.091 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:19:04.091 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:19:04.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.092+0000 7faee6a74700 1 -- 172.21.15.19:0/3165661357 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7faecc0b2b20 msgr2=0x7faecc0b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:04.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.092+0000 7faee6a74700 1 --2- 172.21.15.19:0/3165661357 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7faecc0b2b20 0x7faecc0b5030 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faed000cf60 tx=0x7faed0005960 comp rx=0 tx=0).stop 2024-05-09T02:19:04.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.092+0000 7faee6a74700 1 -- 172.21.15.19:0/3165661357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faee0108560 msgr2=0x7faee0113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:04.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.092+0000 7faee6a74700 1 --2- 172.21.15.19:0/3165661357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faee0108560 0x7faee0113ed0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7faed000c7a0 tx=0x7faed000cb60 comp rx=0 tx=0).stop 2024-05-09T02:19:04.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.093+0000 7faee6a74700 1 -- 172.21.15.19:0/3165661357 shutdown_connections 2024-05-09T02:19:04.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.093+0000 7faee6a74700 1 --2- 172.21.15.19:0/3165661357 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7faecc0b2b20 0x7faecc0b5030 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.093+0000 7faee6a74700 1 --2- 172.21.15.19:0/3165661357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faee0108560 0x7faee0113ed0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.093+0000 7faee6a74700 1 --2- 172.21.15.19:0/3165661357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee01032e0 0x7faee0113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.093+0000 7faee6a74700 1 --2- 172.21.15.19:0/3165661357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee0107c10 0x7faee0113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:04.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.093+0000 7faee6a74700 1 -- 172.21.15.19:0/3165661357 >> 172.21.15.19:0/3165661357 conn(0x7faee0078ac0 msgr2=0x7faee00ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:04.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.093+0000 7faee6a74700 1 -- 172.21.15.19:0/3165661357 shutdown_connections 2024-05-09T02:19:04.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:04.093+0000 7faee6a74700 1 -- 172.21.15.19:0/3165661357 wait complete. 2024-05-09T02:19:04.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:04 smithi059 ceph-mon[34781]: pgmap v238: 97 pgs: 97 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 684 op/s 2024-05-09T02:19:04.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:04 smithi019 ceph-mon[30151]: pgmap v238: 97 pgs: 97 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 684 op/s 2024-05-09T02:19:04.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:04 smithi196 ceph-mon[34718]: pgmap v238: 97 pgs: 97 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 684 op/s 2024-05-09T02:19:04.814 INFO:tasks.cephfs.filesystem:scrub status for tag:d5778f62-c011-40ca-82ce-34c1e20e7ad4 - {'path': '/', 'tag': 'd5778f62-c011-40ca-82ce-34c1e20e7ad4', 'options': 'recursive,force'} 2024-05-09T02:19:06.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:06 smithi059 ceph-mon[34781]: pgmap v239: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 24 MiB/s wr, 1.18k op/s 2024-05-09T02:19:06.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:06 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:19:06.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:06 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:19:06.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:06 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:19:06.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:06 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:19:06.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:06 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:19:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:06 smithi196 ceph-mon[34718]: pgmap v239: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 24 MiB/s wr, 1.18k op/s 2024-05-09T02:19:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:06 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:19:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:06 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:19:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:06 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:19:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:06 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:19:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:06 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:19:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:06 smithi019 ceph-mon[30151]: pgmap v239: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 24 MiB/s wr, 1.18k op/s 2024-05-09T02:19:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:06 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:19:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:06 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:19:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:06 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:19:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:06 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:19:06.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:06 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:19:07.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 445.0 s, 1292.8 tps, lat 78.498 ms stddev 155.494 2024-05-09T02:19:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:08 smithi059 ceph-mon[34781]: pgmap v240: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 MiB/s wr, 812 op/s 2024-05-09T02:19:08.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:08 smithi196 ceph-mon[34718]: pgmap v240: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 MiB/s wr, 812 op/s 2024-05-09T02:19:08.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:08 smithi019 ceph-mon[30151]: pgmap v240: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 MiB/s wr, 812 op/s 2024-05-09T02:19:10.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:10 smithi196 ceph-mon[34718]: pgmap v241: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 31 MiB/s wr, 1.84k op/s 2024-05-09T02:19:10.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:10 smithi019 ceph-mon[30151]: pgmap v241: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 31 MiB/s wr, 1.84k op/s 2024-05-09T02:19:10.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:10 smithi059 ceph-mon[34781]: pgmap v241: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 31 MiB/s wr, 1.84k op/s 2024-05-09T02:19:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 450.0 s, 2096.4 tps, lat 47.446 ms stddev 57.126 2024-05-09T02:19:12.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:12 smithi196 ceph-mon[34718]: pgmap v242: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 23 MiB/s wr, 1.55k op/s 2024-05-09T02:19:12.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:12 smithi196 ceph-mon[34718]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T02:19:12.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:12 smithi019 ceph-mon[30151]: pgmap v242: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 23 MiB/s wr, 1.55k op/s 2024-05-09T02:19:12.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:12 smithi019 ceph-mon[30151]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T02:19:12.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:12 smithi059 ceph-mon[34781]: pgmap v242: 97 pgs: 97 active+clean; 4.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 23 MiB/s wr, 1.55k op/s 2024-05-09T02:19:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:12 smithi059 ceph-mon[34781]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T02:19:14.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:14 smithi196 ceph-mon[34718]: pgmap v244: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 27 MiB/s wr, 1.85k op/s 2024-05-09T02:19:14.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:14 smithi019 ceph-mon[30151]: pgmap v244: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 27 MiB/s wr, 1.85k op/s 2024-05-09T02:19:14.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:14 smithi059 ceph-mon[34781]: pgmap v244: 97 pgs: 97 active+clean; 4.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 27 MiB/s wr, 1.85k op/s 2024-05-09T02:19:16.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:16 smithi196 ceph-mon[34718]: pgmap v245: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 MiB/s wr, 1.55k op/s 2024-05-09T02:19:16.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:16 smithi019 ceph-mon[30151]: pgmap v245: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 MiB/s wr, 1.55k op/s 2024-05-09T02:19:16.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:16 smithi059 ceph-mon[34781]: pgmap v245: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 MiB/s wr, 1.55k op/s 2024-05-09T02:19:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 455.0 s, 2166.8 tps, lat 46.491 ms stddev 52.679 2024-05-09T02:19:17.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:17 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:19:17.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:17 smithi196 ceph-mon[34718]: pgmap v246: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 MiB/s wr, 1.55k op/s 2024-05-09T02:19:17.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:17 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:19:17.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:17 smithi019 ceph-mon[30151]: pgmap v246: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 MiB/s wr, 1.55k op/s 2024-05-09T02:19:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:17 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:19:17.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:17 smithi059 ceph-mon[34781]: pgmap v246: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 MiB/s wr, 1.55k op/s 2024-05-09T02:19:20.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:20 smithi059 ceph-mon[34781]: pgmap v247: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 13 MiB/s wr, 678 op/s 2024-05-09T02:19:20.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:20 smithi059 ceph-mon[34781]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T02:19:20.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:20 smithi196 ceph-mon[34718]: pgmap v247: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 13 MiB/s wr, 678 op/s 2024-05-09T02:19:20.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:20 smithi196 ceph-mon[34718]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T02:19:20.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:20 smithi019 ceph-mon[30151]: pgmap v247: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 13 MiB/s wr, 678 op/s 2024-05-09T02:19:20.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:20 smithi019 ceph-mon[30151]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T02:19:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 460.0 s, 1879.6 tps, lat 52.708 ms stddev 104.248 2024-05-09T02:19:22.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:22 smithi196 ceph-mon[34718]: pgmap v249: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 13 MiB/s wr, 692 op/s 2024-05-09T02:19:22.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:22 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:19:22.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:22 smithi019 ceph-mon[30151]: pgmap v249: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 13 MiB/s wr, 692 op/s 2024-05-09T02:19:22.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:22 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:19:22.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:22 smithi059 ceph-mon[34781]: pgmap v249: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 13 MiB/s wr, 692 op/s 2024-05-09T02:19:22.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:22 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:19:24.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:24 smithi196 ceph-mon[34718]: pgmap v250: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 13 MiB/s wr, 693 op/s 2024-05-09T02:19:24.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:24 smithi019 ceph-mon[30151]: pgmap v250: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 13 MiB/s wr, 693 op/s 2024-05-09T02:19:24.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:24 smithi059 ceph-mon[34781]: pgmap v250: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 13 MiB/s wr, 693 op/s 2024-05-09T02:19:26.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:26 smithi196 ceph-mon[34718]: pgmap v251: 97 pgs: 97 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 28 MiB/s wr, 2k op/s 2024-05-09T02:19:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:26 smithi019 ceph-mon[30151]: pgmap v251: 97 pgs: 97 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 28 MiB/s wr, 2k op/s 2024-05-09T02:19:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:26 smithi059 ceph-mon[34781]: pgmap v251: 97 pgs: 97 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 28 MiB/s wr, 2k op/s 2024-05-09T02:19:27.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 465.0 s, 2601.4 tps, lat 38.828 ms stddev 43.249 2024-05-09T02:19:28.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:28 smithi196 ceph-mon[34718]: pgmap v252: 97 pgs: 97 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 28 MiB/s wr, 2k op/s 2024-05-09T02:19:28.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:28 smithi019 ceph-mon[30151]: pgmap v252: 97 pgs: 97 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 28 MiB/s wr, 2k op/s 2024-05-09T02:19:28.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:28 smithi059 ceph-mon[34781]: pgmap v252: 97 pgs: 97 active+clean; 4.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 28 MiB/s wr, 2k op/s 2024-05-09T02:19:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:30 smithi196 ceph-mon[34718]: pgmap v253: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 26 MiB/s wr, 1.98k op/s 2024-05-09T02:19:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:30 smithi019 ceph-mon[30151]: pgmap v253: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 26 MiB/s wr, 1.98k op/s 2024-05-09T02:19:30.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:30 smithi059 ceph-mon[34781]: pgmap v253: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 26 MiB/s wr, 1.98k op/s 2024-05-09T02:19:31.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:31 smithi196 ceph-mon[34718]: pgmap v254: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 22 MiB/s wr, 1.67k op/s 2024-05-09T02:19:31.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:31 smithi019 ceph-mon[30151]: pgmap v254: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 22 MiB/s wr, 1.67k op/s 2024-05-09T02:19:31.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:31 smithi059 ceph-mon[34781]: pgmap v254: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 22 MiB/s wr, 1.67k op/s 2024-05-09T02:19:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 470.0 s, 2453.2 tps, lat 40.223 ms stddev 50.744 2024-05-09T02:19:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:34 smithi059 ceph-mon[34781]: pgmap v255: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 22 MiB/s wr, 1.67k op/s 2024-05-09T02:19:34.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:34 smithi196 ceph-mon[34718]: pgmap v255: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 22 MiB/s wr, 1.67k op/s 2024-05-09T02:19:34.815 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:19:34.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:34 smithi019 ceph-mon[30151]: pgmap v255: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 22 MiB/s wr, 1.67k op/s 2024-05-09T02:19:35.212 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:19:36.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:36 smithi059 ceph-mon[34781]: pgmap v256: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 26 MiB/s wr, 1.94k op/s 2024-05-09T02:19:36.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:36 smithi196 ceph-mon[34718]: pgmap v256: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 26 MiB/s wr, 1.94k op/s 2024-05-09T02:19:36.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:36 smithi019 ceph-mon[30151]: pgmap v256: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 26 MiB/s wr, 1.94k op/s 2024-05-09T02:19:37.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 475.0 s, 1757.0 tps, lat 57.525 ms stddev 100.139 2024-05-09T02:19:38.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.045+0000 7fa259a15700 1 -- 172.21.15.19:0/514316382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa254108740 msgr2=0x7fa2541100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:38.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.045+0000 7fa259a15700 1 --2- 172.21.15.19:0/514316382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa254108740 0x7fa2541100a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00f400 tx=0x7fa24c00f710 comp rx=0 tx=0).stop 2024-05-09T02:19:38.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.045+0000 7fa259a15700 1 -- 172.21.15.19:0/514316382 shutdown_connections 2024-05-09T02:19:38.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.045+0000 7fa259a15700 1 --2- 172.21.15.19:0/514316382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa254108740 0x7fa2541100a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.045+0000 7fa259a15700 1 --2- 172.21.15.19:0/514316382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2541034c0 0x7fa2541038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.045+0000 7fa259a15700 1 --2- 172.21.15.19:0/514316382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa254107df0 0x7fa254108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.045+0000 7fa259a15700 1 -- 172.21.15.19:0/514316382 >> 172.21.15.19:0/514316382 conn(0x7fa2540fedc0 msgr2=0x7fa2541011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:38.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.052+0000 7fa259a15700 1 -- 172.21.15.19:0/514316382 shutdown_connections 2024-05-09T02:19:38.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.053+0000 7fa259a15700 1 -- 172.21.15.19:0/514316382 wait complete. 2024-05-09T02:19:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.054+0000 7fa259a15700 1 Processor -- start 2024-05-09T02:19:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.054+0000 7fa259a15700 1 -- start start 2024-05-09T02:19:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.055+0000 7fa259a15700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2541034c0 0x7fa254075360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.055+0000 7fa259a15700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa254107df0 0x7fa25406f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.055+0000 7fa259a15700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa254108740 0x7fa25406fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.055+0000 7fa259a15700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa25406daa0 con 0x7fa254107df0 2024-05-09T02:19:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.055+0000 7fa259a15700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa25406d920 con 0x7fa254108740 2024-05-09T02:19:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.055+0000 7fa259a15700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa25406dc20 con 0x7fa2541034c0 2024-05-09T02:19:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.055+0000 7fa252ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2541034c0 0x7fa254075360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.055+0000 7fa252ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2541034c0 0x7fa254075360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:51084/0 (socket says 172.21.15.19:51084) 2024-05-09T02:19:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.055+0000 7fa252ffd700 1 -- 172.21.15.19:0/4241796826 learned_addr learned my addr 172.21.15.19:0/4241796826 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.056+0000 7fa252ffd700 1 -- 172.21.15.19:0/4241796826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa254108740 msgr2=0x7fa25406fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T02:19:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.056+0000 7fa252ffd700 1 --2- 172.21.15.19:0/4241796826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa254108740 0x7fa25406fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.056+0000 7fa252ffd700 1 -- 172.21.15.19:0/4241796826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa254107df0 msgr2=0x7fa25406f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.056+0000 7fa252ffd700 1 --2- 172.21.15.19:0/4241796826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa254107df0 0x7fa25406f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.056+0000 7fa252ffd700 1 -- 172.21.15.19:0/4241796826 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa254070270 con 0x7fa2541034c0 2024-05-09T02:19:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.056+0000 7fa252ffd700 1 --2- 172.21.15.19:0/4241796826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2541034c0 0x7fa254075360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa24400db20 tx=0x7fa24400dee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.057+0000 7fa258a13700 1 -- 172.21.15.19:0/4241796826 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24400ca90 con 0x7fa2541034c0 2024-05-09T02:19:38.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.057+0000 7fa258a13700 1 -- 172.21.15.19:0/4241796826 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2440044d0 con 0x7fa2541034c0 2024-05-09T02:19:38.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.057+0000 7fa258a13700 1 -- 172.21.15.19:0/4241796826 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244003e30 con 0x7fa2541034c0 2024-05-09T02:19:38.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.057+0000 7fa259a15700 1 -- 172.21.15.19:0/4241796826 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2541ad870 con 0x7fa2541034c0 2024-05-09T02:19:38.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.057+0000 7fa259a15700 1 -- 172.21.15.19:0/4241796826 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2541addd0 con 0x7fa2541034c0 2024-05-09T02:19:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.064+0000 7fa26842bb80 1 Processor -- start 2024-05-09T02:19:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.065+0000 7fa26842bb80 1 -- start start 2024-05-09T02:19:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.065+0000 7fa26842bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c4715d00 0x5643c481e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.065+0000 7fa26842bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c469b4a0 0x5643c481e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.065+0000 7fa26842bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c4818400 0x5643c4818810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.065+0000 7fa26842bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5643c473b0a0 con 0x5643c4818400 2024-05-09T02:19:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.065+0000 7fa26842bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5643c46e09a0 con 0x5643c4715d00 2024-05-09T02:19:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.065+0000 7fa26842bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5643c4737060 con 0x5643c469b4a0 2024-05-09T02:19:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.065+0000 7fa2537fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c4715d00 0x5643c481e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.066+0000 7fa2537fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c4715d00 0x5643c481e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:60406/0 (socket says 172.21.15.19:60406) 2024-05-09T02:19:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.066+0000 7fa2537fe700 1 -- 172.21.15.19:0/3911863236 learned_addr learned my addr 172.21.15.19:0/3911863236 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.067+0000 7fa252ffd700 1 --2- 172.21.15.19:0/3911863236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c469b4a0 0x5643c481e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.067+0000 7fa252ffd700 1 -- 172.21.15.19:0/3911863236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c4715d00 msgr2=0x5643c481e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.067+0000 7fa252ffd700 1 --2- 172.21.15.19:0/3911863236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c4715d00 0x5643c481e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.067+0000 7fa252ffd700 1 -- 172.21.15.19:0/3911863236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c4818400 msgr2=0x5643c4818810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.067+0000 7fa252ffd700 1 --2- 172.21.15.19:0/3911863236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c4818400 0x5643c4818810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.067+0000 7fa252ffd700 1 -- 172.21.15.19:0/3911863236 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c46d1500 con 0x5643c469b4a0 2024-05-09T02:19:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.067+0000 7fa252ffd700 1 --2- 172.21.15.19:0/3911863236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c469b4a0 0x5643c481e780 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2440df360 tx=0x7fa2440ea6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58f69565e5bbc356 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.068+0000 7fa258a13700 1 -- 172.21.15.19:0/4241796826 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa244016410 con 0x7fa2541034c0 2024-05-09T02:19:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.068+0000 7fa239ffb700 1 -- 172.21.15.19:0/3911863236 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2440f4670 con 0x5643c469b4a0 2024-05-09T02:19:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.068+0000 7fa239ffb700 1 -- 172.21.15.19:0/3911863236 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2440ef070 con 0x5643c469b4a0 2024-05-09T02:19:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.068+0000 7fa239ffb700 1 -- 172.21.15.19:0/3911863236 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2440eb6a0 con 0x5643c469b4a0 2024-05-09T02:19:38.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.069+0000 7fa258a13700 1 --2- 172.21.15.19:0/4241796826 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa23c0b2af0 0x7fa23c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.069+0000 7fa2537fe700 1 --2- 172.21.15.19:0/4241796826 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa23c0b2af0 0x7fa23c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.070+0000 7fa26842bb80 1 -- 172.21.15.19:0/3911863236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c469b4a0 msgr2=0x5643c481e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.070+0000 7fa26842bb80 1 --2- 172.21.15.19:0/3911863236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c469b4a0 0x5643c481e780 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2440df360 tx=0x7fa2440ea6b0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.070+0000 7fa26842bb80 1 -- 172.21.15.19:0/3911863236 shutdown_connections 2024-05-09T02:19:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.070+0000 7fa26842bb80 1 --2- 172.21.15.19:0/3911863236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c4818400 0x5643c4818810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.070+0000 7fa26842bb80 1 --2- 172.21.15.19:0/3911863236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c4715d00 0x5643c481e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.070+0000 7fa26842bb80 1 --2- 172.21.15.19:0/3911863236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c469b4a0 0x5643c481e780 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.070+0000 7fa26842bb80 1 -- 172.21.15.19:0/3911863236 >> 172.21.15.19:0/3911863236 conn(0x5643c469ef70 msgr2=0x5643c4813220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.070+0000 7fa2537fe700 1 --2- 172.21.15.19:0/4241796826 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa23c0b2af0 0x7fa23c0b5000 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00f210 tx=0x7fa24c00ded0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.070+0000 7fa258a13700 1 -- 172.21.15.19:0/4241796826 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7fa2440e1ca0 con 0x7fa2541034c0 2024-05-09T02:19:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.071+0000 7fa26842bb80 1 -- 172.21.15.19:0/3911863236 shutdown_connections 2024-05-09T02:19:38.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.071+0000 7fa26842bb80 1 -- 172.21.15.19:0/3911863236 wait complete. 2024-05-09T02:19:38.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.071+0000 7fa26842bb80 1 -- start start 2024-05-09T02:19:38.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.072+0000 7fa26842bb80 1 Processor -- start 2024-05-09T02:19:38.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.072+0000 7fa26842bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c469b4a0 0x5643c4821fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.072+0000 7fa26842bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c4715d00 0x5643c4822500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.072+0000 7fa26842bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4818400 0x5643c4822a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.072+0000 7fa26842bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5643c4819160 con 0x5643c469b4a0 2024-05-09T02:19:38.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.072+0000 7fa26842bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5643c46d5b20 con 0x5643c4715d00 2024-05-09T02:19:38.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.072+0000 7fa26842bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5643c4825020 con 0x5643c4818400 2024-05-09T02:19:38.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.073+0000 7fa26842bb80 10 client.0 _ll_register_callbacks cb 0x5643c45ac6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:19:38.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.073+0000 7fa2537fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4818400 0x5643c4822a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:38.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.073+0000 7fa2527fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c4715d00 0x5643c4822500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:38.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.073+0000 7fa2527fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c4715d00 0x5643c4822500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:60412/0 (socket says 172.21.15.19:60412) 2024-05-09T02:19:38.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.073+0000 7fa2527fc700 1 -- 172.21.15.19:0/3240610295 learned_addr learned my addr 172.21.15.19:0/3240610295 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:38.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.074+0000 7fa2537fe700 1 -- 172.21.15.19:0/3240610295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c4715d00 msgr2=0x5643c4822500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:38.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.074+0000 7fa2537fe700 1 --2- 172.21.15.19:0/3240610295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c4715d00 0x5643c4822500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.074+0000 7fa2537fe700 1 -- 172.21.15.19:0/3240610295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c469b4a0 msgr2=0x5643c4821fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:38.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.074+0000 7fa2537fe700 1 --2- 172.21.15.19:0/3240610295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c469b4a0 0x5643c4821fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.074+0000 7fa2537fe700 1 -- 172.21.15.19:0/3240610295 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c45b6490 con 0x5643c4818400 2024-05-09T02:19:38.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.074+0000 7fa2537fe700 1 --2- 172.21.15.19:0/3240610295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4818400 0x5643c4822a40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00dc50 tx=0x7fa24c00f710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8b145b4d9ed3415 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.075+0000 7fa232ffd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:19:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.075+0000 7fa232ffd700 1 -- 172.21.15.19:0/3240610295 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c0184e0 con 0x5643c4818400 2024-05-09T02:19:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.075+0000 7fa232ffd700 1 -- 172.21.15.19:0/3240610295 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7fa24c018680 con 0x5643c4818400 2024-05-09T02:19:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.075+0000 7fa26842bb80 1 -- 172.21.15.19:0/3240610295 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2440ef070 con 0x5643c4818400 2024-05-09T02:19:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.076+0000 7fa232ffd700 1 -- 172.21.15.19:0/3240610295 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa24c031050 con 0x5643c4818400 2024-05-09T02:19:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.076+0000 7fa26842bb80 10 client.26059 fetch_fsmap learned FSMap version 50 2024-05-09T02:19:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.076+0000 7fa26842bb80 1 -- 172.21.15.19:0/3240610295 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5643c48409f0 con 0x5643c4818400 2024-05-09T02:19:38.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.076+0000 7fa232ffd700 1 -- 172.21.15.19:0/3240610295 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2440ef070 con 0x5643c4818400 2024-05-09T02:19:38.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.077+0000 7fa232ffd700 1 -- 172.21.15.19:0/3240610295 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7fa24c005250 con 0x5643c4818400 2024-05-09T02:19:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.077+0000 7fa26842bb80 10 client.26059 fetch_fsmap finished waiting for FSMap version 50 2024-05-09T02:19:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.077+0000 7fa26842bb80 10 client.26059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:19:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.077+0000 7fa26842bb80 20 client.26059 populate_metadata read hostname 'smithi019' 2024-05-09T02:19:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.077+0000 7fa26842bb80 1 --2- 172.21.15.19:0/3240610295 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5643c4843080 0x5643c48454e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.077+0000 7fa26842bb80 4 client.26059 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:19:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.077+0000 7fa26842bb80 1 -- 172.21.15.19:0/3240610295 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643c4748a80 con 0x5643c4843080 2024-05-09T02:19:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.077+0000 7fa252ffd700 1 --2- 172.21.15.19:0/3240610295 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5643c4843080 0x5643c48454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.078+0000 7fa252ffd700 1 --2- 172.21.15.19:0/3240610295 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5643c4843080 0x5643c48454e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65f71ddeedb9704c server_cookie=90fbc2003d0eb5e8 in_seq=0 out_seq=0 2024-05-09T02:19:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.078+0000 7fa232ffd700 10 client.26059 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:19:38.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.079+0000 7fa232ffd700 1 -- 172.21.15.19:0/3240610295 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5643c4748a80 con 0x5643c4843080 2024-05-09T02:19:38.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.079+0000 7fa232ffd700 10 client.26059 handle_command_reply: tid=0 2024-05-09T02:19:38.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.079+0000 7fa26842bb80 1 client.26059 shutdown 2024-05-09T02:19:38.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.079+0000 7fa26842bb80 1 -- 172.21.15.19:0/3240610295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4818400 msgr2=0x5643c4822a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:38.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.079+0000 7fa26842bb80 1 --2- 172.21.15.19:0/3240610295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4818400 0x5643c4822a40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00dc50 tx=0x7fa24c00f710 comp rx=0 tx=0).stop 2024-05-09T02:19:38.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.080+0000 7fa26842bb80 1 -- 172.21.15.19:0/3240610295 shutdown_connections 2024-05-09T02:19:38.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.080+0000 7fa26842bb80 1 --2- 172.21.15.19:0/3240610295 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5643c4843080 0x5643c48454e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.080+0000 7fa26842bb80 1 --2- 172.21.15.19:0/3240610295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4818400 0x5643c4822a40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.080+0000 7fa26842bb80 1 --2- 172.21.15.19:0/3240610295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c469b4a0 0x5643c4821fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.080+0000 7fa26842bb80 1 --2- 172.21.15.19:0/3240610295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c4715d00 0x5643c4822500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.080+0000 7fa26842bb80 1 -- 172.21.15.19:0/3240610295 >> 172.21.15.19:0/3240610295 conn(0x5643c469ef70 msgr2=0x5643c480cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:38.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.080+0000 7fa232ffd700 0 client.26059 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:19:38.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.080+0000 7fa26842bb80 1 -- 172.21.15.19:0/3240610295 shutdown_connections 2024-05-09T02:19:38.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.080+0000 7fa26842bb80 1 -- 172.21.15.19:0/3240610295 wait complete. 2024-05-09T02:19:38.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.080+0000 7fa26842bb80 20 client.26059 trim_cache size 0 max 16384 2024-05-09T02:19:38.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.116+0000 7fa26842bb80 1 Processor -- start 2024-05-09T02:19:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.116+0000 7fa26842bb80 1 -- start start 2024-05-09T02:19:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.117+0000 7fa26842bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c469b4a0 0x5643c4820f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.117+0000 7fa26842bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c4715d00 0x5643c480da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.117+0000 7fa26842bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4818400 0x5643c4822880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.117+0000 7fa26842bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541b5540 con 0x5643c4715d00 2024-05-09T02:19:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.117+0000 7fa26842bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541b71a0 con 0x5643c469b4a0 2024-05-09T02:19:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.117+0000 7fa26842bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541b5ca0 con 0x5643c4818400 2024-05-09T02:19:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.117+0000 7fa2527fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c4715d00 0x5643c480da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.117+0000 7fa2537fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4818400 0x5643c4822880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.117+0000 7fa2537fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4818400 0x5643c4822880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:51100/0 (socket says 172.21.15.19:51100) 2024-05-09T02:19:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.117+0000 7fa2537fe700 1 -- 172.21.15.19:0/1659531167 learned_addr learned my addr 172.21.15.19:0/1659531167 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:38.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.118+0000 7fa252ffd700 1 --2- 172.21.15.19:0/1659531167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c469b4a0 0x5643c4820f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:38.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.118+0000 7fa2527fc700 1 -- 172.21.15.19:0/1659531167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4818400 msgr2=0x5643c4822880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:38.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.118+0000 7fa2527fc700 1 --2- 172.21.15.19:0/1659531167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4818400 0x5643c4822880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.118+0000 7fa2527fc700 1 -- 172.21.15.19:0/1659531167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c469b4a0 msgr2=0x5643c4820f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:38.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.118+0000 7fa2527fc700 1 --2- 172.21.15.19:0/1659531167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c469b4a0 0x5643c4820f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.118+0000 7fa2527fc700 1 -- 172.21.15.19:0/1659531167 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c480f9d0 con 0x5643c4715d00 2024-05-09T02:19:38.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.118+0000 7fa2527fc700 1 --2- 172.21.15.19:0/1659531167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c4715d00 0x5643c480da20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x5643c4813e30 tx=0x7fa24800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40e9be586034d51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:38.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.118+0000 7fa2397fa700 1 -- 172.21.15.19:0/1659531167 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24800ec20 con 0x5643c4715d00 2024-05-09T02:19:38.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.119+0000 7fa2397fa700 1 -- 172.21.15.19:0/1659531167 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa24800edc0 con 0x5643c4715d00 2024-05-09T02:19:38.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.119+0000 7fa2397fa700 1 -- 172.21.15.19:0/1659531167 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248010430 con 0x5643c4715d00 2024-05-09T02:19:38.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.120+0000 7fa26842bb80 1 -- 172.21.15.19:0/1659531167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c4715d00 msgr2=0x5643c480da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:38.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.120+0000 7fa26842bb80 1 --2- 172.21.15.19:0/1659531167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c4715d00 0x5643c480da20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x5643c4813e30 tx=0x7fa24800ccf0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.120+0000 7fa26842bb80 1 -- 172.21.15.19:0/1659531167 shutdown_connections 2024-05-09T02:19:38.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.120+0000 7fa26842bb80 1 --2- 172.21.15.19:0/1659531167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4818400 0x5643c4822880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.120+0000 7fa26842bb80 1 --2- 172.21.15.19:0/1659531167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c469b4a0 0x5643c4820f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.120+0000 7fa26842bb80 1 --2- 172.21.15.19:0/1659531167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c4715d00 0x5643c480da20 secure :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0x5643c4813e30 tx=0x7fa24800ccf0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.120+0000 7fa26842bb80 1 -- 172.21.15.19:0/1659531167 >> 172.21.15.19:0/1659531167 conn(0x5643c469ef70 msgr2=0x5643c481bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:38.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.121+0000 7fa26842bb80 1 -- 172.21.15.19:0/1659531167 shutdown_connections 2024-05-09T02:19:38.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.121+0000 7fa26842bb80 1 -- 172.21.15.19:0/1659531167 wait complete. 2024-05-09T02:19:38.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.122+0000 7fa26842bb80 1 -- start start 2024-05-09T02:19:38.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.122+0000 7fa26842bb80 1 Processor -- start 2024-05-09T02:19:38.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.123+0000 7fa26842bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c469b4a0 0x5643c482a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.123+0000 7fa26842bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c4818400 0x5643c4825a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.123+0000 7fa26842bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4825f60 0x5643c4826370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.123+0000 7fa26842bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541b6a40 con 0x5643c4818400 2024-05-09T02:19:38.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.123+0000 7fa26842bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541b6370 con 0x5643c469b4a0 2024-05-09T02:19:38.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.123+0000 7fa26842bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541b3500 con 0x5643c4825f60 2024-05-09T02:19:38.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.123+0000 7fa26842bb80 10 client.0 _ll_register_callbacks cb 0x5643c45ac6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:19:38.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.123+0000 7fa2527fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c4818400 0x5643c4825a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:38.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.123+0000 7fa2537fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4825f60 0x5643c4826370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:38.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.123+0000 7fa252ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c469b4a0 0x5643c482a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:38.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.123+0000 7fa2537fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4825f60 0x5643c4826370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:51106/0 (socket says 172.21.15.19:51106) 2024-05-09T02:19:38.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.123+0000 7fa252ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c469b4a0 0x5643c482a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:60430/0 (socket says 172.21.15.19:60430) 2024-05-09T02:19:38.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.123+0000 7fa2537fe700 1 -- 172.21.15.19:0/4009177013 learned_addr learned my addr 172.21.15.19:0/4009177013 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:38.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.124+0000 7fa2537fe700 1 -- 172.21.15.19:0/4009177013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c469b4a0 msgr2=0x5643c482a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:38.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.124+0000 7fa2537fe700 1 --2- 172.21.15.19:0/4009177013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c469b4a0 0x5643c482a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.124+0000 7fa2537fe700 1 -- 172.21.15.19:0/4009177013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c4818400 msgr2=0x5643c4825a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:38.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.124+0000 7fa2537fe700 1 --2- 172.21.15.19:0/4009177013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c4818400 0x5643c4825a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.124+0000 7fa2537fe700 1 -- 172.21.15.19:0/4009177013 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c48ac8c0 con 0x5643c4825f60 2024-05-09T02:19:38.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.126+0000 7fa2537fe700 1 --2- 172.21.15.19:0/4009177013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4825f60 0x5643c4826370 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa24c005810 tx=0x7fa24c018570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95bdf3e8fcbb217e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:38.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.126+0000 7fa232ffd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:19:38.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.128+0000 7fa232ffd700 1 -- 172.21.15.19:0/4009177013 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c039020 con 0x5643c4825f60 2024-05-09T02:19:38.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.128+0000 7fa232ffd700 1 -- 172.21.15.19:0/4009177013 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7fa24c005890 con 0x5643c4825f60 2024-05-09T02:19:38.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.128+0000 7fa26842bb80 1 -- 172.21.15.19:0/4009177013 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa24800edc0 con 0x5643c4825f60 2024-05-09T02:19:38.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.128+0000 7fa232ffd700 1 -- 172.21.15.19:0/4009177013 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa24c044050 con 0x5643c4825f60 2024-05-09T02:19:38.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.128+0000 7fa26842bb80 10 client.26065 fetch_fsmap learned FSMap version 50 2024-05-09T02:19:38.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.128+0000 7fa26842bb80 1 -- 172.21.15.19:0/4009177013 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5643c48adb10 con 0x5643c4825f60 2024-05-09T02:19:38.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.129+0000 7fa232ffd700 1 -- 172.21.15.19:0/4009177013 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa24800edc0 con 0x5643c4825f60 2024-05-09T02:19:38.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.129+0000 7fa232ffd700 1 -- 172.21.15.19:0/4009177013 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 50) ==== 8378+0+0 (secure 0 0 0) 0x7fa24c00d5d0 con 0x5643c4825f60 2024-05-09T02:19:38.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.130+0000 7fa26842bb80 10 client.26065 fetch_fsmap finished waiting for FSMap version 50 2024-05-09T02:19:38.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.130+0000 7fa26842bb80 10 client.26065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:19:38.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.130+0000 7fa26842bb80 20 client.26065 populate_metadata read hostname 'smithi019' 2024-05-09T02:19:38.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.130+0000 7fa26842bb80 1 --2- 172.21.15.19:0/4009177013 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5643c48ae0d0 0x5643c48b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:38.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.130+0000 7fa26842bb80 4 client.26065 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:19:38.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.130+0000 7fa26842bb80 1 -- 172.21.15.19:0/4009177013 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5643c4742b20 con 0x5643c48ae0d0 2024-05-09T02:19:38.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.130+0000 7fa252ffd700 1 --2- 172.21.15.19:0/4009177013 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5643c48ae0d0 0x5643c48b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:38.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.131+0000 7fa252ffd700 1 --2- 172.21.15.19:0/4009177013 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5643c48ae0d0 0x5643c48b04f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1de392cc180ded server_cookie=2a2b029e32c64a39 in_seq=0 out_seq=0 2024-05-09T02:19:38.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.131+0000 7fa232ffd700 10 client.26065 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:19:38.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.131+0000 7fa232ffd700 1 -- 172.21.15.19:0/4009177013 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5643c4742b20 con 0x5643c48ae0d0 2024-05-09T02:19:38.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.131+0000 7fa232ffd700 10 client.26065 handle_command_reply: tid=0 2024-05-09T02:19:38.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.131+0000 7fa26842bb80 1 client.26065 shutdown 2024-05-09T02:19:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.132+0000 7fa26842bb80 1 -- 172.21.15.19:0/4009177013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4825f60 msgr2=0x5643c4826370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.132+0000 7fa26842bb80 1 --2- 172.21.15.19:0/4009177013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4825f60 0x5643c4826370 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa24c005810 tx=0x7fa24c018570 comp rx=0 tx=0).stop 2024-05-09T02:19:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.132+0000 7fa26842bb80 1 -- 172.21.15.19:0/4009177013 shutdown_connections 2024-05-09T02:19:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.132+0000 7fa26842bb80 1 --2- 172.21.15.19:0/4009177013 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5643c48ae0d0 0x5643c48b04f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.132+0000 7fa26842bb80 1 --2- 172.21.15.19:0/4009177013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643c4825f60 0x5643c4826370 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.132+0000 7fa26842bb80 1 --2- 172.21.15.19:0/4009177013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643c469b4a0 0x5643c482a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.132+0000 7fa26842bb80 1 --2- 172.21.15.19:0/4009177013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5643c4818400 0x5643c4825a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.132+0000 7fa26842bb80 1 -- 172.21.15.19:0/4009177013 >> 172.21.15.19:0/4009177013 conn(0x5643c469ef70 msgr2=0x5643c480d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.133+0000 7fa232ffd700 0 client.26065 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:19:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.133+0000 7fa26842bb80 1 -- 172.21.15.19:0/4009177013 shutdown_connections 2024-05-09T02:19:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.133+0000 7fa26842bb80 1 -- 172.21.15.19:0/4009177013 wait complete. 2024-05-09T02:19:38.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.133+0000 7fa26842bb80 20 client.26065 trim_cache size 0 max 16384 2024-05-09T02:19:38.137 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:19:38.137 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T02:19:38.137 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T02:19:38.137 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:19:38.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.134+0000 7fa259a15700 1 -- 172.21.15.19:0/4241796826 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa23c0b2af0 msgr2=0x7fa23c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:38.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.134+0000 7fa259a15700 1 --2- 172.21.15.19:0/4241796826 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa23c0b2af0 0x7fa23c0b5000 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00f210 tx=0x7fa24c00ded0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.134+0000 7fa259a15700 1 -- 172.21.15.19:0/4241796826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2541034c0 msgr2=0x7fa254075360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:38.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.134+0000 7fa259a15700 1 --2- 172.21.15.19:0/4241796826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2541034c0 0x7fa254075360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa24400db20 tx=0x7fa24400dee0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.134+0000 7fa259a15700 1 -- 172.21.15.19:0/4241796826 shutdown_connections 2024-05-09T02:19:38.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.134+0000 7fa259a15700 1 --2- 172.21.15.19:0/4241796826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa254108740 0x7fa25406fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.134+0000 7fa259a15700 1 --2- 172.21.15.19:0/4241796826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2541034c0 0x7fa254075360 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.134+0000 7fa259a15700 1 --2- 172.21.15.19:0/4241796826 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa23c0b2af0 0x7fa23c0b5000 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.134+0000 7fa259a15700 1 --2- 172.21.15.19:0/4241796826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa254107df0 0x7fa25406f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:38.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.134+0000 7fa259a15700 1 -- 172.21.15.19:0/4241796826 >> 172.21.15.19:0/4241796826 conn(0x7fa2540fedc0 msgr2=0x7fa254107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:38.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.134+0000 7fa259a15700 1 -- 172.21.15.19:0/4241796826 shutdown_connections 2024-05-09T02:19:38.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:38.134+0000 7fa259a15700 1 -- 172.21.15.19:0/4241796826 wait complete. 2024-05-09T02:19:38.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:38 smithi019 ceph-mon[30151]: pgmap v257: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.8 MiB/s wr, 594 op/s 2024-05-09T02:19:38.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:38 smithi059 ceph-mon[34781]: pgmap v257: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.8 MiB/s wr, 594 op/s 2024-05-09T02:19:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:38 smithi196 ceph-mon[34718]: pgmap v257: 97 pgs: 97 active+clean; 4.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.8 MiB/s wr, 594 op/s 2024-05-09T02:19:38.986 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T02:19:38.986 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:19:39.457 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:19:39.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:39 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:19:39.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:39 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:19:39.748 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:39 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:19:39.748 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:39 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:19:39.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:39 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:19:39.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:39 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:19:40.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:40 smithi059 ceph-mon[34781]: pgmap v258: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 25 MiB/s wr, 1.86k op/s 2024-05-09T02:19:40.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:40 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:19:40.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:40 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:19:40.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:40 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:19:40.729 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:40 smithi019 ceph-mon[30151]: pgmap v258: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 25 MiB/s wr, 1.86k op/s 2024-05-09T02:19:40.729 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:40 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:19:40.729 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:40 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:19:40.729 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:40 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:19:40.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:40 smithi196 ceph-mon[34718]: pgmap v258: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 25 MiB/s wr, 1.86k op/s 2024-05-09T02:19:40.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:40 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:19:40.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:40 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:19:40.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:40 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:19:41.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.445+0000 7f9af69b2700 1 -- 172.21.15.19:0/4049690448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af010b280 msgr2=0x7f9af011d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:41.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.445+0000 7f9af69b2700 1 --2- 172.21.15.19:0/4049690448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af010b280 0x7f9af011d3f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40098f0 tx=0x7f9ae4009c00 comp rx=0 tx=0).stop 2024-05-09T02:19:41.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.445+0000 7f9af69b2700 1 -- 172.21.15.19:0/4049690448 shutdown_connections 2024-05-09T02:19:41.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.445+0000 7f9af69b2700 1 --2- 172.21.15.19:0/4049690448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af011d930 0x7f9af011fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:41.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.445+0000 7f9af69b2700 1 --2- 172.21.15.19:0/4049690448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9af010a930 0x7f9af010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:41.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.445+0000 7f9af69b2700 1 --2- 172.21.15.19:0/4049690448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af010b280 0x7f9af011d3f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:41.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.445+0000 7f9af69b2700 1 -- 172.21.15.19:0/4049690448 >> 172.21.15.19:0/4049690448 conn(0x7f9af006f4d0 msgr2=0x7f9af006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:41.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.446+0000 7f9af69b2700 1 -- 172.21.15.19:0/4049690448 shutdown_connections 2024-05-09T02:19:41.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.446+0000 7f9af69b2700 1 -- 172.21.15.19:0/4049690448 wait complete. 2024-05-09T02:19:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.447+0000 7f9af69b2700 1 Processor -- start 2024-05-09T02:19:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.447+0000 7f9af69b2700 1 -- start start 2024-05-09T02:19:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.447+0000 7f9af69b2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af010a930 0x7f9af01af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.447+0000 7f9af69b2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af010b280 0x7f9af01afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.447+0000 7f9af69b2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9af011d930 0x7f9af01a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.447+0000 7f9af69b2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9af01232f0 con 0x7f9af010a930 2024-05-09T02:19:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.448+0000 7f9af69b2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9af0123170 con 0x7f9af011d930 2024-05-09T02:19:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.448+0000 7f9af69b2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9af0123470 con 0x7f9af010b280 2024-05-09T02:19:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.448+0000 7f9aeffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af010a930 0x7f9af01af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.448+0000 7f9aeffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af010a930 0x7f9af01af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57962/0 (socket says 172.21.15.19:57962) 2024-05-09T02:19:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.448+0000 7f9aeffff700 1 -- 172.21.15.19:0/2566533372 learned_addr learned my addr 172.21.15.19:0/2566533372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.448+0000 7f9af4f4f700 1 --2- 172.21.15.19:0/2566533372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9af011d930 0x7f9af01a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.448+0000 7f9aef7fe700 1 --2- 172.21.15.19:0/2566533372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af010b280 0x7f9af01afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.448+0000 7f9aeffff700 1 -- 172.21.15.19:0/2566533372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af010b280 msgr2=0x7f9af01afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.448+0000 7f9aeffff700 1 --2- 172.21.15.19:0/2566533372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af010b280 0x7f9af01afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.448+0000 7f9aeffff700 1 -- 172.21.15.19:0/2566533372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9af011d930 msgr2=0x7f9af01a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.448+0000 7f9aeffff700 1 --2- 172.21.15.19:0/2566533372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9af011d930 0x7f9af01a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.448+0000 7f9aeffff700 1 -- 172.21.15.19:0/2566533372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af01aa260 con 0x7f9af010a930 2024-05-09T02:19:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.448+0000 7f9aef7fe700 1 --2- 172.21.15.19:0/2566533372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af010b280 0x7f9af01afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:19:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.448+0000 7f9aeffff700 1 --2- 172.21.15.19:0/2566533372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af010a930 0x7f9af01af9a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8007fa0 tx=0x7f9ae800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.449+0000 7f9aed7fa700 1 -- 172.21.15.19:0/2566533372 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae801a070 con 0x7f9af010a930 2024-05-09T02:19:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.449+0000 7f9af69b2700 1 -- 172.21.15.19:0/2566533372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af01aa550 con 0x7f9af010a930 2024-05-09T02:19:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.449+0000 7f9af69b2700 1 -- 172.21.15.19:0/2566533372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af01b6750 con 0x7f9af010a930 2024-05-09T02:19:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.449+0000 7f9aed7fa700 1 -- 172.21.15.19:0/2566533372 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae800dda0 con 0x7f9af010a930 2024-05-09T02:19:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.449+0000 7f9aed7fa700 1 -- 172.21.15.19:0/2566533372 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae801e610 con 0x7f9af010a930 2024-05-09T02:19:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.453+0000 7f9af69b2700 1 -- 172.21.15.19:0/2566533372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9adc005330 con 0x7f9af010a930 2024-05-09T02:19:41.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.455+0000 7f9aed7fa700 1 -- 172.21.15.19:0/2566533372 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9ae801d070 con 0x7f9af010a930 2024-05-09T02:19:41.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.456+0000 7f9aed7fa700 1 --2- 172.21.15.19:0/2566533372 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f9ad80b2bf0 0x7f9ad80b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:41.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.456+0000 7f9aef7fe700 1 --2- 172.21.15.19:0/2566533372 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f9ad80b2bf0 0x7f9ad80b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:41.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.456+0000 7f9aed7fa700 1 -- 172.21.15.19:0/2566533372 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f9ae801e3b0 con 0x7f9af010a930 2024-05-09T02:19:41.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.457+0000 7f9aef7fe700 1 --2- 172.21.15.19:0/2566533372 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f9ad80b2bf0 0x7f9ad80b5100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009810 tx=0x7f9ae4019040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:41.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.462+0000 7f9aed7fa700 1 -- 172.21.15.19:0/2566533372 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9ae8052df0 con 0x7f9af010a930 2024-05-09T02:19:41.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.858+0000 7f9af69b2700 1 -- 172.21.15.19:0/2566533372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9adc0058f0 con 0x7f9af010a930 2024-05-09T02:19:41.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.861+0000 7f9aed7fa700 1 -- 172.21.15.19:0/2566533372 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11923 (secure 0 0 0) 0x7f9ae8010070 con 0x7f9af010a930 2024-05-09T02:19:41.865 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:19:41.865 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:19:37.874806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24484,24463,24473,24457,24455]},"id":1}]} 2024-05-09T02:19:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.868+0000 7f9af69b2700 1 -- 172.21.15.19:0/2566533372 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f9ad80b2bf0 msgr2=0x7f9ad80b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.868+0000 7f9af69b2700 1 --2- 172.21.15.19:0/2566533372 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f9ad80b2bf0 0x7f9ad80b5100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009810 tx=0x7f9ae4019040 comp rx=0 tx=0).stop 2024-05-09T02:19:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.868+0000 7f9af69b2700 1 -- 172.21.15.19:0/2566533372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af010a930 msgr2=0x7f9af01af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.868+0000 7f9af69b2700 1 --2- 172.21.15.19:0/2566533372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af010a930 0x7f9af01af9a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8007fa0 tx=0x7f9ae800d3b0 comp rx=0 tx=0).stop 2024-05-09T02:19:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.869+0000 7f9af69b2700 1 -- 172.21.15.19:0/2566533372 shutdown_connections 2024-05-09T02:19:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.869+0000 7f9af69b2700 1 --2- 172.21.15.19:0/2566533372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9af011d930 0x7f9af01a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.869+0000 7f9af69b2700 1 --2- 172.21.15.19:0/2566533372 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f9ad80b2bf0 0x7f9ad80b5100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.869+0000 7f9af69b2700 1 --2- 172.21.15.19:0/2566533372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af010a930 0x7f9af01af9a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.869+0000 7f9af69b2700 1 --2- 172.21.15.19:0/2566533372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af010b280 0x7f9af01afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.869+0000 7f9af69b2700 1 -- 172.21.15.19:0/2566533372 >> 172.21.15.19:0/2566533372 conn(0x7f9af006f4d0 msgr2=0x7f9af011b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.869+0000 7f9af69b2700 1 -- 172.21.15.19:0/2566533372 shutdown_connections 2024-05-09T02:19:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:41.870+0000 7f9af69b2700 1 -- 172.21.15.19:0/2566533372 wait complete. 2024-05-09T02:19:41.878 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 51 2024-05-09T02:19:42.523 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:42 smithi019 ceph-mon[30151]: pgmap v259: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 20 MiB/s wr, 1.58k op/s 2024-05-09T02:19:42.524 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:42 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2566533372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:19:42.565 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T02:19:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 480.0 s, 1869.0 tps, lat 53.497 ms stddev 61.045 2024-05-09T02:19:42.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:42 smithi059 ceph-mon[34781]: pgmap v259: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 20 MiB/s wr, 1.58k op/s 2024-05-09T02:19:42.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:42 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2566533372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:19:42.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:42 smithi196 ceph-mon[34718]: pgmap v259: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 20 MiB/s wr, 1.58k op/s 2024-05-09T02:19:42.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:42 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2566533372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:19:42.981 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:19:44.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:44 smithi019 ceph-mon[30151]: pgmap v260: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 MiB/s wr, 1.58k op/s 2024-05-09T02:19:44.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:44 smithi059 ceph-mon[34781]: pgmap v260: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 MiB/s wr, 1.58k op/s 2024-05-09T02:19:44.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:44 smithi196 ceph-mon[34718]: pgmap v260: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 MiB/s wr, 1.58k op/s 2024-05-09T02:19:45.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.204+0000 7f3f74d36700 1 -- 172.21.15.19:0/1886841041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f70078640 msgr2=0x7f3f70114cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:45.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.204+0000 7f3f74d36700 1 --2- 172.21.15.19:0/1886841041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f70078640 0x7f3f70114cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009990 tx=0x7f3f60009ca0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.204+0000 7f3f74d36700 1 -- 172.21.15.19:0/1886841041 shutdown_connections 2024-05-09T02:19:45.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.204+0000 7f3f74d36700 1 --2- 172.21.15.19:0/1886841041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f70115200 0x7f3f70117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.204+0000 7f3f74d36700 1 --2- 172.21.15.19:0/1886841041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f70077cf0 0x7f3f70078100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.204+0000 7f3f74d36700 1 --2- 172.21.15.19:0/1886841041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f70078640 0x7f3f70114cc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.204+0000 7f3f74d36700 1 -- 172.21.15.19:0/1886841041 >> 172.21.15.19:0/1886841041 conn(0x7f3f70102060 msgr2=0x7f3f701044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:45.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.205+0000 7f3f74d36700 1 -- 172.21.15.19:0/1886841041 shutdown_connections 2024-05-09T02:19:45.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.205+0000 7f3f74d36700 1 -- 172.21.15.19:0/1886841041 wait complete. 2024-05-09T02:19:45.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.206+0000 7f3f74d36700 1 Processor -- start 2024-05-09T02:19:45.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.206+0000 7f3f74d36700 1 -- start start 2024-05-09T02:19:45.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.207+0000 7f3f74d36700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f70077cf0 0x7f3f701a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.207+0000 7f3f74d36700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f70078640 0x7f3f701a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.207+0000 7f3f74d36700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f70115200 0x7f3f701a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.207+0000 7f3f6e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f70077cf0 0x7f3f701a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:45.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.207+0000 7f3f6e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f70077cf0 0x7f3f701a14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57978/0 (socket says 172.21.15.19:57978) 2024-05-09T02:19:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.207+0000 7f3f65d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f70078640 0x7f3f701a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.207+0000 7f3f6ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f70115200 0x7f3f701a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.208+0000 7f3f6ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f70115200 0x7f3f701a5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:60460/0 (socket says 172.21.15.19:60460) 2024-05-09T02:19:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.208+0000 7f3f65d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f70078640 0x7f3f701a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:51140/0 (socket says 172.21.15.19:51140) 2024-05-09T02:19:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.208+0000 7f3f6e59c700 1 -- 172.21.15.19:0/3111276315 learned_addr learned my addr 172.21.15.19:0/3111276315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.208+0000 7f3f74d36700 1 -- 172.21.15.19:0/3111276315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7011aaf0 con 0x7f3f70077cf0 2024-05-09T02:19:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.208+0000 7f3f74d36700 1 -- 172.21.15.19:0/3111276315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7011a970 con 0x7f3f70115200 2024-05-09T02:19:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.208+0000 7f3f74d36700 1 -- 172.21.15.19:0/3111276315 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7011ac70 con 0x7f3f70078640 2024-05-09T02:19:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.208+0000 7f3f6ed9d700 1 -- 172.21.15.19:0/3111276315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f70078640 msgr2=0x7f3f701a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.208+0000 7f3f6ed9d700 1 --2- 172.21.15.19:0/3111276315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f70078640 0x7f3f701a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.208+0000 7f3f6ed9d700 1 -- 172.21.15.19:0/3111276315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f70077cf0 msgr2=0x7f3f701a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:45.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.208+0000 7f3f6ed9d700 1 --2- 172.21.15.19:0/3111276315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f70077cf0 0x7f3f701a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.208+0000 7f3f6ed9d700 1 -- 172.21.15.19:0/3111276315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f701a62e0 con 0x7f3f70115200 2024-05-09T02:19:45.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.209+0000 7f3f6ed9d700 1 --2- 172.21.15.19:0/3111276315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f70115200 0x7f3f701a5cf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800be20 tx=0x7f3f68009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.209+0000 7f3f67fff700 1 -- 172.21.15.19:0/3111276315 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68019070 con 0x7f3f70115200 2024-05-09T02:19:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.209+0000 7f3f67fff700 1 -- 172.21.15.19:0/3111276315 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f6800cc10 con 0x7f3f70115200 2024-05-09T02:19:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.210+0000 7f3f67fff700 1 -- 172.21.15.19:0/3111276315 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68012630 con 0x7f3f70115200 2024-05-09T02:19:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.210+0000 7f3f74d36700 1 -- 172.21.15.19:0/3111276315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f701a65d0 con 0x7f3f70115200 2024-05-09T02:19:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.210+0000 7f3f74d36700 1 -- 172.21.15.19:0/3111276315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f701a6a30 con 0x7f3f70115200 2024-05-09T02:19:45.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.215+0000 7f3f67fff700 1 -- 172.21.15.19:0/3111276315 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3f6801c070 con 0x7f3f70115200 2024-05-09T02:19:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.219+0000 7f3f67fff700 1 --2- 172.21.15.19:0/3111276315 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3f5c0b2a70 0x7f3f5c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.219+0000 7f3f67fff700 1 -- 172.21.15.19:0/3111276315 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f3f680e1b10 con 0x7f3f70115200 2024-05-09T02:19:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.219+0000 7f3f6e59c700 1 --2- 172.21.15.19:0/3111276315 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3f5c0b2a70 0x7f3f5c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.220+0000 7f3f8374cb80 1 Processor -- start 2024-05-09T02:19:45.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.220+0000 7f3f8374cb80 1 -- start start 2024-05-09T02:19:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.221+0000 7f3f8374cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685eb6160 0x555685f4f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.221+0000 7f3f8374cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685dcc4a0 0x555685f4fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.221+0000 7f3f8374cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685f496a0 0x555685f49ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.221+0000 7f3f6e59c700 1 --2- 172.21.15.19:0/3111276315 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3f5c0b2a70 0x7f3f5c0b4f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800ac30 tx=0x7f3f58005ef0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.221+0000 7f3f8374cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555685e6bff0 con 0x555685eb6160 2024-05-09T02:19:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.221+0000 7f3f8374cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555685e0a7f0 con 0x555685f496a0 2024-05-09T02:19:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.221+0000 7f3f8374cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555685e67fb0 con 0x555685dcc4a0 2024-05-09T02:19:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.221+0000 7f3f65d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685dcc4a0 0x555685f4fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.221+0000 7f3f65d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685dcc4a0 0x555685f4fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:51150/0 (socket says 172.21.15.19:51150) 2024-05-09T02:19:45.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.221+0000 7f3f65d9b700 1 -- 172.21.15.19:0/1068600740 learned_addr learned my addr 172.21.15.19:0/1068600740 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:45.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.221+0000 7f3f6ed9d700 1 --2- 172.21.15.19:0/1068600740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685f496a0 0x555685f49ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:45.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.221+0000 7f3f6e59c700 1 --2- 172.21.15.19:0/1068600740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685eb6160 0x555685f4f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.222+0000 7f3f6e59c700 1 -- 172.21.15.19:0/1068600740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685dcc4a0 msgr2=0x555685f4fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.222+0000 7f3f6e59c700 1 --2- 172.21.15.19:0/1068600740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685dcc4a0 0x555685f4fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.222+0000 7f3f6e59c700 1 -- 172.21.15.19:0/1068600740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685f496a0 msgr2=0x555685f49ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.222+0000 7f3f6e59c700 1 --2- 172.21.15.19:0/1068600740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685f496a0 0x555685f49ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.222+0000 7f3f6e59c700 1 -- 172.21.15.19:0/1068600740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555685e02370 con 0x555685eb6160 2024-05-09T02:19:45.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.224+0000 7f3f6e59c700 1 --2- 172.21.15.19:0/1068600740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685eb6160 0x555685f4f4f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800f6f0 tx=0x7f3f58012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b9e5ef9e69706c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:45.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.225+0000 7f3f64d99700 1 -- 172.21.15.19:0/1068600740 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58022070 con 0x555685eb6160 2024-05-09T02:19:45.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.225+0000 7f3f64d99700 1 -- 172.21.15.19:0/1068600740 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f58011ce0 con 0x555685eb6160 2024-05-09T02:19:45.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.225+0000 7f3f64d99700 1 -- 172.21.15.19:0/1068600740 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58010610 con 0x555685eb6160 2024-05-09T02:19:45.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.226+0000 7f3f8374cb80 1 -- 172.21.15.19:0/1068600740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685eb6160 msgr2=0x555685f4f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:45.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.226+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/1068600740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685eb6160 0x555685f4f4f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800f6f0 tx=0x7f3f58012000 comp rx=0 tx=0).stop 2024-05-09T02:19:45.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.227+0000 7f3f8374cb80 1 -- 172.21.15.19:0/1068600740 shutdown_connections 2024-05-09T02:19:45.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.227+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/1068600740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685f496a0 0x555685f49ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.227+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/1068600740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685eb6160 0x555685f4f4f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.227+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/1068600740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685dcc4a0 0x555685f4fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.227+0000 7f3f8374cb80 1 -- 172.21.15.19:0/1068600740 >> 172.21.15.19:0/1068600740 conn(0x555685e2c6a0 msgr2=0x555685f44400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:45.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.227+0000 7f3f8374cb80 1 -- 172.21.15.19:0/1068600740 shutdown_connections 2024-05-09T02:19:45.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.227+0000 7f3f8374cb80 1 -- 172.21.15.19:0/1068600740 wait complete. 2024-05-09T02:19:45.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.228+0000 7f3f8374cb80 1 -- start start 2024-05-09T02:19:45.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.228+0000 7f3f8374cb80 1 Processor -- start 2024-05-09T02:19:45.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.229+0000 7f3f8374cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685dcc4a0 0x555685f53270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.229+0000 7f3f8374cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685eb6160 0x555685f537b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.229+0000 7f3f8374cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685f496a0 0x555685f53cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.229+0000 7f3f8374cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555685ce7ba0 con 0x555685dcc4a0 2024-05-09T02:19:45.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.229+0000 7f3f8374cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555685e26150 con 0x555685eb6160 2024-05-09T02:19:45.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.229+0000 7f3f8374cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555685f561d0 con 0x555685f496a0 2024-05-09T02:19:45.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.230+0000 7f3f8374cb80 10 client.0 _ll_register_callbacks cb 0x555685cdd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:19:45.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.230+0000 7f3f65d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685eb6160 0x555685f537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:45.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.230+0000 7f3f65d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685eb6160 0x555685f537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:60478/0 (socket says 172.21.15.19:60478) 2024-05-09T02:19:45.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.230+0000 7f3f65d9b700 1 -- 172.21.15.19:0/4057665445 learned_addr learned my addr 172.21.15.19:0/4057665445 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:45.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.231+0000 7f3f65d9b700 1 -- 172.21.15.19:0/4057665445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685f496a0 msgr2=0x555685f53cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:19:45.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.231+0000 7f3f65d9b700 1 --2- 172.21.15.19:0/4057665445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685f496a0 0x555685f53cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.231+0000 7f3f65d9b700 1 -- 172.21.15.19:0/4057665445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685dcc4a0 msgr2=0x555685f53270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:45.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.231+0000 7f3f65d9b700 1 --2- 172.21.15.19:0/4057665445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685dcc4a0 0x555685f53270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.231+0000 7f3f65d9b700 1 -- 172.21.15.19:0/4057665445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555685ce7490 con 0x555685eb6160 2024-05-09T02:19:45.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.231+0000 7f3f65d9b700 1 --2- 172.21.15.19:0/4057665445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685eb6160 0x555685f537b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3f60005b40 tx=0x7f3f60003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93d3304fccde1e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:45.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.231+0000 7f3f55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:19:45.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.231+0000 7f3f55ffb700 1 -- 172.21.15.19:0/4057665445 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60005540 con 0x555685eb6160 2024-05-09T02:19:45.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.231+0000 7f3f55ffb700 1 -- 172.21.15.19:0/4057665445 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f3f6000b7d0 con 0x555685eb6160 2024-05-09T02:19:45.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.232+0000 7f3f8374cb80 1 -- 172.21.15.19:0/4057665445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f58011ce0 con 0x555685eb6160 2024-05-09T02:19:45.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.232+0000 7f3f55ffb700 1 -- 172.21.15.19:0/4057665445 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f3f600056e0 con 0x555685eb6160 2024-05-09T02:19:45.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.234+0000 7f3f8374cb80 10 client.26051 fetch_fsmap learned FSMap version 51 2024-05-09T02:19:45.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.234+0000 7f3f8374cb80 1 -- 172.21.15.19:0/4057665445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x555685f60a30 con 0x555685eb6160 2024-05-09T02:19:45.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.234+0000 7f3f55ffb700 1 -- 172.21.15.19:0/4057665445 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f60028070 con 0x555685eb6160 2024-05-09T02:19:45.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.235+0000 7f3f55ffb700 1 -- 172.21.15.19:0/4057665445 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7f3f60023bd0 con 0x555685eb6160 2024-05-09T02:19:45.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.235+0000 7f3f8374cb80 10 client.26051 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T02:19:45.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.235+0000 7f3f8374cb80 10 client.26051 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:19:45.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.235+0000 7f3f8374cb80 20 client.26051 populate_metadata read hostname 'smithi019' 2024-05-09T02:19:45.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.235+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/4057665445 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x555685f630c0 0x555685f65560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.235+0000 7f3f8374cb80 4 client.26051 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:19:45.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.235+0000 7f3f8374cb80 1 -- 172.21.15.19:0/4057665445 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3f58011ce0 con 0x555685f630c0 2024-05-09T02:19:45.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.247+0000 7f3f6e59c700 1 --2- 172.21.15.19:0/4057665445 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x555685f630c0 0x555685f65560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:45.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.248+0000 7f3f6e59c700 1 --2- 172.21.15.19:0/4057665445 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x555685f630c0 0x555685f65560 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5d3018b88c8f36aa server_cookie=3f003eff564fb847 in_seq=0 out_seq=0 2024-05-09T02:19:45.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.249+0000 7f3f55ffb700 10 client.26051 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:19:45.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.250+0000 7f3f55ffb700 1 -- 172.21.15.19:0/4057665445 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f3f58011ce0 con 0x555685f630c0 2024-05-09T02:19:45.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.250+0000 7f3f55ffb700 10 client.26051 handle_command_reply: tid=0 2024-05-09T02:19:45.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.250+0000 7f3f8374cb80 1 client.26051 shutdown 2024-05-09T02:19:45.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.254+0000 7f3f8374cb80 1 -- 172.21.15.19:0/4057665445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685eb6160 msgr2=0x555685f537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:45.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.255+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/4057665445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685eb6160 0x555685f537b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3f60005b40 tx=0x7f3f60003680 comp rx=0 tx=0).stop 2024-05-09T02:19:45.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.255+0000 7f3f8374cb80 1 -- 172.21.15.19:0/4057665445 shutdown_connections 2024-05-09T02:19:45.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.255+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/4057665445 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x555685f630c0 0x555685f65560 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.255+0000 7f3f55ffb700 0 client.26051 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:19:45.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.255+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/4057665445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685f496a0 0x555685f53cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.255+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/4057665445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685dcc4a0 0x555685f53270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.255+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/4057665445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685eb6160 0x555685f537b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.255+0000 7f3f8374cb80 1 -- 172.21.15.19:0/4057665445 >> 172.21.15.19:0/4057665445 conn(0x555685e2c6a0 msgr2=0x555685f441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:45.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.256+0000 7f3f8374cb80 1 -- 172.21.15.19:0/4057665445 shutdown_connections 2024-05-09T02:19:45.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.258+0000 7f3f8374cb80 1 -- 172.21.15.19:0/4057665445 wait complete. 2024-05-09T02:19:45.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.259+0000 7f3f8374cb80 20 client.26051 trim_cache size 0 max 16384 2024-05-09T02:19:45.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.300+0000 7f3f8374cb80 1 Processor -- start 2024-05-09T02:19:45.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.300+0000 7f3f8374cb80 1 -- start start 2024-05-09T02:19:45.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.300+0000 7f3f8374cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685dcc4a0 0x555685f52590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.300+0000 7f3f8374cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685eb6160 0x555685f53b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.300+0000 7f3f8374cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685f496a0 0x555685f48da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.300+0000 7f3f8374cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f701b53a0 con 0x555685f496a0 2024-05-09T02:19:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.300+0000 7f3f8374cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3f701b6fa0 con 0x555685eb6160 2024-05-09T02:19:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.300+0000 7f3f8374cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f701b5aa0 con 0x555685dcc4a0 2024-05-09T02:19:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.301+0000 7f3f6e59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685dcc4a0 0x555685f52590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.301+0000 7f3f6e59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685dcc4a0 0x555685f52590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:51174/0 (socket says 172.21.15.19:51174) 2024-05-09T02:19:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.301+0000 7f3f6e59c700 1 -- 172.21.15.19:0/2899969155 learned_addr learned my addr 172.21.15.19:0/2899969155 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.301+0000 7f3f6ed9d700 1 --2- 172.21.15.19:0/2899969155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685f496a0 0x555685f48da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.301+0000 7f3f65d9b700 1 --2- 172.21.15.19:0/2899969155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685eb6160 0x555685f53b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.302+0000 7f3f6e59c700 1 -- 172.21.15.19:0/2899969155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685eb6160 msgr2=0x555685f53b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.302+0000 7f3f6e59c700 1 --2- 172.21.15.19:0/2899969155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685eb6160 0x555685f53b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.302+0000 7f3f6e59c700 1 -- 172.21.15.19:0/2899969155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685f496a0 msgr2=0x555685f48da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.302+0000 7f3f6e59c700 1 --2- 172.21.15.19:0/2899969155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685f496a0 0x555685f48da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.302+0000 7f3f6e59c700 1 -- 172.21.15.19:0/2899969155 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555685f54330 con 0x555685dcc4a0 2024-05-09T02:19:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.302+0000 7f3f6e59c700 1 --2- 172.21.15.19:0/2899969155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685dcc4a0 0x555685f52590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3f58011720 tx=0x7f3f58000dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f6c1306e2fb7d80 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.304+0000 7f3f57fff700 1 -- 172.21.15.19:0/2899969155 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58015070 con 0x555685dcc4a0 2024-05-09T02:19:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.304+0000 7f3f57fff700 1 -- 172.21.15.19:0/2899969155 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f580328e0 con 0x555685dcc4a0 2024-05-09T02:19:45.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.307+0000 7f3f8374cb80 1 -- 172.21.15.19:0/2899969155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685dcc4a0 msgr2=0x555685f52590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:45.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.307+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/2899969155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685dcc4a0 0x555685f52590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3f58011720 tx=0x7f3f58000dc0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.309+0000 7f3f8374cb80 1 -- 172.21.15.19:0/2899969155 shutdown_connections 2024-05-09T02:19:45.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.309+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/2899969155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685f496a0 0x555685f48da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.309+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/2899969155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685dcc4a0 0x555685f52590 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.309+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/2899969155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685eb6160 0x555685f53b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.309+0000 7f3f8374cb80 1 -- 172.21.15.19:0/2899969155 >> 172.21.15.19:0/2899969155 conn(0x555685e2c6a0 msgr2=0x555685f48690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.311+0000 7f3f8374cb80 1 -- 172.21.15.19:0/2899969155 shutdown_connections 2024-05-09T02:19:45.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.311+0000 7f3f8374cb80 1 -- 172.21.15.19:0/2899969155 wait complete. 2024-05-09T02:19:45.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.311+0000 7f3f8374cb80 1 -- start start 2024-05-09T02:19:45.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.311+0000 7f3f8374cb80 1 Processor -- start 2024-05-09T02:19:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.312+0000 7f3f8374cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685dcc4a0 0x555685f5b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.312+0000 7f3f8374cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685eb6160 0x555685f6a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.312+0000 7f3f8374cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685f496a0 0x555685f6a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.312+0000 7f3f8374cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f701b6840 con 0x555685f496a0 2024-05-09T02:19:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.312+0000 7f3f8374cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3f701b6170 con 0x555685dcc4a0 2024-05-09T02:19:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.312+0000 7f3f8374cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f701b3360 con 0x555685eb6160 2024-05-09T02:19:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.312+0000 7f3f8374cb80 10 client.0 _ll_register_callbacks cb 0x555685cdd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:19:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.312+0000 7f3f6ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685f496a0 0x555685f6a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.313+0000 7f3f6ed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685f496a0 0x555685f6a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58008/0 (socket says 172.21.15.19:58008) 2024-05-09T02:19:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.313+0000 7f3f6ed9d700 1 -- 172.21.15.19:0/988650950 learned_addr learned my addr 172.21.15.19:0/988650950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.314+0000 7f3f6ed9d700 1 -- 172.21.15.19:0/988650950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685eb6160 msgr2=0x555685f6a2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:19:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.314+0000 7f3f6ed9d700 1 --2- 172.21.15.19:0/988650950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685eb6160 0x555685f6a2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.314+0000 7f3f6ed9d700 1 -- 172.21.15.19:0/988650950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685dcc4a0 msgr2=0x555685f5b590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:19:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.314+0000 7f3f6ed9d700 1 --2- 172.21.15.19:0/988650950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685dcc4a0 0x555685f5b590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.314+0000 7f3f6ed9d700 1 -- 172.21.15.19:0/988650950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555685f6ad70 con 0x555685f496a0 2024-05-09T02:19:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.315+0000 7f3f6ed9d700 1 --2- 172.21.15.19:0/988650950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685f496a0 0x555685f6a830 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3f680a4f50 tx=0x7f3f68014bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4dd4910645e0b84 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.315+0000 7f3f55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:19:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.316+0000 7f3f55ffb700 1 -- 172.21.15.19:0/988650950 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f680ad280 con 0x555685f496a0 2024-05-09T02:19:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.316+0000 7f3f55ffb700 1 -- 172.21.15.19:0/988650950 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f3f6800cdc0 con 0x555685f496a0 2024-05-09T02:19:45.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.319+0000 7f3f8374cb80 1 -- 172.21.15.19:0/988650950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f580328e0 con 0x555685f496a0 2024-05-09T02:19:45.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.319+0000 7f3f55ffb700 1 -- 172.21.15.19:0/988650950 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f3f680b1d30 con 0x555685f496a0 2024-05-09T02:19:45.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.320+0000 7f3f8374cb80 10 client.16308 fetch_fsmap learned FSMap version 51 2024-05-09T02:19:45.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.320+0000 7f3f8374cb80 1 -- 172.21.15.19:0/988650950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x555685fde520 con 0x555685f496a0 2024-05-09T02:19:45.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.320+0000 7f3f55ffb700 1 -- 172.21.15.19:0/988650950 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f580328e0 con 0x555685f496a0 2024-05-09T02:19:45.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.320+0000 7f3f55ffb700 1 -- 172.21.15.19:0/988650950 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7f3f680a4910 con 0x555685f496a0 2024-05-09T02:19:45.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.321+0000 7f3f8374cb80 10 client.16308 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T02:19:45.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.321+0000 7f3f8374cb80 10 client.16308 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:19:45.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.321+0000 7f3f8374cb80 20 client.16308 populate_metadata read hostname 'smithi019' 2024-05-09T02:19:45.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.321+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/988650950 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x555685fdec90 0x555685fe10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:45.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.321+0000 7f3f8374cb80 4 client.16308 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:19:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.321+0000 7f3f8374cb80 1 -- 172.21.15.19:0/988650950 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555685e799d0 con 0x555685fdec90 2024-05-09T02:19:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.321+0000 7f3f6e59c700 1 --2- 172.21.15.19:0/988650950 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x555685fdec90 0x555685fe10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.322+0000 7f3f55ffb700 10 client.16308 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:19:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.322+0000 7f3f6e59c700 1 --2- 172.21.15.19:0/988650950 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x555685fdec90 0x555685fe10b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=42ef2a443bc44ee9 server_cookie=7a7fd416d27cd98c in_seq=0 out_seq=0 2024-05-09T02:19:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.322+0000 7f3f55ffb700 1 -- 172.21.15.19:0/988650950 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555685e799d0 con 0x555685fdec90 2024-05-09T02:19:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.322+0000 7f3f55ffb700 10 client.16308 handle_command_reply: tid=0 2024-05-09T02:19:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.322+0000 7f3f8374cb80 1 client.16308 shutdown 2024-05-09T02:19:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.322+0000 7f3f8374cb80 1 -- 172.21.15.19:0/988650950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685f496a0 msgr2=0x555685f6a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.322+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/988650950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685f496a0 0x555685f6a830 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3f680a4f50 tx=0x7f3f68014bb0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.322+0000 7f3f8374cb80 1 -- 172.21.15.19:0/988650950 shutdown_connections 2024-05-09T02:19:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.322+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/988650950 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x555685fdec90 0x555685fe10b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.322+0000 7f3f55ffb700 0 client.16308 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:19:45.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.322+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/988650950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555685f496a0 0x555685f6a830 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.322+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/988650950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555685dcc4a0 0x555685f5b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.322+0000 7f3f8374cb80 1 --2- 172.21.15.19:0/988650950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555685eb6160 0x555685f6a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.322+0000 7f3f8374cb80 1 -- 172.21.15.19:0/988650950 >> 172.21.15.19:0/988650950 conn(0x555685e2c6a0 msgr2=0x555685f522f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:45.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.322+0000 7f3f8374cb80 1 -- 172.21.15.19:0/988650950 shutdown_connections 2024-05-09T02:19:45.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.323+0000 7f3f8374cb80 1 -- 172.21.15.19:0/988650950 wait complete. 2024-05-09T02:19:45.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.327+0000 7f3f8374cb80 20 client.16308 trim_cache size 0 max 16384 2024-05-09T02:19:45.330 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:19:45.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.329+0000 7f3f74d36700 1 -- 172.21.15.19:0/3111276315 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3f5c0b2a70 msgr2=0x7f3f5c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:45.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.329+0000 7f3f74d36700 1 --2- 172.21.15.19:0/3111276315 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3f5c0b2a70 0x7f3f5c0b4f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800ac30 tx=0x7f3f58005ef0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.329+0000 7f3f74d36700 1 -- 172.21.15.19:0/3111276315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f70115200 msgr2=0x7f3f701a5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:45.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.329+0000 7f3f74d36700 1 --2- 172.21.15.19:0/3111276315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f70115200 0x7f3f701a5cf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800be20 tx=0x7f3f68009ed0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.329+0000 7f3f74d36700 1 -- 172.21.15.19:0/3111276315 shutdown_connections 2024-05-09T02:19:45.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.329+0000 7f3f74d36700 1 --2- 172.21.15.19:0/3111276315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f70115200 0x7f3f701a5cf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.329+0000 7f3f74d36700 1 --2- 172.21.15.19:0/3111276315 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f3f5c0b2a70 0x7f3f5c0b4f80 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800ac30 tx=0x7f3f58005ef0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.329+0000 7f3f74d36700 1 --2- 172.21.15.19:0/3111276315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f70077cf0 0x7f3f701a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.329+0000 7f3f74d36700 1 --2- 172.21.15.19:0/3111276315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f70078640 0x7f3f701a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:45.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.329+0000 7f3f74d36700 1 -- 172.21.15.19:0/3111276315 >> 172.21.15.19:0/3111276315 conn(0x7f3f70102060 msgr2=0x7f3f70104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:45.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.330+0000 7f3f74d36700 1 -- 172.21.15.19:0/3111276315 shutdown_connections 2024-05-09T02:19:45.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:45.330+0000 7f3f74d36700 1 -- 172.21.15.19:0/3111276315 wait complete. 2024-05-09T02:19:46.398 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T02:19:46.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:46 smithi059 ceph-mon[34781]: pgmap v261: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 23 MiB/s wr, 1.82k op/s 2024-05-09T02:19:46.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:46 smithi196 ceph-mon[34718]: pgmap v261: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 23 MiB/s wr, 1.82k op/s 2024-05-09T02:19:46.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:46 smithi019 ceph-mon[30151]: pgmap v261: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 23 MiB/s wr, 1.82k op/s 2024-05-09T02:19:46.859 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:19:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 485.0 s, 2844.4 tps, lat 35.019 ms stddev 41.275 2024-05-09T02:19:47.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:47 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:19:47.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:47 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:19:47.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:47 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:19:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:48 smithi059 ceph-mon[34781]: pgmap v262: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-05-09T02:19:48.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:48 smithi019 ceph-mon[30151]: pgmap v262: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-05-09T02:19:48.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:48 smithi196 ceph-mon[34718]: pgmap v262: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-05-09T02:19:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.930+0000 7ff84f656700 1 -- 172.21.15.19:0/3330017109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff84810b280 msgr2=0x7ff8481198b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.930+0000 7ff84f656700 1 --2- 172.21.15.19:0/3330017109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff84810b280 0x7ff8481198b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff838009990 tx=0x7ff838009ca0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.930+0000 7ff84f656700 1 -- 172.21.15.19:0/3330017109 shutdown_connections 2024-05-09T02:19:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.930+0000 7ff84f656700 1 --2- 172.21.15.19:0/3330017109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff84810b280 0x7ff8481198b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.930+0000 7ff84f656700 1 --2- 172.21.15.19:0/3330017109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff848111bd0 0x7ff848111fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.930+0000 7ff84f656700 1 --2- 172.21.15.19:0/3330017109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff84810a930 0x7ff84810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.930+0000 7ff84f656700 1 -- 172.21.15.19:0/3330017109 >> 172.21.15.19:0/3330017109 conn(0x7ff84806f4d0 msgr2=0x7ff84806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.930+0000 7ff84f656700 1 -- 172.21.15.19:0/3330017109 shutdown_connections 2024-05-09T02:19:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.931+0000 7ff84f656700 1 -- 172.21.15.19:0/3330017109 wait complete. 2024-05-09T02:19:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.932+0000 7ff84f656700 1 Processor -- start 2024-05-09T02:19:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.932+0000 7ff84f656700 1 -- start start 2024-05-09T02:19:48.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.932+0000 7ff84f656700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff84810a930 0x7ff8481aa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.932+0000 7ff84f656700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84810b280 0x7ff8481aae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.932+0000 7ff84f656700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff848111bd0 0x7ff8481b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.932+0000 7ff84f656700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff84811cc60 con 0x7ff848111bd0 2024-05-09T02:19:48.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.932+0000 7ff84f656700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff84811cae0 con 0x7ff84810a930 2024-05-09T02:19:48.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.932+0000 7ff84f656700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff84811cde0 con 0x7ff84810b280 2024-05-09T02:19:48.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.932+0000 7ff84dbf3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff848111bd0 0x7ff8481b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.932+0000 7ff84cbf1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84810b280 0x7ff8481aae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.932+0000 7ff84d3f2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff84810a930 0x7ff8481aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.932+0000 7ff84dbf3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff848111bd0 0x7ff8481b59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53504/0 (socket says 172.21.15.19:53504) 2024-05-09T02:19:48.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.933+0000 7ff84dbf3700 1 -- 172.21.15.19:0/1284860767 learned_addr learned my addr 172.21.15.19:0/1284860767 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:48.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.933+0000 7ff84dbf3700 1 -- 172.21.15.19:0/1284860767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84810b280 msgr2=0x7ff8481aae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:48.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.933+0000 7ff84dbf3700 1 --2- 172.21.15.19:0/1284860767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84810b280 0x7ff8481aae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.933+0000 7ff84dbf3700 1 -- 172.21.15.19:0/1284860767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff84810a930 msgr2=0x7ff8481aa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:48.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.933+0000 7ff84dbf3700 1 --2- 172.21.15.19:0/1284860767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff84810a930 0x7ff8481aa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.933+0000 7ff84dbf3700 1 -- 172.21.15.19:0/1284860767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8481b5f00 con 0x7ff848111bd0 2024-05-09T02:19:48.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.933+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/1284860767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84810b280 0x7ff8481aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:19:48.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.933+0000 7ff84dbf3700 1 --2- 172.21.15.19:0/1284860767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff848111bd0 0x7ff8481b59c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff8380098e0 tx=0x7ff8380047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:48.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.934+0000 7ff83e7fc700 1 -- 172.21.15.19:0/1284860767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838026070 con 0x7ff848111bd0 2024-05-09T02:19:48.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.934+0000 7ff83e7fc700 1 -- 172.21.15.19:0/1284860767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff838021410 con 0x7ff848111bd0 2024-05-09T02:19:48.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.934+0000 7ff83e7fc700 1 -- 172.21.15.19:0/1284860767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838017650 con 0x7ff848111bd0 2024-05-09T02:19:48.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.934+0000 7ff84f656700 1 -- 172.21.15.19:0/1284860767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8481b60d0 con 0x7ff848111bd0 2024-05-09T02:19:48.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.934+0000 7ff84f656700 1 -- 172.21.15.19:0/1284860767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8481b6470 con 0x7ff848111bd0 2024-05-09T02:19:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.939+0000 7ff85e06cb80 1 Processor -- start 2024-05-09T02:19:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.940+0000 7ff83e7fc700 1 -- 172.21.15.19:0/1284860767 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff838009d10 con 0x7ff848111bd0 2024-05-09T02:19:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.940+0000 7ff85e06cb80 1 -- start start 2024-05-09T02:19:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.941+0000 7ff83e7fc700 1 --2- 172.21.15.19:0/1284860767 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff8340b2a70 0x7ff8340b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.942+0000 7ff83e7fc700 1 -- 172.21.15.19:0/1284860767 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7ff8380e32e0 con 0x7ff848111bd0 2024-05-09T02:19:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.942+0000 7ff85e06cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 0x55839cb87240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.942+0000 7ff85e06cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca044a0 0x55839cb87780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.942+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/1284860767 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff8340b2a70 0x7ff8340b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.942+0000 7ff85e06cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb81810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.942+0000 7ff85e06cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55839caa40a0 con 0x55839ca7ed00 2024-05-09T02:19:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.942+0000 7ff85e06cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55839ca499a0 con 0x55839ca044a0 2024-05-09T02:19:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.942+0000 7ff85e06cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55839caa0060 con 0x55839cb81400 2024-05-09T02:19:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.942+0000 7ff84cbf1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca044a0 0x55839cb87780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.942+0000 7ff84cbf1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca044a0 0x55839cb87780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:55986/0 (socket says 172.21.15.19:55986) 2024-05-09T02:19:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.942+0000 7ff84cbf1700 1 -- 172.21.15.19:0/2298736487 learned_addr learned my addr 172.21.15.19:0/2298736487 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.942+0000 7ff84d3f2700 1 --2- 172.21.15.19:0/2298736487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 0x55839cb87240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.942+0000 7ff84dbf3700 1 --2- 172.21.15.19:0/2298736487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb81810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.943+0000 7ff84cbf1700 1 -- 172.21.15.19:0/2298736487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 msgr2=0x55839cb81810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.943+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/2298736487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb81810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.943+0000 7ff84cbf1700 1 -- 172.21.15.19:0/2298736487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 msgr2=0x55839cb87240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.943+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/2298736487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 0x55839cb87240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.943+0000 7ff84cbf1700 1 -- 172.21.15.19:0/2298736487 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55839ca3a500 con 0x55839ca044a0 2024-05-09T02:19:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.943+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/1284860767 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff8340b2a70 0x7ff8340b4f80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff840007c00 tx=0x7ff8400172f0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.943+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/2298736487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca044a0 0x55839cb87780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff840009550 tx=0x7ff84001c030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d55c4293e2d04c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.943+0000 7ff8337fe700 1 -- 172.21.15.19:0/2298736487 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84001ab60 con 0x55839ca044a0 2024-05-09T02:19:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.943+0000 7ff8337fe700 1 -- 172.21.15.19:0/2298736487 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff84001ad00 con 0x55839ca044a0 2024-05-09T02:19:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.943+0000 7ff8337fe700 1 -- 172.21.15.19:0/2298736487 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840014670 con 0x55839ca044a0 2024-05-09T02:19:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.944+0000 7ff85e06cb80 1 -- 172.21.15.19:0/2298736487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca044a0 msgr2=0x55839cb87780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.944+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/2298736487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca044a0 0x55839cb87780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff840009550 tx=0x7ff84001c030 comp rx=0 tx=0).stop 2024-05-09T02:19:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.944+0000 7ff85e06cb80 1 -- 172.21.15.19:0/2298736487 shutdown_connections 2024-05-09T02:19:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.944+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/2298736487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb81810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.944+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/2298736487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 0x55839cb87240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.944+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/2298736487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca044a0 0x55839cb87780 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.944+0000 7ff85e06cb80 1 -- 172.21.15.19:0/2298736487 >> 172.21.15.19:0/2298736487 conn(0x55839ca07f70 msgr2=0x55839cb7c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.944+0000 7ff85e06cb80 1 -- 172.21.15.19:0/2298736487 shutdown_connections 2024-05-09T02:19:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.944+0000 7ff85e06cb80 1 -- 172.21.15.19:0/2298736487 wait complete. 2024-05-09T02:19:48.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.945+0000 7ff85e06cb80 1 -- start start 2024-05-09T02:19:48.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.945+0000 7ff85e06cb80 1 Processor -- start 2024-05-09T02:19:48.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.945+0000 7ff85e06cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca044a0 0x55839cb8afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.945+0000 7ff85e06cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 0x55839cb8b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.945+0000 7ff85e06cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb8ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.945+0000 7ff85e06cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55839cb820a0 con 0x55839ca7ed00 2024-05-09T02:19:48.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.945+0000 7ff85e06cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55839ca3eb20 con 0x55839ca044a0 2024-05-09T02:19:48.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.945+0000 7ff85e06cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55839cb82220 con 0x55839cb81400 2024-05-09T02:19:48.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.945+0000 7ff85e06cb80 10 client.0 _ll_register_callbacks cb 0x55839c9156c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:19:48.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff84cbf1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 0x55839cb8b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff84d3f2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca044a0 0x55839cb8afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff84cbf1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 0x55839cb8b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53510/0 (socket says 172.21.15.19:53510) 2024-05-09T02:19:48.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff84cbf1700 1 -- 172.21.15.19:0/1217696193 learned_addr learned my addr 172.21.15.19:0/1217696193 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:48.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff84dbf3700 1 --2- 172.21.15.19:0/1217696193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb8ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff84d3f2700 1 -- 172.21.15.19:0/1217696193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 msgr2=0x55839cb8ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:48.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff84d3f2700 1 --2- 172.21.15.19:0/1217696193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb8ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff84d3f2700 1 -- 172.21.15.19:0/1217696193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 msgr2=0x55839cb8b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:48.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff84d3f2700 1 --2- 172.21.15.19:0/1217696193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 0x55839cb8b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff84d3f2700 1 -- 172.21.15.19:0/1217696193 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55839c91f490 con 0x55839ca044a0 2024-05-09T02:19:48.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/1217696193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 0x55839cb8b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:19:48.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff84d3f2700 1 --2- 172.21.15.19:0/1217696193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca044a0 0x55839cb8afb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff844012680 tx=0x7ff844012990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c45c9bc62692b3c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:48.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff830ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:19:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff830ff9700 1 -- 172.21.15.19:0/1217696193 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844004020 con 0x55839ca044a0 2024-05-09T02:19:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff85e06cb80 1 -- 172.21.15.19:0/1217696193 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff84001ad00 con 0x55839ca044a0 2024-05-09T02:19:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff830ff9700 1 -- 172.21.15.19:0/1217696193 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7ff8440041c0 con 0x55839ca044a0 2024-05-09T02:19:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff830ff9700 1 -- 172.21.15.19:0/1217696193 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff8440043c0 con 0x55839ca044a0 2024-05-09T02:19:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff85e06cb80 10 client.26063 fetch_fsmap learned FSMap version 52 2024-05-09T02:19:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.946+0000 7ff85e06cb80 1 -- 172.21.15.19:0/1217696193 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55839cb98990 con 0x55839ca044a0 2024-05-09T02:19:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.947+0000 7ff830ff9700 1 -- 172.21.15.19:0/1217696193 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff84001ad00 con 0x55839ca044a0 2024-05-09T02:19:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.947+0000 7ff830ff9700 1 -- 172.21.15.19:0/1217696193 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 52) ==== 8374+0+0 (secure 0 0 0) 0x7ff844023070 con 0x55839ca044a0 2024-05-09T02:19:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.947+0000 7ff85e06cb80 10 client.26063 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T02:19:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.947+0000 7ff85e06cb80 10 client.26063 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:19:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.947+0000 7ff85e06cb80 20 client.26063 populate_metadata read hostname 'smithi019' 2024-05-09T02:19:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.947+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/1217696193 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55839cb9af20 0x55839cb9d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.947+0000 7ff85e06cb80 4 client.26063 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:19:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.947+0000 7ff85e06cb80 1 -- 172.21.15.19:0/1217696193 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55839cab1a80 con 0x55839cb9af20 2024-05-09T02:19:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.947+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/1217696193 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55839cb9af20 0x55839cb9d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.948+0000 7ff830ff9700 10 client.26063 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:19:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.948+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/1217696193 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55839cb9af20 0x55839cb9d3b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dff526eaafcaf725 server_cookie=f6805698ed6d2236 in_seq=0 out_seq=0 2024-05-09T02:19:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.949+0000 7ff830ff9700 1 -- 172.21.15.19:0/1217696193 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55839cab1a80 con 0x55839cb9af20 2024-05-09T02:19:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.949+0000 7ff830ff9700 10 client.26063 handle_command_reply: tid=0 2024-05-09T02:19:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.949+0000 7ff85e06cb80 1 client.26063 shutdown 2024-05-09T02:19:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.949+0000 7ff85e06cb80 1 -- 172.21.15.19:0/1217696193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca044a0 msgr2=0x55839cb8afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.949+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/1217696193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca044a0 0x55839cb8afb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff844012680 tx=0x7ff844012990 comp rx=0 tx=0).stop 2024-05-09T02:19:48.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.950+0000 7ff85e06cb80 1 -- 172.21.15.19:0/1217696193 shutdown_connections 2024-05-09T02:19:48.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.950+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/1217696193 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55839cb9af20 0x55839cb9d3b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.950+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/1217696193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb8ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.950+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/1217696193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca044a0 0x55839cb8afb0 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff844012680 tx=0x7ff844012990 comp rx=0 tx=0).stop 2024-05-09T02:19:48.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.950+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/1217696193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 0x55839cb8b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.950+0000 7ff85e06cb80 1 -- 172.21.15.19:0/1217696193 >> 172.21.15.19:0/1217696193 conn(0x55839ca07f70 msgr2=0x55839cb75ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:48.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.950+0000 7ff830ff9700 0 client.26063 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:19:48.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.951+0000 7ff85e06cb80 1 -- 172.21.15.19:0/1217696193 shutdown_connections 2024-05-09T02:19:48.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.951+0000 7ff85e06cb80 1 -- 172.21.15.19:0/1217696193 wait complete. 2024-05-09T02:19:48.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.951+0000 7ff85e06cb80 20 client.26063 trim_cache size 0 max 16384 2024-05-09T02:19:48.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.974+0000 7ff85e06cb80 1 Processor -- start 2024-05-09T02:19:48.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.974+0000 7ff85e06cb80 1 -- start start 2024-05-09T02:19:48.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.975+0000 7ff85e06cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 0x55839cb8a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.975+0000 7ff85e06cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.975+0000 7ff85e06cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839cb76700 0x55839cb96d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.975+0000 7ff85e06cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481bdc50 con 0x55839ca7ed00 2024-05-09T02:19:48.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.975+0000 7ff85e06cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481bf880 con 0x55839cb76700 2024-05-09T02:19:48.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.975+0000 7ff85e06cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481be350 con 0x55839cb81400 2024-05-09T02:19:48.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.975+0000 7ff84d3f2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 0x55839cb8a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.975+0000 7ff84d3f2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 0x55839cb8a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53526/0 (socket says 172.21.15.19:53526) 2024-05-09T02:19:48.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.975+0000 7ff84d3f2700 1 -- 172.21.15.19:0/1056186238 learned_addr learned my addr 172.21.15.19:0/1056186238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:48.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.975+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/1056186238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb8b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.975+0000 7ff84dbf3700 1 --2- 172.21.15.19:0/1056186238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839cb76700 0x55839cb96d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.975+0000 7ff84cbf1700 1 -- 172.21.15.19:0/1056186238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839cb76700 msgr2=0x55839cb96d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:48.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.976+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/1056186238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839cb76700 0x55839cb96d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.976+0000 7ff84cbf1700 1 -- 172.21.15.19:0/1056186238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 msgr2=0x55839cb8a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:48.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.976+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/1056186238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 0x55839cb8a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.976+0000 7ff84cbf1700 1 -- 172.21.15.19:0/1056186238 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55839cb89520 con 0x55839cb81400 2024-05-09T02:19:48.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.976+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/1056186238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb8b8a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff840006080 tx=0x7ff84001c030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95586a4969f4ba7b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:48.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.976+0000 7ff832ffd700 1 -- 172.21.15.19:0/1056186238 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84003c070 con 0x55839cb81400 2024-05-09T02:19:48.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.976+0000 7ff832ffd700 1 -- 172.21.15.19:0/1056186238 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff840014610 con 0x55839cb81400 2024-05-09T02:19:48.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.976+0000 7ff832ffd700 1 -- 172.21.15.19:0/1056186238 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840014b40 con 0x55839cb81400 2024-05-09T02:19:48.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.977+0000 7ff85e06cb80 1 -- 172.21.15.19:0/1056186238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 msgr2=0x55839cb8b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:48.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.977+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/1056186238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb8b8a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff840006080 tx=0x7ff84001c030 comp rx=0 tx=0).stop 2024-05-09T02:19:48.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.977+0000 7ff85e06cb80 1 -- 172.21.15.19:0/1056186238 shutdown_connections 2024-05-09T02:19:48.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.977+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/1056186238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839cb76700 0x55839cb96d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.977+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/1056186238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839ca7ed00 0x55839cb8a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.977+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/1056186238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb8b8a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.977+0000 7ff85e06cb80 1 -- 172.21.15.19:0/1056186238 >> 172.21.15.19:0/1056186238 conn(0x55839ca07f70 msgr2=0x55839cb75ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:48.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.977+0000 7ff85e06cb80 1 -- 172.21.15.19:0/1056186238 shutdown_connections 2024-05-09T02:19:48.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.977+0000 7ff85e06cb80 1 -- 172.21.15.19:0/1056186238 wait complete. 2024-05-09T02:19:48.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.978+0000 7ff85e06cb80 1 -- start start 2024-05-09T02:19:48.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.978+0000 7ff85e06cb80 1 Processor -- start 2024-05-09T02:19:48.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.979+0000 7ff85e06cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca7ed00 0x55839cb88d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.979+0000 7ff85e06cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839cb76700 0x55839cb930f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.979+0000 7ff85e06cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb93790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.979+0000 7ff85e06cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481bf120 con 0x55839cb76700 2024-05-09T02:19:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.979+0000 7ff85e06cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481bea20 con 0x55839ca7ed00 2024-05-09T02:19:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.979+0000 7ff85e06cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481bbc10 con 0x55839cb81400 2024-05-09T02:19:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.979+0000 7ff85e06cb80 10 client.0 _ll_register_callbacks cb 0x55839c9156c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:19:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.979+0000 7ff84cbf1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839cb76700 0x55839cb930f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.979+0000 7ff84cbf1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839cb76700 0x55839cb930f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53528/0 (socket says 172.21.15.19:53528) 2024-05-09T02:19:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.979+0000 7ff84cbf1700 1 -- 172.21.15.19:0/2996027633 learned_addr learned my addr 172.21.15.19:0/2996027633 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.979+0000 7ff84dbf3700 1 --2- 172.21.15.19:0/2996027633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb93790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.979+0000 7ff84d3f2700 1 --2- 172.21.15.19:0/2996027633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca7ed00 0x55839cb88d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.980+0000 7ff84cbf1700 1 -- 172.21.15.19:0/2996027633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 msgr2=0x55839cb93790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.980+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/2996027633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb93790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.980+0000 7ff84cbf1700 1 -- 172.21.15.19:0/2996027633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca7ed00 msgr2=0x55839cb88d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.980+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/2996027633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca7ed00 0x55839cb88d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.980+0000 7ff84cbf1700 1 -- 172.21.15.19:0/2996027633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55839cb87430 con 0x55839cb76700 2024-05-09T02:19:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.981+0000 7ff84cbf1700 1 --2- 172.21.15.19:0/2996027633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839cb76700 0x55839cb930f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff840018010 tx=0x7ff840017710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c219f4cbe3c83ce9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:48.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.981+0000 7ff830ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:19:48.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.981+0000 7ff830ff9700 1 -- 172.21.15.19:0/2996027633 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84003c040 con 0x55839cb76700 2024-05-09T02:19:48.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.981+0000 7ff830ff9700 1 -- 172.21.15.19:0/2996027633 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7ff840004c90 con 0x55839cb76700 2024-05-09T02:19:48.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.981+0000 7ff85e06cb80 1 -- 172.21.15.19:0/2996027633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff840014610 con 0x55839cb76700 2024-05-09T02:19:48.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.981+0000 7ff830ff9700 1 -- 172.21.15.19:0/2996027633 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff840024410 con 0x55839cb76700 2024-05-09T02:19:48.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.981+0000 7ff85e06cb80 10 client.16326 fetch_fsmap learned FSMap version 52 2024-05-09T02:19:48.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.981+0000 7ff85e06cb80 1 -- 172.21.15.19:0/2996027633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55839cc04bb0 con 0x55839cb76700 2024-05-09T02:19:48.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.982+0000 7ff830ff9700 1 -- 172.21.15.19:0/2996027633 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff840024a60 con 0x55839cb76700 2024-05-09T02:19:48.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.982+0000 7ff830ff9700 1 -- 172.21.15.19:0/2996027633 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 8374+0+0 (secure 0 0 0) 0x7ff84003f070 con 0x55839cb76700 2024-05-09T02:19:48.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.983+0000 7ff85e06cb80 10 client.16326 fetch_fsmap finished waiting for FSMap version 52 2024-05-09T02:19:48.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.983+0000 7ff85e06cb80 10 client.16326 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:19:48.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.983+0000 7ff85e06cb80 20 client.16326 populate_metadata read hostname 'smithi019' 2024-05-09T02:19:48.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.983+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/2996027633 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55839cc05010 0x55839cc07430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:48.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.983+0000 7ff85e06cb80 4 client.16326 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:19:48.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.983+0000 7ff85e06cb80 1 -- 172.21.15.19:0/2996027633 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff840014610 con 0x55839cc05010 2024-05-09T02:19:48.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.983+0000 7ff84d3f2700 1 --2- 172.21.15.19:0/2996027633 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55839cc05010 0x55839cc07430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.984+0000 7ff830ff9700 10 client.16326 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:19:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.984+0000 7ff84d3f2700 1 --2- 172.21.15.19:0/2996027633 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55839cc05010 0x55839cc07430 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e8fd0aeb55bf2a0d server_cookie=428b63aa904c59a4 in_seq=0 out_seq=0 2024-05-09T02:19:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.984+0000 7ff830ff9700 1 -- 172.21.15.19:0/2996027633 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff840014610 con 0x55839cc05010 2024-05-09T02:19:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.984+0000 7ff830ff9700 10 client.16326 handle_command_reply: tid=0 2024-05-09T02:19:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.984+0000 7ff85e06cb80 1 client.16326 shutdown 2024-05-09T02:19:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.984+0000 7ff85e06cb80 1 -- 172.21.15.19:0/2996027633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839cb76700 msgr2=0x55839cb930f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.984+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/2996027633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839cb76700 0x55839cb930f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff840018010 tx=0x7ff840017710 comp rx=0 tx=0).stop 2024-05-09T02:19:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.984+0000 7ff85e06cb80 1 -- 172.21.15.19:0/2996027633 shutdown_connections 2024-05-09T02:19:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.984+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/2996027633 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55839cc05010 0x55839cc07430 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.984+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/2996027633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55839cb81400 0x55839cb93790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.984+0000 7ff830ff9700 0 client.16326 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:19:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.984+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/2996027633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55839ca7ed00 0x55839cb88d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.984+0000 7ff85e06cb80 1 --2- 172.21.15.19:0/2996027633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55839cb76700 0x55839cb930f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.984+0000 7ff85e06cb80 1 -- 172.21.15.19:0/2996027633 >> 172.21.15.19:0/2996027633 conn(0x55839ca07f70 msgr2=0x55839cb75ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.985+0000 7ff85e06cb80 1 -- 172.21.15.19:0/2996027633 shutdown_connections 2024-05-09T02:19:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.985+0000 7ff85e06cb80 1 -- 172.21.15.19:0/2996027633 wait complete. 2024-05-09T02:19:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.985+0000 7ff85e06cb80 20 client.16326 trim_cache size 0 max 16384 2024-05-09T02:19:48.990 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:19:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.985+0000 7ff84f656700 1 -- 172.21.15.19:0/1284860767 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff8340b2a70 msgr2=0x7ff8340b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.985+0000 7ff84f656700 1 --2- 172.21.15.19:0/1284860767 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff8340b2a70 0x7ff8340b4f80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff840007c00 tx=0x7ff8400172f0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.985+0000 7ff84f656700 1 -- 172.21.15.19:0/1284860767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff848111bd0 msgr2=0x7ff8481b59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.985+0000 7ff84f656700 1 --2- 172.21.15.19:0/1284860767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff848111bd0 0x7ff8481b59c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff8380098e0 tx=0x7ff8380047b0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.985+0000 7ff84f656700 1 -- 172.21.15.19:0/1284860767 shutdown_connections 2024-05-09T02:19:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.985+0000 7ff84f656700 1 --2- 172.21.15.19:0/1284860767 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff8340b2a70 0x7ff8340b4f80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.985+0000 7ff84f656700 1 --2- 172.21.15.19:0/1284860767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff848111bd0 0x7ff8481b59c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.985+0000 7ff84f656700 1 --2- 172.21.15.19:0/1284860767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff84810a930 0x7ff8481aa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.986+0000 7ff84f656700 1 --2- 172.21.15.19:0/1284860767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84810b280 0x7ff8481aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.986+0000 7ff84f656700 1 -- 172.21.15.19:0/1284860767 >> 172.21.15.19:0/1284860767 conn(0x7ff84806f4d0 msgr2=0x7ff84810c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.986+0000 7ff84f656700 1 -- 172.21.15.19:0/1284860767 shutdown_connections 2024-05-09T02:19:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:48.986+0000 7ff84f656700 1 -- 172.21.15.19:0/1284860767 wait complete. 2024-05-09T02:19:50.188 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T02:19:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:19:50.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:50 smithi059 ceph-mon[34781]: pgmap v263: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 24 MiB/s wr, 1.82k op/s 2024-05-09T02:19:50.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:50 smithi019 ceph-mon[30151]: pgmap v263: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 24 MiB/s wr, 1.82k op/s 2024-05-09T02:19:50.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:50 smithi196 ceph-mon[34718]: pgmap v263: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 24 MiB/s wr, 1.82k op/s 2024-05-09T02:19:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 490.0 s, 2270.6 tps, lat 44.097 ms stddev 52.131 2024-05-09T02:19:52.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:52 smithi059 ceph-mon[34781]: pgmap v264: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.0 MiB/s wr, 558 op/s 2024-05-09T02:19:52.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:52 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:19:52.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:52 smithi019 ceph-mon[30151]: pgmap v264: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.0 MiB/s wr, 558 op/s 2024-05-09T02:19:52.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:52 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:19:52.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:52 smithi196 ceph-mon[34718]: pgmap v264: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.0 MiB/s wr, 558 op/s 2024-05-09T02:19:52.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:52 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:19:53.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.682+0000 7f84f810c700 1 -- 172.21.15.19:0/3377475870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84f0107e10 msgr2=0x7f84f0108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:53.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.682+0000 7f84f810c700 1 --2- 172.21.15.19:0/3377475870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84f0107e10 0x7f84f0108220 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f84e00099c0 tx=0x7f84e0009cd0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.682+0000 7f84f810c700 1 -- 172.21.15.19:0/3377475870 shutdown_connections 2024-05-09T02:19:53.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.682+0000 7f84f810c700 1 --2- 172.21.15.19:0/3377475870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f0108760 0x7f84f0110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.682+0000 7f84f810c700 1 --2- 172.21.15.19:0/3377475870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84f0103ca0 0x7f84f01040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.682+0000 7f84f810c700 1 --2- 172.21.15.19:0/3377475870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84f0107e10 0x7f84f0108220 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.682+0000 7f84f810c700 1 -- 172.21.15.19:0/3377475870 >> 172.21.15.19:0/3377475870 conn(0x7f84f00fe6f0 msgr2=0x7f84f0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:53.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.683+0000 7f84f810c700 1 -- 172.21.15.19:0/3377475870 shutdown_connections 2024-05-09T02:19:53.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.684+0000 7f84f810c700 1 -- 172.21.15.19:0/3377475870 wait complete. 2024-05-09T02:19:53.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.685+0000 7f84f810c700 1 Processor -- start 2024-05-09T02:19:53.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.685+0000 7f84f810c700 1 -- start start 2024-05-09T02:19:53.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.685+0000 7f84f810c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84f0103ca0 0x7f84f010c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.685+0000 7f84f810c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84f0107e10 0x7f84f010c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.685+0000 7f84f810c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f0108760 0x7f84f010cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.685+0000 7f84f56a7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84f0107e10 0x7f84f010c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.686+0000 7f84f810c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84f0113360 con 0x7f84f0103ca0 2024-05-09T02:19:53.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.686+0000 7f84f810c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01131e0 con 0x7f84f0108760 2024-05-09T02:19:53.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.686+0000 7f84f810c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01134e0 con 0x7f84f0107e10 2024-05-09T02:19:53.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.686+0000 7f84f5ea8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84f0103ca0 0x7f84f010c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.686+0000 7f84f56a7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84f0107e10 0x7f84f010c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:45014/0 (socket says 172.21.15.19:45014) 2024-05-09T02:19:53.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.686+0000 7f84f5ea8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84f0103ca0 0x7f84f010c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53548/0 (socket says 172.21.15.19:53548) 2024-05-09T02:19:53.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.686+0000 7f84f56a7700 1 -- 172.21.15.19:0/3485839116 learned_addr learned my addr 172.21.15.19:0/3485839116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:53.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.686+0000 7f84f66a9700 1 --2- 172.21.15.19:0/3485839116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f0108760 0x7f84f010cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.686+0000 7f84f56a7700 1 -- 172.21.15.19:0/3485839116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f0108760 msgr2=0x7f84f010cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:53.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.686+0000 7f84f56a7700 1 --2- 172.21.15.19:0/3485839116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f0108760 0x7f84f010cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.686+0000 7f84f56a7700 1 -- 172.21.15.19:0/3485839116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84f0103ca0 msgr2=0x7f84f010c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:53.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.686+0000 7f84f56a7700 1 --2- 172.21.15.19:0/3485839116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84f0103ca0 0x7f84f010c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.686+0000 7f84f56a7700 1 -- 172.21.15.19:0/3485839116 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f01ad520 con 0x7f84f0107e10 2024-05-09T02:19:53.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.687+0000 7f84f66a9700 1 --2- 172.21.15.19:0/3485839116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f0108760 0x7f84f010cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:19:53.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.687+0000 7f84f56a7700 1 --2- 172.21.15.19:0/3485839116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84f0107e10 0x7f84f010c980 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f84e00095c0 tx=0x7f84e0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:53.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.687+0000 7f84e6ffd700 1 -- 172.21.15.19:0/3485839116 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0026070 con 0x7f84f0107e10 2024-05-09T02:19:53.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.688+0000 7f84e6ffd700 1 -- 172.21.15.19:0/3485839116 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84e0003c70 con 0x7f84f0107e10 2024-05-09T02:19:53.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.688+0000 7f84f810c700 1 -- 172.21.15.19:0/3485839116 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f01ad7b0 con 0x7f84f0107e10 2024-05-09T02:19:53.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.688+0000 7f84f810c700 1 -- 172.21.15.19:0/3485839116 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f01adc60 con 0x7f84f0107e10 2024-05-09T02:19:53.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.688+0000 7f84e6ffd700 1 -- 172.21.15.19:0/3485839116 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e000f9f0 con 0x7f84f0107e10 2024-05-09T02:19:53.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.691+0000 7f84e6ffd700 1 -- 172.21.15.19:0/3485839116 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f84e0029070 con 0x7f84f0107e10 2024-05-09T02:19:53.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.692+0000 7f84e6ffd700 1 --2- 172.21.15.19:0/3485839116 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f84dc0b2a70 0x7f84dc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.692+0000 7f84e6ffd700 1 -- 172.21.15.19:0/3485839116 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f84e00e2b50 con 0x7f84f0107e10 2024-05-09T02:19:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.692+0000 7f84f5ea8700 1 --2- 172.21.15.19:0/3485839116 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f84dc0b2a70 0x7f84dc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.693+0000 7f8506b22b80 1 Processor -- start 2024-05-09T02:19:53.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.693+0000 7f84f5ea8700 1 --2- 172.21.15.19:0/3485839116 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f84dc0b2a70 0x7f84dc0b4f80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009c60 tx=0x7f84ec009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:53.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.693+0000 7f8506b22b80 1 -- start start 2024-05-09T02:19:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.694+0000 7f8506b22b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791f09d00 0x55b792012240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.694+0000 7f8506b22b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b791e8f4a0 0x55b792012780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.694+0000 7f8506b22b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b79200c400 0x55b79200c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.694+0000 7f84f5ea8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791f09d00 0x55b792012240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.694+0000 7f84f5ea8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791f09d00 0x55b792012240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56028/0 (socket says 172.21.15.19:56028) 2024-05-09T02:19:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.694+0000 7f84f5ea8700 1 -- 172.21.15.19:0/1560380137 learned_addr learned my addr 172.21.15.19:0/1560380137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.694+0000 7f8506b22b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b791f2f0a0 con 0x55b791e8f4a0 2024-05-09T02:19:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.694+0000 7f8506b22b80 1 -- 172.21.15.19:0/1560380137 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b791ed49a0 con 0x55b791f09d00 2024-05-09T02:19:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.694+0000 7f8506b22b80 1 -- 172.21.15.19:0/1560380137 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b791f2b060 con 0x55b79200c400 2024-05-09T02:19:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.694+0000 7f84f56a7700 1 --2- 172.21.15.19:0/1560380137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b791e8f4a0 0x55b792012780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.694+0000 7f84f66a9700 1 --2- 172.21.15.19:0/1560380137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b79200c400 0x55b79200c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.694+0000 7f84f5ea8700 1 -- 172.21.15.19:0/1560380137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b79200c400 msgr2=0x55b79200c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:53.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.695+0000 7f84f5ea8700 1 --2- 172.21.15.19:0/1560380137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b79200c400 0x55b79200c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.695+0000 7f84f5ea8700 1 -- 172.21.15.19:0/1560380137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b791e8f4a0 msgr2=0x55b792012780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:53.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.695+0000 7f84f5ea8700 1 --2- 172.21.15.19:0/1560380137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b791e8f4a0 0x55b792012780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.695+0000 7f84f5ea8700 1 -- 172.21.15.19:0/1560380137 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b791ec5500 con 0x55b791f09d00 2024-05-09T02:19:53.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.695+0000 7f84f56a7700 1 --2- 172.21.15.19:0/1560380137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b791e8f4a0 0x55b792012780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:19:53.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.695+0000 7f84f5ea8700 1 --2- 172.21.15.19:0/1560380137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791f09d00 0x55b792012240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00e4e0 tx=0x7f84ec014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d556ffb9302f11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:53.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.695+0000 7f84dbfff700 1 -- 172.21.15.19:0/1560380137 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec00ee00 con 0x55b791f09d00 2024-05-09T02:19:53.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.695+0000 7f84dbfff700 1 -- 172.21.15.19:0/1560380137 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84ec01c8c0 con 0x55b791f09d00 2024-05-09T02:19:53.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.695+0000 7f84dbfff700 1 -- 172.21.15.19:0/1560380137 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec01da20 con 0x55b791f09d00 2024-05-09T02:19:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.698+0000 7f8506b22b80 1 -- 172.21.15.19:0/1560380137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791f09d00 msgr2=0x55b792012240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.698+0000 7f8506b22b80 1 --2- 172.21.15.19:0/1560380137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791f09d00 0x55b792012240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00e4e0 tx=0x7f84ec014000 comp rx=0 tx=0).stop 2024-05-09T02:19:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.698+0000 7f8506b22b80 1 -- 172.21.15.19:0/1560380137 shutdown_connections 2024-05-09T02:19:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.698+0000 7f8506b22b80 1 --2- 172.21.15.19:0/1560380137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b79200c400 0x55b79200c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.698+0000 7f8506b22b80 1 --2- 172.21.15.19:0/1560380137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791f09d00 0x55b792012240 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.698+0000 7f8506b22b80 1 --2- 172.21.15.19:0/1560380137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b791e8f4a0 0x55b792012780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.698+0000 7f8506b22b80 1 -- 172.21.15.19:0/1560380137 >> 172.21.15.19:0/1560380137 conn(0x55b791e92f70 msgr2=0x55b792007220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.698+0000 7f8506b22b80 1 -- 172.21.15.19:0/1560380137 shutdown_connections 2024-05-09T02:19:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.699+0000 7f8506b22b80 1 -- 172.21.15.19:0/1560380137 wait complete. 2024-05-09T02:19:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.699+0000 7f8506b22b80 1 -- start start 2024-05-09T02:19:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.699+0000 7f8506b22b80 1 Processor -- start 2024-05-09T02:19:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.700+0000 7f8506b22b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791e8f4a0 0x55b792016020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.700+0000 7f8506b22b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791f09d00 0x55b792016560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.700+0000 7f8506b22b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 0x55b792016aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.700+0000 7f8506b22b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b79200d160 con 0x55b79200c400 2024-05-09T02:19:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.700+0000 7f8506b22b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b791ec9b20 con 0x55b791e8f4a0 2024-05-09T02:19:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.700+0000 7f8506b22b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b792019020 con 0x55b791f09d00 2024-05-09T02:19:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.700+0000 7f8506b22b80 10 client.0 _ll_register_callbacks cb 0x55b791da06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:19:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.700+0000 7f84f66a9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 0x55b792016aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.700+0000 7f84f5ea8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791e8f4a0 0x55b792016020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.700+0000 7f84f66a9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 0x55b792016aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53564/0 (socket says 172.21.15.19:53564) 2024-05-09T02:19:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.700+0000 7f84f66a9700 1 -- 172.21.15.19:0/3550659011 learned_addr learned my addr 172.21.15.19:0/3550659011 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.700+0000 7f84f56a7700 1 --2- 172.21.15.19:0/3550659011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791f09d00 0x55b792016560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.701+0000 7f84f56a7700 1 -- 172.21.15.19:0/3550659011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791e8f4a0 msgr2=0x55b792016020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.701+0000 7f84f56a7700 1 --2- 172.21.15.19:0/3550659011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791e8f4a0 0x55b792016020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.701+0000 7f84f56a7700 1 -- 172.21.15.19:0/3550659011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 msgr2=0x55b792016aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.701+0000 7f84f56a7700 1 --2- 172.21.15.19:0/3550659011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 0x55b792016aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.701+0000 7f84f56a7700 1 -- 172.21.15.19:0/3550659011 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b791daa490 con 0x55b791f09d00 2024-05-09T02:19:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.701+0000 7f84f5ea8700 1 --2- 172.21.15.19:0/3550659011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791e8f4a0 0x55b792016020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:19:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.701+0000 7f84f56a7700 1 --2- 172.21.15.19:0/3550659011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791f09d00 0x55b792016560 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f84e00e0210 tx=0x7f84e00aa370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=226e874d83306b85 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:53.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.701+0000 7f84d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:19:53.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.702+0000 7f84d97fa700 1 -- 172.21.15.19:0/3550659011 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e00a9940 con 0x55b791f09d00 2024-05-09T02:19:53.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.702+0000 7f8506b22b80 1 -- 172.21.15.19:0/3550659011 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84ec01c8c0 con 0x55b791f09d00 2024-05-09T02:19:53.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.702+0000 7f84d97fa700 1 -- 172.21.15.19:0/3550659011 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f84e000fda0 con 0x55b791f09d00 2024-05-09T02:19:53.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.702+0000 7f84d97fa700 1 -- 172.21.15.19:0/3550659011 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f84e0003e20 con 0x55b791f09d00 2024-05-09T02:19:53.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.702+0000 7f8506b22b80 10 client.26107 fetch_fsmap learned FSMap version 53 2024-05-09T02:19:53.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.702+0000 7f8506b22b80 1 -- 172.21.15.19:0/3550659011 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55b792023820 con 0x55b791f09d00 2024-05-09T02:19:53.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.702+0000 7f84d97fa700 1 -- 172.21.15.19:0/3550659011 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84ec01c8c0 con 0x55b791f09d00 2024-05-09T02:19:53.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.703+0000 7f84d97fa700 1 -- 172.21.15.19:0/3550659011 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 53) ==== 8386+0+0 (secure 0 0 0) 0x7f84e00c2020 con 0x55b791f09d00 2024-05-09T02:19:53.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.703+0000 7f8506b22b80 10 client.26107 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T02:19:53.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.703+0000 7f8506b22b80 10 client.26107 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:19:53.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.703+0000 7f8506b22b80 20 client.26107 populate_metadata read hostname 'smithi019' 2024-05-09T02:19:53.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.703+0000 7f8506b22b80 1 --2- 172.21.15.19:0/3550659011 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b792025e10 0x55b792028280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.703+0000 7f8506b22b80 4 client.26107 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:19:53.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.703+0000 7f8506b22b80 1 -- 172.21.15.19:0/3550659011 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b791f3ca80 con 0x55b792025e10 2024-05-09T02:19:53.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.703+0000 7f84f5ea8700 1 --2- 172.21.15.19:0/3550659011 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b792025e10 0x55b792028280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.704+0000 7f84f5ea8700 1 --2- 172.21.15.19:0/3550659011 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b792025e10 0x55b792028280 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=deb11c1a80972413 server_cookie=8c4baf12c8796e4c in_seq=0 out_seq=0 2024-05-09T02:19:53.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.704+0000 7f84d97fa700 10 client.26107 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:19:53.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.705+0000 7f84d97fa700 1 -- 172.21.15.19:0/3550659011 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b791f3ca80 con 0x55b792025e10 2024-05-09T02:19:53.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.705+0000 7f84d97fa700 10 client.26107 handle_command_reply: tid=0 2024-05-09T02:19:53.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.705+0000 7f8506b22b80 1 client.26107 shutdown 2024-05-09T02:19:53.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.712+0000 7f8506b22b80 1 -- 172.21.15.19:0/3550659011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791f09d00 msgr2=0x55b792016560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:53.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.712+0000 7f8506b22b80 1 --2- 172.21.15.19:0/3550659011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791f09d00 0x55b792016560 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f84e00e0210 tx=0x7f84e00aa370 comp rx=0 tx=0).stop 2024-05-09T02:19:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.717+0000 7f8506b22b80 1 -- 172.21.15.19:0/3550659011 shutdown_connections 2024-05-09T02:19:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.717+0000 7f8506b22b80 1 --2- 172.21.15.19:0/3550659011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 0x55b792016aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.717+0000 7f8506b22b80 1 --2- 172.21.15.19:0/3550659011 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b792025e10 0x55b792028280 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.717+0000 7f8506b22b80 1 --2- 172.21.15.19:0/3550659011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791e8f4a0 0x55b792016020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.717+0000 7f8506b22b80 1 --2- 172.21.15.19:0/3550659011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791f09d00 0x55b792016560 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.717+0000 7f8506b22b80 1 -- 172.21.15.19:0/3550659011 >> 172.21.15.19:0/3550659011 conn(0x55b791e92f70 msgr2=0x55b792000ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.718+0000 7f84d97fa700 0 client.26107 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:19:53.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.725+0000 7f8506b22b80 1 -- 172.21.15.19:0/3550659011 shutdown_connections 2024-05-09T02:19:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.728+0000 7f8506b22b80 1 -- 172.21.15.19:0/3550659011 wait complete. 2024-05-09T02:19:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.729+0000 7f8506b22b80 20 client.26107 trim_cache size 0 max 16384 2024-05-09T02:19:53.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.762+0000 7f8506b22b80 1 Processor -- start 2024-05-09T02:19:53.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.762+0000 7f8506b22b80 1 -- start start 2024-05-09T02:19:53.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.763+0000 7f8506b22b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791e8f4a0 0x55b792011820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.763+0000 7f8506b22b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791f09d00 0x55b79202da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.763+0000 7f8506b22b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 0x55b79202dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.763+0000 7f84f5ea8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791e8f4a0 0x55b792011820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.763+0000 7f84f5ea8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791e8f4a0 0x55b792011820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56046/0 (socket says 172.21.15.19:56046) 2024-05-09T02:19:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.763+0000 7f84f5ea8700 1 -- 172.21.15.19:0/1550731818 learned_addr learned my addr 172.21.15.19:0/1550731818 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.763+0000 7f8506b22b80 1 -- 172.21.15.19:0/1550731818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84d400d630 con 0x55b79200c400 2024-05-09T02:19:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.763+0000 7f8506b22b80 1 -- 172.21.15.19:0/1550731818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84d400f2f0 con 0x55b791e8f4a0 2024-05-09T02:19:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.763+0000 7f8506b22b80 1 -- 172.21.15.19:0/1550731818 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84d400dd30 con 0x55b791f09d00 2024-05-09T02:19:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.763+0000 7f84f66a9700 1 --2- 172.21.15.19:0/1550731818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 0x55b79202dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.764+0000 7f84f5ea8700 1 -- 172.21.15.19:0/1550731818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791f09d00 msgr2=0x55b79202da80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:19:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.764+0000 7f84f5ea8700 1 --2- 172.21.15.19:0/1550731818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791f09d00 0x55b79202da80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.764+0000 7f84f5ea8700 1 -- 172.21.15.19:0/1550731818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 msgr2=0x55b79202dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.764+0000 7f84f5ea8700 1 --2- 172.21.15.19:0/1550731818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 0x55b79202dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.764+0000 7f84f5ea8700 1 -- 172.21.15.19:0/1550731818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b792007f90 con 0x55b791e8f4a0 2024-05-09T02:19:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.765+0000 7f84f5ea8700 1 --2- 172.21.15.19:0/1550731818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791e8f4a0 0x55b792011820 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00e4b0 tx=0x7f84ec009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7d076ab8eff7def server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:53.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.765+0000 7f84db7fe700 1 -- 172.21.15.19:0/1550731818 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec00ec90 con 0x55b791e8f4a0 2024-05-09T02:19:53.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.765+0000 7f84db7fe700 1 -- 172.21.15.19:0/1550731818 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84ec00ee30 con 0x55b791e8f4a0 2024-05-09T02:19:53.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.767+0000 7f8506b22b80 1 -- 172.21.15.19:0/1550731818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791e8f4a0 msgr2=0x55b792011820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.767+0000 7f8506b22b80 1 --2- 172.21.15.19:0/1550731818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791e8f4a0 0x55b792011820 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00e4b0 tx=0x7f84ec009da0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.769+0000 7f8506b22b80 1 -- 172.21.15.19:0/1550731818 shutdown_connections 2024-05-09T02:19:53.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.769+0000 7f8506b22b80 1 --2- 172.21.15.19:0/1550731818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 0x55b79202dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.769+0000 7f8506b22b80 1 --2- 172.21.15.19:0/1550731818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791e8f4a0 0x55b792011820 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.769+0000 7f8506b22b80 1 --2- 172.21.15.19:0/1550731818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791f09d00 0x55b79202da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.769+0000 7f8506b22b80 1 -- 172.21.15.19:0/1550731818 >> 172.21.15.19:0/1550731818 conn(0x55b791e92f70 msgr2=0x55b792011110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:53.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.771+0000 7f8506b22b80 1 -- 172.21.15.19:0/1550731818 shutdown_connections 2024-05-09T02:19:53.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.772+0000 7f8506b22b80 1 -- 172.21.15.19:0/1550731818 wait complete. 2024-05-09T02:19:53.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.773+0000 7f8506b22b80 1 -- start start 2024-05-09T02:19:53.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.773+0000 7f8506b22b80 1 Processor -- start 2024-05-09T02:19:53.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.774+0000 7f8506b22b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791e8f4a0 0x55b79201e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.774+0000 7f8506b22b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791f09d00 0x55b79201e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.774+0000 7f8506b22b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 0x55b79202ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.774+0000 7f8506b22b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84d400eb90 con 0x55b79200c400 2024-05-09T02:19:53.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.774+0000 7f8506b22b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84d400e400 con 0x55b791f09d00 2024-05-09T02:19:53.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.774+0000 7f8506b22b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84d400b5f0 con 0x55b791e8f4a0 2024-05-09T02:19:53.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.774+0000 7f8506b22b80 10 client.0 _ll_register_callbacks cb 0x55b791da06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:19:53.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.774+0000 7f84f5ea8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791e8f4a0 0x55b79201e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.774+0000 7f84f5ea8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791e8f4a0 0x55b79201e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:45042/0 (socket says 172.21.15.19:45042) 2024-05-09T02:19:53.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.774+0000 7f84f5ea8700 1 -- 172.21.15.19:0/3101713064 learned_addr learned my addr 172.21.15.19:0/3101713064 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:53.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.775+0000 7f84f56a7700 1 --2- 172.21.15.19:0/3101713064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791f09d00 0x55b79201e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.775+0000 7f84f66a9700 1 --2- 172.21.15.19:0/3101713064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 0x55b79202ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.775+0000 7f84f5ea8700 1 -- 172.21.15.19:0/3101713064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791f09d00 msgr2=0x55b79201e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:53.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.775+0000 7f84f5ea8700 1 --2- 172.21.15.19:0/3101713064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791f09d00 0x55b79201e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.775+0000 7f84f5ea8700 1 -- 172.21.15.19:0/3101713064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 msgr2=0x55b79202ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:53.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.775+0000 7f84f5ea8700 1 --2- 172.21.15.19:0/3101713064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 0x55b79202ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.775+0000 7f84f5ea8700 1 -- 172.21.15.19:0/3101713064 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b792012af0 con 0x55b791e8f4a0 2024-05-09T02:19:53.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.775+0000 7f84f66a9700 1 --2- 172.21.15.19:0/3101713064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 0x55b79202ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:19:53.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.776+0000 7f84f5ea8700 1 --2- 172.21.15.19:0/3101713064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791e8f4a0 0x55b79201e1e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009ec0 tx=0x7f84ec011be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5532f79db5aa2fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:53.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.776+0000 7f84d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:19:53.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.776+0000 7f84d97fa700 1 -- 172.21.15.19:0/3101713064 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec005250 con 0x55b791e8f4a0 2024-05-09T02:19:53.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.776+0000 7f8506b22b80 1 -- 172.21.15.19:0/3101713064 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84ec00ee30 con 0x55b791e8f4a0 2024-05-09T02:19:53.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.776+0000 7f84d97fa700 1 -- 172.21.15.19:0/3101713064 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f84ec0053f0 con 0x55b791e8f4a0 2024-05-09T02:19:53.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.776+0000 7f84d97fa700 1 -- 172.21.15.19:0/3101713064 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f84ec00ec80 con 0x55b791e8f4a0 2024-05-09T02:19:53.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.777+0000 7f8506b22b80 10 client.26110 fetch_fsmap learned FSMap version 53 2024-05-09T02:19:53.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.777+0000 7f8506b22b80 1 -- 172.21.15.19:0/3101713064 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55b7920039d0 con 0x55b791e8f4a0 2024-05-09T02:19:53.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.777+0000 7f84d97fa700 1 -- 172.21.15.19:0/3101713064 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84ec00ee30 con 0x55b791e8f4a0 2024-05-09T02:19:53.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.778+0000 7f84d97fa700 1 -- 172.21.15.19:0/3101713064 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 53) ==== 8386+0+0 (secure 0 0 0) 0x7f84ec0342b0 con 0x55b791e8f4a0 2024-05-09T02:19:53.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.778+0000 7f8506b22b80 10 client.26110 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T02:19:53.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.778+0000 7f8506b22b80 10 client.26110 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:19:53.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.778+0000 7f8506b22b80 20 client.26110 populate_metadata read hostname 'smithi019' 2024-05-09T02:19:53.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.778+0000 7f8506b22b80 1 --2- 172.21.15.19:0/3101713064 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b79208fad0 0x55b792091ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:53.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.778+0000 7f8506b22b80 4 client.26110 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:19:53.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.778+0000 7f8506b22b80 1 -- 172.21.15.19:0/3101713064 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b791f36b20 con 0x55b79208fad0 2024-05-09T02:19:53.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.778+0000 7f84f56a7700 1 --2- 172.21.15.19:0/3101713064 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b79208fad0 0x55b792091ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:53.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.779+0000 7f84d97fa700 10 client.26110 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:19:53.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.779+0000 7f84f56a7700 1 --2- 172.21.15.19:0/3101713064 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b79208fad0 0x55b792091ef0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c0461b7b7467a1d server_cookie=1886469e4cf3e020 in_seq=0 out_seq=0 2024-05-09T02:19:53.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.780+0000 7f84d97fa700 1 -- 172.21.15.19:0/3101713064 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b791f36b20 con 0x55b79208fad0 2024-05-09T02:19:53.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.780+0000 7f84d97fa700 10 client.26110 handle_command_reply: tid=0 2024-05-09T02:19:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.780+0000 7f8506b22b80 1 client.26110 shutdown 2024-05-09T02:19:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.780+0000 7f8506b22b80 1 -- 172.21.15.19:0/3101713064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791e8f4a0 msgr2=0x55b79201e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.780+0000 7f8506b22b80 1 --2- 172.21.15.19:0/3101713064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791e8f4a0 0x55b79201e1e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009ec0 tx=0x7f84ec011be0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.780+0000 7f8506b22b80 1 -- 172.21.15.19:0/3101713064 shutdown_connections 2024-05-09T02:19:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.780+0000 7f8506b22b80 1 --2- 172.21.15.19:0/3101713064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b79200c400 0x55b79202ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.780+0000 7f8506b22b80 1 --2- 172.21.15.19:0/3101713064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b791e8f4a0 0x55b79201e1e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.780+0000 7f8506b22b80 1 --2- 172.21.15.19:0/3101713064 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55b79208fad0 0x55b792091ef0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.780+0000 7f8506b22b80 1 --2- 172.21.15.19:0/3101713064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b791f09d00 0x55b79201e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.780+0000 7f8506b22b80 1 -- 172.21.15.19:0/3101713064 >> 172.21.15.19:0/3101713064 conn(0x55b791e92f70 msgr2=0x55b792000ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.780+0000 7f84d97fa700 0 client.26110 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:19:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.781+0000 7f8506b22b80 1 -- 172.21.15.19:0/3101713064 shutdown_connections 2024-05-09T02:19:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.781+0000 7f8506b22b80 1 -- 172.21.15.19:0/3101713064 wait complete. 2024-05-09T02:19:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.781+0000 7f8506b22b80 20 client.26110 trim_cache size 0 max 16384 2024-05-09T02:19:53.785 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:19:53.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.784+0000 7f84f810c700 1 -- 172.21.15.19:0/3485839116 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f84dc0b2a70 msgr2=0x7f84dc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:53.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.784+0000 7f84f810c700 1 --2- 172.21.15.19:0/3485839116 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f84dc0b2a70 0x7f84dc0b4f80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009c60 tx=0x7f84ec009260 comp rx=0 tx=0).stop 2024-05-09T02:19:53.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.784+0000 7f84f810c700 1 -- 172.21.15.19:0/3485839116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84f0107e10 msgr2=0x7f84f010c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:53.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.784+0000 7f84f810c700 1 --2- 172.21.15.19:0/3485839116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84f0107e10 0x7f84f010c980 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f84e00095c0 tx=0x7f84e0004270 comp rx=0 tx=0).stop 2024-05-09T02:19:53.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.785+0000 7f84f810c700 1 -- 172.21.15.19:0/3485839116 shutdown_connections 2024-05-09T02:19:53.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.785+0000 7f84f810c700 1 --2- 172.21.15.19:0/3485839116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f0108760 0x7f84f010cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.785+0000 7f84f810c700 1 --2- 172.21.15.19:0/3485839116 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f84dc0b2a70 0x7f84dc0b4f80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.785+0000 7f84f810c700 1 --2- 172.21.15.19:0/3485839116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84f0103ca0 0x7f84f010c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.785+0000 7f84f810c700 1 --2- 172.21.15.19:0/3485839116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84f0107e10 0x7f84f010c980 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:53.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.785+0000 7f84f810c700 1 -- 172.21.15.19:0/3485839116 >> 172.21.15.19:0/3485839116 conn(0x7f84f00fe6f0 msgr2=0x7f84f0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:53.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.785+0000 7f84f810c700 1 -- 172.21.15.19:0/3485839116 shutdown_connections 2024-05-09T02:19:53.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:53.785+0000 7f84f810c700 1 -- 172.21.15.19:0/3485839116 wait complete. 2024-05-09T02:19:54.508 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T02:19:54.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:54 smithi019 ceph-mon[30151]: pgmap v265: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.0 MiB/s wr, 563 op/s 2024-05-09T02:19:54.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:54 smithi059 ceph-mon[34781]: pgmap v265: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.0 MiB/s wr, 563 op/s 2024-05-09T02:19:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:54 smithi196 ceph-mon[34718]: pgmap v265: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.0 MiB/s wr, 563 op/s 2024-05-09T02:19:54.932 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:19:56.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:56 smithi059 ceph-mon[34781]: pgmap v266: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 856 op/s 2024-05-09T02:19:56.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:56 smithi019 ceph-mon[30151]: pgmap v266: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 856 op/s 2024-05-09T02:19:56.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:56 smithi196 ceph-mon[34718]: pgmap v266: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 856 op/s 2024-05-09T02:19:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 495.0 s, 1768.2 tps, lat 56.634 ms stddev 64.759 2024-05-09T02:19:58.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.031+0000 7f7c30e44700 1 -- 172.21.15.19:0/989726301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c107df0 msgr2=0x7f7c2c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:58.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.031+0000 7f7c30e44700 1 --2- 172.21.15.19:0/989726301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c107df0 0x7f7c2c108200 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7c200099c0 tx=0x7f7c20009cd0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.031+0000 7f7c30e44700 1 -- 172.21.15.19:0/989726301 shutdown_connections 2024-05-09T02:19:58.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.031+0000 7f7c30e44700 1 --2- 172.21.15.19:0/989726301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c2c108740 0x7f7c2c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.031+0000 7f7c30e44700 1 --2- 172.21.15.19:0/989726301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c2c1034c0 0x7f7c2c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.031+0000 7f7c30e44700 1 --2- 172.21.15.19:0/989726301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c107df0 0x7f7c2c108200 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.031+0000 7f7c30e44700 1 -- 172.21.15.19:0/989726301 >> 172.21.15.19:0/989726301 conn(0x7f7c2c0fedc0 msgr2=0x7f7c2c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:58.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.031+0000 7f7c30e44700 1 -- 172.21.15.19:0/989726301 shutdown_connections 2024-05-09T02:19:58.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.031+0000 7f7c30e44700 1 -- 172.21.15.19:0/989726301 wait complete. 2024-05-09T02:19:58.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.032+0000 7f7c30e44700 1 Processor -- start 2024-05-09T02:19:58.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.033+0000 7f7c30e44700 1 -- start start 2024-05-09T02:19:58.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.033+0000 7f7c30e44700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c2c1034c0 0x7f7c2c19bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.033+0000 7f7c30e44700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c107df0 0x7f7c2c19c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.033+0000 7f7c30e44700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c2c108740 0x7f7c2c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.033+0000 7f7c30e44700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c1133c0 con 0x7f7c2c107df0 2024-05-09T02:19:58.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.033+0000 7f7c30e44700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c113240 con 0x7f7c2c108740 2024-05-09T02:19:58.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.033+0000 7f7c30e44700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c113540 con 0x7f7c2c1034c0 2024-05-09T02:19:58.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.033+0000 7f7c2a59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c2c1034c0 0x7f7c2c19bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.033+0000 7f7c29d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c107df0 0x7f7c2c19c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.033+0000 7f7c2a59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c2c1034c0 0x7f7c2c19bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:37252/0 (socket says 172.21.15.19:37252) 2024-05-09T02:19:58.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.033+0000 7f7c2a59c700 1 -- 172.21.15.19:0/2235085511 learned_addr learned my addr 172.21.15.19:0/2235085511 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:58.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.033+0000 7f7c2ad9d700 1 --2- 172.21.15.19:0/2235085511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c2c108740 0x7f7c2c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.034+0000 7f7c2a59c700 1 -- 172.21.15.19:0/2235085511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c2c108740 msgr2=0x7f7c2c19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:58.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.034+0000 7f7c2a59c700 1 --2- 172.21.15.19:0/2235085511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c2c108740 0x7f7c2c19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.034+0000 7f7c2a59c700 1 -- 172.21.15.19:0/2235085511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c107df0 msgr2=0x7f7c2c19c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:58.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.034+0000 7f7c2a59c700 1 --2- 172.21.15.19:0/2235085511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c107df0 0x7f7c2c19c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.034+0000 7f7c2a59c700 1 -- 172.21.15.19:0/2235085511 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2c1a6fc0 con 0x7f7c2c1034c0 2024-05-09T02:19:58.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.034+0000 7f7c29d9b700 1 --2- 172.21.15.19:0/2235085511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c107df0 0x7f7c2c19c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:19:58.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.034+0000 7f7c2ad9d700 1 --2- 172.21.15.19:0/2235085511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c2c108740 0x7f7c2c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:19:58.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.034+0000 7f7c2a59c700 1 --2- 172.21.15.19:0/2235085511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c2c1034c0 0x7f7c2c19bd00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00e9c0 tx=0x7f7c1c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:58.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.034+0000 7f7c1b7fe700 1 -- 172.21.15.19:0/2235085511 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c00cbd0 con 0x7f7c2c1034c0 2024-05-09T02:19:58.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.034+0000 7f7c1b7fe700 1 -- 172.21.15.19:0/2235085511 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c1c00ee60 con 0x7f7c2c1034c0 2024-05-09T02:19:58.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.034+0000 7f7c30e44700 1 -- 172.21.15.19:0/2235085511 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2c1a72b0 con 0x7f7c2c1034c0 2024-05-09T02:19:58.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.034+0000 7f7c1b7fe700 1 -- 172.21.15.19:0/2235085511 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c0055c0 con 0x7f7c2c1034c0 2024-05-09T02:19:58.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.034+0000 7f7c30e44700 1 -- 172.21.15.19:0/2235085511 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c2c1a77c0 con 0x7f7c2c1034c0 2024-05-09T02:19:58.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.038+0000 7f7c1b7fe700 1 -- 172.21.15.19:0/2235085511 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7c1c004830 con 0x7f7c2c1034c0 2024-05-09T02:19:58.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.038+0000 7f7c3f85ab80 1 Processor -- start 2024-05-09T02:19:58.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.038+0000 7f7c3f85ab80 1 -- start start 2024-05-09T02:19:58.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.039+0000 7f7c3f85ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a1880a160 0x564a1889d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.039+0000 7f7c3f85ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a187204a0 0x564a1889dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.039+0000 7f7c3f85ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 0x564a1889e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.039+0000 7f7c2ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a1880a160 0x564a1889d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.039+0000 7f7c29d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 0x564a1889e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.039+0000 7f7c3f85ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564a187bfff0 con 0x564a187204a0 2024-05-09T02:19:58.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.039+0000 7f7c3f85ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564a1875e7f0 con 0x564a1880a160 2024-05-09T02:19:58.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.039+0000 7f7c2ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a1880a160 0x564a1889d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:49286/0 (socket says 172.21.15.19:49286) 2024-05-09T02:19:58.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.039+0000 7f7c3f85ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564a187bbfb0 con 0x564a188a2620 2024-05-09T02:19:58.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.039+0000 7f7c2ad9d700 1 -- 172.21.15.19:0/1097255965 learned_addr learned my addr 172.21.15.19:0/1097255965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:58.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.040+0000 7f7c2a59c700 1 --2- 172.21.15.19:0/1097255965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a187204a0 0x564a1889dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.040+0000 7f7c29d9b700 1 -- 172.21.15.19:0/1097255965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a1880a160 msgr2=0x564a1889d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:58.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.040+0000 7f7c29d9b700 1 --2- 172.21.15.19:0/1097255965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a1880a160 0x564a1889d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.040+0000 7f7c29d9b700 1 -- 172.21.15.19:0/1097255965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a187204a0 msgr2=0x564a1889dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:58.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.040+0000 7f7c29d9b700 1 --2- 172.21.15.19:0/1097255965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a187204a0 0x564a1889dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.040+0000 7f7c29d9b700 1 -- 172.21.15.19:0/1097255965 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a18756370 con 0x564a188a2620 2024-05-09T02:19:58.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.040+0000 7f7c29d9b700 1 --2- 172.21.15.19:0/1097255965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 0x564a1889e120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009990 tx=0x7f7c20005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9523746740d300f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:58.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.041+0000 7f7c18ff9700 1 -- 172.21.15.19:0/1097255965 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20027070 con 0x564a188a2620 2024-05-09T02:19:58.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.041+0000 7f7c18ff9700 1 -- 172.21.15.19:0/1097255965 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c200095b0 con 0x564a188a2620 2024-05-09T02:19:58.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.041+0000 7f7c18ff9700 1 -- 172.21.15.19:0/1097255965 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20003d70 con 0x564a188a2620 2024-05-09T02:19:58.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.042+0000 7f7c1b7fe700 1 --2- 172.21.15.19:0/2235085511 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7c140b2af0 0x7f7c140b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.042+0000 7f7c1b7fe700 1 -- 172.21.15.19:0/2235085511 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f7c1c0e28c0 con 0x7f7c2c1034c0 2024-05-09T02:19:58.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.042+0000 7f7c2ad9d700 1 --2- 172.21.15.19:0/2235085511 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7c140b2af0 0x7f7c140b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.042+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/1097255965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 msgr2=0x564a1889e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:58.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.042+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/1097255965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 0x564a1889e120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009990 tx=0x7f7c20005fb0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.042+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/1097255965 shutdown_connections 2024-05-09T02:19:58.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.042+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/1097255965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 0x564a1889e120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.042+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/1097255965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a1880a160 0x564a1889d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.042+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/1097255965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a187204a0 0x564a1889dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.042+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/1097255965 >> 172.21.15.19:0/1097255965 conn(0x564a187806a0 msgr2=0x564a188985a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:58.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.043+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/1097255965 shutdown_connections 2024-05-09T02:19:58.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.043+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/1097255965 wait complete. 2024-05-09T02:19:58.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.043+0000 7f7c2ad9d700 1 --2- 172.21.15.19:0/2235085511 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7c140b2af0 0x7f7c140b5000 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x564a1889a090 tx=0x7f7c24007950 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:58.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.043+0000 7f7c3f85ab80 1 -- start start 2024-05-09T02:19:58.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.043+0000 7f7c3f85ab80 1 Processor -- start 2024-05-09T02:19:58.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.043+0000 7f7c3f85ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a187204a0 0x564a188af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.043+0000 7f7c3f85ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1880a160 0x564a188af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.043+0000 7f7c3f85ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a188a2620 0x564a188afe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.043+0000 7f7c3f85ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564a1863bba0 con 0x564a188a2620 2024-05-09T02:19:58.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.043+0000 7f7c3f85ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564a1877a150 con 0x564a187204a0 2024-05-09T02:19:58.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.043+0000 7f7c3f85ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564a188b2820 con 0x564a1880a160 2024-05-09T02:19:58.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c3f85ab80 10 client.0 _ll_register_callbacks cb 0x564a186316c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:19:58.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c29d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1880a160 0x564a188af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c2a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a187204a0 0x564a188af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c29d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1880a160 0x564a188af930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:37272/0 (socket says 172.21.15.19:37272) 2024-05-09T02:19:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c29d9b700 1 -- 172.21.15.19:0/1449572613 learned_addr learned my addr 172.21.15.19:0/1449572613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c2ad9d700 1 --2- 172.21.15.19:0/1449572613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a188a2620 0x564a188afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c2a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a187204a0 0x564a188af3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:49296/0 (socket says 172.21.15.19:49296) 2024-05-09T02:19:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c29d9b700 1 -- 172.21.15.19:0/1449572613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a187204a0 msgr2=0x564a188af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c29d9b700 1 --2- 172.21.15.19:0/1449572613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a187204a0 0x564a188af3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c29d9b700 1 -- 172.21.15.19:0/1449572613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a188a2620 msgr2=0x564a188afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c29d9b700 1 --2- 172.21.15.19:0/1449572613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a188a2620 0x564a188afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c29d9b700 1 -- 172.21.15.19:0/1449572613 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a1863b490 con 0x564a1880a160 2024-05-09T02:19:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c29d9b700 1 --2- 172.21.15.19:0/1449572613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1880a160 0x564a188af930 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000f690 tx=0x7f7c2000f6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43ffcfa00f323099 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:19:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c11ffb700 1 -- 172.21.15.19:0/1449572613 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20027040 con 0x564a1880a160 2024-05-09T02:19:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/1449572613 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c200095b0 con 0x564a1880a160 2024-05-09T02:19:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.044+0000 7f7c11ffb700 1 -- 172.21.15.19:0/1449572613 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f7c20003bb0 con 0x564a1880a160 2024-05-09T02:19:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.045+0000 7f7c11ffb700 1 -- 172.21.15.19:0/1449572613 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f7c20003db0 con 0x564a1880a160 2024-05-09T02:19:58.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.045+0000 7f7c3f85ab80 10 client.26122 fetch_fsmap learned FSMap version 53 2024-05-09T02:19:58.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.045+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/1449572613 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x564a188bd2c0 con 0x564a1880a160 2024-05-09T02:19:58.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.045+0000 7f7c11ffb700 1 -- 172.21.15.19:0/1449572613 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c200095b0 con 0x564a1880a160 2024-05-09T02:19:58.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.045+0000 7f7c11ffb700 1 -- 172.21.15.19:0/1449572613 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 53) ==== 8386+0+0 (secure 0 0 0) 0x7f7c20033030 con 0x564a1880a160 2024-05-09T02:19:58.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.046+0000 7f7c3f85ab80 10 client.26122 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T02:19:58.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.046+0000 7f7c3f85ab80 10 client.26122 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:19:58.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.046+0000 7f7c3f85ab80 20 client.26122 populate_metadata read hostname 'smithi019' 2024-05-09T02:19:58.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.046+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/1449572613 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x564a188bf790 0x564a188c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.046+0000 7f7c3f85ab80 4 client.26122 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:19:58.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.046+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/1449572613 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a187cd9d0 con 0x564a188bf790 2024-05-09T02:19:58.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.047+0000 7f7c2a59c700 1 --2- 172.21.15.19:0/1449572613 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x564a188bf790 0x564a188c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.048+0000 7f7c2a59c700 1 --2- 172.21.15.19:0/1449572613 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x564a188bf790 0x564a188c1bb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5eb67fe14432c36f server_cookie=d58e490d8a89442 in_seq=0 out_seq=0 2024-05-09T02:19:58.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.048+0000 7f7c11ffb700 10 client.26122 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:19:58.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.049+0000 7f7c11ffb700 1 -- 172.21.15.19:0/1449572613 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564a187cd9d0 con 0x564a188bf790 2024-05-09T02:19:58.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.049+0000 7f7c11ffb700 10 client.26122 handle_command_reply: tid=0 2024-05-09T02:19:58.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.049+0000 7f7c3f85ab80 1 client.26122 shutdown 2024-05-09T02:19:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.049+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/1449572613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1880a160 msgr2=0x564a188af930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.049+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/1449572613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1880a160 0x564a188af930 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000f690 tx=0x7f7c2000f6c0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.049+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/1449572613 shutdown_connections 2024-05-09T02:19:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.049+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/1449572613 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x564a188bf790 0x564a188c1bb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.049+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/1449572613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a188a2620 0x564a188afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.049+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/1449572613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a187204a0 0x564a188af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.049+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/1449572613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1880a160 0x564a188af930 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.049+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/1449572613 >> 172.21.15.19:0/1449572613 conn(0x564a187806a0 msgr2=0x564a18898220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.049+0000 7f7c11ffb700 0 client.26122 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:19:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.050+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/1449572613 shutdown_connections 2024-05-09T02:19:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.050+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/1449572613 wait complete. 2024-05-09T02:19:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.050+0000 7f7c3f85ab80 20 client.26122 trim_cache size 0 max 16384 2024-05-09T02:19:58.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.071+0000 7f7c3f85ab80 1 Processor -- start 2024-05-09T02:19:58.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.071+0000 7f7c3f85ab80 1 -- start start 2024-05-09T02:19:58.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.071+0000 7f7c3f85ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a187204a0 0x564a1889def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c3f85ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a1880a160 0x564a188b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c3f85ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 0x564a188af960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c2a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a187204a0 0x564a1889def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c3f85ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c1acd00 con 0x564a1880a160 2024-05-09T02:19:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c2a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a187204a0 0x564a1889def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:49308/0 (socket says 172.21.15.19:49308) 2024-05-09T02:19:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c2a59c700 1 -- 172.21.15.19:0/557800453 learned_addr learned my addr 172.21.15.19:0/557800453 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c29d9b700 1 --2- 172.21.15.19:0/557800453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a1880a160 0x564a188b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/557800453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c1ae9c0 con 0x564a187204a0 2024-05-09T02:19:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/557800453 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c1ad400 con 0x564a188a2620 2024-05-09T02:19:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c2ad9d700 1 --2- 172.21.15.19:0/557800453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 0x564a188af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c2a59c700 1 -- 172.21.15.19:0/557800453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 msgr2=0x564a188af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c2a59c700 1 --2- 172.21.15.19:0/557800453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 0x564a188af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c2a59c700 1 -- 172.21.15.19:0/557800453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a1880a160 msgr2=0x564a188b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c2a59c700 1 --2- 172.21.15.19:0/557800453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a1880a160 0x564a188b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c2a59c700 1 -- 172.21.15.19:0/557800453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a18891f60 con 0x564a187204a0 2024-05-09T02:19:58.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.072+0000 7f7c2a59c700 1 --2- 172.21.15.19:0/557800453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a187204a0 0x564a1889def0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00c5b0 tx=0x7f7c1c0cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd8da320056dfaff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:58.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.073+0000 7f7c13fff700 1 -- 172.21.15.19:0/557800453 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c0a98e0 con 0x564a187204a0 2024-05-09T02:19:58.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.073+0000 7f7c13fff700 1 -- 172.21.15.19:0/557800453 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c1c0b22b0 con 0x564a187204a0 2024-05-09T02:19:58.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.073+0000 7f7c13fff700 1 -- 172.21.15.19:0/557800453 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c0a52a0 con 0x564a187204a0 2024-05-09T02:19:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.074+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/557800453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a187204a0 msgr2=0x564a1889def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.074+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/557800453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a187204a0 0x564a1889def0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00c5b0 tx=0x7f7c1c0cc040 comp rx=0 tx=0).stop 2024-05-09T02:19:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.074+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/557800453 shutdown_connections 2024-05-09T02:19:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.074+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/557800453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 0x564a188af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.074+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/557800453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a187204a0 0x564a1889def0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.074+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/557800453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a1880a160 0x564a188b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.074+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/557800453 >> 172.21.15.19:0/557800453 conn(0x564a187806a0 msgr2=0x564a188b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.075+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/557800453 shutdown_connections 2024-05-09T02:19:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.075+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/557800453 wait complete. 2024-05-09T02:19:58.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.076+0000 7f7c3f85ab80 1 -- start start 2024-05-09T02:19:58.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.076+0000 7f7c3f85ab80 1 Processor -- start 2024-05-09T02:19:58.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.076+0000 7f7c3f85ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a187204a0 0x564a188c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.076+0000 7f7c3f85ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a1880a160 0x564a188c1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.076+0000 7f7c3f85ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 0x564a188c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.076+0000 7f7c3f85ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c1ae260 con 0x564a187204a0 2024-05-09T02:19:58.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.077+0000 7f7c3f85ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c1adad0 con 0x564a1880a160 2024-05-09T02:19:58.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.077+0000 7f7c3f85ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c1aacc0 con 0x564a188a2620 2024-05-09T02:19:58.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.077+0000 7f7c29d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a1880a160 0x564a188c1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.077+0000 7f7c2a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a187204a0 0x564a188c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.077+0000 7f7c3f85ab80 10 client.0 _ll_register_callbacks cb 0x564a186316c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:19:58.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.077+0000 7f7c29d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a1880a160 0x564a188c1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:49322/0 (socket says 172.21.15.19:49322) 2024-05-09T02:19:58.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.077+0000 7f7c2a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a187204a0 0x564a188c17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46286/0 (socket says 172.21.15.19:46286) 2024-05-09T02:19:58.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.077+0000 7f7c29d9b700 1 -- 172.21.15.19:0/3073319641 learned_addr learned my addr 172.21.15.19:0/3073319641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:19:58.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.077+0000 7f7c2ad9d700 1 --2- 172.21.15.19:0/3073319641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 0x564a188c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.077+0000 7f7c2a59c700 1 -- 172.21.15.19:0/3073319641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 msgr2=0x564a188c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:58.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.077+0000 7f7c2a59c700 1 --2- 172.21.15.19:0/3073319641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 0x564a188c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.077+0000 7f7c2a59c700 1 -- 172.21.15.19:0/3073319641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a1880a160 msgr2=0x564a188c1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:19:58.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.077+0000 7f7c2a59c700 1 --2- 172.21.15.19:0/3073319641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a1880a160 0x564a188c1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.077+0000 7f7c2a59c700 1 -- 172.21.15.19:0/3073319641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a18927ec0 con 0x564a187204a0 2024-05-09T02:19:58.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.078+0000 7f7c2a59c700 1 --2- 172.21.15.19:0/3073319641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a187204a0 0x564a188c17a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0a6d80 tx=0x7f7c1c0a5fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c4c60334360fed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:19:58.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.078+0000 7f7c11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:19:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.079+0000 7f7c11ffb700 1 -- 172.21.15.19:0/3073319641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c0a9280 con 0x564a187204a0 2024-05-09T02:19:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.079+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/3073319641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564a187c7a70 con 0x564a187204a0 2024-05-09T02:19:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.079+0000 7f7c11ffb700 1 -- 172.21.15.19:0/3073319641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f7c1c0a9d70 con 0x564a187204a0 2024-05-09T02:19:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.079+0000 7f7c11ffb700 1 -- 172.21.15.19:0/3073319641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f7c1c0a9460 con 0x564a187204a0 2024-05-09T02:19:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.079+0000 7f7c3f85ab80 10 client.16359 fetch_fsmap learned FSMap version 53 2024-05-09T02:19:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.079+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/3073319641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x564a18929150 con 0x564a187204a0 2024-05-09T02:19:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.079+0000 7f7c11ffb700 1 -- 172.21.15.19:0/3073319641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564a187c7a70 con 0x564a187204a0 2024-05-09T02:19:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.080+0000 7f7c11ffb700 1 -- 172.21.15.19:0/3073319641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 53) ==== 8386+0+0 (secure 0 0 0) 0x7f7c1c0b24e0 con 0x564a187204a0 2024-05-09T02:19:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.080+0000 7f7c3f85ab80 10 client.16359 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T02:19:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.080+0000 7f7c3f85ab80 10 client.16359 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:19:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.080+0000 7f7c3f85ab80 20 client.16359 populate_metadata read hostname 'smithi019' 2024-05-09T02:19:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.080+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/3073319641 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x564a18929920 0x564a1892bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:19:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.080+0000 7f7c3f85ab80 4 client.16359 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:19:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.080+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/3073319641 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a1863b970 con 0x564a18929920 2024-05-09T02:19:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.080+0000 7f7c29d9b700 1 --2- 172.21.15.19:0/3073319641 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x564a18929920 0x564a1892bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:19:58.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.080+0000 7f7c11ffb700 10 client.16359 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:19:58.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.081+0000 7f7c29d9b700 1 --2- 172.21.15.19:0/3073319641 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x564a18929920 0x564a1892bd40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=372b66b2a81a04a2 server_cookie=8803d3db4686a55d in_seq=0 out_seq=0 2024-05-09T02:19:58.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.081+0000 7f7c11ffb700 1 -- 172.21.15.19:0/3073319641 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a1863b970 con 0x564a18929920 2024-05-09T02:19:58.086 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:19:58.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.081+0000 7f7c11ffb700 10 client.16359 handle_command_reply: tid=0 2024-05-09T02:19:58.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.081+0000 7f7c3f85ab80 1 client.16359 shutdown 2024-05-09T02:19:58.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.082+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/3073319641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a187204a0 msgr2=0x564a188c17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:58.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.082+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/3073319641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a187204a0 0x564a188c17a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0a6d80 tx=0x7f7c1c0a5fd0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.082+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/3073319641 shutdown_connections 2024-05-09T02:19:58.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.082+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/3073319641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a188a2620 0x564a188c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.082+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/3073319641 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x564a18929920 0x564a1892bd40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.082+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/3073319641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a187204a0 0x564a188c17a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.082+0000 7f7c3f85ab80 1 --2- 172.21.15.19:0/3073319641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a1880a160 0x564a188c1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.082+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/3073319641 >> 172.21.15.19:0/3073319641 conn(0x564a187806a0 msgr2=0x564a18898310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.082+0000 7f7c11ffb700 0 client.16359 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:19:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.082+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/3073319641 shutdown_connections 2024-05-09T02:19:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.082+0000 7f7c3f85ab80 1 -- 172.21.15.19:0/3073319641 wait complete. 2024-05-09T02:19:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.083+0000 7f7c3f85ab80 20 client.16359 trim_cache size 0 max 16384 2024-05-09T02:19:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.084+0000 7f7c30e44700 1 -- 172.21.15.19:0/2235085511 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7c140b2af0 msgr2=0x7f7c140b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.084+0000 7f7c30e44700 1 --2- 172.21.15.19:0/2235085511 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7c140b2af0 0x7f7c140b5000 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x564a1889a090 tx=0x7f7c24007950 comp rx=0 tx=0).stop 2024-05-09T02:19:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.084+0000 7f7c30e44700 1 -- 172.21.15.19:0/2235085511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c2c1034c0 msgr2=0x7f7c2c19bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:19:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.084+0000 7f7c30e44700 1 --2- 172.21.15.19:0/2235085511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c2c1034c0 0x7f7c2c19bd00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00e9c0 tx=0x7f7c1c00ecd0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.084+0000 7f7c30e44700 1 -- 172.21.15.19:0/2235085511 shutdown_connections 2024-05-09T02:19:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.084+0000 7f7c30e44700 1 --2- 172.21.15.19:0/2235085511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c2c108740 0x7f7c2c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.084+0000 7f7c30e44700 1 --2- 172.21.15.19:0/2235085511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c2c1034c0 0x7f7c2c19bd00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.084+0000 7f7c30e44700 1 --2- 172.21.15.19:0/2235085511 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7c140b2af0 0x7f7c140b5000 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.084+0000 7f7c30e44700 1 --2- 172.21.15.19:0/2235085511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c107df0 0x7f7c2c19c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:19:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.084+0000 7f7c30e44700 1 -- 172.21.15.19:0/2235085511 >> 172.21.15.19:0/2235085511 conn(0x7f7c2c0fedc0 msgr2=0x7f7c2c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:19:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.085+0000 7f7c30e44700 1 -- 172.21.15.19:0/2235085511 shutdown_connections 2024-05-09T02:19:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:19:58.085+0000 7f7c30e44700 1 -- 172.21.15.19:0/2235085511 wait complete. 2024-05-09T02:19:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:19:58 smithi059 ceph-mon[34781]: pgmap v267: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.4 MiB/s wr, 602 op/s 2024-05-09T02:19:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:19:58 smithi196 ceph-mon[34718]: pgmap v267: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.4 MiB/s wr, 602 op/s 2024-05-09T02:19:58.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:19:58 smithi019 ceph-mon[30151]: pgmap v267: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.4 MiB/s wr, 602 op/s 2024-05-09T02:19:58.897 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T02:19:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:20:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:00 smithi059 ceph-mon[34781]: pgmap v268: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 11 MiB/s wr, 808 op/s 2024-05-09T02:20:00.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:00 smithi059 ceph-mon[34781]: overall HEALTH_OK 2024-05-09T02:20:00.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:00 smithi019 ceph-mon[30151]: pgmap v268: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 11 MiB/s wr, 808 op/s 2024-05-09T02:20:00.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:00 smithi019 ceph-mon[30151]: overall HEALTH_OK 2024-05-09T02:20:00.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:00 smithi196 ceph-mon[34718]: pgmap v268: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 11 MiB/s wr, 808 op/s 2024-05-09T02:20:00.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:00 smithi196 ceph-mon[34718]: overall HEALTH_OK 2024-05-09T02:20:01.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.194+0000 7f096401f700 1 -- 172.21.15.19:0/3203788781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c1089f0 msgr2=0x7f095c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:01.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.194+0000 7f096401f700 1 --2- 172.21.15.19:0/3203788781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c1089f0 0x7f095c108e00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f095800aaa0 tx=0x7f095800adb0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.195+0000 7f096401f700 1 -- 172.21.15.19:0/3203788781 shutdown_connections 2024-05-09T02:20:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.195+0000 7f096401f700 1 --2- 172.21.15.19:0/3203788781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095c103310 0x7f095c10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.195+0000 7f096401f700 1 --2- 172.21.15.19:0/3203788781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c1089f0 0x7f095c108e00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.195+0000 7f096401f700 1 --2- 172.21.15.19:0/3203788781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f095c1029c0 0x7f095c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.195+0000 7f096401f700 1 -- 172.21.15.19:0/3203788781 >> 172.21.15.19:0/3203788781 conn(0x7f095c0fe6f0 msgr2=0x7f095c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.195+0000 7f096401f700 1 -- 172.21.15.19:0/3203788781 shutdown_connections 2024-05-09T02:20:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.195+0000 7f096401f700 1 -- 172.21.15.19:0/3203788781 wait complete. 2024-05-09T02:20:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.196+0000 7f096401f700 1 Processor -- start 2024-05-09T02:20:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.196+0000 7f096401f700 1 -- start start 2024-05-09T02:20:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.196+0000 7f096401f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095c1029c0 0x7f095c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.196+0000 7f096401f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c103310 0x7f095c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.196+0000 7f096401f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f095c1089f0 0x7f095c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.196+0000 7f096401f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1122e0 con 0x7f095c103310 2024-05-09T02:20:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.196+0000 7f096401f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f095c112160 con 0x7f095c1089f0 2024-05-09T02:20:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.196+0000 7f096401f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f095c112460 con 0x7f095c1029c0 2024-05-09T02:20:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.196+0000 7f09615ba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c103310 0x7f095c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.196+0000 7f09615ba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c103310 0x7f095c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46302/0 (socket says 172.21.15.19:46302) 2024-05-09T02:20:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.196+0000 7f0961dbb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095c1029c0 0x7f095c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.196+0000 7f09615ba700 1 -- 172.21.15.19:0/1671296146 learned_addr learned my addr 172.21.15.19:0/1671296146 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.197+0000 7f09625bc700 1 --2- 172.21.15.19:0/1671296146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f095c1089f0 0x7f095c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.197+0000 7f09615ba700 1 -- 172.21.15.19:0/1671296146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095c1029c0 msgr2=0x7f095c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.197+0000 7f09615ba700 1 --2- 172.21.15.19:0/1671296146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095c1029c0 0x7f095c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.197+0000 7f09615ba700 1 -- 172.21.15.19:0/1671296146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f095c1089f0 msgr2=0x7f095c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.197+0000 7f09615ba700 1 --2- 172.21.15.19:0/1671296146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f095c1089f0 0x7f095c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.197+0000 7f09615ba700 1 -- 172.21.15.19:0/1671296146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c1990f0 con 0x7f095c103310 2024-05-09T02:20:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.197+0000 7f09615ba700 1 --2- 172.21.15.19:0/1671296146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c103310 0x7f095c19ed60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f094c00ea30 tx=0x7f094c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.197+0000 7f0952ffd700 1 -- 172.21.15.19:0/1671296146 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c00cbd0 con 0x7f095c103310 2024-05-09T02:20:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.197+0000 7f0952ffd700 1 -- 172.21.15.19:0/1671296146 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f094c0044d0 con 0x7f095c103310 2024-05-09T02:20:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.197+0000 7f096401f700 1 -- 172.21.15.19:0/1671296146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095c1993e0 con 0x7f095c103310 2024-05-09T02:20:01.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.197+0000 7f0952ffd700 1 -- 172.21.15.19:0/1671296146 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c003dc0 con 0x7f095c103310 2024-05-09T02:20:01.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.197+0000 7f096401f700 1 -- 172.21.15.19:0/1671296146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095c1a56b0 con 0x7f095c103310 2024-05-09T02:20:01.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.200+0000 7f0952ffd700 1 -- 172.21.15.19:0/1671296146 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f094c018430 con 0x7f095c103310 2024-05-09T02:20:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.201+0000 7f0952ffd700 1 --2- 172.21.15.19:0/1671296146 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f09480b29f0 0x7f09480b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.201+0000 7f0972a35b80 1 Processor -- start 2024-05-09T02:20:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.202+0000 7f0961dbb700 1 --2- 172.21.15.19:0/1671296146 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f09480b29f0 0x7f09480b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.202+0000 7f0952ffd700 1 -- 172.21.15.19:0/1671296146 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f094c0e2780 con 0x7f095c103310 2024-05-09T02:20:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.202+0000 7f0972a35b80 1 -- start start 2024-05-09T02:20:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.202+0000 7f0972a35b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb83408160 0x55cb834a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.202+0000 7f0972a35b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.202+0000 7f0972a35b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb8349b6a0 0x55cb8349bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.202+0000 7f0972a35b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cb833bdff0 con 0x55cb8331e4a0 2024-05-09T02:20:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.202+0000 7f0972a35b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cb8335c7f0 con 0x55cb8349b6a0 2024-05-09T02:20:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.202+0000 7f0972a35b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cb833b9fb0 con 0x55cb83408160 2024-05-09T02:20:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.202+0000 7f0961dbb700 1 --2- 172.21.15.19:0/1671296146 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f09480b29f0 0x7f09480b4f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f09580059d0 tx=0x7f0958005960 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.202+0000 7f0961dbb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb83408160 0x55cb834a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.202+0000 7f0961dbb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb83408160 0x55cb834a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:37302/0 (socket says 172.21.15.19:37302) 2024-05-09T02:20:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.202+0000 7f0961dbb700 1 -- 172.21.15.19:0/1309659898 learned_addr learned my addr 172.21.15.19:0/1309659898 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.202+0000 7f09625bc700 1 --2- 172.21.15.19:0/1309659898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb8349b6a0 0x55cb8349bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.202+0000 7f09615ba700 1 --2- 172.21.15.19:0/1309659898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.203+0000 7f09615ba700 1 -- 172.21.15.19:0/1309659898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb83408160 msgr2=0x55cb834a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.203+0000 7f09615ba700 1 --2- 172.21.15.19:0/1309659898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb83408160 0x55cb834a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.203+0000 7f09615ba700 1 -- 172.21.15.19:0/1309659898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb8349b6a0 msgr2=0x55cb8349bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.203+0000 7f09615ba700 1 --2- 172.21.15.19:0/1309659898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb8349b6a0 0x55cb8349bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.203+0000 7f09615ba700 1 -- 172.21.15.19:0/1309659898 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb83354370 con 0x55cb8331e4a0 2024-05-09T02:20:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.203+0000 7f0961dbb700 1 --2- 172.21.15.19:0/1309659898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb83408160 0x55cb834a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:20:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.203+0000 7f09615ba700 1 --2- 172.21.15.19:0/1309659898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834a1a30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f094c0dfea0 tx=0x7f094c0a9220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9c611240e45b13 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:01.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.203+0000 7f0947fff700 1 -- 172.21.15.19:0/1309659898 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c003680 con 0x55cb8331e4a0 2024-05-09T02:20:01.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.203+0000 7f0947fff700 1 -- 172.21.15.19:0/1309659898 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f094c0a9540 con 0x55cb8331e4a0 2024-05-09T02:20:01.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.203+0000 7f0947fff700 1 -- 172.21.15.19:0/1309659898 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c0cc110 con 0x55cb8331e4a0 2024-05-09T02:20:01.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.204+0000 7f0972a35b80 1 -- 172.21.15.19:0/1309659898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 msgr2=0x55cb834a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:01.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.204+0000 7f0972a35b80 1 --2- 172.21.15.19:0/1309659898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834a1a30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f094c0dfea0 tx=0x7f094c0a9220 comp rx=0 tx=0).stop 2024-05-09T02:20:01.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.204+0000 7f0972a35b80 1 -- 172.21.15.19:0/1309659898 shutdown_connections 2024-05-09T02:20:01.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.204+0000 7f0972a35b80 1 --2- 172.21.15.19:0/1309659898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb8349b6a0 0x55cb8349bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.204+0000 7f0972a35b80 1 --2- 172.21.15.19:0/1309659898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb83408160 0x55cb834a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.204+0000 7f0972a35b80 1 --2- 172.21.15.19:0/1309659898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834a1a30 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.204+0000 7f0972a35b80 1 -- 172.21.15.19:0/1309659898 >> 172.21.15.19:0/1309659898 conn(0x55cb8337e6a0 msgr2=0x55cb83496400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:01.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.205+0000 7f0972a35b80 1 -- 172.21.15.19:0/1309659898 shutdown_connections 2024-05-09T02:20:01.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.205+0000 7f0972a35b80 1 -- 172.21.15.19:0/1309659898 wait complete. 2024-05-09T02:20:01.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.205+0000 7f0972a35b80 1 -- start start 2024-05-09T02:20:01.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.205+0000 7f0972a35b80 1 Processor -- start 2024-05-09T02:20:01.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f0972a35b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb8331e4a0 0x55cb834a5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f0972a35b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb83408160 0x55cb834a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f0972a35b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8349b6a0 0x55cb834a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f0972a35b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cb83239ba0 con 0x55cb8349b6a0 2024-05-09T02:20:01.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f0972a35b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cb83378150 con 0x55cb8331e4a0 2024-05-09T02:20:01.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f0972a35b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cb834a81d0 con 0x55cb83408160 2024-05-09T02:20:01.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f0972a35b80 10 client.0 _ll_register_callbacks cb 0x55cb8322f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:20:01.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f09615ba700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb83408160 0x55cb834a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f09615ba700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb83408160 0x55cb834a57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:37308/0 (socket says 172.21.15.19:37308) 2024-05-09T02:20:01.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f09615ba700 1 -- 172.21.15.19:0/3614201478 learned_addr learned my addr 172.21.15.19:0/3614201478 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:01.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f0961dbb700 1 --2- 172.21.15.19:0/3614201478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb8331e4a0 0x55cb834a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f09625bc700 1 --2- 172.21.15.19:0/3614201478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8349b6a0 0x55cb834a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f09615ba700 1 -- 172.21.15.19:0/3614201478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb8331e4a0 msgr2=0x55cb834a5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:01.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f09615ba700 1 --2- 172.21.15.19:0/3614201478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb8331e4a0 0x55cb834a5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f09615ba700 1 -- 172.21.15.19:0/3614201478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8349b6a0 msgr2=0x55cb834a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:01.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f09615ba700 1 --2- 172.21.15.19:0/3614201478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8349b6a0 0x55cb834a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f09615ba700 1 -- 172.21.15.19:0/3614201478 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb83239490 con 0x55cb83408160 2024-05-09T02:20:01.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.206+0000 7f0961dbb700 1 --2- 172.21.15.19:0/3614201478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb8331e4a0 0x55cb834a5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:20:01.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.207+0000 7f09615ba700 1 --2- 172.21.15.19:0/3614201478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb83408160 0x55cb834a57c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f094c0e04a0 tx=0x7f094c0098e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a81f960ac2f15e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:01.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.207+0000 7f09457fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:20:01.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.207+0000 7f09457fa700 1 -- 172.21.15.19:0/3614201478 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c003680 con 0x55cb83408160 2024-05-09T02:20:01.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.207+0000 7f09457fa700 1 -- 172.21.15.19:0/3614201478 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f094c0a9ea0 con 0x55cb83408160 2024-05-09T02:20:01.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.207+0000 7f0972a35b80 1 -- 172.21.15.19:0/3614201478 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f094c0a9540 con 0x55cb83408160 2024-05-09T02:20:01.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.207+0000 7f09457fa700 1 -- 172.21.15.19:0/3614201478 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f094c0a9a30 con 0x55cb83408160 2024-05-09T02:20:01.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.207+0000 7f0972a35b80 10 client.26140 fetch_fsmap learned FSMap version 53 2024-05-09T02:20:01.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.207+0000 7f0972a35b80 1 -- 172.21.15.19:0/3614201478 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55cb834b2c60 con 0x55cb83408160 2024-05-09T02:20:01.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.207+0000 7f09457fa700 1 -- 172.21.15.19:0/3614201478 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f094c0a9540 con 0x55cb83408160 2024-05-09T02:20:01.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.208+0000 7f09457fa700 1 -- 172.21.15.19:0/3614201478 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 53) ==== 8386+0+0 (secure 0 0 0) 0x7f094c06f030 con 0x55cb83408160 2024-05-09T02:20:01.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.208+0000 7f0972a35b80 10 client.26140 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T02:20:01.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.208+0000 7f0972a35b80 10 client.26140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:20:01.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.208+0000 7f0972a35b80 20 client.26140 populate_metadata read hostname 'smithi019' 2024-05-09T02:20:01.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.208+0000 7f0972a35b80 1 --2- 172.21.15.19:0/3614201478 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb834b5130 0x55cb834b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.208+0000 7f0972a35b80 4 client.26140 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:20:01.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.208+0000 7f0972a35b80 1 -- 172.21.15.19:0/3614201478 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb833cb9d0 con 0x55cb834b5130 2024-05-09T02:20:01.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.208+0000 7f0961dbb700 1 --2- 172.21.15.19:0/3614201478 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb834b5130 0x55cb834b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.208+0000 7f09457fa700 10 client.26140 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:01.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.209+0000 7f0961dbb700 1 --2- 172.21.15.19:0/3614201478 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb834b5130 0x55cb834b7550 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf9bc8e21af341b server_cookie=43bdd12b36e8c383 in_seq=0 out_seq=0 2024-05-09T02:20:01.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.209+0000 7f09457fa700 1 -- 172.21.15.19:0/3614201478 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cb833cb9d0 con 0x55cb834b5130 2024-05-09T02:20:01.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.209+0000 7f09457fa700 10 client.26140 handle_command_reply: tid=0 2024-05-09T02:20:01.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.209+0000 7f0972a35b80 1 client.26140 shutdown 2024-05-09T02:20:01.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.209+0000 7f0972a35b80 1 -- 172.21.15.19:0/3614201478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb83408160 msgr2=0x55cb834a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:01.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.209+0000 7f0972a35b80 1 --2- 172.21.15.19:0/3614201478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb83408160 0x55cb834a57c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f094c0e04a0 tx=0x7f094c0098e0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.210+0000 7f0972a35b80 1 -- 172.21.15.19:0/3614201478 shutdown_connections 2024-05-09T02:20:01.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.210+0000 7f0972a35b80 1 --2- 172.21.15.19:0/3614201478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8349b6a0 0x55cb834a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.210+0000 7f0972a35b80 1 --2- 172.21.15.19:0/3614201478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb8331e4a0 0x55cb834a5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.210+0000 7f0972a35b80 1 --2- 172.21.15.19:0/3614201478 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb834b5130 0x55cb834b7550 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.210+0000 7f0972a35b80 1 --2- 172.21.15.19:0/3614201478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb83408160 0x55cb834a57c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.210+0000 7f0972a35b80 1 -- 172.21.15.19:0/3614201478 >> 172.21.15.19:0/3614201478 conn(0x55cb8337e6a0 msgr2=0x55cb834961c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:01.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.210+0000 7f09457fa700 0 client.26140 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:01.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.210+0000 7f0972a35b80 1 -- 172.21.15.19:0/3614201478 shutdown_connections 2024-05-09T02:20:01.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.210+0000 7f0972a35b80 1 -- 172.21.15.19:0/3614201478 wait complete. 2024-05-09T02:20:01.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.210+0000 7f0972a35b80 20 client.26140 trim_cache size 0 max 16384 2024-05-09T02:20:01.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.230+0000 7f0972a35b80 1 Processor -- start 2024-05-09T02:20:01.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.230+0000 7f0972a35b80 1 -- start start 2024-05-09T02:20:01.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.230+0000 7f0972a35b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.230+0000 7f0972a35b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb83408160 0x55cb834a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.230+0000 7f0972a35b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb8349b6a0 0x55cb834a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.231+0000 7f0972a35b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1ace10 con 0x55cb8331e4a0 2024-05-09T02:20:01.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.231+0000 7f0972a35b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1aead0 con 0x55cb83408160 2024-05-09T02:20:01.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.231+0000 7f0972a35b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1ad510 con 0x55cb8349b6a0 2024-05-09T02:20:01.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.231+0000 7f0961dbb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.231+0000 7f0961dbb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834a57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46332/0 (socket says 172.21.15.19:46332) 2024-05-09T02:20:01.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.231+0000 7f0961dbb700 1 -- 172.21.15.19:0/1056201277 learned_addr learned my addr 172.21.15.19:0/1056201277 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:01.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.231+0000 7f09615ba700 1 --2- 172.21.15.19:0/1056201277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb83408160 0x55cb834a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.231+0000 7f09625bc700 1 --2- 172.21.15.19:0/1056201277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb8349b6a0 0x55cb834a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.231+0000 7f0961dbb700 1 -- 172.21.15.19:0/1056201277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb8349b6a0 msgr2=0x55cb834a40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:01.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.231+0000 7f0961dbb700 1 --2- 172.21.15.19:0/1056201277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb8349b6a0 0x55cb834a40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.231+0000 7f0961dbb700 1 -- 172.21.15.19:0/1056201277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb83408160 msgr2=0x55cb834a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:01.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.231+0000 7f0961dbb700 1 --2- 172.21.15.19:0/1056201277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb83408160 0x55cb834a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.231+0000 7f0961dbb700 1 -- 172.21.15.19:0/1056201277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb834a7b20 con 0x55cb8331e4a0 2024-05-09T02:20:01.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.231+0000 7f09615ba700 1 --2- 172.21.15.19:0/1056201277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb83408160 0x55cb834a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:20:01.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.231+0000 7f0961dbb700 1 --2- 172.21.15.19:0/1056201277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834a57f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0958006010 tx=0x7f0958023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683a1020d70afe0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.232+0000 7f09477fe700 1 -- 172.21.15.19:0/1056201277 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958040070 con 0x55cb8331e4a0 2024-05-09T02:20:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.232+0000 7f09477fe700 1 -- 172.21.15.19:0/1056201277 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f095802dd10 con 0x55cb8331e4a0 2024-05-09T02:20:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.232+0000 7f09477fe700 1 -- 172.21.15.19:0/1056201277 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095803ba30 con 0x55cb8331e4a0 2024-05-09T02:20:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.232+0000 7f0972a35b80 1 -- 172.21.15.19:0/1056201277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 msgr2=0x55cb834a57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.232+0000 7f0972a35b80 1 --2- 172.21.15.19:0/1056201277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834a57f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0958006010 tx=0x7f0958023000 comp rx=0 tx=0).stop 2024-05-09T02:20:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.233+0000 7f0972a35b80 1 -- 172.21.15.19:0/1056201277 shutdown_connections 2024-05-09T02:20:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.233+0000 7f0972a35b80 1 --2- 172.21.15.19:0/1056201277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb8349b6a0 0x55cb834a40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.233+0000 7f0972a35b80 1 --2- 172.21.15.19:0/1056201277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834a57f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.233+0000 7f0972a35b80 1 --2- 172.21.15.19:0/1056201277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb83408160 0x55cb834a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.233+0000 7f0972a35b80 1 -- 172.21.15.19:0/1056201277 >> 172.21.15.19:0/1056201277 conn(0x55cb8337e6a0 msgr2=0x55cb834a0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.233+0000 7f0972a35b80 1 -- 172.21.15.19:0/1056201277 shutdown_connections 2024-05-09T02:20:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.233+0000 7f0972a35b80 1 -- 172.21.15.19:0/1056201277 wait complete. 2024-05-09T02:20:01.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.233+0000 7f0972a35b80 1 -- start start 2024-05-09T02:20:01.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.233+0000 7f0972a35b80 1 Processor -- start 2024-05-09T02:20:01.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f0972a35b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834b7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f0972a35b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb83408160 0x55cb834b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f0972a35b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb8349b6a0 0x55cb8351d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f0972a35b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1ae370 con 0x55cb8331e4a0 2024-05-09T02:20:01.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f0972a35b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1adbe0 con 0x55cb83408160 2024-05-09T02:20:01.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f0972a35b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1aadd0 con 0x55cb8349b6a0 2024-05-09T02:20:01.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f0972a35b80 10 client.0 _ll_register_callbacks cb 0x55cb8322f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:20:01.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f0961dbb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f0961dbb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834b7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46342/0 (socket says 172.21.15.19:46342) 2024-05-09T02:20:01.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f0961dbb700 1 -- 172.21.15.19:0/1978161611 learned_addr learned my addr 172.21.15.19:0/1978161611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:01.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f09625bc700 1 --2- 172.21.15.19:0/1978161611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb8349b6a0 0x55cb8351d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f09615ba700 1 --2- 172.21.15.19:0/1978161611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb83408160 0x55cb834b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f09625bc700 1 -- 172.21.15.19:0/1978161611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb83408160 msgr2=0x55cb834b7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f09625bc700 1 --2- 172.21.15.19:0/1978161611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb83408160 0x55cb834b7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f09625bc700 1 -- 172.21.15.19:0/1978161611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 msgr2=0x55cb834b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f09625bc700 1 --2- 172.21.15.19:0/1978161611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f09625bc700 1 -- 172.21.15.19:0/1978161611 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb8351dbb0 con 0x55cb8349b6a0 2024-05-09T02:20:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.234+0000 7f0961dbb700 1 --2- 172.21.15.19:0/1978161611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:20:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.235+0000 7f09625bc700 1 --2- 172.21.15.19:0/1978161611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb8349b6a0 0x55cb8351d670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0954011c50 tx=0x7f0954011f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56ce6fa2bace0dc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.235+0000 7f09457fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:20:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.235+0000 7f09457fa700 1 -- 172.21.15.19:0/1978161611 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954007bc0 con 0x55cb8349b6a0 2024-05-09T02:20:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.235+0000 7f0972a35b80 1 -- 172.21.15.19:0/1978161611 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f095802dd10 con 0x55cb8349b6a0 2024-05-09T02:20:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.235+0000 7f09457fa700 1 -- 172.21.15.19:0/1978161611 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f0954007d60 con 0x55cb8349b6a0 2024-05-09T02:20:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.235+0000 7f09457fa700 1 -- 172.21.15.19:0/1978161611 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f0954004bb0 con 0x55cb8349b6a0 2024-05-09T02:20:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.235+0000 7f0972a35b80 10 client.26146 fetch_fsmap learned FSMap version 53 2024-05-09T02:20:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.235+0000 7f0972a35b80 1 -- 172.21.15.19:0/1978161611 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55cb8351eea0 con 0x55cb8349b6a0 2024-05-09T02:20:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.235+0000 7f09457fa700 1 -- 172.21.15.19:0/1978161611 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f095802dd10 con 0x55cb8349b6a0 2024-05-09T02:20:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.235+0000 7f09457fa700 1 -- 172.21.15.19:0/1978161611 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 53) ==== 8386+0+0 (secure 0 0 0) 0x7f0954016400 con 0x55cb8349b6a0 2024-05-09T02:20:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.236+0000 7f0972a35b80 10 client.26146 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T02:20:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.236+0000 7f0972a35b80 10 client.26146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:20:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.236+0000 7f0972a35b80 20 client.26146 populate_metadata read hostname 'smithi019' 2024-05-09T02:20:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.236+0000 7f0972a35b80 1 --2- 172.21.15.19:0/1978161611 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb8351f670 0x55cb83521a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.236+0000 7f0972a35b80 4 client.26146 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:20:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.236+0000 7f0972a35b80 1 -- 172.21.15.19:0/1978161611 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb833c5a70 con 0x55cb8351f670 2024-05-09T02:20:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.236+0000 7f0961dbb700 1 --2- 172.21.15.19:0/1978161611 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb8351f670 0x55cb83521a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.236+0000 7f09457fa700 10 client.26146 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.236+0000 7f0961dbb700 1 --2- 172.21.15.19:0/1978161611 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb8351f670 0x55cb83521a90 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2abeb1bcf013c6c7 server_cookie=d9404ac232da930f in_seq=0 out_seq=0 2024-05-09T02:20:01.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.237+0000 7f09457fa700 1 -- 172.21.15.19:0/1978161611 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb833c5a70 con 0x55cb8351f670 2024-05-09T02:20:01.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.237+0000 7f09457fa700 10 client.26146 handle_command_reply: tid=0 2024-05-09T02:20:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.237+0000 7f0972a35b80 1 client.26146 shutdown 2024-05-09T02:20:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.237+0000 7f0972a35b80 1 -- 172.21.15.19:0/1978161611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb8349b6a0 msgr2=0x55cb8351d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.237+0000 7f0972a35b80 1 --2- 172.21.15.19:0/1978161611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb8349b6a0 0x55cb8351d670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0954011c50 tx=0x7f0954011f60 comp rx=0 tx=0).stop 2024-05-09T02:20:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.237+0000 7f0972a35b80 1 -- 172.21.15.19:0/1978161611 shutdown_connections 2024-05-09T02:20:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.237+0000 7f0972a35b80 1 --2- 172.21.15.19:0/1978161611 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb8351f670 0x55cb83521a90 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.237+0000 7f09457fa700 0 client.26146 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.237+0000 7f0972a35b80 1 --2- 172.21.15.19:0/1978161611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb8349b6a0 0x55cb8351d670 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.237+0000 7f0972a35b80 1 --2- 172.21.15.19:0/1978161611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb8331e4a0 0x55cb834b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.237+0000 7f0972a35b80 1 --2- 172.21.15.19:0/1978161611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb83408160 0x55cb834b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.237+0000 7f0972a35b80 1 -- 172.21.15.19:0/1978161611 >> 172.21.15.19:0/1978161611 conn(0x55cb8337e6a0 msgr2=0x55cb834a0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.237+0000 7f0972a35b80 1 -- 172.21.15.19:0/1978161611 shutdown_connections 2024-05-09T02:20:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.237+0000 7f0972a35b80 1 -- 172.21.15.19:0/1978161611 wait complete. 2024-05-09T02:20:01.245 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:20:01.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.238+0000 7f0972a35b80 20 client.26146 trim_cache size 0 max 16384 2024-05-09T02:20:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.238+0000 7f096401f700 1 -- 172.21.15.19:0/1671296146 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f09480b29f0 msgr2=0x7f09480b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.238+0000 7f096401f700 1 --2- 172.21.15.19:0/1671296146 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f09480b29f0 0x7f09480b4f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f09580059d0 tx=0x7f0958005960 comp rx=0 tx=0).stop 2024-05-09T02:20:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.238+0000 7f096401f700 1 -- 172.21.15.19:0/1671296146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c103310 msgr2=0x7f095c19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.238+0000 7f096401f700 1 --2- 172.21.15.19:0/1671296146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c103310 0x7f095c19ed60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f094c00ea30 tx=0x7f094c00ed40 comp rx=0 tx=0).stop 2024-05-09T02:20:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.239+0000 7f096401f700 1 -- 172.21.15.19:0/1671296146 shutdown_connections 2024-05-09T02:20:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.239+0000 7f096401f700 1 --2- 172.21.15.19:0/1671296146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f095c1089f0 0x7f095c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.239+0000 7f096401f700 1 --2- 172.21.15.19:0/1671296146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095c1029c0 0x7f095c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.239+0000 7f096401f700 1 --2- 172.21.15.19:0/1671296146 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f09480b29f0 0x7f09480b4f00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.239+0000 7f096401f700 1 --2- 172.21.15.19:0/1671296146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095c103310 0x7f095c19ed60 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.239+0000 7f096401f700 1 -- 172.21.15.19:0/1671296146 >> 172.21.15.19:0/1671296146 conn(0x7f095c0fe6f0 msgr2=0x7f095c100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.239+0000 7f096401f700 1 -- 172.21.15.19:0/1671296146 shutdown_connections 2024-05-09T02:20:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:01.239+0000 7f096401f700 1 -- 172.21.15.19:0/1671296146 wait complete. 2024-05-09T02:20:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 500.0 s, 1400.0 tps, lat 66.259 ms stddev 107.496 2024-05-09T02:20:02.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:02 smithi196 ceph-mon[34718]: pgmap v269: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.6 MiB/s wr, 567 op/s 2024-05-09T02:20:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:02 smithi019 ceph-mon[30151]: pgmap v269: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.6 MiB/s wr, 567 op/s 2024-05-09T02:20:02.846 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:02 smithi059 ceph-mon[34781]: pgmap v269: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.6 MiB/s wr, 567 op/s 2024-05-09T02:20:02.962 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T02:20:02.962 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T02:20:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:20:04.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:04 smithi019 ceph-mon[30151]: pgmap v270: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.6 MiB/s wr, 571 op/s 2024-05-09T02:20:04.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:04 smithi196 ceph-mon[34718]: pgmap v270: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.6 MiB/s wr, 571 op/s 2024-05-09T02:20:04.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.945+0000 7f77d6552700 1 -- 172.21.15.19:0/2153079695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d01034c0 msgr2=0x7f77d01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:04.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.945+0000 7f77d6552700 1 --2- 172.21.15.19:0/2153079695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d01034c0 0x7f77d01038d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f77c00099c0 tx=0x7f77c0009cd0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.945+0000 7f77d6552700 1 -- 172.21.15.19:0/2153079695 shutdown_connections 2024-05-09T02:20:04.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.945+0000 7f77d6552700 1 --2- 172.21.15.19:0/2153079695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d0108740 0x7f77d01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.945+0000 7f77d6552700 1 --2- 172.21.15.19:0/2153079695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d01034c0 0x7f77d01038d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.945+0000 7f77d6552700 1 --2- 172.21.15.19:0/2153079695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77d0107df0 0x7f77d0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.945+0000 7f77d6552700 1 -- 172.21.15.19:0/2153079695 >> 172.21.15.19:0/2153079695 conn(0x7f77d00fedc0 msgr2=0x7f77d01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:04.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.946+0000 7f77d6552700 1 -- 172.21.15.19:0/2153079695 shutdown_connections 2024-05-09T02:20:04.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.946+0000 7f77d6552700 1 -- 172.21.15.19:0/2153079695 wait complete. 2024-05-09T02:20:04.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.946+0000 7f77d6552700 1 Processor -- start 2024-05-09T02:20:04.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77d6552700 1 -- start start 2024-05-09T02:20:04.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77d6552700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d01034c0 0x7f77d019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77d6552700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77d0107df0 0x7f77d019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77d6552700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0108740 0x7f77d019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77d6552700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0113370 con 0x7f77d0108740 2024-05-09T02:20:04.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77d6552700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01131f0 con 0x7f77d0107df0 2024-05-09T02:20:04.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77d6552700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01134f0 con 0x7f77d01034c0 2024-05-09T02:20:04.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77d4aef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0108740 0x7f77d019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77d4aef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0108740 0x7f77d019f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46348/0 (socket says 172.21.15.19:46348) 2024-05-09T02:20:04.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77d4aef700 1 -- 172.21.15.19:0/254319914 learned_addr learned my addr 172.21.15.19:0/254319914 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:04.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77cffff700 1 --2- 172.21.15.19:0/254319914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d01034c0 0x7f77d019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77d4aef700 1 -- 172.21.15.19:0/254319914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d01034c0 msgr2=0x7f77d019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:04.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77d4aef700 1 --2- 172.21.15.19:0/254319914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d01034c0 0x7f77d019adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77d4aef700 1 -- 172.21.15.19:0/254319914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77d0107df0 msgr2=0x7f77d019b390 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T02:20:04.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77d4aef700 1 --2- 172.21.15.19:0/254319914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77d0107df0 0x7f77d019b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77d4aef700 1 -- 172.21.15.19:0/254319914 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d019fcc0 con 0x7f77d0108740 2024-05-09T02:20:04.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.947+0000 7f77cffff700 1 --2- 172.21.15.19:0/254319914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d01034c0 0x7f77d019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:20:04.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.948+0000 7f77d4aef700 1 --2- 172.21.15.19:0/254319914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0108740 0x7f77d019f620 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f77c800ef30 tx=0x7f77c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:04.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.948+0000 7f77cdffb700 1 -- 172.21.15.19:0/254319914 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c80092e0 con 0x7f77d0108740 2024-05-09T02:20:04.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.948+0000 7f77d6552700 1 -- 172.21.15.19:0/254319914 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d019ff50 con 0x7f77d0108740 2024-05-09T02:20:04.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.948+0000 7f77cdffb700 1 -- 172.21.15.19:0/254319914 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c8004bb0 con 0x7f77d0108740 2024-05-09T02:20:04.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.948+0000 7f77cdffb700 1 -- 172.21.15.19:0/254319914 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8004280 con 0x7f77d0108740 2024-05-09T02:20:04.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.948+0000 7f77d6552700 1 -- 172.21.15.19:0/254319914 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d00ff1d0 con 0x7f77d0108740 2024-05-09T02:20:04.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.951+0000 7f77cdffb700 1 -- 172.21.15.19:0/254319914 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f77c801b030 con 0x7f77d0108740 2024-05-09T02:20:04.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.952+0000 7f77cdffb700 1 --2- 172.21.15.19:0/254319914 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f77bc0b29f0 0x7f77bc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.952+0000 7f77cdffb700 1 -- 172.21.15.19:0/254319914 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f77c80e2110 con 0x7f77d0108740 2024-05-09T02:20:04.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.952+0000 7f77cffff700 1 --2- 172.21.15.19:0/254319914 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f77bc0b29f0 0x7f77bc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.952+0000 7f77e4f68b80 1 Processor -- start 2024-05-09T02:20:04.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.952+0000 7f77e4f68b80 1 -- start start 2024-05-09T02:20:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77e4f68b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b32fd80 0x55fe3b440ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77e4f68b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b2b5520 0x55fe3b441010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77e4f68b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b43ac90 0x55fe3b43b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77e4f68b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fe3b355120 con 0x55fe3b32fd80 2024-05-09T02:20:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77e4f68b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fe3b2faa20 con 0x55fe3b2b5520 2024-05-09T02:20:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77e4f68b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fe3b3510e0 con 0x55fe3b43ac90 2024-05-09T02:20:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77c7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b2b5520 0x55fe3b441010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77cffff700 1 --2- 172.21.15.19:0/254319914 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f77bc0b29f0 0x7f77bc0b4f00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f77c0000c00 tx=0x7f77c0005fb0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77d4aef700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b43ac90 0x55fe3b43b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77cffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b32fd80 0x55fe3b440ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77c7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b2b5520 0x55fe3b441010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:49396/0 (socket says 172.21.15.19:49396) 2024-05-09T02:20:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77d4aef700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b43ac90 0x55fe3b43b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:37384/0 (socket says 172.21.15.19:37384) 2024-05-09T02:20:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77c7fff700 1 -- 172.21.15.19:0/1931650923 learned_addr learned my addr 172.21.15.19:0/1931650923 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77c7fff700 1 -- 172.21.15.19:0/1931650923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b43ac90 msgr2=0x55fe3b43b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77c7fff700 1 --2- 172.21.15.19:0/1931650923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b43ac90 0x55fe3b43b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77c7fff700 1 -- 172.21.15.19:0/1931650923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b32fd80 msgr2=0x55fe3b440ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77c7fff700 1 --2- 172.21.15.19:0/1931650923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b32fd80 0x55fe3b440ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.953+0000 7f77c7fff700 1 -- 172.21.15.19:0/1931650923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe3b2eb580 con 0x55fe3b2b5520 2024-05-09T02:20:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.954+0000 7f77c7fff700 1 --2- 172.21.15.19:0/1931650923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b2b5520 0x55fe3b441010 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009bd0 tx=0x7f77b8009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=850d4ea1c517b334 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.954+0000 7f77c67fc700 1 -- 172.21.15.19:0/1931650923 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b800eb00 con 0x55fe3b2b5520 2024-05-09T02:20:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.954+0000 7f77c67fc700 1 -- 172.21.15.19:0/1931650923 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77b8004ce0 con 0x55fe3b2b5520 2024-05-09T02:20:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.954+0000 7f77c67fc700 1 -- 172.21.15.19:0/1931650923 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8005650 con 0x55fe3b2b5520 2024-05-09T02:20:04.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.955+0000 7f77e4f68b80 1 -- 172.21.15.19:0/1931650923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b2b5520 msgr2=0x55fe3b441010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:04.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.955+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/1931650923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b2b5520 0x55fe3b441010 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009bd0 tx=0x7f77b8009ee0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.955+0000 7f77e4f68b80 1 -- 172.21.15.19:0/1931650923 shutdown_connections 2024-05-09T02:20:04.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.955+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/1931650923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b43ac90 0x55fe3b43b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.955+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/1931650923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b32fd80 0x55fe3b440ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.955+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/1931650923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b2b5520 0x55fe3b441010 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.955+0000 7f77e4f68b80 1 -- 172.21.15.19:0/1931650923 >> 172.21.15.19:0/1931650923 conn(0x55fe3b2b8ff0 msgr2=0x55fe3b42d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:04.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.955+0000 7f77e4f68b80 1 -- 172.21.15.19:0/1931650923 shutdown_connections 2024-05-09T02:20:04.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.956+0000 7f77e4f68b80 1 -- 172.21.15.19:0/1931650923 wait complete. 2024-05-09T02:20:04.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.956+0000 7f77e4f68b80 1 -- start start 2024-05-09T02:20:04.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.956+0000 7f77e4f68b80 1 Processor -- start 2024-05-09T02:20:04.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.957+0000 7f77e4f68b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b2b5520 0x55fe3b4448f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.957+0000 7f77e4f68b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b444e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.957+0000 7f77e4f68b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b43ac90 0x55fe3b445370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.957+0000 7f77e4f68b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fe3b43b9f0 con 0x55fe3b2b5520 2024-05-09T02:20:04.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.957+0000 7f77e4f68b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fe3b2efba0 con 0x55fe3b43ac90 2024-05-09T02:20:04.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.957+0000 7f77e4f68b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fe3b4478b0 con 0x55fe3b32fd80 2024-05-09T02:20:04.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.957+0000 7f77c7fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b444e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.957+0000 7f77cffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b2b5520 0x55fe3b4448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.957+0000 7f77c7fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b444e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:37394/0 (socket says 172.21.15.19:37394) 2024-05-09T02:20:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.957+0000 7f77c7fff700 1 -- 172.21.15.19:0/2665672366 learned_addr learned my addr 172.21.15.19:0/2665672366 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.957+0000 7f77e4f68b80 10 client.0 _ll_register_callbacks cb 0x55fe3b1c6740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:20:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.957+0000 7f77d4aef700 1 --2- 172.21.15.19:0/2665672366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b43ac90 0x55fe3b445370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.957+0000 7f77c7fff700 1 -- 172.21.15.19:0/2665672366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b43ac90 msgr2=0x55fe3b445370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.957+0000 7f77c7fff700 1 --2- 172.21.15.19:0/2665672366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b43ac90 0x55fe3b445370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.958+0000 7f77c7fff700 1 -- 172.21.15.19:0/2665672366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b2b5520 msgr2=0x55fe3b4448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.958+0000 7f77c7fff700 1 --2- 172.21.15.19:0/2665672366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b2b5520 0x55fe3b4448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.958+0000 7f77c7fff700 1 -- 172.21.15.19:0/2665672366 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe3b1d0510 con 0x55fe3b32fd80 2024-05-09T02:20:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.958+0000 7f77c7fff700 1 --2- 172.21.15.19:0/2665672366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b444e30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f77b8005230 tx=0x7f77b8005540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ce3d9bc7249cbde server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.958+0000 7f77af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:20:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.958+0000 7f77af7fe700 1 -- 172.21.15.19:0/2665672366 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b801d040 con 0x55fe3b32fd80 2024-05-09T02:20:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.958+0000 7f77af7fe700 1 -- 172.21.15.19:0/2665672366 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f77b800eb00 con 0x55fe3b32fd80 2024-05-09T02:20:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.958+0000 7f77e4f68b80 1 -- 172.21.15.19:0/2665672366 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77b8004ce0 con 0x55fe3b32fd80 2024-05-09T02:20:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.958+0000 7f77af7fe700 1 -- 172.21.15.19:0/2665672366 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f77b8030050 con 0x55fe3b32fd80 2024-05-09T02:20:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.958+0000 7f77e4f68b80 10 client.26158 fetch_fsmap learned FSMap version 53 2024-05-09T02:20:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.958+0000 7f77e4f68b80 1 -- 172.21.15.19:0/2665672366 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55fe3b4522e0 con 0x55fe3b32fd80 2024-05-09T02:20:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.959+0000 7f77af7fe700 1 -- 172.21.15.19:0/2665672366 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77b8004ce0 con 0x55fe3b32fd80 2024-05-09T02:20:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.959+0000 7f77af7fe700 1 -- 172.21.15.19:0/2665672366 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 53) ==== 8386+0+0 (secure 0 0 0) 0x7f77b8003680 con 0x55fe3b32fd80 2024-05-09T02:20:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.959+0000 7f77e4f68b80 10 client.26158 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T02:20:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.959+0000 7f77e4f68b80 10 client.26158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:20:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.959+0000 7f77e4f68b80 20 client.26158 populate_metadata read hostname 'smithi019' 2024-05-09T02:20:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.959+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/2665672366 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55fe3b454860 0x55fe3b456c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.959+0000 7f77e4f68b80 4 client.26158 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:20:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.959+0000 7f77e4f68b80 1 -- 172.21.15.19:0/2665672366 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe3b362b00 con 0x55fe3b454860 2024-05-09T02:20:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.959+0000 7f77cffff700 1 --2- 172.21.15.19:0/2665672366 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55fe3b454860 0x55fe3b456c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.960+0000 7f77af7fe700 10 client.26158 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.960+0000 7f77cffff700 1 --2- 172.21.15.19:0/2665672366 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55fe3b454860 0x55fe3b456c80 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1501046d97c1ea8 server_cookie=c281e475b1778278 in_seq=0 out_seq=0 2024-05-09T02:20:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.960+0000 7f77af7fe700 1 -- 172.21.15.19:0/2665672366 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe3b362b00 con 0x55fe3b454860 2024-05-09T02:20:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.960+0000 7f77af7fe700 10 client.26158 handle_command_reply: tid=0 2024-05-09T02:20:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.960+0000 7f77e4f68b80 1 client.26158 shutdown 2024-05-09T02:20:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.961+0000 7f77e4f68b80 1 -- 172.21.15.19:0/2665672366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 msgr2=0x55fe3b444e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.961+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/2665672366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b444e30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f77b8005230 tx=0x7f77b8005540 comp rx=0 tx=0).stop 2024-05-09T02:20:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.961+0000 7f77e4f68b80 1 -- 172.21.15.19:0/2665672366 shutdown_connections 2024-05-09T02:20:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.961+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/2665672366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b43ac90 0x55fe3b445370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.961+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/2665672366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b2b5520 0x55fe3b4448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.961+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/2665672366 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55fe3b454860 0x55fe3b456c80 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.961+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/2665672366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b444e30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.961+0000 7f77af7fe700 0 client.26158 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.961+0000 7f77e4f68b80 1 -- 172.21.15.19:0/2665672366 >> 172.21.15.19:0/2665672366 conn(0x55fe3b2b8ff0 msgr2=0x55fe3b427070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.961+0000 7f77e4f68b80 1 -- 172.21.15.19:0/2665672366 shutdown_connections 2024-05-09T02:20:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.961+0000 7f77e4f68b80 1 -- 172.21.15.19:0/2665672366 wait complete. 2024-05-09T02:20:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.961+0000 7f77e4f68b80 20 client.26158 trim_cache size 0 max 16384 2024-05-09T02:20:04.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:04 smithi059 ceph-mon[34781]: pgmap v270: 97 pgs: 97 active+clean; 4.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.6 MiB/s wr, 571 op/s 2024-05-09T02:20:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.984+0000 7f77e4f68b80 1 Processor -- start 2024-05-09T02:20:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.984+0000 7f77e4f68b80 1 -- start start 2024-05-09T02:20:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.984+0000 7f77e4f68b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b2b5520 0x55fe3b442580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.984+0000 7f77e4f68b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b43ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.984+0000 7f77e4f68b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b43ac90 0x55fe3b4451b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.984+0000 7f77e4f68b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01ace10 con 0x55fe3b43ac90 2024-05-09T02:20:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.984+0000 7f77e4f68b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01aead0 con 0x55fe3b2b5520 2024-05-09T02:20:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.984+0000 7f77e4f68b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01ad510 con 0x55fe3b32fd80 2024-05-09T02:20:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.984+0000 7f77cffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b2b5520 0x55fe3b442580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.984+0000 7f77c7fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b43ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.984+0000 7f77cffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b2b5520 0x55fe3b442580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:49426/0 (socket says 172.21.15.19:49426) 2024-05-09T02:20:04.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.984+0000 7f77cffff700 1 -- 172.21.15.19:0/857601951 learned_addr learned my addr 172.21.15.19:0/857601951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:04.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.984+0000 7f77d4aef700 1 --2- 172.21.15.19:0/857601951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b43ac90 0x55fe3b4451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.985+0000 7f77cffff700 1 -- 172.21.15.19:0/857601951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 msgr2=0x55fe3b43ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.985+0000 7f77cffff700 1 --2- 172.21.15.19:0/857601951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b43ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.985+0000 7f77cffff700 1 -- 172.21.15.19:0/857601951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b43ac90 msgr2=0x55fe3b4451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.985+0000 7f77cffff700 1 --2- 172.21.15.19:0/857601951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b43ac90 0x55fe3b4451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.985+0000 7f77cffff700 1 -- 172.21.15.19:0/857601951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe3b441380 con 0x55fe3b2b5520 2024-05-09T02:20:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.985+0000 7f77c7fff700 1 --2- 172.21.15.19:0/857601951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b43ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:20:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.985+0000 7f77d4aef700 1 --2- 172.21.15.19:0/857601951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b43ac90 0x55fe3b4451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:20:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.985+0000 7f77cffff700 1 --2- 172.21.15.19:0/857601951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b2b5520 0x55fe3b442580 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009fd0 tx=0x7f77c001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91c805ac7ebb4a71 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.985+0000 7f77c5ffb700 1 -- 172.21.15.19:0/857601951 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0023070 con 0x55fe3b2b5520 2024-05-09T02:20:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.985+0000 7f77c5ffb700 1 -- 172.21.15.19:0/857601951 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c00044d0 con 0x55fe3b2b5520 2024-05-09T02:20:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.985+0000 7f77c5ffb700 1 -- 172.21.15.19:0/857601951 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0003d10 con 0x55fe3b2b5520 2024-05-09T02:20:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.986+0000 7f77e4f68b80 1 -- 172.21.15.19:0/857601951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b2b5520 msgr2=0x55fe3b442580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.986+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/857601951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b2b5520 0x55fe3b442580 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009fd0 tx=0x7f77c001c000 comp rx=0 tx=0).stop 2024-05-09T02:20:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.986+0000 7f77e4f68b80 1 -- 172.21.15.19:0/857601951 shutdown_connections 2024-05-09T02:20:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.986+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/857601951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b43ac90 0x55fe3b4451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.986+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/857601951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b2b5520 0x55fe3b442580 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.986+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/857601951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b43ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.986+0000 7f77e4f68b80 1 -- 172.21.15.19:0/857601951 >> 172.21.15.19:0/857601951 conn(0x55fe3b2b8ff0 msgr2=0x55fe3b4276e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.986+0000 7f77e4f68b80 1 -- 172.21.15.19:0/857601951 shutdown_connections 2024-05-09T02:20:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.986+0000 7f77e4f68b80 1 -- 172.21.15.19:0/857601951 wait complete. 2024-05-09T02:20:04.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.987+0000 7f77e4f68b80 1 -- start start 2024-05-09T02:20:04.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.987+0000 7f77e4f68b80 1 Processor -- start 2024-05-09T02:20:04.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.987+0000 7f77e4f68b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b2b5520 0x55fe3b44cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.987+0000 7f77e4f68b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b449560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.987+0000 7f77e4f68b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b43ac90 0x55fe3b449aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:04.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77e4f68b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01ae370 con 0x55fe3b2b5520 2024-05-09T02:20:04.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77e4f68b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01adbe0 con 0x55fe3b43ac90 2024-05-09T02:20:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77e4f68b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01aadf0 con 0x55fe3b32fd80 2024-05-09T02:20:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77e4f68b80 10 client.0 _ll_register_callbacks cb 0x55fe3b1c6740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:20:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77cffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b2b5520 0x55fe3b44cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77c7fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b449560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77d4aef700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b43ac90 0x55fe3b449aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77c7fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b449560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:37418/0 (socket says 172.21.15.19:37418) 2024-05-09T02:20:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77cffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b2b5520 0x55fe3b44cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46372/0 (socket says 172.21.15.19:46372) 2024-05-09T02:20:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77c7fff700 1 -- 172.21.15.19:0/453563246 learned_addr learned my addr 172.21.15.19:0/453563246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77cffff700 1 -- 172.21.15.19:0/453563246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 msgr2=0x55fe3b449560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77cffff700 1 --2- 172.21.15.19:0/453563246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b449560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77cffff700 1 -- 172.21.15.19:0/453563246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b43ac90 msgr2=0x55fe3b449aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77cffff700 1 --2- 172.21.15.19:0/453563246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b43ac90 0x55fe3b449aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77cffff700 1 -- 172.21.15.19:0/453563246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe3b449fe0 con 0x55fe3b2b5520 2024-05-09T02:20:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77c7fff700 1 --2- 172.21.15.19:0/453563246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b449560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:20:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77cffff700 1 --2- 172.21.15.19:0/453563246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b2b5520 0x55fe3b44cae0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x55fe3b42ea80 tx=0x7f77c0004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb27e14401d0bd6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:20:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.988+0000 7f77af7fe700 1 -- 172.21.15.19:0/453563246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0043060 con 0x55fe3b2b5520 2024-05-09T02:20:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.989+0000 7f77e4f68b80 1 -- 172.21.15.19:0/453563246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c00044d0 con 0x55fe3b2b5520 2024-05-09T02:20:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.989+0000 7f77af7fe700 1 -- 172.21.15.19:0/453563246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f77c0033040 con 0x55fe3b2b5520 2024-05-09T02:20:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.989+0000 7f77af7fe700 1 -- 172.21.15.19:0/453563246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f77c002f030 con 0x55fe3b2b5520 2024-05-09T02:20:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.989+0000 7f77e4f68b80 10 client.16395 fetch_fsmap learned FSMap version 53 2024-05-09T02:20:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.989+0000 7f77e4f68b80 1 -- 172.21.15.19:0/453563246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55fe3b4bfa90 con 0x55fe3b2b5520 2024-05-09T02:20:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.989+0000 7f77af7fe700 1 -- 172.21.15.19:0/453563246 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c00044d0 con 0x55fe3b2b5520 2024-05-09T02:20:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.989+0000 7f77af7fe700 1 -- 172.21.15.19:0/453563246 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 53) ==== 8386+0+0 (secure 0 0 0) 0x7f77c0034ce0 con 0x55fe3b2b5520 2024-05-09T02:20:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.989+0000 7f77e4f68b80 10 client.16395 fetch_fsmap finished waiting for FSMap version 53 2024-05-09T02:20:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.989+0000 7f77e4f68b80 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:20:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.989+0000 7f77e4f68b80 20 client.16395 populate_metadata read hostname 'smithi019' 2024-05-09T02:20:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.989+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/453563246 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55fe3b4c0180 0x55fe3b4c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.989+0000 7f77e4f68b80 4 client.16395 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T02:20:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.989+0000 7f77e4f68b80 1 -- 172.21.15.19:0/453563246 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fe3b35cba0 con 0x55fe3b4c0180 2024-05-09T02:20:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.990+0000 7f77c7fff700 1 --2- 172.21.15.19:0/453563246 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55fe3b4c0180 0x55fe3b4c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.990+0000 7f77af7fe700 10 client.16395 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.990+0000 7f77c7fff700 1 --2- 172.21.15.19:0/453563246 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55fe3b4c0180 0x55fe3b4c25a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d289077cfa748994 server_cookie=3c27018ed9136156 in_seq=0 out_seq=0 2024-05-09T02:20:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.991+0000 7f77af7fe700 1 -- 172.21.15.19:0/453563246 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fe3b35cba0 con 0x55fe3b4c0180 2024-05-09T02:20:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.991+0000 7f77af7fe700 10 client.16395 handle_command_reply: tid=0 2024-05-09T02:20:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.991+0000 7f77e4f68b80 1 client.16395 shutdown 2024-05-09T02:20:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.992+0000 7f77e4f68b80 1 -- 172.21.15.19:0/453563246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b2b5520 msgr2=0x55fe3b44cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.992+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/453563246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b2b5520 0x55fe3b44cae0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x55fe3b42ea80 tx=0x7f77c0004710 comp rx=0 tx=0).stop 2024-05-09T02:20:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.992+0000 7f77e4f68b80 1 -- 172.21.15.19:0/453563246 shutdown_connections 2024-05-09T02:20:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.992+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/453563246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe3b43ac90 0x55fe3b449aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.992+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/453563246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe3b2b5520 0x55fe3b44cae0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.992+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/453563246 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55fe3b4c0180 0x55fe3b4c25a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.992+0000 7f77e4f68b80 1 --2- 172.21.15.19:0/453563246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fe3b32fd80 0x55fe3b449560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.992+0000 7f77e4f68b80 1 -- 172.21.15.19:0/453563246 >> 172.21.15.19:0/453563246 conn(0x55fe3b2b8ff0 msgr2=0x55fe3b439c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.992+0000 7f77af7fe700 0 client.16395 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.992+0000 7f77e4f68b80 1 -- 172.21.15.19:0/453563246 shutdown_connections 2024-05-09T02:20:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.992+0000 7f77e4f68b80 1 -- 172.21.15.19:0/453563246 wait complete. 2024-05-09T02:20:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.992+0000 7f77e4f68b80 20 client.16395 trim_cache size 0 max 16384 2024-05-09T02:20:05.003 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:20:05.003 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T02:20:05.003 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "8d1e7477-864a-4e74-b84e-33e45f653c8c", 2024-05-09T02:20:05.003 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T02:20:05.003 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:20:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.993+0000 7f77d6552700 1 -- 172.21.15.19:0/254319914 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f77bc0b29f0 msgr2=0x7f77bc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.993+0000 7f77d6552700 1 --2- 172.21.15.19:0/254319914 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f77bc0b29f0 0x7f77bc0b4f00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f77c0000c00 tx=0x7f77c0005fb0 comp rx=0 tx=0).stop 2024-05-09T02:20:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.993+0000 7f77d6552700 1 -- 172.21.15.19:0/254319914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0108740 msgr2=0x7f77d019f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.993+0000 7f77d6552700 1 --2- 172.21.15.19:0/254319914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0108740 0x7f77d019f620 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f77c800ef30 tx=0x7f77c800c3a0 comp rx=0 tx=0).stop 2024-05-09T02:20:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.993+0000 7f77d6552700 1 -- 172.21.15.19:0/254319914 shutdown_connections 2024-05-09T02:20:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.993+0000 7f77d6552700 1 --2- 172.21.15.19:0/254319914 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f77bc0b29f0 0x7f77bc0b4f00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.993+0000 7f77d6552700 1 --2- 172.21.15.19:0/254319914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0108740 0x7f77d019f620 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.993+0000 7f77d6552700 1 --2- 172.21.15.19:0/254319914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d01034c0 0x7f77d019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.993+0000 7f77d6552700 1 --2- 172.21.15.19:0/254319914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77d0107df0 0x7f77d019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.993+0000 7f77d6552700 1 -- 172.21.15.19:0/254319914 >> 172.21.15.19:0/254319914 conn(0x7f77d00fedc0 msgr2=0x7f77d006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.993+0000 7f77d6552700 1 -- 172.21.15.19:0/254319914 shutdown_connections 2024-05-09T02:20:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:04.993+0000 7f77d6552700 1 -- 172.21.15.19:0/254319914 wait complete. 2024-05-09T02:20:05.617 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:05 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:20:05.618 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:05 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:05.774 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:20:05.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:05 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:20:05.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:05 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:05.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:05 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:20:05.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:05 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.331 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:20:06.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub queued for path: / 2024-05-09T02:20:06.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:20:06.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:20:06.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:20:06.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:20:06.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:20:06.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:20:06.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:20:06.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:20:06.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:20:06.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:06 smithi019 ceph-mon[30151]: pgmap v271: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.3 MiB/s wr, 796 op/s 2024-05-09T02:20:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub queued for path: / 2024-05-09T02:20:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:20:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:20:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:20:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:20:06.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:20:06.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:20:06.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:20:06.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:20:06.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:20:06.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:06 smithi196 ceph-mon[34718]: pgmap v271: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.3 MiB/s wr, 796 op/s 2024-05-09T02:20:06.868 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub queued for path: / 2024-05-09T02:20:06.868 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.868 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:20:06.868 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:20:06.868 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.868 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:20:06.868 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:20:06.868 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:20:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:20:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:20:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:20:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:20:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:20:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:06 smithi059 ceph-mon[34781]: pgmap v271: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.3 MiB/s wr, 796 op/s 2024-05-09T02:20:07.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 505.0 s, 270.4 tps, lat 378.477 ms stddev 506.766 2024-05-09T02:20:07.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:07 smithi196 ceph-mon[34718]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T02:20:07.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:07 smithi196 ceph-mon[34718]: pgmap v273: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.6 MiB/s wr, 598 op/s 2024-05-09T02:20:07.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:07 smithi019 ceph-mon[30151]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T02:20:07.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:07 smithi019 ceph-mon[30151]: pgmap v273: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.6 MiB/s wr, 598 op/s 2024-05-09T02:20:07.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:07 smithi059 ceph-mon[34781]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T02:20:07.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:07 smithi059 ceph-mon[34781]: pgmap v273: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.6 MiB/s wr, 598 op/s 2024-05-09T02:20:09.628 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:09 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:20:09.628 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:09 smithi019 ceph-mon[30151]: pgmap v274: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 28 MiB/s wr, 1.51k op/s 2024-05-09T02:20:09.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.686+0000 7f8f2e7bf700 1 -- 172.21.15.19:0/351335884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f28107e40 msgr2=0x7f8f28108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:09.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.686+0000 7f8f2e7bf700 1 --2- 172.21.15.19:0/351335884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f28107e40 0x7f8f28108250 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0099c0 tx=0x7f8f1c009cd0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.686+0000 7f8f2e7bf700 1 -- 172.21.15.19:0/351335884 shutdown_connections 2024-05-09T02:20:09.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.686+0000 7f8f2e7bf700 1 --2- 172.21.15.19:0/351335884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f28108790 0x7f8f281100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.686+0000 7f8f2e7bf700 1 --2- 172.21.15.19:0/351335884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f28103f40 0x7f8f28104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.686+0000 7f8f2e7bf700 1 --2- 172.21.15.19:0/351335884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f28107e40 0x7f8f28108250 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.686+0000 7f8f2e7bf700 1 -- 172.21.15.19:0/351335884 >> 172.21.15.19:0/351335884 conn(0x7f8f280fe0d0 msgr2=0x7f8f281004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:09.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.686+0000 7f8f2e7bf700 1 -- 172.21.15.19:0/351335884 shutdown_connections 2024-05-09T02:20:09.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.686+0000 7f8f2e7bf700 1 -- 172.21.15.19:0/351335884 wait complete. 2024-05-09T02:20:09.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.688+0000 7f8f2e7bf700 1 Processor -- start 2024-05-09T02:20:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.689+0000 7f8f2e7bf700 1 -- start start 2024-05-09T02:20:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f2e7bf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f28103f40 0x7f8f281a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f2e7bf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f28107e40 0x7f8f281a75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f2e7bf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f28108790 0x7f8f281a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f2e7bf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f28113380 con 0x7f8f28108790 2024-05-09T02:20:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f2e7bf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f28113200 con 0x7f8f28107e40 2024-05-09T02:20:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f2e7bf700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f28113500 con 0x7f8f28103f40 2024-05-09T02:20:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f277fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f28107e40 0x7f8f281a75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f277fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f28107e40 0x7f8f281a75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:41858/0 (socket says 172.21.15.19:41858) 2024-05-09T02:20:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f277fe700 1 -- 172.21.15.19:0/12085254 learned_addr learned my addr 172.21.15.19:0/12085254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f2cd5c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f28108790 0x7f8f281a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f27fff700 1 --2- 172.21.15.19:0/12085254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f28103f40 0x7f8f281a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f277fe700 1 -- 172.21.15.19:0/12085254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f28103f40 msgr2=0x7f8f281a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f277fe700 1 --2- 172.21.15.19:0/12085254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f28103f40 0x7f8f281a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f277fe700 1 -- 172.21.15.19:0/12085254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f28108790 msgr2=0x7f8f281a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f277fe700 1 --2- 172.21.15.19:0/12085254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f28108790 0x7f8f281a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.690+0000 7f8f277fe700 1 -- 172.21.15.19:0/12085254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f281a1970 con 0x7f8f28107e40 2024-05-09T02:20:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.691+0000 7f8f277fe700 1 --2- 172.21.15.19:0/12085254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f28107e40 0x7f8f281a75e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c006010 tx=0x7f8f1c0055f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.691+0000 7f8f257fa700 1 -- 172.21.15.19:0/12085254 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c026070 con 0x7f8f28107e40 2024-05-09T02:20:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.691+0000 7f8f257fa700 1 -- 172.21.15.19:0/12085254 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f1c00f460 con 0x7f8f28107e40 2024-05-09T02:20:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.691+0000 7f8f2e7bf700 1 -- 172.21.15.19:0/12085254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f281a1c00 con 0x7f8f28107e40 2024-05-09T02:20:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.691+0000 7f8f257fa700 1 -- 172.21.15.19:0/12085254 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c009e60 con 0x7f8f28107e40 2024-05-09T02:20:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.691+0000 7f8f2e7bf700 1 -- 172.21.15.19:0/12085254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f281adec0 con 0x7f8f28107e40 2024-05-09T02:20:09.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.695+0000 7f8f257fa700 1 -- 172.21.15.19:0/12085254 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8f1c021690 con 0x7f8f28107e40 2024-05-09T02:20:09.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.696+0000 7f8f3d1d5b80 1 Processor -- start 2024-05-09T02:20:09.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.696+0000 7f8f3d1d5b80 1 -- start start 2024-05-09T02:20:09.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.696+0000 7f8f257fa700 1 --2- 172.21.15.19:0/12085254 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8f100b2a70 0x7f8f100b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f2cd5c700 1 --2- 172.21.15.19:0/12085254 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8f100b2a70 0x7f8f100b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f3d1d5b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427af31160 0x56427afca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f3d1d5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427ae474a0 0x56427afcaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f3d1d5b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427afc46a0 0x56427afc4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f257fa700 1 -- 172.21.15.19:0/12085254 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8615+0+0 (secure 0 0 0) 0x7f8f1c0e2e70 con 0x7f8f28107e40 2024-05-09T02:20:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f3d1d5b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56427aee6ff0 con 0x56427af31160 2024-05-09T02:20:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f3d1d5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56427ae857f0 con 0x56427ae474a0 2024-05-09T02:20:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f3d1d5b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56427aee2fb0 con 0x56427afc46a0 2024-05-09T02:20:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f27fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427af31160 0x56427afca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f27fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427af31160 0x56427afca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33178/0 (socket says 172.21.15.19:33178) 2024-05-09T02:20:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f27fff700 1 -- 172.21.15.19:0/643438339 learned_addr learned my addr 172.21.15.19:0/643438339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f277fe700 1 --2- 172.21.15.19:0/643438339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427ae474a0 0x56427afcaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f2cd5c700 1 --2- 172.21.15.19:0/643438339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427afc46a0 0x56427afc4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f2cd5c700 1 -- 172.21.15.19:0/643438339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427ae474a0 msgr2=0x56427afcaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f2cd5c700 1 --2- 172.21.15.19:0/643438339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427ae474a0 0x56427afcaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f2cd5c700 1 -- 172.21.15.19:0/643438339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427af31160 msgr2=0x56427afca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f2cd5c700 1 --2- 172.21.15.19:0/643438339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427af31160 0x56427afca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f2cd5c700 1 -- 172.21.15.19:0/643438339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56427ae7d370 con 0x56427afc46a0 2024-05-09T02:20:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f277fe700 1 --2- 172.21.15.19:0/643438339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427ae474a0 0x56427afcaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:20:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f27fff700 1 --2- 172.21.15.19:0/643438339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427af31160 0x56427afca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:20:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f2cd5c700 1 --2- 172.21.15.19:0/12085254 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8f100b2a70 0x7f8f100b4f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000fd40 tx=0x7f8f2000f830 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.697+0000 7f8f2cd5c700 1 --2- 172.21.15.19:0/643438339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427afc46a0 0x56427afc4ad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8f2001ae20 tx=0x7f8f2001d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fb498837441bf76 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.698+0000 7f8f0e7fc700 1 -- 172.21.15.19:0/643438339 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20021070 con 0x56427afc46a0 2024-05-09T02:20:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.698+0000 7f8f0e7fc700 1 -- 172.21.15.19:0/643438339 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f2001be50 con 0x56427afc46a0 2024-05-09T02:20:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.698+0000 7f8f0e7fc700 1 -- 172.21.15.19:0/643438339 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20004020 con 0x56427afc46a0 2024-05-09T02:20:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.699+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/643438339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427afc46a0 msgr2=0x56427afc4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.699+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/643438339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427afc46a0 0x56427afc4ad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8f2001ae20 tx=0x7f8f2001d000 comp rx=0 tx=0).stop 2024-05-09T02:20:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.699+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/643438339 shutdown_connections 2024-05-09T02:20:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.699+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/643438339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427afc46a0 0x56427afc4ad0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.699+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/643438339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427af31160 0x56427afca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.699+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/643438339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427ae474a0 0x56427afcaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.699+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/643438339 >> 172.21.15.19:0/643438339 conn(0x56427aea76a0 msgr2=0x56427afbf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.699+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/643438339 shutdown_connections 2024-05-09T02:20:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.699+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/643438339 wait complete. 2024-05-09T02:20:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.699+0000 7f8f3d1d5b80 1 -- start start 2024-05-09T02:20:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.699+0000 7f8f3d1d5b80 1 Processor -- start 2024-05-09T02:20:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.700+0000 7f8f3d1d5b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427ae474a0 0x56427afce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.700+0000 7f8f3d1d5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427af31160 0x56427afce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.700+0000 7f8f3d1d5b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427afc46a0 0x56427afcebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.700+0000 7f8f3d1d5b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56427ad62ba0 con 0x56427afc46a0 2024-05-09T02:20:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.700+0000 7f8f3d1d5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56427aea1150 con 0x56427af31160 2024-05-09T02:20:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.700+0000 7f8f3d1d5b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56427afc54e0 con 0x56427ae474a0 2024-05-09T02:20:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.700+0000 7f8f3d1d5b80 10 client.0 _ll_register_callbacks cb 0x56427ad586c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:20:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.700+0000 7f8f27fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427ae474a0 0x56427afce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.700+0000 7f8f277fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427af31160 0x56427afce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.700+0000 7f8f27fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427ae474a0 0x56427afce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:34080/0 (socket says 172.21.15.19:34080) 2024-05-09T02:20:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.700+0000 7f8f27fff700 1 -- 172.21.15.19:0/1497068183 learned_addr learned my addr 172.21.15.19:0/1497068183 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.700+0000 7f8f2cd5c700 1 --2- 172.21.15.19:0/1497068183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427afc46a0 0x56427afcebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.701+0000 7f8f2cd5c700 1 -- 172.21.15.19:0/1497068183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427ae474a0 msgr2=0x56427afce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.701+0000 7f8f2cd5c700 1 --2- 172.21.15.19:0/1497068183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427ae474a0 0x56427afce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.701+0000 7f8f2cd5c700 1 -- 172.21.15.19:0/1497068183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427af31160 msgr2=0x56427afce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.701+0000 7f8f2cd5c700 1 --2- 172.21.15.19:0/1497068183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427af31160 0x56427afce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.701+0000 7f8f2cd5c700 1 -- 172.21.15.19:0/1497068183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56427ad62490 con 0x56427afc46a0 2024-05-09T02:20:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.701+0000 7f8f2cd5c700 1 --2- 172.21.15.19:0/1497068183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427afc46a0 0x56427afcebf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8f2001ba10 tx=0x7f8f20004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98206a0c8fa574c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:09.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.701+0000 7f8f077fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:20:09.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.702+0000 7f8f077fe700 1 -- 172.21.15.19:0/1497068183 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20029070 con 0x56427afc46a0 2024-05-09T02:20:09.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.702+0000 7f8f077fe700 1 -- 172.21.15.19:0/1497068183 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8615+0+0 (secure 0 0 0) 0x7f8f20003c10 con 0x56427afc46a0 2024-05-09T02:20:09.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.702+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/1497068183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f2001be50 con 0x56427afc46a0 2024-05-09T02:20:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.702+0000 7f8f077fe700 1 -- 172.21.15.19:0/1497068183 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f8f2003e050 con 0x56427afc46a0 2024-05-09T02:20:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.702+0000 7f8f3d1d5b80 10 client.16407 fetch_fsmap learned FSMap version 54 2024-05-09T02:20:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.702+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/1497068183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56427afdbad0 con 0x56427afc46a0 2024-05-09T02:20:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.703+0000 7f8f077fe700 1 -- 172.21.15.19:0/1497068183 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f2001be50 con 0x56427afc46a0 2024-05-09T02:20:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.703+0000 7f8f077fe700 1 -- 172.21.15.19:0/1497068183 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f8f20030930 con 0x56427afc46a0 2024-05-09T02:20:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.703+0000 7f8f3d1d5b80 10 client.16407 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:20:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.703+0000 7f8f3d1d5b80 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:20:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.703+0000 7f8f3d1d5b80 20 client.16407 populate_metadata read hostname 'smithi019' 2024-05-09T02:20:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.703+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/1497068183 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56427afde440 0x56427afe08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.703+0000 7f8f3d1d5b80 4 client.16407 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:20:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.703+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/1497068183 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56427aef49d0 con 0x56427afde440 2024-05-09T02:20:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.704+0000 7f8f27fff700 1 --2- 172.21.15.19:0/1497068183 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56427afde440 0x56427afe08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.705+0000 7f8f077fe700 10 client.16407 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.705+0000 7f8f27fff700 1 --2- 172.21.15.19:0/1497068183 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56427afde440 0x56427afe08b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6630ca6401ad5df9 server_cookie=7d80a5371bbcf186 in_seq=0 out_seq=0 2024-05-09T02:20:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.705+0000 7f8f077fe700 1 -- 172.21.15.19:0/1497068183 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56427aef49d0 con 0x56427afde440 2024-05-09T02:20:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.705+0000 7f8f077fe700 10 client.16407 handle_command_reply: tid=0 2024-05-09T02:20:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.705+0000 7f8f3d1d5b80 1 client.16407 shutdown 2024-05-09T02:20:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.706+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/1497068183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427afc46a0 msgr2=0x56427afcebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.706+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/1497068183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427afc46a0 0x56427afcebf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8f2001ba10 tx=0x7f8f20004620 comp rx=0 tx=0).stop 2024-05-09T02:20:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.706+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/1497068183 shutdown_connections 2024-05-09T02:20:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.706+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/1497068183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427afc46a0 0x56427afcebf0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.706+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/1497068183 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56427afde440 0x56427afe08b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.706+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/1497068183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427ae474a0 0x56427afce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.706+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/1497068183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427af31160 0x56427afce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.706+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/1497068183 >> 172.21.15.19:0/1497068183 conn(0x56427aea76a0 msgr2=0x56427afbf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.707+0000 7f8f077fe700 0 client.16407 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.707+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/1497068183 shutdown_connections 2024-05-09T02:20:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.707+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/1497068183 wait complete. 2024-05-09T02:20:09.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.707+0000 7f8f3d1d5b80 20 client.16407 trim_cache size 0 max 16384 2024-05-09T02:20:09.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.738+0000 7f8f3d1d5b80 1 Processor -- start 2024-05-09T02:20:09.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.738+0000 7f8f3d1d5b80 1 -- start start 2024-05-09T02:20:09.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.738+0000 7f8f3d1d5b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427ae474a0 0x56427afc8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.738+0000 7f8f3d1d5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427af31160 0x56427afd9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.738+0000 7f8f3d1d5b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427afc46a0 0x56427afe60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.739+0000 7f8f3d1d5b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281b5580 con 0x56427ae474a0 2024-05-09T02:20:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.739+0000 7f8f3d1d5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281b7210 con 0x56427af31160 2024-05-09T02:20:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.739+0000 7f8f3d1d5b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281b5c50 con 0x56427afc46a0 2024-05-09T02:20:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.739+0000 7f8f277fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427af31160 0x56427afd9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.739+0000 7f8f27fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427ae474a0 0x56427afc8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.739+0000 7f8f277fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427af31160 0x56427afd9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:41888/0 (socket says 172.21.15.19:41888) 2024-05-09T02:20:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.739+0000 7f8f277fe700 1 -- 172.21.15.19:0/4167790697 learned_addr learned my addr 172.21.15.19:0/4167790697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.739+0000 7f8f2cd5c700 1 --2- 172.21.15.19:0/4167790697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427afc46a0 0x56427afe60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.739+0000 7f8f277fe700 1 -- 172.21.15.19:0/4167790697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427afc46a0 msgr2=0x56427afe60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.739+0000 7f8f277fe700 1 --2- 172.21.15.19:0/4167790697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427afc46a0 0x56427afe60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.739+0000 7f8f277fe700 1 -- 172.21.15.19:0/4167790697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427ae474a0 msgr2=0x56427afc8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.739+0000 7f8f277fe700 1 --2- 172.21.15.19:0/4167790697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427ae474a0 0x56427afc8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.739+0000 7f8f277fe700 1 -- 172.21.15.19:0/4167790697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56427afb8f60 con 0x56427af31160 2024-05-09T02:20:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.739+0000 7f8f277fe700 1 --2- 172.21.15.19:0/4167790697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427af31160 0x56427afd9f20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009990 tx=0x7f8f1c009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23733808b4fb339e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.740+0000 7f8f0dffb700 1 -- 172.21.15.19:0/4167790697 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0a98f0 con 0x56427af31160 2024-05-09T02:20:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.740+0000 7f8f0dffb700 1 -- 172.21.15.19:0/4167790697 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f1c0a9c90 con 0x56427af31160 2024-05-09T02:20:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.740+0000 7f8f0dffb700 1 -- 172.21.15.19:0/4167790697 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0a6410 con 0x56427af31160 2024-05-09T02:20:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.741+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/4167790697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427af31160 msgr2=0x56427afd9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.741+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/4167790697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427af31160 0x56427afd9f20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009990 tx=0x7f8f1c009710 comp rx=0 tx=0).stop 2024-05-09T02:20:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.741+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/4167790697 shutdown_connections 2024-05-09T02:20:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.741+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/4167790697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427afc46a0 0x56427afe60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.741+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/4167790697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427ae474a0 0x56427afc8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.741+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/4167790697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427af31160 0x56427afd9f20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.741+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/4167790697 >> 172.21.15.19:0/4167790697 conn(0x56427aea76a0 msgr2=0x56427afcca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.741+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/4167790697 shutdown_connections 2024-05-09T02:20:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.741+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/4167790697 wait complete. 2024-05-09T02:20:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.742+0000 7f8f3d1d5b80 1 -- start start 2024-05-09T02:20:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.742+0000 7f8f3d1d5b80 1 Processor -- start 2024-05-09T02:20:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.743+0000 7f8f3d1d5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427ae474a0 0x56427afd6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.743+0000 7f8f3d1d5b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427af31160 0x56427afe5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.743+0000 7f8f3d1d5b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427afc46a0 0x56427afe5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.743+0000 7f8f3d1d5b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281b6ab0 con 0x56427afc46a0 2024-05-09T02:20:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.743+0000 7f8f3d1d5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281b6320 con 0x56427ae474a0 2024-05-09T02:20:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.743+0000 7f8f3d1d5b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281b3540 con 0x56427af31160 2024-05-09T02:20:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.743+0000 7f8f3d1d5b80 10 client.0 _ll_register_callbacks cb 0x56427ad586c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:20:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.745+0000 7f8f2cd5c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427afc46a0 0x56427afe5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.745+0000 7f8f2cd5c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427afc46a0 0x56427afe5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33208/0 (socket says 172.21.15.19:33208) 2024-05-09T02:20:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.745+0000 7f8f2cd5c700 1 -- 172.21.15.19:0/772283201 learned_addr learned my addr 172.21.15.19:0/772283201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:09.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.745+0000 7f8f2cd5c700 1 -- 172.21.15.19:0/772283201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427af31160 msgr2=0x56427afe5680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:20:09.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.746+0000 7f8f2cd5c700 1 --2- 172.21.15.19:0/772283201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427af31160 0x56427afe5680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.746+0000 7f8f2cd5c700 1 -- 172.21.15.19:0/772283201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427ae474a0 msgr2=0x56427afd6760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:20:09.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.746+0000 7f8f2cd5c700 1 --2- 172.21.15.19:0/772283201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427ae474a0 0x56427afd6760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.746+0000 7f8f2cd5c700 1 -- 172.21.15.19:0/772283201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56427afe6230 con 0x56427afc46a0 2024-05-09T02:20:09.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.746+0000 7f8f2cd5c700 1 --2- 172.21.15.19:0/772283201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427afc46a0 0x56427afe5bc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8f20030cd0 tx=0x7f8f2000f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387b7f1f85c473a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.752+0000 7f8f077fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:20:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.752+0000 7f8f077fe700 1 -- 172.21.15.19:0/772283201 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20046020 con 0x56427afc46a0 2024-05-09T02:20:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.752+0000 7f8f077fe700 1 -- 172.21.15.19:0/772283201 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8615+0+0 (secure 0 0 0) 0x7f8f20005e60 con 0x56427afc46a0 2024-05-09T02:20:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.752+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/772283201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f1c0a9c90 con 0x56427afc46a0 2024-05-09T02:20:09.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.765+0000 7f8f077fe700 1 -- 172.21.15.19:0/772283201 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f8f20051050 con 0x56427afc46a0 2024-05-09T02:20:09.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.765+0000 7f8f3d1d5b80 10 client.16413 fetch_fsmap learned FSMap version 54 2024-05-09T02:20:09.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.765+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/772283201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56427b048a70 con 0x56427afc46a0 2024-05-09T02:20:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.774+0000 7f8f077fe700 1 -- 172.21.15.19:0/772283201 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f1c0a9c90 con 0x56427afc46a0 2024-05-09T02:20:09.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.775+0000 7f8f077fe700 1 -- 172.21.15.19:0/772283201 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f8f20030480 con 0x56427afc46a0 2024-05-09T02:20:09.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.775+0000 7f8f3d1d5b80 10 client.16413 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:20:09.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.775+0000 7f8f3d1d5b80 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:20:09.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.775+0000 7f8f3d1d5b80 20 client.16413 populate_metadata read hostname 'smithi019' 2024-05-09T02:20:09.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.775+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/772283201 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56427b049150 0x56427b04b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:09.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.775+0000 7f8f3d1d5b80 4 client.16413 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:20:09.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.776+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/772283201 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56427aeeea70 con 0x56427b049150 2024-05-09T02:20:09.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.777+0000 7f8f27fff700 1 --2- 172.21.15.19:0/772283201 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56427b049150 0x56427b04b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:09.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:09 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:20:09.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:09 smithi196 ceph-mon[34718]: pgmap v274: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 28 MiB/s wr, 1.51k op/s 2024-05-09T02:20:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.789+0000 7f8f27fff700 1 --2- 172.21.15.19:0/772283201 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56427b049150 0x56427b04b570 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd968f6d98ee3395 server_cookie=8c5999776b70de8d in_seq=0 out_seq=0 2024-05-09T02:20:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.789+0000 7f8f077fe700 10 client.16413 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:09.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.792+0000 7f8f077fe700 1 -- 172.21.15.19:0/772283201 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56427aeeea70 con 0x56427b049150 2024-05-09T02:20:09.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.792+0000 7f8f077fe700 10 client.16413 handle_command_reply: tid=0 2024-05-09T02:20:09.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.793+0000 7f8f3d1d5b80 1 client.16413 shutdown 2024-05-09T02:20:09.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.800+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/772283201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427afc46a0 msgr2=0x56427afe5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:09.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.800+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/772283201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427afc46a0 0x56427afe5bc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8f20030cd0 tx=0x7f8f2000f9d0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.801+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/772283201 shutdown_connections 2024-05-09T02:20:09.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.801+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/772283201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56427afc46a0 0x56427afe5bc0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.801+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/772283201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56427ae474a0 0x56427afd6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.801+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/772283201 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56427b049150 0x56427b04b570 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.801+0000 7f8f3d1d5b80 1 --2- 172.21.15.19:0/772283201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56427af31160 0x56427afe5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.801+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/772283201 >> 172.21.15.19:0/772283201 conn(0x56427aea76a0 msgr2=0x56427afc3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:09.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.801+0000 7f8f077fe700 0 client.16413 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:09.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.801+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/772283201 shutdown_connections 2024-05-09T02:20:09.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.801+0000 7f8f3d1d5b80 1 -- 172.21.15.19:0/772283201 wait complete. 2024-05-09T02:20:09.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.801+0000 7f8f3d1d5b80 20 client.16413 trim_cache size 0 max 16384 2024-05-09T02:20:09.805 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:20:09.805 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T02:20:09.805 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T02:20:09.805 INFO:teuthology.orchestra.run.smithi019.stdout: "8d1e7477-864a-4e74-b84e-33e45f653c8c": { 2024-05-09T02:20:09.805 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T02:20:09.805 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "8d1e7477-864a-4e74-b84e-33e45f653c8c", 2024-05-09T02:20:09.805 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T02:20:09.805 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:20:09.806 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:20:09.806 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:20:09.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.805+0000 7f8f2e7bf700 1 -- 172.21.15.19:0/12085254 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8f100b2a70 msgr2=0x7f8f100b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:09.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.805+0000 7f8f2e7bf700 1 --2- 172.21.15.19:0/12085254 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8f100b2a70 0x7f8f100b4f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000fd40 tx=0x7f8f2000f830 comp rx=0 tx=0).stop 2024-05-09T02:20:09.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.805+0000 7f8f2e7bf700 1 -- 172.21.15.19:0/12085254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f28107e40 msgr2=0x7f8f281a75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:09.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.805+0000 7f8f2e7bf700 1 --2- 172.21.15.19:0/12085254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f28107e40 0x7f8f281a75e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c006010 tx=0x7f8f1c0055f0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.806+0000 7f8f2e7bf700 1 -- 172.21.15.19:0/12085254 shutdown_connections 2024-05-09T02:20:09.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.806+0000 7f8f2e7bf700 1 --2- 172.21.15.19:0/12085254 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f8f100b2a70 0x7f8f100b4f80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.806+0000 7f8f2e7bf700 1 --2- 172.21.15.19:0/12085254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f28108790 0x7f8f281a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.806+0000 7f8f2e7bf700 1 --2- 172.21.15.19:0/12085254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f28103f40 0x7f8f281a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.806+0000 7f8f2e7bf700 1 --2- 172.21.15.19:0/12085254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f28107e40 0x7f8f281a75e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:09.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.806+0000 7f8f2e7bf700 1 -- 172.21.15.19:0/12085254 >> 172.21.15.19:0/12085254 conn(0x7f8f280fe0d0 msgr2=0x7f8f28100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:09.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.810+0000 7f8f2e7bf700 1 -- 172.21.15.19:0/12085254 shutdown_connections 2024-05-09T02:20:09.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:09.811+0000 7f8f2e7bf700 1 -- 172.21.15.19:0/12085254 wait complete. 2024-05-09T02:20:09.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:09 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:20:09.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:09 smithi059 ceph-mon[34781]: pgmap v274: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 28 MiB/s wr, 1.51k op/s 2024-05-09T02:20:10.538 INFO:tasks.cephfs.filesystem:scrub status for tag:8d1e7477-864a-4e74-b84e-33e45f653c8c - {'path': '/', 'tag': '8d1e7477-864a-4e74-b84e-33e45f653c8c', 'options': 'recursive,force'} 2024-05-09T02:20:11.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:10 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:20:11.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:10 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:20:11.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:10 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:20:12.216 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:11 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:20:12.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:11 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:20:12.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:11 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:20:12.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:11 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:20:12.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:11 smithi059 ceph-mon[34781]: pgmap v275: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 33 MiB/s wr, 1.70k op/s 2024-05-09T02:20:12.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:11 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:20:12.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:11 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:20:12.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:11 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:20:12.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:11 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:20:12.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:11 smithi196 ceph-mon[34718]: pgmap v275: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 33 MiB/s wr, 1.70k op/s 2024-05-09T02:20:12.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:11 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:20:12.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:11 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:20:12.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:11 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:20:12.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:11 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:20:12.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:11 smithi019 ceph-mon[30151]: pgmap v275: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 33 MiB/s wr, 1.70k op/s 2024-05-09T02:20:12.656 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 510.0 s, 1936.0 tps, lat 54.333 ms stddev 76.559 2024-05-09T02:20:14.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:14 smithi059 ceph-mon[34781]: pgmap v276: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 33 MiB/s wr, 1.70k op/s 2024-05-09T02:20:14.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:14 smithi196 ceph-mon[34718]: pgmap v276: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 33 MiB/s wr, 1.70k op/s 2024-05-09T02:20:14.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:14 smithi019 ceph-mon[30151]: pgmap v276: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 33 MiB/s wr, 1.70k op/s 2024-05-09T02:20:15.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:15 smithi196 ceph-mon[34718]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T02:20:15.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:15 smithi019 ceph-mon[30151]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T02:20:15.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:15 smithi059 ceph-mon[34781]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T02:20:16.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:16 smithi196 ceph-mon[34718]: pgmap v278: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 MiB/s wr, 1.99k op/s 2024-05-09T02:20:16.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:16 smithi019 ceph-mon[30151]: pgmap v278: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 MiB/s wr, 1.99k op/s 2024-05-09T02:20:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:16 smithi059 ceph-mon[34781]: pgmap v278: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 MiB/s wr, 1.99k op/s 2024-05-09T02:20:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 515.0 s, 1870.2 tps, lat 53.126 ms stddev 61.229 2024-05-09T02:20:18.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:18 smithi196 ceph-mon[34718]: pgmap v279: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 MiB/s wr, 1.75k op/s 2024-05-09T02:20:18.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:18 smithi019 ceph-mon[30151]: pgmap v279: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 MiB/s wr, 1.75k op/s 2024-05-09T02:20:18.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:18 smithi059 ceph-mon[34781]: pgmap v279: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 MiB/s wr, 1.75k op/s 2024-05-09T02:20:20.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:20 smithi196 ceph-mon[34718]: pgmap v280: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 877 op/s 2024-05-09T02:20:20.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:20 smithi019 ceph-mon[30151]: pgmap v280: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 877 op/s 2024-05-09T02:20:20.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:20 smithi059 ceph-mon[34781]: pgmap v280: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 877 op/s 2024-05-09T02:20:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 520.0 s, 2218.0 tps, lat 45.111 ms stddev 56.276 2024-05-09T02:20:22.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:22 smithi196 ceph-mon[34718]: pgmap v281: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 661 op/s 2024-05-09T02:20:22.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:22 smithi019 ceph-mon[30151]: pgmap v281: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 661 op/s 2024-05-09T02:20:22.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:22 smithi059 ceph-mon[34781]: pgmap v281: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 661 op/s 2024-05-09T02:20:23.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:23 smithi019 ceph-mon[30151]: pgmap v282: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 661 op/s 2024-05-09T02:20:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:23 smithi059 ceph-mon[34781]: pgmap v282: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 661 op/s 2024-05-09T02:20:24.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:23 smithi196 ceph-mon[34718]: pgmap v282: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 661 op/s 2024-05-09T02:20:26.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:26 smithi059 ceph-mon[34781]: pgmap v283: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.2 MiB/s wr, 668 op/s 2024-05-09T02:20:26.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:26 smithi196 ceph-mon[34718]: pgmap v283: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.2 MiB/s wr, 668 op/s 2024-05-09T02:20:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:26 smithi019 ceph-mon[30151]: pgmap v283: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.2 MiB/s wr, 668 op/s 2024-05-09T02:20:27.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 525.0 s, 1831.4 tps, lat 54.707 ms stddev 63.565 2024-05-09T02:20:28.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:28 smithi059 ceph-mon[34781]: pgmap v284: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.5 MiB/s wr, 602 op/s 2024-05-09T02:20:28.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:28 smithi196 ceph-mon[34718]: pgmap v284: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.5 MiB/s wr, 602 op/s 2024-05-09T02:20:28.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:28 smithi019 ceph-mon[30151]: pgmap v284: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.5 MiB/s wr, 602 op/s 2024-05-09T02:20:30.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:30 smithi059 ceph-mon[34781]: pgmap v285: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.3 MiB/s wr, 798 op/s 2024-05-09T02:20:30.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:30 smithi196 ceph-mon[34718]: pgmap v285: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.3 MiB/s wr, 798 op/s 2024-05-09T02:20:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:30 smithi019 ceph-mon[30151]: pgmap v285: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.3 MiB/s wr, 798 op/s 2024-05-09T02:20:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 530.0 s, 1554.2 tps, lat 64.149 ms stddev 68.963 2024-05-09T02:20:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:32 smithi059 ceph-mon[34781]: pgmap v286: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.3 MiB/s wr, 618 op/s 2024-05-09T02:20:32.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:32 smithi196 ceph-mon[34718]: pgmap v286: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.3 MiB/s wr, 618 op/s 2024-05-09T02:20:32.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:32 smithi019 ceph-mon[30151]: pgmap v286: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.3 MiB/s wr, 618 op/s 2024-05-09T02:20:34.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:34 smithi019 ceph-mon[30151]: pgmap v287: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.1 MiB/s wr, 588 op/s 2024-05-09T02:20:34.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:34 smithi059 ceph-mon[34781]: pgmap v287: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.1 MiB/s wr, 588 op/s 2024-05-09T02:20:34.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:34 smithi196 ceph-mon[34718]: pgmap v287: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.1 MiB/s wr, 588 op/s 2024-05-09T02:20:36.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:36 smithi059 ceph-mon[34781]: pgmap v288: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.8 MiB/s wr, 875 op/s 2024-05-09T02:20:36.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:36 smithi196 ceph-mon[34718]: pgmap v288: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.8 MiB/s wr, 875 op/s 2024-05-09T02:20:36.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:36 smithi019 ceph-mon[30151]: pgmap v288: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.8 MiB/s wr, 875 op/s 2024-05-09T02:20:37.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 535.0 s, 1294.2 tps, lat 76.508 ms stddev 151.325 2024-05-09T02:20:38.722 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:38 smithi059 ceph-mon[34781]: pgmap v289: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.2 MiB/s wr, 554 op/s 2024-05-09T02:20:38.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:38 smithi196 ceph-mon[34718]: pgmap v289: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.2 MiB/s wr, 554 op/s 2024-05-09T02:20:38.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:38 smithi019 ceph-mon[30151]: pgmap v289: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.2 MiB/s wr, 554 op/s 2024-05-09T02:20:40.540 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:20:40.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:40 smithi059 ceph-mon[34781]: pgmap v290: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 1.63k op/s 2024-05-09T02:20:40.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:40 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:20:40.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:40 smithi196 ceph-mon[34718]: pgmap v290: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 1.63k op/s 2024-05-09T02:20:40.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:40 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:20:40.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:40 smithi019 ceph-mon[30151]: pgmap v290: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 1.63k op/s 2024-05-09T02:20:40.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:40 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:20:40.938 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:20:42.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:42 smithi019 ceph-mon[30151]: pgmap v291: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-05-09T02:20:42.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:42 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:20:42.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:42 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:20:42.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:42 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:20:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 540.0 s, 2005.2 tps, lat 50.513 ms stddev 57.838 2024-05-09T02:20:42.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:42 smithi059 ceph-mon[34781]: pgmap v291: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-05-09T02:20:42.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:42 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:20:42.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:42 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:20:42.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:42 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:20:42.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:42 smithi196 ceph-mon[34718]: pgmap v291: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-05-09T02:20:42.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:42 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:20:42.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:42 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:20:42.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:42 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:20:42.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.853+0000 7f63b9535700 1 -- 172.21.15.19:0/2165923707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b41034c0 msgr2=0x7f63b41038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:42.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.853+0000 7f63b9535700 1 --2- 172.21.15.19:0/2165923707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b41034c0 0x7f63b41038d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f63a40099c0 tx=0x7f63a4009cd0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.854+0000 7f63b9535700 1 -- 172.21.15.19:0/2165923707 shutdown_connections 2024-05-09T02:20:42.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.854+0000 7f63b9535700 1 --2- 172.21.15.19:0/2165923707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b4108740 0x7f63b41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.854+0000 7f63b9535700 1 --2- 172.21.15.19:0/2165923707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b41034c0 0x7f63b41038d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.854+0000 7f63b9535700 1 --2- 172.21.15.19:0/2165923707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63b4107df0 0x7f63b4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.854+0000 7f63b9535700 1 -- 172.21.15.19:0/2165923707 >> 172.21.15.19:0/2165923707 conn(0x7f63b40fedc0 msgr2=0x7f63b41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:42.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.854+0000 7f63b9535700 1 -- 172.21.15.19:0/2165923707 shutdown_connections 2024-05-09T02:20:42.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.854+0000 7f63b9535700 1 -- 172.21.15.19:0/2165923707 wait complete. 2024-05-09T02:20:42.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.855+0000 7f63b9535700 1 Processor -- start 2024-05-09T02:20:42.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.855+0000 7f63b9535700 1 -- start start 2024-05-09T02:20:42.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b9535700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63b41034c0 0x7f63b419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b9535700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b4107df0 0x7f63b419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b9535700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b4108740 0x7f63b4198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b9535700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63b41122d0 con 0x7f63b4107df0 2024-05-09T02:20:42.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b9535700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63b4112150 con 0x7f63b41034c0 2024-05-09T02:20:42.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b9535700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63b4112450 con 0x7f63b4108740 2024-05-09T02:20:42.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b37fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b4108740 0x7f63b4198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b37fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b4108740 0x7f63b4198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:35358/0 (socket says 172.21.15.19:35358) 2024-05-09T02:20:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b37fe700 1 -- 172.21.15.19:0/203031660 learned_addr learned my addr 172.21.15.19:0/203031660 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b2ffd700 1 --2- 172.21.15.19:0/203031660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63b41034c0 0x7f63b419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b27fc700 1 --2- 172.21.15.19:0/203031660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b4107df0 0x7f63b419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b37fe700 1 -- 172.21.15.19:0/203031660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63b41034c0 msgr2=0x7f63b419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b37fe700 1 --2- 172.21.15.19:0/203031660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63b41034c0 0x7f63b419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b37fe700 1 -- 172.21.15.19:0/203031660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b4107df0 msgr2=0x7f63b419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b37fe700 1 --2- 172.21.15.19:0/203031660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b4107df0 0x7f63b419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.856+0000 7f63b37fe700 1 -- 172.21.15.19:0/203031660 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b41990e0 con 0x7f63b4108740 2024-05-09T02:20:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.857+0000 7f63b37fe700 1 --2- 172.21.15.19:0/203031660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b4108740 0x7f63b4198a30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00ef30 tx=0x7f63ac00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.857+0000 7f639bfff700 1 -- 172.21.15.19:0/203031660 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac0092e0 con 0x7f63b4108740 2024-05-09T02:20:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.857+0000 7f63b9535700 1 -- 172.21.15.19:0/203031660 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b4199370 con 0x7f63b4108740 2024-05-09T02:20:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.857+0000 7f63b9535700 1 -- 172.21.15.19:0/203031660 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b41a56a0 con 0x7f63b4108740 2024-05-09T02:20:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.857+0000 7f639bfff700 1 -- 172.21.15.19:0/203031660 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63ac004bb0 con 0x7f63b4108740 2024-05-09T02:20:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.857+0000 7f639bfff700 1 -- 172.21.15.19:0/203031660 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac004020 con 0x7f63b4108740 2024-05-09T02:20:42.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.861+0000 7f639bfff700 1 -- 172.21.15.19:0/203031660 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f63ac01b030 con 0x7f63b4108740 2024-05-09T02:20:42.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.862+0000 7f639bfff700 1 --2- 172.21.15.19:0/203031660 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f639c0b2a70 0x7f639c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.862+0000 7f639bfff700 1 -- 172.21.15.19:0/203031660 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f63ac0e2080 con 0x7f63b4108740 2024-05-09T02:20:42.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.862+0000 7f63b2ffd700 1 --2- 172.21.15.19:0/203031660 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f639c0b2a70 0x7f639c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.863+0000 7f63c7f4bb80 1 Processor -- start 2024-05-09T02:20:42.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.863+0000 7f63c7f4bb80 1 -- start start 2024-05-09T02:20:42.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.863+0000 7f63b2ffd700 1 --2- 172.21.15.19:0/203031660 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f639c0b2a70 0x7f639c0b4f80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009990 tx=0x7f63a4005fb0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:42.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.863+0000 7f63c7f4bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d480814160 0x55d4808ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.863+0000 7f63c7f4bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d48072a4a0 0x55d4808ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.863+0000 7f63c7f4bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4808a76a0 0x55d4808a7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.863+0000 7f63c7f4bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d4807c9ff0 con 0x55d48072a4a0 2024-05-09T02:20:42.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.863+0000 7f63c7f4bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d4807687f0 con 0x55d480814160 2024-05-09T02:20:42.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.863+0000 7f63c7f4bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d4807c5fb0 con 0x55d4808a76a0 2024-05-09T02:20:42.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.863+0000 7f63b37fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4808a76a0 0x55d4808a7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.863+0000 7f63b27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d48072a4a0 0x55d4808ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.863+0000 7f63b37fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4808a76a0 0x55d4808a7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:35370/0 (socket says 172.21.15.19:35370) 2024-05-09T02:20:42.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.863+0000 7f63b27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d48072a4a0 0x55d4808ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48898/0 (socket says 172.21.15.19:48898) 2024-05-09T02:20:42.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.863+0000 7f63b37fe700 1 -- 172.21.15.19:0/3728634090 learned_addr learned my addr 172.21.15.19:0/3728634090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:42.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.863+0000 7f63b2ffd700 1 --2- 172.21.15.19:0/3728634090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d480814160 0x55d4808ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.864+0000 7f63b37fe700 1 -- 172.21.15.19:0/3728634090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d480814160 msgr2=0x55d4808ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:42.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.864+0000 7f63b37fe700 1 --2- 172.21.15.19:0/3728634090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d480814160 0x55d4808ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.864+0000 7f63b37fe700 1 -- 172.21.15.19:0/3728634090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d48072a4a0 msgr2=0x55d4808ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:42.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.864+0000 7f63b37fe700 1 --2- 172.21.15.19:0/3728634090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d48072a4a0 0x55d4808ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.864+0000 7f63b37fe700 1 -- 172.21.15.19:0/3728634090 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d480760370 con 0x55d4808a76a0 2024-05-09T02:20:42.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.864+0000 7f63b37fe700 1 --2- 172.21.15.19:0/3728634090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4808a76a0 0x55d4808a7ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0a53e0 tx=0x7f63ac0a8f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58556771bca09e29 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:42.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.864+0000 7f63997fa700 1 -- 172.21.15.19:0/3728634090 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac0ad710 con 0x55d4808a76a0 2024-05-09T02:20:42.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.864+0000 7f63997fa700 1 -- 172.21.15.19:0/3728634090 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63ac0b2130 con 0x55d4808a76a0 2024-05-09T02:20:42.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.864+0000 7f63997fa700 1 -- 172.21.15.19:0/3728634090 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac003c60 con 0x55d4808a76a0 2024-05-09T02:20:42.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.865+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/3728634090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4808a76a0 msgr2=0x55d4808a7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:42.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.865+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/3728634090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4808a76a0 0x55d4808a7ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0a53e0 tx=0x7f63ac0a8f40 comp rx=0 tx=0).stop 2024-05-09T02:20:42.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.866+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/3728634090 shutdown_connections 2024-05-09T02:20:42.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.866+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/3728634090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4808a76a0 0x55d4808a7ad0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.866+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/3728634090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d480814160 0x55d4808ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.866+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/3728634090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d48072a4a0 0x55d4808ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.866+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/3728634090 >> 172.21.15.19:0/3728634090 conn(0x55d48078a6a0 msgr2=0x55d4808a2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:42.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.867+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/3728634090 shutdown_connections 2024-05-09T02:20:42.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.867+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/3728634090 wait complete. 2024-05-09T02:20:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.867+0000 7f63c7f4bb80 1 -- start start 2024-05-09T02:20:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.867+0000 7f63c7f4bb80 1 Processor -- start 2024-05-09T02:20:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63c7f4bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d48072a4a0 0x55d4808b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63c7f4bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d480814160 0x55d4808b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63c7f4bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d4808a76a0 0x55d4808b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63c7f4bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d480645ba0 con 0x55d480814160 2024-05-09T02:20:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63c7f4bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d480784150 con 0x55d4808a76a0 2024-05-09T02:20:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63c7f4bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d4808b41d0 con 0x55d48072a4a0 2024-05-09T02:20:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63b2ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d48072a4a0 0x55d4808b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63c7f4bb80 10 client.0 _ll_register_callbacks cb 0x55d48063b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:20:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63b2ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d48072a4a0 0x55d4808b12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:35376/0 (socket says 172.21.15.19:35376) 2024-05-09T02:20:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63b2ffd700 1 -- 172.21.15.19:0/1358492287 learned_addr learned my addr 172.21.15.19:0/1358492287 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63b27fc700 1 --2- 172.21.15.19:0/1358492287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d480814160 0x55d4808b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63b37fe700 1 --2- 172.21.15.19:0/1358492287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d4808a76a0 0x55d4808b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63b2ffd700 1 -- 172.21.15.19:0/1358492287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d4808a76a0 msgr2=0x55d4808b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63b2ffd700 1 --2- 172.21.15.19:0/1358492287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d4808a76a0 0x55d4808b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63b2ffd700 1 -- 172.21.15.19:0/1358492287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d480814160 msgr2=0x55d4808b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63b2ffd700 1 --2- 172.21.15.19:0/1358492287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d480814160 0x55d4808b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.868+0000 7f63b2ffd700 1 -- 172.21.15.19:0/1358492287 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d480645490 con 0x55d48072a4a0 2024-05-09T02:20:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.869+0000 7f6396ff5700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:20:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.869+0000 7f63b2ffd700 1 --2- 172.21.15.19:0/1358492287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d48072a4a0 0x55d4808b12a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f63a40046c0 tx=0x7f63a4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be458c8697068463 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.869+0000 7f6396ff5700 1 -- 172.21.15.19:0/1358492287 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4030060 con 0x55d48072a4a0 2024-05-09T02:20:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.869+0000 7f6396ff5700 1 -- 172.21.15.19:0/1358492287 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f63a40049f0 con 0x55d48072a4a0 2024-05-09T02:20:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.869+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/1358492287 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63ac0b2130 con 0x55d48072a4a0 2024-05-09T02:20:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.870+0000 7f6396ff5700 1 -- 172.21.15.19:0/1358492287 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f63a401a040 con 0x55d48072a4a0 2024-05-09T02:20:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.870+0000 7f63c7f4bb80 10 client.26191 fetch_fsmap learned FSMap version 54 2024-05-09T02:20:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.870+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/1358492287 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d4808c7270 con 0x55d48072a4a0 2024-05-09T02:20:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.870+0000 7f6396ff5700 1 -- 172.21.15.19:0/1358492287 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63ac0b2130 con 0x55d48072a4a0 2024-05-09T02:20:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.870+0000 7f6396ff5700 1 -- 172.21.15.19:0/1358492287 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f63a4003dc0 con 0x55d48072a4a0 2024-05-09T02:20:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.870+0000 7f63c7f4bb80 10 client.26191 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:20:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.870+0000 7f63c7f4bb80 10 client.26191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:20:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.870+0000 7f63c7f4bb80 20 client.26191 populate_metadata read hostname 'smithi019' 2024-05-09T02:20:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.871+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/1358492287 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d4808c97e0 0x55d4808cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.871+0000 7f63c7f4bb80 4 client.26191 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:20:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.871+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/1358492287 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4807d79d0 con 0x55d4808c97e0 2024-05-09T02:20:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.871+0000 7f63b27fc700 1 --2- 172.21.15.19:0/1358492287 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d4808c97e0 0x55d4808cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.872+0000 7f63b27fc700 1 --2- 172.21.15.19:0/1358492287 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d4808c97e0 0x55d4808cbc30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=991fdedb1242e4ea server_cookie=a987fa7e167f9d82 in_seq=0 out_seq=0 2024-05-09T02:20:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.872+0000 7f6396ff5700 10 client.26191 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.872+0000 7f6396ff5700 1 -- 172.21.15.19:0/1358492287 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d4807d79d0 con 0x55d4808c97e0 2024-05-09T02:20:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.872+0000 7f6396ff5700 10 client.26191 handle_command_reply: tid=0 2024-05-09T02:20:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.872+0000 7f63c7f4bb80 1 client.26191 shutdown 2024-05-09T02:20:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.873+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/1358492287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d48072a4a0 msgr2=0x55d4808b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.873+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/1358492287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d48072a4a0 0x55d4808b12a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f63a40046c0 tx=0x7f63a4009310 comp rx=0 tx=0).stop 2024-05-09T02:20:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.873+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/1358492287 shutdown_connections 2024-05-09T02:20:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.873+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/1358492287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d4808a76a0 0x55d4808b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.873+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/1358492287 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d4808c97e0 0x55d4808cbc30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.873+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/1358492287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d48072a4a0 0x55d4808b12a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.873+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/1358492287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d480814160 0x55d4808b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.873+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/1358492287 >> 172.21.15.19:0/1358492287 conn(0x55d48078a6a0 msgr2=0x55d4808a21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.873+0000 7f6396ff5700 0 client.26191 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.873+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/1358492287 shutdown_connections 2024-05-09T02:20:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.873+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/1358492287 wait complete. 2024-05-09T02:20:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.873+0000 7f63c7f4bb80 20 client.26191 trim_cache size 0 max 16384 2024-05-09T02:20:42.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.912+0000 7f63c7f4bb80 1 Processor -- start 2024-05-09T02:20:42.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.912+0000 7f63c7f4bb80 1 -- start start 2024-05-09T02:20:42.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.913+0000 7f63c7f4bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d48072a4a0 0x55d4808aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.913+0000 7f63c7f4bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d480814160 0x55d4808a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.913+0000 7f63c7f4bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d4808a76a0 0x55d4808b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.913+0000 7f63c7f4bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63b41acd60 con 0x55d48072a4a0 2024-05-09T02:20:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.913+0000 7f63c7f4bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63b41ae9f0 con 0x55d4808a76a0 2024-05-09T02:20:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.913+0000 7f63c7f4bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63b41ad430 con 0x55d480814160 2024-05-09T02:20:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.914+0000 7f63b37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d4808a76a0 0x55d4808b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.914+0000 7f63b37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d4808a76a0 0x55d4808b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:58546/0 (socket says 172.21.15.19:58546) 2024-05-09T02:20:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.914+0000 7f63b37fe700 1 -- 172.21.15.19:0/1994723034 learned_addr learned my addr 172.21.15.19:0/1994723034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.914+0000 7f63b2ffd700 1 --2- 172.21.15.19:0/1994723034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d48072a4a0 0x55d4808aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.914+0000 7f63b27fc700 1 --2- 172.21.15.19:0/1994723034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d480814160 0x55d4808a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.914+0000 7f63b37fe700 1 -- 172.21.15.19:0/1994723034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d480814160 msgr2=0x55d4808a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.914+0000 7f63b37fe700 1 --2- 172.21.15.19:0/1994723034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d480814160 0x55d4808a70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.914+0000 7f63b37fe700 1 -- 172.21.15.19:0/1994723034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d48072a4a0 msgr2=0x55d4808aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.914+0000 7f63b37fe700 1 --2- 172.21.15.19:0/1994723034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d48072a4a0 0x55d4808aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.914+0000 7f63b37fe700 1 -- 172.21.15.19:0/1994723034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4808a3d00 con 0x55d4808a76a0 2024-05-09T02:20:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.915+0000 7f63b37fe700 1 --2- 172.21.15.19:0/1994723034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d4808a76a0 0x55d4808b1b60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00ef00 tx=0x7f63ac009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e41b2a43d67e7e20 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:42.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.915+0000 7f6398ff9700 1 -- 172.21.15.19:0/1994723034 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac0ad710 con 0x55d4808a76a0 2024-05-09T02:20:42.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.915+0000 7f6398ff9700 1 -- 172.21.15.19:0/1994723034 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63ac0b1eb0 con 0x55d4808a76a0 2024-05-09T02:20:42.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.915+0000 7f6398ff9700 1 -- 172.21.15.19:0/1994723034 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac0a4cd0 con 0x55d4808a76a0 2024-05-09T02:20:42.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.916+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/1994723034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d4808a76a0 msgr2=0x55d4808b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:42.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.916+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/1994723034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d4808a76a0 0x55d4808b1b60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00ef00 tx=0x7f63ac009ed0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.917+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/1994723034 shutdown_connections 2024-05-09T02:20:42.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.917+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/1994723034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d4808a76a0 0x55d4808b1b60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.917+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/1994723034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d48072a4a0 0x55d4808aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.917+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/1994723034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d480814160 0x55d4808a70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.917+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/1994723034 >> 172.21.15.19:0/1994723034 conn(0x55d48078a6a0 msgr2=0x55d4808aae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:42.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.918+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/1994723034 shutdown_connections 2024-05-09T02:20:42.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.918+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/1994723034 wait complete. 2024-05-09T02:20:42.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.919+0000 7f63c7f4bb80 1 -- start start 2024-05-09T02:20:42.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.919+0000 7f63c7f4bb80 1 Processor -- start 2024-05-09T02:20:42.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.920+0000 7f63c7f4bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d48072a4a0 0x55d4808c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.920+0000 7f63c7f4bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d480814160 0x55d4808c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.920+0000 7f63b2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d48072a4a0 0x55d4808c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.920+0000 7f63b2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d48072a4a0 0x55d4808c5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:58550/0 (socket says 172.21.15.19:58550) 2024-05-09T02:20:42.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.920+0000 7f63c7f4bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4808a76a0 0x55d4808c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.920+0000 7f63b27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d480814160 0x55d4808c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.920+0000 7f63b2ffd700 1 -- 172.21.15.19:0/2709678309 learned_addr learned my addr 172.21.15.19:0/2709678309 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.920+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/2709678309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63b41ae290 con 0x55d480814160 2024-05-09T02:20:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.920+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/2709678309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63b41adb00 con 0x55d48072a4a0 2024-05-09T02:20:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.920+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/2709678309 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63b41aad20 con 0x55d4808a76a0 2024-05-09T02:20:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.921+0000 7f63b37fe700 1 --2- 172.21.15.19:0/2709678309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4808a76a0 0x55d4808c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.921+0000 7f63c7f4bb80 10 client.0 _ll_register_callbacks cb 0x55d48063b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:20:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.921+0000 7f63b2ffd700 1 -- 172.21.15.19:0/2709678309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4808a76a0 msgr2=0x55d4808c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.921+0000 7f63b2ffd700 1 --2- 172.21.15.19:0/2709678309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4808a76a0 0x55d4808c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.921+0000 7f63b2ffd700 1 -- 172.21.15.19:0/2709678309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d480814160 msgr2=0x55d4808c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.921+0000 7f63b2ffd700 1 --2- 172.21.15.19:0/2709678309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d480814160 0x55d4808c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.921+0000 7f63b2ffd700 1 -- 172.21.15.19:0/2709678309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d48089bf60 con 0x55d48072a4a0 2024-05-09T02:20:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.921+0000 7f63b2ffd700 1 --2- 172.21.15.19:0/2709678309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d48072a4a0 0x55d4808c5060 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009960 tx=0x7f63a4009660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec3286122139bb9e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.921+0000 7f6396ff5700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:20:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.921+0000 7f6396ff5700 1 -- 172.21.15.19:0/2709678309 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a40048f0 con 0x55d48072a4a0 2024-05-09T02:20:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.922+0000 7f6396ff5700 1 -- 172.21.15.19:0/2709678309 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f63a4004a90 con 0x55d48072a4a0 2024-05-09T02:20:42.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.922+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/2709678309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63ac0b1eb0 con 0x55d48072a4a0 2024-05-09T02:20:42.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.922+0000 7f6396ff5700 1 -- 172.21.15.19:0/2709678309 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f63a4004450 con 0x55d48072a4a0 2024-05-09T02:20:42.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.922+0000 7f63c7f4bb80 10 client.26168 fetch_fsmap learned FSMap version 54 2024-05-09T02:20:42.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.922+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/2709678309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d480933f60 con 0x55d48072a4a0 2024-05-09T02:20:42.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.922+0000 7f6396ff5700 1 -- 172.21.15.19:0/2709678309 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63ac0b1eb0 con 0x55d48072a4a0 2024-05-09T02:20:42.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.923+0000 7f6396ff5700 1 -- 172.21.15.19:0/2709678309 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f63a402cc00 con 0x55d48072a4a0 2024-05-09T02:20:42.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.923+0000 7f63c7f4bb80 10 client.26168 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:20:42.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.923+0000 7f63c7f4bb80 10 client.26168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:20:42.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.923+0000 7f63c7f4bb80 20 client.26168 populate_metadata read hostname 'smithi019' 2024-05-09T02:20:42.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.923+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/2709678309 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d480934600 0x55d480936a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:42.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.923+0000 7f63c7f4bb80 4 client.26168 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:20:42.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.923+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/2709678309 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d4807d1a70 con 0x55d480934600 2024-05-09T02:20:42.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.924+0000 7f63b27fc700 1 --2- 172.21.15.19:0/2709678309 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d480934600 0x55d480936a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.924+0000 7f63b27fc700 1 --2- 172.21.15.19:0/2709678309 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d480934600 0x55d480936a20 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea828a121666e1e3 server_cookie=2399f16991096cd0 in_seq=0 out_seq=0 2024-05-09T02:20:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.924+0000 7f6396ff5700 10 client.26168 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.925+0000 7f6396ff5700 1 -- 172.21.15.19:0/2709678309 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d4807d1a70 con 0x55d480934600 2024-05-09T02:20:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.925+0000 7f6396ff5700 10 client.26168 handle_command_reply: tid=0 2024-05-09T02:20:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.925+0000 7f63c7f4bb80 1 client.26168 shutdown 2024-05-09T02:20:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.925+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/2709678309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d48072a4a0 msgr2=0x55d4808c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.925+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/2709678309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d48072a4a0 0x55d4808c5060 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009960 tx=0x7f63a4009660 comp rx=0 tx=0).stop 2024-05-09T02:20:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.925+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/2709678309 shutdown_connections 2024-05-09T02:20:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.925+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/2709678309 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d480934600 0x55d480936a20 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.925+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/2709678309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4808a76a0 0x55d4808c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.925+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/2709678309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d48072a4a0 0x55d4808c5060 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.925+0000 7f63c7f4bb80 1 --2- 172.21.15.19:0/2709678309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d480814160 0x55d4808c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.925+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/2709678309 >> 172.21.15.19:0/2709678309 conn(0x55d48078a6a0 msgr2=0x55d4808a6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.925+0000 7f6396ff5700 0 client.26168 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:20:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.925+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/2709678309 shutdown_connections 2024-05-09T02:20:42.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.925+0000 7f63c7f4bb80 1 -- 172.21.15.19:0/2709678309 wait complete. 2024-05-09T02:20:42.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.926+0000 7f63c7f4bb80 20 client.26168 trim_cache size 0 max 16384 2024-05-09T02:20:42.932 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:20:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T02:20:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T02:20:42.932 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:20:42.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.927+0000 7f63b9535700 1 -- 172.21.15.19:0/203031660 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f639c0b2a70 msgr2=0x7f639c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:42.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.927+0000 7f63b9535700 1 --2- 172.21.15.19:0/203031660 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f639c0b2a70 0x7f639c0b4f80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009990 tx=0x7f63a4005fb0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.927+0000 7f63b9535700 1 -- 172.21.15.19:0/203031660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b4108740 msgr2=0x7f63b4198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:42.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.927+0000 7f63b9535700 1 --2- 172.21.15.19:0/203031660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b4108740 0x7f63b4198a30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00ef30 tx=0x7f63ac00c3a0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.927+0000 7f63b9535700 1 -- 172.21.15.19:0/203031660 shutdown_connections 2024-05-09T02:20:42.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.927+0000 7f63b9535700 1 --2- 172.21.15.19:0/203031660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b4108740 0x7f63b4198a30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.927+0000 7f63b9535700 1 --2- 172.21.15.19:0/203031660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63b41034c0 0x7f63b419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.927+0000 7f63b9535700 1 --2- 172.21.15.19:0/203031660 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f639c0b2a70 0x7f639c0b4f80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.927+0000 7f63b9535700 1 --2- 172.21.15.19:0/203031660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b4107df0 0x7f63b419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:42.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.927+0000 7f63b9535700 1 -- 172.21.15.19:0/203031660 >> 172.21.15.19:0/203031660 conn(0x7f63b40fedc0 msgr2=0x7f63b41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:42.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.928+0000 7f63b9535700 1 -- 172.21.15.19:0/203031660 shutdown_connections 2024-05-09T02:20:42.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:42.928+0000 7f63b9535700 1 -- 172.21.15.19:0/203031660 wait complete. 2024-05-09T02:20:43.731 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T02:20:43.731 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:20:44.155 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:20:44.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:44 smithi019 ceph-mon[30151]: pgmap v292: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 20 MiB/s wr, 1.60k op/s 2024-05-09T02:20:44.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:44 smithi059 ceph-mon[34781]: pgmap v292: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 20 MiB/s wr, 1.60k op/s 2024-05-09T02:20:44.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:44 smithi196 ceph-mon[34718]: pgmap v292: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 20 MiB/s wr, 1.60k op/s 2024-05-09T02:20:46.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:46 smithi059 ceph-mon[34781]: pgmap v293: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 23 MiB/s wr, 1.86k op/s 2024-05-09T02:20:46.751 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:46 smithi019 ceph-mon[30151]: pgmap v293: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 23 MiB/s wr, 1.86k op/s 2024-05-09T02:20:46.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:46 smithi196 ceph-mon[34718]: pgmap v293: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 23 MiB/s wr, 1.86k op/s 2024-05-09T02:20:47.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.169+0000 7fc16a7fc700 1 -- 172.21.15.19:0/1516655100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160003b40 con 0x7fc16c1102a0 2024-05-09T02:20:47.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.170+0000 7fc171eb4700 1 -- 172.21.15.19:0/1516655100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c1102a0 msgr2=0x7fc16c117c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:47.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.170+0000 7fc171eb4700 1 --2- 172.21.15.19:0/1516655100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c1102a0 0x7fc16c117c00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc1600099c0 tx=0x7fc160009cd0 comp rx=0 tx=0).stop 2024-05-09T02:20:47.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.170+0000 7fc171eb4700 1 -- 172.21.15.19:0/1516655100 shutdown_connections 2024-05-09T02:20:47.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.170+0000 7fc171eb4700 1 --2- 172.21.15.19:0/1516655100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c1102a0 0x7fc16c117c00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:47.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.170+0000 7fc171eb4700 1 --2- 172.21.15.19:0/1516655100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c10b020 0x7fc16c10b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:47.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.170+0000 7fc171eb4700 1 --2- 172.21.15.19:0/1516655100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc16c10f950 0x7fc16c10fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:47.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.170+0000 7fc171eb4700 1 -- 172.21.15.19:0/1516655100 >> 172.21.15.19:0/1516655100 conn(0x7fc16c06f990 msgr2=0x7fc16c06fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:47.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.171+0000 7fc171eb4700 1 -- 172.21.15.19:0/1516655100 shutdown_connections 2024-05-09T02:20:47.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.171+0000 7fc171eb4700 1 -- 172.21.15.19:0/1516655100 wait complete. 2024-05-09T02:20:47.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.172+0000 7fc171eb4700 1 Processor -- start 2024-05-09T02:20:47.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.172+0000 7fc171eb4700 1 -- start start 2024-05-09T02:20:47.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.172+0000 7fc171eb4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc16c10b020 0x7fc16c1a6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:47.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.172+0000 7fc171eb4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c10f950 0x7fc16c1a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:47.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.172+0000 7fc171eb4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c1102a0 0x7fc16c1a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:47.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.172+0000 7fc171eb4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c11bbe0 con 0x7fc16c1102a0 2024-05-09T02:20:47.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.172+0000 7fc171eb4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c11ba60 con 0x7fc16c10b020 2024-05-09T02:20:47.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.172+0000 7fc171eb4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c11bd60 con 0x7fc16c10f950 2024-05-09T02:20:47.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.173+0000 7fc16b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc16c10b020 0x7fc16c1a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:47.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.173+0000 7fc16affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c10f950 0x7fc16c1a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:47.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.173+0000 7fc16affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c10f950 0x7fc16c1a65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:52064/0 (socket says 172.21.15.19:52064) 2024-05-09T02:20:47.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.173+0000 7fc16affd700 1 -- 172.21.15.19:0/2111040207 learned_addr learned my addr 172.21.15.19:0/2111040207 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:47.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.173+0000 7fc16affd700 1 -- 172.21.15.19:0/2111040207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc16c10b020 msgr2=0x7fc16c1a6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:47.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.173+0000 7fc16affd700 1 --2- 172.21.15.19:0/2111040207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc16c10b020 0x7fc16c1a6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:47.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.173+0000 7fc16affd700 1 -- 172.21.15.19:0/2111040207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c1102a0 msgr2=0x7fc16c1a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:47.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.173+0000 7fc16affd700 1 --2- 172.21.15.19:0/2111040207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c1102a0 0x7fc16c1a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:47.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.173+0000 7fc16affd700 1 -- 172.21.15.19:0/2111040207 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16c1a1a40 con 0x7fc16c10f950 2024-05-09T02:20:47.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.174+0000 7fc16affd700 1 --2- 172.21.15.19:0/2111040207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c10f950 0x7fc16c1a65b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc16400ada0 tx=0x7fc16400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:47.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.174+0000 7fc168ff9700 1 -- 172.21.15.19:0/2111040207 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164017020 con 0x7fc16c10f950 2024-05-09T02:20:47.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.174+0000 7fc168ff9700 1 -- 172.21.15.19:0/2111040207 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc16400cdb0 con 0x7fc16c10f950 2024-05-09T02:20:47.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.174+0000 7fc168ff9700 1 -- 172.21.15.19:0/2111040207 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164005310 con 0x7fc16c10f950 2024-05-09T02:20:47.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.174+0000 7fc171eb4700 1 -- 172.21.15.19:0/2111040207 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16c1a1d30 con 0x7fc16c10f950 2024-05-09T02:20:47.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.175+0000 7fc171eb4700 1 -- 172.21.15.19:0/2111040207 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16c1abd70 con 0x7fc16c10f950 2024-05-09T02:20:47.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.177+0000 7fc171eb4700 1 -- 172.21.15.19:0/2111040207 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc16c0512c0 con 0x7fc16c10f950 2024-05-09T02:20:47.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.181+0000 7fc168ff9700 1 -- 172.21.15.19:0/2111040207 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc164004ad0 con 0x7fc16c10f950 2024-05-09T02:20:47.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.182+0000 7fc168ff9700 1 --2- 172.21.15.19:0/2111040207 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc1540b2aa0 0x7fc1540b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:47.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.182+0000 7fc168ff9700 1 -- 172.21.15.19:0/2111040207 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fc1640e1cc0 con 0x7fc16c10f950 2024-05-09T02:20:47.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.182+0000 7fc16b7fe700 1 --2- 172.21.15.19:0/2111040207 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc1540b2aa0 0x7fc1540b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:47.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.183+0000 7fc16b7fe700 1 --2- 172.21.15.19:0/2111040207 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc1540b2aa0 0x7fc1540b4fb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0096d0 tx=0x7fc15c009af0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:47.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.184+0000 7fc168ff9700 1 -- 172.21.15.19:0/2111040207 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc164021020 con 0x7fc16c10f950 2024-05-09T02:20:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.580+0000 7fc171eb4700 1 -- 172.21.15.19:0/2111040207 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc16c1a2740 con 0x7fc16c10f950 2024-05-09T02:20:47.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.586+0000 7fc168ff9700 1 -- 172.21.15.19:0/2111040207 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11929 (secure 0 0 0) 0x7fc1640a9700 con 0x7fc16c10f950 2024-05-09T02:20:47.591 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:20:47.592 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:20:07.875180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24457,24455,24473,24463,24484]},"id":1}]} 2024-05-09T02:20:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.598+0000 7fc171eb4700 1 -- 172.21.15.19:0/2111040207 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc1540b2aa0 msgr2=0x7fc1540b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.598+0000 7fc171eb4700 1 --2- 172.21.15.19:0/2111040207 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc1540b2aa0 0x7fc1540b4fb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0096d0 tx=0x7fc15c009af0 comp rx=0 tx=0).stop 2024-05-09T02:20:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.598+0000 7fc171eb4700 1 -- 172.21.15.19:0/2111040207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c10f950 msgr2=0x7fc16c1a65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.598+0000 7fc171eb4700 1 --2- 172.21.15.19:0/2111040207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c10f950 0x7fc16c1a65b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc16400ada0 tx=0x7fc16400c330 comp rx=0 tx=0).stop 2024-05-09T02:20:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.598+0000 7fc171eb4700 1 -- 172.21.15.19:0/2111040207 shutdown_connections 2024-05-09T02:20:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.598+0000 7fc171eb4700 1 --2- 172.21.15.19:0/2111040207 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc1540b2aa0 0x7fc1540b4fb0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0096d0 tx=0x7fc15c009af0 comp rx=0 tx=0).stop 2024-05-09T02:20:47.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.598+0000 7fc171eb4700 1 --2- 172.21.15.19:0/2111040207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c1102a0 0x7fc16c1a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.598+0000 7fc171eb4700 1 --2- 172.21.15.19:0/2111040207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc16c10b020 0x7fc16c1a6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.598+0000 7fc171eb4700 1 --2- 172.21.15.19:0/2111040207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c10f950 0x7fc16c1a65b0 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc16400ada0 tx=0x7fc16400c330 comp rx=0 tx=0).stop 2024-05-09T02:20:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.598+0000 7fc171eb4700 1 -- 172.21.15.19:0/2111040207 >> 172.21.15.19:0/2111040207 conn(0x7fc16c06f990 msgr2=0x7fc16c10e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.599+0000 7fc171eb4700 1 -- 172.21.15.19:0/2111040207 shutdown_connections 2024-05-09T02:20:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:47.600+0000 7fc171eb4700 1 -- 172.21.15.19:0/2111040207 wait complete. 2024-05-09T02:20:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 54 2024-05-09T02:20:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 545.0 s, 1734.6 tps, lat 57.556 ms stddev 66.957 2024-05-09T02:20:48.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T02:20:48.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:48 smithi059 ceph-mon[34781]: pgmap v294: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 1.57k op/s 2024-05-09T02:20:48.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:48 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2111040207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:20:48.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:48 smithi196 ceph-mon[34718]: pgmap v294: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 1.57k op/s 2024-05-09T02:20:48.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:48 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2111040207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:20:48.795 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:20:48.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:48 smithi019 ceph-mon[30151]: pgmap v294: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 1.57k op/s 2024-05-09T02:20:48.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:48 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2111040207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:20:50.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:50 smithi059 ceph-mon[34781]: pgmap v295: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 22 MiB/s wr, 1.79k op/s 2024-05-09T02:20:50.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:50 smithi196 ceph-mon[34718]: pgmap v295: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 22 MiB/s wr, 1.79k op/s 2024-05-09T02:20:50.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:50 smithi019 ceph-mon[30151]: pgmap v295: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 22 MiB/s wr, 1.79k op/s 2024-05-09T02:20:52.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.298+0000 7f3515a7a700 1 -- 172.21.15.19:0/2792788736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3510103310 msgr2=0x7f351010f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:52.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.298+0000 7f3515a7a700 1 --2- 172.21.15.19:0/2792788736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3510103310 0x7f351010f890 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f350800f450 tx=0x7f350800f760 comp rx=0 tx=0).stop 2024-05-09T02:20:52.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.298+0000 7f3515a7a700 1 -- 172.21.15.19:0/2792788736 shutdown_connections 2024-05-09T02:20:52.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.298+0000 7f3515a7a700 1 --2- 172.21.15.19:0/2792788736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3510103310 0x7f351010f890 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.298+0000 7f3515a7a700 1 --2- 172.21.15.19:0/2792788736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35101089f0 0x7f3510108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.298+0000 7f3515a7a700 1 --2- 172.21.15.19:0/2792788736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35101029c0 0x7f3510102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.298+0000 7f3515a7a700 1 -- 172.21.15.19:0/2792788736 >> 172.21.15.19:0/2792788736 conn(0x7f35100fe500 msgr2=0x7f3510100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:52.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.299+0000 7f3515a7a700 1 -- 172.21.15.19:0/2792788736 shutdown_connections 2024-05-09T02:20:52.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.299+0000 7f3515a7a700 1 -- 172.21.15.19:0/2792788736 wait complete. 2024-05-09T02:20:52.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.300+0000 7f3515a7a700 1 Processor -- start 2024-05-09T02:20:52.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.301+0000 7f3515a7a700 1 -- start start 2024-05-09T02:20:52.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.301+0000 7f3515a7a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35101029c0 0x7f351019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.301+0000 7f3515a7a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3510103310 0x7f351019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.301+0000 7f3515a7a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35101089f0 0x7f3510198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.301+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35101029c0 0x7f351019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.301+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35101029c0 0x7f351019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:44730/0 (socket says 172.21.15.19:44730) 2024-05-09T02:20:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.301+0000 7f350f7fe700 1 -- 172.21.15.19:0/3576673334 learned_addr learned my addr 172.21.15.19:0/3576673334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.301+0000 7f3515a7a700 1 -- 172.21.15.19:0/3576673334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3510112300 con 0x7f3510103310 2024-05-09T02:20:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.302+0000 7f350effd700 1 --2- 172.21.15.19:0/3576673334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3510103310 0x7f351019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.302+0000 7f3515a7a700 1 -- 172.21.15.19:0/3576673334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3510112180 con 0x7f35101029c0 2024-05-09T02:20:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.302+0000 7f3515a7a700 1 -- 172.21.15.19:0/3576673334 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3510112480 con 0x7f35101089f0 2024-05-09T02:20:52.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.302+0000 7f350ffff700 1 --2- 172.21.15.19:0/3576673334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35101089f0 0x7f3510198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.302+0000 7f350f7fe700 1 -- 172.21.15.19:0/3576673334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35101089f0 msgr2=0x7f3510198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:52.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.302+0000 7f350f7fe700 1 --2- 172.21.15.19:0/3576673334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35101089f0 0x7f3510198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.302+0000 7f350f7fe700 1 -- 172.21.15.19:0/3576673334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3510103310 msgr2=0x7f351019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:52.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.302+0000 7f350f7fe700 1 --2- 172.21.15.19:0/3576673334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3510103310 0x7f351019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.302+0000 7f350f7fe700 1 -- 172.21.15.19:0/3576673334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35101990d0 con 0x7f35101029c0 2024-05-09T02:20:52.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.303+0000 7f350f7fe700 1 --2- 172.21.15.19:0/3576673334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35101029c0 0x7f351019e800 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f350000b500 tx=0x7f350000b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:52.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.303+0000 7f350cff9700 1 -- 172.21.15.19:0/3576673334 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500015020 con 0x7f35101029c0 2024-05-09T02:20:52.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.303+0000 7f350cff9700 1 -- 172.21.15.19:0/3576673334 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3500011ce0 con 0x7f35101029c0 2024-05-09T02:20:52.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.303+0000 7f3515a7a700 1 -- 172.21.15.19:0/3576673334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35101993c0 con 0x7f35101029c0 2024-05-09T02:20:52.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.303+0000 7f3515a7a700 1 -- 172.21.15.19:0/3576673334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35101a5690 con 0x7f35101029c0 2024-05-09T02:20:52.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.303+0000 7f350cff9700 1 -- 172.21.15.19:0/3576673334 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500010da0 con 0x7f35101029c0 2024-05-09T02:20:52.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.307+0000 7f350cff9700 1 -- 172.21.15.19:0/3576673334 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3500018020 con 0x7f35101029c0 2024-05-09T02:20:52.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.309+0000 7f3524490b80 1 Processor -- start 2024-05-09T02:20:52.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.309+0000 7f3524490b80 1 -- start start 2024-05-09T02:20:52.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.310+0000 7f3524490b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.310+0000 7f3524490b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 0x55be7ceaca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.310+0000 7f350ffff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f350ffff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceac4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:52090/0 (socket says 172.21.15.19:52090) 2024-05-09T02:20:52.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f3524490b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cea66a0 0x55be7cea6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f3524490b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55be7cdc8ff0 con 0x55be7cd294a0 2024-05-09T02:20:52.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f3524490b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55be7cd677f0 con 0x55be7cea66a0 2024-05-09T02:20:52.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f3524490b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55be7cdc4fb0 con 0x55be7ce13160 2024-05-09T02:20:52.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 0x55be7ceaca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 0x55be7ceaca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32966/0 (socket says 172.21.15.19:32966) 2024-05-09T02:20:52.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f350f7fe700 1 -- 172.21.15.19:0/2553128417 learned_addr learned my addr 172.21.15.19:0/2553128417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:52.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f350cff9700 1 --2- 172.21.15.19:0/3576673334 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f34f80b2b40 0x7f34f80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f350effd700 1 --2- 172.21.15.19:0/2553128417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cea66a0 0x55be7cea6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f350ffff700 1 -- 172.21.15.19:0/2553128417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cea66a0 msgr2=0x55be7cea6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:52.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f350ffff700 1 --2- 172.21.15.19:0/2553128417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cea66a0 0x55be7cea6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f350ffff700 1 -- 172.21.15.19:0/2553128417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 msgr2=0x55be7ceaca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f350ffff700 1 --2- 172.21.15.19:0/2553128417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 0x55be7ceaca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f350ffff700 1 -- 172.21.15.19:0/2553128417 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be7cd5f370 con 0x55be7ce13160 2024-05-09T02:20:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.311+0000 7f350ffff700 1 --2- 172.21.15.19:0/3576673334 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f34f80b2b40 0x7f34f80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.312+0000 7f350cff9700 1 -- 172.21.15.19:0/3576673334 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f35000e1af0 con 0x7f35101029c0 2024-05-09T02:20:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.312+0000 7f350ffff700 1 --2- 172.21.15.19:0/2553128417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceac4f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f350800fb30 tx=0x7f350800abf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae6da76e08626fe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.312+0000 7f34f5ffb700 1 -- 172.21.15.19:0/2553128417 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508027070 con 0x55be7ce13160 2024-05-09T02:20:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.312+0000 7f350ffff700 1 --2- 172.21.15.19:0/3576673334 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f34f80b2b40 0x7f34f80b5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55be7cea7270 tx=0x7f350802d000 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.312+0000 7f34f5ffb700 1 -- 172.21.15.19:0/2553128417 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35080088a0 con 0x55be7ce13160 2024-05-09T02:20:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.312+0000 7f34f5ffb700 1 -- 172.21.15.19:0/2553128417 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508022440 con 0x55be7ce13160 2024-05-09T02:20:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.313+0000 7f3524490b80 1 -- 172.21.15.19:0/2553128417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 msgr2=0x55be7ceac4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.313+0000 7f3524490b80 1 --2- 172.21.15.19:0/2553128417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceac4f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f350800fb30 tx=0x7f350800abf0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.313+0000 7f3524490b80 1 -- 172.21.15.19:0/2553128417 shutdown_connections 2024-05-09T02:20:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.313+0000 7f3524490b80 1 --2- 172.21.15.19:0/2553128417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cea66a0 0x55be7cea6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.313+0000 7f3524490b80 1 --2- 172.21.15.19:0/2553128417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceac4f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.313+0000 7f3524490b80 1 --2- 172.21.15.19:0/2553128417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 0x55be7ceaca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.313+0000 7f3524490b80 1 -- 172.21.15.19:0/2553128417 >> 172.21.15.19:0/2553128417 conn(0x55be7cd896a0 msgr2=0x55be7cea1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:52.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.314+0000 7f3524490b80 1 -- 172.21.15.19:0/2553128417 shutdown_connections 2024-05-09T02:20:52.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.314+0000 7f3524490b80 1 -- 172.21.15.19:0/2553128417 wait complete. 2024-05-09T02:20:52.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.314+0000 7f3524490b80 1 -- start start 2024-05-09T02:20:52.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.315+0000 7f3524490b80 1 Processor -- start 2024-05-09T02:20:52.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.316+0000 7f3524490b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 0x55be7ceb0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.316+0000 7f3524490b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceb0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.316+0000 7f3524490b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cea66a0 0x55be7ceb0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.316+0000 7f3524490b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55be7cc44ba0 con 0x55be7cd294a0 2024-05-09T02:20:52.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.316+0000 7f3524490b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55be7cd83150 con 0x55be7cea66a0 2024-05-09T02:20:52.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.316+0000 7f3524490b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55be7ceb31d0 con 0x55be7ce13160 2024-05-09T02:20:52.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.316+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 0x55be7ceb0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.316+0000 7f350effd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceb0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.316+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 0x55be7ceb0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32970/0 (socket says 172.21.15.19:32970) 2024-05-09T02:20:52.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.316+0000 7f350f7fe700 1 -- 172.21.15.19:0/1714766381 learned_addr learned my addr 172.21.15.19:0/1714766381 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:52.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.316+0000 7f350ffff700 1 --2- 172.21.15.19:0/1714766381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cea66a0 0x55be7ceb0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.316+0000 7f3524490b80 10 client.0 _ll_register_callbacks cb 0x55be7cc3a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:20:52.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.317+0000 7f350effd700 1 -- 172.21.15.19:0/1714766381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cea66a0 msgr2=0x55be7ceb0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:52.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.317+0000 7f350effd700 1 --2- 172.21.15.19:0/1714766381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cea66a0 0x55be7ceb0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.317+0000 7f350effd700 1 -- 172.21.15.19:0/1714766381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 msgr2=0x55be7ceb0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:52.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.317+0000 7f350effd700 1 --2- 172.21.15.19:0/1714766381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 0x55be7ceb0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.317+0000 7f350effd700 1 -- 172.21.15.19:0/1714766381 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be7cc44490 con 0x55be7ce13160 2024-05-09T02:20:52.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.317+0000 7f350ffff700 1 --2- 172.21.15.19:0/1714766381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cea66a0 0x55be7ceb0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:20:52.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.317+0000 7f3524490b80 1 -- 172.21.15.19:0/1714766381 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35080088a0 con 0x55be7ce13160 2024-05-09T02:20:52.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.317+0000 7f350effd700 1 --2- 172.21.15.19:0/1714766381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceb0770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3510101b10 tx=0x7f3504015a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4528aae6cd36a7aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:52.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.317+0000 7f34eeffd700 10 client.26209 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:20:52.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.317+0000 7f34eeffd700 1 -- 172.21.15.19:0/1714766381 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35040178c0 con 0x55be7ce13160 2024-05-09T02:20:52.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.317+0000 7f34eeffd700 1 -- 172.21.15.19:0/1714766381 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f3504017a60 con 0x55be7ce13160 2024-05-09T02:20:52.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.318+0000 7f34eeffd700 1 -- 172.21.15.19:0/1714766381 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3504017ca0 con 0x55be7ce13160 2024-05-09T02:20:52.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.318+0000 7f3524490b80 10 client.26209 fetch_fsmap learned FSMap version 54 2024-05-09T02:20:52.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.318+0000 7f3524490b80 1 -- 172.21.15.19:0/1714766381 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=54}) -- 0x55be7cebdbd0 con 0x55be7ce13160 2024-05-09T02:20:52.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.318+0000 7f34eeffd700 1 -- 172.21.15.19:0/1714766381 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f350401f420 con 0x55be7ce13160 2024-05-09T02:20:52.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.318+0000 7f3524490b80 10 client.26209 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:20:52.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.318+0000 7f3524490b80 10 client.26209 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:20:52.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.318+0000 7f3524490b80 20 client.26209 populate_metadata read hostname 'smithi019' 2024-05-09T02:20:52.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.318+0000 7f3524490b80 1 --2- 172.21.15.19:0/1714766381 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be7cec0140 0x55be7cec2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.318+0000 7f3524490b80 4 client.26209 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:20:52.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.318+0000 7f3524490b80 1 -- 172.21.15.19:0/1714766381 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be7cdd69d0 con 0x55be7cec0140 2024-05-09T02:20:52.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.319+0000 7f350f7fe700 1 --2- 172.21.15.19:0/1714766381 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be7cec0140 0x55be7cec2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.319+0000 7f350f7fe700 1 --2- 172.21.15.19:0/1714766381 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be7cec0140 0x55be7cec2580 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3abe6026663c3e2f server_cookie=35acecae011a2ea7 in_seq=0 out_seq=0 2024-05-09T02:20:52.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.319+0000 7f34eeffd700 10 client.26209 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:20:52.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.320+0000 7f34eeffd700 1 -- 172.21.15.19:0/1714766381 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55be7cdd69d0 con 0x55be7cec0140 2024-05-09T02:20:52.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.320+0000 7f34eeffd700 10 client.26209 handle_command_reply: tid=0 2024-05-09T02:20:52.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.320+0000 7f3524490b80 1 client.26209 shutdown 2024-05-09T02:20:52.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.320+0000 7f3524490b80 1 -- 172.21.15.19:0/1714766381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 msgr2=0x55be7ceb0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:52.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.320+0000 7f3524490b80 1 --2- 172.21.15.19:0/1714766381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceb0770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3510101b10 tx=0x7f3504015a80 comp rx=0 tx=0).stop 2024-05-09T02:20:52.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.320+0000 7f3524490b80 1 -- 172.21.15.19:0/1714766381 shutdown_connections 2024-05-09T02:20:52.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.320+0000 7f3524490b80 1 --2- 172.21.15.19:0/1714766381 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be7cec0140 0x55be7cec2580 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.320+0000 7f3524490b80 1 --2- 172.21.15.19:0/1714766381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cea66a0 0x55be7ceb0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.320+0000 7f3524490b80 1 --2- 172.21.15.19:0/1714766381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 0x55be7ceb0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.320+0000 7f34eeffd700 0 client.26209 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:20:52.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.320+0000 7f3524490b80 1 --2- 172.21.15.19:0/1714766381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceb0770 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.320+0000 7f3524490b80 1 -- 172.21.15.19:0/1714766381 >> 172.21.15.19:0/1714766381 conn(0x55be7cd896a0 msgr2=0x55be7cea11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:52.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.320+0000 7f3524490b80 1 -- 172.21.15.19:0/1714766381 shutdown_connections 2024-05-09T02:20:52.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.320+0000 7f3524490b80 1 -- 172.21.15.19:0/1714766381 wait complete. 2024-05-09T02:20:52.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.320+0000 7f3524490b80 20 client.26209 trim_cache size 0 max 16384 2024-05-09T02:20:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.340+0000 7f3524490b80 1 Processor -- start 2024-05-09T02:20:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.340+0000 7f3524490b80 1 -- start start 2024-05-09T02:20:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.340+0000 7f3524490b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cd294a0 0x55be7ceaa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.340+0000 7f3524490b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceb0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.340+0000 7f3524490b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cea66a0 0x55be7ceab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.340+0000 7f3524490b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc00d600 con 0x55be7cea66a0 2024-05-09T02:20:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.340+0000 7f3524490b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc00f290 con 0x55be7cd294a0 2024-05-09T02:20:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.340+0000 7f3524490b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc00dcd0 con 0x55be7ce13160 2024-05-09T02:20:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.340+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cd294a0 0x55be7ceaa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.340+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cd294a0 0x55be7ceaa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:44754/0 (socket says 172.21.15.19:44754) 2024-05-09T02:20:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.340+0000 7f350f7fe700 1 -- 172.21.15.19:0/3206316708 learned_addr learned my addr 172.21.15.19:0/3206316708 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.340+0000 7f350ffff700 1 --2- 172.21.15.19:0/3206316708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cea66a0 0x55be7ceab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.340+0000 7f350effd700 1 --2- 172.21.15.19:0/3206316708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceb0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.341+0000 7f350effd700 1 -- 172.21.15.19:0/3206316708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cd294a0 msgr2=0x55be7ceaa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.341+0000 7f350effd700 1 --2- 172.21.15.19:0/3206316708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cd294a0 0x55be7ceaa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.341+0000 7f350effd700 1 -- 172.21.15.19:0/3206316708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cea66a0 msgr2=0x55be7ceab1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.341+0000 7f350effd700 1 --2- 172.21.15.19:0/3206316708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cea66a0 0x55be7ceab1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.341+0000 7f350effd700 1 -- 172.21.15.19:0/3206316708 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be7cea2290 con 0x55be7ce13160 2024-05-09T02:20:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.341+0000 7f350effd700 1 --2- 172.21.15.19:0/3206316708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceb0af0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3504015ac0 tx=0x7f350401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67e71c214c34b75c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:52.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.341+0000 7f34f57fa700 1 -- 172.21.15.19:0/3206316708 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350401f420 con 0x55be7ce13160 2024-05-09T02:20:52.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.341+0000 7f34f57fa700 1 -- 172.21.15.19:0/3206316708 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3504038070 con 0x55be7ce13160 2024-05-09T02:20:52.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.341+0000 7f34f57fa700 1 -- 172.21.15.19:0/3206316708 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504003680 con 0x55be7ce13160 2024-05-09T02:20:52.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.343+0000 7f3524490b80 1 -- 172.21.15.19:0/3206316708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 msgr2=0x55be7ceb0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:52.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.343+0000 7f3524490b80 1 --2- 172.21.15.19:0/3206316708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceb0af0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3504015ac0 tx=0x7f350401a000 comp rx=0 tx=0).stop 2024-05-09T02:20:52.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.343+0000 7f3524490b80 1 -- 172.21.15.19:0/3206316708 shutdown_connections 2024-05-09T02:20:52.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.343+0000 7f3524490b80 1 --2- 172.21.15.19:0/3206316708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cea66a0 0x55be7ceab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.343+0000 7f3524490b80 1 --2- 172.21.15.19:0/3206316708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7cd294a0 0x55be7ceaa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.343+0000 7f3524490b80 1 --2- 172.21.15.19:0/3206316708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7ce13160 0x55be7ceb0af0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.343+0000 7f3524490b80 1 -- 172.21.15.19:0/3206316708 >> 172.21.15.19:0/3206316708 conn(0x55be7cd896a0 msgr2=0x55be7ceaeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.343+0000 7f3524490b80 1 -- 172.21.15.19:0/3206316708 shutdown_connections 2024-05-09T02:20:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.343+0000 7f3524490b80 1 -- 172.21.15.19:0/3206316708 wait complete. 2024-05-09T02:20:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.344+0000 7f3524490b80 1 -- start start 2024-05-09T02:20:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.344+0000 7f3524490b80 1 Processor -- start 2024-05-09T02:20:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.344+0000 7f3524490b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 0x55be7ceafab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.344+0000 7f3524490b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7cea66a0 0x55be7ceafff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f3524490b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7ceb8b60 0x55be7ceb0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f3524490b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc00eb30 con 0x55be7cd294a0 2024-05-09T02:20:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f3524490b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc00e3a0 con 0x55be7ceb8b60 2024-05-09T02:20:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f3524490b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc00b5c0 con 0x55be7cea66a0 2024-05-09T02:20:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f3524490b80 10 client.0 _ll_register_callbacks cb 0x55be7cc3a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:20:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 0x55be7ceafab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f350ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7ceb8b60 0x55be7ceb0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 0x55be7ceafab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32984/0 (socket says 172.21.15.19:32984) 2024-05-09T02:20:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f350ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7ceb8b60 0x55be7ceb0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:44766/0 (socket says 172.21.15.19:44766) 2024-05-09T02:20:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f350f7fe700 1 -- 172.21.15.19:0/253345350 learned_addr learned my addr 172.21.15.19:0/253345350 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f350effd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7cea66a0 0x55be7ceafff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f350ffff700 1 -- 172.21.15.19:0/253345350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7cea66a0 msgr2=0x55be7ceafff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f350ffff700 1 --2- 172.21.15.19:0/253345350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7cea66a0 0x55be7ceafff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f350ffff700 1 -- 172.21.15.19:0/253345350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 msgr2=0x55be7ceafab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f350ffff700 1 --2- 172.21.15.19:0/253345350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 0x55be7ceafab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.345+0000 7f350ffff700 1 -- 172.21.15.19:0/253345350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be7ceb6600 con 0x55be7ceb8b60 2024-05-09T02:20:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.346+0000 7f350ffff700 1 --2- 172.21.15.19:0/253345350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7ceb8b60 0x55be7ceb0530 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f350800a810 tx=0x7f350800f360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19fe2a9a9345bc79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.346+0000 7f34eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:20:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.346+0000 7f34eeffd700 1 -- 172.21.15.19:0/253345350 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508027040 con 0x55be7ceb8b60 2024-05-09T02:20:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.346+0000 7f34eeffd700 1 -- 172.21.15.19:0/253345350 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f3508003a00 con 0x55be7ceb8b60 2024-05-09T02:20:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.346+0000 7f3524490b80 1 -- 172.21.15.19:0/253345350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3504038070 con 0x55be7ceb8b60 2024-05-09T02:20:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.346+0000 7f34eeffd700 1 -- 172.21.15.19:0/253345350 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3508003c00 con 0x55be7ceb8b60 2024-05-09T02:20:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.346+0000 7f3524490b80 10 client.26189 fetch_fsmap learned FSMap version 54 2024-05-09T02:20:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.346+0000 7f3524490b80 1 -- 172.21.15.19:0/253345350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55be7cf334f0 con 0x55be7ceb8b60 2024-05-09T02:20:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.347+0000 7f34eeffd700 1 -- 172.21.15.19:0/253345350 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3504038070 con 0x55be7ceb8b60 2024-05-09T02:20:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.347+0000 7f34eeffd700 1 -- 172.21.15.19:0/253345350 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f3508045070 con 0x55be7ceb8b60 2024-05-09T02:20:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.347+0000 7f3524490b80 10 client.26189 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:20:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.347+0000 7f3524490b80 10 client.26189 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:20:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.347+0000 7f3524490b80 20 client.26189 populate_metadata read hostname 'smithi019' 2024-05-09T02:20:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.347+0000 7f3524490b80 1 --2- 172.21.15.19:0/253345350 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be7cf33770 0x55be7cf35b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.347+0000 7f3524490b80 4 client.26189 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:20:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.347+0000 7f3524490b80 1 -- 172.21.15.19:0/253345350 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be7cdd0a70 con 0x55be7cf33770 2024-05-09T02:20:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.347+0000 7f350f7fe700 1 --2- 172.21.15.19:0/253345350 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be7cf33770 0x55be7cf35b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.347+0000 7f34eeffd700 10 client.26189 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:20:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f350f7fe700 1 --2- 172.21.15.19:0/253345350 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be7cf33770 0x55be7cf35b90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c5e06e860b71e032 server_cookie=2841eb8f5e98f0ed in_seq=0 out_seq=0 2024-05-09T02:20:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f34eeffd700 1 -- 172.21.15.19:0/253345350 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be7cdd0a70 con 0x55be7cf33770 2024-05-09T02:20:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f34eeffd700 10 client.26189 handle_command_reply: tid=0 2024-05-09T02:20:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f3524490b80 1 client.26189 shutdown 2024-05-09T02:20:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f3524490b80 1 -- 172.21.15.19:0/253345350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7ceb8b60 msgr2=0x55be7ceb0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f3524490b80 1 --2- 172.21.15.19:0/253345350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7ceb8b60 0x55be7ceb0530 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f350800a810 tx=0x7f350800f360 comp rx=0 tx=0).stop 2024-05-09T02:20:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f3524490b80 1 -- 172.21.15.19:0/253345350 shutdown_connections 2024-05-09T02:20:52.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f3524490b80 1 --2- 172.21.15.19:0/253345350 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55be7cf33770 0x55be7cf35b90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f3524490b80 1 --2- 172.21.15.19:0/253345350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55be7ceb8b60 0x55be7ceb0530 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f3524490b80 1 --2- 172.21.15.19:0/253345350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be7cd294a0 0x55be7ceafab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f3524490b80 1 --2- 172.21.15.19:0/253345350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55be7cea66a0 0x55be7ceafff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f34eeffd700 0 client.26189 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:20:52.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f3524490b80 1 -- 172.21.15.19:0/253345350 >> 172.21.15.19:0/253345350 conn(0x55be7cd896a0 msgr2=0x55be7ceab1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:52.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f3524490b80 1 -- 172.21.15.19:0/253345350 shutdown_connections 2024-05-09T02:20:52.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f3524490b80 1 -- 172.21.15.19:0/253345350 wait complete. 2024-05-09T02:20:52.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.348+0000 7f3524490b80 20 client.26189 trim_cache size 0 max 16384 2024-05-09T02:20:52.353 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:20:52.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.349+0000 7f3515a7a700 1 -- 172.21.15.19:0/3576673334 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f34f80b2b40 msgr2=0x7f34f80b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:52.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.349+0000 7f3515a7a700 1 --2- 172.21.15.19:0/3576673334 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f34f80b2b40 0x7f34f80b5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55be7cea7270 tx=0x7f350802d000 comp rx=0 tx=0).stop 2024-05-09T02:20:52.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.349+0000 7f3515a7a700 1 -- 172.21.15.19:0/3576673334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35101029c0 msgr2=0x7f351019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:52.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.349+0000 7f3515a7a700 1 --2- 172.21.15.19:0/3576673334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35101029c0 0x7f351019e800 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f350000b500 tx=0x7f350000b8c0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.349+0000 7f3515a7a700 1 -- 172.21.15.19:0/3576673334 shutdown_connections 2024-05-09T02:20:52.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.349+0000 7f3515a7a700 1 --2- 172.21.15.19:0/3576673334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35101089f0 0x7f3510198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.349+0000 7f3515a7a700 1 --2- 172.21.15.19:0/3576673334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35101029c0 0x7f351019e800 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.349+0000 7f3515a7a700 1 --2- 172.21.15.19:0/3576673334 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f34f80b2b40 0x7f34f80b5050 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.349+0000 7f3515a7a700 1 --2- 172.21.15.19:0/3576673334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3510103310 0x7f351019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:52.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.349+0000 7f3515a7a700 1 -- 172.21.15.19:0/3576673334 >> 172.21.15.19:0/3576673334 conn(0x7f35100fe500 msgr2=0x7f3510107c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:52.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.349+0000 7f3515a7a700 1 -- 172.21.15.19:0/3576673334 shutdown_connections 2024-05-09T02:20:52.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:52.349+0000 7f3515a7a700 1 -- 172.21.15.19:0/3576673334 wait complete. 2024-05-09T02:20:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 550.0 s, 1536.0 tps, lat 64.487 ms stddev 70.462 2024-05-09T02:20:52.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:52 smithi059 ceph-mon[34781]: pgmap v296: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.3 MiB/s wr, 755 op/s 2024-05-09T02:20:52.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:52 smithi196 ceph-mon[34718]: pgmap v296: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.3 MiB/s wr, 755 op/s 2024-05-09T02:20:52.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:52 smithi019 ceph-mon[30151]: pgmap v296: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.3 MiB/s wr, 755 op/s 2024-05-09T02:20:53.331 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T02:20:53.763 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:20:54.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:54 smithi019 ceph-mon[30151]: pgmap v297: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.2 MiB/s wr, 530 op/s 2024-05-09T02:20:54.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:54 smithi059 ceph-mon[34781]: pgmap v297: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.2 MiB/s wr, 530 op/s 2024-05-09T02:20:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:54 smithi196 ceph-mon[34718]: pgmap v297: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.2 MiB/s wr, 530 op/s 2024-05-09T02:20:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.656+0000 7fd3eab17700 1 -- 172.21.15.19:0/1444842999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4103250 msgr2=0x7fd3e4103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.656+0000 7fd3eab17700 1 --2- 172.21.15.19:0/1444842999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4103250 0x7fd3e4103660 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd3d801a9a0 tx=0x7fd3d801acb0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.656+0000 7fd3eab17700 1 -- 172.21.15.19:0/1444842999 shutdown_connections 2024-05-09T02:20:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.656+0000 7fd3eab17700 1 --2- 172.21.15.19:0/1444842999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e41086a0 0x7fd3e4108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.656+0000 7fd3eab17700 1 --2- 172.21.15.19:0/1444842999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4103250 0x7fd3e4103660 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.656+0000 7fd3eab17700 1 --2- 172.21.15.19:0/1444842999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e4103ba0 0x7fd3e41080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.656+0000 7fd3eab17700 1 -- 172.21.15.19:0/1444842999 >> 172.21.15.19:0/1444842999 conn(0x7fd3e40fedc0 msgr2=0x7fd3e41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.656+0000 7fd3eab17700 1 -- 172.21.15.19:0/1444842999 shutdown_connections 2024-05-09T02:20:55.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.657+0000 7fd3eab17700 1 -- 172.21.15.19:0/1444842999 wait complete. 2024-05-09T02:20:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.657+0000 7fd3eab17700 1 Processor -- start 2024-05-09T02:20:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.657+0000 7fd3eab17700 1 -- start start 2024-05-09T02:20:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.658+0000 7fd3eab17700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4103250 0x7fd3e4117840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.658+0000 7fd3eab17700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e4103ba0 0x7fd3e4113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.658+0000 7fd3eab17700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e41086a0 0x7fd3e4113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.658+0000 7fd3eab17700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e410bec0 con 0x7fd3e4103ba0 2024-05-09T02:20:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.658+0000 7fd3eab17700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e410bd40 con 0x7fd3e41086a0 2024-05-09T02:20:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.658+0000 7fd3eab17700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e410c040 con 0x7fd3e4103250 2024-05-09T02:20:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.658+0000 7fd3e88b3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4103250 0x7fd3e4117840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.658+0000 7fd3e3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e4103ba0 0x7fd3e4113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.658+0000 7fd3e3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e4103ba0 0x7fd3e4113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32990/0 (socket says 172.21.15.19:32990) 2024-05-09T02:20:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.658+0000 7fd3e3fff700 1 -- 172.21.15.19:0/2581615855 learned_addr learned my addr 172.21.15.19:0/2581615855 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.658+0000 7fd3e90b4700 1 --2- 172.21.15.19:0/2581615855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e41086a0 0x7fd3e4113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.659+0000 7fd3e3fff700 1 -- 172.21.15.19:0/2581615855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4103250 msgr2=0x7fd3e4117840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:55.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.659+0000 7fd3e3fff700 1 --2- 172.21.15.19:0/2581615855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4103250 0x7fd3e4117840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.659+0000 7fd3e3fff700 1 -- 172.21.15.19:0/2581615855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e41086a0 msgr2=0x7fd3e4113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:55.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.659+0000 7fd3e3fff700 1 --2- 172.21.15.19:0/2581615855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e41086a0 0x7fd3e4113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.659+0000 7fd3e3fff700 1 -- 172.21.15.19:0/2581615855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e41146f0 con 0x7fd3e4103ba0 2024-05-09T02:20:55.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.659+0000 7fd3e3fff700 1 --2- 172.21.15.19:0/2581615855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e4103ba0 0x7fd3e4113980 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000b550 tx=0x7fd3d000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:55.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.659+0000 7fd3e1ffb700 1 -- 172.21.15.19:0/2581615855 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0015020 con 0x7fd3e4103ba0 2024-05-09T02:20:55.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.659+0000 7fd3eab17700 1 -- 172.21.15.19:0/2581615855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e41a50e0 con 0x7fd3e4103ba0 2024-05-09T02:20:55.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.659+0000 7fd3e1ffb700 1 -- 172.21.15.19:0/2581615855 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d0011ce0 con 0x7fd3e4103ba0 2024-05-09T02:20:55.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.659+0000 7fd3e1ffb700 1 -- 172.21.15.19:0/2581615855 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0010c80 con 0x7fd3e4103ba0 2024-05-09T02:20:55.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.660+0000 7fd3eab17700 1 -- 172.21.15.19:0/2581615855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e41a5670 con 0x7fd3e4103ba0 2024-05-09T02:20:55.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.664+0000 7fd3e1ffb700 1 -- 172.21.15.19:0/2581615855 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd3d0018020 con 0x7fd3e4103ba0 2024-05-09T02:20:55.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.666+0000 7fd3e1ffb700 1 --2- 172.21.15.19:0/2581615855 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fd3d40b2af0 0x7fd3d40b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.666+0000 7fd3e1ffb700 1 -- 172.21.15.19:0/2581615855 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fd3d00e1ca0 con 0x7fd3e4103ba0 2024-05-09T02:20:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.666+0000 7fd3f952db80 1 Processor -- start 2024-05-09T02:20:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.666+0000 7fd3e88b3700 1 --2- 172.21.15.19:0/2581615855 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fd3d40b2af0 0x7fd3d40b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.666+0000 7fd3f952db80 1 -- start start 2024-05-09T02:20:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.666+0000 7fd3f952db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca91897d00 0x55ca919a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.666+0000 7fd3f952db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9181d4a0 0x55ca919a0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.666+0000 7fd3f952db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca9199a400 0x55ca9199a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.667+0000 7fd3f952db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ca918bd0a0 con 0x55ca9181d4a0 2024-05-09T02:20:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.667+0000 7fd3f952db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ca918629a0 con 0x55ca9199a400 2024-05-09T02:20:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.667+0000 7fd3f952db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ca918b9060 con 0x55ca91897d00 2024-05-09T02:20:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.667+0000 7fd3e3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9181d4a0 0x55ca919a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.667+0000 7fd3e3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9181d4a0 0x55ca919a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32996/0 (socket says 172.21.15.19:32996) 2024-05-09T02:20:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.667+0000 7fd3e90b4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca9199a400 0x55ca9199a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.667+0000 7fd3e3fff700 1 -- 172.21.15.19:0/2344926852 learned_addr learned my addr 172.21.15.19:0/2344926852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.667+0000 7fd3e88b3700 1 --2- 172.21.15.19:0/2344926852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca91897d00 0x55ca919a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.667+0000 7fd3e88b3700 1 --2- 172.21.15.19:0/2581615855 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fd3d40b2af0 0x7fd3d40b5000 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd3d801c5d0 tx=0x7fd3d802a040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.667+0000 7fd3e90b4700 1 -- 172.21.15.19:0/2344926852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca91897d00 msgr2=0x55ca919a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.667+0000 7fd3e90b4700 1 --2- 172.21.15.19:0/2344926852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca91897d00 0x55ca919a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.667+0000 7fd3e90b4700 1 -- 172.21.15.19:0/2344926852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9181d4a0 msgr2=0x55ca919a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.667+0000 7fd3e90b4700 1 --2- 172.21.15.19:0/2344926852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9181d4a0 0x55ca919a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.667+0000 7fd3e90b4700 1 -- 172.21.15.19:0/2344926852 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca91853500 con 0x55ca9199a400 2024-05-09T02:20:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.668+0000 7fd3e90b4700 1 --2- 172.21.15.19:0/2344926852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca9199a400 0x55ca9199a810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009f70 tx=0x7fd3dc009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef0a2b82cd386433 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.668+0000 7fd3ceffd700 1 -- 172.21.15.19:0/2344926852 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc014620 con 0x55ca9199a400 2024-05-09T02:20:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.668+0000 7fd3ceffd700 1 -- 172.21.15.19:0/2344926852 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3dc019070 con 0x55ca9199a400 2024-05-09T02:20:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.668+0000 7fd3ceffd700 1 -- 172.21.15.19:0/2344926852 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc013a20 con 0x55ca9199a400 2024-05-09T02:20:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.669+0000 7fd3f952db80 1 -- 172.21.15.19:0/2344926852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca9199a400 msgr2=0x55ca9199a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.669+0000 7fd3f952db80 1 --2- 172.21.15.19:0/2344926852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca9199a400 0x55ca9199a810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009f70 tx=0x7fd3dc009650 comp rx=0 tx=0).stop 2024-05-09T02:20:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.669+0000 7fd3f952db80 1 -- 172.21.15.19:0/2344926852 shutdown_connections 2024-05-09T02:20:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.669+0000 7fd3f952db80 1 --2- 172.21.15.19:0/2344926852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca9199a400 0x55ca9199a810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.669+0000 7fd3f952db80 1 --2- 172.21.15.19:0/2344926852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca91897d00 0x55ca919a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.669+0000 7fd3f952db80 1 --2- 172.21.15.19:0/2344926852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9181d4a0 0x55ca919a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.669+0000 7fd3f952db80 1 -- 172.21.15.19:0/2344926852 >> 172.21.15.19:0/2344926852 conn(0x55ca91820f70 msgr2=0x55ca91995220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.669+0000 7fd3f952db80 1 -- 172.21.15.19:0/2344926852 shutdown_connections 2024-05-09T02:20:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.669+0000 7fd3f952db80 1 -- 172.21.15.19:0/2344926852 wait complete. 2024-05-09T02:20:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.670+0000 7fd3f952db80 1 -- start start 2024-05-09T02:20:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.670+0000 7fd3f952db80 1 Processor -- start 2024-05-09T02:20:55.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.670+0000 7fd3f952db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9181d4a0 0x55ca919a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.671+0000 7fd3f952db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 0x55ca919a4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.671+0000 7fd3f952db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9199a400 0x55ca919a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.671+0000 7fd3f952db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ca9199b160 con 0x55ca9199a400 2024-05-09T02:20:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.671+0000 7fd3f952db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ca91857b20 con 0x55ca91897d00 2024-05-09T02:20:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.671+0000 7fd3f952db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ca919a7020 con 0x55ca9181d4a0 2024-05-09T02:20:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.671+0000 7fd3f952db80 10 client.0 _ll_register_callbacks cb 0x55ca9172e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:20:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.672+0000 7fd3e90b4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9199a400 0x55ca919a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.672+0000 7fd3e88b3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9181d4a0 0x55ca919a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.672+0000 7fd3e90b4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9199a400 0x55ca919a4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33002/0 (socket says 172.21.15.19:33002) 2024-05-09T02:20:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.672+0000 7fd3e90b4700 1 -- 172.21.15.19:0/467263347 learned_addr learned my addr 172.21.15.19:0/467263347 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.673+0000 7fd3e3fff700 1 --2- 172.21.15.19:0/467263347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 0x55ca919a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.673+0000 7fd3e90b4700 1 -- 172.21.15.19:0/467263347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9181d4a0 msgr2=0x55ca919a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.673+0000 7fd3e90b4700 1 --2- 172.21.15.19:0/467263347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9181d4a0 0x55ca919a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.673+0000 7fd3e90b4700 1 -- 172.21.15.19:0/467263347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 msgr2=0x55ca919a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:55.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.673+0000 7fd3e90b4700 1 --2- 172.21.15.19:0/467263347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 0x55ca919a4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.673+0000 7fd3e90b4700 1 -- 172.21.15.19:0/467263347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca91738490 con 0x55ca9199a400 2024-05-09T02:20:55.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.673+0000 7fd3e90b4700 1 --2- 172.21.15.19:0/467263347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9199a400 0x55ca919a4a70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00a9c0 tx=0x7fd3dc013980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6000e03a333a5fb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:55.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.673+0000 7fd3c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:20:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.674+0000 7fd3c3fff700 1 -- 172.21.15.19:0/467263347 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc013da0 con 0x55ca9199a400 2024-05-09T02:20:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.675+0000 7fd3c3fff700 1 -- 172.21.15.19:0/467263347 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fd3dc014420 con 0x55ca9199a400 2024-05-09T02:20:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.675+0000 7fd3f952db80 1 -- 172.21.15.19:0/467263347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3dc019070 con 0x55ca9199a400 2024-05-09T02:20:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.675+0000 7fd3c3fff700 1 -- 172.21.15.19:0/467263347 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fd3dc0133f0 con 0x55ca9199a400 2024-05-09T02:20:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.675+0000 7fd3f952db80 10 client.16461 fetch_fsmap learned FSMap version 54 2024-05-09T02:20:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.675+0000 7fd3f952db80 1 -- 172.21.15.19:0/467263347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55ca919b19c0 con 0x55ca9199a400 2024-05-09T02:20:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.675+0000 7fd3c3fff700 1 -- 172.21.15.19:0/467263347 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3dc014ad0 con 0x55ca9199a400 2024-05-09T02:20:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.675+0000 7fd3c3fff700 1 -- 172.21.15.19:0/467263347 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7fd3dc02f9e0 con 0x55ca9199a400 2024-05-09T02:20:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.675+0000 7fd3f952db80 10 client.16461 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:20:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.675+0000 7fd3f952db80 10 client.16461 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:20:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.675+0000 7fd3f952db80 20 client.16461 populate_metadata read hostname 'smithi019' 2024-05-09T02:20:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.675+0000 7fd3f952db80 1 --2- 172.21.15.19:0/467263347 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ca919b4000 0x55ca919b6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.675+0000 7fd3f952db80 4 client.16461 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:20:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.675+0000 7fd3f952db80 1 -- 172.21.15.19:0/467263347 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd3dc019070 con 0x55ca919b4000 2024-05-09T02:20:55.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.676+0000 7fd3e88b3700 1 --2- 172.21.15.19:0/467263347 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ca919b4000 0x55ca919b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.676+0000 7fd3c3fff700 10 client.16461 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:20:55.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.676+0000 7fd3e88b3700 1 --2- 172.21.15.19:0/467263347 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ca919b4000 0x55ca919b6420 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=219c884350fa5224 server_cookie=b1b6eb197930c55a in_seq=0 out_seq=0 2024-05-09T02:20:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.677+0000 7fd3c3fff700 1 -- 172.21.15.19:0/467263347 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fd3dc019070 con 0x55ca919b4000 2024-05-09T02:20:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.677+0000 7fd3c3fff700 10 client.16461 handle_command_reply: tid=0 2024-05-09T02:20:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.677+0000 7fd3f952db80 1 client.16461 shutdown 2024-05-09T02:20:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.678+0000 7fd3f952db80 1 -- 172.21.15.19:0/467263347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9199a400 msgr2=0x55ca919a4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.678+0000 7fd3f952db80 1 --2- 172.21.15.19:0/467263347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9199a400 0x55ca919a4a70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00a9c0 tx=0x7fd3dc013980 comp rx=0 tx=0).stop 2024-05-09T02:20:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.678+0000 7fd3f952db80 1 -- 172.21.15.19:0/467263347 shutdown_connections 2024-05-09T02:20:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.678+0000 7fd3f952db80 1 --2- 172.21.15.19:0/467263347 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ca919b4000 0x55ca919b6420 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.678+0000 7fd3f952db80 1 --2- 172.21.15.19:0/467263347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9199a400 0x55ca919a4a70 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.678+0000 7fd3f952db80 1 --2- 172.21.15.19:0/467263347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9181d4a0 0x55ca919a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.678+0000 7fd3f952db80 1 --2- 172.21.15.19:0/467263347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 0x55ca919a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.678+0000 7fd3c3fff700 0 client.16461 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:20:55.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.678+0000 7fd3f952db80 1 -- 172.21.15.19:0/467263347 >> 172.21.15.19:0/467263347 conn(0x55ca91820f70 msgr2=0x55ca9198eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:55.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.678+0000 7fd3f952db80 1 -- 172.21.15.19:0/467263347 shutdown_connections 2024-05-09T02:20:55.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.678+0000 7fd3f952db80 1 -- 172.21.15.19:0/467263347 wait complete. 2024-05-09T02:20:55.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.678+0000 7fd3f952db80 20 client.16461 trim_cache size 0 max 16384 2024-05-09T02:20:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.697+0000 7fd3f952db80 1 Processor -- start 2024-05-09T02:20:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.697+0000 7fd3f952db80 1 -- start start 2024-05-09T02:20:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.697+0000 7fd3f952db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9181d4a0 0x55ca919a6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3f952db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 0x55ca9198f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3f952db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9199a400 0x55ca919afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3f952db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e41acd40 con 0x55ca9199a400 2024-05-09T02:20:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3f952db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e41ae9d0 con 0x55ca91897d00 2024-05-09T02:20:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3f952db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e41ad410 con 0x55ca9181d4a0 2024-05-09T02:20:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3e88b3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9181d4a0 0x55ca919a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3e88b3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9181d4a0 0x55ca919a6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:52176/0 (socket says 172.21.15.19:52176) 2024-05-09T02:20:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3e88b3700 1 -- 172.21.15.19:0/2371378782 learned_addr learned my addr 172.21.15.19:0/2371378782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3e90b4700 1 --2- 172.21.15.19:0/2371378782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9199a400 0x55ca919afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3e3fff700 1 --2- 172.21.15.19:0/2371378782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 0x55ca9198f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3e88b3700 1 -- 172.21.15.19:0/2371378782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 msgr2=0x55ca9198f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3e88b3700 1 --2- 172.21.15.19:0/2371378782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 0x55ca9198f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3e88b3700 1 -- 172.21.15.19:0/2371378782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9199a400 msgr2=0x55ca919afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3e88b3700 1 --2- 172.21.15.19:0/2371378782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9199a400 0x55ca919afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3e88b3700 1 -- 172.21.15.19:0/2371378782 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca919a3dc0 con 0x55ca9181d4a0 2024-05-09T02:20:55.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.698+0000 7fd3e88b3700 1 --2- 172.21.15.19:0/2371378782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9181d4a0 0x55ca919a6490 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8005f50 tx=0x7fd3d8033000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f74867b108015bf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:55.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.699+0000 7fd3ce7fc700 1 -- 172.21.15.19:0/2371378782 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d804c070 con 0x55ca9181d4a0 2024-05-09T02:20:55.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.699+0000 7fd3ce7fc700 1 -- 172.21.15.19:0/2371378782 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d80039a0 con 0x55ca9181d4a0 2024-05-09T02:20:55.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.699+0000 7fd3ce7fc700 1 -- 172.21.15.19:0/2371378782 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80056d0 con 0x55ca9181d4a0 2024-05-09T02:20:55.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.700+0000 7fd3f952db80 1 -- 172.21.15.19:0/2371378782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9181d4a0 msgr2=0x55ca919a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:55.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.700+0000 7fd3f952db80 1 --2- 172.21.15.19:0/2371378782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9181d4a0 0x55ca919a6490 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8005f50 tx=0x7fd3d8033000 comp rx=0 tx=0).stop 2024-05-09T02:20:55.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.700+0000 7fd3f952db80 1 -- 172.21.15.19:0/2371378782 shutdown_connections 2024-05-09T02:20:55.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.700+0000 7fd3f952db80 1 --2- 172.21.15.19:0/2371378782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9199a400 0x55ca919afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.700+0000 7fd3f952db80 1 --2- 172.21.15.19:0/2371378782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9181d4a0 0x55ca919a6490 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.700+0000 7fd3f952db80 1 --2- 172.21.15.19:0/2371378782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 0x55ca9198f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.700+0000 7fd3f952db80 1 -- 172.21.15.19:0/2371378782 >> 172.21.15.19:0/2371378782 conn(0x55ca91820f70 msgr2=0x55ca9198eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:55.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.701+0000 7fd3f952db80 1 -- 172.21.15.19:0/2371378782 shutdown_connections 2024-05-09T02:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.701+0000 7fd3f952db80 1 -- 172.21.15.19:0/2371378782 wait complete. 2024-05-09T02:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.701+0000 7fd3f952db80 1 -- start start 2024-05-09T02:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.702+0000 7fd3f952db80 1 Processor -- start 2024-05-09T02:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.703+0000 7fd3f952db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9181d4a0 0x55ca919bbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.703+0000 7fd3f952db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 0x55ca919baa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.703+0000 7fd3f952db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9199a400 0x55ca919baf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.703+0000 7fd3e88b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9181d4a0 0x55ca919bbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.703+0000 7fd3e88b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9181d4a0 0x55ca919bbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33016/0 (socket says 172.21.15.19:33016) 2024-05-09T02:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.704+0000 7fd3e88b3700 1 -- 172.21.15.19:0/2235747267 learned_addr learned my addr 172.21.15.19:0/2235747267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.704+0000 7fd3f952db80 1 -- 172.21.15.19:0/2235747267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e41ae270 con 0x55ca9181d4a0 2024-05-09T02:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.704+0000 7fd3f952db80 1 -- 172.21.15.19:0/2235747267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e41adae0 con 0x55ca91897d00 2024-05-09T02:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.704+0000 7fd3f952db80 1 -- 172.21.15.19:0/2235747267 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e41aad00 con 0x55ca9199a400 2024-05-09T02:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.704+0000 7fd3e3fff700 1 --2- 172.21.15.19:0/2235747267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 0x55ca919baa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.704+0000 7fd3f952db80 10 client.0 _ll_register_callbacks cb 0x55ca9172e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.704+0000 7fd3e90b4700 1 --2- 172.21.15.19:0/2235747267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9199a400 0x55ca919baf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.704+0000 7fd3e88b3700 1 -- 172.21.15.19:0/2235747267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9199a400 msgr2=0x55ca919baf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.704+0000 7fd3e88b3700 1 --2- 172.21.15.19:0/2235747267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9199a400 0x55ca919baf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.704+0000 7fd3e88b3700 1 -- 172.21.15.19:0/2235747267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 msgr2=0x55ca919baa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.704+0000 7fd3e88b3700 1 --2- 172.21.15.19:0/2235747267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 0x55ca919baa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.704+0000 7fd3e88b3700 1 -- 172.21.15.19:0/2235747267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca919bb490 con 0x55ca9181d4a0 2024-05-09T02:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.704+0000 7fd3e3fff700 1 --2- 172.21.15.19:0/2235747267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 0x55ca919baa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.704+0000 7fd3e88b3700 1 --2- 172.21.15.19:0/2235747267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9181d4a0 0x55ca919bbd20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8000c00 tx=0x7fd3d8005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34e844b8268ac08 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.704+0000 7fd3c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.705+0000 7fd3c3fff700 1 -- 172.21.15.19:0/2235747267 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d804c070 con 0x55ca9181d4a0 2024-05-09T02:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.705+0000 7fd3f952db80 1 -- 172.21.15.19:0/2235747267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3d80039a0 con 0x55ca9181d4a0 2024-05-09T02:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.705+0000 7fd3c3fff700 1 -- 172.21.15.19:0/2235747267 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fd3d8034b00 con 0x55ca9181d4a0 2024-05-09T02:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.705+0000 7fd3c3fff700 1 -- 172.21.15.19:0/2235747267 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fd3d8034d00 con 0x55ca9181d4a0 2024-05-09T02:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.705+0000 7fd3f952db80 10 client.16467 fetch_fsmap learned FSMap version 54 2024-05-09T02:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.705+0000 7fd3f952db80 1 -- 172.21.15.19:0/2235747267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55ca91a1d230 con 0x55ca9181d4a0 2024-05-09T02:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.705+0000 7fd3c3fff700 1 -- 172.21.15.19:0/2235747267 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d8046170 con 0x55ca9181d4a0 2024-05-09T02:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.705+0000 7fd3c3fff700 1 -- 172.21.15.19:0/2235747267 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7fd3d8057070 con 0x55ca9181d4a0 2024-05-09T02:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.705+0000 7fd3f952db80 10 client.16467 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.706+0000 7fd3f952db80 10 client.16467 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.706+0000 7fd3f952db80 20 client.16467 populate_metadata read hostname 'smithi019' 2024-05-09T02:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.706+0000 7fd3f952db80 1 --2- 172.21.15.19:0/2235747267 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ca91a1d770 0x55ca91a1fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.706+0000 7fd3f952db80 4 client.16467 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.706+0000 7fd3f952db80 1 -- 172.21.15.19:0/2235747267 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd3d80039a0 con 0x55ca91a1d770 2024-05-09T02:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.706+0000 7fd3e3fff700 1 --2- 172.21.15.19:0/2235747267 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ca91a1d770 0x55ca91a1fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.706+0000 7fd3e3fff700 1 --2- 172.21.15.19:0/2235747267 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ca91a1d770 0x55ca91a1fb90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=13ac4249d5b75d39 server_cookie=3e6582d0fb4986b7 in_seq=0 out_seq=0 2024-05-09T02:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.706+0000 7fd3c3fff700 10 client.16467 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.706+0000 7fd3c3fff700 1 -- 172.21.15.19:0/2235747267 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd3d80039a0 con 0x55ca91a1d770 2024-05-09T02:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.706+0000 7fd3c3fff700 10 client.16467 handle_command_reply: tid=0 2024-05-09T02:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.707+0000 7fd3f952db80 1 client.16467 shutdown 2024-05-09T02:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.707+0000 7fd3f952db80 1 -- 172.21.15.19:0/2235747267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9181d4a0 msgr2=0x55ca919bbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.707+0000 7fd3f952db80 1 --2- 172.21.15.19:0/2235747267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9181d4a0 0x55ca919bbd20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8000c00 tx=0x7fd3d8005160 comp rx=0 tx=0).stop 2024-05-09T02:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.707+0000 7fd3f952db80 1 -- 172.21.15.19:0/2235747267 shutdown_connections 2024-05-09T02:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.707+0000 7fd3f952db80 1 --2- 172.21.15.19:0/2235747267 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ca91a1d770 0x55ca91a1fb90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.707+0000 7fd3f952db80 1 --2- 172.21.15.19:0/2235747267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca9199a400 0x55ca919baf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.707+0000 7fd3f952db80 1 --2- 172.21.15.19:0/2235747267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca9181d4a0 0x55ca919bbd20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.707+0000 7fd3f952db80 1 --2- 172.21.15.19:0/2235747267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca91897d00 0x55ca919baa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.707+0000 7fd3f952db80 1 -- 172.21.15.19:0/2235747267 >> 172.21.15.19:0/2235747267 conn(0x55ca91820f70 msgr2=0x55ca919a27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.707+0000 7fd3c3fff700 0 client.16467 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.707+0000 7fd3f952db80 1 -- 172.21.15.19:0/2235747267 shutdown_connections 2024-05-09T02:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.707+0000 7fd3f952db80 1 -- 172.21.15.19:0/2235747267 wait complete. 2024-05-09T02:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.708+0000 7fd3f952db80 20 client.16467 trim_cache size 0 max 16384 2024-05-09T02:20:55.715 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:20:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.708+0000 7fd3eab17700 1 -- 172.21.15.19:0/2581615855 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fd3d40b2af0 msgr2=0x7fd3d40b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.708+0000 7fd3eab17700 1 --2- 172.21.15.19:0/2581615855 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fd3d40b2af0 0x7fd3d40b5000 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd3d801c5d0 tx=0x7fd3d802a040 comp rx=0 tx=0).stop 2024-05-09T02:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.709+0000 7fd3eab17700 1 -- 172.21.15.19:0/2581615855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e4103ba0 msgr2=0x7fd3e4113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.709+0000 7fd3eab17700 1 --2- 172.21.15.19:0/2581615855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e4103ba0 0x7fd3e4113980 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000b550 tx=0x7fd3d000b860 comp rx=0 tx=0).stop 2024-05-09T02:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.709+0000 7fd3eab17700 1 -- 172.21.15.19:0/2581615855 shutdown_connections 2024-05-09T02:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.709+0000 7fd3eab17700 1 --2- 172.21.15.19:0/2581615855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3e41086a0 0x7fd3e4113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.709+0000 7fd3eab17700 1 --2- 172.21.15.19:0/2581615855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4103250 0x7fd3e4117840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.709+0000 7fd3eab17700 1 --2- 172.21.15.19:0/2581615855 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fd3d40b2af0 0x7fd3d40b5000 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.709+0000 7fd3eab17700 1 --2- 172.21.15.19:0/2581615855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3e4103ba0 0x7fd3e4113980 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.709+0000 7fd3eab17700 1 -- 172.21.15.19:0/2581615855 >> 172.21.15.19:0/2581615855 conn(0x7fd3e40fedc0 msgr2=0x7fd3e4106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:55.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.709+0000 7fd3eab17700 1 -- 172.21.15.19:0/2581615855 shutdown_connections 2024-05-09T02:20:55.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:55.709+0000 7fd3eab17700 1 -- 172.21.15.19:0/2581615855 wait complete. 2024-05-09T02:20:56.538 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:56 smithi019 ceph-mon[30151]: pgmap v298: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.9 MiB/s wr, 812 op/s 2024-05-09T02:20:56.578 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T02:20:56.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:56 smithi059 ceph-mon[34781]: pgmap v298: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.9 MiB/s wr, 812 op/s 2024-05-09T02:20:56.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:56 smithi196 ceph-mon[34718]: pgmap v298: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.9 MiB/s wr, 812 op/s 2024-05-09T02:20:56.959 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:20:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 555.0 s, 1423.8 tps, lat 70.605 ms stddev 76.869 2024-05-09T02:20:58.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:20:58 smithi059 ceph-mon[34781]: pgmap v299: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.2 MiB/s wr, 549 op/s 2024-05-09T02:20:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:20:58 smithi196 ceph-mon[34718]: pgmap v299: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.2 MiB/s wr, 549 op/s 2024-05-09T02:20:58.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:20:58 smithi019 ceph-mon[30151]: pgmap v299: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.2 MiB/s wr, 549 op/s 2024-05-09T02:20:59.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.248+0000 7f047dbb2700 1 -- 172.21.15.19:0/2691454801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0478108740 msgr2=0x7f04781100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:59.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.248+0000 7f047dbb2700 1 --2- 172.21.15.19:0/2691454801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0478108740 0x7f04781100a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f047000b0d0 tx=0x7f047000b3e0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.248+0000 7f047dbb2700 1 -- 172.21.15.19:0/2691454801 shutdown_connections 2024-05-09T02:20:59.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.248+0000 7f047dbb2700 1 --2- 172.21.15.19:0/2691454801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0478108740 0x7f04781100a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.248+0000 7f047dbb2700 1 --2- 172.21.15.19:0/2691454801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04781034c0 0x7f04781038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.248+0000 7f047dbb2700 1 --2- 172.21.15.19:0/2691454801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0478107df0 0x7f0478108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.248+0000 7f047dbb2700 1 -- 172.21.15.19:0/2691454801 >> 172.21.15.19:0/2691454801 conn(0x7f04780fedc0 msgr2=0x7f04781011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:59.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.248+0000 7f047dbb2700 1 -- 172.21.15.19:0/2691454801 shutdown_connections 2024-05-09T02:20:59.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.248+0000 7f047dbb2700 1 -- 172.21.15.19:0/2691454801 wait complete. 2024-05-09T02:20:59.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.249+0000 7f047dbb2700 1 Processor -- start 2024-05-09T02:20:59.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.249+0000 7f047dbb2700 1 -- start start 2024-05-09T02:20:59.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f047dbb2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04781034c0 0x7f047810a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f047dbb2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0478107df0 0x7f047810a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f047dbb2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0478108740 0x7f047810aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f047dbb2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0478113370 con 0x7f04781034c0 2024-05-09T02:20:59.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f047dbb2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04781131f0 con 0x7f0478107df0 2024-05-09T02:20:59.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f047dbb2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04781134f0 con 0x7f0478108740 2024-05-09T02:20:59.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f0476ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0478107df0 0x7f047810a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f04777fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04781034c0 0x7f047810a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f0476ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0478107df0 0x7f047810a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:59538/0 (socket says 172.21.15.19:59538) 2024-05-09T02:20:59.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f04777fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04781034c0 0x7f047810a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37706/0 (socket says 172.21.15.19:37706) 2024-05-09T02:20:59.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f0476ffd700 1 -- 172.21.15.19:0/3279037492 learned_addr learned my addr 172.21.15.19:0/3279037492 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:59.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f0477fff700 1 --2- 172.21.15.19:0/3279037492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0478108740 0x7f047810aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f0476ffd700 1 -- 172.21.15.19:0/3279037492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0478108740 msgr2=0x7f047810aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:59.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f0476ffd700 1 --2- 172.21.15.19:0/3279037492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0478108740 0x7f047810aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f0476ffd700 1 -- 172.21.15.19:0/3279037492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04781034c0 msgr2=0x7f047810a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:59.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f0476ffd700 1 --2- 172.21.15.19:0/3279037492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04781034c0 0x7f047810a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f0476ffd700 1 -- 172.21.15.19:0/3279037492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04781ad560 con 0x7f0478107df0 2024-05-09T02:20:59.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f0477fff700 1 --2- 172.21.15.19:0/3279037492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0478108740 0x7f047810aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:20:59.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.250+0000 7f04777fe700 1 --2- 172.21.15.19:0/3279037492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04781034c0 0x7f047810a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:20:59.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.251+0000 7f0476ffd700 1 --2- 172.21.15.19:0/3279037492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0478107df0 0x7f047810a660 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f046c00e970 tx=0x7f046c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:59.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.251+0000 7f0474ff9700 1 -- 172.21.15.19:0/3279037492 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c004a30 con 0x7f0478107df0 2024-05-09T02:20:59.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.251+0000 7f0474ff9700 1 -- 172.21.15.19:0/3279037492 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f046c00cc90 con 0x7f0478107df0 2024-05-09T02:20:59.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.251+0000 7f0474ff9700 1 -- 172.21.15.19:0/3279037492 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c005610 con 0x7f0478107df0 2024-05-09T02:20:59.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.251+0000 7f047dbb2700 1 -- 172.21.15.19:0/3279037492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04781ad7f0 con 0x7f0478107df0 2024-05-09T02:20:59.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.251+0000 7f047dbb2700 1 -- 172.21.15.19:0/3279037492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04781adce0 con 0x7f0478107df0 2024-05-09T02:20:59.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.254+0000 7f0474ff9700 1 -- 172.21.15.19:0/3279037492 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f046c021440 con 0x7f0478107df0 2024-05-09T02:20:59.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.254+0000 7f0474ff9700 1 --2- 172.21.15.19:0/3279037492 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f04600b2a70 0x7f04600b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.255+0000 7f04777fe700 1 --2- 172.21.15.19:0/3279037492 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f04600b2a70 0x7f04600b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.255+0000 7f0474ff9700 1 -- 172.21.15.19:0/3279037492 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f046c0e26f0 con 0x7f0478107df0 2024-05-09T02:20:59.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.255+0000 7f04777fe700 1 --2- 172.21.15.19:0/3279037492 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f04600b2a70 0x7f04600b4f80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0468005950 tx=0x7f046800b410 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:59.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.258+0000 7f048c5c8b80 1 Processor -- start 2024-05-09T02:20:59.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.258+0000 7f048c5c8b80 1 -- start start 2024-05-09T02:20:59.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.259+0000 7f048c5c8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2bafd00 0x559ca2cb8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.259+0000 7f048c5c8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2b354a0 0x559ca2cb8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.259+0000 7f048c5c8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2cb2400 0x559ca2cb2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.259+0000 7f048c5c8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559ca2bd50a0 con 0x559ca2bafd00 2024-05-09T02:20:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.259+0000 7f048c5c8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559ca2b7a9a0 con 0x559ca2cb2400 2024-05-09T02:20:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.259+0000 7f048c5c8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559ca2bd1060 con 0x559ca2b354a0 2024-05-09T02:20:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.259+0000 7f0477fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2cb2400 0x559ca2cb2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.259+0000 7f0477fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2cb2400 0x559ca2cb2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:59544/0 (socket says 172.21.15.19:59544) 2024-05-09T02:20:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.259+0000 7f0477fff700 1 -- 172.21.15.19:0/2092952137 learned_addr learned my addr 172.21.15.19:0/2092952137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.259+0000 7f0476ffd700 1 --2- 172.21.15.19:0/2092952137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2b354a0 0x559ca2cb8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.259+0000 7f04777fe700 1 --2- 172.21.15.19:0/2092952137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2bafd00 0x559ca2cb8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.259+0000 7f0477fff700 1 -- 172.21.15.19:0/2092952137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2b354a0 msgr2=0x559ca2cb8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.259+0000 7f0477fff700 1 --2- 172.21.15.19:0/2092952137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2b354a0 0x559ca2cb8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.259+0000 7f0477fff700 1 -- 172.21.15.19:0/2092952137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2bafd00 msgr2=0x559ca2cb8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.259+0000 7f0477fff700 1 --2- 172.21.15.19:0/2092952137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2bafd00 0x559ca2cb8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.260+0000 7f0477fff700 1 -- 172.21.15.19:0/2092952137 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ca2b6b500 con 0x559ca2cb2400 2024-05-09T02:20:59.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.260+0000 7f04777fe700 1 --2- 172.21.15.19:0/2092952137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2bafd00 0x559ca2cb8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:20:59.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.260+0000 7f0477fff700 1 --2- 172.21.15.19:0/2092952137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2cb2400 0x559ca2cb2810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0470005ba0 tx=0x7f0470009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c704daab0355341 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:59.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.260+0000 7f045dffb700 1 -- 172.21.15.19:0/2092952137 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047000be50 con 0x559ca2cb2400 2024-05-09T02:20:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.260+0000 7f045dffb700 1 -- 172.21.15.19:0/2092952137 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04700094e0 con 0x559ca2cb2400 2024-05-09T02:20:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.260+0000 7f045dffb700 1 -- 172.21.15.19:0/2092952137 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04700075c0 con 0x559ca2cb2400 2024-05-09T02:20:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.261+0000 7f048c5c8b80 1 -- 172.21.15.19:0/2092952137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2cb2400 msgr2=0x559ca2cb2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.261+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/2092952137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2cb2400 0x559ca2cb2810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0470005ba0 tx=0x7f0470009b80 comp rx=0 tx=0).stop 2024-05-09T02:20:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.261+0000 7f048c5c8b80 1 -- 172.21.15.19:0/2092952137 shutdown_connections 2024-05-09T02:20:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.261+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/2092952137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2cb2400 0x559ca2cb2810 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.261+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/2092952137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2bafd00 0x559ca2cb8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.261+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/2092952137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2b354a0 0x559ca2cb8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.261+0000 7f048c5c8b80 1 -- 172.21.15.19:0/2092952137 >> 172.21.15.19:0/2092952137 conn(0x559ca2b38f70 msgr2=0x559ca2cad220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.262+0000 7f048c5c8b80 1 -- 172.21.15.19:0/2092952137 shutdown_connections 2024-05-09T02:20:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.262+0000 7f048c5c8b80 1 -- 172.21.15.19:0/2092952137 wait complete. 2024-05-09T02:20:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.262+0000 7f048c5c8b80 1 -- start start 2024-05-09T02:20:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.263+0000 7f048c5c8b80 1 Processor -- start 2024-05-09T02:20:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.263+0000 7f048c5c8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2b354a0 0x559ca2cbc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.263+0000 7f048c5c8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2bafd00 0x559ca2cbc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.263+0000 7f048c5c8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2cb2400 0x559ca2cbcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.263+0000 7f048c5c8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559ca2cb3100 con 0x559ca2cb2400 2024-05-09T02:20:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.263+0000 7f048c5c8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559ca2b6fb20 con 0x559ca2b354a0 2024-05-09T02:20:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.263+0000 7f048c5c8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559ca2cc8200 con 0x559ca2bafd00 2024-05-09T02:20:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.264+0000 7f048c5c8b80 10 client.0 _ll_register_callbacks cb 0x559ca2a466c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:20:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.264+0000 7f04777fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2b354a0 0x559ca2cbc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.264+0000 7f0477fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2cb2400 0x559ca2cbcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.264+0000 7f0477fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2cb2400 0x559ca2cbcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37728/0 (socket says 172.21.15.19:37728) 2024-05-09T02:20:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.264+0000 7f0476ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2bafd00 0x559ca2cbc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.264+0000 7f0477fff700 1 -- 172.21.15.19:0/2659000967 learned_addr learned my addr 172.21.15.19:0/2659000967 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.264+0000 7f04777fe700 1 -- 172.21.15.19:0/2659000967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2bafd00 msgr2=0x559ca2cbc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.264+0000 7f04777fe700 1 --2- 172.21.15.19:0/2659000967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2bafd00 0x559ca2cbc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.264+0000 7f04777fe700 1 -- 172.21.15.19:0/2659000967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2cb2400 msgr2=0x559ca2cbcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.264+0000 7f04777fe700 1 --2- 172.21.15.19:0/2659000967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2cb2400 0x559ca2cbcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.264+0000 7f04777fe700 1 -- 172.21.15.19:0/2659000967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ca2a50490 con 0x559ca2b354a0 2024-05-09T02:20:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.265+0000 7f04777fe700 1 --2- 172.21.15.19:0/2659000967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2b354a0 0x559ca2cbc0b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0468010c00 tx=0x7f04680047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a83cf909d52965c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.265+0000 7f0456ffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:20:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.265+0000 7f0456ffd700 1 -- 172.21.15.19:0/2659000967 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468004a30 con 0x559ca2b354a0 2024-05-09T02:20:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.265+0000 7f0456ffd700 1 -- 172.21.15.19:0/2659000967 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f0468004bd0 con 0x559ca2b354a0 2024-05-09T02:20:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.265+0000 7f048c5c8b80 1 -- 172.21.15.19:0/2659000967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04700094e0 con 0x559ca2b354a0 2024-05-09T02:20:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.265+0000 7f0456ffd700 1 -- 172.21.15.19:0/2659000967 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0468004e10 con 0x559ca2b354a0 2024-05-09T02:20:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.265+0000 7f048c5c8b80 10 client.26213 fetch_fsmap learned FSMap version 54 2024-05-09T02:20:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.265+0000 7f048c5c8b80 1 -- 172.21.15.19:0/2659000967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559ca2cc9ba0 con 0x559ca2b354a0 2024-05-09T02:20:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.265+0000 7f0456ffd700 1 -- 172.21.15.19:0/2659000967 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04700094e0 con 0x559ca2b354a0 2024-05-09T02:20:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.265+0000 7f0456ffd700 1 -- 172.21.15.19:0/2659000967 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f046801ead0 con 0x559ca2b354a0 2024-05-09T02:20:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.266+0000 7f048c5c8b80 10 client.26213 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:20:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.266+0000 7f048c5c8b80 10 client.26213 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:20:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.266+0000 7f048c5c8b80 20 client.26213 populate_metadata read hostname 'smithi019' 2024-05-09T02:20:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.266+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/2659000967 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559ca2ccc120 0x559ca2cce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.266+0000 7f048c5c8b80 4 client.26213 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:20:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.266+0000 7f048c5c8b80 1 -- 172.21.15.19:0/2659000967 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ca2be2a80 con 0x559ca2ccc120 2024-05-09T02:20:59.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.266+0000 7f0476ffd700 1 --2- 172.21.15.19:0/2659000967 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559ca2ccc120 0x559ca2cce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.267+0000 7f0476ffd700 1 --2- 172.21.15.19:0/2659000967 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559ca2ccc120 0x559ca2cce580 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ebe919a6798561c0 server_cookie=7e2b61ceab98fc80 in_seq=0 out_seq=0 2024-05-09T02:20:59.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.267+0000 7f0456ffd700 10 client.26213 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:20:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.267+0000 7f0456ffd700 1 -- 172.21.15.19:0/2659000967 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559ca2be2a80 con 0x559ca2ccc120 2024-05-09T02:20:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.267+0000 7f0456ffd700 10 client.26213 handle_command_reply: tid=0 2024-05-09T02:20:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.267+0000 7f048c5c8b80 1 client.26213 shutdown 2024-05-09T02:20:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.268+0000 7f048c5c8b80 1 -- 172.21.15.19:0/2659000967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2b354a0 msgr2=0x559ca2cbc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.268+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/2659000967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2b354a0 0x559ca2cbc0b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0468010c00 tx=0x7f04680047c0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.268+0000 7f048c5c8b80 1 -- 172.21.15.19:0/2659000967 shutdown_connections 2024-05-09T02:20:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.268+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/2659000967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2cb2400 0x559ca2cbcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.268+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/2659000967 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559ca2ccc120 0x559ca2cce580 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.268+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/2659000967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2b354a0 0x559ca2cbc0b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.268+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/2659000967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2bafd00 0x559ca2cbc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.268+0000 7f0456ffd700 0 client.26213 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:20:59.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.268+0000 7f048c5c8b80 1 -- 172.21.15.19:0/2659000967 >> 172.21.15.19:0/2659000967 conn(0x559ca2b38f70 msgr2=0x559ca2ca6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:59.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.268+0000 7f048c5c8b80 1 -- 172.21.15.19:0/2659000967 shutdown_connections 2024-05-09T02:20:59.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.268+0000 7f048c5c8b80 1 -- 172.21.15.19:0/2659000967 wait complete. 2024-05-09T02:20:59.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.268+0000 7f048c5c8b80 20 client.26213 trim_cache size 0 max 16384 2024-05-09T02:20:59.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.297+0000 7f048c5c8b80 1 Processor -- start 2024-05-09T02:20:59.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.297+0000 7f048c5c8b80 1 -- start start 2024-05-09T02:20:59.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.298+0000 7f048c5c8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2b354a0 0x559ca2ca7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.298+0000 7f048c5c8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2bafd00 0x559ca2cd3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.298+0000 7f048c5c8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2cb2400 0x559ca2cd42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.298+0000 7f048c5c8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b5400 con 0x559ca2bafd00 2024-05-09T02:20:59.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.298+0000 7f048c5c8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b7090 con 0x559ca2cb2400 2024-05-09T02:20:59.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.298+0000 7f048c5c8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b5ad0 con 0x559ca2b354a0 2024-05-09T02:20:59.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.299+0000 7f0476ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2bafd00 0x559ca2cd3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.299+0000 7f0476ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2bafd00 0x559ca2cd3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37730/0 (socket says 172.21.15.19:37730) 2024-05-09T02:20:59.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.299+0000 7f0476ffd700 1 -- 172.21.15.19:0/4028097943 learned_addr learned my addr 172.21.15.19:0/4028097943 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:59.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.300+0000 7f0477fff700 1 --2- 172.21.15.19:0/4028097943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2cb2400 0x559ca2cd42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.300+0000 7f04777fe700 1 --2- 172.21.15.19:0/4028097943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2b354a0 0x559ca2ca7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.300+0000 7f0476ffd700 1 -- 172.21.15.19:0/4028097943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2b354a0 msgr2=0x559ca2ca7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.300+0000 7f0476ffd700 1 --2- 172.21.15.19:0/4028097943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2b354a0 0x559ca2ca7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.300+0000 7f0476ffd700 1 -- 172.21.15.19:0/4028097943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2cb2400 msgr2=0x559ca2cd42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.300+0000 7f0476ffd700 1 --2- 172.21.15.19:0/4028097943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2cb2400 0x559ca2cd42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.300+0000 7f0476ffd700 1 -- 172.21.15.19:0/4028097943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ca2cbbe80 con 0x559ca2bafd00 2024-05-09T02:20:59.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.303+0000 7f0476ffd700 1 --2- 172.21.15.19:0/4028097943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2bafd00 0x559ca2cd3d80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f046c0dfe70 tx=0x7f046c017fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e231435417b52c6f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:59.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.311+0000 7f045d7fa700 1 -- 172.21.15.19:0/4028097943 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c0b30c0 con 0x559ca2bafd00 2024-05-09T02:20:59.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.311+0000 7f045d7fa700 1 -- 172.21.15.19:0/4028097943 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f046c0aa430 con 0x559ca2bafd00 2024-05-09T02:20:59.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.311+0000 7f045d7fa700 1 -- 172.21.15.19:0/4028097943 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c0a6bb0 con 0x559ca2bafd00 2024-05-09T02:20:59.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.313+0000 7f048c5c8b80 1 -- 172.21.15.19:0/4028097943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2bafd00 msgr2=0x559ca2cd3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:59.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.313+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/4028097943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2bafd00 0x559ca2cd3d80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f046c0dfe70 tx=0x7f046c017fc0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.320+0000 7f048c5c8b80 1 -- 172.21.15.19:0/4028097943 shutdown_connections 2024-05-09T02:20:59.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.320+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/4028097943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2cb2400 0x559ca2cd42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.320+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/4028097943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2b354a0 0x559ca2ca7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.320+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/4028097943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2bafd00 0x559ca2cd3d80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.321+0000 7f048c5c8b80 1 -- 172.21.15.19:0/4028097943 >> 172.21.15.19:0/4028097943 conn(0x559ca2b38f70 msgr2=0x559ca2ca6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.326+0000 7f048c5c8b80 1 -- 172.21.15.19:0/4028097943 shutdown_connections 2024-05-09T02:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.327+0000 7f048c5c8b80 1 -- 172.21.15.19:0/4028097943 wait complete. 2024-05-09T02:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.327+0000 7f048c5c8b80 1 -- start start 2024-05-09T02:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.328+0000 7f048c5c8b80 1 Processor -- start 2024-05-09T02:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.328+0000 7f048c5c8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2b354a0 0x559ca2cc4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.328+0000 7f048c5c8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2bafd00 0x559ca2cc19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.328+0000 7f048c5c8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2cb2400 0x559ca2cc1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.328+0000 7f048c5c8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b6930 con 0x559ca2cb2400 2024-05-09T02:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.328+0000 7f048c5c8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b61a0 con 0x559ca2b354a0 2024-05-09T02:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.328+0000 7f048c5c8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b33c0 con 0x559ca2bafd00 2024-05-09T02:20:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.328+0000 7f04777fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2b354a0 0x559ca2cc4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.328+0000 7f04777fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2b354a0 0x559ca2cc4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:59578/0 (socket says 172.21.15.19:59578) 2024-05-09T02:20:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.328+0000 7f04777fe700 1 -- 172.21.15.19:0/3740931646 learned_addr learned my addr 172.21.15.19:0/3740931646 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:20:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.328+0000 7f048c5c8b80 10 client.0 _ll_register_callbacks cb 0x559ca2a466c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:20:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.329+0000 7f0476ffd700 1 --2- 172.21.15.19:0/3740931646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2bafd00 0x559ca2cc19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.329+0000 7f04777fe700 1 -- 172.21.15.19:0/3740931646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2bafd00 msgr2=0x559ca2cc19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.329+0000 7f04777fe700 1 --2- 172.21.15.19:0/3740931646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2bafd00 0x559ca2cc19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.329+0000 7f04777fe700 1 -- 172.21.15.19:0/3740931646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2cb2400 msgr2=0x559ca2cc1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:20:59.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.329+0000 7f04777fe700 1 --2- 172.21.15.19:0/3740931646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2cb2400 0x559ca2cc1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.329+0000 7f04777fe700 1 -- 172.21.15.19:0/3740931646 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ca2cc2640 con 0x559ca2b354a0 2024-05-09T02:20:59.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.334+0000 7f04777fe700 1 --2- 172.21.15.19:0/3740931646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2b354a0 0x559ca2cc4650 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f046801f010 tx=0x7f04680048e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb01071c14f37758 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:20:59.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.336+0000 7f0456ffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:20:59.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.336+0000 7f0456ffd700 1 -- 172.21.15.19:0/3740931646 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468057070 con 0x559ca2b354a0 2024-05-09T02:20:59.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.336+0000 7f048c5c8b80 1 -- 172.21.15.19:0/3740931646 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f046c0aa430 con 0x559ca2b354a0 2024-05-09T02:20:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.338+0000 7f0456ffd700 1 -- 172.21.15.19:0/3740931646 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f0468028aa0 con 0x559ca2b354a0 2024-05-09T02:20:59.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.343+0000 7f0456ffd700 1 -- 172.21.15.19:0/3740931646 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0468028ca0 con 0x559ca2b354a0 2024-05-09T02:20:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.343+0000 7f048c5c8b80 10 client.26216 fetch_fsmap learned FSMap version 54 2024-05-09T02:20:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.343+0000 7f048c5c8b80 1 -- 172.21.15.19:0/3740931646 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559ca2d2d7a0 con 0x559ca2b354a0 2024-05-09T02:20:59.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.347+0000 7f0456ffd700 1 -- 172.21.15.19:0/3740931646 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f046c0aa430 con 0x559ca2b354a0 2024-05-09T02:20:59.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.347+0000 7f0456ffd700 1 -- 172.21.15.19:0/3740931646 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f046801e6b0 con 0x559ca2b354a0 2024-05-09T02:20:59.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.347+0000 7f048c5c8b80 10 client.26216 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:20:59.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.347+0000 7f048c5c8b80 10 client.26216 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:20:59.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.347+0000 7f048c5c8b80 20 client.26216 populate_metadata read hostname 'smithi019' 2024-05-09T02:20:59.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.347+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/3740931646 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559ca2d2df00 0x559ca2d30320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:20:59.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.347+0000 7f048c5c8b80 4 client.26216 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:20:59.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.347+0000 7f048c5c8b80 1 -- 172.21.15.19:0/3740931646 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ca2bdcb20 con 0x559ca2d2df00 2024-05-09T02:20:59.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.349+0000 7f0476ffd700 1 --2- 172.21.15.19:0/3740931646 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559ca2d2df00 0x559ca2d30320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:20:59.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.351+0000 7f0476ffd700 1 --2- 172.21.15.19:0/3740931646 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559ca2d2df00 0x559ca2d30320 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4e63f3c9ced03bc server_cookie=3496c257d907eeae in_seq=0 out_seq=0 2024-05-09T02:20:59.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.351+0000 7f0456ffd700 10 client.26216 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:20:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.352+0000 7f0456ffd700 1 -- 172.21.15.19:0/3740931646 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ca2bdcb20 con 0x559ca2d2df00 2024-05-09T02:20:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.352+0000 7f0456ffd700 10 client.26216 handle_command_reply: tid=0 2024-05-09T02:20:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.352+0000 7f048c5c8b80 1 client.26216 shutdown 2024-05-09T02:20:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.357+0000 7f048c5c8b80 1 -- 172.21.15.19:0/3740931646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2b354a0 msgr2=0x559ca2cc4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.357+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/3740931646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2b354a0 0x559ca2cc4650 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f046801f010 tx=0x7f04680048e0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.358+0000 7f048c5c8b80 1 -- 172.21.15.19:0/3740931646 shutdown_connections 2024-05-09T02:20:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.358+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/3740931646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ca2cb2400 0x559ca2cc1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.358+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/3740931646 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559ca2d2df00 0x559ca2d30320 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.358+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/3740931646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ca2b354a0 0x559ca2cc4650 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f046801f010 tx=0x7f04680048e0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.358+0000 7f048c5c8b80 1 --2- 172.21.15.19:0/3740931646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ca2bafd00 0x559ca2cc19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.358+0000 7f048c5c8b80 1 -- 172.21.15.19:0/3740931646 >> 172.21.15.19:0/3740931646 conn(0x559ca2b38f70 msgr2=0x559ca2cbc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.358+0000 7f0456ffd700 0 client.26216 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:20:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.361+0000 7f048c5c8b80 1 -- 172.21.15.19:0/3740931646 shutdown_connections 2024-05-09T02:20:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.362+0000 7f048c5c8b80 1 -- 172.21.15.19:0/3740931646 wait complete. 2024-05-09T02:20:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.362+0000 7f048c5c8b80 20 client.26216 trim_cache size 0 max 16384 2024-05-09T02:20:59.365 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:20:59.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.365+0000 7f047dbb2700 1 -- 172.21.15.19:0/3279037492 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f04600b2a70 msgr2=0x7f04600b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:59.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.365+0000 7f047dbb2700 1 --2- 172.21.15.19:0/3279037492 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f04600b2a70 0x7f04600b4f80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0468005950 tx=0x7f046800b410 comp rx=0 tx=0).stop 2024-05-09T02:20:59.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.365+0000 7f047dbb2700 1 -- 172.21.15.19:0/3279037492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0478107df0 msgr2=0x7f047810a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:20:59.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.365+0000 7f047dbb2700 1 --2- 172.21.15.19:0/3279037492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0478107df0 0x7f047810a660 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f046c00e970 tx=0x7f046c00ec80 comp rx=0 tx=0).stop 2024-05-09T02:20:59.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.366+0000 7f047dbb2700 1 -- 172.21.15.19:0/3279037492 shutdown_connections 2024-05-09T02:20:59.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.366+0000 7f047dbb2700 1 --2- 172.21.15.19:0/3279037492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0478108740 0x7f047810aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.366+0000 7f047dbb2700 1 --2- 172.21.15.19:0/3279037492 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f04600b2a70 0x7f04600b4f80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.366+0000 7f047dbb2700 1 --2- 172.21.15.19:0/3279037492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04781034c0 0x7f047810a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.366+0000 7f047dbb2700 1 --2- 172.21.15.19:0/3279037492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0478107df0 0x7f047810a660 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:20:59.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.366+0000 7f047dbb2700 1 -- 172.21.15.19:0/3279037492 >> 172.21.15.19:0/3279037492 conn(0x7f04780fedc0 msgr2=0x7f04781096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:20:59.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.369+0000 7f047dbb2700 1 -- 172.21.15.19:0/3279037492 shutdown_connections 2024-05-09T02:20:59.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:20:59.371+0000 7f047dbb2700 1 -- 172.21.15.19:0/3279037492 wait complete. 2024-05-09T02:21:00.036 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T02:21:00.348 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:21:00.715 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:00 smithi019 ceph-mon[30151]: pgmap v300: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.1 MiB/s wr, 759 op/s 2024-05-09T02:21:00.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:00 smithi059 ceph-mon[34781]: pgmap v300: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.1 MiB/s wr, 759 op/s 2024-05-09T02:21:00.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:00 smithi196 ceph-mon[34718]: pgmap v300: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.1 MiB/s wr, 759 op/s 2024-05-09T02:21:02.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.068+0000 7f0c71b9b700 1 -- 172.21.15.19:0/2259238251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c6c1045e0 msgr2=0x7f0c6c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:02.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.068+0000 7f0c71b9b700 1 --2- 172.21.15.19:0/2259238251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c6c1045e0 0x7f0c6c1049f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0099c0 tx=0x7f0c5c009cd0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.068+0000 7f0c71b9b700 1 -- 172.21.15.19:0/2259238251 shutdown_connections 2024-05-09T02:21:02.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.068+0000 7f0c71b9b700 1 --2- 172.21.15.19:0/2259238251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c6c109860 0x7f0c6c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.068+0000 7f0c71b9b700 1 --2- 172.21.15.19:0/2259238251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c6c1045e0 0x7f0c6c1049f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.068+0000 7f0c71b9b700 1 --2- 172.21.15.19:0/2259238251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c6c108f10 0x7f0c6c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.068+0000 7f0c71b9b700 1 -- 172.21.15.19:0/2259238251 >> 172.21.15.19:0/2259238251 conn(0x7f0c6c0ffe70 msgr2=0x7f0c6c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.068+0000 7f0c71b9b700 1 -- 172.21.15.19:0/2259238251 shutdown_connections 2024-05-09T02:21:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.068+0000 7f0c71b9b700 1 -- 172.21.15.19:0/2259238251 wait complete. 2024-05-09T02:21:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.069+0000 7f0c71b9b700 1 Processor -- start 2024-05-09T02:21:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.069+0000 7f0c71b9b700 1 -- start start 2024-05-09T02:21:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.069+0000 7f0c71b9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c6c1045e0 0x7f0c6c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.069+0000 7f0c71b9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c6c108f10 0x7f0c6c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c71b9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c6c109860 0x7f0c6c19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c71b9b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1144b0 con 0x7f0c6c109860 2024-05-09T02:21:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c71b9b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c114330 con 0x7f0c6c108f10 2024-05-09T02:21:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c71b9b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c114630 con 0x7f0c6c1045e0 2024-05-09T02:21:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c6b7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c6c1045e0 0x7f0c6c19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c6b7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c6c1045e0 0x7f0c6c19ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36254/0 (socket says 172.21.15.19:36254) 2024-05-09T02:21:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c6b7fe700 1 -- 172.21.15.19:0/2208829569 learned_addr learned my addr 172.21.15.19:0/2208829569 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c63fff700 1 --2- 172.21.15.19:0/2208829569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c6c108f10 0x7f0c6c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c6bfff700 1 --2- 172.21.15.19:0/2208829569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c6c109860 0x7f0c6c19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c6b7fe700 1 -- 172.21.15.19:0/2208829569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c6c108f10 msgr2=0x7f0c6c19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c6b7fe700 1 --2- 172.21.15.19:0/2208829569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c6c108f10 0x7f0c6c19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c6b7fe700 1 -- 172.21.15.19:0/2208829569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c6c109860 msgr2=0x7f0c6c19f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c6b7fe700 1 --2- 172.21.15.19:0/2208829569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c6c109860 0x7f0c6c19f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c6b7fe700 1 -- 172.21.15.19:0/2208829569 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c19fcb0 con 0x7f0c6c1045e0 2024-05-09T02:21:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c63fff700 1 --2- 172.21.15.19:0/2208829569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c6c108f10 0x7f0c6c19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:21:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.070+0000 7f0c6b7fe700 1 --2- 172.21.15.19:0/2208829569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c6c1045e0 0x7f0c6c19ade0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c006010 tx=0x7f0c5c005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:02.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.071+0000 7f0c697fa700 1 -- 172.21.15.19:0/2208829569 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c026070 con 0x7f0c6c1045e0 2024-05-09T02:21:02.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.071+0000 7f0c697fa700 1 -- 172.21.15.19:0/2208829569 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c5c005680 con 0x7f0c6c1045e0 2024-05-09T02:21:02.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.071+0000 7f0c71b9b700 1 -- 172.21.15.19:0/2208829569 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6c19ff40 con 0x7f0c6c1045e0 2024-05-09T02:21:02.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.071+0000 7f0c71b9b700 1 -- 172.21.15.19:0/2208829569 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c6c1a0420 con 0x7f0c6c1045e0 2024-05-09T02:21:02.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.071+0000 7f0c697fa700 1 -- 172.21.15.19:0/2208829569 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c00fb20 con 0x7f0c6c1045e0 2024-05-09T02:21:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.074+0000 7f0c697fa700 1 -- 172.21.15.19:0/2208829569 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0c5c029070 con 0x7f0c6c1045e0 2024-05-09T02:21:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.075+0000 7f0c697fa700 1 --2- 172.21.15.19:0/2208829569 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0c580b29f0 0x7f0c580b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.075+0000 7f0c63fff700 1 --2- 172.21.15.19:0/2208829569 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0c580b29f0 0x7f0c580b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.075+0000 7f0c697fa700 1 -- 172.21.15.19:0/2208829569 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f0c5c0e2d60 con 0x7f0c6c1045e0 2024-05-09T02:21:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.075+0000 7f0c805b1b80 1 Processor -- start 2024-05-09T02:21:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.075+0000 7f0c805b1b80 1 -- start start 2024-05-09T02:21:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.075+0000 7f0c63fff700 1 --2- 172.21.15.19:0/2208829569 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0c580b29f0 0x7f0c580b4f00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0c54009680 tx=0x7f0c54009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.075+0000 7f0c805b1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5e46d00 0x55abe5f4dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.075+0000 7f0c805b1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5dcc4a0 0x55abe5f4e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.075+0000 7f0c805b1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 0x55abe5f525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.075+0000 7f0c805b1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55abe5e6c0a0 con 0x55abe5dcc4a0 2024-05-09T02:21:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.075+0000 7f0c805b1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55abe5e119a0 con 0x55abe5f521e0 2024-05-09T02:21:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.075+0000 7f0c805b1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55abe5e68060 con 0x55abe5e46d00 2024-05-09T02:21:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c63fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5dcc4a0 0x55abe5f4e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c6b7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5e46d00 0x55abe5f4dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c6b7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5e46d00 0x55abe5f4dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36262/0 (socket says 172.21.15.19:36262) 2024-05-09T02:21:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c63fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5dcc4a0 0x55abe5f4e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37766/0 (socket says 172.21.15.19:37766) 2024-05-09T02:21:02.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c6b7fe700 1 -- 172.21.15.19:0/2678409536 learned_addr learned my addr 172.21.15.19:0/2678409536 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:02.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c6bfff700 1 --2- 172.21.15.19:0/2678409536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 0x55abe5f525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c6b7fe700 1 -- 172.21.15.19:0/2678409536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 msgr2=0x55abe5f525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:02.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c6b7fe700 1 --2- 172.21.15.19:0/2678409536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 0x55abe5f525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c6b7fe700 1 -- 172.21.15.19:0/2678409536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5dcc4a0 msgr2=0x55abe5f4e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:02.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c6b7fe700 1 --2- 172.21.15.19:0/2678409536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5dcc4a0 0x55abe5f4e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c6b7fe700 1 -- 172.21.15.19:0/2678409536 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abe5e02500 con 0x55abe5e46d00 2024-05-09T02:21:02.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c63fff700 1 --2- 172.21.15.19:0/2678409536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5dcc4a0 0x55abe5f4e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:21:02.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c6b7fe700 1 --2- 172.21.15.19:0/2678409536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5e46d00 0x55abe5f4dad0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0a9d40 tx=0x7f0c5c0a9b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df3779d36963e717 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:02.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c61ffb700 1 -- 172.21.15.19:0/2678409536 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c0ae3f0 con 0x55abe5e46d00 2024-05-09T02:21:02.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c61ffb700 1 -- 172.21.15.19:0/2678409536 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c5c0b2e10 con 0x55abe5e46d00 2024-05-09T02:21:02.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.076+0000 7f0c61ffb700 1 -- 172.21.15.19:0/2678409536 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c0e0b70 con 0x55abe5e46d00 2024-05-09T02:21:02.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.078+0000 7f0c805b1b80 1 -- 172.21.15.19:0/2678409536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5e46d00 msgr2=0x55abe5f4dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:02.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.078+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/2678409536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5e46d00 0x55abe5f4dad0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0a9d40 tx=0x7f0c5c0a9b50 comp rx=0 tx=0).stop 2024-05-09T02:21:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.078+0000 7f0c805b1b80 1 -- 172.21.15.19:0/2678409536 shutdown_connections 2024-05-09T02:21:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.078+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/2678409536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 0x55abe5f525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.078+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/2678409536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5e46d00 0x55abe5f4dad0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.078+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/2678409536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5dcc4a0 0x55abe5f4e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.078+0000 7f0c805b1b80 1 -- 172.21.15.19:0/2678409536 >> 172.21.15.19:0/2678409536 conn(0x55abe5dcff70 msgr2=0x55abe5f44220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.078+0000 7f0c805b1b80 1 -- 172.21.15.19:0/2678409536 shutdown_connections 2024-05-09T02:21:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.078+0000 7f0c805b1b80 1 -- 172.21.15.19:0/2678409536 wait complete. 2024-05-09T02:21:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.078+0000 7f0c805b1b80 1 -- start start 2024-05-09T02:21:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.078+0000 7f0c805b1b80 1 Processor -- start 2024-05-09T02:21:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c805b1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5dcc4a0 0x55abe5f5cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c805b1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5e46d00 0x55abe5f5d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c805b1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 0x55abe5f57330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c805b1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55abe5f52d50 con 0x55abe5e46d00 2024-05-09T02:21:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c805b1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55abe5e06b20 con 0x55abe5f521e0 2024-05-09T02:21:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c805b1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55abe5f52ed0 con 0x55abe5dcc4a0 2024-05-09T02:21:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c805b1b80 10 client.0 _ll_register_callbacks cb 0x55abe5cdd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c63fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5e46d00 0x55abe5f5d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c6b7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5dcc4a0 0x55abe5f5cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c63fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5e46d00 0x55abe5f5d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37774/0 (socket says 172.21.15.19:37774) 2024-05-09T02:21:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c63fff700 1 -- 172.21.15.19:0/4001784728 learned_addr learned my addr 172.21.15.19:0/4001784728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c6bfff700 1 --2- 172.21.15.19:0/4001784728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 0x55abe5f57330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c6b7fe700 1 -- 172.21.15.19:0/4001784728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 msgr2=0x55abe5f57330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c6b7fe700 1 --2- 172.21.15.19:0/4001784728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 0x55abe5f57330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c6b7fe700 1 -- 172.21.15.19:0/4001784728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5e46d00 msgr2=0x55abe5f5d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c6b7fe700 1 --2- 172.21.15.19:0/4001784728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5e46d00 0x55abe5f5d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.079+0000 7f0c6b7fe700 1 -- 172.21.15.19:0/4001784728 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abe5ce7490 con 0x55abe5dcc4a0 2024-05-09T02:21:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.080+0000 7f0c6b7fe700 1 --2- 172.21.15.19:0/4001784728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5dcc4a0 0x55abe5f5cf30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55abe5f52c60 tx=0x7f0c5c0a6030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd86b1d4b5f2362b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.080+0000 7f0c4affd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:21:02.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.080+0000 7f0c4affd700 1 -- 172.21.15.19:0/4001784728 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c0ae3f0 con 0x55abe5dcc4a0 2024-05-09T02:21:02.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.080+0000 7f0c4affd700 1 -- 172.21.15.19:0/4001784728 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f0c5c0aa2a0 con 0x55abe5dcc4a0 2024-05-09T02:21:02.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.080+0000 7f0c805b1b80 1 -- 172.21.15.19:0/4001784728 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c5c0b2e10 con 0x55abe5dcc4a0 2024-05-09T02:21:02.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.080+0000 7f0c4affd700 1 -- 172.21.15.19:0/4001784728 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0c5c0aa4a0 con 0x55abe5dcc4a0 2024-05-09T02:21:02.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.080+0000 7f0c805b1b80 10 client.26260 fetch_fsmap learned FSMap version 54 2024-05-09T02:21:02.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.080+0000 7f0c805b1b80 1 -- 172.21.15.19:0/4001784728 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55abe5f52d50 con 0x55abe5dcc4a0 2024-05-09T02:21:02.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.080+0000 7f0c4affd700 1 -- 172.21.15.19:0/4001784728 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c5c0b2e10 con 0x55abe5dcc4a0 2024-05-09T02:21:02.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.080+0000 7f0c4affd700 1 -- 172.21.15.19:0/4001784728 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f0c5c0a8d10 con 0x55abe5dcc4a0 2024-05-09T02:21:02.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.081+0000 7f0c805b1b80 10 client.26260 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:21:02.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.081+0000 7f0c805b1b80 10 client.26260 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:21:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.081+0000 7f0c805b1b80 20 client.26260 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.081+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/4001784728 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55abe5f66fa0 0x55abe5f69450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.081+0000 7f0c805b1b80 4 client.26260 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:21:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.081+0000 7f0c805b1b80 1 -- 172.21.15.19:0/4001784728 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abe5e79a80 con 0x55abe5f66fa0 2024-05-09T02:21:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.081+0000 7f0c63fff700 1 --2- 172.21.15.19:0/4001784728 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55abe5f66fa0 0x55abe5f69450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.081+0000 7f0c4affd700 10 client.26260 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:21:02.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.081+0000 7f0c63fff700 1 --2- 172.21.15.19:0/4001784728 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55abe5f66fa0 0x55abe5f69450 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=235ddf054242ae05 server_cookie=fe73017ae70435b1 in_seq=0 out_seq=0 2024-05-09T02:21:02.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.082+0000 7f0c4affd700 1 -- 172.21.15.19:0/4001784728 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55abe5e79a80 con 0x55abe5f66fa0 2024-05-09T02:21:02.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.082+0000 7f0c4affd700 10 client.26260 handle_command_reply: tid=0 2024-05-09T02:21:02.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.082+0000 7f0c805b1b80 1 client.26260 shutdown 2024-05-09T02:21:02.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.082+0000 7f0c805b1b80 1 -- 172.21.15.19:0/4001784728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5dcc4a0 msgr2=0x55abe5f5cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:02.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.082+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/4001784728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5dcc4a0 0x55abe5f5cf30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55abe5f52c60 tx=0x7f0c5c0a6030 comp rx=0 tx=0).stop 2024-05-09T02:21:02.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.082+0000 7f0c805b1b80 1 -- 172.21.15.19:0/4001784728 shutdown_connections 2024-05-09T02:21:02.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.082+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/4001784728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 0x55abe5f57330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.082+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/4001784728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5dcc4a0 0x55abe5f5cf30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.082+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/4001784728 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55abe5f66fa0 0x55abe5f69450 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.082+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/4001784728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5e46d00 0x55abe5f5d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.082+0000 7f0c4affd700 0 client.26260 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:21:02.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.082+0000 7f0c805b1b80 1 -- 172.21.15.19:0/4001784728 >> 172.21.15.19:0/4001784728 conn(0x55abe5dcff70 msgr2=0x55abe5f483f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:02.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.082+0000 7f0c805b1b80 1 -- 172.21.15.19:0/4001784728 shutdown_connections 2024-05-09T02:21:02.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.082+0000 7f0c805b1b80 1 -- 172.21.15.19:0/4001784728 wait complete. 2024-05-09T02:21:02.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.083+0000 7f0c805b1b80 20 client.26260 trim_cache size 0 max 16384 2024-05-09T02:21:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.098+0000 7f0c805b1b80 1 Processor -- start 2024-05-09T02:21:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.098+0000 7f0c805b1b80 1 -- start start 2024-05-09T02:21:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.098+0000 7f0c805b1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5dcc4a0 0x55abe5f57250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.098+0000 7f0c805b1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5e46d00 0x55abe5f51a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.098+0000 7f0c805b1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5f521e0 0x55abe5f6ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.098+0000 7f0c805b1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c00d640 con 0x55abe5e46d00 2024-05-09T02:21:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.098+0000 7f0c805b1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c00f2d0 con 0x55abe5dcc4a0 2024-05-09T02:21:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.098+0000 7f0c805b1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c00dd10 con 0x55abe5f521e0 2024-05-09T02:21:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.099+0000 7f0c63fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5e46d00 0x55abe5f51a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.099+0000 7f0c63fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5e46d00 0x55abe5f51a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37790/0 (socket says 172.21.15.19:37790) 2024-05-09T02:21:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.099+0000 7f0c63fff700 1 -- 172.21.15.19:0/1001513685 learned_addr learned my addr 172.21.15.19:0/1001513685 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.099+0000 7f0c6bfff700 1 --2- 172.21.15.19:0/1001513685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5f521e0 0x55abe5f6ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.099+0000 7f0c6b7fe700 1 --2- 172.21.15.19:0/1001513685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5dcc4a0 0x55abe5f57250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.099+0000 7f0c63fff700 1 -- 172.21.15.19:0/1001513685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5f521e0 msgr2=0x55abe5f6ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:02.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.099+0000 7f0c63fff700 1 --2- 172.21.15.19:0/1001513685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5f521e0 0x55abe5f6ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.099+0000 7f0c63fff700 1 -- 172.21.15.19:0/1001513685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5dcc4a0 msgr2=0x55abe5f57250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:02.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.099+0000 7f0c63fff700 1 --2- 172.21.15.19:0/1001513685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5dcc4a0 0x55abe5f57250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.099+0000 7f0c63fff700 1 -- 172.21.15.19:0/1001513685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abe5f51f90 con 0x55abe5e46d00 2024-05-09T02:21:02.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.099+0000 7f0c63fff700 1 --2- 172.21.15.19:0/1001513685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5e46d00 0x55abe5f51a50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0c54009870 tx=0x7f0c54014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb23c64b66d3c811 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:02.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.099+0000 7f0c617fa700 1 -- 172.21.15.19:0/1001513685 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5401d770 con 0x55abe5e46d00 2024-05-09T02:21:02.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.099+0000 7f0c617fa700 1 -- 172.21.15.19:0/1001513685 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c54030070 con 0x55abe5e46d00 2024-05-09T02:21:02.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.099+0000 7f0c617fa700 1 -- 172.21.15.19:0/1001513685 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5402c970 con 0x55abe5e46d00 2024-05-09T02:21:02.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.100+0000 7f0c805b1b80 1 -- 172.21.15.19:0/1001513685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5e46d00 msgr2=0x55abe5f51a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:02.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.100+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/1001513685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5e46d00 0x55abe5f51a50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0c54009870 tx=0x7f0c54014000 comp rx=0 tx=0).stop 2024-05-09T02:21:02.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.100+0000 7f0c805b1b80 1 -- 172.21.15.19:0/1001513685 shutdown_connections 2024-05-09T02:21:02.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.100+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/1001513685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5f521e0 0x55abe5f6ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.100+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/1001513685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5dcc4a0 0x55abe5f57250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.100+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/1001513685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5e46d00 0x55abe5f51a50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.100+0000 7f0c805b1b80 1 -- 172.21.15.19:0/1001513685 >> 172.21.15.19:0/1001513685 conn(0x55abe5dcff70 msgr2=0x55abe5f51340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.100+0000 7f0c805b1b80 1 -- 172.21.15.19:0/1001513685 shutdown_connections 2024-05-09T02:21:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.100+0000 7f0c805b1b80 1 -- 172.21.15.19:0/1001513685 wait complete. 2024-05-09T02:21:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.101+0000 7f0c805b1b80 1 -- start start 2024-05-09T02:21:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.101+0000 7f0c805b1b80 1 Processor -- start 2024-05-09T02:21:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.101+0000 7f0c805b1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5dcc4a0 0x55abe5f5fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.101+0000 7f0c805b1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5e46d00 0x55abe5f60080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.101+0000 7f0c805b1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 0x55abe5f605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.101+0000 7f0c805b1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c00eb70 con 0x55abe5dcc4a0 2024-05-09T02:21:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.101+0000 7f0c805b1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c00e3e0 con 0x55abe5f521e0 2024-05-09T02:21:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.101+0000 7f0c805b1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c00b600 con 0x55abe5e46d00 2024-05-09T02:21:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.101+0000 7f0c805b1b80 10 client.0 _ll_register_callbacks cb 0x55abe5cdd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.101+0000 7f0c6b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5dcc4a0 0x55abe5f5fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.101+0000 7f0c6b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5dcc4a0 0x55abe5f5fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37794/0 (socket says 172.21.15.19:37794) 2024-05-09T02:21:02.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.102+0000 7f0c6b7fe700 1 -- 172.21.15.19:0/1266220256 learned_addr learned my addr 172.21.15.19:0/1266220256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:02.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.102+0000 7f0c63fff700 1 --2- 172.21.15.19:0/1266220256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5e46d00 0x55abe5f60080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.102+0000 7f0c6bfff700 1 --2- 172.21.15.19:0/1266220256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 0x55abe5f605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.102+0000 7f0c6b7fe700 1 -- 172.21.15.19:0/1266220256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5e46d00 msgr2=0x55abe5f60080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:02.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.102+0000 7f0c6b7fe700 1 --2- 172.21.15.19:0/1266220256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5e46d00 0x55abe5f60080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.102+0000 7f0c6b7fe700 1 -- 172.21.15.19:0/1266220256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 msgr2=0x55abe5f605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:02.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.103+0000 7f0c6b7fe700 1 --2- 172.21.15.19:0/1266220256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 0x55abe5f605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.103+0000 7f0c6b7fe700 1 -- 172.21.15.19:0/1266220256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abe5fd07f0 con 0x55abe5dcc4a0 2024-05-09T02:21:02.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.103+0000 7f0c6bfff700 1 --2- 172.21.15.19:0/1266220256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 0x55abe5f605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:21:02.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.103+0000 7f0c6b7fe700 1 --2- 172.21.15.19:0/1266220256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5dcc4a0 0x55abe5f5fb40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0a60c0 tx=0x7f0c5c0e0520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0f5f98466153a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:02.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.103+0000 7f0c4affd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:21:02.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.103+0000 7f0c4affd700 1 -- 172.21.15.19:0/1266220256 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c0b2e10 con 0x55abe5dcc4a0 2024-05-09T02:21:02.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.103+0000 7f0c805b1b80 1 -- 172.21.15.19:0/1266220256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c54030070 con 0x55abe5dcc4a0 2024-05-09T02:21:02.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.104+0000 7f0c4affd700 1 -- 172.21.15.19:0/1266220256 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f0c5c0a6630 con 0x55abe5dcc4a0 2024-05-09T02:21:02.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.104+0000 7f0c4affd700 1 -- 172.21.15.19:0/1266220256 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0c5c0ae3f0 con 0x55abe5dcc4a0 2024-05-09T02:21:02.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.104+0000 7f0c805b1b80 10 client.16500 fetch_fsmap learned FSMap version 54 2024-05-09T02:21:02.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.104+0000 7f0c805b1b80 1 -- 172.21.15.19:0/1266220256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55abe5fd07f0 con 0x55abe5dcc4a0 2024-05-09T02:21:02.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.105+0000 7f0c4affd700 1 -- 172.21.15.19:0/1266220256 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c54030070 con 0x55abe5dcc4a0 2024-05-09T02:21:02.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.105+0000 7f0c4affd700 1 -- 172.21.15.19:0/1266220256 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f0c5c0c4440 con 0x55abe5dcc4a0 2024-05-09T02:21:02.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.105+0000 7f0c805b1b80 10 client.16500 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:21:02.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.105+0000 7f0c805b1b80 10 client.16500 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:21:02.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.105+0000 7f0c805b1b80 20 client.16500 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.105+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/1266220256 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55abe5fd1ea0 0x55abe5fd42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.105+0000 7f0c805b1b80 4 client.16500 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:21:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.105+0000 7f0c805b1b80 1 -- 172.21.15.19:0/1266220256 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abe5e73b20 con 0x55abe5fd1ea0 2024-05-09T02:21:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.105+0000 7f0c63fff700 1 --2- 172.21.15.19:0/1266220256 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55abe5fd1ea0 0x55abe5fd42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.106+0000 7f0c4affd700 10 client.16500 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:21:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.106+0000 7f0c63fff700 1 --2- 172.21.15.19:0/1266220256 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55abe5fd1ea0 0x55abe5fd42c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3b72f93904ef042 server_cookie=379d7d12aa879cd7 in_seq=0 out_seq=0 2024-05-09T02:21:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.106+0000 7f0c4affd700 1 -- 172.21.15.19:0/1266220256 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abe5e73b20 con 0x55abe5fd1ea0 2024-05-09T02:21:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.106+0000 7f0c4affd700 10 client.16500 handle_command_reply: tid=0 2024-05-09T02:21:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.106+0000 7f0c805b1b80 1 client.16500 shutdown 2024-05-09T02:21:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.107+0000 7f0c805b1b80 1 -- 172.21.15.19:0/1266220256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5dcc4a0 msgr2=0x55abe5f5fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.107+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/1266220256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5dcc4a0 0x55abe5f5fb40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0a60c0 tx=0x7f0c5c0e0520 comp rx=0 tx=0).stop 2024-05-09T02:21:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.107+0000 7f0c805b1b80 1 -- 172.21.15.19:0/1266220256 shutdown_connections 2024-05-09T02:21:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.107+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/1266220256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abe5f521e0 0x55abe5f605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.107+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/1266220256 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55abe5fd1ea0 0x55abe5fd42c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.107+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/1266220256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abe5dcc4a0 0x55abe5f5fb40 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.107+0000 7f0c805b1b80 1 --2- 172.21.15.19:0/1266220256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abe5e46d00 0x55abe5f60080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.107+0000 7f0c805b1b80 1 -- 172.21.15.19:0/1266220256 >> 172.21.15.19:0/1266220256 conn(0x55abe5dcff70 msgr2=0x55abe5f3dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:02.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.107+0000 7f0c4affd700 0 client.16500 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:21:02.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.107+0000 7f0c805b1b80 1 -- 172.21.15.19:0/1266220256 shutdown_connections 2024-05-09T02:21:02.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.107+0000 7f0c805b1b80 1 -- 172.21.15.19:0/1266220256 wait complete. 2024-05-09T02:21:02.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.107+0000 7f0c805b1b80 20 client.16500 trim_cache size 0 max 16384 2024-05-09T02:21:02.113 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:21:02.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.108+0000 7f0c71b9b700 1 -- 172.21.15.19:0/2208829569 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0c580b29f0 msgr2=0x7f0c580b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:02.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.108+0000 7f0c71b9b700 1 --2- 172.21.15.19:0/2208829569 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0c580b29f0 0x7f0c580b4f00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0c54009680 tx=0x7f0c54009260 comp rx=0 tx=0).stop 2024-05-09T02:21:02.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.108+0000 7f0c71b9b700 1 -- 172.21.15.19:0/2208829569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c6c1045e0 msgr2=0x7f0c6c19ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:02.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.108+0000 7f0c71b9b700 1 --2- 172.21.15.19:0/2208829569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c6c1045e0 0x7f0c6c19ade0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c006010 tx=0x7f0c5c005280 comp rx=0 tx=0).stop 2024-05-09T02:21:02.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.108+0000 7f0c71b9b700 1 -- 172.21.15.19:0/2208829569 shutdown_connections 2024-05-09T02:21:02.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.108+0000 7f0c71b9b700 1 --2- 172.21.15.19:0/2208829569 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0c580b29f0 0x7f0c580b4f00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.108+0000 7f0c71b9b700 1 --2- 172.21.15.19:0/2208829569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c6c109860 0x7f0c6c19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.108+0000 7f0c71b9b700 1 --2- 172.21.15.19:0/2208829569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c6c1045e0 0x7f0c6c19ade0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.108+0000 7f0c71b9b700 1 --2- 172.21.15.19:0/2208829569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c6c108f10 0x7f0c6c19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:02.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.108+0000 7f0c71b9b700 1 -- 172.21.15.19:0/2208829569 >> 172.21.15.19:0/2208829569 conn(0x7f0c6c0ffe70 msgr2=0x7f0c6c107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:02.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.108+0000 7f0c71b9b700 1 -- 172.21.15.19:0/2208829569 shutdown_connections 2024-05-09T02:21:02.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:02.108+0000 7f0c71b9b700 1 -- 172.21.15.19:0/2208829569 wait complete. 2024-05-09T02:21:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 560.0 s, 883.2 tps, lat 109.539 ms stddev 112.925 2024-05-09T02:21:02.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:02 smithi059 ceph-mon[34781]: pgmap v301: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.2 MiB/s wr, 574 op/s 2024-05-09T02:21:02.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:02 smithi196 ceph-mon[34718]: pgmap v301: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.2 MiB/s wr, 574 op/s 2024-05-09T02:21:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:02 smithi019 ceph-mon[30151]: pgmap v301: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.2 MiB/s wr, 574 op/s 2024-05-09T02:21:02.938 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T02:21:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:21:04.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:04 smithi019 ceph-mon[30151]: pgmap v302: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 540 op/s 2024-05-09T02:21:04.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:04 smithi059 ceph-mon[34781]: pgmap v302: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 540 op/s 2024-05-09T02:21:04.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:04 smithi196 ceph-mon[34718]: pgmap v302: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 540 op/s 2024-05-09T02:21:05.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.213+0000 7ffab6f99700 1 -- 172.21.15.19:0/1081786708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffab010b280 msgr2=0x7ffab011d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:05.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.213+0000 7ffab6f99700 1 --2- 172.21.15.19:0/1081786708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffab010b280 0x7ffab011d3f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffaa401a9a0 tx=0x7ffaa401acb0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.220+0000 7ffab6f99700 1 -- 172.21.15.19:0/1081786708 shutdown_connections 2024-05-09T02:21:05.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.220+0000 7ffab6f99700 1 --2- 172.21.15.19:0/1081786708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab011d930 0x7ffab011fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.220+0000 7ffab6f99700 1 --2- 172.21.15.19:0/1081786708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab010a930 0x7ffab010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.220+0000 7ffab6f99700 1 --2- 172.21.15.19:0/1081786708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffab010b280 0x7ffab011d3f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.220+0000 7ffab6f99700 1 -- 172.21.15.19:0/1081786708 >> 172.21.15.19:0/1081786708 conn(0x7ffab006f4d0 msgr2=0x7ffab006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:05.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.221+0000 7ffab6f99700 1 -- 172.21.15.19:0/1081786708 shutdown_connections 2024-05-09T02:21:05.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.223+0000 7ffab6f99700 1 -- 172.21.15.19:0/1081786708 wait complete. 2024-05-09T02:21:05.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.224+0000 7ffab6f99700 1 Processor -- start 2024-05-09T02:21:05.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.224+0000 7ffab6f99700 1 -- start start 2024-05-09T02:21:05.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.225+0000 7ffab6f99700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab010a930 0x7ffab011d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.225+0000 7ffab6f99700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab010b280 0x7ffab01193f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.225+0000 7ffab6f99700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffab011d930 0x7ffab0119950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.225+0000 7ffab6f99700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffab0115410 con 0x7ffab010b280 2024-05-09T02:21:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.225+0000 7ffab6f99700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffab0115290 con 0x7ffab010a930 2024-05-09T02:21:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.225+0000 7ffab6f99700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffab0115590 con 0x7ffab011d930 2024-05-09T02:21:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.227+0000 7ffab5536700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffab011d930 0x7ffab0119950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.227+0000 7ffab4d35700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab010a930 0x7ffab011d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.227+0000 7ffab5536700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffab011d930 0x7ffab0119950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36322/0 (socket says 172.21.15.19:36322) 2024-05-09T02:21:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.227+0000 7ffab5536700 1 -- 172.21.15.19:0/620706550 learned_addr learned my addr 172.21.15.19:0/620706550 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:05.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.229+0000 7ffaaffff700 1 --2- 172.21.15.19:0/620706550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab010b280 0x7ffab01193f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.229+0000 7ffab4d35700 1 -- 172.21.15.19:0/620706550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffab011d930 msgr2=0x7ffab0119950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.229+0000 7ffab4d35700 1 --2- 172.21.15.19:0/620706550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffab011d930 0x7ffab0119950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.229+0000 7ffab4d35700 1 -- 172.21.15.19:0/620706550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab010b280 msgr2=0x7ffab01193f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.229+0000 7ffab4d35700 1 --2- 172.21.15.19:0/620706550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab010b280 0x7ffab01193f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.229+0000 7ffab4d35700 1 -- 172.21.15.19:0/620706550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab011a180 con 0x7ffab010a930 2024-05-09T02:21:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.229+0000 7ffab4d35700 1 --2- 172.21.15.19:0/620706550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab010a930 0x7ffab011d2b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800ef10 tx=0x7ffaa800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.231+0000 7ffaadffb700 1 -- 172.21.15.19:0/620706550 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa800ca70 con 0x7ffab010a930 2024-05-09T02:21:05.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.231+0000 7ffab6f99700 1 -- 172.21.15.19:0/620706550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab01ad920 con 0x7ffab010a930 2024-05-09T02:21:05.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.231+0000 7ffab6f99700 1 -- 172.21.15.19:0/620706550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab01ade30 con 0x7ffab010a930 2024-05-09T02:21:05.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.231+0000 7ffaadffb700 1 -- 172.21.15.19:0/620706550 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaa8019070 con 0x7ffab010a930 2024-05-09T02:21:05.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.231+0000 7ffaadffb700 1 -- 172.21.15.19:0/620706550 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa80146e0 con 0x7ffab010a930 2024-05-09T02:21:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.241+0000 7ffac59afb80 1 Processor -- start 2024-05-09T02:21:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.241+0000 7ffac59afb80 1 -- start start 2024-05-09T02:21:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.242+0000 7ffac59afb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046c2d00 0x560b047cb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.242+0000 7ffac59afb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b046484a0 0x560b047cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.242+0000 7ffac59afb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b047c5400 0x560b047c5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.242+0000 7ffac59afb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560b046e80a0 con 0x560b046484a0 2024-05-09T02:21:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.242+0000 7ffac59afb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560b0468d9a0 con 0x560b046c2d00 2024-05-09T02:21:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.242+0000 7ffac59afb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560b046e4060 con 0x560b047c5400 2024-05-09T02:21:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.243+0000 7ffaadffb700 1 -- 172.21.15.19:0/620706550 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ffaa8012050 con 0x7ffab010a930 2024-05-09T02:21:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.243+0000 7ffaaffff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b047c5400 0x560b047c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.243+0000 7ffaaffff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b047c5400 0x560b047c5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36336/0 (socket says 172.21.15.19:36336) 2024-05-09T02:21:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.243+0000 7ffaaffff700 1 -- 172.21.15.19:0/2861420631 learned_addr learned my addr 172.21.15.19:0/2861420631 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.243+0000 7ffab4d35700 1 --2- 172.21.15.19:0/2861420631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b046484a0 0x560b047cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.244+0000 7ffaaffff700 1 -- 172.21.15.19:0/2861420631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046c2d00 msgr2=0x560b047cb240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:21:05.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.244+0000 7ffaaffff700 1 --2- 172.21.15.19:0/2861420631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046c2d00 0x560b047cb240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.244+0000 7ffaaffff700 1 -- 172.21.15.19:0/2861420631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b046484a0 msgr2=0x560b047cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:05.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.244+0000 7ffaaffff700 1 --2- 172.21.15.19:0/2861420631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b046484a0 0x560b047cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.244+0000 7ffaaffff700 1 -- 172.21.15.19:0/2861420631 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b0467e500 con 0x560b047c5400 2024-05-09T02:21:05.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.245+0000 7ffaadffb700 1 --2- 172.21.15.19:0/620706550 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ffa980b2c00 0x7ffa980b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.245+0000 7ffaaffff700 1 --2- 172.21.15.19:0/2861420631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b047c5400 0x560b047c5810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffaa401a9a0 tx=0x7ffaa401ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e85111aea3823c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.245+0000 7ffaadffb700 1 -- 172.21.15.19:0/620706550 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7ffaa80e2530 con 0x7ffab010a930 2024-05-09T02:21:05.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.245+0000 7ffab5536700 1 --2- 172.21.15.19:0/620706550 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ffa980b2c00 0x7ffa980b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.246+0000 7ffab4d35700 1 --2- 172.21.15.19:0/2861420631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b046484a0 0x560b047cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:21:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.248+0000 7ffa96ffd700 1 -- 172.21.15.19:0/2861420631 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa401cc20 con 0x560b047c5400 2024-05-09T02:21:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.248+0000 7ffa96ffd700 1 -- 172.21.15.19:0/2861420631 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaa401cdc0 con 0x560b047c5400 2024-05-09T02:21:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.248+0000 7ffa96ffd700 1 -- 172.21.15.19:0/2861420631 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4003e60 con 0x560b047c5400 2024-05-09T02:21:05.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.251+0000 7ffab5536700 1 --2- 172.21.15.19:0/620706550 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ffa980b2c00 0x7ffa980b5110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009a20 tx=0x7ffaa00091c0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:05.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.251+0000 7ffac59afb80 1 -- 172.21.15.19:0/2861420631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b047c5400 msgr2=0x560b047c5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:05.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.251+0000 7ffac59afb80 1 --2- 172.21.15.19:0/2861420631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b047c5400 0x560b047c5810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffaa401a9a0 tx=0x7ffaa401ca00 comp rx=0 tx=0).stop 2024-05-09T02:21:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.254+0000 7ffac59afb80 1 -- 172.21.15.19:0/2861420631 shutdown_connections 2024-05-09T02:21:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.254+0000 7ffac59afb80 1 --2- 172.21.15.19:0/2861420631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b047c5400 0x560b047c5810 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.254+0000 7ffac59afb80 1 --2- 172.21.15.19:0/2861420631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046c2d00 0x560b047cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.254+0000 7ffac59afb80 1 --2- 172.21.15.19:0/2861420631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b046484a0 0x560b047cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.254+0000 7ffac59afb80 1 -- 172.21.15.19:0/2861420631 >> 172.21.15.19:0/2861420631 conn(0x560b0464bf70 msgr2=0x560b047c0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.254+0000 7ffac59afb80 1 -- 172.21.15.19:0/2861420631 shutdown_connections 2024-05-09T02:21:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.254+0000 7ffac59afb80 1 -- 172.21.15.19:0/2861420631 wait complete. 2024-05-09T02:21:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.255+0000 7ffac59afb80 1 -- start start 2024-05-09T02:21:05.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.255+0000 7ffac59afb80 1 Processor -- start 2024-05-09T02:21:05.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.256+0000 7ffac59afb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046484a0 0x560b047cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.256+0000 7ffac59afb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046c2d00 0x560b047cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.256+0000 7ffac59afb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b047c5400 0x560b047cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.257+0000 7ffac59afb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560b047c6100 con 0x560b047c5400 2024-05-09T02:21:05.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.257+0000 7ffac59afb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560b04682b20 con 0x560b046484a0 2024-05-09T02:21:05.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.257+0000 7ffac59afb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560b047db230 con 0x560b046c2d00 2024-05-09T02:21:05.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.257+0000 7ffab5536700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b047c5400 0x560b047cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:05.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.257+0000 7ffab5536700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b047c5400 0x560b047cfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37820/0 (socket says 172.21.15.19:37820) 2024-05-09T02:21:05.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.257+0000 7ffab5536700 1 -- 172.21.15.19:0/3816996933 learned_addr learned my addr 172.21.15.19:0/3816996933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:05.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.257+0000 7ffac59afb80 10 client.0 _ll_register_callbacks cb 0x560b045596c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:05.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.258+0000 7ffab5536700 1 -- 172.21.15.19:0/3816996933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046c2d00 msgr2=0x560b047cf510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:21:05.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.258+0000 7ffab5536700 1 --2- 172.21.15.19:0/3816996933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046c2d00 0x560b047cf510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.258+0000 7ffab5536700 1 -- 172.21.15.19:0/3816996933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046484a0 msgr2=0x560b047cefd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:21:05.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.258+0000 7ffab5536700 1 --2- 172.21.15.19:0/3816996933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046484a0 0x560b047cefd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.258+0000 7ffab5536700 1 -- 172.21.15.19:0/3816996933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b04563490 con 0x560b047c5400 2024-05-09T02:21:05.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.258+0000 7ffab5536700 1 --2- 172.21.15.19:0/3816996933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b047c5400 0x560b047cfa50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00154d0 tx=0x7ffaa0004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c3373431afe2b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:05.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.259+0000 7ffa8ffff700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:21:05.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.259+0000 7ffa8ffff700 1 -- 172.21.15.19:0/3816996933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0023020 con 0x560b047c5400 2024-05-09T02:21:05.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.259+0000 7ffa8ffff700 1 -- 172.21.15.19:0/3816996933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7ffaa0015af0 con 0x560b047c5400 2024-05-09T02:21:05.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.259+0000 7ffac59afb80 1 -- 172.21.15.19:0/3816996933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaa401cdc0 con 0x560b047c5400 2024-05-09T02:21:05.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.260+0000 7ffa8ffff700 1 -- 172.21.15.19:0/3816996933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ffaa002e050 con 0x560b047c5400 2024-05-09T02:21:05.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.260+0000 7ffac59afb80 10 client.16509 fetch_fsmap learned FSMap version 54 2024-05-09T02:21:05.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.260+0000 7ffac59afb80 1 -- 172.21.15.19:0/3816996933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x560b047dc980 con 0x560b047c5400 2024-05-09T02:21:05.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.261+0000 7ffa8ffff700 1 -- 172.21.15.19:0/3816996933 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaa401cdc0 con 0x560b047c5400 2024-05-09T02:21:05.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.261+0000 7ffa8ffff700 1 -- 172.21.15.19:0/3816996933 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7ffaa001e030 con 0x560b047c5400 2024-05-09T02:21:05.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.261+0000 7ffac59afb80 10 client.16509 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:21:05.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.261+0000 7ffac59afb80 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:21:05.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.261+0000 7ffac59afb80 20 client.16509 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:05.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.261+0000 7ffac59afb80 1 --2- 172.21.15.19:0/3816996933 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560b047df060 0x560b047e1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.261+0000 7ffac59afb80 4 client.16509 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:21:05.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.261+0000 7ffac59afb80 1 -- 172.21.15.19:0/3816996933 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b046f5a80 con 0x560b047df060 2024-05-09T02:21:05.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.262+0000 7ffab4d35700 1 --2- 172.21.15.19:0/3816996933 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560b047df060 0x560b047e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:05.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.263+0000 7ffa8ffff700 10 client.16509 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:05.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.263+0000 7ffab4d35700 1 --2- 172.21.15.19:0/3816996933 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560b047df060 0x560b047e1530 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=562c4cd6142116d9 server_cookie=b9b93c7b798179e5 in_seq=0 out_seq=0 2024-05-09T02:21:05.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.265+0000 7ffa8ffff700 1 -- 172.21.15.19:0/3816996933 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560b046f5a80 con 0x560b047df060 2024-05-09T02:21:05.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.265+0000 7ffa8ffff700 10 client.16509 handle_command_reply: tid=0 2024-05-09T02:21:05.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.265+0000 7ffac59afb80 1 client.16509 shutdown 2024-05-09T02:21:05.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.266+0000 7ffac59afb80 1 -- 172.21.15.19:0/3816996933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b047c5400 msgr2=0x560b047cfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:05.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.266+0000 7ffac59afb80 1 --2- 172.21.15.19:0/3816996933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b047c5400 0x560b047cfa50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00154d0 tx=0x7ffaa0004860 comp rx=0 tx=0).stop 2024-05-09T02:21:05.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.267+0000 7ffac59afb80 1 -- 172.21.15.19:0/3816996933 shutdown_connections 2024-05-09T02:21:05.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.267+0000 7ffac59afb80 1 --2- 172.21.15.19:0/3816996933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b047c5400 0x560b047cfa50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.267+0000 7ffac59afb80 1 --2- 172.21.15.19:0/3816996933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046484a0 0x560b047cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.267+0000 7ffac59afb80 1 --2- 172.21.15.19:0/3816996933 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560b047df060 0x560b047e1530 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.267+0000 7ffac59afb80 1 --2- 172.21.15.19:0/3816996933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046c2d00 0x560b047cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.267+0000 7ffac59afb80 1 -- 172.21.15.19:0/3816996933 >> 172.21.15.19:0/3816996933 conn(0x560b0464bf70 msgr2=0x560b047b9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:05.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.267+0000 7ffa8ffff700 0 client.16509 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:05.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.267+0000 7ffac59afb80 1 -- 172.21.15.19:0/3816996933 shutdown_connections 2024-05-09T02:21:05.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.267+0000 7ffac59afb80 1 -- 172.21.15.19:0/3816996933 wait complete. 2024-05-09T02:21:05.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.267+0000 7ffac59afb80 20 client.16509 trim_cache size 0 max 16384 2024-05-09T02:21:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.293+0000 7ffac59afb80 1 Processor -- start 2024-05-09T02:21:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.294+0000 7ffac59afb80 1 -- start start 2024-05-09T02:21:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.294+0000 7ffac59afb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b0464bf70 0x560b047cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.294+0000 7ffac59afb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046484a0 0x560b047e16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.295+0000 7ffab4d35700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b0464bf70 0x560b047cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.295+0000 7ffab4d35700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b0464bf70 0x560b047cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37824/0 (socket says 172.21.15.19:37824) 2024-05-09T02:21:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.295+0000 7ffac59afb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046c2d00 0x560b047cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.295+0000 7ffaaffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046484a0 0x560b047e16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.295+0000 7ffac59afb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c00d7c0 con 0x560b0464bf70 2024-05-09T02:21:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.295+0000 7ffac59afb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c00f450 con 0x560b046484a0 2024-05-09T02:21:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.295+0000 7ffaaffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046484a0 0x560b047e16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:59662/0 (socket says 172.21.15.19:59662) 2024-05-09T02:21:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.295+0000 7ffaaffff700 1 -- 172.21.15.19:0/761277314 learned_addr learned my addr 172.21.15.19:0/761277314 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.295+0000 7ffac59afb80 1 -- 172.21.15.19:0/761277314 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c00de90 con 0x560b046c2d00 2024-05-09T02:21:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.295+0000 7ffab5536700 1 --2- 172.21.15.19:0/761277314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046c2d00 0x560b047cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.295+0000 7ffaaffff700 1 -- 172.21.15.19:0/761277314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046c2d00 msgr2=0x560b047cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.295+0000 7ffaaffff700 1 --2- 172.21.15.19:0/761277314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046c2d00 0x560b047cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.295+0000 7ffaaffff700 1 -- 172.21.15.19:0/761277314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b0464bf70 msgr2=0x560b047cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.295+0000 7ffaaffff700 1 --2- 172.21.15.19:0/761277314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b0464bf70 0x560b047cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.295+0000 7ffaaffff700 1 -- 172.21.15.19:0/761277314 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b047cffe0 con 0x560b046484a0 2024-05-09T02:21:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.296+0000 7ffaaffff700 1 --2- 172.21.15.19:0/761277314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046484a0 0x560b047e16e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffaa401c9c0 tx=0x7ffaa401a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17da86a0380f6604 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.296+0000 7ffa967fc700 1 -- 172.21.15.19:0/761277314 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4000c80 con 0x560b046484a0 2024-05-09T02:21:05.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.296+0000 7ffa967fc700 1 -- 172.21.15.19:0/761277314 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaa40054f0 con 0x560b046484a0 2024-05-09T02:21:05.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.296+0000 7ffa967fc700 1 -- 172.21.15.19:0/761277314 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4003ad0 con 0x560b046484a0 2024-05-09T02:21:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.297+0000 7ffac59afb80 1 -- 172.21.15.19:0/761277314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046484a0 msgr2=0x560b047e16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.297+0000 7ffac59afb80 1 --2- 172.21.15.19:0/761277314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046484a0 0x560b047e16e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffaa401c9c0 tx=0x7ffaa401a250 comp rx=0 tx=0).stop 2024-05-09T02:21:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.297+0000 7ffac59afb80 1 -- 172.21.15.19:0/761277314 shutdown_connections 2024-05-09T02:21:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.297+0000 7ffac59afb80 1 --2- 172.21.15.19:0/761277314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046c2d00 0x560b047cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.297+0000 7ffac59afb80 1 --2- 172.21.15.19:0/761277314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b0464bf70 0x560b047cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.297+0000 7ffac59afb80 1 --2- 172.21.15.19:0/761277314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046484a0 0x560b047e16e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.297+0000 7ffac59afb80 1 -- 172.21.15.19:0/761277314 >> 172.21.15.19:0/761277314 conn(0x560b047c0520 msgr2=0x560b047b9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.297+0000 7ffac59afb80 1 -- 172.21.15.19:0/761277314 shutdown_connections 2024-05-09T02:21:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.297+0000 7ffac59afb80 1 -- 172.21.15.19:0/761277314 wait complete. 2024-05-09T02:21:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.298+0000 7ffac59afb80 1 -- start start 2024-05-09T02:21:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.298+0000 7ffac59afb80 1 Processor -- start 2024-05-09T02:21:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.298+0000 7ffac59afb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046484a0 0x560b047d7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.298+0000 7ffac59afb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b0464bf70 0x560b047d76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.298+0000 7ffac59afb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046c2d00 0x560b047e5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.298+0000 7ffac59afb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c00ecf0 con 0x560b0464bf70 2024-05-09T02:21:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.298+0000 7ffac59afb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c00e560 con 0x560b046c2d00 2024-05-09T02:21:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.298+0000 7ffac59afb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c00b780 con 0x560b046484a0 2024-05-09T02:21:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffab4d35700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046484a0 0x560b047d7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffaaffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b0464bf70 0x560b047d76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffab4d35700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046484a0 0x560b047d7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36366/0 (socket says 172.21.15.19:36366) 2024-05-09T02:21:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffab4d35700 1 -- 172.21.15.19:0/3845412311 learned_addr learned my addr 172.21.15.19:0/3845412311 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffac59afb80 10 client.0 _ll_register_callbacks cb 0x560b045596c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffab5536700 1 --2- 172.21.15.19:0/3845412311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046c2d00 0x560b047e5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffab4d35700 1 -- 172.21.15.19:0/3845412311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046c2d00 msgr2=0x560b047e5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffab4d35700 1 --2- 172.21.15.19:0/3845412311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046c2d00 0x560b047e5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffab4d35700 1 -- 172.21.15.19:0/3845412311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b0464bf70 msgr2=0x560b047d76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffab4d35700 1 --2- 172.21.15.19:0/3845412311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b0464bf70 0x560b047d76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffab4d35700 1 -- 172.21.15.19:0/3845412311 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b047bc9d0 con 0x560b046484a0 2024-05-09T02:21:05.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffab4d35700 1 --2- 172.21.15.19:0/3845412311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046484a0 0x560b047d7190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80a69f0 tx=0x7ffaa80a5760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b45401171cf2fd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:05.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffa8ffff700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:21:05.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffa8ffff700 1 -- 172.21.15.19:0/3845412311 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa80a8ef0 con 0x560b046484a0 2024-05-09T02:21:05.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffac59afb80 1 -- 172.21.15.19:0/3845412311 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaa40054f0 con 0x560b046484a0 2024-05-09T02:21:05.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.299+0000 7ffa8ffff700 1 -- 172.21.15.19:0/3845412311 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7ffaa80a9bf0 con 0x560b046484a0 2024-05-09T02:21:05.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.300+0000 7ffa8ffff700 1 -- 172.21.15.19:0/3845412311 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ffaa8007e30 con 0x560b046484a0 2024-05-09T02:21:05.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.300+0000 7ffac59afb80 10 client.26281 fetch_fsmap learned FSMap version 54 2024-05-09T02:21:05.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.300+0000 7ffac59afb80 1 -- 172.21.15.19:0/3845412311 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x560b04848e40 con 0x560b046484a0 2024-05-09T02:21:05.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.300+0000 7ffa8ffff700 1 -- 172.21.15.19:0/3845412311 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaa40054f0 con 0x560b046484a0 2024-05-09T02:21:05.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.300+0000 7ffa8ffff700 1 -- 172.21.15.19:0/3845412311 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7ffaa80b2150 con 0x560b046484a0 2024-05-09T02:21:05.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.300+0000 7ffac59afb80 10 client.26281 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:21:05.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.300+0000 7ffac59afb80 10 client.26281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:21:05.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.300+0000 7ffac59afb80 20 client.26281 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:05.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.300+0000 7ffac59afb80 1 --2- 172.21.15.19:0/3845412311 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560b047c5400 0x560b0484b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:05.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.300+0000 7ffac59afb80 4 client.26281 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:21:05.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.300+0000 7ffac59afb80 1 -- 172.21.15.19:0/3845412311 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b046efb20 con 0x560b047c5400 2024-05-09T02:21:05.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.300+0000 7ffaaffff700 1 --2- 172.21.15.19:0/3845412311 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560b047c5400 0x560b0484b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:05.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.301+0000 7ffa8ffff700 10 client.26281 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:05.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.301+0000 7ffaaffff700 1 --2- 172.21.15.19:0/3845412311 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560b047c5400 0x560b0484b5a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c25d5c38f530e9b0 server_cookie=fe23e10cd28c8b94 in_seq=0 out_seq=0 2024-05-09T02:21:05.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.302+0000 7ffa8ffff700 1 -- 172.21.15.19:0/3845412311 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b046efb20 con 0x560b047c5400 2024-05-09T02:21:05.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.302+0000 7ffa8ffff700 10 client.26281 handle_command_reply: tid=0 2024-05-09T02:21:05.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.302+0000 7ffac59afb80 1 client.26281 shutdown 2024-05-09T02:21:05.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.303+0000 7ffac59afb80 1 -- 172.21.15.19:0/3845412311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046484a0 msgr2=0x560b047d7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:05.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.303+0000 7ffac59afb80 1 --2- 172.21.15.19:0/3845412311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046484a0 0x560b047d7190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80a69f0 tx=0x7ffaa80a5760 comp rx=0 tx=0).stop 2024-05-09T02:21:05.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.303+0000 7ffac59afb80 1 -- 172.21.15.19:0/3845412311 shutdown_connections 2024-05-09T02:21:05.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.303+0000 7ffac59afb80 1 --2- 172.21.15.19:0/3845412311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560b046c2d00 0x560b047e5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.303+0000 7ffac59afb80 1 --2- 172.21.15.19:0/3845412311 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560b047c5400 0x560b0484b5a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.303+0000 7ffac59afb80 1 --2- 172.21.15.19:0/3845412311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b046484a0 0x560b047d7190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.303+0000 7ffac59afb80 1 --2- 172.21.15.19:0/3845412311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b0464bf70 0x560b047d76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.303+0000 7ffac59afb80 1 -- 172.21.15.19:0/3845412311 >> 172.21.15.19:0/3845412311 conn(0x560b047c0520 msgr2=0x560b047c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.303+0000 7ffa8ffff700 0 client.26281 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.304+0000 7ffac59afb80 1 -- 172.21.15.19:0/3845412311 shutdown_connections 2024-05-09T02:21:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.304+0000 7ffac59afb80 1 -- 172.21.15.19:0/3845412311 wait complete. 2024-05-09T02:21:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.304+0000 7ffac59afb80 20 client.26281 trim_cache size 0 max 16384 2024-05-09T02:21:05.309 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:21:05.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.305+0000 7ffab6f99700 1 -- 172.21.15.19:0/620706550 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ffa980b2c00 msgr2=0x7ffa980b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:05.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.305+0000 7ffab6f99700 1 --2- 172.21.15.19:0/620706550 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ffa980b2c00 0x7ffa980b5110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009a20 tx=0x7ffaa00091c0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.305+0000 7ffab6f99700 1 -- 172.21.15.19:0/620706550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab010a930 msgr2=0x7ffab011d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:05.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.305+0000 7ffab6f99700 1 --2- 172.21.15.19:0/620706550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab010a930 0x7ffab011d2b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800ef10 tx=0x7ffaa800c3a0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.305+0000 7ffab6f99700 1 -- 172.21.15.19:0/620706550 shutdown_connections 2024-05-09T02:21:05.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.305+0000 7ffab6f99700 1 --2- 172.21.15.19:0/620706550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffab011d930 0x7ffab0119950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.305+0000 7ffab6f99700 1 --2- 172.21.15.19:0/620706550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab010a930 0x7ffab011d2b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.305+0000 7ffab6f99700 1 --2- 172.21.15.19:0/620706550 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ffa980b2c00 0x7ffa980b5110 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.305+0000 7ffab6f99700 1 --2- 172.21.15.19:0/620706550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab010b280 0x7ffab01193f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:05.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.305+0000 7ffab6f99700 1 -- 172.21.15.19:0/620706550 >> 172.21.15.19:0/620706550 conn(0x7ffab006f4d0 msgr2=0x7ffab011df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:05.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.305+0000 7ffab6f99700 1 -- 172.21.15.19:0/620706550 shutdown_connections 2024-05-09T02:21:05.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:05.305+0000 7ffab6f99700 1 -- 172.21.15.19:0/620706550 wait complete. 2024-05-09T02:21:06.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:06 smithi059 ceph-mon[34781]: pgmap v303: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 1.43k op/s 2024-05-09T02:21:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:06 smithi196 ceph-mon[34718]: pgmap v303: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 1.43k op/s 2024-05-09T02:21:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:06 smithi019 ceph-mon[30151]: pgmap v303: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 1.43k op/s 2024-05-09T02:21:07.030 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T02:21:07.031 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T02:21:07.509 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:21:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 565.0 s, 872.0 tps, lat 119.193 ms stddev 191.643 2024-05-09T02:21:07.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:07 smithi059 ceph-mon[34781]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T02:21:07.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:07 smithi196 ceph-mon[34718]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T02:21:07.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:07 smithi019 ceph-mon[30151]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T02:21:08.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:08 smithi196 ceph-mon[34718]: pgmap v305: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 1.38k op/s 2024-05-09T02:21:08.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:08 smithi019 ceph-mon[30151]: pgmap v305: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 1.38k op/s 2024-05-09T02:21:08.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:08 smithi059 ceph-mon[34781]: pgmap v305: 97 pgs: 97 active+clean; 4.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 1.38k op/s 2024-05-09T02:21:10.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.317+0000 7f62e55b7700 1 -- 172.21.15.19:0/531921422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62e01086a0 msgr2=0x7f62e0108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:10.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.317+0000 7f62e55b7700 1 --2- 172.21.15.19:0/531921422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62e01086a0 0x7f62e0108b70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009f90 tx=0x7f62d800b330 comp rx=0 tx=0).stop 2024-05-09T02:21:10.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.317+0000 7f62e55b7700 1 -- 172.21.15.19:0/531921422 shutdown_connections 2024-05-09T02:21:10.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.317+0000 7f62e55b7700 1 --2- 172.21.15.19:0/531921422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62e01086a0 0x7f62e0108b70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.317+0000 7f62e55b7700 1 --2- 172.21.15.19:0/531921422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0103250 0x7f62e0103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.317+0000 7f62e55b7700 1 --2- 172.21.15.19:0/531921422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0103ba0 0x7f62e01080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.317+0000 7f62e55b7700 1 -- 172.21.15.19:0/531921422 >> 172.21.15.19:0/531921422 conn(0x7f62e00fedc0 msgr2=0x7f62e01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.317+0000 7f62e55b7700 1 -- 172.21.15.19:0/531921422 shutdown_connections 2024-05-09T02:21:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.317+0000 7f62e55b7700 1 -- 172.21.15.19:0/531921422 wait complete. 2024-05-09T02:21:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.318+0000 7f62e55b7700 1 Processor -- start 2024-05-09T02:21:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.319+0000 7f62e55b7700 1 -- start start 2024-05-09T02:21:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.319+0000 7f62e55b7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0103250 0x7f62e01a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.319+0000 7f62e55b7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0103ba0 0x7f62e01a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.319+0000 7f62e55b7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62e01086a0 0x7f62e01a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.319+0000 7f62e55b7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62e011aa70 con 0x7f62e0103250 2024-05-09T02:21:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.319+0000 7f62e55b7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f62e011a8f0 con 0x7f62e0103ba0 2024-05-09T02:21:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.319+0000 7f62e55b7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62e011abf0 con 0x7f62e01086a0 2024-05-09T02:21:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.319+0000 7f62de7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0103ba0 0x7f62e01a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.319+0000 7f62df7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62e01086a0 0x7f62e01a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.319+0000 7f62deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0103250 0x7f62e01a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.319+0000 7f62df7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62e01086a0 0x7f62e01a5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:60038/0 (socket says 172.21.15.19:60038) 2024-05-09T02:21:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.319+0000 7f62de7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0103ba0 0x7f62e01a1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:51752/0 (socket says 172.21.15.19:51752) 2024-05-09T02:21:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.319+0000 7f62df7fe700 1 -- 172.21.15.19:0/1973791538 learned_addr learned my addr 172.21.15.19:0/1973791538 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.320+0000 7f62de7fc700 1 -- 172.21.15.19:0/1973791538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62e01086a0 msgr2=0x7f62e01a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.320+0000 7f62de7fc700 1 --2- 172.21.15.19:0/1973791538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62e01086a0 0x7f62e01a5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.320+0000 7f62de7fc700 1 -- 172.21.15.19:0/1973791538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0103250 msgr2=0x7f62e01a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.320+0000 7f62de7fc700 1 --2- 172.21.15.19:0/1973791538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0103250 0x7f62e01a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.320+0000 7f62de7fc700 1 -- 172.21.15.19:0/1973791538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e01a6290 con 0x7f62e0103ba0 2024-05-09T02:21:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.320+0000 7f62df7fe700 1 --2- 172.21.15.19:0/1973791538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62e01086a0 0x7f62e01a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:21:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.320+0000 7f62de7fc700 1 --2- 172.21.15.19:0/1973791538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0103ba0 0x7f62e01a1960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f62d400e9c0 tx=0x7f62d400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.320+0000 7f62c7fff700 1 -- 172.21.15.19:0/1973791538 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d400c9c0 con 0x7f62e0103ba0 2024-05-09T02:21:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.320+0000 7f62c7fff700 1 -- 172.21.15.19:0/1973791538 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d400cb60 con 0x7f62e0103ba0 2024-05-09T02:21:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.320+0000 7f62c7fff700 1 -- 172.21.15.19:0/1973791538 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d4018420 con 0x7f62e0103ba0 2024-05-09T02:21:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.321+0000 7f62e55b7700 1 -- 172.21.15.19:0/1973791538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e01a6580 con 0x7f62e0103ba0 2024-05-09T02:21:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.321+0000 7f62e55b7700 1 -- 172.21.15.19:0/1973791538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e01a69b0 con 0x7f62e0103ba0 2024-05-09T02:21:10.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.325+0000 7f62c7fff700 1 -- 172.21.15.19:0/1973791538 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f62d4010430 con 0x7f62e0103ba0 2024-05-09T02:21:10.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.326+0000 7f62c7fff700 1 --2- 172.21.15.19:0/1973791538 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f62c80b2bc0 0x7f62c80b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.326+0000 7f62c7fff700 1 -- 172.21.15.19:0/1973791538 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f62d40e2690 con 0x7f62e0103ba0 2024-05-09T02:21:10.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.326+0000 7f62deffd700 1 --2- 172.21.15.19:0/1973791538 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f62c80b2bc0 0x7f62c80b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.327+0000 7f62deffd700 1 --2- 172.21.15.19:0/1973791538 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f62c80b2bc0 0x7f62c80b50d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009c60 tx=0x7f62d0009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:10.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.330+0000 7f62f3fcdb80 1 Processor -- start 2024-05-09T02:21:10.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.330+0000 7f62f3fcdb80 1 -- start start 2024-05-09T02:21:10.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.330+0000 7f62f3fcdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778f541e0 0x564778fed570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.330+0000 7f62f3fcdb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778e6a520 0x564778fedab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62f3fcdb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778fe7720 0x564778fe7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62f3fcdb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564778f0a070 con 0x564778f541e0 2024-05-09T02:21:10.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62f3fcdb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564778ea8870 con 0x564778fe7720 2024-05-09T02:21:10.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62f3fcdb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564778f06030 con 0x564778e6a520 2024-05-09T02:21:10.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778f541e0 0x564778fed570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778f541e0 0x564778fed570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53498/0 (socket says 172.21.15.19:53498) 2024-05-09T02:21:10.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62deffd700 1 -- 172.21.15.19:0/1937050567 learned_addr learned my addr 172.21.15.19:0/1937050567 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:10.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62df7fe700 1 --2- 172.21.15.19:0/1937050567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778fe7720 0x564778fe7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62de7fc700 1 --2- 172.21.15.19:0/1937050567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778e6a520 0x564778fedab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62de7fc700 1 -- 172.21.15.19:0/1937050567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778fe7720 msgr2=0x564778fe7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:10.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62de7fc700 1 --2- 172.21.15.19:0/1937050567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778fe7720 0x564778fe7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62de7fc700 1 -- 172.21.15.19:0/1937050567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778f541e0 msgr2=0x564778fed570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:10.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62de7fc700 1 --2- 172.21.15.19:0/1937050567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778f541e0 0x564778fed570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62de7fc700 1 -- 172.21.15.19:0/1937050567 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564778ea03f0 con 0x564778e6a520 2024-05-09T02:21:10.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62df7fe700 1 --2- 172.21.15.19:0/1937050567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778fe7720 0x564778fe7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:21:10.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.331+0000 7f62de7fc700 1 --2- 172.21.15.19:0/1937050567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778e6a520 0x564778fedab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f62d40dfd20 tx=0x7f62d40a97b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69cc3ab897171799 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:10.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.332+0000 7f62c57fa700 1 -- 172.21.15.19:0/1937050567 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d40a98b0 con 0x564778e6a520 2024-05-09T02:21:10.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.332+0000 7f62c57fa700 1 -- 172.21.15.19:0/1937050567 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d40adc60 con 0x564778e6a520 2024-05-09T02:21:10.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.332+0000 7f62c57fa700 1 -- 172.21.15.19:0/1937050567 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d40a9b10 con 0x564778e6a520 2024-05-09T02:21:10.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.333+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/1937050567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778e6a520 msgr2=0x564778fedab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:10.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.333+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/1937050567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778e6a520 0x564778fedab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f62d40dfd20 tx=0x7f62d40a97b0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.333+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/1937050567 shutdown_connections 2024-05-09T02:21:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.333+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/1937050567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778fe7720 0x564778fe7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.333+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/1937050567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778f541e0 0x564778fed570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.333+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/1937050567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778e6a520 0x564778fedab0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.333+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/1937050567 >> 172.21.15.19:0/1937050567 conn(0x564778eca720 msgr2=0x564778fe2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.334+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/1937050567 shutdown_connections 2024-05-09T02:21:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.334+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/1937050567 wait complete. 2024-05-09T02:21:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.334+0000 7f62f3fcdb80 1 -- start start 2024-05-09T02:21:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.334+0000 7f62f3fcdb80 1 Processor -- start 2024-05-09T02:21:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.335+0000 7f62f3fcdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778e6a520 0x564778ff1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.335+0000 7f62f3fcdb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778f541e0 0x564778ff1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.335+0000 7f62f3fcdb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778fe7720 0x564778ff1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.335+0000 7f62deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778e6a520 0x564778ff1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.335+0000 7f62deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778e6a520 0x564778ff1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53502/0 (socket says 172.21.15.19:53502) 2024-05-09T02:21:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.335+0000 7f62deffd700 1 -- 172.21.15.19:0/445886946 learned_addr learned my addr 172.21.15.19:0/445886946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.335+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/445886946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564778d85c20 con 0x564778e6a520 2024-05-09T02:21:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.335+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/445886946 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564778ec41d0 con 0x564778f541e0 2024-05-09T02:21:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.335+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/445886946 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564778fe8560 con 0x564778fe7720 2024-05-09T02:21:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.336+0000 7f62f3fcdb80 10 client.0 _ll_register_callbacks cb 0x564778d7b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.336+0000 7f62df7fe700 1 --2- 172.21.15.19:0/445886946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778fe7720 0x564778ff1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.336+0000 7f62de7fc700 1 --2- 172.21.15.19:0/445886946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778f541e0 0x564778ff1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.336+0000 7f62deffd700 1 -- 172.21.15.19:0/445886946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778fe7720 msgr2=0x564778ff1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.336+0000 7f62deffd700 1 --2- 172.21.15.19:0/445886946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778fe7720 0x564778ff1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.336+0000 7f62deffd700 1 -- 172.21.15.19:0/445886946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778f541e0 msgr2=0x564778ff1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.336+0000 7f62deffd700 1 --2- 172.21.15.19:0/445886946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778f541e0 0x564778ff1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.336+0000 7f62deffd700 1 -- 172.21.15.19:0/445886946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564778d85510 con 0x564778e6a520 2024-05-09T02:21:10.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.337+0000 7f62deffd700 1 --2- 172.21.15.19:0/445886946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778e6a520 0x564778ff1230 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f62d0003930 tx=0x7f62d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf499a94957efea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:10.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.337+0000 7f62c2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:21:10.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.337+0000 7f62c2ff5700 1 -- 172.21.15.19:0/445886946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d001d550 con 0x564778e6a520 2024-05-09T02:21:10.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.337+0000 7f62c2ff5700 1 -- 172.21.15.19:0/445886946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f62d001d6f0 con 0x564778e6a520 2024-05-09T02:21:10.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.337+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/445886946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62d40adc60 con 0x564778e6a520 2024-05-09T02:21:10.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.337+0000 7f62c2ff5700 1 -- 172.21.15.19:0/445886946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f62d001d8f0 con 0x564778e6a520 2024-05-09T02:21:10.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.337+0000 7f62f3fcdb80 10 client.16527 fetch_fsmap learned FSMap version 54 2024-05-09T02:21:10.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.337+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/445886946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x564778ffebe0 con 0x564778e6a520 2024-05-09T02:21:10.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.338+0000 7f62c2ff5700 1 -- 172.21.15.19:0/445886946 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d40adc60 con 0x564778e6a520 2024-05-09T02:21:10.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.338+0000 7f62c2ff5700 1 -- 172.21.15.19:0/445886946 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f62d002fbd0 con 0x564778e6a520 2024-05-09T02:21:10.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.338+0000 7f62f3fcdb80 10 client.16527 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:21:10.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.338+0000 7f62f3fcdb80 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:21:10.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.338+0000 7f62f3fcdb80 20 client.16527 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:10.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.338+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/445886946 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564779001630 0x564779003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.338+0000 7f62f3fcdb80 4 client.16527 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:21:10.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.338+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/445886946 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564778f17a50 con 0x564779001630 2024-05-09T02:21:10.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.339+0000 7f62de7fc700 1 --2- 172.21.15.19:0/445886946 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564779001630 0x564779003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.339+0000 7f62c2ff5700 10 client.16527 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:10.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.340+0000 7f62de7fc700 1 --2- 172.21.15.19:0/445886946 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564779001630 0x564779003b00 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19da170512d2929d server_cookie=5be34df5e09acff6 in_seq=0 out_seq=0 2024-05-09T02:21:10.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.340+0000 7f62c2ff5700 1 -- 172.21.15.19:0/445886946 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564778f17a50 con 0x564779001630 2024-05-09T02:21:10.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.340+0000 7f62c2ff5700 10 client.16527 handle_command_reply: tid=0 2024-05-09T02:21:10.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.341+0000 7f62f3fcdb80 1 client.16527 shutdown 2024-05-09T02:21:10.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.341+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/445886946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778e6a520 msgr2=0x564778ff1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:10.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.341+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/445886946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778e6a520 0x564778ff1230 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f62d0003930 tx=0x7f62d00047c0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.341+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/445886946 shutdown_connections 2024-05-09T02:21:10.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.341+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/445886946 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x564779001630 0x564779003b00 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.341+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/445886946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778fe7720 0x564778ff1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.341+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/445886946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778e6a520 0x564778ff1230 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.341+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/445886946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778f541e0 0x564778ff1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.341+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/445886946 >> 172.21.15.19:0/445886946 conn(0x564778eca720 msgr2=0x564778fe2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:10.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.341+0000 7f62c2ff5700 0 client.16527 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:10.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.341+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/445886946 shutdown_connections 2024-05-09T02:21:10.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.341+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/445886946 wait complete. 2024-05-09T02:21:10.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.341+0000 7f62f3fcdb80 20 client.16527 trim_cache size 0 max 16384 2024-05-09T02:21:10.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.370+0000 7f62f3fcdb80 1 Processor -- start 2024-05-09T02:21:10.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.370+0000 7f62f3fcdb80 1 -- start start 2024-05-09T02:21:10.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.370+0000 7f62f3fcdb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778e6a520 0x564778fec940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.370+0000 7f62f3fcdb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778f541e0 0x564778ffcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.370+0000 7f62f3fcdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778fe7720 0x5647790092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.370+0000 7f62f3fcdb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01b55e0 con 0x564778fe7720 2024-05-09T02:21:10.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.371+0000 7f62f3fcdb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01b7270 con 0x564778f541e0 2024-05-09T02:21:10.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.371+0000 7f62deffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778e6a520 0x564778fec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.371+0000 7f62df7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778fe7720 0x5647790092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.371+0000 7f62deffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778e6a520 0x564778fec940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:60080/0 (socket says 172.21.15.19:60080) 2024-05-09T02:21:10.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.371+0000 7f62deffd700 1 -- 172.21.15.19:0/4211768314 learned_addr learned my addr 172.21.15.19:0/4211768314 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:10.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.371+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/4211768314 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01b5cb0 con 0x564778e6a520 2024-05-09T02:21:10.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.371+0000 7f62de7fc700 1 --2- 172.21.15.19:0/4211768314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778f541e0 0x564778ffcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.372+0000 7f62deffd700 1 -- 172.21.15.19:0/4211768314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778f541e0 msgr2=0x564778ffcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:10.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.372+0000 7f62deffd700 1 --2- 172.21.15.19:0/4211768314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778f541e0 0x564778ffcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.372+0000 7f62deffd700 1 -- 172.21.15.19:0/4211768314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778fe7720 msgr2=0x5647790092c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:10.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.372+0000 7f62deffd700 1 --2- 172.21.15.19:0/4211768314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778fe7720 0x5647790092c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.372+0000 7f62deffd700 1 -- 172.21.15.19:0/4211768314 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564778fe3450 con 0x564778e6a520 2024-05-09T02:21:10.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.372+0000 7f62de7fc700 1 --2- 172.21.15.19:0/4211768314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778f541e0 0x564778ffcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:21:10.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.372+0000 7f62deffd700 1 --2- 172.21.15.19:0/4211768314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778e6a520 0x564778fec940 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f62d0011ab0 tx=0x7f62d0009350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef58b90196550694 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:10.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.372+0000 7f62c4ff9700 1 -- 172.21.15.19:0/4211768314 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d001c6f0 con 0x564778e6a520 2024-05-09T02:21:10.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.372+0000 7f62c4ff9700 1 -- 172.21.15.19:0/4211768314 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d001d550 con 0x564778e6a520 2024-05-09T02:21:10.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.372+0000 7f62c4ff9700 1 -- 172.21.15.19:0/4211768314 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d002f670 con 0x564778e6a520 2024-05-09T02:21:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.373+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/4211768314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778e6a520 msgr2=0x564778fec940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.373+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/4211768314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778e6a520 0x564778fec940 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f62d0011ab0 tx=0x7f62d0009350 comp rx=0 tx=0).stop 2024-05-09T02:21:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.373+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/4211768314 shutdown_connections 2024-05-09T02:21:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.373+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/4211768314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778fe7720 0x5647790092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.373+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/4211768314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778e6a520 0x564778fec940 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.373+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/4211768314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778f541e0 0x564778ffcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.373+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/4211768314 >> 172.21.15.19:0/4211768314 conn(0x564778eca720 msgr2=0x564778fec230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.373+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/4211768314 shutdown_connections 2024-05-09T02:21:10.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.373+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/4211768314 wait complete. 2024-05-09T02:21:10.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.374+0000 7f62f3fcdb80 1 -- start start 2024-05-09T02:21:10.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.374+0000 7f62f3fcdb80 1 Processor -- start 2024-05-09T02:21:10.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.374+0000 7f62f3fcdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778e6a520 0x564779008a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.374+0000 7f62f3fcdb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778f541e0 0x564779008f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.374+0000 7f62f3fcdb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778fe7720 0x564779009480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.374+0000 7f62f3fcdb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01b6b10 con 0x564778e6a520 2024-05-09T02:21:10.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.374+0000 7f62f3fcdb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01b6380 con 0x564778fe7720 2024-05-09T02:21:10.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.375+0000 7f62f3fcdb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01b35a0 con 0x564778f541e0 2024-05-09T02:21:10.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.375+0000 7f62f3fcdb80 10 client.0 _ll_register_callbacks cb 0x564778d7b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:10.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.375+0000 7f62de7fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778f541e0 0x564779008f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.375+0000 7f62deffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778e6a520 0x564779008a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.375+0000 7f62de7fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778f541e0 0x564779008f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:60086/0 (socket says 172.21.15.19:60086) 2024-05-09T02:21:10.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.375+0000 7f62de7fc700 1 -- 172.21.15.19:0/3385984803 learned_addr learned my addr 172.21.15.19:0/3385984803 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:10.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.375+0000 7f62df7fe700 1 --2- 172.21.15.19:0/3385984803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778fe7720 0x564779009480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.375+0000 7f62de7fc700 1 -- 172.21.15.19:0/3385984803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778fe7720 msgr2=0x564779009480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:10.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.375+0000 7f62de7fc700 1 --2- 172.21.15.19:0/3385984803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778fe7720 0x564779009480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.375+0000 7f62de7fc700 1 -- 172.21.15.19:0/3385984803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778e6a520 msgr2=0x564779008a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:10.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.375+0000 7f62de7fc700 1 --2- 172.21.15.19:0/3385984803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778e6a520 0x564779008a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.375+0000 7f62de7fc700 1 -- 172.21.15.19:0/3385984803 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56477906ae70 con 0x564778f541e0 2024-05-09T02:21:10.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.376+0000 7f62de7fc700 1 --2- 172.21.15.19:0/3385984803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778f541e0 0x564779008f40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f62d4005fd0 tx=0x7f62d4009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3c6c2b76d1b9943 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:10.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.376+0000 7f62c2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:21:10.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.376+0000 7f62c2ff5700 1 -- 172.21.15.19:0/3385984803 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d40c2020 con 0x564778f541e0 2024-05-09T02:21:10.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.377+0000 7f62c2ff5700 1 -- 172.21.15.19:0/3385984803 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f62d40dffa0 con 0x564778f541e0 2024-05-09T02:21:10.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.377+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/3385984803 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62d001d550 con 0x564778f541e0 2024-05-09T02:21:10.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.377+0000 7f62c2ff5700 1 -- 172.21.15.19:0/3385984803 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f62d40a5770 con 0x564778f541e0 2024-05-09T02:21:10.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.377+0000 7f62f3fcdb80 10 client.26299 fetch_fsmap learned FSMap version 54 2024-05-09T02:21:10.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.377+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/3385984803 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56477906bf10 con 0x564778f541e0 2024-05-09T02:21:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.377+0000 7f62c2ff5700 1 -- 172.21.15.19:0/3385984803 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d001d550 con 0x564778f541e0 2024-05-09T02:21:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.377+0000 7f62c2ff5700 1 -- 172.21.15.19:0/3385984803 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 54) ==== 8382+0+0 (secure 0 0 0) 0x7f62d40aabe0 con 0x564778f541e0 2024-05-09T02:21:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.377+0000 7f62f3fcdb80 10 client.26299 fetch_fsmap finished waiting for FSMap version 54 2024-05-09T02:21:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.377+0000 7f62f3fcdb80 10 client.26299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:21:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.377+0000 7f62f3fcdb80 20 client.26299 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.377+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/3385984803 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56477906c290 0x56477906e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.377+0000 7f62f3fcdb80 4 client.26299 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T02:21:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.377+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/3385984803 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564778f11af0 con 0x56477906c290 2024-05-09T02:21:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.378+0000 7f62deffd700 1 --2- 172.21.15.19:0/3385984803 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56477906c290 0x56477906e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.378+0000 7f62c2ff5700 10 client.26299 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.378+0000 7f62deffd700 1 --2- 172.21.15.19:0/3385984803 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56477906c290 0x56477906e6b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c6f0d8a0c0b3c3 server_cookie=b733fe4e902c774d in_seq=0 out_seq=0 2024-05-09T02:21:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.379+0000 7f62c2ff5700 1 -- 172.21.15.19:0/3385984803 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564778f11af0 con 0x56477906c290 2024-05-09T02:21:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.379+0000 7f62c2ff5700 10 client.26299 handle_command_reply: tid=0 2024-05-09T02:21:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.379+0000 7f62f3fcdb80 1 client.26299 shutdown 2024-05-09T02:21:10.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.379+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/3385984803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778f541e0 msgr2=0x564779008f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.379+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/3385984803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778f541e0 0x564779008f40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f62d4005fd0 tx=0x7f62d4009f70 comp rx=0 tx=0).stop 2024-05-09T02:21:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.379+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/3385984803 shutdown_connections 2024-05-09T02:21:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.379+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/3385984803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564778fe7720 0x564779009480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.379+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/3385984803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564778e6a520 0x564779008a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.379+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/3385984803 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56477906c290 0x56477906e6b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.379+0000 7f62f3fcdb80 1 --2- 172.21.15.19:0/3385984803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564778f541e0 0x564779008f40 secure :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f62d4005fd0 tx=0x7f62d4009f70 comp rx=0 tx=0).stop 2024-05-09T02:21:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.379+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/3385984803 >> 172.21.15.19:0/3385984803 conn(0x564778eca720 msgr2=0x564778feb610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.380+0000 7f62c2ff5700 0 client.26299 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.380+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/3385984803 shutdown_connections 2024-05-09T02:21:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.380+0000 7f62f3fcdb80 1 -- 172.21.15.19:0/3385984803 wait complete. 2024-05-09T02:21:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.380+0000 7f62f3fcdb80 20 client.26299 trim_cache size 0 max 16384 2024-05-09T02:21:10.386 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:21:10.386 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T02:21:10.386 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "55ce619a-8f4c-4e98-9e1f-fc84e0baef8a", 2024-05-09T02:21:10.386 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T02:21:10.386 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:21:10.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.381+0000 7f62e55b7700 1 -- 172.21.15.19:0/1973791538 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f62c80b2bc0 msgr2=0x7f62c80b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:10.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.381+0000 7f62e55b7700 1 --2- 172.21.15.19:0/1973791538 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f62c80b2bc0 0x7f62c80b50d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009c60 tx=0x7f62d0009260 comp rx=0 tx=0).stop 2024-05-09T02:21:10.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.381+0000 7f62e55b7700 1 -- 172.21.15.19:0/1973791538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0103ba0 msgr2=0x7f62e01a1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:10.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.381+0000 7f62e55b7700 1 --2- 172.21.15.19:0/1973791538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0103ba0 0x7f62e01a1960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f62d400e9c0 tx=0x7f62d400ecd0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.381+0000 7f62e55b7700 1 -- 172.21.15.19:0/1973791538 shutdown_connections 2024-05-09T02:21:10.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.381+0000 7f62e55b7700 1 --2- 172.21.15.19:0/1973791538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62e01086a0 0x7f62e01a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.381+0000 7f62e55b7700 1 --2- 172.21.15.19:0/1973791538 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f62c80b2bc0 0x7f62c80b50d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.381+0000 7f62e55b7700 1 --2- 172.21.15.19:0/1973791538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e0103250 0x7f62e01a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.381+0000 7f62e55b7700 1 --2- 172.21.15.19:0/1973791538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0103ba0 0x7f62e01a1960 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:10.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.381+0000 7f62e55b7700 1 -- 172.21.15.19:0/1973791538 >> 172.21.15.19:0/1973791538 conn(0x7f62e00fedc0 msgr2=0x7f62e01060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:10.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.381+0000 7f62e55b7700 1 -- 172.21.15.19:0/1973791538 shutdown_connections 2024-05-09T02:21:10.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:10.381+0000 7f62e55b7700 1 -- 172.21.15.19:0/1973791538 wait complete. 2024-05-09T02:21:10.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:10 smithi019 ceph-mon[30151]: pgmap v306: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 1.75k op/s 2024-05-09T02:21:10.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:10 smithi196 ceph-mon[34718]: pgmap v306: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 1.75k op/s 2024-05-09T02:21:10.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:10 smithi059 ceph-mon[34781]: pgmap v306: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 1.75k op/s 2024-05-09T02:21:11.056 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:21:11.428 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:21:11.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub queued for path: / 2024-05-09T02:21:11.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:11.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:11.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:11.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:11.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:11.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:11.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:11.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:11.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:11.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:11 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub queued for path: / 2024-05-09T02:21:11.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:11.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:11.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:11.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:11.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:11.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:11.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:11.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:11.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:11.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:11 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub queued for path: / 2024-05-09T02:21:11.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:11.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:11.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:11.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:11.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:11.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:11.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:11.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:11.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:11.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:11.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:11 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:12.656 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 570.0 s, 2064.2 tps, lat 48.358 ms stddev 54.478 2024-05-09T02:21:12.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:12 smithi196 ceph-mon[34718]: pgmap v307: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 1.75k op/s 2024-05-09T02:21:12.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:12 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:21:12.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:12 smithi019 ceph-mon[30151]: pgmap v307: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 1.75k op/s 2024-05-09T02:21:12.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:12 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:21:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:12 smithi059 ceph-mon[34781]: pgmap v307: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 1.75k op/s 2024-05-09T02:21:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:12 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:21:14.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:14 smithi196 ceph-mon[34718]: pgmap v308: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 19 MiB/s wr, 1.80k op/s 2024-05-09T02:21:14.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:14 smithi019 ceph-mon[30151]: pgmap v308: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 19 MiB/s wr, 1.80k op/s 2024-05-09T02:21:14.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:14 smithi059 ceph-mon[34781]: pgmap v308: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 19 MiB/s wr, 1.80k op/s 2024-05-09T02:21:15.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:15 smithi019 ceph-mon[30151]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T02:21:15.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:15 smithi196 ceph-mon[34718]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T02:21:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.961+0000 7f65797d1700 1 -- 172.21.15.19:0/3761526929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65741029c0 msgr2=0x7f6574102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.961+0000 7f65797d1700 1 --2- 172.21.15.19:0/3761526929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65741029c0 0x7f6574102dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6564009990 tx=0x7f6564009ca0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.961+0000 7f65797d1700 1 -- 172.21.15.19:0/3761526929 shutdown_connections 2024-05-09T02:21:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.961+0000 7f65797d1700 1 --2- 172.21.15.19:0/3761526929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6574103310 0x7f657406ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.961+0000 7f65797d1700 1 --2- 172.21.15.19:0/3761526929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65741089f0 0x7f6574108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.961+0000 7f65797d1700 1 --2- 172.21.15.19:0/3761526929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65741029c0 0x7f6574102dd0 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6564009990 tx=0x7f6564009ca0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.961+0000 7f65797d1700 1 -- 172.21.15.19:0/3761526929 >> 172.21.15.19:0/3761526929 conn(0x7f65740fe6f0 msgr2=0x7f6574100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:15.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.963+0000 7f65797d1700 1 -- 172.21.15.19:0/3761526929 shutdown_connections 2024-05-09T02:21:15.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.963+0000 7f65797d1700 1 -- 172.21.15.19:0/3761526929 wait complete. 2024-05-09T02:21:15.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:15 smithi059 ceph-mon[34781]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T02:21:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.964+0000 7f65797d1700 1 Processor -- start 2024-05-09T02:21:15.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.965+0000 7f65797d1700 1 -- start start 2024-05-09T02:21:15.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.965+0000 7f65797d1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6574103310 0x7f657419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:15.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.965+0000 7f65797d1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65741089f0 0x7f657419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:15.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.965+0000 7f65797d1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657419f9c0 0x7f657419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:15.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.966+0000 7f6572ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6574103310 0x7f657419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:15.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.966+0000 7f6572ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6574103310 0x7f657419b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:51788/0 (socket says 172.21.15.19:51788) 2024-05-09T02:21:15.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.966+0000 7f6572ffd700 1 -- 172.21.15.19:0/3385047663 learned_addr learned my addr 172.21.15.19:0/3385047663 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.966+0000 7f65797d1700 1 -- 172.21.15.19:0/3385047663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6574113530 con 0x7f657419f9c0 2024-05-09T02:21:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.966+0000 7f65797d1700 1 -- 172.21.15.19:0/3385047663 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65741133b0 con 0x7f6574103310 2024-05-09T02:21:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.966+0000 7f65797d1700 1 -- 172.21.15.19:0/3385047663 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65741136b0 con 0x7f65741089f0 2024-05-09T02:21:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.966+0000 7f65737fe700 1 --2- 172.21.15.19:0/3385047663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657419f9c0 0x7f657419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.967+0000 7f6572ffd700 1 -- 172.21.15.19:0/3385047663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65741089f0 msgr2=0x7f657419b840 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T02:21:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.967+0000 7f6572ffd700 1 --2- 172.21.15.19:0/3385047663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65741089f0 0x7f657419b840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.967+0000 7f6572ffd700 1 -- 172.21.15.19:0/3385047663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657419f9c0 msgr2=0x7f657419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.967+0000 7f6572ffd700 1 --2- 172.21.15.19:0/3385047663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657419f9c0 0x7f657419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.967+0000 7f6572ffd700 1 -- 172.21.15.19:0/3385047663 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65741a0510 con 0x7f6574103310 2024-05-09T02:21:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.967+0000 7f6572ffd700 1 --2- 172.21.15.19:0/3385047663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6574103310 0x7f657419b2a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f655c00b500 tx=0x7f655c00b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.967+0000 7f6570ff9700 1 -- 172.21.15.19:0/3385047663 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c015020 con 0x7f6574103310 2024-05-09T02:21:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.967+0000 7f65797d1700 1 -- 172.21.15.19:0/3385047663 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65741a0800 con 0x7f6574103310 2024-05-09T02:21:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.967+0000 7f6570ff9700 1 -- 172.21.15.19:0/3385047663 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f655c011ce0 con 0x7f6574103310 2024-05-09T02:21:15.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.968+0000 7f6570ff9700 1 -- 172.21.15.19:0/3385047663 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c010dd0 con 0x7f6574103310 2024-05-09T02:21:15.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.968+0000 7f65797d1700 1 -- 172.21.15.19:0/3385047663 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6574114050 con 0x7f6574103310 2024-05-09T02:21:15.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.977+0000 7f65881e7b80 1 Processor -- start 2024-05-09T02:21:15.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.977+0000 7f65881e7b80 1 -- start start 2024-05-09T02:21:15.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.978+0000 7f6570ff9700 1 -- 172.21.15.19:0/3385047663 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f655c018020 con 0x7f6574103310 2024-05-09T02:21:15.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.978+0000 7f65881e7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30f1f160 0x55cb30fb84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:15.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.978+0000 7f65881e7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 0x55cb30fb8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:15.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.978+0000 7f65881e7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30fb26a0 0x55cb30fb2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:15.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.978+0000 7f65737fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30f1f160 0x55cb30fb84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:15.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.979+0000 7f65737fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30f1f160 0x55cb30fb84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:51792/0 (socket says 172.21.15.19:51792) 2024-05-09T02:21:15.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.979+0000 7f65737fe700 1 -- 172.21.15.19:0/3268227804 learned_addr learned my addr 172.21.15.19:0/3268227804 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:15.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.978+0000 7f65881e7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cb30ed4ff0 con 0x55cb30e354a0 2024-05-09T02:21:15.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.979+0000 7f65881e7b80 1 -- 172.21.15.19:0/3268227804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cb30e737f0 con 0x55cb30f1f160 2024-05-09T02:21:15.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.979+0000 7f65881e7b80 1 -- 172.21.15.19:0/3268227804 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cb30ed0fb0 con 0x55cb30fb26a0 2024-05-09T02:21:15.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.979+0000 7f656bfff700 1 --2- 172.21.15.19:0/3268227804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30fb26a0 0x55cb30fb2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:15.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.979+0000 7f6572ffd700 1 --2- 172.21.15.19:0/3268227804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 0x55cb30fb8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:15.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.980+0000 7f65737fe700 1 -- 172.21.15.19:0/3268227804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30fb26a0 msgr2=0x55cb30fb2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:15.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.980+0000 7f65737fe700 1 --2- 172.21.15.19:0/3268227804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30fb26a0 0x55cb30fb2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.980+0000 7f65737fe700 1 -- 172.21.15.19:0/3268227804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 msgr2=0x55cb30fb8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:15.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.980+0000 7f65737fe700 1 --2- 172.21.15.19:0/3268227804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 0x55cb30fb8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.980+0000 7f65737fe700 1 -- 172.21.15.19:0/3268227804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb30e6b370 con 0x55cb30f1f160 2024-05-09T02:21:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.980+0000 7f656bfff700 1 --2- 172.21.15.19:0/3268227804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30fb26a0 0x55cb30fb2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:21:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.980+0000 7f65737fe700 1 --2- 172.21.15.19:0/3268227804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30f1f160 0x55cb30fb84f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f656c00a4d0 tx=0x7f656c009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6bb0dfaaa3c963f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.981+0000 7f65697fa700 1 -- 172.21.15.19:0/3268227804 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c014420 con 0x55cb30f1f160 2024-05-09T02:21:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.981+0000 7f65697fa700 1 -- 172.21.15.19:0/3268227804 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f656c019070 con 0x55cb30f1f160 2024-05-09T02:21:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.981+0000 7f6570ff9700 1 --2- 172.21.15.19:0/3385047663 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65600b2b20 0x7f65600b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.982+0000 7f6572ffd700 1 --2- 172.21.15.19:0/3385047663 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65600b2b20 0x7f65600b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:15.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.982+0000 7f6570ff9700 1 -- 172.21.15.19:0/3385047663 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f655c0e1c80 con 0x7f6574103310 2024-05-09T02:21:15.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.983+0000 7f65697fa700 1 -- 172.21.15.19:0/3268227804 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c0138c0 con 0x55cb30f1f160 2024-05-09T02:21:15.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.983+0000 7f65881e7b80 1 -- 172.21.15.19:0/3268227804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30f1f160 msgr2=0x55cb30fb84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:15.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.983+0000 7f65881e7b80 1 --2- 172.21.15.19:0/3268227804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30f1f160 0x55cb30fb84f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f656c00a4d0 tx=0x7f656c009650 comp rx=0 tx=0).stop 2024-05-09T02:21:15.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.983+0000 7f6572ffd700 1 --2- 172.21.15.19:0/3385047663 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65600b2b20 0x7f65600b5030 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x55cb30faeed0 tx=0x7f655c0e56b0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:15.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.984+0000 7f65881e7b80 1 -- 172.21.15.19:0/3268227804 shutdown_connections 2024-05-09T02:21:15.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.984+0000 7f65881e7b80 1 --2- 172.21.15.19:0/3268227804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30fb26a0 0x55cb30fb2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.984+0000 7f65881e7b80 1 --2- 172.21.15.19:0/3268227804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30f1f160 0x55cb30fb84f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.984+0000 7f65881e7b80 1 --2- 172.21.15.19:0/3268227804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 0x55cb30fb8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.984+0000 7f65881e7b80 1 -- 172.21.15.19:0/3268227804 >> 172.21.15.19:0/3268227804 conn(0x55cb30e956a0 msgr2=0x55cb30fad400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:15.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.984+0000 7f65881e7b80 1 -- 172.21.15.19:0/3268227804 shutdown_connections 2024-05-09T02:21:15.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.984+0000 7f65881e7b80 1 -- 172.21.15.19:0/3268227804 wait complete. 2024-05-09T02:21:15.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.985+0000 7f65881e7b80 1 -- start start 2024-05-09T02:21:15.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.985+0000 7f65881e7b80 1 Processor -- start 2024-05-09T02:21:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.985+0000 7f65881e7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30e354a0 0x55cb30fbc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.985+0000 7f65881e7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30f1f160 0x55cb30fbc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.985+0000 7f65881e7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30fb26a0 0x55cb30fbcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.985+0000 7f65881e7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cb30d50ba0 con 0x55cb30f1f160 2024-05-09T02:21:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.985+0000 7f65881e7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cb30e8f150 con 0x55cb30fb26a0 2024-05-09T02:21:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.985+0000 7f65881e7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cb30fbf1d0 con 0x55cb30e354a0 2024-05-09T02:21:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.986+0000 7f65737fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30e354a0 0x55cb30fbc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.986+0000 7f65737fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30fb26a0 0x55cb30fbcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.986+0000 7f65881e7b80 10 client.0 _ll_register_callbacks cb 0x55cb30d466c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.986+0000 7f65737fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30e354a0 0x55cb30fbc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:60128/0 (socket says 172.21.15.19:60128) 2024-05-09T02:21:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.986+0000 7f65737fe700 1 -- 172.21.15.19:0/4105943933 learned_addr learned my addr 172.21.15.19:0/4105943933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.986+0000 7f656bfff700 1 --2- 172.21.15.19:0/4105943933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30f1f160 0x55cb30fbc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.986+0000 7f65737fe700 1 -- 172.21.15.19:0/4105943933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30fb26a0 msgr2=0x55cb30fbcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.986+0000 7f65737fe700 1 --2- 172.21.15.19:0/4105943933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30fb26a0 0x55cb30fbcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.986+0000 7f65737fe700 1 -- 172.21.15.19:0/4105943933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30f1f160 msgr2=0x55cb30fbc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:15.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.986+0000 7f65737fe700 1 --2- 172.21.15.19:0/4105943933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30f1f160 0x55cb30fbc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.986+0000 7f65737fe700 1 -- 172.21.15.19:0/4105943933 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb30d50490 con 0x55cb30e354a0 2024-05-09T02:21:15.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.986+0000 7f656bfff700 1 --2- 172.21.15.19:0/4105943933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30f1f160 0x55cb30fbc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:21:15.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.987+0000 7f65737fe700 1 --2- 172.21.15.19:0/4105943933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30e354a0 0x55cb30fbc1a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f656c013a60 tx=0x7f656c009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b3c8701d897775b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:15.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.987+0000 7f655a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:21:15.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.988+0000 7f655a7fc700 1 -- 172.21.15.19:0/4105943933 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c024020 con 0x55cb30e354a0 2024-05-09T02:21:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.988+0000 7f655a7fc700 1 -- 172.21.15.19:0/4105943933 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f656c01e430 con 0x55cb30e354a0 2024-05-09T02:21:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.988+0000 7f65881e7b80 1 -- 172.21.15.19:0/4105943933 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f656c019070 con 0x55cb30e354a0 2024-05-09T02:21:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.990+0000 7f655a7fc700 1 -- 172.21.15.19:0/4105943933 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f656c032050 con 0x55cb30e354a0 2024-05-09T02:21:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.990+0000 7f65881e7b80 10 client.26305 fetch_fsmap learned FSMap version 55 2024-05-09T02:21:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.990+0000 7f65881e7b80 1 -- 172.21.15.19:0/4105943933 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55cb30fc9c30 con 0x55cb30e354a0 2024-05-09T02:21:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.990+0000 7f655a7fc700 1 -- 172.21.15.19:0/4105943933 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f656c019070 con 0x55cb30e354a0 2024-05-09T02:21:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.990+0000 7f655a7fc700 1 -- 172.21.15.19:0/4105943933 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 55) ==== 8374+0+0 (secure 0 0 0) 0x7f656c0145c0 con 0x55cb30e354a0 2024-05-09T02:21:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.990+0000 7f65881e7b80 10 client.26305 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T02:21:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.990+0000 7f65881e7b80 10 client.26305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:21:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.990+0000 7f65881e7b80 20 client.26305 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.990+0000 7f65881e7b80 1 --2- 172.21.15.19:0/4105943933 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb30fcc270 0x55cb30fce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.991+0000 7f65881e7b80 4 client.26305 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:21:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.991+0000 7f65881e7b80 1 -- 172.21.15.19:0/4105943933 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb30ee29d0 con 0x55cb30fcc270 2024-05-09T02:21:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.991+0000 7f656bfff700 1 --2- 172.21.15.19:0/4105943933 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb30fcc270 0x55cb30fce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.992+0000 7f655a7fc700 10 client.26305 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.992+0000 7f656bfff700 1 --2- 172.21.15.19:0/4105943933 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb30fcc270 0x55cb30fce690 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91875441d13e81e6 server_cookie=415964652237fdd3 in_seq=0 out_seq=0 2024-05-09T02:21:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.993+0000 7f655a7fc700 1 -- 172.21.15.19:0/4105943933 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cb30ee29d0 con 0x55cb30fcc270 2024-05-09T02:21:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.993+0000 7f655a7fc700 10 client.26305 handle_command_reply: tid=0 2024-05-09T02:21:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.993+0000 7f65881e7b80 1 client.26305 shutdown 2024-05-09T02:21:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.993+0000 7f65881e7b80 1 -- 172.21.15.19:0/4105943933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30e354a0 msgr2=0x55cb30fbc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.993+0000 7f65881e7b80 1 --2- 172.21.15.19:0/4105943933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30e354a0 0x55cb30fbc1a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f656c013a60 tx=0x7f656c009c60 comp rx=0 tx=0).stop 2024-05-09T02:21:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.994+0000 7f65881e7b80 1 -- 172.21.15.19:0/4105943933 shutdown_connections 2024-05-09T02:21:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.994+0000 7f65881e7b80 1 --2- 172.21.15.19:0/4105943933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30fb26a0 0x55cb30fbcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.994+0000 7f65881e7b80 1 --2- 172.21.15.19:0/4105943933 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb30fcc270 0x55cb30fce690 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.994+0000 7f65881e7b80 1 --2- 172.21.15.19:0/4105943933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30e354a0 0x55cb30fbc1a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.994+0000 7f65881e7b80 1 --2- 172.21.15.19:0/4105943933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30f1f160 0x55cb30fbc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.994+0000 7f65881e7b80 1 -- 172.21.15.19:0/4105943933 >> 172.21.15.19:0/4105943933 conn(0x55cb30e956a0 msgr2=0x55cb30fad1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.994+0000 7f655a7fc700 0 client.26305 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.994+0000 7f65881e7b80 1 -- 172.21.15.19:0/4105943933 shutdown_connections 2024-05-09T02:21:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.994+0000 7f65881e7b80 1 -- 172.21.15.19:0/4105943933 wait complete. 2024-05-09T02:21:15.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:15.994+0000 7f65881e7b80 20 client.26305 trim_cache size 0 max 16384 2024-05-09T02:21:16.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.022+0000 7f65881e7b80 1 Processor -- start 2024-05-09T02:21:16.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.022+0000 7f65881e7b80 1 -- start start 2024-05-09T02:21:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.023+0000 7f65881e7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 0x55cb30fbb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.023+0000 7f65881e7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30f1f160 0x55cb30fc8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.023+0000 7f65881e7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30fb26a0 0x55cb30fb1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.023+0000 7f65881e7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65741b19f0 con 0x55cb30e354a0 2024-05-09T02:21:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.023+0000 7f65881e7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65741b3620 con 0x55cb30fb26a0 2024-05-09T02:21:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.023+0000 7f65881e7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65741b20c0 con 0x55cb30f1f160 2024-05-09T02:21:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.023+0000 7f656bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30f1f160 0x55cb30fc8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.024+0000 7f656bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30f1f160 0x55cb30fc8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:60144/0 (socket says 172.21.15.19:60144) 2024-05-09T02:21:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.024+0000 7f656bfff700 1 -- 172.21.15.19:0/2833651431 learned_addr learned my addr 172.21.15.19:0/2833651431 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.024+0000 7f65737fe700 1 --2- 172.21.15.19:0/2833651431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 0x55cb30fbb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.024+0000 7f65737fe700 1 --2- 172.21.15.19:0/2833651431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30fb26a0 0x55cb30fb1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.024+0000 7f656bfff700 1 -- 172.21.15.19:0/2833651431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30fb26a0 msgr2=0x55cb30fb1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.024+0000 7f656bfff700 1 --2- 172.21.15.19:0/2833651431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30fb26a0 0x55cb30fb1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.024+0000 7f656bfff700 1 -- 172.21.15.19:0/2833651431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 msgr2=0x55cb30fbb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.024+0000 7f656bfff700 1 --2- 172.21.15.19:0/2833651431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 0x55cb30fbb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.024+0000 7f656bfff700 1 -- 172.21.15.19:0/2833651431 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb30faed00 con 0x55cb30f1f160 2024-05-09T02:21:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.024+0000 7f65737fe700 1 --2- 172.21.15.19:0/2833651431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 0x55cb30fbb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:21:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.024+0000 7f656bfff700 1 --2- 172.21.15.19:0/2833651431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30f1f160 0x55cb30fc8430 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6564006010 tx=0x7f6564005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22fb6ad04455f74e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.024+0000 7f6568ff9700 1 -- 172.21.15.19:0/2833651431 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564003e40 con 0x55cb30f1f160 2024-05-09T02:21:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.025+0000 7f6568ff9700 1 -- 172.21.15.19:0/2833651431 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6564029070 con 0x55cb30f1f160 2024-05-09T02:21:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.025+0000 7f6568ff9700 1 -- 172.21.15.19:0/2833651431 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656400bcd0 con 0x55cb30f1f160 2024-05-09T02:21:16.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.026+0000 7f65881e7b80 1 -- 172.21.15.19:0/2833651431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30f1f160 msgr2=0x55cb30fc8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:16.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.026+0000 7f65881e7b80 1 --2- 172.21.15.19:0/2833651431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30f1f160 0x55cb30fc8430 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6564006010 tx=0x7f6564005040 comp rx=0 tx=0).stop 2024-05-09T02:21:16.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.026+0000 7f65881e7b80 1 -- 172.21.15.19:0/2833651431 shutdown_connections 2024-05-09T02:21:16.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.027+0000 7f65881e7b80 1 --2- 172.21.15.19:0/2833651431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30fb26a0 0x55cb30fb1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.027+0000 7f65881e7b80 1 --2- 172.21.15.19:0/2833651431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 0x55cb30fbb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.027+0000 7f65881e7b80 1 --2- 172.21.15.19:0/2833651431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30f1f160 0x55cb30fc8430 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.027+0000 7f65881e7b80 1 -- 172.21.15.19:0/2833651431 >> 172.21.15.19:0/2833651431 conn(0x55cb30e956a0 msgr2=0x55cb30fb1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:16.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.027+0000 7f65881e7b80 1 -- 172.21.15.19:0/2833651431 shutdown_connections 2024-05-09T02:21:16.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.027+0000 7f65881e7b80 1 -- 172.21.15.19:0/2833651431 wait complete. 2024-05-09T02:21:16.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.027+0000 7f65881e7b80 1 -- start start 2024-05-09T02:21:16.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.028+0000 7f65881e7b80 1 Processor -- start 2024-05-09T02:21:16.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.028+0000 7f65881e7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 0x55cb30fcee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.028+0000 7f65881e7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30f1f160 0x55cb30fcf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.028+0000 7f65881e7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30fb26a0 0x55cb30fcf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.028+0000 7f65737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 0x55cb30fcee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.028+0000 7f65737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 0x55cb30fcee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53558/0 (socket says 172.21.15.19:53558) 2024-05-09T02:21:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.028+0000 7f65737fe700 1 -- 172.21.15.19:0/2523491141 learned_addr learned my addr 172.21.15.19:0/2523491141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.028+0000 7f65881e7b80 1 -- 172.21.15.19:0/2523491141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65741b2f20 con 0x55cb30e354a0 2024-05-09T02:21:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.028+0000 7f65881e7b80 1 -- 172.21.15.19:0/2523491141 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65741b2790 con 0x55cb30f1f160 2024-05-09T02:21:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.028+0000 7f65881e7b80 1 -- 172.21.15.19:0/2523491141 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65741af9b0 con 0x55cb30fb26a0 2024-05-09T02:21:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.028+0000 7f656bfff700 1 --2- 172.21.15.19:0/2523491141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30f1f160 0x55cb30fcf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.028+0000 7f65881e7b80 10 client.0 _ll_register_callbacks cb 0x55cb30d466c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.029+0000 7f65737fe700 1 --2- 172.21.15.19:0/2523491141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30fb26a0 0x55cb30fcf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.029+0000 7f656bfff700 1 -- 172.21.15.19:0/2523491141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30fb26a0 msgr2=0x55cb30fcf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.029+0000 7f656bfff700 1 --2- 172.21.15.19:0/2523491141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30fb26a0 0x55cb30fcf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.029+0000 7f656bfff700 1 -- 172.21.15.19:0/2523491141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 msgr2=0x55cb30fcee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:16.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.029+0000 7f656bfff700 1 --2- 172.21.15.19:0/2523491141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 0x55cb30fcee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.029+0000 7f656bfff700 1 -- 172.21.15.19:0/2523491141 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb3103dfd0 con 0x55cb30f1f160 2024-05-09T02:21:16.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.029+0000 7f656bfff700 1 --2- 172.21.15.19:0/2523491141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30f1f160 0x55cb30fcf3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55cb30fc9be0 tx=0x7f6564004e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11f04c3926778787 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:16.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.029+0000 7f655a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:21:16.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.030+0000 7f655a7fc700 1 -- 172.21.15.19:0/2523491141 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564033020 con 0x55cb30f1f160 2024-05-09T02:21:16.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.030+0000 7f65881e7b80 1 -- 172.21.15.19:0/2523491141 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6564029070 con 0x55cb30f1f160 2024-05-09T02:21:16.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.030+0000 7f655a7fc700 1 -- 172.21.15.19:0/2523491141 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f656400bd20 con 0x55cb30f1f160 2024-05-09T02:21:16.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.030+0000 7f655a7fc700 1 -- 172.21.15.19:0/2523491141 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f656403e050 con 0x55cb30f1f160 2024-05-09T02:21:16.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.031+0000 7f65881e7b80 10 client.26276 fetch_fsmap learned FSMap version 55 2024-05-09T02:21:16.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.031+0000 7f65881e7b80 1 -- 172.21.15.19:0/2523491141 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55cb3103f1c0 con 0x55cb30f1f160 2024-05-09T02:21:16.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.032+0000 7f655a7fc700 1 -- 172.21.15.19:0/2523491141 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6564029070 con 0x55cb30f1f160 2024-05-09T02:21:16.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.032+0000 7f655a7fc700 1 -- 172.21.15.19:0/2523491141 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 55) ==== 8374+0+0 (secure 0 0 0) 0x7f6564020070 con 0x55cb30f1f160 2024-05-09T02:21:16.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.032+0000 7f65881e7b80 10 client.26276 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T02:21:16.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.032+0000 7f65881e7b80 10 client.26276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:21:16.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.032+0000 7f65881e7b80 20 client.26276 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:16.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.032+0000 7f65881e7b80 1 --2- 172.21.15.19:0/2523491141 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb3103f890 0x55cb31041cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:16.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.032+0000 7f65881e7b80 4 client.26276 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:21:16.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.032+0000 7f65881e7b80 1 -- 172.21.15.19:0/2523491141 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cb30edca70 con 0x55cb3103f890 2024-05-09T02:21:16.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.033+0000 7f65737fe700 1 --2- 172.21.15.19:0/2523491141 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb3103f890 0x55cb31041cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:16.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.034+0000 7f65737fe700 1 --2- 172.21.15.19:0/2523491141 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb3103f890 0x55cb31041cb0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17e94aefcac8d38e server_cookie=cb3d20a28761f76f in_seq=0 out_seq=0 2024-05-09T02:21:16.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.034+0000 7f655a7fc700 10 client.26276 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:16.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.035+0000 7f655a7fc700 1 -- 172.21.15.19:0/2523491141 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cb30edca70 con 0x55cb3103f890 2024-05-09T02:21:16.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.035+0000 7f655a7fc700 10 client.26276 handle_command_reply: tid=0 2024-05-09T02:21:16.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.035+0000 7f65881e7b80 1 client.26276 shutdown 2024-05-09T02:21:16.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.035+0000 7f65881e7b80 1 -- 172.21.15.19:0/2523491141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30f1f160 msgr2=0x55cb30fcf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:16.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.035+0000 7f65881e7b80 1 --2- 172.21.15.19:0/2523491141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30f1f160 0x55cb30fcf3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55cb30fc9be0 tx=0x7f6564004e80 comp rx=0 tx=0).stop 2024-05-09T02:21:16.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.035+0000 7f65881e7b80 1 -- 172.21.15.19:0/2523491141 shutdown_connections 2024-05-09T02:21:16.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.035+0000 7f65881e7b80 1 --2- 172.21.15.19:0/2523491141 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55cb3103f890 0x55cb31041cb0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.035+0000 7f65881e7b80 1 --2- 172.21.15.19:0/2523491141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb30fb26a0 0x55cb30fcf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.039 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:21:16.040 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T02:21:16.040 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T02:21:16.040 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:21:16.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.035+0000 7f65881e7b80 1 --2- 172.21.15.19:0/2523491141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cb30e354a0 0x55cb30fcee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.035+0000 7f655a7fc700 0 client.26276 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:16.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.035+0000 7f65881e7b80 1 --2- 172.21.15.19:0/2523491141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb30f1f160 0x55cb30fcf3c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.035+0000 7f65881e7b80 1 -- 172.21.15.19:0/2523491141 >> 172.21.15.19:0/2523491141 conn(0x55cb30e956a0 msgr2=0x55cb30fb6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:16.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.035+0000 7f65881e7b80 1 -- 172.21.15.19:0/2523491141 shutdown_connections 2024-05-09T02:21:16.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.035+0000 7f65881e7b80 1 -- 172.21.15.19:0/2523491141 wait complete. 2024-05-09T02:21:16.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.035+0000 7f65881e7b80 20 client.26276 trim_cache size 0 max 16384 2024-05-09T02:21:16.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.036+0000 7f65797d1700 1 -- 172.21.15.19:0/3385047663 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65600b2b20 msgr2=0x7f65600b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.036+0000 7f65797d1700 1 --2- 172.21.15.19:0/3385047663 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65600b2b20 0x7f65600b5030 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x55cb30faeed0 tx=0x7f655c0e56b0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.036+0000 7f65797d1700 1 -- 172.21.15.19:0/3385047663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6574103310 msgr2=0x7f657419b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.036+0000 7f65797d1700 1 --2- 172.21.15.19:0/3385047663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6574103310 0x7f657419b2a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f655c00b500 tx=0x7f655c00b810 comp rx=0 tx=0).stop 2024-05-09T02:21:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.036+0000 7f65797d1700 1 -- 172.21.15.19:0/3385047663 shutdown_connections 2024-05-09T02:21:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.036+0000 7f65797d1700 1 --2- 172.21.15.19:0/3385047663 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65600b2b20 0x7f65600b5030 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.036+0000 7f65797d1700 1 --2- 172.21.15.19:0/3385047663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657419f9c0 0x7f657419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.036+0000 7f65797d1700 1 --2- 172.21.15.19:0/3385047663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6574103310 0x7f657419b2a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.036+0000 7f65797d1700 1 --2- 172.21.15.19:0/3385047663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65741089f0 0x7f657419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.036+0000 7f65797d1700 1 -- 172.21.15.19:0/3385047663 >> 172.21.15.19:0/3385047663 conn(0x7f65740fe6f0 msgr2=0x7f6574105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.036+0000 7f65797d1700 1 -- 172.21.15.19:0/3385047663 shutdown_connections 2024-05-09T02:21:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:16.036+0000 7f65797d1700 1 -- 172.21.15.19:0/3385047663 wait complete. 2024-05-09T02:21:16.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:16 smithi019 ceph-mon[30151]: pgmap v310: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-05-09T02:21:16.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:16 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:21:16.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:16 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:21:16.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:16 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:21:16.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:16 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:21:16.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:16 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:21:16.697 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T02:21:16.698 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:21:16.799 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:16 smithi196 ceph-mon[34718]: pgmap v310: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-05-09T02:21:16.799 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:16 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:21:16.799 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:16 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:21:16.799 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:16 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:21:16.799 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:16 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:21:16.799 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:16 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:21:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:16 smithi059 ceph-mon[34781]: pgmap v310: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-05-09T02:21:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:16 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:21:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:16 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:21:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:16 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:21:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:16 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:21:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:16 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:21:17.086 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:21:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 575.0 s, 1895.2 tps, lat 52.457 ms stddev 57.475 2024-05-09T02:21:18.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:18 smithi196 ceph-mon[34718]: pgmap v311: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 998 op/s 2024-05-09T02:21:18.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:18 smithi019 ceph-mon[30151]: pgmap v311: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 998 op/s 2024-05-09T02:21:18.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:18 smithi059 ceph-mon[34781]: pgmap v311: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 998 op/s 2024-05-09T02:21:18.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.976+0000 7f5a6ab88700 1 -- 172.21.15.19:0/3959057462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a6410b2e0 msgr2=0x7f5a64117810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:18.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.976+0000 7f5a6ab88700 1 --2- 172.21.15.19:0/3959057462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a6410b2e0 0x7f5a64117810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0098f0 tx=0x7f5a5c009c00 comp rx=0 tx=0).stop 2024-05-09T02:21:18.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.976+0000 7f5a6ab88700 1 -- 172.21.15.19:0/3959057462 shutdown_connections 2024-05-09T02:21:18.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.976+0000 7f5a6ab88700 1 --2- 172.21.15.19:0/3959057462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a6410b2e0 0x7f5a64117810 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:18.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.976+0000 7f5a6ab88700 1 --2- 172.21.15.19:0/3959057462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a64110970 0x7f5a64110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:18.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.976+0000 7f5a6ab88700 1 --2- 172.21.15.19:0/3959057462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6410a990 0x7f5a6410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:18.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.976+0000 7f5a6ab88700 1 -- 172.21.15.19:0/3959057462 >> 172.21.15.19:0/3959057462 conn(0x7f5a64077c80 msgr2=0x7f5a640780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:18.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.976+0000 7f5a6ab88700 1 -- 172.21.15.19:0/3959057462 shutdown_connections 2024-05-09T02:21:18.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.976+0000 7f5a6ab88700 1 -- 172.21.15.19:0/3959057462 wait complete. 2024-05-09T02:21:18.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.977+0000 7f5a6ab88700 1 Processor -- start 2024-05-09T02:21:18.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.977+0000 7f5a6ab88700 1 -- start start 2024-05-09T02:21:18.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.977+0000 7f5a6ab88700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6410a990 0x7f5a641a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:18.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.977+0000 7f5a6ab88700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6410b2e0 0x7f5a641a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:18.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.977+0000 7f5a6ab88700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a64110970 0x7f5a641a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:18.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.977+0000 7f5a6ab88700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6411aa90 con 0x7f5a6410b2e0 2024-05-09T02:21:18.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.977+0000 7f5a6ab88700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6411a910 con 0x7f5a6410a990 2024-05-09T02:21:18.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.977+0000 7f5a6ab88700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6411ac10 con 0x7f5a64110970 2024-05-09T02:21:18.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.978+0000 7f5a68924700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6410a990 0x7f5a641a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:18.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.978+0000 7f5a69125700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a64110970 0x7f5a641a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:18.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.978+0000 7f5a68924700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6410a990 0x7f5a641a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:32952/0 (socket says 172.21.15.19:32952) 2024-05-09T02:21:18.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.978+0000 7f5a69125700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a64110970 0x7f5a641a5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50058/0 (socket says 172.21.15.19:50058) 2024-05-09T02:21:18.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.978+0000 7f5a68924700 1 -- 172.21.15.19:0/1828586577 learned_addr learned my addr 172.21.15.19:0/1828586577 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:18.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.978+0000 7f5a5bfff700 1 --2- 172.21.15.19:0/1828586577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6410b2e0 0x7f5a641a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:18.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.978+0000 7f5a68924700 1 -- 172.21.15.19:0/1828586577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a64110970 msgr2=0x7f5a641a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:18.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.978+0000 7f5a68924700 1 --2- 172.21.15.19:0/1828586577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a64110970 0x7f5a641a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:18.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.978+0000 7f5a68924700 1 -- 172.21.15.19:0/1828586577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6410b2e0 msgr2=0x7f5a641a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:18.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.978+0000 7f5a68924700 1 --2- 172.21.15.19:0/1828586577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6410b2e0 0x7f5a641a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:18.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.978+0000 7f5a68924700 1 -- 172.21.15.19:0/1828586577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a641a6310 con 0x7f5a6410a990 2024-05-09T02:21:18.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.979+0000 7f5a68924700 1 --2- 172.21.15.19:0/1828586577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6410a990 0x7f5a641a1440 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400ec70 tx=0x7f5a5400ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:18.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.979+0000 7f5a5bfff700 1 --2- 172.21.15.19:0/1828586577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6410b2e0 0x7f5a641a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:21:18.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.980+0000 7f5a59ffb700 1 -- 172.21.15.19:0/1828586577 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5400cd40 con 0x7f5a6410a990 2024-05-09T02:21:18.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.980+0000 7f5a59ffb700 1 -- 172.21.15.19:0/1828586577 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a540044d0 con 0x7f5a6410a990 2024-05-09T02:21:18.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.980+0000 7f5a59ffb700 1 -- 172.21.15.19:0/1828586577 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54010690 con 0x7f5a6410a990 2024-05-09T02:21:18.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.980+0000 7f5a6ab88700 1 -- 172.21.15.19:0/1828586577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a641a6600 con 0x7f5a6410a990 2024-05-09T02:21:18.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.981+0000 7f5a6ab88700 1 -- 172.21.15.19:0/1828586577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a641adf20 con 0x7f5a6410a990 2024-05-09T02:21:18.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.982+0000 7f5a6ab88700 1 -- 172.21.15.19:0/1828586577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a640512c0 con 0x7f5a6410a990 2024-05-09T02:21:18.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.984+0000 7f5a59ffb700 1 -- 172.21.15.19:0/1828586577 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5a540037a0 con 0x7f5a6410a990 2024-05-09T02:21:18.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.985+0000 7f5a59ffb700 1 --2- 172.21.15.19:0/1828586577 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5a4c0b2a70 0x7f5a4c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:18.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.985+0000 7f5a5bfff700 1 --2- 172.21.15.19:0/1828586577 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5a4c0b2a70 0x7f5a4c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:18.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.985+0000 7f5a59ffb700 1 -- 172.21.15.19:0/1828586577 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f5a540e28e0 con 0x7f5a6410a990 2024-05-09T02:21:18.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.986+0000 7f5a5bfff700 1 --2- 172.21.15.19:0/1828586577 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5a4c0b2a70 0x7f5a4c0b4f80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5a600094f0 tx=0x7f5a60009040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:18.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:18.991+0000 7f5a59ffb700 1 -- 172.21.15.19:0/1828586577 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5a540aa290 con 0x7f5a6410a990 2024-05-09T02:21:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:19.429+0000 7f5a6ab88700 1 -- 172.21.15.19:0/1828586577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a641a27a0 con 0x7f5a6410a990 2024-05-09T02:21:19.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:19.430+0000 7f5a59ffb700 1 -- 172.21.15.19:0/1828586577 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11925 (secure 0 0 0) 0x7f5a540a9b90 con 0x7f5a6410a990 2024-05-09T02:21:19.432 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:21:19.433 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:21:10.734722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24484,24463,24473,24457,24455]},"id":1}]} 2024-05-09T02:21:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:19.432+0000 7f5a6ab88700 1 -- 172.21.15.19:0/1828586577 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5a4c0b2a70 msgr2=0x7f5a4c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:19.432+0000 7f5a6ab88700 1 --2- 172.21.15.19:0/1828586577 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5a4c0b2a70 0x7f5a4c0b4f80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5a600094f0 tx=0x7f5a60009040 comp rx=0 tx=0).stop 2024-05-09T02:21:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:19.432+0000 7f5a6ab88700 1 -- 172.21.15.19:0/1828586577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6410a990 msgr2=0x7f5a641a1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:19.432+0000 7f5a6ab88700 1 --2- 172.21.15.19:0/1828586577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6410a990 0x7f5a641a1440 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400ec70 tx=0x7f5a5400ef80 comp rx=0 tx=0).stop 2024-05-09T02:21:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:19.433+0000 7f5a6ab88700 1 -- 172.21.15.19:0/1828586577 shutdown_connections 2024-05-09T02:21:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:19.433+0000 7f5a6ab88700 1 --2- 172.21.15.19:0/1828586577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a64110970 0x7f5a641a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:19.433+0000 7f5a6ab88700 1 --2- 172.21.15.19:0/1828586577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6410a990 0x7f5a641a1440 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:19.433+0000 7f5a6ab88700 1 --2- 172.21.15.19:0/1828586577 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5a4c0b2a70 0x7f5a4c0b4f80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:19.433+0000 7f5a6ab88700 1 --2- 172.21.15.19:0/1828586577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6410b2e0 0x7f5a641a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:19.433+0000 7f5a6ab88700 1 -- 172.21.15.19:0/1828586577 >> 172.21.15.19:0/1828586577 conn(0x7f5a64077c80 msgr2=0x7f5a6407b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:19.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:19.435+0000 7f5a6ab88700 1 -- 172.21.15.19:0/1828586577 shutdown_connections 2024-05-09T02:21:19.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:19.435+0000 7f5a6ab88700 1 -- 172.21.15.19:0/1828586577 wait complete. 2024-05-09T02:21:19.439 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 55 2024-05-09T02:21:19.771 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:19 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1828586577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:21:19.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:19 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1828586577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:21:19.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:19 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1828586577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:21:20.170 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T02:21:20.541 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:21:20.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:20 smithi196 ceph-mon[34718]: pgmap v312: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.4 MiB/s wr, 573 op/s 2024-05-09T02:21:20.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:20 smithi019 ceph-mon[30151]: pgmap v312: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.4 MiB/s wr, 573 op/s 2024-05-09T02:21:20.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:20 smithi059 ceph-mon[34781]: pgmap v312: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.4 MiB/s wr, 573 op/s 2024-05-09T02:21:21.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:21 smithi019 ceph-mon[30151]: pgmap v313: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.4 MiB/s wr, 657 op/s 2024-05-09T02:21:21.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:21 smithi059 ceph-mon[34781]: pgmap v313: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.4 MiB/s wr, 657 op/s 2024-05-09T02:21:22.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:21 smithi196 ceph-mon[34718]: pgmap v313: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.4 MiB/s wr, 657 op/s 2024-05-09T02:21:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.384+0000 7f2c8b79e700 1 -- 172.21.15.19:0/150568343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8411d930 msgr2=0x7f2c8411fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.384+0000 7f2c8b79e700 1 --2- 172.21.15.19:0/150568343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8411d930 0x7f2c8411fd50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00b0a0 tx=0x7f2c7c00b3b0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.384+0000 7f2c8b79e700 1 -- 172.21.15.19:0/150568343 shutdown_connections 2024-05-09T02:21:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.384+0000 7f2c8b79e700 1 --2- 172.21.15.19:0/150568343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8411d930 0x7f2c8411fd50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.384+0000 7f2c8b79e700 1 --2- 172.21.15.19:0/150568343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c8410a930 0x7f2c8410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.384+0000 7f2c8b79e700 1 --2- 172.21.15.19:0/150568343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c8410b280 0x7f2c8411d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.384+0000 7f2c8b79e700 1 -- 172.21.15.19:0/150568343 >> 172.21.15.19:0/150568343 conn(0x7f2c8406f4d0 msgr2=0x7f2c8406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.384+0000 7f2c8b79e700 1 -- 172.21.15.19:0/150568343 shutdown_connections 2024-05-09T02:21:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.384+0000 7f2c8b79e700 1 -- 172.21.15.19:0/150568343 wait complete. 2024-05-09T02:21:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.385+0000 7f2c8b79e700 1 Processor -- start 2024-05-09T02:21:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.385+0000 7f2c8b79e700 1 -- start start 2024-05-09T02:21:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.385+0000 7f2c8b79e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c8410a930 0x7f2c841af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c8b79e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8410b280 0x7f2c841afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c8b79e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c8411d930 0x7f2c841a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c8b79e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84123270 con 0x7f2c8410b280 2024-05-09T02:21:22.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c8b79e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841230f0 con 0x7f2c8411d930 2024-05-09T02:21:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c8b79e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841233f0 con 0x7f2c8410a930 2024-05-09T02:21:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c88d39700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8410b280 0x7f2c841afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c88d39700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8410b280 0x7f2c841afcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34184/0 (socket says 172.21.15.19:34184) 2024-05-09T02:21:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c88d39700 1 -- 172.21.15.19:0/1747357263 learned_addr learned my addr 172.21.15.19:0/1747357263 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c89d3b700 1 --2- 172.21.15.19:0/1747357263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c8411d930 0x7f2c841a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c8953a700 1 --2- 172.21.15.19:0/1747357263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c8410a930 0x7f2c841af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c89d3b700 1 -- 172.21.15.19:0/1747357263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c8410a930 msgr2=0x7f2c841af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c89d3b700 1 --2- 172.21.15.19:0/1747357263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c8410a930 0x7f2c841af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c89d3b700 1 -- 172.21.15.19:0/1747357263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8410b280 msgr2=0x7f2c841afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c89d3b700 1 --2- 172.21.15.19:0/1747357263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8410b280 0x7f2c841afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c89d3b700 1 -- 172.21.15.19:0/1747357263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c841aa080 con 0x7f2c8411d930 2024-05-09T02:21:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.386+0000 7f2c88d39700 1 --2- 172.21.15.19:0/1747357263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8410b280 0x7f2c841afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:21:22.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.387+0000 7f2c89d3b700 1 --2- 172.21.15.19:0/1747357263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c8411d930 0x7f2c841a99d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00b070 tx=0x7f2c7c004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.387+0000 7f2c7a7fc700 1 -- 172.21.15.19:0/1747357263 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c021070 con 0x7f2c8411d930 2024-05-09T02:21:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.387+0000 7f2c7a7fc700 1 -- 172.21.15.19:0/1747357263 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7c007e30 con 0x7f2c8411d930 2024-05-09T02:21:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.387+0000 7f2c8b79e700 1 -- 172.21.15.19:0/1747357263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c841aa310 con 0x7f2c8411d930 2024-05-09T02:21:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.387+0000 7f2c8b79e700 1 -- 172.21.15.19:0/1747357263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c841b6640 con 0x7f2c8411d930 2024-05-09T02:21:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.387+0000 7f2c7a7fc700 1 -- 172.21.15.19:0/1747357263 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c004590 con 0x7f2c8411d930 2024-05-09T02:21:22.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.390+0000 7f2c7a7fc700 1 -- 172.21.15.19:0/1747357263 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2c7c004730 con 0x7f2c8411d930 2024-05-09T02:21:22.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.391+0000 7f2c7a7fc700 1 --2- 172.21.15.19:0/1747357263 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2c700b2a70 0x7f2c700b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.391+0000 7f2c7a7fc700 1 -- 172.21.15.19:0/1747357263 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f2c7c00f020 con 0x7f2c8411d930 2024-05-09T02:21:22.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.391+0000 7f2c8953a700 1 --2- 172.21.15.19:0/1747357263 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2c700b2a70 0x7f2c700b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.392+0000 7f2c8953a700 1 --2- 172.21.15.19:0/1747357263 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2c700b2a70 0x7f2c700b4f80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2c841aab10 tx=0x7f2c74009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:22.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.392+0000 7f2c9a1b4b80 1 Processor -- start 2024-05-09T02:21:22.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.393+0000 7f2c9a1b4b80 1 -- start start 2024-05-09T02:21:22.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.393+0000 7f2c9a1b4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12d3ad00 0x558c12e43240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.393+0000 7f2c9a1b4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 0x558c12e43780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.393+0000 7f2c9a1b4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12e3d400 0x558c12e3d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.393+0000 7f2c9a1b4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558c12d600a0 con 0x558c12cc04a0 2024-05-09T02:21:22.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.393+0000 7f2c9a1b4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558c12d059a0 con 0x558c12d3ad00 2024-05-09T02:21:22.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.393+0000 7f2c9a1b4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558c12d5c060 con 0x558c12e3d400 2024-05-09T02:21:22.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.393+0000 7f2c8953a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12d3ad00 0x558c12e43240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.393+0000 7f2c8953a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12d3ad00 0x558c12e43240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:32988/0 (socket says 172.21.15.19:32988) 2024-05-09T02:21:22.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.393+0000 7f2c8953a700 1 -- 172.21.15.19:0/4258166004 learned_addr learned my addr 172.21.15.19:0/4258166004 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.393+0000 7f2c89d3b700 1 --2- 172.21.15.19:0/4258166004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12e3d400 0x558c12e3d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.394+0000 7f2c8953a700 1 -- 172.21.15.19:0/4258166004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12e3d400 msgr2=0x558c12e3d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.394+0000 7f2c8953a700 1 --2- 172.21.15.19:0/4258166004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12e3d400 0x558c12e3d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.394+0000 7f2c8953a700 1 -- 172.21.15.19:0/4258166004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 msgr2=0x558c12e43780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.394+0000 7f2c8953a700 1 --2- 172.21.15.19:0/4258166004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 0x558c12e43780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.394+0000 7f2c8953a700 1 -- 172.21.15.19:0/4258166004 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c12cf6500 con 0x558c12d3ad00 2024-05-09T02:21:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.394+0000 7f2c8953a700 1 --2- 172.21.15.19:0/4258166004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12d3ad00 0x558c12e43240 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2c74011d70 tx=0x7f2c74014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5cd916ffedbc62c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.394+0000 7f2c6f7fe700 1 -- 172.21.15.19:0/4258166004 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7400eb20 con 0x558c12d3ad00 2024-05-09T02:21:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.395+0000 7f2c6f7fe700 1 -- 172.21.15.19:0/4258166004 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c740044d0 con 0x558c12d3ad00 2024-05-09T02:21:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.395+0000 7f2c6f7fe700 1 -- 172.21.15.19:0/4258166004 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74003e30 con 0x558c12d3ad00 2024-05-09T02:21:22.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.396+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/4258166004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12d3ad00 msgr2=0x558c12e43240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:22.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.396+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/4258166004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12d3ad00 0x558c12e43240 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2c74011d70 tx=0x7f2c74014000 comp rx=0 tx=0).stop 2024-05-09T02:21:22.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.396+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/4258166004 shutdown_connections 2024-05-09T02:21:22.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.396+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/4258166004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12e3d400 0x558c12e3d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.396+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/4258166004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12d3ad00 0x558c12e43240 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.396+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/4258166004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 0x558c12e43780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.396+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/4258166004 >> 172.21.15.19:0/4258166004 conn(0x558c12cc3f70 msgr2=0x558c12e38220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.396+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/4258166004 shutdown_connections 2024-05-09T02:21:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.396+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/4258166004 wait complete. 2024-05-09T02:21:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.397+0000 7f2c9a1b4b80 1 -- start start 2024-05-09T02:21:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.397+0000 7f2c9a1b4b80 1 Processor -- start 2024-05-09T02:21:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c9a1b4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 0x558c12e47110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c9a1b4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12d3ad00 0x558c12e47650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c9a1b4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12e3d400 0x558c12e47b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c9a1b4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558c12e3e160 con 0x558c12cc04a0 2024-05-09T02:21:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c9a1b4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558c12cfab20 con 0x558c12e3d400 2024-05-09T02:21:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c9a1b4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558c12e4a020 con 0x558c12d3ad00 2024-05-09T02:21:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c9a1b4b80 10 client.0 _ll_register_callbacks cb 0x558c12bd16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c8953a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 0x558c12e47110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c89d3b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12e3d400 0x558c12e47b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c8953a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 0x558c12e47110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34194/0 (socket says 172.21.15.19:34194) 2024-05-09T02:21:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c8953a700 1 -- 172.21.15.19:0/216248601 learned_addr learned my addr 172.21.15.19:0/216248601 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c88d39700 1 --2- 172.21.15.19:0/216248601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12d3ad00 0x558c12e47650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c89d3b700 1 -- 172.21.15.19:0/216248601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12d3ad00 msgr2=0x558c12e47650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c89d3b700 1 --2- 172.21.15.19:0/216248601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12d3ad00 0x558c12e47650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c89d3b700 1 -- 172.21.15.19:0/216248601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 msgr2=0x558c12e47110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c89d3b700 1 --2- 172.21.15.19:0/216248601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 0x558c12e47110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c89d3b700 1 -- 172.21.15.19:0/216248601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c12bdb490 con 0x558c12e3d400 2024-05-09T02:21:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.398+0000 7f2c88d39700 1 --2- 172.21.15.19:0/216248601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12d3ad00 0x558c12e47650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:21:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.399+0000 7f2c89d3b700 1 --2- 172.21.15.19:0/216248601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12e3d400 0x558c12e47b90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c0e10b0 tx=0x7f2c7c0a8020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e29a39aeec1cb283 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.399+0000 7f2c6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:21:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.399+0000 7f2c6cff9700 1 -- 172.21.15.19:0/216248601 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c0b1060 con 0x558c12e3d400 2024-05-09T02:21:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.399+0000 7f2c6cff9700 1 -- 172.21.15.19:0/216248601 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f2c7c0b3320 con 0x558c12e3d400 2024-05-09T02:21:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.399+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/216248601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c740044d0 con 0x558c12e3d400 2024-05-09T02:21:22.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.399+0000 7f2c6cff9700 1 -- 172.21.15.19:0/216248601 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2c7c0ab620 con 0x558c12e3d400 2024-05-09T02:21:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.399+0000 7f2c9a1b4b80 10 client.26294 fetch_fsmap learned FSMap version 55 2024-05-09T02:21:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.399+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/216248601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558c12e548d0 con 0x558c12e3d400 2024-05-09T02:21:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.399+0000 7f2c6cff9700 1 -- 172.21.15.19:0/216248601 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c740044d0 con 0x558c12e3d400 2024-05-09T02:21:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.399+0000 7f2c6cff9700 1 -- 172.21.15.19:0/216248601 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 55) ==== 8374+0+0 (secure 0 0 0) 0x7f2c7c011070 con 0x558c12e3d400 2024-05-09T02:21:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.399+0000 7f2c9a1b4b80 10 client.26294 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T02:21:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.400+0000 7f2c9a1b4b80 10 client.26294 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:21:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.400+0000 7f2c9a1b4b80 20 client.26294 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.400+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/216248601 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x558c12e56f30 0x558c12e59380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.400+0000 7f2c9a1b4b80 4 client.26294 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:21:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.400+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/216248601 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c12d6da80 con 0x558c12e56f30 2024-05-09T02:21:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.400+0000 7f2c8953a700 1 --2- 172.21.15.19:0/216248601 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x558c12e56f30 0x558c12e59380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.400+0000 7f2c6cff9700 10 client.26294 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:21:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.400+0000 7f2c8953a700 1 --2- 172.21.15.19:0/216248601 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x558c12e56f30 0x558c12e59380 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ba05b0472edc572c server_cookie=2c80a256b041e173 in_seq=0 out_seq=0 2024-05-09T02:21:22.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.401+0000 7f2c6cff9700 1 -- 172.21.15.19:0/216248601 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558c12d6da80 con 0x558c12e56f30 2024-05-09T02:21:22.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.401+0000 7f2c6cff9700 10 client.26294 handle_command_reply: tid=0 2024-05-09T02:21:22.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.401+0000 7f2c9a1b4b80 1 client.26294 shutdown 2024-05-09T02:21:22.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.401+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/216248601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12e3d400 msgr2=0x558c12e47b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:22.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.401+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/216248601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12e3d400 0x558c12e47b90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c0e10b0 tx=0x7f2c7c0a8020 comp rx=0 tx=0).stop 2024-05-09T02:21:22.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.401+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/216248601 shutdown_connections 2024-05-09T02:21:22.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.401+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/216248601 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x558c12e56f30 0x558c12e59380 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.401+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/216248601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12e3d400 0x558c12e47b90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.401+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/216248601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 0x558c12e47110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.401+0000 7f2c6cff9700 0 client.26294 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:21:22.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.401+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/216248601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12d3ad00 0x558c12e47650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.401+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/216248601 >> 172.21.15.19:0/216248601 conn(0x558c12cc3f70 msgr2=0x558c12e31ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:22.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.402+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/216248601 shutdown_connections 2024-05-09T02:21:22.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.402+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/216248601 wait complete. 2024-05-09T02:21:22.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.402+0000 7f2c9a1b4b80 20 client.26294 trim_cache size 0 max 16384 2024-05-09T02:21:22.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.430+0000 7f2c9a1b4b80 1 Processor -- start 2024-05-09T02:21:22.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.431+0000 7f2c9a1b4b80 1 -- start start 2024-05-09T02:21:22.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.431+0000 7f2c9a1b4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12cc3f70 0x558c12e32a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.431+0000 7f2c9a1b4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12cc04a0 0x558c12e44ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.431+0000 7f2c9a1b4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12d3ad00 0x558c12e477e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.431+0000 7f2c9a1b4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841bdd20 con 0x558c12d3ad00 2024-05-09T02:21:22.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.431+0000 7f2c9a1b4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841bf950 con 0x558c12cc04a0 2024-05-09T02:21:22.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.431+0000 7f2c9a1b4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841be3f0 con 0x558c12cc3f70 2024-05-09T02:21:22.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.431+0000 7f2c8953a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12cc3f70 0x558c12e32a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.431+0000 7f2c8953a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12cc3f70 0x558c12e32a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50102/0 (socket says 172.21.15.19:50102) 2024-05-09T02:21:22.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.431+0000 7f2c8953a700 1 -- 172.21.15.19:0/2313322784 learned_addr learned my addr 172.21.15.19:0/2313322784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:22.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.432+0000 7f2c88d39700 1 --2- 172.21.15.19:0/2313322784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12cc04a0 0x558c12e44ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.432+0000 7f2c8953a700 1 -- 172.21.15.19:0/2313322784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12cc04a0 msgr2=0x558c12e44ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:22.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.432+0000 7f2c8953a700 1 --2- 172.21.15.19:0/2313322784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12cc04a0 0x558c12e44ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.432+0000 7f2c8953a700 1 -- 172.21.15.19:0/2313322784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12d3ad00 msgr2=0x558c12e477e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:21:22.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.432+0000 7f2c8953a700 1 --2- 172.21.15.19:0/2313322784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12d3ad00 0x558c12e477e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.432+0000 7f2c8953a700 1 -- 172.21.15.19:0/2313322784 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c12e46ee0 con 0x558c12cc3f70 2024-05-09T02:21:22.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.432+0000 7f2c8953a700 1 --2- 172.21.15.19:0/2313322784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12cc3f70 0x558c12e32a20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c74012d80 tx=0x7f2c74009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93a9c18b0d1e7630 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:22.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.433+0000 7f2c6effd700 1 -- 172.21.15.19:0/2313322784 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74037070 con 0x558c12cc3f70 2024-05-09T02:21:22.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.433+0000 7f2c6effd700 1 -- 172.21.15.19:0/2313322784 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c74005040 con 0x558c12cc3f70 2024-05-09T02:21:22.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.434+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/2313322784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12cc3f70 msgr2=0x558c12e32a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:22.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.434+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/2313322784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12cc3f70 0x558c12e32a20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c74012d80 tx=0x7f2c74009800 comp rx=0 tx=0).stop 2024-05-09T02:21:22.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.437+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/2313322784 shutdown_connections 2024-05-09T02:21:22.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.437+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/2313322784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12d3ad00 0x558c12e477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.437+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/2313322784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12cc3f70 0x558c12e32a20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.437+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/2313322784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12cc04a0 0x558c12e44ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.437+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/2313322784 >> 172.21.15.19:0/2313322784 conn(0x558c12e38520 msgr2=0x558c12e470d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:22.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.437+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/2313322784 shutdown_connections 2024-05-09T02:21:22.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.437+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/2313322784 wait complete. 2024-05-09T02:21:22.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.438+0000 7f2c9a1b4b80 1 -- start start 2024-05-09T02:21:22.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.438+0000 7f2c9a1b4b80 1 Processor -- start 2024-05-09T02:21:22.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.439+0000 7f2c9a1b4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 0x558c12ebcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.439+0000 7f2c9a1b4b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12cc3f70 0x558c12e4f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.440+0000 7f2c9a1b4b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12d3ad00 0x558c12e4f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.440+0000 7f2c9a1b4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841bf250 con 0x558c12cc04a0 2024-05-09T02:21:22.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.440+0000 7f2c9a1b4b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841beac0 con 0x558c12d3ad00 2024-05-09T02:21:22.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.440+0000 7f2c9a1b4b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841bbce0 con 0x558c12cc3f70 2024-05-09T02:21:22.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.440+0000 7f2c9a1b4b80 10 client.0 _ll_register_callbacks cb 0x558c12bd16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:22.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.440+0000 7f2c89d3b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12d3ad00 0x558c12e4f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.440+0000 7f2c8953a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 0x558c12ebcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.440+0000 7f2c8953a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 0x558c12ebcf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34208/0 (socket says 172.21.15.19:34208) 2024-05-09T02:21:22.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.440+0000 7f2c8953a700 1 -- 172.21.15.19:0/3462109494 learned_addr learned my addr 172.21.15.19:0/3462109494 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:22.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.440+0000 7f2c88d39700 1 --2- 172.21.15.19:0/3462109494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12cc3f70 0x558c12e4f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.441+0000 7f2c88d39700 1 -- 172.21.15.19:0/3462109494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12d3ad00 msgr2=0x558c12e4f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:22.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.441+0000 7f2c88d39700 1 --2- 172.21.15.19:0/3462109494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12d3ad00 0x558c12e4f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.441+0000 7f2c88d39700 1 -- 172.21.15.19:0/3462109494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 msgr2=0x558c12ebcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:22.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.441+0000 7f2c88d39700 1 --2- 172.21.15.19:0/3462109494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 0x558c12ebcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.441+0000 7f2c88d39700 1 -- 172.21.15.19:0/3462109494 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c12e5e340 con 0x558c12cc3f70 2024-05-09T02:21:22.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.441+0000 7f2c88d39700 1 --2- 172.21.15.19:0/3462109494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12cc3f70 0x558c12e4f1e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2c80004790 tx=0x7f2c80003be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cad9fd742fbea46 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:22.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.441+0000 7f2c6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:21:22.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.442+0000 7f2c6cff9700 1 -- 172.21.15.19:0/3462109494 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80004830 con 0x558c12cc3f70 2024-05-09T02:21:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.442+0000 7f2c6cff9700 1 -- 172.21.15.19:0/3462109494 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f2c800049d0 con 0x558c12cc3f70 2024-05-09T02:21:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.443+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/3462109494 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c74005040 con 0x558c12cc3f70 2024-05-09T02:21:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.443+0000 7f2c6cff9700 1 -- 172.21.15.19:0/3462109494 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2c80004bd0 con 0x558c12cc3f70 2024-05-09T02:21:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.443+0000 7f2c9a1b4b80 10 client.26335 fetch_fsmap learned FSMap version 55 2024-05-09T02:21:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.443+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/3462109494 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558c12ed2940 con 0x558c12cc3f70 2024-05-09T02:21:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.446+0000 7f2c6cff9700 1 -- 172.21.15.19:0/3462109494 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c74005040 con 0x558c12cc3f70 2024-05-09T02:21:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.446+0000 7f2c6cff9700 1 -- 172.21.15.19:0/3462109494 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 55) ==== 8374+0+0 (secure 0 0 0) 0x7f2c80005380 con 0x558c12cc3f70 2024-05-09T02:21:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.446+0000 7f2c9a1b4b80 10 client.26335 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T02:21:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.446+0000 7f2c9a1b4b80 10 client.26335 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:21:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.446+0000 7f2c9a1b4b80 20 client.26335 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.446+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/3462109494 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x558c12e3d400 0x558c12ed50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.446+0000 7f2c9a1b4b80 4 client.26335 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:21:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.446+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/3462109494 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c12d67b20 con 0x558c12e3d400 2024-05-09T02:21:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.448+0000 7f2c8953a700 1 --2- 172.21.15.19:0/3462109494 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x558c12e3d400 0x558c12ed50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.449+0000 7f2c8953a700 1 --2- 172.21.15.19:0/3462109494 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x558c12e3d400 0x558c12ed50e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3561850dd5475a1a server_cookie=35d41c3799c1490e in_seq=0 out_seq=0 2024-05-09T02:21:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.449+0000 7f2c6cff9700 10 client.26335 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:21:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.453+0000 7f2c6cff9700 1 -- 172.21.15.19:0/3462109494 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c12d67b20 con 0x558c12e3d400 2024-05-09T02:21:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.453+0000 7f2c6cff9700 10 client.26335 handle_command_reply: tid=0 2024-05-09T02:21:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.453+0000 7f2c9a1b4b80 1 client.26335 shutdown 2024-05-09T02:21:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.458+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/3462109494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12cc3f70 msgr2=0x558c12e4f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.458+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/3462109494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12cc3f70 0x558c12e4f1e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2c80004790 tx=0x7f2c80003be0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.460+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/3462109494 shutdown_connections 2024-05-09T02:21:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.460+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/3462109494 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x558c12e3d400 0x558c12ed50e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.460+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/3462109494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558c12d3ad00 0x558c12e4f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.460+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/3462109494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c12cc04a0 0x558c12ebcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.460+0000 7f2c9a1b4b80 1 --2- 172.21.15.19:0/3462109494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c12cc3f70 0x558c12e4f1e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.460+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/3462109494 >> 172.21.15.19:0/3462109494 conn(0x558c12e38520 msgr2=0x558c12e32780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.460+0000 7f2c6cff9700 0 client.26335 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:21:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.461+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/3462109494 shutdown_connections 2024-05-09T02:21:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.461+0000 7f2c9a1b4b80 1 -- 172.21.15.19:0/3462109494 wait complete. 2024-05-09T02:21:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.462+0000 7f2c9a1b4b80 20 client.26335 trim_cache size 0 max 16384 2024-05-09T02:21:22.466 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:21:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.467+0000 7f2c8b79e700 1 -- 172.21.15.19:0/1747357263 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2c700b2a70 msgr2=0x7f2c700b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.467+0000 7f2c8b79e700 1 --2- 172.21.15.19:0/1747357263 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2c700b2a70 0x7f2c700b4f80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2c841aab10 tx=0x7f2c74009260 comp rx=0 tx=0).stop 2024-05-09T02:21:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.467+0000 7f2c8b79e700 1 -- 172.21.15.19:0/1747357263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c8411d930 msgr2=0x7f2c841a99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.467+0000 7f2c8b79e700 1 --2- 172.21.15.19:0/1747357263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c8411d930 0x7f2c841a99d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00b070 tx=0x7f2c7c004d00 comp rx=0 tx=0).stop 2024-05-09T02:21:22.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.469+0000 7f2c8b79e700 1 -- 172.21.15.19:0/1747357263 shutdown_connections 2024-05-09T02:21:22.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.469+0000 7f2c8b79e700 1 --2- 172.21.15.19:0/1747357263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c8411d930 0x7f2c841a99d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.469+0000 7f2c8b79e700 1 --2- 172.21.15.19:0/1747357263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c8410a930 0x7f2c841af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.469+0000 7f2c8b79e700 1 --2- 172.21.15.19:0/1747357263 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2c700b2a70 0x7f2c700b4f80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.470+0000 7f2c8b79e700 1 --2- 172.21.15.19:0/1747357263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8410b280 0x7f2c841afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:22.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.470+0000 7f2c8b79e700 1 -- 172.21.15.19:0/1747357263 >> 172.21.15.19:0/1747357263 conn(0x7f2c8406f4d0 msgr2=0x7f2c8411b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:22.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.471+0000 7f2c8b79e700 1 -- 172.21.15.19:0/1747357263 shutdown_connections 2024-05-09T02:21:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:22.475+0000 7f2c8b79e700 1 -- 172.21.15.19:0/1747357263 wait complete. 2024-05-09T02:21:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 580.0 s, 1438.4 tps, lat 69.348 ms stddev 78.892 2024-05-09T02:21:23.153 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T02:21:23.535 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:21:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:24 smithi059 ceph-mon[34781]: pgmap v314: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 602 op/s 2024-05-09T02:21:24.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:24 smithi196 ceph-mon[34718]: pgmap v314: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 602 op/s 2024-05-09T02:21:24.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:24 smithi019 ceph-mon[30151]: pgmap v314: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 602 op/s 2024-05-09T02:21:25.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.940+0000 7fa13040f700 1 -- 172.21.15.19:0/4004438769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa128108740 msgr2=0x7fa1281100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:25.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.940+0000 7fa13040f700 1 --2- 172.21.15.19:0/4004438769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa128108740 0x7fa1281100a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa1180099c0 tx=0x7fa118009cd0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.940+0000 7fa13040f700 1 -- 172.21.15.19:0/4004438769 shutdown_connections 2024-05-09T02:21:25.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.940+0000 7fa13040f700 1 --2- 172.21.15.19:0/4004438769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa128108740 0x7fa1281100a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.940+0000 7fa13040f700 1 --2- 172.21.15.19:0/4004438769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1281034c0 0x7fa1281038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.940+0000 7fa13040f700 1 --2- 172.21.15.19:0/4004438769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa128107df0 0x7fa128108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.940+0000 7fa13040f700 1 -- 172.21.15.19:0/4004438769 >> 172.21.15.19:0/4004438769 conn(0x7fa1280fedc0 msgr2=0x7fa1281011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:25.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.940+0000 7fa13040f700 1 -- 172.21.15.19:0/4004438769 shutdown_connections 2024-05-09T02:21:25.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.940+0000 7fa13040f700 1 -- 172.21.15.19:0/4004438769 wait complete. 2024-05-09T02:21:25.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.941+0000 7fa13040f700 1 Processor -- start 2024-05-09T02:21:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.941+0000 7fa13040f700 1 -- start start 2024-05-09T02:21:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.941+0000 7fa13040f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1281034c0 0x7fa12819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.941+0000 7fa13040f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa128107df0 0x7fa12819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.941+0000 7fa13040f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa128108740 0x7fa12819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.941+0000 7fa13040f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa128113370 con 0x7fa128108740 2024-05-09T02:21:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.941+0000 7fa13040f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281131f0 con 0x7fa1281034c0 2024-05-09T02:21:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.941+0000 7fa13040f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281134f0 con 0x7fa128107df0 2024-05-09T02:21:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.942+0000 7fa12e1ab700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1281034c0 0x7fa12819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.942+0000 7fa12d9aa700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa128107df0 0x7fa12819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.942+0000 7fa12e1ab700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1281034c0 0x7fa12819bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:33022/0 (socket says 172.21.15.19:33022) 2024-05-09T02:21:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.942+0000 7fa12e1ab700 1 -- 172.21.15.19:0/2658812278 learned_addr learned my addr 172.21.15.19:0/2658812278 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.942+0000 7fa12e9ac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa128108740 0x7fa12819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.943+0000 7fa12d9aa700 1 -- 172.21.15.19:0/2658812278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1281034c0 msgr2=0x7fa12819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.943+0000 7fa12d9aa700 1 --2- 172.21.15.19:0/2658812278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1281034c0 0x7fa12819bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.943+0000 7fa12d9aa700 1 -- 172.21.15.19:0/2658812278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa128108740 msgr2=0x7fa12819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.943+0000 7fa12d9aa700 1 --2- 172.21.15.19:0/2658812278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa128108740 0x7fa12819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.943+0000 7fa12d9aa700 1 -- 172.21.15.19:0/2658812278 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1281a6f80 con 0x7fa128107df0 2024-05-09T02:21:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.943+0000 7fa12d9aa700 1 --2- 172.21.15.19:0/2658812278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa128107df0 0x7fa12819c150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa12000eee0 tx=0x7fa12000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.944+0000 7fa11f7fe700 1 -- 172.21.15.19:0/2658812278 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12000c9c0 con 0x7fa128107df0 2024-05-09T02:21:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.944+0000 7fa13040f700 1 -- 172.21.15.19:0/2658812278 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa128199a00 con 0x7fa128107df0 2024-05-09T02:21:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.944+0000 7fa11f7fe700 1 -- 172.21.15.19:0/2658812278 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa12000cb60 con 0x7fa128107df0 2024-05-09T02:21:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.944+0000 7fa11f7fe700 1 -- 172.21.15.19:0/2658812278 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120007740 con 0x7fa128107df0 2024-05-09T02:21:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.944+0000 7fa13040f700 1 -- 172.21.15.19:0/2658812278 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa128199f90 con 0x7fa128107df0 2024-05-09T02:21:25.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.948+0000 7fa11f7fe700 1 -- 172.21.15.19:0/2658812278 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa1200079a0 con 0x7fa128107df0 2024-05-09T02:21:25.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.948+0000 7fa13ee25b80 1 Processor -- start 2024-05-09T02:21:25.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.949+0000 7fa13ee25b80 1 -- start start 2024-05-09T02:21:25.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.949+0000 7fa11f7fe700 1 --2- 172.21.15.19:0/2658812278 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa1140b2a70 0x7fa1140b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:25.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.949+0000 7fa13ee25b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0ace160 0x5634a0b674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:25.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.949+0000 7fa13ee25b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a09e44a0 0x5634a0b67a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:25.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.949+0000 7fa13ee25b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b616a0 0x5634a0b61ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:25.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.949+0000 7fa13ee25b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5634a0a83ff0 con 0x5634a0b616a0 2024-05-09T02:21:25.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.949+0000 7fa13ee25b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5634a0a227f0 con 0x5634a09e44a0 2024-05-09T02:21:25.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.949+0000 7fa13ee25b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5634a0a7ffb0 con 0x5634a0ace160 2024-05-09T02:21:25.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.949+0000 7fa12e9ac700 1 --2- 172.21.15.19:0/2658812278 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa1140b2a70 0x7fa1140b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:25.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.949+0000 7fa12e1ab700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0ace160 0x5634a0b674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:25.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.949+0000 7fa12e1ab700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0ace160 0x5634a0b674f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50140/0 (socket says 172.21.15.19:50140) 2024-05-09T02:21:25.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.949+0000 7fa12e1ab700 1 -- 172.21.15.19:0/2473043969 learned_addr learned my addr 172.21.15.19:0/2473043969 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:25.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.949+0000 7fa12e9ac700 1 --2- 172.21.15.19:0/2473043969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b616a0 0x5634a0b61ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:25.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.949+0000 7fa12d9aa700 1 --2- 172.21.15.19:0/2473043969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a09e44a0 0x5634a0b67a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:25.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.949+0000 7fa11f7fe700 1 -- 172.21.15.19:0/2658812278 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fa1200e63d0 con 0x7fa128107df0 2024-05-09T02:21:25.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.950+0000 7fa12e1ab700 1 -- 172.21.15.19:0/2473043969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a09e44a0 msgr2=0x5634a0b67a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:25.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.950+0000 7fa12e1ab700 1 --2- 172.21.15.19:0/2473043969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a09e44a0 0x5634a0b67a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.950+0000 7fa12e1ab700 1 -- 172.21.15.19:0/2473043969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b616a0 msgr2=0x5634a0b61ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:25.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.950+0000 7fa12e1ab700 1 --2- 172.21.15.19:0/2473043969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b616a0 0x5634a0b61ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.950+0000 7fa12e1ab700 1 -- 172.21.15.19:0/2473043969 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a0a1a370 con 0x5634a0ace160 2024-05-09T02:21:25.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.950+0000 7fa12e9ac700 1 --2- 172.21.15.19:0/2658812278 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa1140b2a70 0x7fa1140b4f80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa118005950 tx=0x7fa1180050d0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:25.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.950+0000 7fa12e1ab700 1 --2- 172.21.15.19:0/2473043969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0ace160 0x5634a0b674f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa124009e50 tx=0x7fa124011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf66f1fd01993ba7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:25.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.951+0000 7fa11cd47700 1 -- 172.21.15.19:0/2473043969 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124004a30 con 0x5634a0ace160 2024-05-09T02:21:25.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.951+0000 7fa11cd47700 1 -- 172.21.15.19:0/2473043969 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1240054d0 con 0x5634a0ace160 2024-05-09T02:21:25.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.951+0000 7fa11cd47700 1 -- 172.21.15.19:0/2473043969 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124019970 con 0x5634a0ace160 2024-05-09T02:21:25.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.952+0000 7fa13ee25b80 1 -- 172.21.15.19:0/2473043969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0ace160 msgr2=0x5634a0b674f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:25.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.952+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/2473043969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0ace160 0x5634a0b674f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa124009e50 tx=0x7fa124011720 comp rx=0 tx=0).stop 2024-05-09T02:21:25.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.952+0000 7fa13ee25b80 1 -- 172.21.15.19:0/2473043969 shutdown_connections 2024-05-09T02:21:25.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.952+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/2473043969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b616a0 0x5634a0b61ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.952+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/2473043969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0ace160 0x5634a0b674f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.952+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/2473043969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a09e44a0 0x5634a0b67a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.952+0000 7fa13ee25b80 1 -- 172.21.15.19:0/2473043969 >> 172.21.15.19:0/2473043969 conn(0x5634a0a446a0 msgr2=0x5634a0b5c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:25.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.953+0000 7fa13ee25b80 1 -- 172.21.15.19:0/2473043969 shutdown_connections 2024-05-09T02:21:25.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.953+0000 7fa13ee25b80 1 -- 172.21.15.19:0/2473043969 wait complete. 2024-05-09T02:21:25.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.953+0000 7fa13ee25b80 1 -- start start 2024-05-09T02:21:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.953+0000 7fa13ee25b80 1 Processor -- start 2024-05-09T02:21:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa13ee25b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a09e44a0 0x5634a0b6b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa13ee25b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a0ace160 0x5634a0b6b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa13ee25b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b616a0 0x5634a0b6bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa13ee25b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5634a08ffba0 con 0x5634a0b616a0 2024-05-09T02:21:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa13ee25b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5634a0a3e150 con 0x5634a0ace160 2024-05-09T02:21:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa13ee25b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5634a0b6e1d0 con 0x5634a09e44a0 2024-05-09T02:21:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa13ee25b80 10 client.0 _ll_register_callbacks cb 0x5634a08f56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa12e1ab700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a09e44a0 0x5634a0b6b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa12e1ab700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a09e44a0 0x5634a0b6b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:50144/0 (socket says 172.21.15.19:50144) 2024-05-09T02:21:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa12e1ab700 1 -- 172.21.15.19:0/902489138 learned_addr learned my addr 172.21.15.19:0/902489138 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa12e9ac700 1 --2- 172.21.15.19:0/902489138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b616a0 0x5634a0b6bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa12d9aa700 1 --2- 172.21.15.19:0/902489138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a0ace160 0x5634a0b6b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa12e1ab700 1 -- 172.21.15.19:0/902489138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a0ace160 msgr2=0x5634a0b6b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa12e1ab700 1 --2- 172.21.15.19:0/902489138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a0ace160 0x5634a0b6b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa12e1ab700 1 -- 172.21.15.19:0/902489138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b616a0 msgr2=0x5634a0b6bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa12e1ab700 1 --2- 172.21.15.19:0/902489138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b616a0 0x5634a0b6bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa12e1ab700 1 -- 172.21.15.19:0/902489138 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634a08ff490 con 0x5634a09e44a0 2024-05-09T02:21:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.954+0000 7fa12d9aa700 1 --2- 172.21.15.19:0/902489138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a0ace160 0x5634a0b6b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:21:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.955+0000 7fa12e1ab700 1 --2- 172.21.15.19:0/902489138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a09e44a0 0x5634a0b6b1f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa124019510 tx=0x7fa124004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=997acdb0b4c37124 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.955+0000 7fa111ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:21:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.955+0000 7fa111ffb700 1 -- 172.21.15.19:0/902489138 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124027070 con 0x5634a09e44a0 2024-05-09T02:21:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.955+0000 7fa13ee25b80 1 -- 172.21.15.19:0/902489138 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1240054d0 con 0x5634a09e44a0 2024-05-09T02:21:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.955+0000 7fa111ffb700 1 -- 172.21.15.19:0/902489138 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fa124004b60 con 0x5634a09e44a0 2024-05-09T02:21:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.955+0000 7fa111ffb700 1 -- 172.21.15.19:0/902489138 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa124033050 con 0x5634a09e44a0 2024-05-09T02:21:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.955+0000 7fa13ee25b80 10 client.26347 fetch_fsmap learned FSMap version 55 2024-05-09T02:21:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.955+0000 7fa13ee25b80 1 -- 172.21.15.19:0/902489138 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5634a0b78bc0 con 0x5634a09e44a0 2024-05-09T02:21:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.956+0000 7fa111ffb700 1 -- 172.21.15.19:0/902489138 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1240054d0 con 0x5634a09e44a0 2024-05-09T02:21:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.956+0000 7fa111ffb700 1 -- 172.21.15.19:0/902489138 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 55) ==== 8374+0+0 (secure 0 0 0) 0x7fa124019570 con 0x5634a09e44a0 2024-05-09T02:21:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.956+0000 7fa13ee25b80 10 client.26347 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T02:21:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.956+0000 7fa13ee25b80 10 client.26347 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:21:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.956+0000 7fa13ee25b80 20 client.26347 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.956+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/902489138 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5634a0b7b150 0x5634a0b7d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.956+0000 7fa13ee25b80 4 client.26347 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:21:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.956+0000 7fa13ee25b80 1 -- 172.21.15.19:0/902489138 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634a0a919d0 con 0x5634a0b7b150 2024-05-09T02:21:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.956+0000 7fa12d9aa700 1 --2- 172.21.15.19:0/902489138 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5634a0b7b150 0x5634a0b7d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.957+0000 7fa12d9aa700 1 --2- 172.21.15.19:0/902489138 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5634a0b7b150 0x5634a0b7d570 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=93689e85a55e85d6 server_cookie=b34a782aef0805e in_seq=0 out_seq=0 2024-05-09T02:21:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.957+0000 7fa111ffb700 10 client.26347 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:21:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.957+0000 7fa111ffb700 1 -- 172.21.15.19:0/902489138 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5634a0a919d0 con 0x5634a0b7b150 2024-05-09T02:21:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.957+0000 7fa111ffb700 10 client.26347 handle_command_reply: tid=0 2024-05-09T02:21:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.958+0000 7fa13ee25b80 1 client.26347 shutdown 2024-05-09T02:21:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.958+0000 7fa13ee25b80 1 -- 172.21.15.19:0/902489138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a09e44a0 msgr2=0x5634a0b6b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.958+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/902489138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a09e44a0 0x5634a0b6b1f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa124019510 tx=0x7fa124004320 comp rx=0 tx=0).stop 2024-05-09T02:21:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.958+0000 7fa13ee25b80 1 -- 172.21.15.19:0/902489138 shutdown_connections 2024-05-09T02:21:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.958+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/902489138 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5634a0b7b150 0x5634a0b7d570 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.958+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/902489138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b616a0 0x5634a0b6bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.958+0000 7fa111ffb700 0 client.26347 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:21:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.958+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/902489138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a09e44a0 0x5634a0b6b1f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.958+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/902489138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a0ace160 0x5634a0b6b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.958+0000 7fa13ee25b80 1 -- 172.21.15.19:0/902489138 >> 172.21.15.19:0/902489138 conn(0x5634a0a446a0 msgr2=0x5634a0b5c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.958+0000 7fa13ee25b80 1 -- 172.21.15.19:0/902489138 shutdown_connections 2024-05-09T02:21:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.958+0000 7fa13ee25b80 1 -- 172.21.15.19:0/902489138 wait complete. 2024-05-09T02:21:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.958+0000 7fa13ee25b80 20 client.26347 trim_cache size 0 max 16384 2024-05-09T02:21:25.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.986+0000 7fa13ee25b80 1 Processor -- start 2024-05-09T02:21:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.987+0000 7fa13ee25b80 1 -- start start 2024-05-09T02:21:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.988+0000 7fa13ee25b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0ace160 0x5634a0b657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.988+0000 7fa13ee25b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a0b616a0 0x5634a0b6bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.988+0000 7fa13ee25b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b7b150 0x5634a0b6a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.988+0000 7fa13ee25b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281ace90 con 0x5634a0b7b150 2024-05-09T02:21:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.988+0000 7fa13ee25b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281aeac0 con 0x5634a0b616a0 2024-05-09T02:21:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.988+0000 7fa13ee25b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281ad560 con 0x5634a0ace160 2024-05-09T02:21:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.988+0000 7fa12e9ac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b7b150 0x5634a0b6a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.988+0000 7fa12e9ac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b7b150 0x5634a0b6a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34260/0 (socket says 172.21.15.19:34260) 2024-05-09T02:21:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.988+0000 7fa12e9ac700 1 -- 172.21.15.19:0/301512049 learned_addr learned my addr 172.21.15.19:0/301512049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.989+0000 7fa12e9ac700 1 -- 172.21.15.19:0/301512049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0ace160 msgr2=0x5634a0b657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.989+0000 7fa12e9ac700 1 --2- 172.21.15.19:0/301512049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0ace160 0x5634a0b657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.989+0000 7fa12e9ac700 1 -- 172.21.15.19:0/301512049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a0b616a0 msgr2=0x5634a0b6bae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:21:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.989+0000 7fa12e9ac700 1 --2- 172.21.15.19:0/301512049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a0b616a0 0x5634a0b6bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.989+0000 7fa12e9ac700 1 -- 172.21.15.19:0/301512049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a0b55f60 con 0x5634a0b7b150 2024-05-09T02:21:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.989+0000 7fa12e9ac700 1 --2- 172.21.15.19:0/301512049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b7b150 0x5634a0b6a1f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa1180099c0 tx=0x7fa118005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb579c397c47627a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.990+0000 7fa113fff700 1 -- 172.21.15.19:0/301512049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118047070 con 0x5634a0b7b150 2024-05-09T02:21:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.990+0000 7fa113fff700 1 -- 172.21.15.19:0/301512049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1180095b0 con 0x5634a0b7b150 2024-05-09T02:21:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.990+0000 7fa113fff700 1 -- 172.21.15.19:0/301512049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11800fd30 con 0x5634a0b7b150 2024-05-09T02:21:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.991+0000 7fa13ee25b80 1 -- 172.21.15.19:0/301512049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b7b150 msgr2=0x5634a0b6a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.991+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/301512049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b7b150 0x5634a0b6a1f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa1180099c0 tx=0x7fa118005040 comp rx=0 tx=0).stop 2024-05-09T02:21:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.994+0000 7fa13ee25b80 1 -- 172.21.15.19:0/301512049 shutdown_connections 2024-05-09T02:21:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.994+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/301512049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b7b150 0x5634a0b6a1f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.994+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/301512049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0ace160 0x5634a0b657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.994+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/301512049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a0b616a0 0x5634a0b6bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:25.994+0000 7fa13ee25b80 1 -- 172.21.15.19:0/301512049 >> 172.21.15.19:0/301512049 conn(0x5634a0a446a0 msgr2=0x5634a0b69ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.000+0000 7fa13ee25b80 1 -- 172.21.15.19:0/301512049 shutdown_connections 2024-05-09T02:21:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.000+0000 7fa13ee25b80 1 -- 172.21.15.19:0/301512049 wait complete. 2024-05-09T02:21:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.001+0000 7fa13ee25b80 1 -- start start 2024-05-09T02:21:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.001+0000 7fa13ee25b80 1 Processor -- start 2024-05-09T02:21:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.005+0000 7fa13ee25b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0ace160 0x5634a0b762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.005+0000 7fa13ee25b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a0b616a0 0x5634a0b76800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.005+0000 7fa13ee25b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b7b150 0x5634a0b71fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.005+0000 7fa13ee25b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281ae3c0 con 0x5634a0b7b150 2024-05-09T02:21:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.005+0000 7fa13ee25b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281adc30 con 0x5634a0b616a0 2024-05-09T02:21:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.005+0000 7fa13ee25b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281aae70 con 0x5634a0ace160 2024-05-09T02:21:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.005+0000 7fa13ee25b80 10 client.0 _ll_register_callbacks cb 0x5634a08f56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.005+0000 7fa12e9ac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b7b150 0x5634a0b71fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.005+0000 7fa12e9ac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b7b150 0x5634a0b71fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34262/0 (socket says 172.21.15.19:34262) 2024-05-09T02:21:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.005+0000 7fa12e9ac700 1 -- 172.21.15.19:0/2378253237 learned_addr learned my addr 172.21.15.19:0/2378253237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.006+0000 7fa12e9ac700 1 -- 172.21.15.19:0/2378253237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0ace160 msgr2=0x5634a0b762c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.006+0000 7fa12e9ac700 1 --2- 172.21.15.19:0/2378253237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0ace160 0x5634a0b762c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.006+0000 7fa12e9ac700 1 -- 172.21.15.19:0/2378253237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a0b616a0 msgr2=0x5634a0b76800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.006+0000 7fa12e9ac700 1 --2- 172.21.15.19:0/2378253237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a0b616a0 0x5634a0b76800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.006+0000 7fa12e9ac700 1 -- 172.21.15.19:0/2378253237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634a0b72510 con 0x5634a0b7b150 2024-05-09T02:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.006+0000 7fa12e9ac700 1 --2- 172.21.15.19:0/2378253237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b7b150 0x5634a0b71fd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa1180052d0 tx=0x7fa118003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f4a186d360d746 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.009+0000 7fa111ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:21:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.011+0000 7fa111ffb700 1 -- 172.21.15.19:0/2378253237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118051020 con 0x5634a0b7b150 2024-05-09T02:21:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.011+0000 7fa111ffb700 1 -- 172.21.15.19:0/2378253237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fa11800bba0 con 0x5634a0b7b150 2024-05-09T02:21:26.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.013+0000 7fa13ee25b80 1 -- 172.21.15.19:0/2378253237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1180095b0 con 0x5634a0b7b150 2024-05-09T02:21:26.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.014+0000 7fa111ffb700 1 -- 172.21.15.19:0/2378253237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa118047070 con 0x5634a0b7b150 2024-05-09T02:21:26.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.016+0000 7fa13ee25b80 10 client.16581 fetch_fsmap learned FSMap version 55 2024-05-09T02:21:26.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.016+0000 7fa13ee25b80 1 -- 172.21.15.19:0/2378253237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5634a0b73790 con 0x5634a0b7b150 2024-05-09T02:21:26.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.017+0000 7fa111ffb700 1 -- 172.21.15.19:0/2378253237 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1180095b0 con 0x5634a0b7b150 2024-05-09T02:21:26.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.017+0000 7fa111ffb700 1 -- 172.21.15.19:0/2378253237 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 55) ==== 8374+0+0 (secure 0 0 0) 0x7fa118041070 con 0x5634a0b7b150 2024-05-09T02:21:26.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.017+0000 7fa13ee25b80 10 client.16581 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T02:21:26.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.017+0000 7fa13ee25b80 10 client.16581 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:21:26.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.017+0000 7fa13ee25b80 20 client.16581 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:26.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.017+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/2378253237 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5634a0be5780 0x5634a0be7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:26.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.017+0000 7fa13ee25b80 4 client.16581 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:21:26.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.017+0000 7fa13ee25b80 1 -- 172.21.15.19:0/2378253237 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634a0a8ba70 con 0x5634a0be5780 2024-05-09T02:21:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.023+0000 7fa12e1ab700 1 --2- 172.21.15.19:0/2378253237 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5634a0be5780 0x5634a0be7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:26.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.038+0000 7fa12e1ab700 1 --2- 172.21.15.19:0/2378253237 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5634a0be5780 0x5634a0be7ba0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8b4d7b176a64f5ba server_cookie=f68708b4dc80f255 in_seq=0 out_seq=0 2024-05-09T02:21:26.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.039+0000 7fa111ffb700 10 client.16581 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:21:26.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.039+0000 7fa111ffb700 1 -- 172.21.15.19:0/2378253237 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634a0a8ba70 con 0x5634a0be5780 2024-05-09T02:21:26.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.039+0000 7fa111ffb700 10 client.16581 handle_command_reply: tid=0 2024-05-09T02:21:26.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.039+0000 7fa13ee25b80 1 client.16581 shutdown 2024-05-09T02:21:26.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.043+0000 7fa13ee25b80 1 -- 172.21.15.19:0/2378253237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b7b150 msgr2=0x5634a0b71fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:26.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.043+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/2378253237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b7b150 0x5634a0b71fd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa1180052d0 tx=0x7fa118003eb0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.043+0000 7fa13ee25b80 1 -- 172.21.15.19:0/2378253237 shutdown_connections 2024-05-09T02:21:26.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.043+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/2378253237 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x5634a0be5780 0x5634a0be7ba0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.043+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/2378253237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a0b7b150 0x5634a0b71fd0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.043+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/2378253237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0ace160 0x5634a0b762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.043+0000 7fa13ee25b80 1 --2- 172.21.15.19:0/2378253237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634a0b616a0 0x5634a0b76800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.043+0000 7fa13ee25b80 1 -- 172.21.15.19:0/2378253237 >> 172.21.15.19:0/2378253237 conn(0x5634a0a446a0 msgr2=0x5634a0b64e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:26.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.043+0000 7fa111ffb700 0 client.16581 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:21:26.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.046+0000 7fa13ee25b80 1 -- 172.21.15.19:0/2378253237 shutdown_connections 2024-05-09T02:21:26.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.050+0000 7fa13ee25b80 1 -- 172.21.15.19:0/2378253237 wait complete. 2024-05-09T02:21:26.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.050+0000 7fa13ee25b80 20 client.16581 trim_cache size 0 max 16384 2024-05-09T02:21:26.054 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:21:26.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.052+0000 7fa13040f700 1 -- 172.21.15.19:0/2658812278 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa1140b2a70 msgr2=0x7fa1140b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:26.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.052+0000 7fa13040f700 1 --2- 172.21.15.19:0/2658812278 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa1140b2a70 0x7fa1140b4f80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa118005950 tx=0x7fa1180050d0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.052+0000 7fa13040f700 1 -- 172.21.15.19:0/2658812278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa128107df0 msgr2=0x7fa12819c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:26.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.052+0000 7fa13040f700 1 --2- 172.21.15.19:0/2658812278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa128107df0 0x7fa12819c150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa12000eee0 tx=0x7fa12000c3a0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.052+0000 7fa13040f700 1 -- 172.21.15.19:0/2658812278 shutdown_connections 2024-05-09T02:21:26.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.052+0000 7fa13040f700 1 --2- 172.21.15.19:0/2658812278 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa1140b2a70 0x7fa1140b4f80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.052+0000 7fa13040f700 1 --2- 172.21.15.19:0/2658812278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa128108740 0x7fa12819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.052+0000 7fa13040f700 1 --2- 172.21.15.19:0/2658812278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1281034c0 0x7fa12819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.052+0000 7fa13040f700 1 --2- 172.21.15.19:0/2658812278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa128107df0 0x7fa12819c150 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:26.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.052+0000 7fa13040f700 1 -- 172.21.15.19:0/2658812278 >> 172.21.15.19:0/2658812278 conn(0x7fa1280fedc0 msgr2=0x7fa128107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:26.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.053+0000 7fa13040f700 1 -- 172.21.15.19:0/2658812278 shutdown_connections 2024-05-09T02:21:26.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:26.053+0000 7fa13040f700 1 -- 172.21.15.19:0/2658812278 wait complete. 2024-05-09T02:21:26.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:26 smithi059 ceph-mon[34781]: pgmap v315: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.4 MiB/s wr, 566 op/s 2024-05-09T02:21:26.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:26 smithi196 ceph-mon[34718]: pgmap v315: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.4 MiB/s wr, 566 op/s 2024-05-09T02:21:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:26 smithi019 ceph-mon[30151]: pgmap v315: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.4 MiB/s wr, 566 op/s 2024-05-09T02:21:26.888 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T02:21:27.259 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:21:27.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 585.0 s, 1141.2 tps, lat 87.740 ms stddev 96.724 2024-05-09T02:21:28.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:28 smithi059 ceph-mon[34781]: pgmap v316: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.8 MiB/s wr, 511 op/s 2024-05-09T02:21:28.741 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:28 smithi019 ceph-mon[30151]: pgmap v316: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.8 MiB/s wr, 511 op/s 2024-05-09T02:21:28.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:28 smithi196 ceph-mon[34718]: pgmap v316: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.8 MiB/s wr, 511 op/s 2024-05-09T02:21:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.225+0000 7efed4366700 1 -- 172.21.15.19:0/3306945222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc10b2c0 msgr2=0x7efecc117840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.225+0000 7efed4366700 1 --2- 172.21.15.19:0/3306945222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc10b2c0 0x7efecc117840 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efec4009f90 tx=0x7efec400b330 comp rx=0 tx=0).stop 2024-05-09T02:21:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.225+0000 7efed4366700 1 -- 172.21.15.19:0/3306945222 shutdown_connections 2024-05-09T02:21:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.225+0000 7efed4366700 1 --2- 172.21.15.19:0/3306945222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc10b2c0 0x7efecc117840 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.225+0000 7efed4366700 1 --2- 172.21.15.19:0/3306945222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc1109a0 0x7efecc110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.225+0000 7efed4366700 1 --2- 172.21.15.19:0/3306945222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efecc10a970 0x7efecc10ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.225+0000 7efed4366700 1 -- 172.21.15.19:0/3306945222 >> 172.21.15.19:0/3306945222 conn(0x7efecc076cc0 msgr2=0x7efecc0770d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.225+0000 7efed4366700 1 -- 172.21.15.19:0/3306945222 shutdown_connections 2024-05-09T02:21:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.225+0000 7efed4366700 1 -- 172.21.15.19:0/3306945222 wait complete. 2024-05-09T02:21:29.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.226+0000 7efed4366700 1 Processor -- start 2024-05-09T02:21:29.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.227+0000 7efed4366700 1 -- start start 2024-05-09T02:21:29.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.227+0000 7efed4366700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efecc10a970 0x7efecc1a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.227+0000 7efed4366700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc10b2c0 0x7efecc1a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.227+0000 7efed4366700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc1109a0 0x7efecc1a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.227+0000 7efed4366700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efecc11aac0 con 0x7efecc10a970 2024-05-09T02:21:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.227+0000 7efed4366700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efecc11a940 con 0x7efecc1109a0 2024-05-09T02:21:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.227+0000 7efed4366700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efecc11ac40 con 0x7efecc10b2c0 2024-05-09T02:21:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.227+0000 7efed1901700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc10b2c0 0x7efecc1a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.227+0000 7efed2102700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efecc10a970 0x7efecc1a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.227+0000 7efed1901700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc10b2c0 0x7efecc1a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:55370/0 (socket says 172.21.15.19:55370) 2024-05-09T02:21:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.227+0000 7efed1901700 1 -- 172.21.15.19:0/1420342309 learned_addr learned my addr 172.21.15.19:0/1420342309 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.228+0000 7efed2903700 1 --2- 172.21.15.19:0/1420342309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc1109a0 0x7efecc1a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.228+0000 7efed1901700 1 -- 172.21.15.19:0/1420342309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc1109a0 msgr2=0x7efecc1a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.228+0000 7efed1901700 1 --2- 172.21.15.19:0/1420342309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc1109a0 0x7efecc1a5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.228+0000 7efed1901700 1 -- 172.21.15.19:0/1420342309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efecc10a970 msgr2=0x7efecc1a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.228+0000 7efed1901700 1 --2- 172.21.15.19:0/1420342309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efecc10a970 0x7efecc1a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.228+0000 7efed1901700 1 -- 172.21.15.19:0/1420342309 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc1a6400 con 0x7efecc10b2c0 2024-05-09T02:21:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.228+0000 7efed2903700 1 --2- 172.21.15.19:0/1420342309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc1109a0 0x7efecc1a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:21:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.229+0000 7efed1901700 1 --2- 172.21.15.19:0/1420342309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc10b2c0 0x7efecc1a1ad0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efebc00e9f0 tx=0x7efebc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.229+0000 7efec37fe700 1 -- 172.21.15.19:0/1420342309 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc00c9c0 con 0x7efecc10b2c0 2024-05-09T02:21:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.229+0000 7efec37fe700 1 -- 172.21.15.19:0/1420342309 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efebc00cb60 con 0x7efecc10b2c0 2024-05-09T02:21:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.229+0000 7efed4366700 1 -- 172.21.15.19:0/1420342309 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efecc1a66f0 con 0x7efecc10b2c0 2024-05-09T02:21:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.229+0000 7efec37fe700 1 -- 172.21.15.19:0/1420342309 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc018680 con 0x7efecc10b2c0 2024-05-09T02:21:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.229+0000 7efed4366700 1 -- 172.21.15.19:0/1420342309 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efecc1ae010 con 0x7efecc10b2c0 2024-05-09T02:21:29.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.232+0000 7efec37fe700 1 -- 172.21.15.19:0/1420342309 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efebc010430 con 0x7efecc10b2c0 2024-05-09T02:21:29.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.233+0000 7efec37fe700 1 --2- 172.21.15.19:0/1420342309 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efeb80b2a70 0x7efeb80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.233+0000 7efec37fe700 1 -- 172.21.15.19:0/1420342309 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7efebc0e27e0 con 0x7efecc10b2c0 2024-05-09T02:21:29.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.233+0000 7efed2102700 1 --2- 172.21.15.19:0/1420342309 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efeb80b2a70 0x7efeb80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.234+0000 7efed2102700 1 --2- 172.21.15.19:0/1420342309 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efeb80b2a70 0x7efeb80b4f80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efecc10bab0 tx=0x7efec80073d0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:29.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.234+0000 7efee2d7cb80 1 Processor -- start 2024-05-09T02:21:29.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.234+0000 7efee2d7cb80 1 -- start start 2024-05-09T02:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.235+0000 7efee2d7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51d05d00 0x55eb51e0e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.235+0000 7efee2d7cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51c8b4a0 0x55eb51e0e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.235+0000 7efee2d7cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 0x55eb51e08810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.235+0000 7efed2102700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51d05d00 0x55eb51e0e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.235+0000 7efee2d7cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eb51d2b0a0 con 0x55eb51d05d00 2024-05-09T02:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.235+0000 7efee2d7cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55eb51cd09a0 con 0x55eb51c8b4a0 2024-05-09T02:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.235+0000 7efee2d7cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55eb51d27060 con 0x55eb51e08400 2024-05-09T02:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.235+0000 7efed2102700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51d05d00 0x55eb51e0e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48810/0 (socket says 172.21.15.19:48810) 2024-05-09T02:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.235+0000 7efed2102700 1 -- 172.21.15.19:0/4059960472 learned_addr learned my addr 172.21.15.19:0/4059960472 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.235+0000 7efed1901700 1 --2- 172.21.15.19:0/4059960472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51c8b4a0 0x55eb51e0e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.235+0000 7efed2903700 1 --2- 172.21.15.19:0/4059960472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 0x55eb51e08810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.236+0000 7efed2102700 1 -- 172.21.15.19:0/4059960472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 msgr2=0x55eb51e08810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.236+0000 7efed2102700 1 --2- 172.21.15.19:0/4059960472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 0x55eb51e08810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.236+0000 7efed2102700 1 -- 172.21.15.19:0/4059960472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51c8b4a0 msgr2=0x55eb51e0e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.236+0000 7efed2102700 1 --2- 172.21.15.19:0/4059960472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51c8b4a0 0x55eb51e0e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.236+0000 7efed2102700 1 -- 172.21.15.19:0/4059960472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb51cc1500 con 0x55eb51d05d00 2024-05-09T02:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.236+0000 7efed2102700 1 --2- 172.21.15.19:0/4059960472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51d05d00 0x55eb51e0e240 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7efec8010d60 tx=0x7efec8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0296d247ed5a9e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.237+0000 7efec0d47700 1 -- 172.21.15.19:0/4059960472 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8011bb0 con 0x55eb51d05d00 2024-05-09T02:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.237+0000 7efec0d47700 1 -- 172.21.15.19:0/4059960472 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec8011d50 con 0x55eb51d05d00 2024-05-09T02:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.237+0000 7efec0d47700 1 -- 172.21.15.19:0/4059960472 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec801c610 con 0x55eb51d05d00 2024-05-09T02:21:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.238+0000 7efee2d7cb80 1 -- 172.21.15.19:0/4059960472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51d05d00 msgr2=0x55eb51e0e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.238+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/4059960472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51d05d00 0x55eb51e0e240 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7efec8010d60 tx=0x7efec8013000 comp rx=0 tx=0).stop 2024-05-09T02:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.238+0000 7efee2d7cb80 1 -- 172.21.15.19:0/4059960472 shutdown_connections 2024-05-09T02:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.238+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/4059960472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 0x55eb51e08810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.238+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/4059960472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51d05d00 0x55eb51e0e240 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.238+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/4059960472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51c8b4a0 0x55eb51e0e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.238+0000 7efee2d7cb80 1 -- 172.21.15.19:0/4059960472 >> 172.21.15.19:0/4059960472 conn(0x55eb51c8ef70 msgr2=0x55eb51e03220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.239+0000 7efee2d7cb80 1 -- 172.21.15.19:0/4059960472 shutdown_connections 2024-05-09T02:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.239+0000 7efee2d7cb80 1 -- 172.21.15.19:0/4059960472 wait complete. 2024-05-09T02:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.239+0000 7efee2d7cb80 1 -- start start 2024-05-09T02:21:29.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.240+0000 7efee2d7cb80 1 Processor -- start 2024-05-09T02:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.240+0000 7efee2d7cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51c8b4a0 0x55eb51e12050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.240+0000 7efee2d7cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51d05d00 0x55eb51e12590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.240+0000 7efee2d7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51e08400 0x55eb51e12ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.240+0000 7efee2d7cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eb51e09160 con 0x55eb51e08400 2024-05-09T02:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.240+0000 7efee2d7cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55eb51cc5b20 con 0x55eb51d05d00 2024-05-09T02:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.240+0000 7efee2d7cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55eb51e15020 con 0x55eb51c8b4a0 2024-05-09T02:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.241+0000 7efed1901700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51d05d00 0x55eb51e12590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.241+0000 7efed1901700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51d05d00 0x55eb51e12590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:33990/0 (socket says 172.21.15.19:33990) 2024-05-09T02:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.241+0000 7efed1901700 1 -- 172.21.15.19:0/23680840 learned_addr learned my addr 172.21.15.19:0/23680840 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.241+0000 7efed2903700 1 --2- 172.21.15.19:0/23680840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51e08400 0x55eb51e12ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.241+0000 7efee2d7cb80 10 client.0 _ll_register_callbacks cb 0x55eb51b9c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.241+0000 7efed2102700 1 --2- 172.21.15.19:0/23680840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51c8b4a0 0x55eb51e12050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.241+0000 7efed1901700 1 -- 172.21.15.19:0/23680840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51c8b4a0 msgr2=0x55eb51e12050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.241+0000 7efed1901700 1 --2- 172.21.15.19:0/23680840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51c8b4a0 0x55eb51e12050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.241+0000 7efed1901700 1 -- 172.21.15.19:0/23680840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51e08400 msgr2=0x55eb51e12ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.241+0000 7efed1901700 1 --2- 172.21.15.19:0/23680840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51e08400 0x55eb51e12ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.241+0000 7efed1901700 1 -- 172.21.15.19:0/23680840 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb51ba6490 con 0x55eb51d05d00 2024-05-09T02:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.241+0000 7efed2903700 1 --2- 172.21.15.19:0/23680840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51e08400 0x55eb51e12ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.241+0000 7efed2102700 1 --2- 172.21.15.19:0/23680840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51c8b4a0 0x55eb51e12050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.241+0000 7efed1901700 1 --2- 172.21.15.19:0/23680840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51d05d00 0x55eb51e12590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efebc000c00 tx=0x7efebc0a9670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5526e0c9824574b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.241+0000 7efeb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.242+0000 7efeb5ffb700 1 -- 172.21.15.19:0/23680840 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc0adde0 con 0x55eb51d05d00 2024-05-09T02:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.242+0000 7efeb5ffb700 1 -- 172.21.15.19:0/23680840 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7efebc0b2610 con 0x55eb51d05d00 2024-05-09T02:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.242+0000 7efee2d7cb80 1 -- 172.21.15.19:0/23680840 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efec8011d50 con 0x55eb51d05d00 2024-05-09T02:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.242+0000 7efeb5ffb700 1 -- 172.21.15.19:0/23680840 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7efebc0b2890 con 0x55eb51d05d00 2024-05-09T02:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.242+0000 7efee2d7cb80 10 client.26321 fetch_fsmap learned FSMap version 55 2024-05-09T02:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.242+0000 7efee2d7cb80 1 -- 172.21.15.19:0/23680840 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55eb51e1f880 con 0x55eb51d05d00 2024-05-09T02:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.242+0000 7efeb5ffb700 1 -- 172.21.15.19:0/23680840 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efebc0cc0e0 con 0x55eb51d05d00 2024-05-09T02:21:29.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.242+0000 7efeb5ffb700 1 -- 172.21.15.19:0/23680840 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 55) ==== 8374+0+0 (secure 0 0 0) 0x7efebc0cace0 con 0x55eb51d05d00 2024-05-09T02:21:29.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.243+0000 7efee2d7cb80 10 client.26321 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T02:21:29.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.243+0000 7efee2d7cb80 10 client.26321 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:21:29.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.243+0000 7efee2d7cb80 20 client.26321 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:29.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.243+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/23680840 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55eb51e21e60 0x55eb51e24280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.243+0000 7efee2d7cb80 4 client.26321 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:21:29.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.243+0000 7efee2d7cb80 1 -- 172.21.15.19:0/23680840 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efec8011d50 con 0x55eb51e21e60 2024-05-09T02:21:29.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.243+0000 7efed2102700 1 --2- 172.21.15.19:0/23680840 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55eb51e21e60 0x55eb51e24280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.243+0000 7efeb5ffb700 10 client.26321 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:21:29.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.244+0000 7efed2102700 1 --2- 172.21.15.19:0/23680840 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55eb51e21e60 0x55eb51e24280 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bfa608925c64439f server_cookie=ac89d487679b83d7 in_seq=0 out_seq=0 2024-05-09T02:21:29.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.245+0000 7efeb5ffb700 1 -- 172.21.15.19:0/23680840 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7efec8011d50 con 0x55eb51e21e60 2024-05-09T02:21:29.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.245+0000 7efeb5ffb700 10 client.26321 handle_command_reply: tid=0 2024-05-09T02:21:29.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.245+0000 7efee2d7cb80 1 client.26321 shutdown 2024-05-09T02:21:29.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.245+0000 7efee2d7cb80 1 -- 172.21.15.19:0/23680840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51d05d00 msgr2=0x55eb51e12590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:29.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.245+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/23680840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51d05d00 0x55eb51e12590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efebc000c00 tx=0x7efebc0a9670 comp rx=0 tx=0).stop 2024-05-09T02:21:29.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.245+0000 7efee2d7cb80 1 -- 172.21.15.19:0/23680840 shutdown_connections 2024-05-09T02:21:29.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.245+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/23680840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51e08400 0x55eb51e12ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.245+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/23680840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51c8b4a0 0x55eb51e12050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.245+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/23680840 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55eb51e21e60 0x55eb51e24280 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.245+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/23680840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51d05d00 0x55eb51e12590 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.245+0000 7efee2d7cb80 1 -- 172.21.15.19:0/23680840 >> 172.21.15.19:0/23680840 conn(0x55eb51c8ef70 msgr2=0x55eb51dfcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:29.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.245+0000 7efeb5ffb700 0 client.26321 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:21:29.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.245+0000 7efee2d7cb80 1 -- 172.21.15.19:0/23680840 shutdown_connections 2024-05-09T02:21:29.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.246+0000 7efee2d7cb80 1 -- 172.21.15.19:0/23680840 wait complete. 2024-05-09T02:21:29.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.246+0000 7efee2d7cb80 20 client.26321 trim_cache size 0 max 16384 2024-05-09T02:21:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.265+0000 7efee2d7cb80 1 Processor -- start 2024-05-09T02:21:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.265+0000 7efee2d7cb80 1 -- start start 2024-05-09T02:21:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.265+0000 7efee2d7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51c8b4a0 0x55eb51e12910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.265+0000 7efee2d7cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51d05d00 0x55eb51dfd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.265+0000 7efee2d7cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 0x55eb51e1de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.265+0000 7efee2d7cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efecc1b56a0 con 0x55eb51c8b4a0 2024-05-09T02:21:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.265+0000 7efee2d7cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efecc1b72d0 con 0x55eb51d05d00 2024-05-09T02:21:29.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.265+0000 7efee2d7cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efecc1b5d70 con 0x55eb51e08400 2024-05-09T02:21:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.265+0000 7efed2102700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51c8b4a0 0x55eb51e12910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.265+0000 7efed1901700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51d05d00 0x55eb51dfd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.265+0000 7efed1901700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51d05d00 0x55eb51dfd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:34006/0 (socket says 172.21.15.19:34006) 2024-05-09T02:21:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.265+0000 7efed1901700 1 -- 172.21.15.19:0/2490167335 learned_addr learned my addr 172.21.15.19:0/2490167335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:29.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.265+0000 7efed2903700 1 --2- 172.21.15.19:0/2490167335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 0x55eb51e1de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.266+0000 7efed1901700 1 -- 172.21.15.19:0/2490167335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 msgr2=0x55eb51e1de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.266+0000 7efed1901700 1 --2- 172.21.15.19:0/2490167335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 0x55eb51e1de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.266+0000 7efed1901700 1 -- 172.21.15.19:0/2490167335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51c8b4a0 msgr2=0x55eb51e12910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.266+0000 7efed1901700 1 --2- 172.21.15.19:0/2490167335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51c8b4a0 0x55eb51e12910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.266+0000 7efed1901700 1 -- 172.21.15.19:0/2490167335 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb51e11370 con 0x55eb51d05d00 2024-05-09T02:21:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.266+0000 7efed1901700 1 --2- 172.21.15.19:0/2490167335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51d05d00 0x55eb51dfd700 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efebc00c5b0 tx=0x7efebc00ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b36fa84c017397c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.266+0000 7efeb7fff700 1 -- 172.21.15.19:0/2490167335 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc0adde0 con 0x55eb51d05d00 2024-05-09T02:21:29.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.266+0000 7efeb7fff700 1 -- 172.21.15.19:0/2490167335 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efebc0a53a0 con 0x55eb51d05d00 2024-05-09T02:21:29.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.266+0000 7efeb7fff700 1 -- 172.21.15.19:0/2490167335 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc0a51c0 con 0x55eb51d05d00 2024-05-09T02:21:29.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.267+0000 7efee2d7cb80 1 -- 172.21.15.19:0/2490167335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51d05d00 msgr2=0x55eb51dfd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:29.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.267+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/2490167335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51d05d00 0x55eb51dfd700 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efebc00c5b0 tx=0x7efebc00ef80 comp rx=0 tx=0).stop 2024-05-09T02:21:29.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.267+0000 7efee2d7cb80 1 -- 172.21.15.19:0/2490167335 shutdown_connections 2024-05-09T02:21:29.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.267+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/2490167335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 0x55eb51e1de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.267+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/2490167335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51c8b4a0 0x55eb51e12910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.267+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/2490167335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51d05d00 0x55eb51dfd700 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.267+0000 7efee2d7cb80 1 -- 172.21.15.19:0/2490167335 >> 172.21.15.19:0/2490167335 conn(0x55eb51c8ef70 msgr2=0x55eb51dfcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:29.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.267+0000 7efee2d7cb80 1 -- 172.21.15.19:0/2490167335 shutdown_connections 2024-05-09T02:21:29.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.267+0000 7efee2d7cb80 1 -- 172.21.15.19:0/2490167335 wait complete. 2024-05-09T02:21:29.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.268+0000 7efee2d7cb80 1 -- start start 2024-05-09T02:21:29.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.268+0000 7efee2d7cb80 1 Processor -- start 2024-05-09T02:21:29.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.268+0000 7efee2d7cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51c8b4a0 0x55eb51e15110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.268+0000 7efee2d7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51d05d00 0x55eb51e15650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.268+0000 7efee2d7cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 0x55eb51e15b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.268+0000 7efee2d7cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efecc1b6bd0 con 0x55eb51d05d00 2024-05-09T02:21:29.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.268+0000 7efee2d7cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efecc1b6440 con 0x55eb51c8b4a0 2024-05-09T02:21:29.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.268+0000 7efee2d7cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efecc1b3660 con 0x55eb51e08400 2024-05-09T02:21:29.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.268+0000 7efee2d7cb80 10 client.0 _ll_register_callbacks cb 0x55eb51e03880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:29.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efed2102700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51c8b4a0 0x55eb51e15110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efed2903700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 0x55eb51e15b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efed2102700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51c8b4a0 0x55eb51e15110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:34012/0 (socket says 172.21.15.19:34012) 2024-05-09T02:21:29.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efed2903700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 0x55eb51e15b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:55404/0 (socket says 172.21.15.19:55404) 2024-05-09T02:21:29.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efed2102700 1 -- 172.21.15.19:0/842966976 learned_addr learned my addr 172.21.15.19:0/842966976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:29.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efed1901700 1 --2- 172.21.15.19:0/842966976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51d05d00 0x55eb51e15650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efed2102700 1 -- 172.21.15.19:0/842966976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 msgr2=0x55eb51e15b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:29.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efed2102700 1 --2- 172.21.15.19:0/842966976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 0x55eb51e15b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efed2102700 1 -- 172.21.15.19:0/842966976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51d05d00 msgr2=0x55eb51e15650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:29.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efed2102700 1 --2- 172.21.15.19:0/842966976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51d05d00 0x55eb51e15650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efed2102700 1 -- 172.21.15.19:0/842966976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb51e1abf0 con 0x55eb51c8b4a0 2024-05-09T02:21:29.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efed2903700 1 --2- 172.21.15.19:0/842966976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 0x55eb51e15b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:21:29.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efed2102700 1 --2- 172.21.15.19:0/842966976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51c8b4a0 0x55eb51e15110 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efec8010d00 tx=0x7efec8010fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a54a0cecac280a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:29.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efeb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:21:29.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efeb5ffb700 1 -- 172.21.15.19:0/842966976 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80050b0 con 0x55eb51c8b4a0 2024-05-09T02:21:29.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efeb5ffb700 1 -- 172.21.15.19:0/842966976 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7efec8005250 con 0x55eb51c8b4a0 2024-05-09T02:21:29.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.269+0000 7efee2d7cb80 1 -- 172.21.15.19:0/842966976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efebc0a53a0 con 0x55eb51c8b4a0 2024-05-09T02:21:29.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.270+0000 7efeb5ffb700 1 -- 172.21.15.19:0/842966976 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7efec8041050 con 0x55eb51c8b4a0 2024-05-09T02:21:29.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.270+0000 7efee2d7cb80 10 client.26327 fetch_fsmap learned FSMap version 55 2024-05-09T02:21:29.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.270+0000 7efee2d7cb80 1 -- 172.21.15.19:0/842966976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55eb51e8b610 con 0x55eb51c8b4a0 2024-05-09T02:21:29.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.270+0000 7efeb5ffb700 1 -- 172.21.15.19:0/842966976 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efebc0a53a0 con 0x55eb51c8b4a0 2024-05-09T02:21:29.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.270+0000 7efeb5ffb700 1 -- 172.21.15.19:0/842966976 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 55) ==== 8374+0+0 (secure 0 0 0) 0x7efec8003680 con 0x55eb51c8b4a0 2024-05-09T02:21:29.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.270+0000 7efee2d7cb80 10 client.26327 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T02:21:29.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.270+0000 7efee2d7cb80 10 client.26327 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:21:29.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.270+0000 7efee2d7cb80 20 client.26327 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:29.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.270+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/842966976 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55eb51e8ba40 0x55eb51e8de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:29.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.270+0000 7efee2d7cb80 4 client.26327 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:21:29.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.270+0000 7efee2d7cb80 1 -- 172.21.15.19:0/842966976 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb51d38a80 con 0x55eb51e8ba40 2024-05-09T02:21:29.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.270+0000 7efed1901700 1 --2- 172.21.15.19:0/842966976 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55eb51e8ba40 0x55eb51e8de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:29.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.271+0000 7efed1901700 1 --2- 172.21.15.19:0/842966976 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55eb51e8ba40 0x55eb51e8de60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28650745dfed98f8 server_cookie=9dba59aff3c56a35 in_seq=0 out_seq=0 2024-05-09T02:21:29.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.271+0000 7efeb5ffb700 10 client.26327 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:21:29.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.272+0000 7efeb5ffb700 1 -- 172.21.15.19:0/842966976 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb51d38a80 con 0x55eb51e8ba40 2024-05-09T02:21:29.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.272+0000 7efeb5ffb700 10 client.26327 handle_command_reply: tid=0 2024-05-09T02:21:29.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.272+0000 7efee2d7cb80 1 client.26327 shutdown 2024-05-09T02:21:29.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.272+0000 7efee2d7cb80 1 -- 172.21.15.19:0/842966976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51c8b4a0 msgr2=0x55eb51e15110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:29.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.272+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/842966976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51c8b4a0 0x55eb51e15110 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efec8010d00 tx=0x7efec8010fc0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.273+0000 7efee2d7cb80 1 -- 172.21.15.19:0/842966976 shutdown_connections 2024-05-09T02:21:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.273+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/842966976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eb51e08400 0x55eb51e15b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.273+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/842966976 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55eb51e8ba40 0x55eb51e8de60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.273+0000 7efeb5ffb700 0 client.26327 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:21:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.273+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/842966976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb51c8b4a0 0x55eb51e15110 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.273+0000 7efee2d7cb80 1 --2- 172.21.15.19:0/842966976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb51d05d00 0x55eb51e15650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.273+0000 7efee2d7cb80 1 -- 172.21.15.19:0/842966976 >> 172.21.15.19:0/842966976 conn(0x55eb51c8ef70 msgr2=0x55eb51e073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.273+0000 7efee2d7cb80 1 -- 172.21.15.19:0/842966976 shutdown_connections 2024-05-09T02:21:29.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.274+0000 7efee2d7cb80 1 -- 172.21.15.19:0/842966976 wait complete. 2024-05-09T02:21:29.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.274+0000 7efee2d7cb80 20 client.26327 trim_cache size 0 max 16384 2024-05-09T02:21:29.281 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:21:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.275+0000 7efed4366700 1 -- 172.21.15.19:0/1420342309 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efeb80b2a70 msgr2=0x7efeb80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.275+0000 7efed4366700 1 --2- 172.21.15.19:0/1420342309 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efeb80b2a70 0x7efeb80b4f80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efecc10bab0 tx=0x7efec80073d0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.275+0000 7efed4366700 1 -- 172.21.15.19:0/1420342309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc10b2c0 msgr2=0x7efecc1a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.275+0000 7efed4366700 1 --2- 172.21.15.19:0/1420342309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc10b2c0 0x7efecc1a1ad0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efebc00e9f0 tx=0x7efebc00ed00 comp rx=0 tx=0).stop 2024-05-09T02:21:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.275+0000 7efed4366700 1 -- 172.21.15.19:0/1420342309 shutdown_connections 2024-05-09T02:21:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.275+0000 7efed4366700 1 --2- 172.21.15.19:0/1420342309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc1109a0 0x7efecc1a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.275+0000 7efed4366700 1 --2- 172.21.15.19:0/1420342309 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efeb80b2a70 0x7efeb80b4f80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.275+0000 7efed4366700 1 --2- 172.21.15.19:0/1420342309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efecc10a970 0x7efecc1a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.275+0000 7efed4366700 1 --2- 172.21.15.19:0/1420342309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc10b2c0 0x7efecc1a1ad0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.275+0000 7efed4366700 1 -- 172.21.15.19:0/1420342309 >> 172.21.15.19:0/1420342309 conn(0x7efecc076cc0 msgr2=0x7efecc1089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:29.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.275+0000 7efed4366700 1 -- 172.21.15.19:0/1420342309 shutdown_connections 2024-05-09T02:21:29.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:29.275+0000 7efed4366700 1 -- 172.21.15.19:0/1420342309 wait complete. 2024-05-09T02:21:30.128 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T02:21:30.511 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:21:30.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:30 smithi196 ceph-mon[34718]: pgmap v317: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.7 MiB/s wr, 706 op/s 2024-05-09T02:21:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:30 smithi019 ceph-mon[30151]: pgmap v317: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.7 MiB/s wr, 706 op/s 2024-05-09T02:21:30.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:30 smithi059 ceph-mon[34781]: pgmap v317: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.7 MiB/s wr, 706 op/s 2024-05-09T02:21:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 590.0 s, 1164.6 tps, lat 85.354 ms stddev 95.280 2024-05-09T02:21:32.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.689+0000 7f1acba63700 1 -- 172.21.15.19:0/3344309171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ac410b280 msgr2=0x7f1ac411d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:32.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.689+0000 7f1acba63700 1 --2- 172.21.15.19:0/3344309171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ac410b280 0x7f1ac411d3f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40098f0 tx=0x7f1ab4009c00 comp rx=0 tx=0).stop 2024-05-09T02:21:32.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.691+0000 7f1acba63700 1 -- 172.21.15.19:0/3344309171 shutdown_connections 2024-05-09T02:21:32.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.691+0000 7f1acba63700 1 --2- 172.21.15.19:0/3344309171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ac411d930 0x7f1ac411fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.691+0000 7f1acba63700 1 --2- 172.21.15.19:0/3344309171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac410a930 0x7f1ac410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.691+0000 7f1acba63700 1 --2- 172.21.15.19:0/3344309171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ac410b280 0x7f1ac411d3f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.691+0000 7f1acba63700 1 -- 172.21.15.19:0/3344309171 >> 172.21.15.19:0/3344309171 conn(0x7f1ac406f4d0 msgr2=0x7f1ac406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:32.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.694+0000 7f1acba63700 1 -- 172.21.15.19:0/3344309171 shutdown_connections 2024-05-09T02:21:32.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.695+0000 7f1acba63700 1 -- 172.21.15.19:0/3344309171 wait complete. 2024-05-09T02:21:32.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.696+0000 7f1acba63700 1 Processor -- start 2024-05-09T02:21:32.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.696+0000 7f1acba63700 1 -- start start 2024-05-09T02:21:32.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.696+0000 7f1acba63700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac410a930 0x7f1ac41af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.696+0000 7f1acba63700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ac410b280 0x7f1ac41afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.696+0000 7f1acba63700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ac411d930 0x7f1ac41a9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.697+0000 7f1acba63700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac41232f0 con 0x7f1ac410a930 2024-05-09T02:21:32.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.697+0000 7f1acba63700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4123170 con 0x7f1ac410b280 2024-05-09T02:21:32.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.697+0000 7f1acba63700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4123470 con 0x7f1ac411d930 2024-05-09T02:21:32.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.697+0000 7f1aca000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ac411d930 0x7f1ac41a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:32.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.697+0000 7f1aca000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ac411d930 0x7f1ac41a9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:55426/0 (socket says 172.21.15.19:55426) 2024-05-09T02:21:32.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.697+0000 7f1aca000700 1 -- 172.21.15.19:0/1253216309 learned_addr learned my addr 172.21.15.19:0/1253216309 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:32.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.698+0000 7f1aca000700 1 -- 172.21.15.19:0/1253216309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ac410b280 msgr2=0x7f1ac41afc40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T02:21:32.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.698+0000 7f1aca000700 1 --2- 172.21.15.19:0/1253216309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ac410b280 0x7f1ac41afc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.698+0000 7f1aca000700 1 -- 172.21.15.19:0/1253216309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac410a930 msgr2=0x7f1ac41af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:32.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.698+0000 7f1aca000700 1 --2- 172.21.15.19:0/1253216309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac410a930 0x7f1ac41af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.698+0000 7f1aca000700 1 -- 172.21.15.19:0/1253216309 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac41a9fd0 con 0x7f1ac411d930 2024-05-09T02:21:32.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.699+0000 7f1aca000700 1 --2- 172.21.15.19:0/1253216309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ac411d930 0x7f1ac41a9920 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00ef90 tx=0x7f1abc00c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:32.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.701+0000 7f1aba7fc700 1 -- 172.21.15.19:0/1253216309 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc0092e0 con 0x7f1ac411d930 2024-05-09T02:21:32.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.701+0000 7f1aba7fc700 1 -- 172.21.15.19:0/1253216309 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1abc004bb0 con 0x7f1ac411d930 2024-05-09T02:21:32.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.701+0000 7f1aba7fc700 1 -- 172.21.15.19:0/1253216309 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc004280 con 0x7f1ac411d930 2024-05-09T02:21:32.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.701+0000 7f1acba63700 1 -- 172.21.15.19:0/1253216309 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac41aa260 con 0x7f1ac411d930 2024-05-09T02:21:32.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.701+0000 7f1acba63700 1 -- 172.21.15.19:0/1253216309 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac41aa690 con 0x7f1ac411d930 2024-05-09T02:21:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.707+0000 7f1ada479b80 1 Processor -- start 2024-05-09T02:21:32.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.719+0000 7f1aba7fc700 1 -- 172.21.15.19:0/1253216309 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1abc01b030 con 0x7f1ac411d930 2024-05-09T02:21:32.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.719+0000 7f1ada479b80 1 -- start start 2024-05-09T02:21:32.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.720+0000 7f1ada479b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a07cd00 0x55d96a185240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.720+0000 7f1ada479b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a0024a0 0x55d96a185780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.720+0000 7f1ada479b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a17f400 0x55d96a17f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.720+0000 7f1ada479b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d96a0a20a0 con 0x55d96a17f400 2024-05-09T02:21:32.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.720+0000 7f1ada479b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d96a0479a0 con 0x55d96a07cd00 2024-05-09T02:21:32.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.720+0000 7f1ada479b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d96a09e060 con 0x55d96a0024a0 2024-05-09T02:21:32.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.721+0000 7f1ac8ffe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a17f400 0x55d96a17f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:32.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.721+0000 7f1ac8ffe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a17f400 0x55d96a17f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48864/0 (socket says 172.21.15.19:48864) 2024-05-09T02:21:32.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.721+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/3523100566 learned_addr learned my addr 172.21.15.19:0/3523100566 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:32.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.721+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/3523100566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a07cd00 0x55d96a185240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:32.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.721+0000 7f1ac97ff700 1 --2- 172.21.15.19:0/3523100566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a0024a0 0x55d96a185780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:32.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.722+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/3523100566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a0024a0 msgr2=0x55d96a185780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:32.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.722+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/3523100566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a0024a0 0x55d96a185780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.722+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/3523100566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a07cd00 msgr2=0x55d96a185240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:32.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.722+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/3523100566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a07cd00 0x55d96a185240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.722+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/3523100566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96a038500 con 0x55d96a17f400 2024-05-09T02:21:32.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.722+0000 7f1aba7fc700 1 --2- 172.21.15.19:0/1253216309 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1ab00b2b40 0x7f1ab00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.722+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/3523100566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a17f400 0x55d96a17f810 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009810 tx=0x7f1ab4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1429f9af4ee751a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:32.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.722+0000 7f1aba7fc700 1 -- 172.21.15.19:0/1253216309 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f1abc0e2130 con 0x7f1ac411d930 2024-05-09T02:21:32.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.722+0000 7f1aca000700 1 --2- 172.21.15.19:0/1253216309 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1ab00b2b40 0x7f1ab00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:32.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.723+0000 7f1aaf7fe700 1 -- 172.21.15.19:0/3523100566 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4004030 con 0x55d96a17f400 2024-05-09T02:21:32.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.723+0000 7f1aaf7fe700 1 -- 172.21.15.19:0/3523100566 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ab400f460 con 0x55d96a17f400 2024-05-09T02:21:32.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.725+0000 7f1ada479b80 1 -- 172.21.15.19:0/3523100566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a17f400 msgr2=0x55d96a17f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:32.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.725+0000 7f1ada479b80 1 --2- 172.21.15.19:0/3523100566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a17f400 0x55d96a17f810 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009810 tx=0x7f1ab4000bc0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.725+0000 7f1ada479b80 1 -- 172.21.15.19:0/3523100566 shutdown_connections 2024-05-09T02:21:32.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.725+0000 7f1ada479b80 1 --2- 172.21.15.19:0/3523100566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a17f400 0x55d96a17f810 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.725+0000 7f1ada479b80 1 --2- 172.21.15.19:0/3523100566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a07cd00 0x55d96a185240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.725+0000 7f1ada479b80 1 --2- 172.21.15.19:0/3523100566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a0024a0 0x55d96a185780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.725+0000 7f1ada479b80 1 -- 172.21.15.19:0/3523100566 >> 172.21.15.19:0/3523100566 conn(0x55d96a005f70 msgr2=0x55d96a17a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:32.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.725+0000 7f1aca000700 1 --2- 172.21.15.19:0/1253216309 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1ab00b2b40 0x7f1ab00b5050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0df900 tx=0x7f1abc003c50 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:32.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.728+0000 7f1ada479b80 1 -- 172.21.15.19:0/3523100566 shutdown_connections 2024-05-09T02:21:32.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.728+0000 7f1ada479b80 1 -- 172.21.15.19:0/3523100566 wait complete. 2024-05-09T02:21:32.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.728+0000 7f1ada479b80 1 -- start start 2024-05-09T02:21:32.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.729+0000 7f1ada479b80 1 Processor -- start 2024-05-09T02:21:32.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.729+0000 7f1ada479b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a188f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.729+0000 7f1ada479b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a07cd00 0x55d96a189480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.729+0000 7f1ada479b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a17f400 0x55d96a1899c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.729+0000 7f1ada479b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d96a180150 con 0x55d96a17f400 2024-05-09T02:21:32.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.729+0000 7f1ada479b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d96a03cb20 con 0x55d96a0024a0 2024-05-09T02:21:32.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.729+0000 7f1ada479b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d96a19d030 con 0x55d96a07cd00 2024-05-09T02:21:32.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.729+0000 7f1ada479b80 10 client.0 _ll_register_callbacks cb 0x55d969f136c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:32.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.730+0000 7f1ac97ff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a07cd00 0x55d96a189480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:32.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.730+0000 7f1ac8ffe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a188f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:32.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.730+0000 7f1ac8ffe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a188f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:34054/0 (socket says 172.21.15.19:34054) 2024-05-09T02:21:32.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.730+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/2118010262 learned_addr learned my addr 172.21.15.19:0/2118010262 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:32.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.730+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/2118010262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a17f400 0x55d96a1899c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:32.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.731+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/2118010262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a07cd00 msgr2=0x55d96a189480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:32.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.731+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/2118010262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a07cd00 0x55d96a189480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.731+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/2118010262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a17f400 msgr2=0x55d96a1899c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:32.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.731+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/2118010262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a17f400 0x55d96a1899c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.731+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/2118010262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d969f1d490 con 0x55d96a0024a0 2024-05-09T02:21:32.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.731+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/2118010262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a188f40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009680 tx=0x7f1ab40058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a03458491863851 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:32.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.731+0000 7f1aacff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:21:32.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.731+0000 7f1aacff9700 1 -- 172.21.15.19:0/2118010262 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4004ad0 con 0x55d96a0024a0 2024-05-09T02:21:32.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.732+0000 7f1ada479b80 1 -- 172.21.15.19:0/2118010262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ab400f460 con 0x55d96a0024a0 2024-05-09T02:21:32.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.732+0000 7f1aacff9700 1 -- 172.21.15.19:0/2118010262 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f1ab402f040 con 0x55d96a0024a0 2024-05-09T02:21:32.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.734+0000 7f1aacff9700 1 -- 172.21.15.19:0/2118010262 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1ab4004c70 con 0x55d96a0024a0 2024-05-09T02:21:32.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.734+0000 7f1ada479b80 10 client.26336 fetch_fsmap learned FSMap version 55 2024-05-09T02:21:32.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.734+0000 7f1ada479b80 1 -- 172.21.15.19:0/2118010262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55d96a1a7960 con 0x55d96a0024a0 2024-05-09T02:21:32.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.735+0000 7f1aacff9700 1 -- 172.21.15.19:0/2118010262 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ab400f460 con 0x55d96a0024a0 2024-05-09T02:21:32.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.736+0000 7f1aacff9700 1 -- 172.21.15.19:0/2118010262 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 55) ==== 8374+0+0 (secure 0 0 0) 0x7f1ab4032020 con 0x55d96a0024a0 2024-05-09T02:21:32.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.736+0000 7f1ada479b80 10 client.26336 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T02:21:32.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.736+0000 7f1ada479b80 10 client.26336 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:21:32.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.736+0000 7f1ada479b80 20 client.26336 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:32.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.736+0000 7f1ada479b80 1 --2- 172.21.15.19:0/2118010262 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d96a1a9e80 0x55d96a1ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.736+0000 7f1ada479b80 4 client.26336 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:21:32.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.736+0000 7f1ada479b80 1 -- 172.21.15.19:0/2118010262 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d96a0afa80 con 0x55d96a1a9e80 2024-05-09T02:21:32.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.736+0000 7f1ac97ff700 1 --2- 172.21.15.19:0/2118010262 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d96a1a9e80 0x55d96a1ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:32.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.737+0000 7f1aacff9700 10 client.26336 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:21:32.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.737+0000 7f1ac97ff700 1 --2- 172.21.15.19:0/2118010262 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d96a1a9e80 0x55d96a1ac2a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=197cdf9944848aab server_cookie=87da620185496fd4 in_seq=0 out_seq=0 2024-05-09T02:21:32.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.739+0000 7f1aacff9700 1 -- 172.21.15.19:0/2118010262 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d96a0afa80 con 0x55d96a1a9e80 2024-05-09T02:21:32.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.739+0000 7f1aacff9700 10 client.26336 handle_command_reply: tid=0 2024-05-09T02:21:32.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.739+0000 7f1ada479b80 1 client.26336 shutdown 2024-05-09T02:21:32.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.740+0000 7f1ada479b80 1 -- 172.21.15.19:0/2118010262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 msgr2=0x55d96a188f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:32.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.740+0000 7f1ada479b80 1 --2- 172.21.15.19:0/2118010262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a188f40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009680 tx=0x7f1ab40058e0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.741+0000 7f1ada479b80 1 -- 172.21.15.19:0/2118010262 shutdown_connections 2024-05-09T02:21:32.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.741+0000 7f1ada479b80 1 --2- 172.21.15.19:0/2118010262 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d96a1a9e80 0x55d96a1ac2a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.741+0000 7f1ada479b80 1 --2- 172.21.15.19:0/2118010262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a17f400 0x55d96a1899c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.741+0000 7f1ada479b80 1 --2- 172.21.15.19:0/2118010262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a188f40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.741+0000 7f1ada479b80 1 --2- 172.21.15.19:0/2118010262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a07cd00 0x55d96a189480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.741+0000 7f1ada479b80 1 -- 172.21.15.19:0/2118010262 >> 172.21.15.19:0/2118010262 conn(0x55d96a005f70 msgr2=0x55d96a173ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.741+0000 7f1aacff9700 0 client.26336 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:21:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.741+0000 7f1ada479b80 1 -- 172.21.15.19:0/2118010262 shutdown_connections 2024-05-09T02:21:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.741+0000 7f1ada479b80 1 -- 172.21.15.19:0/2118010262 wait complete. 2024-05-09T02:21:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.741+0000 7f1ada479b80 20 client.26336 trim_cache size 0 max 16384 2024-05-09T02:21:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.766+0000 7f1ada479b80 1 Processor -- start 2024-05-09T02:21:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.766+0000 7f1ada479b80 1 -- start start 2024-05-09T02:21:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.766+0000 7f1ada479b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a189800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.766+0000 7f1ada479b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a07cd00 0x55d96a184740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.766+0000 7f1ada479b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a17f400 0x55d96a1a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.766+0000 7f1ada479b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac411a220 con 0x55d96a07cd00 2024-05-09T02:21:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.766+0000 7f1ada479b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac41b7060 con 0x55d96a0024a0 2024-05-09T02:21:32.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.766+0000 7f1ada479b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac411a8f0 con 0x55d96a17f400 2024-05-09T02:21:32.771 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:32 smithi019 ceph-mon[30151]: pgmap v318: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.8 MiB/s wr, 615 op/s 2024-05-09T02:21:32.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.767+0000 7f1ac8ffe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a189800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:32.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.767+0000 7f1ac8ffe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a189800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:34064/0 (socket says 172.21.15.19:34064) 2024-05-09T02:21:32.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.767+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/1878712156 learned_addr learned my addr 172.21.15.19:0/1878712156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:32.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.767+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/1878712156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a17f400 0x55d96a1a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:32.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.767+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/1878712156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a17f400 msgr2=0x55d96a1a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:32.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.767+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/1878712156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a17f400 0x55d96a1a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.767+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/1878712156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a07cd00 msgr2=0x55d96a184740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:32.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.768+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/1878712156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a07cd00 0x55d96a184740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.768+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/1878712156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96a1769d0 con 0x55d96a0024a0 2024-05-09T02:21:32.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.768+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/1878712156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a189800 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40047b0 tx=0x7f1ab4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7aa753fd1486e16 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:32.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.768+0000 7f1aaeffd700 1 -- 172.21.15.19:0/1878712156 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4004ad0 con 0x55d96a0024a0 2024-05-09T02:21:32.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.768+0000 7f1aaeffd700 1 -- 172.21.15.19:0/1878712156 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ab400f460 con 0x55d96a0024a0 2024-05-09T02:21:32.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.768+0000 7f1aaeffd700 1 -- 172.21.15.19:0/1878712156 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4003b30 con 0x55d96a0024a0 2024-05-09T02:21:32.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.769+0000 7f1ada479b80 1 -- 172.21.15.19:0/1878712156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 msgr2=0x55d96a189800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.770+0000 7f1ada479b80 1 --2- 172.21.15.19:0/1878712156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a189800 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40047b0 tx=0x7f1ab4000bc0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.770+0000 7f1ada479b80 1 -- 172.21.15.19:0/1878712156 shutdown_connections 2024-05-09T02:21:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.770+0000 7f1ada479b80 1 --2- 172.21.15.19:0/1878712156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a17f400 0x55d96a1a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.770+0000 7f1ada479b80 1 --2- 172.21.15.19:0/1878712156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a189800 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.770+0000 7f1ada479b80 1 --2- 172.21.15.19:0/1878712156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a07cd00 0x55d96a184740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.770+0000 7f1ada479b80 1 -- 172.21.15.19:0/1878712156 >> 172.21.15.19:0/1878712156 conn(0x55d96a005f70 msgr2=0x55d96a184030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.770+0000 7f1ada479b80 1 -- 172.21.15.19:0/1878712156 shutdown_connections 2024-05-09T02:21:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.770+0000 7f1ada479b80 1 -- 172.21.15.19:0/1878712156 wait complete. 2024-05-09T02:21:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.770+0000 7f1ada479b80 1 -- start start 2024-05-09T02:21:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.770+0000 7f1ada479b80 1 Processor -- start 2024-05-09T02:21:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.771+0000 7f1ada479b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a1b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.771+0000 7f1ada479b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a07cd00 0x55d96a1a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.771+0000 7f1ada479b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a17f400 0x55d96a1a2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.771+0000 7f1ada479b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac41b6960 con 0x55d96a07cd00 2024-05-09T02:21:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.771+0000 7f1ada479b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac411afc0 con 0x55d96a0024a0 2024-05-09T02:21:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.771+0000 7f1ada479b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac41181e0 con 0x55d96a17f400 2024-05-09T02:21:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.771+0000 7f1ada479b80 10 client.0 _ll_register_callbacks cb 0x55d969f136c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.772+0000 7f1ac8ffe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a1b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.772+0000 7f1ac8ffe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a1b1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:34076/0 (socket says 172.21.15.19:34076) 2024-05-09T02:21:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.772+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/3033263951 learned_addr learned my addr 172.21.15.19:0/3033263951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.772+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/3033263951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a17f400 0x55d96a1a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.772+0000 7f1ac97ff700 1 --2- 172.21.15.19:0/3033263951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a07cd00 0x55d96a1a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.772+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/3033263951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a17f400 msgr2=0x55d96a1a2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.772+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/3033263951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a17f400 0x55d96a1a2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.772+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/3033263951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a07cd00 msgr2=0x55d96a1a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.772+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/3033263951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a07cd00 0x55d96a1a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.772+0000 7f1ac8ffe700 1 -- 172.21.15.19:0/3033263951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96a184c80 con 0x55d96a0024a0 2024-05-09T02:21:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.773+0000 7f1ac8ffe700 1 --2- 172.21.15.19:0/3033263951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a1b1da0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400fa30 tx=0x7f1ab4009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=611dad82425f5970 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.773+0000 7f1aacff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:21:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.773+0000 7f1aacff9700 1 -- 172.21.15.19:0/3033263951 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4003680 con 0x55d96a0024a0 2024-05-09T02:21:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.773+0000 7f1aacff9700 1 -- 172.21.15.19:0/3033263951 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f1ab4004210 con 0x55d96a0024a0 2024-05-09T02:21:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.773+0000 7f1ada479b80 1 -- 172.21.15.19:0/3033263951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ab400f460 con 0x55d96a0024a0 2024-05-09T02:21:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.773+0000 7f1aacff9700 1 -- 172.21.15.19:0/3033263951 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1ab4004410 con 0x55d96a0024a0 2024-05-09T02:21:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.773+0000 7f1ada479b80 10 client.26342 fetch_fsmap learned FSMap version 55 2024-05-09T02:21:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.773+0000 7f1ada479b80 1 -- 172.21.15.19:0/3033263951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55d96a213650 con 0x55d96a0024a0 2024-05-09T02:21:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.773+0000 7f1aacff9700 1 -- 172.21.15.19:0/3033263951 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ab400f460 con 0x55d96a0024a0 2024-05-09T02:21:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.774+0000 7f1aacff9700 1 -- 172.21.15.19:0/3033263951 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 55) ==== 8374+0+0 (secure 0 0 0) 0x7f1ab4032070 con 0x55d96a0024a0 2024-05-09T02:21:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.774+0000 7f1ada479b80 10 client.26342 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T02:21:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.774+0000 7f1ada479b80 10 client.26342 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:21:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.774+0000 7f1ada479b80 20 client.26342 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.774+0000 7f1ada479b80 1 --2- 172.21.15.19:0/3033263951 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d96a213930 0x55d96a215d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.774+0000 7f1ada479b80 4 client.26342 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:21:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.774+0000 7f1ada479b80 1 -- 172.21.15.19:0/3033263951 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d96a0a9b20 con 0x55d96a213930 2024-05-09T02:21:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.774+0000 7f1ac97ff700 1 --2- 172.21.15.19:0/3033263951 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d96a213930 0x55d96a215d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.775+0000 7f1aacff9700 10 client.26342 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:21:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.775+0000 7f1ac97ff700 1 --2- 172.21.15.19:0/3033263951 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d96a213930 0x55d96a215d50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3fd1c66b4b2d2256 server_cookie=fa322d65ffc02442 in_seq=0 out_seq=0 2024-05-09T02:21:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.775+0000 7f1aacff9700 1 -- 172.21.15.19:0/3033263951 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d96a0a9b20 con 0x55d96a213930 2024-05-09T02:21:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.775+0000 7f1aacff9700 10 client.26342 handle_command_reply: tid=0 2024-05-09T02:21:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.775+0000 7f1ada479b80 1 client.26342 shutdown 2024-05-09T02:21:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.775+0000 7f1ada479b80 1 -- 172.21.15.19:0/3033263951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 msgr2=0x55d96a1b1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.775+0000 7f1ada479b80 1 --2- 172.21.15.19:0/3033263951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a1b1da0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400fa30 tx=0x7f1ab4009870 comp rx=0 tx=0).stop 2024-05-09T02:21:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.775+0000 7f1ada479b80 1 -- 172.21.15.19:0/3033263951 shutdown_connections 2024-05-09T02:21:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.775+0000 7f1ada479b80 1 --2- 172.21.15.19:0/3033263951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d96a17f400 0x55d96a1a2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.775+0000 7f1ada479b80 1 --2- 172.21.15.19:0/3033263951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d96a0024a0 0x55d96a1b1da0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.775+0000 7f1ada479b80 1 --2- 172.21.15.19:0/3033263951 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d96a213930 0x55d96a215d50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.775+0000 7f1ada479b80 1 --2- 172.21.15.19:0/3033263951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d96a07cd00 0x55d96a1a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.775+0000 7f1ada479b80 1 -- 172.21.15.19:0/3033263951 >> 172.21.15.19:0/3033263951 conn(0x55d96a005f70 msgr2=0x55d96a173ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.775+0000 7f1aacff9700 0 client.26342 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:21:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.776+0000 7f1ada479b80 1 -- 172.21.15.19:0/3033263951 shutdown_connections 2024-05-09T02:21:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.776+0000 7f1ada479b80 1 -- 172.21.15.19:0/3033263951 wait complete. 2024-05-09T02:21:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.776+0000 7f1ada479b80 20 client.26342 trim_cache size 0 max 16384 2024-05-09T02:21:32.785 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:21:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.777+0000 7f1acba63700 1 -- 172.21.15.19:0/1253216309 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1ab00b2b40 msgr2=0x7f1ab00b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.777+0000 7f1acba63700 1 --2- 172.21.15.19:0/1253216309 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1ab00b2b40 0x7f1ab00b5050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0df900 tx=0x7f1abc003c50 comp rx=0 tx=0).stop 2024-05-09T02:21:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.777+0000 7f1acba63700 1 -- 172.21.15.19:0/1253216309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ac411d930 msgr2=0x7f1ac41a9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.777+0000 7f1acba63700 1 --2- 172.21.15.19:0/1253216309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ac411d930 0x7f1ac41a9920 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00ef90 tx=0x7f1abc00c450 comp rx=0 tx=0).stop 2024-05-09T02:21:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.777+0000 7f1acba63700 1 -- 172.21.15.19:0/1253216309 shutdown_connections 2024-05-09T02:21:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.777+0000 7f1acba63700 1 --2- 172.21.15.19:0/1253216309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ac411d930 0x7f1ac41a9920 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.777+0000 7f1acba63700 1 --2- 172.21.15.19:0/1253216309 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1ab00b2b40 0x7f1ab00b5050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.777+0000 7f1acba63700 1 --2- 172.21.15.19:0/1253216309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac410a930 0x7f1ac41af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.777+0000 7f1acba63700 1 --2- 172.21.15.19:0/1253216309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ac410b280 0x7f1ac41afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:32.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.777+0000 7f1acba63700 1 -- 172.21.15.19:0/1253216309 >> 172.21.15.19:0/1253216309 conn(0x7f1ac406f4d0 msgr2=0x7f1ac411b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:32.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.777+0000 7f1acba63700 1 -- 172.21.15.19:0/1253216309 shutdown_connections 2024-05-09T02:21:32.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:32.777+0000 7f1acba63700 1 -- 172.21.15.19:0/1253216309 wait complete. 2024-05-09T02:21:32.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:32 smithi196 ceph-mon[34718]: pgmap v318: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.8 MiB/s wr, 615 op/s 2024-05-09T02:21:32.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:32 smithi059 ceph-mon[34781]: pgmap v318: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.8 MiB/s wr, 615 op/s 2024-05-09T02:21:33.525 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T02:21:33.893 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:21:34.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:34 smithi019 ceph-mon[30151]: pgmap v319: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.7 MiB/s wr, 511 op/s 2024-05-09T02:21:34.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:34 smithi196 ceph-mon[34718]: pgmap v319: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.7 MiB/s wr, 511 op/s 2024-05-09T02:21:34.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:34 smithi059 ceph-mon[34781]: pgmap v319: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.7 MiB/s wr, 511 op/s 2024-05-09T02:21:36.769 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:36 smithi059 ceph-mon[34781]: pgmap v320: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.9 MiB/s wr, 761 op/s 2024-05-09T02:21:36.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:36 smithi196 ceph-mon[34718]: pgmap v320: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.9 MiB/s wr, 761 op/s 2024-05-09T02:21:36.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:36 smithi019 ceph-mon[30151]: pgmap v320: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.9 MiB/s wr, 761 op/s 2024-05-09T02:21:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.341+0000 7fc83dbf2700 1 -- 172.21.15.19:0/581782402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8381034c0 msgr2=0x7fc8381038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.341+0000 7fc83dbf2700 1 --2- 172.21.15.19:0/581782402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8381034c0 0x7fc8381038d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc828009990 tx=0x7fc828009ca0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.341+0000 7fc83dbf2700 1 -- 172.21.15.19:0/581782402 shutdown_connections 2024-05-09T02:21:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.341+0000 7fc83dbf2700 1 --2- 172.21.15.19:0/581782402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc838108740 0x7fc83806b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.341+0000 7fc83dbf2700 1 --2- 172.21.15.19:0/581782402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8381034c0 0x7fc8381038d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.341+0000 7fc83dbf2700 1 --2- 172.21.15.19:0/581782402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc838107df0 0x7fc838108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.341+0000 7fc83dbf2700 1 -- 172.21.15.19:0/581782402 >> 172.21.15.19:0/581782402 conn(0x7fc8380fedc0 msgr2=0x7fc8381011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.341+0000 7fc83dbf2700 1 -- 172.21.15.19:0/581782402 shutdown_connections 2024-05-09T02:21:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.341+0000 7fc83dbf2700 1 -- 172.21.15.19:0/581782402 wait complete. 2024-05-09T02:21:37.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.342+0000 7fc83dbf2700 1 Processor -- start 2024-05-09T02:21:37.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.342+0000 7fc83dbf2700 1 -- start start 2024-05-09T02:21:37.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc83dbf2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8381034c0 0x7fc838111960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc83dbf2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc838107df0 0x7fc83810fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc83dbf2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc838108740 0x7fc8381100f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc83dbf2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc838112430 con 0x7fc838108740 2024-05-09T02:21:37.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc83dbf2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381122b0 con 0x7fc838107df0 2024-05-09T02:21:37.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc83dbf2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381125b0 con 0x7fc8381034c0 2024-05-09T02:21:37.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc836ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc838107df0 0x7fc83810fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:37.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc836ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc838107df0 0x7fc83810fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:58118/0 (socket says 172.21.15.19:58118) 2024-05-09T02:21:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc837fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc838108740 0x7fc8381100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc836ffd700 1 -- 172.21.15.19:0/1572249730 learned_addr learned my addr 172.21.15.19:0/1572249730 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc8377fe700 1 --2- 172.21.15.19:0/1572249730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8381034c0 0x7fc838111960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc836ffd700 1 -- 172.21.15.19:0/1572249730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8381034c0 msgr2=0x7fc838111960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc836ffd700 1 --2- 172.21.15.19:0/1572249730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8381034c0 0x7fc838111960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc836ffd700 1 -- 172.21.15.19:0/1572249730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc838108740 msgr2=0x7fc8381100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc836ffd700 1 --2- 172.21.15.19:0/1572249730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc838108740 0x7fc8381100f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.343+0000 7fc836ffd700 1 -- 172.21.15.19:0/1572249730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc838110730 con 0x7fc838107df0 2024-05-09T02:21:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.344+0000 7fc836ffd700 1 --2- 172.21.15.19:0/1572249730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc838107df0 0x7fc83810fbb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00b580 tx=0x7fc82c00b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.344+0000 7fc834ff9700 1 -- 172.21.15.19:0/1572249730 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c015020 con 0x7fc838107df0 2024-05-09T02:21:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.344+0000 7fc834ff9700 1 -- 172.21.15.19:0/1572249730 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82c011ce0 con 0x7fc838107df0 2024-05-09T02:21:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.344+0000 7fc834ff9700 1 -- 172.21.15.19:0/1572249730 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c010c30 con 0x7fc838107df0 2024-05-09T02:21:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.344+0000 7fc83dbf2700 1 -- 172.21.15.19:0/1572249730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8381adc50 con 0x7fc838107df0 2024-05-09T02:21:37.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.344+0000 7fc83dbf2700 1 -- 172.21.15.19:0/1572249730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8381adff0 con 0x7fc838107df0 2024-05-09T02:21:37.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.350+0000 7fc84c608b80 1 Processor -- start 2024-05-09T02:21:37.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.358+0000 7fc834ff9700 1 -- 172.21.15.19:0/1572249730 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc82c018020 con 0x7fc838107df0 2024-05-09T02:21:37.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.359+0000 7fc834ff9700 1 --2- 172.21.15.19:0/1572249730 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc8200b2a20 0x7fc8200b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.359+0000 7fc834ff9700 1 -- 172.21.15.19:0/1572249730 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fc82c0a53f0 con 0x7fc838107df0 2024-05-09T02:21:37.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.364+0000 7fc84c608b80 1 -- start start 2024-05-09T02:21:37.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.364+0000 7fc84c608b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b13014160 0x558b130ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.364+0000 7fc84c608b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 0x558b130ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.364+0000 7fc84c608b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b130a76a0 0x558b130a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.364+0000 7fc84c608b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558b12fc9ff0 con 0x558b13014160 2024-05-09T02:21:37.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.364+0000 7fc84c608b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558b12f687f0 con 0x558b130a76a0 2024-05-09T02:21:37.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.364+0000 7fc84c608b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558b12fc5fb0 con 0x558b12f2a4a0 2024-05-09T02:21:37.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.365+0000 7fc837fff700 1 --2- 172.21.15.19:0/1572249730 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc8200b2a20 0x7fc8200b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:37.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.366+0000 7fc8377fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b13014160 0x558b130ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:37.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.366+0000 7fc8377fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b13014160 0x558b130ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33228/0 (socket says 172.21.15.19:33228) 2024-05-09T02:21:37.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.366+0000 7fc8377fe700 1 -- 172.21.15.19:0/867367580 learned_addr learned my addr 172.21.15.19:0/867367580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:37.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.366+0000 7fc8377fe700 1 -- 172.21.15.19:0/867367580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 msgr2=0x558b130ada20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:21:37.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.366+0000 7fc8377fe700 1 --2- 172.21.15.19:0/867367580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 0x558b130ada20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.366+0000 7fc8377fe700 1 -- 172.21.15.19:0/867367580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b130a76a0 msgr2=0x558b130a7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:37.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.366+0000 7fc8377fe700 1 --2- 172.21.15.19:0/867367580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b130a76a0 0x558b130a7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.366+0000 7fc8377fe700 1 -- 172.21.15.19:0/867367580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b12f60370 con 0x558b13014160 2024-05-09T02:21:37.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.366+0000 7fc8377fe700 1 --2- 172.21.15.19:0/867367580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b13014160 0x558b130ad480 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc8280037f0 tx=0x7fc828003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659448ebbe2f4703 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:37.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.367+0000 7fc83dbf2700 1 -- 172.21.15.19:0/867367580 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828003cb0 con 0x558b13014160 2024-05-09T02:21:37.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.367+0000 7fc83dbf2700 1 -- 172.21.15.19:0/867367580 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc828003e50 con 0x558b13014160 2024-05-09T02:21:37.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.368+0000 7fc84c608b80 1 -- 172.21.15.19:0/867367580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b13014160 msgr2=0x558b130ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:37.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.368+0000 7fc84c608b80 1 --2- 172.21.15.19:0/867367580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b13014160 0x558b130ad480 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc8280037f0 tx=0x7fc828003860 comp rx=0 tx=0).stop 2024-05-09T02:21:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.381+0000 7fc84c608b80 1 -- 172.21.15.19:0/867367580 shutdown_connections 2024-05-09T02:21:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.381+0000 7fc84c608b80 1 --2- 172.21.15.19:0/867367580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b130a76a0 0x558b130a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.381+0000 7fc84c608b80 1 --2- 172.21.15.19:0/867367580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b13014160 0x558b130ad480 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.381+0000 7fc84c608b80 1 --2- 172.21.15.19:0/867367580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 0x558b130ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.381+0000 7fc84c608b80 1 -- 172.21.15.19:0/867367580 >> 172.21.15.19:0/867367580 conn(0x558b12f8a6a0 msgr2=0x558b130a2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:37.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.383+0000 7fc837fff700 1 --2- 172.21.15.19:0/1572249730 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc8200b2a20 0x7fc8200b4f30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc83000a4c0 tx=0x7fc830009650 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:37.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.387+0000 7fc84c608b80 1 -- 172.21.15.19:0/867367580 shutdown_connections 2024-05-09T02:21:37.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.391+0000 7fc84c608b80 1 -- 172.21.15.19:0/867367580 wait complete. 2024-05-09T02:21:37.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.391+0000 7fc84c608b80 1 -- start start 2024-05-09T02:21:37.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.391+0000 7fc84c608b80 1 Processor -- start 2024-05-09T02:21:37.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.392+0000 7fc84c608b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 0x558b130b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.392+0000 7fc84c608b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b13014160 0x558b130b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.392+0000 7fc84c608b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 0x558b130b1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.392+0000 7fc84c608b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558b12e45ba0 con 0x558b130a76a0 2024-05-09T02:21:37.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.392+0000 7fc84c608b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558b12f84150 con 0x558b13014160 2024-05-09T02:21:37.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.392+0000 7fc84c608b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558b130a8480 con 0x558b12f2a4a0 2024-05-09T02:21:37.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.392+0000 7fc84c608b80 10 client.0 _ll_register_callbacks cb 0x558b12e3b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.394+0000 7fc8377fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 0x558b130b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.394+0000 7fc8377fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 0x558b130b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:52092/0 (socket says 172.21.15.19:52092) 2024-05-09T02:21:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.394+0000 7fc8377fe700 1 -- 172.21.15.19:0/1196232291 learned_addr learned my addr 172.21.15.19:0/1196232291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.395+0000 7fc837fff700 1 --2- 172.21.15.19:0/1196232291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 0x558b130b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.395+0000 7fc8377fe700 1 -- 172.21.15.19:0/1196232291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b13014160 msgr2=0x558b130b0fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:21:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.395+0000 7fc8377fe700 1 --2- 172.21.15.19:0/1196232291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b13014160 0x558b130b0fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.395+0000 7fc8377fe700 1 -- 172.21.15.19:0/1196232291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 msgr2=0x558b130b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.395+0000 7fc8377fe700 1 --2- 172.21.15.19:0/1196232291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 0x558b130b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.395+0000 7fc8377fe700 1 -- 172.21.15.19:0/1196232291 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b12e45490 con 0x558b12f2a4a0 2024-05-09T02:21:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.397+0000 7fc8377fe700 1 --2- 172.21.15.19:0/1196232291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 0x558b130b3c30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc828000c00 tx=0x7fc828009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1827acca48e3ec77 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.397+0000 7fc816ffd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:21:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.397+0000 7fc816ffd700 1 -- 172.21.15.19:0/1196232291 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8280056b0 con 0x558b12f2a4a0 2024-05-09T02:21:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.397+0000 7fc816ffd700 1 -- 172.21.15.19:0/1196232291 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fc82800f640 con 0x558b12f2a4a0 2024-05-09T02:21:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.397+0000 7fc84c608b80 1 -- 172.21.15.19:0/1196232291 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc828003e50 con 0x558b12f2a4a0 2024-05-09T02:21:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.398+0000 7fc816ffd700 1 -- 172.21.15.19:0/1196232291 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fc82800f840 con 0x558b12f2a4a0 2024-05-09T02:21:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.398+0000 7fc84c608b80 10 client.26386 fetch_fsmap learned FSMap version 55 2024-05-09T02:21:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.398+0000 7fc84c608b80 1 -- 172.21.15.19:0/1196232291 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558b130e06f0 con 0x558b12f2a4a0 2024-05-09T02:21:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.398+0000 7fc816ffd700 1 -- 172.21.15.19:0/1196232291 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc828003e50 con 0x558b12f2a4a0 2024-05-09T02:21:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.399+0000 7fc816ffd700 1 -- 172.21.15.19:0/1196232291 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 55) ==== 8374+0+0 (secure 0 0 0) 0x7fc828034a00 con 0x558b12f2a4a0 2024-05-09T02:21:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.399+0000 7fc84c608b80 10 client.26386 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T02:21:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.399+0000 7fc84c608b80 10 client.26386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:21:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.399+0000 7fc84c608b80 20 client.26386 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.399+0000 7fc84c608b80 1 --2- 172.21.15.19:0/1196232291 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558b130e2d00 0x558b130e5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.399+0000 7fc84c608b80 4 client.26386 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:21:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.399+0000 7fc84c608b80 1 -- 172.21.15.19:0/1196232291 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b12fd79d0 con 0x558b130e2d00 2024-05-09T02:21:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.399+0000 7fc836ffd700 1 --2- 172.21.15.19:0/1196232291 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558b130e2d00 0x558b130e5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.400+0000 7fc816ffd700 10 client.26386 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.400+0000 7fc836ffd700 1 --2- 172.21.15.19:0/1196232291 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558b130e2d00 0x558b130e5160 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=137eea57949ad98 server_cookie=1a0f3450d02aeecc in_seq=0 out_seq=0 2024-05-09T02:21:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.401+0000 7fc816ffd700 1 -- 172.21.15.19:0/1196232291 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558b12fd79d0 con 0x558b130e2d00 2024-05-09T02:21:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.401+0000 7fc816ffd700 10 client.26386 handle_command_reply: tid=0 2024-05-09T02:21:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.402+0000 7fc84c608b80 1 client.26386 shutdown 2024-05-09T02:21:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.404+0000 7fc84c608b80 1 -- 172.21.15.19:0/1196232291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 msgr2=0x558b130b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.404+0000 7fc84c608b80 1 --2- 172.21.15.19:0/1196232291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 0x558b130b3c30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc828000c00 tx=0x7fc828009310 comp rx=0 tx=0).stop 2024-05-09T02:21:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.404+0000 7fc84c608b80 1 -- 172.21.15.19:0/1196232291 shutdown_connections 2024-05-09T02:21:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.404+0000 7fc84c608b80 1 --2- 172.21.15.19:0/1196232291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 0x558b130b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.404+0000 7fc84c608b80 1 --2- 172.21.15.19:0/1196232291 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558b130e2d00 0x558b130e5160 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.404+0000 7fc84c608b80 1 --2- 172.21.15.19:0/1196232291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 0x558b130b3c30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.404+0000 7fc84c608b80 1 --2- 172.21.15.19:0/1196232291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b13014160 0x558b130b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.404+0000 7fc84c608b80 1 -- 172.21.15.19:0/1196232291 >> 172.21.15.19:0/1196232291 conn(0x558b12f8a6a0 msgr2=0x558b130a2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.404+0000 7fc816ffd700 0 client.26386 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.404+0000 7fc84c608b80 1 -- 172.21.15.19:0/1196232291 shutdown_connections 2024-05-09T02:21:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.404+0000 7fc84c608b80 1 -- 172.21.15.19:0/1196232291 wait complete. 2024-05-09T02:21:37.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.404+0000 7fc84c608b80 20 client.26386 trim_cache size 0 max 16384 2024-05-09T02:21:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.423+0000 7fc84c608b80 1 Processor -- start 2024-05-09T02:21:37.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.423+0000 7fc84c608b80 1 -- start start 2024-05-09T02:21:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.423+0000 7fc84c608b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b12f2a4a0 0x558b130aca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.423+0000 7fc84c608b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b13014160 0x558b130a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.423+0000 7fc84c608b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 0x558b130ae670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.424+0000 7fc836ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b13014160 0x558b130a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.424+0000 7fc836ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b13014160 0x558b130a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:52102/0 (socket says 172.21.15.19:52102) 2024-05-09T02:21:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.424+0000 7fc8377fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b12f2a4a0 0x558b130aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.424+0000 7fc836ffd700 1 -- 172.21.15.19:0/2469492611 learned_addr learned my addr 172.21.15.19:0/2469492611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.424+0000 7fc837fff700 1 --2- 172.21.15.19:0/2469492611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 0x558b130ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.424+0000 7fc84c608b80 1 -- 172.21.15.19:0/2469492611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc83811dc20 con 0x558b130a76a0 2024-05-09T02:21:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.424+0000 7fc84c608b80 1 -- 172.21.15.19:0/2469492611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc83806db60 con 0x558b12f2a4a0 2024-05-09T02:21:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.424+0000 7fc84c608b80 1 -- 172.21.15.19:0/2469492611 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc83811e2f0 con 0x558b13014160 2024-05-09T02:21:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.425+0000 7fc837fff700 1 -- 172.21.15.19:0/2469492611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b13014160 msgr2=0x558b130a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.425+0000 7fc837fff700 1 --2- 172.21.15.19:0/2469492611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b13014160 0x558b130a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.425+0000 7fc837fff700 1 -- 172.21.15.19:0/2469492611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b12f2a4a0 msgr2=0x558b130aca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.425+0000 7fc837fff700 1 --2- 172.21.15.19:0/2469492611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b12f2a4a0 0x558b130aca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.425+0000 7fc837fff700 1 -- 172.21.15.19:0/2469492611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b130b2070 con 0x558b130a76a0 2024-05-09T02:21:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.425+0000 7fc8377fe700 1 --2- 172.21.15.19:0/2469492611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b12f2a4a0 0x558b130aca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:21:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.425+0000 7fc836ffd700 1 --2- 172.21.15.19:0/2469492611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b13014160 0x558b130a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:21:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.425+0000 7fc837fff700 1 --2- 172.21.15.19:0/2469492611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 0x558b130ae670 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc83000afd0 tx=0x7fc830012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff8e3937a69a8f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.426+0000 7fc81d7fa700 1 -- 172.21.15.19:0/2469492611 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830028070 con 0x558b130a76a0 2024-05-09T02:21:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.426+0000 7fc81d7fa700 1 -- 172.21.15.19:0/2469492611 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc830023aa0 con 0x558b130a76a0 2024-05-09T02:21:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.426+0000 7fc81d7fa700 1 -- 172.21.15.19:0/2469492611 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830017ae0 con 0x558b130a76a0 2024-05-09T02:21:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.427+0000 7fc84c608b80 1 -- 172.21.15.19:0/2469492611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 msgr2=0x558b130ae670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.427+0000 7fc84c608b80 1 --2- 172.21.15.19:0/2469492611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 0x558b130ae670 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc83000afd0 tx=0x7fc830012000 comp rx=0 tx=0).stop 2024-05-09T02:21:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.427+0000 7fc84c608b80 1 -- 172.21.15.19:0/2469492611 shutdown_connections 2024-05-09T02:21:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.427+0000 7fc84c608b80 1 --2- 172.21.15.19:0/2469492611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 0x558b130ae670 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.427+0000 7fc84c608b80 1 --2- 172.21.15.19:0/2469492611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b12f2a4a0 0x558b130aca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.428+0000 7fc84c608b80 1 --2- 172.21.15.19:0/2469492611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b13014160 0x558b130a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.428+0000 7fc84c608b80 1 -- 172.21.15.19:0/2469492611 >> 172.21.15.19:0/2469492611 conn(0x558b12f8a6a0 msgr2=0x558b130ac310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.428+0000 7fc84c608b80 1 -- 172.21.15.19:0/2469492611 shutdown_connections 2024-05-09T02:21:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.428+0000 7fc84c608b80 1 -- 172.21.15.19:0/2469492611 wait complete. 2024-05-09T02:21:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.428+0000 7fc84c608b80 1 -- start start 2024-05-09T02:21:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.429+0000 7fc84c608b80 1 Processor -- start 2024-05-09T02:21:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.429+0000 7fc84c608b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 0x558b130ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.429+0000 7fc84c608b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b13014160 0x558b130ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.429+0000 7fc84c608b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 0x558b130ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.429+0000 7fc84c608b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc83806d460 con 0x558b130a76a0 2024-05-09T02:21:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.429+0000 7fc84c608b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc83811e9c0 con 0x558b13014160 2024-05-09T02:21:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.429+0000 7fc84c608b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc83811bbe0 con 0x558b12f2a4a0 2024-05-09T02:21:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.429+0000 7fc84c608b80 10 client.0 _ll_register_callbacks cb 0x558b12e3b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.430+0000 7fc8377fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 0x558b130ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.430+0000 7fc837fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 0x558b130ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.430+0000 7fc8377fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 0x558b130ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:52106/0 (socket says 172.21.15.19:52106) 2024-05-09T02:21:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.430+0000 7fc8377fe700 1 -- 172.21.15.19:0/1414824554 learned_addr learned my addr 172.21.15.19:0/1414824554 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.430+0000 7fc836ffd700 1 --2- 172.21.15.19:0/1414824554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b13014160 0x558b130ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.430+0000 7fc8377fe700 1 -- 172.21.15.19:0/1414824554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b13014160 msgr2=0x558b130ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.430+0000 7fc8377fe700 1 --2- 172.21.15.19:0/1414824554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b13014160 0x558b130ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.430+0000 7fc8377fe700 1 -- 172.21.15.19:0/1414824554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 msgr2=0x558b130ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.430+0000 7fc8377fe700 1 --2- 172.21.15.19:0/1414824554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 0x558b130ad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.430+0000 7fc8377fe700 1 -- 172.21.15.19:0/1414824554 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b130aeea0 con 0x558b12f2a4a0 2024-05-09T02:21:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.430+0000 7fc837fff700 1 --2- 172.21.15.19:0/1414824554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 0x558b130ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:21:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.430+0000 7fc836ffd700 1 --2- 172.21.15.19:0/1414824554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b13014160 0x558b130ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:21:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.430+0000 7fc8377fe700 1 --2- 172.21.15.19:0/1414824554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 0x558b130ca360 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc828009990 tx=0x7fc82802a490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2350a6f87afcce6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.430+0000 7fc816ffd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:21:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.431+0000 7fc816ffd700 1 -- 172.21.15.19:0/1414824554 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82803c020 con 0x558b12f2a4a0 2024-05-09T02:21:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.431+0000 7fc84c608b80 1 -- 172.21.15.19:0/1414824554 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc830023aa0 con 0x558b12f2a4a0 2024-05-09T02:21:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.431+0000 7fc816ffd700 1 -- 172.21.15.19:0/1414824554 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fc82802a8c0 con 0x558b12f2a4a0 2024-05-09T02:21:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.431+0000 7fc816ffd700 1 -- 172.21.15.19:0/1414824554 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fc828047050 con 0x558b12f2a4a0 2024-05-09T02:21:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.431+0000 7fc84c608b80 10 client.26392 fetch_fsmap learned FSMap version 55 2024-05-09T02:21:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.431+0000 7fc84c608b80 1 -- 172.21.15.19:0/1414824554 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558b1314c900 con 0x558b12f2a4a0 2024-05-09T02:21:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.431+0000 7fc816ffd700 1 -- 172.21.15.19:0/1414824554 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc830023aa0 con 0x558b12f2a4a0 2024-05-09T02:21:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.431+0000 7fc816ffd700 1 -- 172.21.15.19:0/1414824554 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 55) ==== 8374+0+0 (secure 0 0 0) 0x7fc828034570 con 0x558b12f2a4a0 2024-05-09T02:21:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.431+0000 7fc84c608b80 10 client.26392 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T02:21:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.431+0000 7fc84c608b80 10 client.26392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:21:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.431+0000 7fc84c608b80 20 client.26392 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.431+0000 7fc84c608b80 1 --2- 172.21.15.19:0/1414824554 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558b1314d060 0x558b1314f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.431+0000 7fc84c608b80 4 client.26392 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:21:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.431+0000 7fc84c608b80 1 -- 172.21.15.19:0/1414824554 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b12fd1a70 con 0x558b1314d060 2024-05-09T02:21:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.432+0000 7fc836ffd700 1 --2- 172.21.15.19:0/1414824554 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558b1314d060 0x558b1314f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.432+0000 7fc816ffd700 10 client.26392 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.432+0000 7fc836ffd700 1 --2- 172.21.15.19:0/1414824554 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558b1314d060 0x558b1314f480 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f1896ad1237becc server_cookie=57fd673d9b15b121 in_seq=0 out_seq=0 2024-05-09T02:21:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.432+0000 7fc816ffd700 1 -- 172.21.15.19:0/1414824554 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b12fd1a70 con 0x558b1314d060 2024-05-09T02:21:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.432+0000 7fc816ffd700 10 client.26392 handle_command_reply: tid=0 2024-05-09T02:21:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.432+0000 7fc84c608b80 1 client.26392 shutdown 2024-05-09T02:21:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.433+0000 7fc84c608b80 1 -- 172.21.15.19:0/1414824554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 msgr2=0x558b130ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.433+0000 7fc84c608b80 1 --2- 172.21.15.19:0/1414824554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 0x558b130ca360 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc828009990 tx=0x7fc82802a490 comp rx=0 tx=0).stop 2024-05-09T02:21:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.433+0000 7fc84c608b80 1 -- 172.21.15.19:0/1414824554 shutdown_connections 2024-05-09T02:21:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.433+0000 7fc84c608b80 1 --2- 172.21.15.19:0/1414824554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b130a76a0 0x558b130ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.433+0000 7fc84c608b80 1 --2- 172.21.15.19:0/1414824554 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558b1314d060 0x558b1314f480 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.433+0000 7fc84c608b80 1 --2- 172.21.15.19:0/1414824554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b12f2a4a0 0x558b130ca360 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.433+0000 7fc84c608b80 1 --2- 172.21.15.19:0/1414824554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b13014160 0x558b130ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.433+0000 7fc84c608b80 1 -- 172.21.15.19:0/1414824554 >> 172.21.15.19:0/1414824554 conn(0x558b12f8a6a0 msgr2=0x558b130af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.433+0000 7fc816ffd700 0 client.26392 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.434+0000 7fc84c608b80 1 -- 172.21.15.19:0/1414824554 shutdown_connections 2024-05-09T02:21:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.434+0000 7fc84c608b80 1 -- 172.21.15.19:0/1414824554 wait complete. 2024-05-09T02:21:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.434+0000 7fc84c608b80 20 client.26392 trim_cache size 0 max 16384 2024-05-09T02:21:37.439 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:21:37.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.435+0000 7fc81e7fc700 1 -- 172.21.15.19:0/1572249730 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc8200b2a20 msgr2=0x7fc8200b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:37.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.435+0000 7fc81e7fc700 1 --2- 172.21.15.19:0/1572249730 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc8200b2a20 0x7fc8200b4f30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc83000a4c0 tx=0x7fc830009650 comp rx=0 tx=0).stop 2024-05-09T02:21:37.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.435+0000 7fc81e7fc700 1 -- 172.21.15.19:0/1572249730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc838107df0 msgr2=0x7fc83810fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:37.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.435+0000 7fc81e7fc700 1 --2- 172.21.15.19:0/1572249730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc838107df0 0x7fc83810fbb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00b580 tx=0x7fc82c00b890 comp rx=0 tx=0).stop 2024-05-09T02:21:37.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.435+0000 7fc81e7fc700 1 -- 172.21.15.19:0/1572249730 shutdown_connections 2024-05-09T02:21:37.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.435+0000 7fc81e7fc700 1 --2- 172.21.15.19:0/1572249730 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc8200b2a20 0x7fc8200b4f30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.435+0000 7fc81e7fc700 1 --2- 172.21.15.19:0/1572249730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc838108740 0x7fc8381100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.435+0000 7fc81e7fc700 1 --2- 172.21.15.19:0/1572249730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8381034c0 0x7fc838111960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.435+0000 7fc81e7fc700 1 --2- 172.21.15.19:0/1572249730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc838107df0 0x7fc83810fbb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.435+0000 7fc81e7fc700 1 -- 172.21.15.19:0/1572249730 >> 172.21.15.19:0/1572249730 conn(0x7fc8380fedc0 msgr2=0x7fc838107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.435+0000 7fc81e7fc700 1 -- 172.21.15.19:0/1572249730 shutdown_connections 2024-05-09T02:21:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:37.435+0000 7fc81e7fc700 1 -- 172.21.15.19:0/1572249730 wait complete. 2024-05-09T02:21:37.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 595.0 s, 1098.6 tps, lat 91.557 ms stddev 155.330 2024-05-09T02:21:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:38 smithi196 ceph-mon[34718]: pgmap v321: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.8 MiB/s wr, 523 op/s 2024-05-09T02:21:38.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:38 smithi019 ceph-mon[30151]: pgmap v321: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.8 MiB/s wr, 523 op/s 2024-05-09T02:21:38.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:38 smithi059 ceph-mon[34781]: pgmap v321: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.8 MiB/s wr, 523 op/s 2024-05-09T02:21:39.219 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T02:21:39.219 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T02:21:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:21:39.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:39 smithi059 ceph-mon[34781]: pgmap v322: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 1.44k op/s 2024-05-09T02:21:40.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:39 smithi196 ceph-mon[34718]: pgmap v322: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 1.44k op/s 2024-05-09T02:21:40.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:39 smithi019 ceph-mon[30151]: pgmap v322: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 1.44k op/s 2024-05-09T02:21:42.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:42 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:21:42.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:42 smithi019 ceph-mon[30151]: pgmap v323: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 1.59k op/s 2024-05-09T02:21:42.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:42 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:21:42.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:42 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:21:42.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:42 smithi059 ceph-mon[34781]: pgmap v323: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 1.59k op/s 2024-05-09T02:21:42.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:42 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:21:42.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:42 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:21:42.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:42 smithi196 ceph-mon[34718]: pgmap v323: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 1.59k op/s 2024-05-09T02:21:42.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:42 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:21:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 600.0 s, 1324.0 tps, lat 75.158 ms stddev 85.429 2024-05-09T02:21:42.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.936+0000 7f7730acd700 1 -- 172.21.15.19:0/2915674085 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c003c70 con 0x7f772c10b3a0 2024-05-09T02:21:42.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.937+0000 7f7732ad1700 1 -- 172.21.15.19:0/2915674085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772c10b3a0 msgr2=0x7f772c116820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:42.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.937+0000 7f7732ad1700 1 --2- 172.21.15.19:0/2915674085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772c10b3a0 0x7f772c116820 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f771c009990 tx=0x7f771c009ca0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.937+0000 7f7732ad1700 1 -- 172.21.15.19:0/2915674085 shutdown_connections 2024-05-09T02:21:42.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.937+0000 7f7732ad1700 1 --2- 172.21.15.19:0/2915674085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f772c116eb0 0x7f772c1193c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.937+0000 7f7732ad1700 1 --2- 172.21.15.19:0/2915674085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f772c10aa50 0x7f772c10ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.937+0000 7f7732ad1700 1 --2- 172.21.15.19:0/2915674085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772c10b3a0 0x7f772c116820 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.937+0000 7f7732ad1700 1 -- 172.21.15.19:0/2915674085 >> 172.21.15.19:0/2915674085 conn(0x7f772c06f290 msgr2=0x7f772c06f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:42.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.937+0000 7f7732ad1700 1 -- 172.21.15.19:0/2915674085 shutdown_connections 2024-05-09T02:21:42.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.937+0000 7f7732ad1700 1 -- 172.21.15.19:0/2915674085 wait complete. 2024-05-09T02:21:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.938+0000 7f7732ad1700 1 Processor -- start 2024-05-09T02:21:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.938+0000 7f7732ad1700 1 -- start start 2024-05-09T02:21:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.939+0000 7f7732ad1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f772c10aa50 0x7f772c113c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.939+0000 7f7732ad1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772c10b3a0 0x7f772c114180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:42.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.939+0000 7f7731acf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f772c10aa50 0x7f772c113c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:42.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.940+0000 7f7731acf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f772c10aa50 0x7f772c113c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:58166/0 (socket says 172.21.15.19:58166) 2024-05-09T02:21:42.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.940+0000 7f7731acf700 1 -- 172.21.15.19:0/1790269475 learned_addr learned my addr 172.21.15.19:0/1790269475 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:42.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.940+0000 7f7732ad1700 1 --2- 172.21.15.19:0/1790269475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f772c116eb0 0x7f772c1146c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:42.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.940+0000 7f7732ad1700 1 -- 172.21.15.19:0/1790269475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f772c11ed70 con 0x7f772c10b3a0 2024-05-09T02:21:42.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.940+0000 7f77322d0700 1 --2- 172.21.15.19:0/1790269475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f772c116eb0 0x7f772c1146c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:42.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.940+0000 7f7732ad1700 1 -- 172.21.15.19:0/1790269475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f772c11ebf0 con 0x7f772c10aa50 2024-05-09T02:21:42.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.940+0000 7f7732ad1700 1 -- 172.21.15.19:0/1790269475 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f772c11eef0 con 0x7f772c116eb0 2024-05-09T02:21:42.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.941+0000 7f77312ce700 1 --2- 172.21.15.19:0/1790269475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772c10b3a0 0x7f772c114180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.942+0000 7f7731acf700 1 -- 172.21.15.19:0/1790269475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f772c116eb0 msgr2=0x7f772c1146c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.942+0000 7f7731acf700 1 --2- 172.21.15.19:0/1790269475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f772c116eb0 0x7f772c1146c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.942+0000 7f7731acf700 1 -- 172.21.15.19:0/1790269475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772c10b3a0 msgr2=0x7f772c114180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.942+0000 7f7731acf700 1 --2- 172.21.15.19:0/1790269475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772c10b3a0 0x7f772c114180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.942+0000 7f7731acf700 1 -- 172.21.15.19:0/1790269475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772c1ab380 con 0x7f772c10aa50 2024-05-09T02:21:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.942+0000 7f7731acf700 1 --2- 172.21.15.19:0/1790269475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f772c10aa50 0x7f772c113c40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f772800d950 tx=0x7f772800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.943+0000 7f7722ffd700 1 -- 172.21.15.19:0/1790269475 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7728016070 con 0x7f772c10aa50 2024-05-09T02:21:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.943+0000 7f7722ffd700 1 -- 172.21.15.19:0/1790269475 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7728009df0 con 0x7f772c10aa50 2024-05-09T02:21:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.943+0000 7f7722ffd700 1 -- 172.21.15.19:0/1790269475 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772800f3c0 con 0x7f772c10aa50 2024-05-09T02:21:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.943+0000 7f7732ad1700 1 -- 172.21.15.19:0/1790269475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772c1ab670 con 0x7f772c10aa50 2024-05-09T02:21:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.943+0000 7f7732ad1700 1 -- 172.21.15.19:0/1790269475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f772c1abbb0 con 0x7f772c10aa50 2024-05-09T02:21:42.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.949+0000 7f7722ffd700 1 -- 172.21.15.19:0/1790269475 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7728019070 con 0x7f772c10aa50 2024-05-09T02:21:42.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.955+0000 7f7742749b80 1 Processor -- start 2024-05-09T02:21:42.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.956+0000 7f7742749b80 1 -- start start 2024-05-09T02:21:42.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.956+0000 7f7722ffd700 1 --2- 172.21.15.19:0/1790269475 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f77180b2b40 0x7f77180b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.956+0000 7f7742749b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6091e1e0 0x558a609b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.956+0000 7f7742749b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a60834520 0x558a609b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.956+0000 7f7742749b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a609b1880 0x558a609b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.956+0000 7f7722ffd700 1 -- 172.21.15.19:0/1790269475 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f77280e21e0 con 0x7f772c10aa50 2024-05-09T02:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.956+0000 7f7742749b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a608d4070 con 0x558a60834520 2024-05-09T02:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.956+0000 7f7742749b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558a60872870 con 0x558a609b1880 2024-05-09T02:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.956+0000 7f7742749b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558a608d0030 con 0x558a6091e1e0 2024-05-09T02:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.957+0000 7f7731acf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6091e1e0 0x558a609b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.957+0000 7f7731acf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6091e1e0 0x558a609b76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:52130/0 (socket says 172.21.15.19:52130) 2024-05-09T02:21:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.957+0000 7f7731acf700 1 -- 172.21.15.19:0/2700358521 learned_addr learned my addr 172.21.15.19:0/2700358521 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.957+0000 7f77312ce700 1 --2- 172.21.15.19:0/2700358521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a60834520 0x558a609b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.957+0000 7f7731acf700 1 -- 172.21.15.19:0/2700358521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a609b1880 msgr2=0x558a609b1c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:21:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.957+0000 7f7731acf700 1 --2- 172.21.15.19:0/2700358521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a609b1880 0x558a609b1c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.957+0000 7f7731acf700 1 -- 172.21.15.19:0/2700358521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a60834520 msgr2=0x558a609b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.957+0000 7f7731acf700 1 --2- 172.21.15.19:0/2700358521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a60834520 0x558a609b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.957+0000 7f7731acf700 1 -- 172.21.15.19:0/2700358521 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a6086a3f0 con 0x558a6091e1e0 2024-05-09T02:21:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.957+0000 7f7731acf700 1 --2- 172.21.15.19:0/2700358521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6091e1e0 0x558a609b76c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f77280a54b0 tx=0x7f77280a9bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18f896acecf67046 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.958+0000 7f77322d0700 1 --2- 172.21.15.19:0/1790269475 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f77180b2b40 0x7f77180b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.958+0000 7f7717fff700 1 -- 172.21.15.19:0/2700358521 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7728020040 con 0x558a6091e1e0 2024-05-09T02:21:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.958+0000 7f7717fff700 1 -- 172.21.15.19:0/2700358521 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77280ad7e0 con 0x558a6091e1e0 2024-05-09T02:21:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.959+0000 7f7742749b80 1 -- 172.21.15.19:0/2700358521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6091e1e0 msgr2=0x558a609b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.959+0000 7f7742749b80 1 --2- 172.21.15.19:0/2700358521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6091e1e0 0x558a609b76c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f77280a54b0 tx=0x7f77280a9bb0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.959+0000 7f77322d0700 1 --2- 172.21.15.19:0/1790269475 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f77180b2b40 0x7f77180b5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7724009f60 tx=0x7f772400d040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.960+0000 7f7742749b80 1 -- 172.21.15.19:0/2700358521 shutdown_connections 2024-05-09T02:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.960+0000 7f7742749b80 1 --2- 172.21.15.19:0/2700358521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a609b1880 0x558a609b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.960+0000 7f7742749b80 1 --2- 172.21.15.19:0/2700358521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6091e1e0 0x558a609b76c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.960+0000 7f7742749b80 1 --2- 172.21.15.19:0/2700358521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a60834520 0x558a609b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.960+0000 7f7742749b80 1 -- 172.21.15.19:0/2700358521 >> 172.21.15.19:0/2700358521 conn(0x558a60894720 msgr2=0x558a609ac6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.960+0000 7f7742749b80 1 -- 172.21.15.19:0/2700358521 shutdown_connections 2024-05-09T02:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.960+0000 7f7742749b80 1 -- 172.21.15.19:0/2700358521 wait complete. 2024-05-09T02:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.961+0000 7f7742749b80 1 -- start start 2024-05-09T02:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.961+0000 7f7742749b80 1 Processor -- start 2024-05-09T02:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.962+0000 7f7742749b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a60834520 0x558a609bb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.962+0000 7f7742749b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a6091e1e0 0x558a609bb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.962+0000 7f7742749b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a609b1880 0x558a609bbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.962+0000 7f7742749b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a6074fc20 con 0x558a6091e1e0 2024-05-09T02:21:42.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.962+0000 7f7742749b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558a6088e1d0 con 0x558a609b1880 2024-05-09T02:21:42.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.962+0000 7f7742749b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558a609b2650 con 0x558a60834520 2024-05-09T02:21:42.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.962+0000 7f77312ce700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a6091e1e0 0x558a609bb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:42.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.962+0000 7f77312ce700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a6091e1e0 0x558a609bb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33302/0 (socket says 172.21.15.19:33302) 2024-05-09T02:21:42.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.962+0000 7f77312ce700 1 -- 172.21.15.19:0/1554421453 learned_addr learned my addr 172.21.15.19:0/1554421453 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:42.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.962+0000 7f7742749b80 10 client.0 _ll_register_callbacks cb 0x558a60745740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:42.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.962+0000 7f7731acf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a60834520 0x558a609bb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:42.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.962+0000 7f77322d0700 1 --2- 172.21.15.19:0/1554421453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a609b1880 0x558a609bbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:42.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.963+0000 7f77322d0700 1 -- 172.21.15.19:0/1554421453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a60834520 msgr2=0x558a609bb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:42.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.963+0000 7f77322d0700 1 --2- 172.21.15.19:0/1554421453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a60834520 0x558a609bb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.963+0000 7f77322d0700 1 -- 172.21.15.19:0/1554421453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a6091e1e0 msgr2=0x558a609bb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.963+0000 7f77322d0700 1 --2- 172.21.15.19:0/1554421453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a6091e1e0 0x558a609bb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.963+0000 7f77322d0700 1 -- 172.21.15.19:0/1554421453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a6074f510 con 0x558a609b1880 2024-05-09T02:21:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.963+0000 7f7742749b80 1 -- 172.21.15.19:0/1554421453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77280ad7e0 con 0x558a609b1880 2024-05-09T02:21:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.963+0000 7f77322d0700 1 --2- 172.21.15.19:0/1554421453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a609b1880 0x558a609bbe60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f772400ab20 tx=0x7f7724004b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce63e8b748fa8fbd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.964+0000 7f77157fa700 10 client.26363 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:21:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.964+0000 7f77157fa700 1 -- 172.21.15.19:0/1554421453 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724013750 con 0x558a609b1880 2024-05-09T02:21:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.964+0000 7f77157fa700 1 -- 172.21.15.19:0/1554421453 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f77240138f0 con 0x558a609b1880 2024-05-09T02:21:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.964+0000 7f77157fa700 1 -- 172.21.15.19:0/1554421453 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f7724013af0 con 0x558a609b1880 2024-05-09T02:21:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.965+0000 7f7742749b80 10 client.26363 fetch_fsmap learned FSMap version 56 2024-05-09T02:21:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.965+0000 7f7742749b80 1 -- 172.21.15.19:0/1554421453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=56}) -- 0x558a609c8a70 con 0x558a609b1880 2024-05-09T02:21:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.965+0000 7f77157fa700 1 -- 172.21.15.19:0/1554421453 <== mon.2 v2:172.21.15.59:3300/0 4 ==== fsmap(e 56) ==== 8362+0+0 (secure 0 0 0) 0x7f7724013c90 con 0x558a609b1880 2024-05-09T02:21:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.965+0000 7f7742749b80 10 client.26363 fetch_fsmap finished waiting for FSMap version 56 2024-05-09T02:21:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.965+0000 7f7742749b80 10 client.26363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:21:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.965+0000 7f7742749b80 20 client.26363 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:42.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.965+0000 7f7742749b80 1 --2- 172.21.15.19:0/1554421453 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558a609cb0c0 0x558a609cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:42.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.965+0000 7f7742749b80 4 client.26363 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:21:42.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.965+0000 7f7742749b80 1 -- 172.21.15.19:0/1554421453 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a608e1a50 con 0x558a609cb0c0 2024-05-09T02:21:42.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.965+0000 7f7731acf700 1 --2- 172.21.15.19:0/1554421453 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558a609cb0c0 0x558a609cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:42.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.966+0000 7f77157fa700 10 client.26363 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:42.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.966+0000 7f7731acf700 1 --2- 172.21.15.19:0/1554421453 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558a609cb0c0 0x558a609cd550 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba0e2dd2098e80bd server_cookie=780bff5b57256655 in_seq=0 out_seq=0 2024-05-09T02:21:42.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.967+0000 7f77157fa700 1 -- 172.21.15.19:0/1554421453 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a608e1a50 con 0x558a609cb0c0 2024-05-09T02:21:42.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.967+0000 7f77157fa700 10 client.26363 handle_command_reply: tid=0 2024-05-09T02:21:42.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.967+0000 7f7742749b80 1 client.26363 shutdown 2024-05-09T02:21:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.968+0000 7f7742749b80 1 -- 172.21.15.19:0/1554421453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a609b1880 msgr2=0x558a609bbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.968+0000 7f7742749b80 1 --2- 172.21.15.19:0/1554421453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a609b1880 0x558a609bbe60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f772400ab20 tx=0x7f7724004b60 comp rx=0 tx=0).stop 2024-05-09T02:21:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.968+0000 7f7742749b80 1 -- 172.21.15.19:0/1554421453 shutdown_connections 2024-05-09T02:21:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.968+0000 7f7742749b80 1 --2- 172.21.15.19:0/1554421453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a609b1880 0x558a609bbe60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.968+0000 7f7742749b80 1 --2- 172.21.15.19:0/1554421453 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558a609cb0c0 0x558a609cd550 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.968+0000 7f7742749b80 1 --2- 172.21.15.19:0/1554421453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a60834520 0x558a609bb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.968+0000 7f77157fa700 0 client.26363 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.968+0000 7f7742749b80 1 --2- 172.21.15.19:0/1554421453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a6091e1e0 0x558a609bb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.968+0000 7f7742749b80 1 -- 172.21.15.19:0/1554421453 >> 172.21.15.19:0/1554421453 conn(0x558a60894720 msgr2=0x558a609ac400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.968+0000 7f7742749b80 1 -- 172.21.15.19:0/1554421453 shutdown_connections 2024-05-09T02:21:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.969+0000 7f7742749b80 1 -- 172.21.15.19:0/1554421453 wait complete. 2024-05-09T02:21:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:42.969+0000 7f7742749b80 20 client.26363 trim_cache size 0 max 16384 2024-05-09T02:21:43.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.023+0000 7f7742749b80 1 Processor -- start 2024-05-09T02:21:43.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.023+0000 7f7742749b80 1 -- start start 2024-05-09T02:21:43.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.024+0000 7f7742749b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a60834520 0x558a609b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.024+0000 7f7742749b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a6091e1e0 0x558a609bd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.024+0000 7f7742749b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a609b1880 0x558a609ba3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.024+0000 7f7731acf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a60834520 0x558a609b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.024+0000 7f7731acf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a60834520 0x558a609b6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33316/0 (socket says 172.21.15.19:33316) 2024-05-09T02:21:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.025+0000 7f7731acf700 1 -- 172.21.15.19:0/1045517170 learned_addr learned my addr 172.21.15.19:0/1045517170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.025+0000 7f7742749b80 1 -- 172.21.15.19:0/1045517170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f772c1c64f0 con 0x558a60834520 2024-05-09T02:21:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.025+0000 7f7742749b80 1 -- 172.21.15.19:0/1045517170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f772c1c8090 con 0x558a6091e1e0 2024-05-09T02:21:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.025+0000 7f7742749b80 1 -- 172.21.15.19:0/1045517170 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f772c1c6bc0 con 0x558a609b1880 2024-05-09T02:21:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.025+0000 7f77312ce700 1 --2- 172.21.15.19:0/1045517170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a6091e1e0 0x558a609bd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.025+0000 7f77322d0700 1 --2- 172.21.15.19:0/1045517170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a609b1880 0x558a609ba3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:43.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.025+0000 7f77312ce700 1 -- 172.21.15.19:0/1045517170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a609b1880 msgr2=0x558a609ba3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:43.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.025+0000 7f77312ce700 1 --2- 172.21.15.19:0/1045517170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a609b1880 0x558a609ba3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.025+0000 7f77312ce700 1 -- 172.21.15.19:0/1045517170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a60834520 msgr2=0x558a609b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:43.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.025+0000 7f77312ce700 1 --2- 172.21.15.19:0/1045517170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a60834520 0x558a609b6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.025+0000 7f77312ce700 1 -- 172.21.15.19:0/1045517170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a609b7d00 con 0x558a6091e1e0 2024-05-09T02:21:43.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.025+0000 7f77312ce700 1 --2- 172.21.15.19:0/1045517170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a6091e1e0 0x558a609bd880 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x558a609ad7a0 tx=0x7f771c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6890b5a546c9a432 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:43.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.026+0000 7f77177fe700 1 -- 172.21.15.19:0/1045517170 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c003af0 con 0x558a6091e1e0 2024-05-09T02:21:43.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.026+0000 7f77177fe700 1 -- 172.21.15.19:0/1045517170 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f771c029070 con 0x558a6091e1e0 2024-05-09T02:21:43.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.026+0000 7f77177fe700 1 -- 172.21.15.19:0/1045517170 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c00bc70 con 0x558a6091e1e0 2024-05-09T02:21:43.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.027+0000 7f7742749b80 1 -- 172.21.15.19:0/1045517170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a6091e1e0 msgr2=0x558a609bd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:43.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.027+0000 7f7742749b80 1 --2- 172.21.15.19:0/1045517170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a6091e1e0 0x558a609bd880 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x558a609ad7a0 tx=0x7f771c0095c0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.027+0000 7f7742749b80 1 -- 172.21.15.19:0/1045517170 shutdown_connections 2024-05-09T02:21:43.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.027+0000 7f7742749b80 1 --2- 172.21.15.19:0/1045517170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a609b1880 0x558a609ba3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.027+0000 7f7742749b80 1 --2- 172.21.15.19:0/1045517170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a60834520 0x558a609b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.027+0000 7f7742749b80 1 --2- 172.21.15.19:0/1045517170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a6091e1e0 0x558a609bd880 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.028+0000 7f7742749b80 1 -- 172.21.15.19:0/1045517170 >> 172.21.15.19:0/1045517170 conn(0x558a60894720 msgr2=0x558a609b9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.028+0000 7f7742749b80 1 -- 172.21.15.19:0/1045517170 shutdown_connections 2024-05-09T02:21:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.028+0000 7f7742749b80 1 -- 172.21.15.19:0/1045517170 wait complete. 2024-05-09T02:21:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.029+0000 7f7742749b80 1 -- start start 2024-05-09T02:21:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.029+0000 7f7742749b80 1 Processor -- start 2024-05-09T02:21:43.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.030+0000 7f7742749b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a60834520 0x558a609c3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:43.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.030+0000 7f7742749b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a6091e1e0 0x558a609c0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:43.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.030+0000 7f7742749b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a609b1880 0x558a609c0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:43.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.030+0000 7f7742749b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f772c1c7990 con 0x558a609b1880 2024-05-09T02:21:43.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.030+0000 7f77312ce700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a6091e1e0 0x558a609c0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:43.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.030+0000 7f7742749b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f772c1c7290 con 0x558a6091e1e0 2024-05-09T02:21:43.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.030+0000 7f77312ce700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a6091e1e0 0x558a609c0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:58194/0 (socket says 172.21.15.19:58194) 2024-05-09T02:21:43.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.030+0000 7f77312ce700 1 -- 172.21.15.19:0/2738943902 learned_addr learned my addr 172.21.15.19:0/2738943902 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:43.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.030+0000 7f77322d0700 1 --2- 172.21.15.19:0/2738943902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a609b1880 0x558a609c0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:43.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.030+0000 7f7742749b80 1 -- 172.21.15.19:0/2738943902 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f772c1c44b0 con 0x558a60834520 2024-05-09T02:21:43.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.030+0000 7f7731acf700 1 --2- 172.21.15.19:0/2738943902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a60834520 0x558a609c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:43.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.031+0000 7f7742749b80 10 client.0 _ll_register_callbacks cb 0x558a60745740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.031+0000 7f77322d0700 1 -- 172.21.15.19:0/2738943902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a60834520 msgr2=0x558a609c3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.031+0000 7f77322d0700 1 --2- 172.21.15.19:0/2738943902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a60834520 0x558a609c3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.031+0000 7f77322d0700 1 -- 172.21.15.19:0/2738943902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a6091e1e0 msgr2=0x558a609c0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.031+0000 7f77322d0700 1 --2- 172.21.15.19:0/2738943902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a6091e1e0 0x558a609c0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.031+0000 7f77322d0700 1 -- 172.21.15.19:0/2738943902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a609c0fc0 con 0x558a609b1880 2024-05-09T02:21:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.031+0000 7f77312ce700 1 --2- 172.21.15.19:0/2738943902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a6091e1e0 0x558a609c0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:21:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.031+0000 7f77322d0700 1 --2- 172.21.15.19:0/2738943902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a609b1880 0x558a609c0a80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7724009d40 tx=0x7f7724013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6014310a1b18bbd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.031+0000 7f77157fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:21:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.032+0000 7f77157fa700 1 -- 172.21.15.19:0/2738943902 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724009330 con 0x558a609b1880 2024-05-09T02:21:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.032+0000 7f77157fa700 1 -- 172.21.15.19:0/2738943902 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f7724003cb0 con 0x558a609b1880 2024-05-09T02:21:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.032+0000 7f7742749b80 1 -- 172.21.15.19:0/2738943902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f771c029070 con 0x558a609b1880 2024-05-09T02:21:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.032+0000 7f77157fa700 1 -- 172.21.15.19:0/2738943902 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f77240206d0 con 0x558a609b1880 2024-05-09T02:21:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.032+0000 7f7742749b80 10 client.16641 fetch_fsmap learned FSMap version 56 2024-05-09T02:21:43.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.032+0000 7f7742749b80 1 -- 172.21.15.19:0/2738943902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x558a60a364d0 con 0x558a609b1880 2024-05-09T02:21:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.033+0000 7f77157fa700 1 -- 172.21.15.19:0/2738943902 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77280ad7e0 con 0x558a609b1880 2024-05-09T02:21:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.033+0000 7f77157fa700 1 -- 172.21.15.19:0/2738943902 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 56) ==== 8362+0+0 (secure 0 0 0) 0x7f7724004040 con 0x558a609b1880 2024-05-09T02:21:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.033+0000 7f7742749b80 10 client.16641 fetch_fsmap finished waiting for FSMap version 56 2024-05-09T02:21:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.033+0000 7f7742749b80 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:21:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.033+0000 7f7742749b80 20 client.16641 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.033+0000 7f7742749b80 1 --2- 172.21.15.19:0/2738943902 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558a609cb0c0 0x558a60a38b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.033+0000 7f7742749b80 4 client.16641 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T02:21:43.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.033+0000 7f7742749b80 1 -- 172.21.15.19:0/2738943902 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f771c029070 con 0x558a609cb0c0 2024-05-09T02:21:43.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.033+0000 7f7731acf700 1 --2- 172.21.15.19:0/2738943902 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558a609cb0c0 0x558a60a38b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:43.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.034+0000 7f77157fa700 10 client.16641 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:43.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.034+0000 7f7731acf700 1 --2- 172.21.15.19:0/2738943902 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558a609cb0c0 0x558a60a38b90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c53243c3b95dae2 server_cookie=b8a5e17242538d31 in_seq=0 out_seq=0 2024-05-09T02:21:43.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.035+0000 7f77157fa700 1 -- 172.21.15.19:0/2738943902 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f771c029070 con 0x558a609cb0c0 2024-05-09T02:21:43.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.035+0000 7f77157fa700 10 client.16641 handle_command_reply: tid=0 2024-05-09T02:21:43.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.035+0000 7f7742749b80 1 client.16641 shutdown 2024-05-09T02:21:43.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.035+0000 7f7742749b80 1 -- 172.21.15.19:0/2738943902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a609b1880 msgr2=0x558a609c0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:43.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.036+0000 7f7742749b80 1 --2- 172.21.15.19:0/2738943902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a609b1880 0x558a609c0a80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7724009d40 tx=0x7f7724013000 comp rx=0 tx=0).stop 2024-05-09T02:21:43.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.036+0000 7f7742749b80 1 -- 172.21.15.19:0/2738943902 shutdown_connections 2024-05-09T02:21:43.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.036+0000 7f7742749b80 1 --2- 172.21.15.19:0/2738943902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a609b1880 0x558a609c0a80 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.036+0000 7f7742749b80 1 --2- 172.21.15.19:0/2738943902 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x558a609cb0c0 0x558a60a38b90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.036+0000 7f7742749b80 1 --2- 172.21.15.19:0/2738943902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a60834520 0x558a609c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.036+0000 7f7742749b80 1 --2- 172.21.15.19:0/2738943902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a6091e1e0 0x558a609c0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.036+0000 7f7742749b80 1 -- 172.21.15.19:0/2738943902 >> 172.21.15.19:0/2738943902 conn(0x558a60894720 msgr2=0x558a609b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.036+0000 7f77157fa700 0 client.16641 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.036+0000 7f7742749b80 1 -- 172.21.15.19:0/2738943902 shutdown_connections 2024-05-09T02:21:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.036+0000 7f7742749b80 1 -- 172.21.15.19:0/2738943902 wait complete. 2024-05-09T02:21:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.036+0000 7f7742749b80 20 client.16641 trim_cache size 0 max 16384 2024-05-09T02:21:43.040 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:21:43.040 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T02:21:43.040 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "8748d7cd-f7ee-451f-8c1c-b1fc30b83cc1", 2024-05-09T02:21:43.040 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T02:21:43.040 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:21:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.037+0000 7f7732ad1700 1 -- 172.21.15.19:0/1790269475 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f77180b2b40 msgr2=0x7f77180b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.037+0000 7f7732ad1700 1 --2- 172.21.15.19:0/1790269475 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f77180b2b40 0x7f77180b5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7724009f60 tx=0x7f772400d040 comp rx=0 tx=0).stop 2024-05-09T02:21:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.037+0000 7f7732ad1700 1 -- 172.21.15.19:0/1790269475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f772c10aa50 msgr2=0x7f772c113c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.038+0000 7f7732ad1700 1 --2- 172.21.15.19:0/1790269475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f772c10aa50 0x7f772c113c40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f772800d950 tx=0x7f772800dc60 comp rx=0 tx=0).stop 2024-05-09T02:21:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.038+0000 7f7732ad1700 1 -- 172.21.15.19:0/1790269475 shutdown_connections 2024-05-09T02:21:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.038+0000 7f7732ad1700 1 --2- 172.21.15.19:0/1790269475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f772c116eb0 0x7f772c1146c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.038+0000 7f7732ad1700 1 --2- 172.21.15.19:0/1790269475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f772c10aa50 0x7f772c113c40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.038+0000 7f7732ad1700 1 --2- 172.21.15.19:0/1790269475 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f77180b2b40 0x7f77180b5050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.038+0000 7f7732ad1700 1 --2- 172.21.15.19:0/1790269475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772c10b3a0 0x7f772c114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.038+0000 7f7732ad1700 1 -- 172.21.15.19:0/1790269475 >> 172.21.15.19:0/1790269475 conn(0x7f772c06f290 msgr2=0x7f772c0724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.039+0000 7f7732ad1700 1 -- 172.21.15.19:0/1790269475 shutdown_connections 2024-05-09T02:21:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:43.039+0000 7f7732ad1700 1 -- 172.21.15.19:0/1790269475 wait complete. 2024-05-09T02:21:43.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:43 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:43 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:43 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:43 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:43 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:43 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:43 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:43.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:43 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:43.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:43 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:43 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:43 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:43 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:43 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:43 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:43 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:43.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:43 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:43.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:43 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:43 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:43 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:43 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:43 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:43 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:43.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:43 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:43.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:43 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:43.890 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:21:44.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub queued for path: / 2024-05-09T02:21:44.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:44.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:44.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:44.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:44.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:44.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:44.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:21:44.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.718 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:44 smithi059 ceph-mon[34781]: pgmap v324: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.52k op/s 2024-05-09T02:21:44.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub queued for path: / 2024-05-09T02:21:44.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:44.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:44.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:44.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:44.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:44.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:44.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:21:44.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.798 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:44 smithi196 ceph-mon[34718]: pgmap v324: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.52k op/s 2024-05-09T02:21:44.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub queued for path: / 2024-05-09T02:21:44.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:44.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:44.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:44.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:44.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:44.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:44.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:21:44.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:21:44.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:44 smithi019 ceph-mon[30151]: pgmap v324: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.52k op/s 2024-05-09T02:21:45.036 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:21:46.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:45 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:46.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:45 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:46.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:45 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:46.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:45 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:46.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:45 smithi059 ceph-mon[34781]: pgmap v325: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 1.76k op/s 2024-05-09T02:21:46.217 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:45 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.218 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:45 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.218 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:45 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.218 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:45 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.218 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:45 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.218 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:45 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.218 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:45 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.218 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:45 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:45 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:46.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:45 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:46.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:45 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:46.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:45 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:46.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:45 smithi196 ceph-mon[34718]: pgmap v325: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 1.76k op/s 2024-05-09T02:21:46.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:45 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:45 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:45 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.296 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:45 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.296 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:45 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.296 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:45 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.296 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:45 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.296 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:45 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:45 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:46.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:45 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:46.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:45 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:46.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:45 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:46.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:45 smithi019 ceph-mon[30151]: pgmap v325: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 1.76k op/s 2024-05-09T02:21:46.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:45 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:45 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:45 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:45 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:45 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:45 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:45 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:46.323 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:45 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:47.254 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:47 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi059 to 177.2M 2024-05-09T02:21:47.255 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:47 smithi019 ceph-mon[30151]: Unable to set osd_memory_target on smithi059 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-05-09T02:21:47.255 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:47 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-05-09T02:21:47.255 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:47 smithi019 ceph-mon[30151]: Unable to set osd_memory_target on smithi196 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-05-09T02:21:47.255 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:47 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:21:47.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:47 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi059 to 177.2M 2024-05-09T02:21:47.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:47 smithi059 ceph-mon[34781]: Unable to set osd_memory_target on smithi059 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-05-09T02:21:47.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:47 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-05-09T02:21:47.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:47 smithi059 ceph-mon[34781]: Unable to set osd_memory_target on smithi196 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-05-09T02:21:47.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:47 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:21:47.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:47 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi059 to 177.2M 2024-05-09T02:21:47.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:47 smithi196 ceph-mon[34718]: Unable to set osd_memory_target on smithi059 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-05-09T02:21:47.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:47 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-05-09T02:21:47.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:47 smithi196 ceph-mon[34718]: Unable to set osd_memory_target on smithi196 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-05-09T02:21:47.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:47 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:21:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 605.0 s, 1518.0 tps, lat 66.874 ms stddev 89.113 2024-05-09T02:21:47.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.702+0000 7f39d3b6a700 1 -- 172.21.15.19:0/2543496896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39cc10a930 msgr2=0x7f39cc10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:47.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.703+0000 7f39d3b6a700 1 --2- 172.21.15.19:0/2543496896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39cc10a930 0x7f39cc10ad40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009990 tx=0x7f39c8009ca0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.706+0000 7f39d3b6a700 1 -- 172.21.15.19:0/2543496896 shutdown_connections 2024-05-09T02:21:47.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.706+0000 7f39d3b6a700 1 --2- 172.21.15.19:0/2543496896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39cc11d930 0x7f39cc11fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.706+0000 7f39d3b6a700 1 --2- 172.21.15.19:0/2543496896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39cc10a930 0x7f39cc10ad40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.706+0000 7f39d3b6a700 1 --2- 172.21.15.19:0/2543496896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39cc10b280 0x7f39cc11d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.706+0000 7f39d3b6a700 1 -- 172.21.15.19:0/2543496896 >> 172.21.15.19:0/2543496896 conn(0x7f39cc06f4d0 msgr2=0x7f39cc06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.711+0000 7f39d3b6a700 1 -- 172.21.15.19:0/2543496896 shutdown_connections 2024-05-09T02:21:47.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.715+0000 7f39d3b6a700 1 -- 172.21.15.19:0/2543496896 wait complete. 2024-05-09T02:21:47.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.716+0000 7f39d3b6a700 1 Processor -- start 2024-05-09T02:21:47.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.716+0000 7f39d3b6a700 1 -- start start 2024-05-09T02:21:47.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.716+0000 7f39d3b6a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39cc10a930 0x7f39cc117570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.716+0000 7f39d3b6a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39cc10b280 0x7f39cc117ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.716+0000 7f39d3b6a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39cc11d930 0x7f39cc117ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.716+0000 7f39d3b6a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc1134d0 con 0x7f39cc10b280 2024-05-09T02:21:47.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.716+0000 7f39d3b6a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc113350 con 0x7f39cc11d930 2024-05-09T02:21:47.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.716+0000 7f39d3b6a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc113650 con 0x7f39cc10a930 2024-05-09T02:21:47.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.718+0000 7f39d2107700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39cc11d930 0x7f39cc117ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:47.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.720+0000 7f39d1906700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39cc10a930 0x7f39cc117570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:47.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.720+0000 7f39d1906700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39cc10a930 0x7f39cc117570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:42582/0 (socket says 172.21.15.19:42582) 2024-05-09T02:21:47.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.720+0000 7f39d1906700 1 -- 172.21.15.19:0/103280435 learned_addr learned my addr 172.21.15.19:0/103280435 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:47.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.723+0000 7f39d1105700 1 --2- 172.21.15.19:0/103280435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39cc10b280 0x7f39cc117ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:47.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.723+0000 7f39d1906700 1 -- 172.21.15.19:0/103280435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39cc11d930 msgr2=0x7f39cc117ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:47.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.723+0000 7f39d1906700 1 --2- 172.21.15.19:0/103280435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39cc11d930 0x7f39cc117ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.723+0000 7f39d1906700 1 -- 172.21.15.19:0/103280435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39cc10b280 msgr2=0x7f39cc117ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:47.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.723+0000 7f39d1906700 1 --2- 172.21.15.19:0/103280435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39cc10b280 0x7f39cc117ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.723+0000 7f39d1906700 1 -- 172.21.15.19:0/103280435 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39cc07ba20 con 0x7f39cc10a930 2024-05-09T02:21:47.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.724+0000 7f39d1906700 1 --2- 172.21.15.19:0/103280435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39cc10a930 0x7f39cc117570 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f39c80057f0 tx=0x7f39c80037b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:47.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.728+0000 7f39c2ffd700 1 -- 172.21.15.19:0/103280435 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8027070 con 0x7f39cc10a930 2024-05-09T02:21:47.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.728+0000 7f39c2ffd700 1 -- 172.21.15.19:0/103280435 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c8004080 con 0x7f39cc10a930 2024-05-09T02:21:47.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.728+0000 7f39c2ffd700 1 -- 172.21.15.19:0/103280435 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8022d40 con 0x7f39cc10a930 2024-05-09T02:21:47.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.728+0000 7f39d3b6a700 1 -- 172.21.15.19:0/103280435 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39cc07bcb0 con 0x7f39cc10a930 2024-05-09T02:21:47.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.729+0000 7f39d3b6a700 1 -- 172.21.15.19:0/103280435 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39cc07c140 con 0x7f39cc10a930 2024-05-09T02:21:47.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.738+0000 7f39e2580b80 1 Processor -- start 2024-05-09T02:21:47.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.738+0000 7f39c2ffd700 1 -- 172.21.15.19:0/103280435 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f39c8022470 con 0x7f39cc10a930 2024-05-09T02:21:47.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.740+0000 7f39e2580b80 1 -- start start 2024-05-09T02:21:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.740+0000 7f39e2580b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e89611160 0x560e896aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.740+0000 7f39e2580b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e895274a0 0x560e896aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.740+0000 7f39e2580b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 0x560e896a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.740+0000 7f39e2580b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560e895c6ff0 con 0x560e89611160 2024-05-09T02:21:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.740+0000 7f39e2580b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560e895657f0 con 0x560e896a46a0 2024-05-09T02:21:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.740+0000 7f39e2580b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560e895c2fb0 con 0x560e895274a0 2024-05-09T02:21:47.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.740+0000 7f39d1906700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e895274a0 0x560e896aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:47.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.740+0000 7f39d1105700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 0x560e896a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:47.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.740+0000 7f39d1906700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e895274a0 0x560e896aaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:42588/0 (socket says 172.21.15.19:42588) 2024-05-09T02:21:47.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.740+0000 7f39d1906700 1 -- 172.21.15.19:0/2811710154 learned_addr learned my addr 172.21.15.19:0/2811710154 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:47.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.741+0000 7f39c2ffd700 1 --2- 172.21.15.19:0/103280435 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f39b80b2b40 0x7f39b80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.741+0000 7f39d1105700 1 -- 172.21.15.19:0/2811710154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e895274a0 msgr2=0x560e896aaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:47.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.741+0000 7f39d1105700 1 --2- 172.21.15.19:0/2811710154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e895274a0 0x560e896aaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.741+0000 7f39d1105700 1 -- 172.21.15.19:0/2811710154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e89611160 msgr2=0x560e896aa4f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:21:47.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.741+0000 7f39d1105700 1 --2- 172.21.15.19:0/2811710154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e89611160 0x560e896aa4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.741+0000 7f39d1105700 1 -- 172.21.15.19:0/2811710154 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e8955d370 con 0x560e896a46a0 2024-05-09T02:21:47.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.741+0000 7f39d1105700 1 --2- 172.21.15.19:0/2811710154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 0x560e896a4ad0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00b7b0 tx=0x7f39bc00bac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4844b1ba8acf2023 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:47.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.741+0000 7f39c2ffd700 1 -- 172.21.15.19:0/103280435 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f39c80e38a0 con 0x7f39cc10a930 2024-05-09T02:21:47.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.742+0000 7f39b7fff700 1 -- 172.21.15.19:0/2811710154 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0125f0 con 0x560e896a46a0 2024-05-09T02:21:47.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.742+0000 7f39b7fff700 1 -- 172.21.15.19:0/2811710154 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39bc018070 con 0x560e896a46a0 2024-05-09T02:21:47.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.742+0000 7f39b7fff700 1 -- 172.21.15.19:0/2811710154 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc013ca0 con 0x560e896a46a0 2024-05-09T02:21:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.743+0000 7f39e2580b80 1 -- 172.21.15.19:0/2811710154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 msgr2=0x560e896a4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.743+0000 7f39e2580b80 1 --2- 172.21.15.19:0/2811710154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 0x560e896a4ad0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00b7b0 tx=0x7f39bc00bac0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.744+0000 7f39e2580b80 1 -- 172.21.15.19:0/2811710154 shutdown_connections 2024-05-09T02:21:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.744+0000 7f39e2580b80 1 --2- 172.21.15.19:0/2811710154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 0x560e896a4ad0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.744+0000 7f39e2580b80 1 --2- 172.21.15.19:0/2811710154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e89611160 0x560e896aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.744+0000 7f39e2580b80 1 --2- 172.21.15.19:0/2811710154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e895274a0 0x560e896aaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.744+0000 7f39e2580b80 1 -- 172.21.15.19:0/2811710154 >> 172.21.15.19:0/2811710154 conn(0x560e895876a0 msgr2=0x560e8969f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.745+0000 7f39d2107700 1 --2- 172.21.15.19:0/103280435 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f39b80b2b40 0x7f39b80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:47.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.745+0000 7f39e2580b80 1 -- 172.21.15.19:0/2811710154 shutdown_connections 2024-05-09T02:21:47.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.745+0000 7f39e2580b80 1 -- 172.21.15.19:0/2811710154 wait complete. 2024-05-09T02:21:47.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.746+0000 7f39d2107700 1 --2- 172.21.15.19:0/103280435 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f39b80b2b40 0x7f39b80b5050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f39cc112ba0 tx=0x7f39c400d040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:47.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.746+0000 7f39e2580b80 1 -- start start 2024-05-09T02:21:47.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.747+0000 7f39e2580b80 1 Processor -- start 2024-05-09T02:21:47.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.748+0000 7f39e2580b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e895274a0 0x560e896ae180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.748+0000 7f39e2580b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e89611160 0x560e896ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.748+0000 7f39e2580b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e896a46a0 0x560e896aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.748+0000 7f39d1906700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e895274a0 0x560e896ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:47.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.748+0000 7f39d1105700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e89611160 0x560e896ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:47.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.748+0000 7f39d1906700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e895274a0 0x560e896ae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:42594/0 (socket says 172.21.15.19:42594) 2024-05-09T02:21:47.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.748+0000 7f39d1906700 1 -- 172.21.15.19:0/1690872611 learned_addr learned my addr 172.21.15.19:0/1690872611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:47.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.748+0000 7f39d2107700 1 --2- 172.21.15.19:0/1690872611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e896a46a0 0x560e896aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:47.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.748+0000 7f39e2580b80 1 -- 172.21.15.19:0/1690872611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560e89442ba0 con 0x560e896a46a0 2024-05-09T02:21:47.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.748+0000 7f39e2580b80 1 -- 172.21.15.19:0/1690872611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560e89581150 con 0x560e89611160 2024-05-09T02:21:47.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.748+0000 7f39e2580b80 1 -- 172.21.15.19:0/1690872611 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560e896b11d0 con 0x560e895274a0 2024-05-09T02:21:47.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.749+0000 7f39d1906700 1 -- 172.21.15.19:0/1690872611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e89611160 msgr2=0x560e896ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:47.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.749+0000 7f39d1906700 1 --2- 172.21.15.19:0/1690872611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e89611160 0x560e896ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.749+0000 7f39d1906700 1 -- 172.21.15.19:0/1690872611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e896a46a0 msgr2=0x560e896aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:47.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.749+0000 7f39d1906700 1 --2- 172.21.15.19:0/1690872611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e896a46a0 0x560e896aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.749+0000 7f39d1906700 1 -- 172.21.15.19:0/1690872611 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e89442490 con 0x560e895274a0 2024-05-09T02:21:47.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.750+0000 7f39d1906700 1 --2- 172.21.15.19:0/1690872611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e895274a0 0x560e896ae180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f39c80a6c60 tx=0x7f39c80ab1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d97b741f0a1edd88 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:47.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.750+0000 7f39b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:21:47.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.750+0000 7f39e2580b80 10 client.0 _ll_register_callbacks cb 0x560e894386c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:47.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.750+0000 7f39b57fa700 1 -- 172.21.15.19:0/1690872611 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c80aa9b0 con 0x560e895274a0 2024-05-09T02:21:47.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.750+0000 7f39b57fa700 1 -- 172.21.15.19:0/1690872611 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f39c80a71d0 con 0x560e895274a0 2024-05-09T02:21:47.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.751+0000 7f39d1105700 1 --2- 172.21.15.19:0/1690872611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e89611160 0x560e896ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:21:47.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.751+0000 7f39e2580b80 1 -- 172.21.15.19:0/1690872611 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39bc018070 con 0x560e895274a0 2024-05-09T02:21:47.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.756+0000 7f39b57fa700 1 -- 172.21.15.19:0/1690872611 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f39c80a6370 con 0x560e895274a0 2024-05-09T02:21:47.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.756+0000 7f39e2580b80 10 client.26416 fetch_fsmap learned FSMap version 57 2024-05-09T02:21:47.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.756+0000 7f39e2580b80 1 -- 172.21.15.19:0/1690872611 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=57}) -- 0x560e896bb9b0 con 0x560e895274a0 2024-05-09T02:21:47.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.756+0000 7f39b57fa700 1 -- 172.21.15.19:0/1690872611 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 57) ==== 8358+0+0 (secure 0 0 0) 0x7f39c80a9e40 con 0x560e895274a0 2024-05-09T02:21:47.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.756+0000 7f39e2580b80 10 client.26416 fetch_fsmap finished waiting for FSMap version 57 2024-05-09T02:21:47.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.756+0000 7f39e2580b80 10 client.26416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:21:47.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.756+0000 7f39e2580b80 20 client.26416 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:47.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.756+0000 7f39e2580b80 1 --2- 172.21.15.19:0/1690872611 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560e896bde80 0x560e896c02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.756+0000 7f39e2580b80 4 client.26416 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:21:47.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.756+0000 7f39e2580b80 1 -- 172.21.15.19:0/1690872611 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f39bc018070 con 0x560e896bde80 2024-05-09T02:21:47.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.757+0000 7f39d1105700 1 --2- 172.21.15.19:0/1690872611 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560e896bde80 0x560e896c02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:47.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.758+0000 7f39b57fa700 10 client.26416 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:47.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.758+0000 7f39d1105700 1 --2- 172.21.15.19:0/1690872611 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560e896bde80 0x560e896c02c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3edf61147b8654 server_cookie=9f529b07d3385748 in_seq=0 out_seq=0 2024-05-09T02:21:47.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.759+0000 7f39b57fa700 1 -- 172.21.15.19:0/1690872611 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f39bc018070 con 0x560e896bde80 2024-05-09T02:21:47.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.759+0000 7f39b57fa700 10 client.26416 handle_command_reply: tid=0 2024-05-09T02:21:47.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.759+0000 7f39e2580b80 1 client.26416 shutdown 2024-05-09T02:21:47.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.759+0000 7f39e2580b80 1 -- 172.21.15.19:0/1690872611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e895274a0 msgr2=0x560e896ae180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:47.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.759+0000 7f39e2580b80 1 --2- 172.21.15.19:0/1690872611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e895274a0 0x560e896ae180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f39c80a6c60 tx=0x7f39c80ab1c0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.759+0000 7f39e2580b80 1 -- 172.21.15.19:0/1690872611 shutdown_connections 2024-05-09T02:21:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.759+0000 7f39e2580b80 1 --2- 172.21.15.19:0/1690872611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e896a46a0 0x560e896aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.759+0000 7f39e2580b80 1 --2- 172.21.15.19:0/1690872611 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560e896bde80 0x560e896c02c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.759+0000 7f39e2580b80 1 --2- 172.21.15.19:0/1690872611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e895274a0 0x560e896ae180 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.759+0000 7f39e2580b80 1 --2- 172.21.15.19:0/1690872611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e89611160 0x560e896ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.759+0000 7f39e2580b80 1 -- 172.21.15.19:0/1690872611 >> 172.21.15.19:0/1690872611 conn(0x560e895876a0 msgr2=0x560e8969f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.760+0000 7f39b57fa700 0 client.26416 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.760+0000 7f39e2580b80 1 -- 172.21.15.19:0/1690872611 shutdown_connections 2024-05-09T02:21:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.760+0000 7f39e2580b80 1 -- 172.21.15.19:0/1690872611 wait complete. 2024-05-09T02:21:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.760+0000 7f39e2580b80 20 client.26416 trim_cache size 0 max 16384 2024-05-09T02:21:47.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.792+0000 7f39e2580b80 1 Processor -- start 2024-05-09T02:21:47.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.792+0000 7f39e2580b80 1 -- start start 2024-05-09T02:21:47.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.792+0000 7f39e2580b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e895274a0 0x560e896aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.792+0000 7f39e2580b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e89611160 0x560e896b9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.792+0000 7f39e2580b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 0x560e896ba470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.792+0000 7f39e2580b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc0837b0 con 0x560e895274a0 2024-05-09T02:21:47.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.792+0000 7f39e2580b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc085320 con 0x560e896a46a0 2024-05-09T02:21:47.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.792+0000 7f39e2580b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc083e80 con 0x560e89611160 2024-05-09T02:21:47.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.793+0000 7f39d1906700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e895274a0 0x560e896aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:47.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.793+0000 7f39d1906700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e895274a0 0x560e896aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48018/0 (socket says 172.21.15.19:48018) 2024-05-09T02:21:47.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.793+0000 7f39d1906700 1 -- 172.21.15.19:0/3776556418 learned_addr learned my addr 172.21.15.19:0/3776556418 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:47.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.794+0000 7f39d2107700 1 --2- 172.21.15.19:0/3776556418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 0x560e896ba470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:47.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.794+0000 7f39d1906700 1 -- 172.21.15.19:0/3776556418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e89611160 msgr2=0x560e896b9f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:21:47.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.794+0000 7f39d1906700 1 --2- 172.21.15.19:0/3776556418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e89611160 0x560e896b9f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.794+0000 7f39d1906700 1 -- 172.21.15.19:0/3776556418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 msgr2=0x560e896ba470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:47.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.794+0000 7f39d1906700 1 --2- 172.21.15.19:0/3776556418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 0x560e896ba470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.794+0000 7f39d1906700 1 -- 172.21.15.19:0/3776556418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e89698f60 con 0x560e895274a0 2024-05-09T02:21:47.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.794+0000 7f39d1906700 1 --2- 172.21.15.19:0/3776556418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e895274a0 0x560e896aeb70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f39c80aab70 tx=0x7f39c8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d00b04bbca3690 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:47.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.795+0000 7f39b77fe700 1 -- 172.21.15.19:0/3776556418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c80aa350 con 0x560e895274a0 2024-05-09T02:21:47.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.795+0000 7f39b77fe700 1 -- 172.21.15.19:0/3776556418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c80aa6f0 con 0x560e895274a0 2024-05-09T02:21:47.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.795+0000 7f39b77fe700 1 -- 172.21.15.19:0/3776556418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c80aa530 con 0x560e895274a0 2024-05-09T02:21:47.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.796+0000 7f39e2580b80 1 -- 172.21.15.19:0/3776556418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e895274a0 msgr2=0x560e896aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:47.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.796+0000 7f39e2580b80 1 --2- 172.21.15.19:0/3776556418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e895274a0 0x560e896aeb70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f39c80aab70 tx=0x7f39c8018000 comp rx=0 tx=0).stop 2024-05-09T02:21:47.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.796+0000 7f39e2580b80 1 -- 172.21.15.19:0/3776556418 shutdown_connections 2024-05-09T02:21:47.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.796+0000 7f39e2580b80 1 --2- 172.21.15.19:0/3776556418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 0x560e896ba470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.796+0000 7f39e2580b80 1 --2- 172.21.15.19:0/3776556418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e895274a0 0x560e896aeb70 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.796+0000 7f39e2580b80 1 --2- 172.21.15.19:0/3776556418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e89611160 0x560e896b9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.796+0000 7f39e2580b80 1 -- 172.21.15.19:0/3776556418 >> 172.21.15.19:0/3776556418 conn(0x560e895876a0 msgr2=0x560e896a9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:47.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.796+0000 7f39e2580b80 1 -- 172.21.15.19:0/3776556418 shutdown_connections 2024-05-09T02:21:47.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.797+0000 7f39e2580b80 1 -- 172.21.15.19:0/3776556418 wait complete. 2024-05-09T02:21:47.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.797+0000 7f39e2580b80 1 -- start start 2024-05-09T02:21:47.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.797+0000 7f39e2580b80 1 Processor -- start 2024-05-09T02:21:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.798+0000 7f39e2580b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e895274a0 0x560e896b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.798+0000 7f39e2580b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e89611160 0x560e896b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.798+0000 7f39e2580b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 0x560e896b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.798+0000 7f39e2580b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc084c50 con 0x560e895274a0 2024-05-09T02:21:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.798+0000 7f39e2580b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc084550 con 0x560e896a46a0 2024-05-09T02:21:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.798+0000 7f39e2580b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc081770 con 0x560e89611160 2024-05-09T02:21:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.798+0000 7f39e2580b80 10 client.0 _ll_register_callbacks cb 0x560e894386c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:21:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.798+0000 7f39d2107700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 0x560e896b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.798+0000 7f39d2107700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 0x560e896b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:57124/0 (socket says 172.21.15.19:57124) 2024-05-09T02:21:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.798+0000 7f39d2107700 1 -- 172.21.15.19:0/3075488808 learned_addr learned my addr 172.21.15.19:0/3075488808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:21:47.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.799+0000 7f39d2107700 1 -- 172.21.15.19:0/3075488808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e89611160 msgr2=0x560e896b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:21:47.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.799+0000 7f39d2107700 1 --2- 172.21.15.19:0/3075488808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e89611160 0x560e896b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.799+0000 7f39d2107700 1 -- 172.21.15.19:0/3075488808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e895274a0 msgr2=0x560e896b60b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:21:47.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.799+0000 7f39d2107700 1 --2- 172.21.15.19:0/3075488808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e895274a0 0x560e896b60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.799+0000 7f39d2107700 1 -- 172.21.15.19:0/3075488808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e896b6cd0 con 0x560e896a46a0 2024-05-09T02:21:47.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.799+0000 7f39d2107700 1 --2- 172.21.15.19:0/3075488808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 0x560e896b3080 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39c4016c00 tx=0x7f39c4004b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a76efe79325032e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:21:47.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.799+0000 7f39b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:21:47.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.799+0000 7f39b57fa700 1 -- 172.21.15.19:0/3075488808 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4027020 con 0x560e896a46a0 2024-05-09T02:21:47.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.799+0000 7f39b57fa700 1 -- 172.21.15.19:0/3075488808 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f39c4013a20 con 0x560e896a46a0 2024-05-09T02:21:47.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.799+0000 7f39e2580b80 1 -- 172.21.15.19:0/3075488808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39c80aa6f0 con 0x560e896a46a0 2024-05-09T02:21:47.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.800+0000 7f39b57fa700 1 -- 172.21.15.19:0/3075488808 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f39c4032050 con 0x560e896a46a0 2024-05-09T02:21:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.800+0000 7f39e2580b80 10 client.26384 fetch_fsmap learned FSMap version 57 2024-05-09T02:21:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.800+0000 7f39e2580b80 1 -- 172.21.15.19:0/3075488808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x560e89728bd0 con 0x560e896a46a0 2024-05-09T02:21:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.800+0000 7f39b57fa700 1 -- 172.21.15.19:0/3075488808 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c80aa6f0 con 0x560e896a46a0 2024-05-09T02:21:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.801+0000 7f39b57fa700 1 -- 172.21.15.19:0/3075488808 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 57) ==== 8358+0+0 (secure 0 0 0) 0x7f39c40223b0 con 0x560e896a46a0 2024-05-09T02:21:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.801+0000 7f39e2580b80 10 client.26384 fetch_fsmap finished waiting for FSMap version 57 2024-05-09T02:21:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.801+0000 7f39e2580b80 10 client.26384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:21:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.801+0000 7f39e2580b80 20 client.26384 populate_metadata read hostname 'smithi019' 2024-05-09T02:21:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.801+0000 7f39e2580b80 1 --2- 172.21.15.19:0/3075488808 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560e89729300 0x560e8972b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:21:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.801+0000 7f39e2580b80 4 client.26384 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:21:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.801+0000 7f39e2580b80 1 -- 172.21.15.19:0/3075488808 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e895d49d0 con 0x560e89729300 2024-05-09T02:21:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.801+0000 7f39d1906700 1 --2- 172.21.15.19:0/3075488808 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560e89729300 0x560e8972b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:21:47.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.802+0000 7f39d1906700 1 --2- 172.21.15.19:0/3075488808 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560e89729300 0x560e8972b720 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff8a24190e6b5421 server_cookie=aacc3a4260db9b1c in_seq=0 out_seq=0 2024-05-09T02:21:47.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.803+0000 7f39b57fa700 10 client.26384 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:47.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.803+0000 7f39b57fa700 1 -- 172.21.15.19:0/3075488808 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560e895d49d0 con 0x560e89729300 2024-05-09T02:21:47.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.803+0000 7f39b57fa700 10 client.26384 handle_command_reply: tid=0 2024-05-09T02:21:47.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.803+0000 7f39e2580b80 1 client.26384 shutdown 2024-05-09T02:21:47.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.803+0000 7f39e2580b80 1 -- 172.21.15.19:0/3075488808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 msgr2=0x560e896b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:47.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.803+0000 7f39e2580b80 1 --2- 172.21.15.19:0/3075488808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 0x560e896b3080 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39c4016c00 tx=0x7f39c4004b60 comp rx=0 tx=0).stop 2024-05-09T02:21:47.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.804+0000 7f39e2580b80 1 -- 172.21.15.19:0/3075488808 shutdown_connections 2024-05-09T02:21:47.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.804+0000 7f39e2580b80 1 --2- 172.21.15.19:0/3075488808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e896a46a0 0x560e896b3080 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.804+0000 7f39e2580b80 1 --2- 172.21.15.19:0/3075488808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e895274a0 0x560e896b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.804+0000 7f39e2580b80 1 --2- 172.21.15.19:0/3075488808 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x560e89729300 0x560e8972b720 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.804+0000 7f39e2580b80 1 --2- 172.21.15.19:0/3075488808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e89611160 0x560e896b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.804+0000 7f39e2580b80 1 -- 172.21.15.19:0/3075488808 >> 172.21.15.19:0/3075488808 conn(0x560e895876a0 msgr2=0x560e896af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:47.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.804+0000 7f39b57fa700 0 client.26384 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:21:47.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.804+0000 7f39e2580b80 1 -- 172.21.15.19:0/3075488808 shutdown_connections 2024-05-09T02:21:47.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.804+0000 7f39e2580b80 1 -- 172.21.15.19:0/3075488808 wait complete. 2024-05-09T02:21:47.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.804+0000 7f39e2580b80 20 client.26384 trim_cache size 0 max 16384 2024-05-09T02:21:47.809 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:21:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T02:21:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T02:21:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "8748d7cd-f7ee-451f-8c1c-b1fc30b83cc1": { 2024-05-09T02:21:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T02:21:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "8748d7cd-f7ee-451f-8c1c-b1fc30b83cc1", 2024-05-09T02:21:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T02:21:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:21:47.809 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:21:47.809 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:21:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.805+0000 7f39d3b6a700 1 -- 172.21.15.19:0/103280435 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f39b80b2b40 msgr2=0x7f39b80b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.805+0000 7f39d3b6a700 1 --2- 172.21.15.19:0/103280435 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f39b80b2b40 0x7f39b80b5050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f39cc112ba0 tx=0x7f39c400d040 comp rx=0 tx=0).stop 2024-05-09T02:21:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.805+0000 7f39d3b6a700 1 -- 172.21.15.19:0/103280435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39cc10a930 msgr2=0x7f39cc117570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:21:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.805+0000 7f39d3b6a700 1 --2- 172.21.15.19:0/103280435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39cc10a930 0x7f39cc117570 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f39c80057f0 tx=0x7f39c80037b0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.805+0000 7f39d3b6a700 1 -- 172.21.15.19:0/103280435 shutdown_connections 2024-05-09T02:21:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.805+0000 7f39d3b6a700 1 --2- 172.21.15.19:0/103280435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39cc11d930 0x7f39cc117ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.805+0000 7f39d3b6a700 1 --2- 172.21.15.19:0/103280435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39cc10a930 0x7f39cc117570 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.805+0000 7f39d3b6a700 1 --2- 172.21.15.19:0/103280435 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f39b80b2b40 0x7f39b80b5050 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.805+0000 7f39d3b6a700 1 --2- 172.21.15.19:0/103280435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39cc10b280 0x7f39cc117ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:21:47.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.805+0000 7f39d3b6a700 1 -- 172.21.15.19:0/103280435 >> 172.21.15.19:0/103280435 conn(0x7f39cc06f4d0 msgr2=0x7f39cc11e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:21:47.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.806+0000 7f39d3b6a700 1 -- 172.21.15.19:0/103280435 shutdown_connections 2024-05-09T02:21:47.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:21:47.806+0000 7f39d3b6a700 1 -- 172.21.15.19:0/103280435 wait complete. 2024-05-09T02:21:48.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:48 smithi059 ceph-mon[34781]: pgmap v326: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 1.51k op/s 2024-05-09T02:21:48.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:48 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:48.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:48 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:48.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:48 smithi196 ceph-mon[34718]: pgmap v326: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 1.51k op/s 2024-05-09T02:21:48.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:48 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:48.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:48 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:48.564 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:48 smithi019 ceph-mon[30151]: pgmap v326: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 1.51k op/s 2024-05-09T02:21:48.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:48 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:48.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:48 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:48.613 INFO:tasks.cephfs.filesystem:scrub status for tag:8748d7cd-f7ee-451f-8c1c-b1fc30b83cc1 - {'path': '/', 'tag': '8748d7cd-f7ee-451f-8c1c-b1fc30b83cc1', 'options': 'recursive,force'} 2024-05-09T02:21:49.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:49 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:49.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:49 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:49.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:49 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:49.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:49 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:49.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:49 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:21:49.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:49 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:21:49.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:49 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:49.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:49 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:49.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:49 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:49.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:49 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:49.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:49 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:49.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:49 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:21:49.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:49 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:21:49.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:49 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:49.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:49 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:49.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:49 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:49.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:49 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:49.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:49 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T02:21:49.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:49 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:21:49.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:49 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:21:49.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:49 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:21:50.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:50 smithi059 ceph-mon[34781]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-09T02:21:50.468 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:50 smithi059 ceph-mon[34781]: Unable to set osd_memory_target on smithi019 to 185876992: error parsing value: Value '185876992' is below minimum 939524096 2024-05-09T02:21:50.468 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:50 smithi059 ceph-mon[34781]: pgmap v327: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 19 MiB/s wr, 1.70k op/s 2024-05-09T02:21:50.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:50 smithi196 ceph-mon[34718]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-09T02:21:50.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:50 smithi196 ceph-mon[34718]: Unable to set osd_memory_target on smithi019 to 185876992: error parsing value: Value '185876992' is below minimum 939524096 2024-05-09T02:21:50.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:50 smithi196 ceph-mon[34718]: pgmap v327: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 19 MiB/s wr, 1.70k op/s 2024-05-09T02:21:50.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:50 smithi019 ceph-mon[30151]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-05-09T02:21:50.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:50 smithi019 ceph-mon[30151]: Unable to set osd_memory_target on smithi019 to 185876992: error parsing value: Value '185876992' is below minimum 939524096 2024-05-09T02:21:50.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:50 smithi019 ceph-mon[30151]: pgmap v327: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 19 MiB/s wr, 1.70k op/s 2024-05-09T02:21:51.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:51 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:21:51.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:51 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:21:51.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:51 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:21:51.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:51 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:21:51.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:51 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:21:51.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:51 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:21:51.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:51 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:21:51.545 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:51 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:21:51.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:51 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:21:51.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:51 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:21:51.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:51 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:21:51.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:51 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:21:52.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:52 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:21:52.467 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:52 smithi059 ceph-mon[34781]: pgmap v328: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.6 MiB/s wr, 843 op/s 2024-05-09T02:21:52.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:52 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:21:52.547 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:52 smithi196 ceph-mon[34718]: pgmap v328: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.6 MiB/s wr, 843 op/s 2024-05-09T02:21:52.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:52 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:21:52.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:52 smithi019 ceph-mon[30151]: pgmap v328: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.6 MiB/s wr, 843 op/s 2024-05-09T02:21:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 610.0 s, 2495.6 tps, lat 39.680 ms stddev 45.844 2024-05-09T02:21:54.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:54 smithi059 ceph-mon[34781]: pgmap v329: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-05-09T02:21:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:54 smithi196 ceph-mon[34718]: pgmap v329: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-05-09T02:21:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:54 smithi019 ceph-mon[30151]: pgmap v329: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-05-09T02:21:56.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:56 smithi059 ceph-mon[34781]: pgmap v330: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.6 MiB/s wr, 780 op/s 2024-05-09T02:21:56.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:56 smithi196 ceph-mon[34718]: pgmap v330: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.6 MiB/s wr, 780 op/s 2024-05-09T02:21:56.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:56 smithi019 ceph-mon[30151]: pgmap v330: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.6 MiB/s wr, 780 op/s 2024-05-09T02:21:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 615.0 s, 2121.4 tps, lat 47.589 ms stddev 49.858 2024-05-09T02:21:58.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:21:58 smithi059 ceph-mon[34781]: pgmap v331: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 535 op/s 2024-05-09T02:21:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:21:58 smithi196 ceph-mon[34718]: pgmap v331: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 535 op/s 2024-05-09T02:21:58.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:21:58 smithi019 ceph-mon[30151]: pgmap v331: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 535 op/s 2024-05-09T02:22:00.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:00 smithi059 ceph-mon[34781]: pgmap v332: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.7 MiB/s wr, 765 op/s 2024-05-09T02:22:00.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:00 smithi196 ceph-mon[34718]: pgmap v332: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.7 MiB/s wr, 765 op/s 2024-05-09T02:22:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:00 smithi019 ceph-mon[30151]: pgmap v332: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.7 MiB/s wr, 765 op/s 2024-05-09T02:22:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 620.0 s, 1454.6 tps, lat 68.235 ms stddev 143.664 2024-05-09T02:22:02.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:02 smithi059 ceph-mon[34781]: pgmap v333: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.5 MiB/s wr, 640 op/s 2024-05-09T02:22:02.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:02 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:22:02.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:02 smithi196 ceph-mon[34718]: pgmap v333: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.5 MiB/s wr, 640 op/s 2024-05-09T02:22:02.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:02 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:22:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:02 smithi019 ceph-mon[30151]: pgmap v333: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.5 MiB/s wr, 640 op/s 2024-05-09T02:22:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:02 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:22:04.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:04 smithi059 ceph-mon[34781]: pgmap v334: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.0 MiB/s wr, 584 op/s 2024-05-09T02:22:04.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:04 smithi196 ceph-mon[34718]: pgmap v334: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.0 MiB/s wr, 584 op/s 2024-05-09T02:22:04.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:04 smithi019 ceph-mon[30151]: pgmap v334: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.0 MiB/s wr, 584 op/s 2024-05-09T02:22:06.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:06 smithi059 ceph-mon[34781]: pgmap v335: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 23 MiB/s wr, 1.89k op/s 2024-05-09T02:22:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:06 smithi196 ceph-mon[34718]: pgmap v335: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 23 MiB/s wr, 1.89k op/s 2024-05-09T02:22:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:06 smithi019 ceph-mon[30151]: pgmap v335: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 23 MiB/s wr, 1.89k op/s 2024-05-09T02:22:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 625.0 s, 1392.2 tps, lat 71.914 ms stddev 101.460 2024-05-09T02:22:07.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:07 smithi059 ceph-mon[34781]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T02:22:07.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:07 smithi196 ceph-mon[34718]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T02:22:07.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:07 smithi019 ceph-mon[30151]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T02:22:08.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:08 smithi059 ceph-mon[34781]: pgmap v337: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 23 MiB/s wr, 1.95k op/s 2024-05-09T02:22:08.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:08 smithi196 ceph-mon[34718]: pgmap v337: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 23 MiB/s wr, 1.95k op/s 2024-05-09T02:22:08.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:08 smithi019 ceph-mon[30151]: pgmap v337: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 23 MiB/s wr, 1.95k op/s 2024-05-09T02:22:10.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:10 smithi059 ceph-mon[34781]: pgmap v338: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 1.92k op/s 2024-05-09T02:22:10.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:10 smithi196 ceph-mon[34718]: pgmap v338: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 1.92k op/s 2024-05-09T02:22:10.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:10 smithi019 ceph-mon[30151]: pgmap v338: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 1.92k op/s 2024-05-09T02:22:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 630.0 s, 1362.8 tps, lat 73.074 ms stddev 75.204 2024-05-09T02:22:12.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:12 smithi059 ceph-mon[34781]: pgmap v339: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 1.91k op/s 2024-05-09T02:22:12.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:12 smithi196 ceph-mon[34718]: pgmap v339: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 1.91k op/s 2024-05-09T02:22:12.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:12 smithi019 ceph-mon[30151]: pgmap v339: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 1.91k op/s 2024-05-09T02:22:14.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:14 smithi059 ceph-mon[34781]: pgmap v340: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 1.91k op/s 2024-05-09T02:22:14.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:14 smithi196 ceph-mon[34718]: pgmap v340: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 1.91k op/s 2024-05-09T02:22:14.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:14 smithi019 ceph-mon[30151]: pgmap v340: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 1.91k op/s 2024-05-09T02:22:15.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:15 smithi196 ceph-mon[34718]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T02:22:15.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:15 smithi019 ceph-mon[30151]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T02:22:15.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:15 smithi059 ceph-mon[34781]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T02:22:16.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:16 smithi196 ceph-mon[34718]: pgmap v342: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.1 MiB/s wr, 683 op/s 2024-05-09T02:22:16.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:16 smithi019 ceph-mon[30151]: pgmap v342: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.1 MiB/s wr, 683 op/s 2024-05-09T02:22:16.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:16 smithi059 ceph-mon[34781]: pgmap v342: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.1 MiB/s wr, 683 op/s 2024-05-09T02:22:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 635.0 s, 1361.0 tps, lat 73.370 ms stddev 81.588 2024-05-09T02:22:18.614 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:22:18.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:18 smithi196 ceph-mon[34718]: pgmap v343: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 609 op/s 2024-05-09T02:22:18.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:18 smithi019 ceph-mon[30151]: pgmap v343: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 609 op/s 2024-05-09T02:22:18.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:18 smithi059 ceph-mon[34781]: pgmap v343: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 609 op/s 2024-05-09T02:22:18.985 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:22:20.773 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:20 smithi019 ceph-mon[30151]: pgmap v344: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.3 MiB/s wr, 599 op/s 2024-05-09T02:22:20.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:20 smithi196 ceph-mon[34718]: pgmap v344: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.3 MiB/s wr, 599 op/s 2024-05-09T02:22:20.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.924+0000 7f702a2b6700 1 -- 172.21.15.19:0/1761786011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70241034c0 msgr2=0x7f70241038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.924+0000 7f702a2b6700 1 --2- 172.21.15.19:0/1761786011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70241034c0 0x7f70241038d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f70140099c0 tx=0x7f7014009cd0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.924+0000 7f702a2b6700 1 -- 172.21.15.19:0/1761786011 shutdown_connections 2024-05-09T02:22:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.924+0000 7f702a2b6700 1 --2- 172.21.15.19:0/1761786011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7024108740 0x7f70241100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.924+0000 7f702a2b6700 1 --2- 172.21.15.19:0/1761786011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70241034c0 0x7f70241038d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.924+0000 7f702a2b6700 1 --2- 172.21.15.19:0/1761786011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7024107df0 0x7f7024108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.924+0000 7f702a2b6700 1 -- 172.21.15.19:0/1761786011 >> 172.21.15.19:0/1761786011 conn(0x7f70240fedc0 msgr2=0x7f70241011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.924+0000 7f702a2b6700 1 -- 172.21.15.19:0/1761786011 shutdown_connections 2024-05-09T02:22:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.924+0000 7f702a2b6700 1 -- 172.21.15.19:0/1761786011 wait complete. 2024-05-09T02:22:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.925+0000 7f702a2b6700 1 Processor -- start 2024-05-09T02:22:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.925+0000 7f702a2b6700 1 -- start start 2024-05-09T02:22:20.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.925+0000 7f702a2b6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70241034c0 0x7f702419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:20.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.925+0000 7f702a2b6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7024107df0 0x7f702419c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.925+0000 7f702a2b6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024108740 0x7f702419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.925+0000 7f702a2b6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7024113370 con 0x7f7024108740 2024-05-09T02:22:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.925+0000 7f702a2b6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f70241131f0 con 0x7f70241034c0 2024-05-09T02:22:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.925+0000 7f702a2b6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70241134f0 con 0x7f7024107df0 2024-05-09T02:22:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.926+0000 7f70237fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7024107df0 0x7f702419c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.926+0000 7f7028853700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024108740 0x7f702419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.926+0000 7f70237fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7024107df0 0x7f702419c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:35978/0 (socket says 172.21.15.19:35978) 2024-05-09T02:22:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.926+0000 7f7028853700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024108740 0x7f702419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48802/0 (socket says 172.21.15.19:48802) 2024-05-09T02:22:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.926+0000 7f70237fe700 1 -- 172.21.15.19:0/410427294 learned_addr learned my addr 172.21.15.19:0/410427294 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.926+0000 7f7023fff700 1 --2- 172.21.15.19:0/410427294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70241034c0 0x7f702419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.926+0000 7f70237fe700 1 -- 172.21.15.19:0/410427294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70241034c0 msgr2=0x7f702419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.926+0000 7f70237fe700 1 --2- 172.21.15.19:0/410427294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70241034c0 0x7f702419bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.926+0000 7f70237fe700 1 -- 172.21.15.19:0/410427294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024108740 msgr2=0x7f702419c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.926+0000 7f70237fe700 1 --2- 172.21.15.19:0/410427294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024108740 0x7f702419c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.926+0000 7f70237fe700 1 -- 172.21.15.19:0/410427294 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70241a6f80 con 0x7f7024107df0 2024-05-09T02:22:20.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.926+0000 7f70237fe700 1 --2- 172.21.15.19:0/410427294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7024107df0 0x7f702419c170 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f701800e970 tx=0x7f701800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:20.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.926+0000 7f70217fa700 1 -- 172.21.15.19:0/410427294 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018004a30 con 0x7f7024107df0 2024-05-09T02:22:20.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.927+0000 7f70217fa700 1 -- 172.21.15.19:0/410427294 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f701800cc90 con 0x7f7024107df0 2024-05-09T02:22:20.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.927+0000 7f70217fa700 1 -- 172.21.15.19:0/410427294 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018005610 con 0x7f7024107df0 2024-05-09T02:22:20.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.927+0000 7f702a2b6700 1 -- 172.21.15.19:0/410427294 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7024199a80 con 0x7f7024107df0 2024-05-09T02:22:20.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.927+0000 7f702a2b6700 1 -- 172.21.15.19:0/410427294 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7024199f10 con 0x7f7024107df0 2024-05-09T02:22:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.930+0000 7f70217fa700 1 -- 172.21.15.19:0/410427294 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7018021440 con 0x7f7024107df0 2024-05-09T02:22:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.931+0000 7f7038cccb80 1 Processor -- start 2024-05-09T02:22:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.931+0000 7f7038cccb80 1 -- start start 2024-05-09T02:22:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.931+0000 7f7038cccb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8b74d00 0x557bd8c7d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.931+0000 7f7038cccb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8afa4a0 0x557bd8c7d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.931+0000 7f70217fa700 1 --2- 172.21.15.19:0/410427294 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f700c0b2a70 0x7f700c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f7038cccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8c77400 0x557bd8c77810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f7038cccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557bd8b9a0a0 con 0x557bd8c77400 2024-05-09T02:22:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f7028853700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8b74d00 0x557bd8c7d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f70237fe700 1 --2- 172.21.15.19:0/410427294 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f700c0b2a70 0x7f700c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f7028853700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8b74d00 0x557bd8c7d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:35980/0 (socket says 172.21.15.19:35980) 2024-05-09T02:22:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f7028853700 1 -- 172.21.15.19:0/1226987570 learned_addr learned my addr 172.21.15.19:0/1226987570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f7038cccb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557bd8b3f9a0 con 0x557bd8afa4a0 2024-05-09T02:22:20.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f7038cccb80 1 -- 172.21.15.19:0/1226987570 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557bd8b96060 con 0x557bd8b74d00 2024-05-09T02:22:20.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f7028853700 1 --2- 172.21.15.19:0/1226987570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8c77400 0x557bd8c77810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:20.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f7023fff700 1 --2- 172.21.15.19:0/1226987570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8afa4a0 0x557bd8c7d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:20.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f70217fa700 1 -- 172.21.15.19:0/410427294 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f70180e27f0 con 0x7f7024107df0 2024-05-09T02:22:20.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f7028853700 1 -- 172.21.15.19:0/1226987570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8afa4a0 msgr2=0x557bd8c7d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:20.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f7028853700 1 --2- 172.21.15.19:0/1226987570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8afa4a0 0x557bd8c7d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f7028853700 1 -- 172.21.15.19:0/1226987570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8c77400 msgr2=0x557bd8c77810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:20.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f7028853700 1 --2- 172.21.15.19:0/1226987570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8c77400 0x557bd8c77810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.932+0000 7f7028853700 1 -- 172.21.15.19:0/1226987570 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bd8b30500 con 0x557bd8b74d00 2024-05-09T02:22:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.933+0000 7f7028853700 1 --2- 172.21.15.19:0/1226987570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8b74d00 0x557bd8c7d240 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f701c007990 tx=0x7f701c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60e1b29a7fa45bc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.934+0000 7f700a7fc700 1 -- 172.21.15.19:0/1226987570 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c004bb0 con 0x557bd8b74d00 2024-05-09T02:22:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.934+0000 7f700a7fc700 1 -- 172.21.15.19:0/1226987570 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f701c004d50 con 0x557bd8b74d00 2024-05-09T02:22:20.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.936+0000 7f7038cccb80 1 -- 172.21.15.19:0/1226987570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8b74d00 msgr2=0x557bd8c7d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:20.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.936+0000 7f7038cccb80 1 --2- 172.21.15.19:0/1226987570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8b74d00 0x557bd8c7d240 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f701c007990 tx=0x7f701c00b040 comp rx=0 tx=0).stop 2024-05-09T02:22:20.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.936+0000 7f70237fe700 1 --2- 172.21.15.19:0/410427294 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f700c0b2a70 0x7f700c0b4f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f70180a5ac0 tx=0x7f70180cc040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:20.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.936+0000 7f7038cccb80 1 -- 172.21.15.19:0/1226987570 shutdown_connections 2024-05-09T02:22:20.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.936+0000 7f7038cccb80 1 --2- 172.21.15.19:0/1226987570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8c77400 0x557bd8c77810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.936+0000 7f7038cccb80 1 --2- 172.21.15.19:0/1226987570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8b74d00 0x557bd8c7d240 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.936+0000 7f7038cccb80 1 --2- 172.21.15.19:0/1226987570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8afa4a0 0x557bd8c7d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.936+0000 7f7038cccb80 1 -- 172.21.15.19:0/1226987570 >> 172.21.15.19:0/1226987570 conn(0x557bd8afdf70 msgr2=0x557bd8c72220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:20.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.937+0000 7f7038cccb80 1 -- 172.21.15.19:0/1226987570 shutdown_connections 2024-05-09T02:22:20.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.937+0000 7f7038cccb80 1 -- 172.21.15.19:0/1226987570 wait complete. 2024-05-09T02:22:20.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.937+0000 7f7038cccb80 1 -- start start 2024-05-09T02:22:20.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.937+0000 7f7038cccb80 1 Processor -- start 2024-05-09T02:22:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.938+0000 7f7038cccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8afa4a0 0x557bd8c80f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.938+0000 7f7038cccb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 0x557bd8c814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.938+0000 7f7038cccb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8c77400 0x557bd8c819f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.938+0000 7f7038cccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557bd8c78150 con 0x557bd8afa4a0 2024-05-09T02:22:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.938+0000 7f7038cccb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557bd8b34b20 con 0x557bd8b74d00 2024-05-09T02:22:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.938+0000 7f7038cccb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557bd8c84020 con 0x557bd8c77400 2024-05-09T02:22:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.938+0000 7f7038cccb80 10 client.0 _ll_register_callbacks cb 0x557bd8a0b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.938+0000 7f7028853700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8afa4a0 0x557bd8c80f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.938+0000 7f7028853700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8afa4a0 0x557bd8c80f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48822/0 (socket says 172.21.15.19:48822) 2024-05-09T02:22:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.938+0000 7f7028853700 1 -- 172.21.15.19:0/3490775379 learned_addr learned my addr 172.21.15.19:0/3490775379 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.938+0000 7f7023fff700 1 --2- 172.21.15.19:0/3490775379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 0x557bd8c814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.939+0000 7f7028853700 1 --2- 172.21.15.19:0/3490775379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8c77400 0x557bd8c819f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.939+0000 7f7023fff700 1 -- 172.21.15.19:0/3490775379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8c77400 msgr2=0x557bd8c819f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.939+0000 7f7023fff700 1 --2- 172.21.15.19:0/3490775379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8c77400 0x557bd8c819f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.939+0000 7f7023fff700 1 -- 172.21.15.19:0/3490775379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8afa4a0 msgr2=0x557bd8c80f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.939+0000 7f7023fff700 1 --2- 172.21.15.19:0/3490775379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8afa4a0 0x557bd8c80f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.939+0000 7f7023fff700 1 -- 172.21.15.19:0/3490775379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bd8a15490 con 0x557bd8b74d00 2024-05-09T02:22:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.939+0000 7f7028853700 1 --2- 172.21.15.19:0/3490775379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8c77400 0x557bd8c819f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:22:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.939+0000 7f7023fff700 1 --2- 172.21.15.19:0/3490775379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 0x557bd8c814b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7014009990 tx=0x7f7014005790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be7329b87ba64960 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.939+0000 7f70037fe700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:22:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.939+0000 7f70037fe700 1 -- 172.21.15.19:0/3490775379 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70140039c0 con 0x557bd8b74d00 2024-05-09T02:22:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.939+0000 7f7038cccb80 1 -- 172.21.15.19:0/3490775379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f701c004d50 con 0x557bd8b74d00 2024-05-09T02:22:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.939+0000 7f70037fe700 1 -- 172.21.15.19:0/3490775379 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f7014003b60 con 0x557bd8b74d00 2024-05-09T02:22:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.940+0000 7f70037fe700 1 -- 172.21.15.19:0/3490775379 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f7014031050 con 0x557bd8b74d00 2024-05-09T02:22:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.940+0000 7f7038cccb80 10 client.26396 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.940+0000 7f7038cccb80 1 -- 172.21.15.19:0/3490775379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557bd8c8e9d0 con 0x557bd8b74d00 2024-05-09T02:22:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.940+0000 7f70037fe700 1 -- 172.21.15.19:0/3490775379 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f701c004d50 con 0x557bd8b74d00 2024-05-09T02:22:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.941+0000 7f70037fe700 1 -- 172.21.15.19:0/3490775379 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f701402c030 con 0x557bd8b74d00 2024-05-09T02:22:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.941+0000 7f7038cccb80 10 client.26396 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.941+0000 7f7038cccb80 10 client.26396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:22:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.941+0000 7f7038cccb80 20 client.26396 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.941+0000 7f7038cccb80 1 --2- 172.21.15.19:0/3490775379 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557bd8c91060 0x557bd8c93480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.941+0000 7f7038cccb80 4 client.26396 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:22:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.941+0000 7f7038cccb80 1 -- 172.21.15.19:0/3490775379 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bd8ba7a80 con 0x557bd8c91060 2024-05-09T02:22:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.941+0000 7f7028853700 1 --2- 172.21.15.19:0/3490775379 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557bd8c91060 0x557bd8c93480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.942+0000 7f70037fe700 10 client.26396 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.942+0000 7f7028853700 1 --2- 172.21.15.19:0/3490775379 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557bd8c91060 0x557bd8c93480 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9351cbe1f0fbf7fc server_cookie=17b88616c7f4e1a2 in_seq=0 out_seq=0 2024-05-09T02:22:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.942+0000 7f70037fe700 1 -- 172.21.15.19:0/3490775379 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557bd8ba7a80 con 0x557bd8c91060 2024-05-09T02:22:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.942+0000 7f70037fe700 10 client.26396 handle_command_reply: tid=0 2024-05-09T02:22:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.942+0000 7f7038cccb80 1 client.26396 shutdown 2024-05-09T02:22:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.943+0000 7f7038cccb80 1 -- 172.21.15.19:0/3490775379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 msgr2=0x557bd8c814b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.943+0000 7f7038cccb80 1 --2- 172.21.15.19:0/3490775379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 0x557bd8c814b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7014009990 tx=0x7f7014005790 comp rx=0 tx=0).stop 2024-05-09T02:22:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.943+0000 7f7038cccb80 1 -- 172.21.15.19:0/3490775379 shutdown_connections 2024-05-09T02:22:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.943+0000 7f7038cccb80 1 --2- 172.21.15.19:0/3490775379 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557bd8c91060 0x557bd8c93480 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.943+0000 7f7038cccb80 1 --2- 172.21.15.19:0/3490775379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8c77400 0x557bd8c819f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.943+0000 7f7038cccb80 1 --2- 172.21.15.19:0/3490775379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8afa4a0 0x557bd8c80f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.943+0000 7f7038cccb80 1 --2- 172.21.15.19:0/3490775379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 0x557bd8c814b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.943+0000 7f70037fe700 0 client.26396 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.943+0000 7f7038cccb80 1 -- 172.21.15.19:0/3490775379 >> 172.21.15.19:0/3490775379 conn(0x557bd8afdf70 msgr2=0x557bd8c6bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.943+0000 7f7038cccb80 1 -- 172.21.15.19:0/3490775379 shutdown_connections 2024-05-09T02:22:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.943+0000 7f7038cccb80 1 -- 172.21.15.19:0/3490775379 wait complete. 2024-05-09T02:22:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.944+0000 7f7038cccb80 20 client.26396 trim_cache size 0 max 16384 2024-05-09T02:22:20.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:20 smithi059 ceph-mon[34781]: pgmap v344: 97 pgs: 97 active+clean; 4.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.3 MiB/s wr, 599 op/s 2024-05-09T02:22:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.982+0000 7f7038cccb80 1 Processor -- start 2024-05-09T02:22:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.982+0000 7f7038cccb80 1 -- start start 2024-05-09T02:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.983+0000 7f7038cccb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8afa4a0 0x557bd8c6ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.983+0000 7f7038cccb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 0x557bd8c98c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.983+0000 7f7028853700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8afa4a0 0x557bd8c6ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.983+0000 7f7028853700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8afa4a0 0x557bd8c6ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36004/0 (socket says 172.21.15.19:36004) 2024-05-09T02:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.983+0000 7f7038cccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8c77400 0x557bd8c99170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.983+0000 7f7028853700 1 -- 172.21.15.19:0/506394651 learned_addr learned my addr 172.21.15.19:0/506394651 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.983+0000 7f7023fff700 1 --2- 172.21.15.19:0/506394651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 0x557bd8c98c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.983+0000 7f7038cccb80 1 -- 172.21.15.19:0/506394651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70241b0fe0 con 0x557bd8c77400 2024-05-09T02:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.983+0000 7f7038cccb80 1 -- 172.21.15.19:0/506394651 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f70241b2b20 con 0x557bd8b74d00 2024-05-09T02:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.983+0000 7f7038cccb80 1 -- 172.21.15.19:0/506394651 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70241b16b0 con 0x557bd8afa4a0 2024-05-09T02:22:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.988+0000 7f7023fff700 1 -- 172.21.15.19:0/506394651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8afa4a0 msgr2=0x557bd8c6ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.988+0000 7f7023fff700 1 --2- 172.21.15.19:0/506394651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8afa4a0 0x557bd8c6ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.988+0000 7f7023fff700 1 -- 172.21.15.19:0/506394651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8c77400 msgr2=0x557bd8c99170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.988+0000 7f7023fff700 1 --2- 172.21.15.19:0/506394651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8c77400 0x557bd8c99170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.988+0000 7f7023fff700 1 -- 172.21.15.19:0/506394651 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bd8c80d40 con 0x557bd8b74d00 2024-05-09T02:22:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.988+0000 7f7023fff700 1 --2- 172.21.15.19:0/506394651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 0x557bd8c98c30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f70140099c0 tx=0x7f701400b5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ca3b2762ba5afca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:20.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.992+0000 7f7009ffb700 1 -- 172.21.15.19:0/506394651 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014024c00 con 0x557bd8b74d00 2024-05-09T02:22:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.992+0000 7f7009ffb700 1 -- 172.21.15.19:0/506394651 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7014003c70 con 0x557bd8b74d00 2024-05-09T02:22:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.992+0000 7f7009ffb700 1 -- 172.21.15.19:0/506394651 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701400be70 con 0x557bd8b74d00 2024-05-09T02:22:20.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.996+0000 7f7038cccb80 1 -- 172.21.15.19:0/506394651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 msgr2=0x557bd8c98c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:20.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.996+0000 7f7038cccb80 1 --2- 172.21.15.19:0/506394651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 0x557bd8c98c30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f70140099c0 tx=0x7f701400b5c0 comp rx=0 tx=0).stop 2024-05-09T02:22:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.998+0000 7f7038cccb80 1 -- 172.21.15.19:0/506394651 shutdown_connections 2024-05-09T02:22:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.998+0000 7f7038cccb80 1 --2- 172.21.15.19:0/506394651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8c77400 0x557bd8c99170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.998+0000 7f7038cccb80 1 --2- 172.21.15.19:0/506394651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8afa4a0 0x557bd8c6ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.998+0000 7f7038cccb80 1 --2- 172.21.15.19:0/506394651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 0x557bd8c98c30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:21.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.998+0000 7f7038cccb80 1 -- 172.21.15.19:0/506394651 >> 172.21.15.19:0/506394651 conn(0x557bd8afdf70 msgr2=0x557bd8c7c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:21.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.998+0000 7f7038cccb80 1 -- 172.21.15.19:0/506394651 shutdown_connections 2024-05-09T02:22:21.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.998+0000 7f7038cccb80 1 -- 172.21.15.19:0/506394651 wait complete. 2024-05-09T02:22:21.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.999+0000 7f7038cccb80 1 -- start start 2024-05-09T02:22:21.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.999+0000 7f7038cccb80 1 Processor -- start 2024-05-09T02:22:21.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:20.999+0000 7f7038cccb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8afa4a0 0x557bd8c897f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.000+0000 7f7038cccb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 0x557bd8c7dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.000+0000 7f7038cccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8c77400 0x557bd8c7e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.000+0000 7f7028853700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8afa4a0 0x557bd8c897f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.000+0000 7f7028853700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8afa4a0 0x557bd8c897f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36018/0 (socket says 172.21.15.19:36018) 2024-05-09T02:22:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.000+0000 7f7023fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 0x557bd8c7dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.000+0000 7f7023fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 0x557bd8c7dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:57986/0 (socket says 172.21.15.19:57986) 2024-05-09T02:22:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.000+0000 7f7028853700 1 -- 172.21.15.19:0/1706880680 learned_addr learned my addr 172.21.15.19:0/1706880680 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.000+0000 7f7038cccb80 1 -- 172.21.15.19:0/1706880680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70241b2450 con 0x557bd8c77400 2024-05-09T02:22:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.000+0000 7f7038cccb80 1 -- 172.21.15.19:0/1706880680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f70241b1d80 con 0x557bd8b74d00 2024-05-09T02:22:21.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.000+0000 7f7038cccb80 1 -- 172.21.15.19:0/1706880680 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70241aefa0 con 0x557bd8afa4a0 2024-05-09T02:22:21.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.000+0000 7f7028853700 1 --2- 172.21.15.19:0/1706880680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8c77400 0x557bd8c7e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:21.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.000+0000 7f7038cccb80 10 client.0 _ll_register_callbacks cb 0x557bd8a0b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:21.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.003+0000 7f7028853700 1 -- 172.21.15.19:0/1706880680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 msgr2=0x557bd8c7dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:21.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.003+0000 7f7028853700 1 --2- 172.21.15.19:0/1706880680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 0x557bd8c7dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:21.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.003+0000 7f7028853700 1 -- 172.21.15.19:0/1706880680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8c77400 msgr2=0x557bd8c7e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:21.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.003+0000 7f7028853700 1 --2- 172.21.15.19:0/1706880680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8c77400 0x557bd8c7e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:21.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.003+0000 7f7028853700 1 -- 172.21.15.19:0/1706880680 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bd8c7e8e0 con 0x557bd8afa4a0 2024-05-09T02:22:21.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.003+0000 7f7028853700 1 --2- 172.21.15.19:0/1706880680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8afa4a0 0x557bd8c897f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f701c008fb0 tx=0x7f701c01aaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dac2d836f0fa15c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:21.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.003+0000 7f70037fe700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:22:21.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.005+0000 7f70037fe700 1 -- 172.21.15.19:0/1706880680 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c035060 con 0x557bd8afa4a0 2024-05-09T02:22:21.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.005+0000 7f70037fe700 1 -- 172.21.15.19:0/1706880680 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f701c012040 con 0x557bd8afa4a0 2024-05-09T02:22:21.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.006+0000 7f7038cccb80 1 -- 172.21.15.19:0/1706880680 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7014003c70 con 0x557bd8afa4a0 2024-05-09T02:22:21.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.009+0000 7f70037fe700 1 -- 172.21.15.19:0/1706880680 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f701c01a6b0 con 0x557bd8afa4a0 2024-05-09T02:22:21.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.009+0000 7f7038cccb80 10 client.26434 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:21.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.009+0000 7f7038cccb80 1 -- 172.21.15.19:0/1706880680 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557bd8cf4740 con 0x557bd8afa4a0 2024-05-09T02:22:21.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.010+0000 7f70037fe700 1 -- 172.21.15.19:0/1706880680 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7014003c70 con 0x557bd8afa4a0 2024-05-09T02:22:21.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.011+0000 7f70037fe700 1 -- 172.21.15.19:0/1706880680 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f701c015070 con 0x557bd8afa4a0 2024-05-09T02:22:21.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.011+0000 7f7038cccb80 10 client.26434 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:21.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.011+0000 7f7038cccb80 10 client.26434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:22:21.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.011+0000 7f7038cccb80 20 client.26434 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:21.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.011+0000 7f7038cccb80 1 --2- 172.21.15.19:0/1706880680 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557bd8cf49d0 0x557bd8cf6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:21.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.011+0000 7f7038cccb80 4 client.26434 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:22:21.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.011+0000 7f7038cccb80 1 -- 172.21.15.19:0/1706880680 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557bd8ba1b20 con 0x557bd8cf49d0 2024-05-09T02:22:21.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.011+0000 7f7023fff700 1 --2- 172.21.15.19:0/1706880680 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557bd8cf49d0 0x557bd8cf6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:21.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.014+0000 7f7023fff700 1 --2- 172.21.15.19:0/1706880680 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557bd8cf49d0 0x557bd8cf6df0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a7aab87ad3df331 server_cookie=8aa55ac6b1422aa3 in_seq=0 out_seq=0 2024-05-09T02:22:21.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.014+0000 7f70037fe700 10 client.26434 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:21.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.014+0000 7f70037fe700 1 -- 172.21.15.19:0/1706880680 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557bd8ba1b20 con 0x557bd8cf49d0 2024-05-09T02:22:21.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.014+0000 7f70037fe700 10 client.26434 handle_command_reply: tid=0 2024-05-09T02:22:21.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.014+0000 7f7038cccb80 1 client.26434 shutdown 2024-05-09T02:22:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.017+0000 7f7038cccb80 1 -- 172.21.15.19:0/1706880680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8afa4a0 msgr2=0x557bd8c897f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.017+0000 7f7038cccb80 1 --2- 172.21.15.19:0/1706880680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8afa4a0 0x557bd8c897f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f701c008fb0 tx=0x7f701c01aaf0 comp rx=0 tx=0).stop 2024-05-09T02:22:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.018+0000 7f7038cccb80 1 -- 172.21.15.19:0/1706880680 shutdown_connections 2024-05-09T02:22:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.018+0000 7f7038cccb80 1 --2- 172.21.15.19:0/1706880680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bd8c77400 0x557bd8c7e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.018+0000 7f7038cccb80 1 --2- 172.21.15.19:0/1706880680 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557bd8cf49d0 0x557bd8cf6df0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.018+0000 7f7038cccb80 1 --2- 172.21.15.19:0/1706880680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557bd8afa4a0 0x557bd8c897f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.018+0000 7f70037fe700 0 client.26434 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.018+0000 7f7038cccb80 1 --2- 172.21.15.19:0/1706880680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bd8b74d00 0x557bd8c7dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.018+0000 7f7038cccb80 1 -- 172.21.15.19:0/1706880680 >> 172.21.15.19:0/1706880680 conn(0x557bd8afdf70 msgr2=0x557bd8c7c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.019+0000 7f7038cccb80 1 -- 172.21.15.19:0/1706880680 shutdown_connections 2024-05-09T02:22:21.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.021+0000 7f7038cccb80 1 -- 172.21.15.19:0/1706880680 wait complete. 2024-05-09T02:22:21.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.021+0000 7f7038cccb80 20 client.26434 trim_cache size 0 max 16384 2024-05-09T02:22:21.025 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:22:21.025 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T02:22:21.025 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T02:22:21.025 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:22:21.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.026+0000 7f702a2b6700 1 -- 172.21.15.19:0/410427294 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f700c0b2a70 msgr2=0x7f700c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:21.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.026+0000 7f702a2b6700 1 --2- 172.21.15.19:0/410427294 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f700c0b2a70 0x7f700c0b4f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f70180a5ac0 tx=0x7f70180cc040 comp rx=0 tx=0).stop 2024-05-09T02:22:21.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.026+0000 7f702a2b6700 1 -- 172.21.15.19:0/410427294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7024107df0 msgr2=0x7f702419c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:21.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.027+0000 7f702a2b6700 1 --2- 172.21.15.19:0/410427294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7024107df0 0x7f702419c170 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f701800e970 tx=0x7f701800ec80 comp rx=0 tx=0).stop 2024-05-09T02:22:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.028+0000 7f702a2b6700 1 -- 172.21.15.19:0/410427294 shutdown_connections 2024-05-09T02:22:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.028+0000 7f702a2b6700 1 --2- 172.21.15.19:0/410427294 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f700c0b2a70 0x7f700c0b4f80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.028+0000 7f702a2b6700 1 --2- 172.21.15.19:0/410427294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7024108740 0x7f702419c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.028+0000 7f702a2b6700 1 --2- 172.21.15.19:0/410427294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70241034c0 0x7f702419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:21.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.028+0000 7f702a2b6700 1 --2- 172.21.15.19:0/410427294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7024107df0 0x7f702419c170 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:21.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.028+0000 7f702a2b6700 1 -- 172.21.15.19:0/410427294 >> 172.21.15.19:0/410427294 conn(0x7f70240fedc0 msgr2=0x7f7024101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:21.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.029+0000 7f702a2b6700 1 -- 172.21.15.19:0/410427294 shutdown_connections 2024-05-09T02:22:21.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:21.029+0000 7f702a2b6700 1 -- 172.21.15.19:0/410427294 wait complete. 2024-05-09T02:22:21.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:21 smithi019 ceph-mon[30151]: pgmap v345: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.3 MiB/s wr, 597 op/s 2024-05-09T02:22:21.775 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T02:22:21.776 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:22:21.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:21 smithi059 ceph-mon[34781]: pgmap v345: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.3 MiB/s wr, 597 op/s 2024-05-09T02:22:22.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:21 smithi196 ceph-mon[34718]: pgmap v345: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.3 MiB/s wr, 597 op/s 2024-05-09T02:22:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:22:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 640.0 s, 1120.8 tps, lat 89.210 ms stddev 94.056 2024-05-09T02:22:24.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.272+0000 7f7069246700 1 -- 172.21.15.19:0/4199586310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70641034e0 msgr2=0x7f70641038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:24.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.272+0000 7f7069246700 1 --2- 172.21.15.19:0/4199586310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70641034e0 0x7f70641038f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f70540099c0 tx=0x7f7054009cd0 comp rx=0 tx=0).stop 2024-05-09T02:22:24.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.272+0000 7f7069246700 1 -- 172.21.15.19:0/4199586310 shutdown_connections 2024-05-09T02:22:24.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.272+0000 7f7069246700 1 --2- 172.21.15.19:0/4199586310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7064108760 0x7f70641100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:24.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.272+0000 7f7069246700 1 --2- 172.21.15.19:0/4199586310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70641034e0 0x7f70641038f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:24.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.272+0000 7f7069246700 1 --2- 172.21.15.19:0/4199586310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7064107e10 0x7f7064108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:24.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.272+0000 7f7069246700 1 -- 172.21.15.19:0/4199586310 >> 172.21.15.19:0/4199586310 conn(0x7f70640fef30 msgr2=0x7f7064101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:24.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.272+0000 7f7069246700 1 -- 172.21.15.19:0/4199586310 shutdown_connections 2024-05-09T02:22:24.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.272+0000 7f7069246700 1 -- 172.21.15.19:0/4199586310 wait complete. 2024-05-09T02:22:24.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.274+0000 7f7069246700 1 Processor -- start 2024-05-09T02:22:24.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.274+0000 7f7069246700 1 -- start start 2024-05-09T02:22:24.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.274+0000 7f7069246700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70641034e0 0x7f70641a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:24.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.274+0000 7f7069246700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7064107e10 0x7f7064198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:24.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.274+0000 7f7069246700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7064108760 0x7f70641990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:24.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.274+0000 7f7069246700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7064113390 con 0x7f7064107e10 2024-05-09T02:22:24.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.274+0000 7f7069246700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7064113210 con 0x7f70641034e0 2024-05-09T02:22:24.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.274+0000 7f7069246700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7064113510 con 0x7f7064108760 2024-05-09T02:22:24.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.275+0000 7f706359e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7064108760 0x7f70641990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:24.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.275+0000 7f706359e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7064108760 0x7f70641990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36034/0 (socket says 172.21.15.19:36034) 2024-05-09T02:22:24.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.275+0000 7f706359e700 1 -- 172.21.15.19:0/1640066639 learned_addr learned my addr 172.21.15.19:0/1640066639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:24.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.275+0000 7f706259c700 1 --2- 172.21.15.19:0/1640066639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7064107e10 0x7f7064198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:24.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.275+0000 7f706359e700 1 -- 172.21.15.19:0/1640066639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70641034e0 msgr2=0x7f70641a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:24.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.275+0000 7f706359e700 1 --2- 172.21.15.19:0/1640066639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70641034e0 0x7f70641a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:24.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.275+0000 7f706359e700 1 -- 172.21.15.19:0/1640066639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7064107e10 msgr2=0x7f7064198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:24.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.275+0000 7f706359e700 1 --2- 172.21.15.19:0/1640066639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7064107e10 0x7f7064198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:24.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.275+0000 7f706359e700 1 -- 172.21.15.19:0/1640066639 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7064199900 con 0x7f7064108760 2024-05-09T02:22:24.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.276+0000 7f706359e700 1 --2- 172.21.15.19:0/1640066639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7064108760 0x7f70641990d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f705c00ef30 tx=0x7f705c00c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:24.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.276+0000 7f704bfff700 1 -- 172.21.15.19:0/1640066639 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c0092e0 con 0x7f7064108760 2024-05-09T02:22:24.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.276+0000 7f704bfff700 1 -- 172.21.15.19:0/1640066639 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f705c004bb0 con 0x7f7064108760 2024-05-09T02:22:24.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.276+0000 7f704bfff700 1 -- 172.21.15.19:0/1640066639 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c004280 con 0x7f7064108760 2024-05-09T02:22:24.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.276+0000 7f7069246700 1 -- 172.21.15.19:0/1640066639 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70641b6100 con 0x7f7064108760 2024-05-09T02:22:24.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.276+0000 7f7069246700 1 -- 172.21.15.19:0/1640066639 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70641b6530 con 0x7f7064108760 2024-05-09T02:22:24.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.278+0000 7f7069246700 1 -- 172.21.15.19:0/1640066639 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7064104990 con 0x7f7064108760 2024-05-09T02:22:24.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.279+0000 7f704bfff700 1 -- 172.21.15.19:0/1640066639 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f705c01b030 con 0x7f7064108760 2024-05-09T02:22:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.280+0000 7f704bfff700 1 --2- 172.21.15.19:0/1640066639 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f704c0b2a70 0x7f704c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.281+0000 7f704bfff700 1 -- 172.21.15.19:0/1640066639 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f705c0e21e0 con 0x7f7064108760 2024-05-09T02:22:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.281+0000 7f7062d9d700 1 --2- 172.21.15.19:0/1640066639 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f704c0b2a70 0x7f704c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.282+0000 7f7062d9d700 1 --2- 172.21.15.19:0/1640066639 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f704c0b2a70 0x7f704c0b4f80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f70540098e0 tx=0x7f7054000bc0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.288+0000 7f704bfff700 1 -- 172.21.15.19:0/1640066639 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f705c022020 con 0x7f7064108760 2024-05-09T02:22:24.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:24 smithi059 ceph-mon[34781]: pgmap v346: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.3 MiB/s wr, 596 op/s 2024-05-09T02:22:24.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.723+0000 7f7069246700 1 -- 172.21.15.19:0/1640066639 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70640512c0 con 0x7f7064108760 2024-05-09T02:22:24.727 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:24 smithi019 ceph-mon[30151]: pgmap v346: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.3 MiB/s wr, 596 op/s 2024-05-09T02:22:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.725+0000 7f704bfff700 1 -- 172.21.15.19:0/1640066639 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11917 (secure 0 0 0) 0x7f705c010040 con 0x7f7064108760 2024-05-09T02:22:24.728 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:22:24.729 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:22:01.074232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24457,24455,24473,24463,24484]},"id":1}]} 2024-05-09T02:22:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.732+0000 7f7069246700 1 -- 172.21.15.19:0/1640066639 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f704c0b2a70 msgr2=0x7f704c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.732+0000 7f7069246700 1 --2- 172.21.15.19:0/1640066639 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f704c0b2a70 0x7f704c0b4f80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f70540098e0 tx=0x7f7054000bc0 comp rx=0 tx=0).stop 2024-05-09T02:22:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.732+0000 7f7069246700 1 -- 172.21.15.19:0/1640066639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7064108760 msgr2=0x7f70641990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.732+0000 7f7069246700 1 --2- 172.21.15.19:0/1640066639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7064108760 0x7f70641990d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f705c00ef30 tx=0x7f705c00c450 comp rx=0 tx=0).stop 2024-05-09T02:22:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.735+0000 7f7069246700 1 -- 172.21.15.19:0/1640066639 shutdown_connections 2024-05-09T02:22:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.735+0000 7f7069246700 1 --2- 172.21.15.19:0/1640066639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7064108760 0x7f70641990d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.735+0000 7f7069246700 1 --2- 172.21.15.19:0/1640066639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70641034e0 0x7f70641a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.735+0000 7f7069246700 1 --2- 172.21.15.19:0/1640066639 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f704c0b2a70 0x7f704c0b4f80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.735+0000 7f7069246700 1 --2- 172.21.15.19:0/1640066639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7064107e10 0x7f7064198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.735+0000 7f7069246700 1 -- 172.21.15.19:0/1640066639 >> 172.21.15.19:0/1640066639 conn(0x7f70640fef30 msgr2=0x7f7064101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.738+0000 7f7069246700 1 -- 172.21.15.19:0/1640066639 shutdown_connections 2024-05-09T02:22:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:24.740+0000 7f7069246700 1 -- 172.21.15.19:0/1640066639 wait complete. 2024-05-09T02:22:24.746 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-05-09T02:22:24.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:24 smithi196 ceph-mon[34718]: pgmap v346: 97 pgs: 97 active+clean; 4.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.3 MiB/s wr, 596 op/s 2024-05-09T02:22:25.470 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T02:22:25.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:25 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1640066639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:22:25.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:25 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1640066639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:22:25.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:25 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1640066639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:22:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:22:26.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:26 smithi019 ceph-mon[30151]: pgmap v347: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.4 MiB/s wr, 630 op/s 2024-05-09T02:22:26.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:26 smithi059 ceph-mon[34781]: pgmap v347: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.4 MiB/s wr, 630 op/s 2024-05-09T02:22:26.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:26 smithi196 ceph-mon[34718]: pgmap v347: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.4 MiB/s wr, 630 op/s 2024-05-09T02:22:27.656 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 645.0 s, 1219.0 tps, lat 82.639 ms stddev 93.531 2024-05-09T02:22:27.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.741+0000 7f6cd69a2700 1 -- 172.21.15.19:0/1586893714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd01032e0 msgr2=0x7f6cd01036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:27.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.741+0000 7f6cd69a2700 1 --2- 172.21.15.19:0/1586893714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd01032e0 0x7f6cd01036f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800b0d0 tx=0x7f6cc800b3e0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.748+0000 7f6cd69a2700 1 -- 172.21.15.19:0/1586893714 shutdown_connections 2024-05-09T02:22:27.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.748+0000 7f6cd69a2700 1 --2- 172.21.15.19:0/1586893714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cd0108560 0x7f6cd010fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.748+0000 7f6cd69a2700 1 --2- 172.21.15.19:0/1586893714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd01032e0 0x7f6cd01036f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.748+0000 7f6cd69a2700 1 --2- 172.21.15.19:0/1586893714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd0107c10 0x7f6cd0108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.748+0000 7f6cd69a2700 1 -- 172.21.15.19:0/1586893714 >> 172.21.15.19:0/1586893714 conn(0x7f6cd0078ac0 msgr2=0x7f6cd0078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:27.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.752+0000 7f6cd69a2700 1 -- 172.21.15.19:0/1586893714 shutdown_connections 2024-05-09T02:22:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.754+0000 7f6cd69a2700 1 -- 172.21.15.19:0/1586893714 wait complete. 2024-05-09T02:22:27.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.755+0000 7f6cd69a2700 1 Processor -- start 2024-05-09T02:22:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.757+0000 7f6cd69a2700 1 -- start start 2024-05-09T02:22:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.757+0000 7f6cd69a2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cd01032e0 0x7f6cd010c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.757+0000 7f6cd69a2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd0107c10 0x7f6cd010c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.757+0000 7f6cd69a2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd0108560 0x7f6cd01ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.758+0000 7f6cd59a0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cd01032e0 0x7f6cd010c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.758+0000 7f6cd59a0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cd01032e0 0x7f6cd010c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59006/0 (socket says 172.21.15.19:59006) 2024-05-09T02:22:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.758+0000 7f6cd59a0700 1 -- 172.21.15.19:0/3317931535 learned_addr learned my addr 172.21.15.19:0/3317931535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.758+0000 7f6cd69a2700 1 -- 172.21.15.19:0/3317931535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd0113320 con 0x7f6cd01032e0 2024-05-09T02:22:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.758+0000 7f6cd69a2700 1 -- 172.21.15.19:0/3317931535 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01131a0 con 0x7f6cd0108560 2024-05-09T02:22:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.758+0000 7f6cd69a2700 1 -- 172.21.15.19:0/3317931535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01134a0 con 0x7f6cd0107c10 2024-05-09T02:22:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.762+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/3317931535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd0108560 0x7f6cd01ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.764+0000 7f6cd59a0700 1 -- 172.21.15.19:0/3317931535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd0107c10 msgr2=0x7f6cd010c990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T02:22:27.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.764+0000 7f6cd59a0700 1 --2- 172.21.15.19:0/3317931535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd0107c10 0x7f6cd010c990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.764+0000 7f6cd59a0700 1 -- 172.21.15.19:0/3317931535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd0108560 msgr2=0x7f6cd01ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:27.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.764+0000 7f6cd59a0700 1 --2- 172.21.15.19:0/3317931535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd0108560 0x7f6cd01ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.764+0000 7f6cd59a0700 1 -- 172.21.15.19:0/3317931535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd010d240 con 0x7f6cd01032e0 2024-05-09T02:22:27.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.765+0000 7f6cd59a0700 1 --2- 172.21.15.19:0/3317931535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cd01032e0 0x7f6cd010c450 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8000f80 tx=0x7f6cc8007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:27.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.767+0000 7f6cc6ffd700 1 -- 172.21.15.19:0/3317931535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8003c00 con 0x7f6cd01032e0 2024-05-09T02:22:27.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.767+0000 7f6cc6ffd700 1 -- 172.21.15.19:0/3317931535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cc8003da0 con 0x7f6cd01032e0 2024-05-09T02:22:27.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.771+0000 7f6cd69a2700 1 -- 172.21.15.19:0/3317931535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd019ccc0 con 0x7f6cd01032e0 2024-05-09T02:22:27.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.771+0000 7f6cd69a2700 1 -- 172.21.15.19:0/3317931535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd019d150 con 0x7f6cd01032e0 2024-05-09T02:22:27.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.771+0000 7f6cc6ffd700 1 -- 172.21.15.19:0/3317931535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc802c970 con 0x7f6cd01032e0 2024-05-09T02:22:27.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.778+0000 7f6ce661ab80 1 Processor -- start 2024-05-09T02:22:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.778+0000 7f6ce661ab80 1 -- start start 2024-05-09T02:22:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.779+0000 7f6ce661ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc5663160 0x562dc56fc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.779+0000 7f6ce661ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc55794a0 0x562dc56fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.779+0000 7f6ce661ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc56f6800 0x562dc56f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.779+0000 7f6ce661ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562dc5618ff0 con 0x562dc55794a0 2024-05-09T02:22:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.779+0000 7f6ce661ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562dc55b77f0 con 0x562dc56f6800 2024-05-09T02:22:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.779+0000 7f6ce661ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562dc5614fb0 con 0x562dc5663160 2024-05-09T02:22:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.780+0000 7f6cd61a1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc5663160 0x562dc56fc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.780+0000 7f6cd61a1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc5663160 0x562dc56fc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:59290/0 (socket says 172.21.15.19:59290) 2024-05-09T02:22:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.780+0000 7f6cd61a1700 1 -- 172.21.15.19:0/1704051578 learned_addr learned my addr 172.21.15.19:0/1704051578 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:27.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.784+0000 7f6cc6ffd700 1 -- 172.21.15.19:0/3317931535 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6cc802a040 con 0x7f6cd01032e0 2024-05-09T02:22:27.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.785+0000 7f6cd519f700 1 --2- 172.21.15.19:0/1704051578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc56f6800 0x562dc56f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.785+0000 7f6cd59a0700 1 --2- 172.21.15.19:0/1704051578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc55794a0 0x562dc56fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.786+0000 7f6cd61a1700 1 -- 172.21.15.19:0/1704051578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc56f6800 msgr2=0x562dc56f6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:27.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.786+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/1704051578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc56f6800 0x562dc56f6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.786+0000 7f6cd61a1700 1 -- 172.21.15.19:0/1704051578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc55794a0 msgr2=0x562dc56fcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:27.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.786+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/1704051578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc55794a0 0x562dc56fcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.786+0000 7f6cd61a1700 1 -- 172.21.15.19:0/1704051578 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dc55af370 con 0x562dc5663160 2024-05-09T02:22:27.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.786+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/1704051578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc5663160 0x562dc56fc640 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00b810 tx=0x7f6ccc00bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec8ee9bd82ef4e10 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:27.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.787+0000 7f6cbbfff700 1 -- 172.21.15.19:0/1704051578 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc014620 con 0x562dc5663160 2024-05-09T02:22:27.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.787+0000 7f6cbbfff700 1 -- 172.21.15.19:0/1704051578 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ccc014c70 con 0x562dc5663160 2024-05-09T02:22:27.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.787+0000 7f6cc6ffd700 1 --2- 172.21.15.19:0/3317931535 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6cbc0b29d0 0x7f6cbc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.787+0000 7f6cbbfff700 1 -- 172.21.15.19:0/1704051578 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc012d50 con 0x562dc5663160 2024-05-09T02:22:27.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.788+0000 7f6cc6ffd700 1 -- 172.21.15.19:0/3317931535 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f6cc80f4a20 con 0x7f6cd01032e0 2024-05-09T02:22:27.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.788+0000 7f6ce661ab80 1 -- 172.21.15.19:0/1704051578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc5663160 msgr2=0x562dc56fc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:27.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.788+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/1704051578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc5663160 0x562dc56fc640 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00b810 tx=0x7f6ccc00bb20 comp rx=0 tx=0).stop 2024-05-09T02:22:27.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.788+0000 7f6ce661ab80 1 -- 172.21.15.19:0/1704051578 shutdown_connections 2024-05-09T02:22:27.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.788+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/1704051578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc56f6800 0x562dc56f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.788+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/1704051578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc5663160 0x562dc56fc640 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.788+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/1704051578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc55794a0 0x562dc56fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.788+0000 7f6ce661ab80 1 -- 172.21.15.19:0/1704051578 >> 172.21.15.19:0/1704051578 conn(0x562dc55d96a0 msgr2=0x562dc56f1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:27.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.791+0000 7f6cd59a0700 1 --2- 172.21.15.19:0/3317931535 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6cbc0b29d0 0x7f6cbc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.792+0000 7f6ce661ab80 1 -- 172.21.15.19:0/1704051578 shutdown_connections 2024-05-09T02:22:27.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.792+0000 7f6ce661ab80 1 -- 172.21.15.19:0/1704051578 wait complete. 2024-05-09T02:22:27.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.793+0000 7f6ce661ab80 1 -- start start 2024-05-09T02:22:27.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.793+0000 7f6ce661ab80 1 Processor -- start 2024-05-09T02:22:27.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.793+0000 7f6cd59a0700 1 --2- 172.21.15.19:0/3317931535 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6cbc0b29d0 0x7f6cbc0b4ee0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x562dc56f3090 tx=0x7f6cc8009040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:27.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.798+0000 7f6ce661ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc55794a0 0x562dc5700300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.798+0000 7f6ce661ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc5663160 0x562dc5700840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.798+0000 7f6ce661ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc56f6800 0x562dc5700d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.798+0000 7f6ce661ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562dc5494ba0 con 0x562dc55794a0 2024-05-09T02:22:27.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.798+0000 7f6ce661ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562dc55d3150 con 0x562dc56f6800 2024-05-09T02:22:27.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.798+0000 7f6ce661ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562dc56f7640 con 0x562dc5663160 2024-05-09T02:22:27.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.799+0000 7f6cd519f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc5663160 0x562dc5700840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.800+0000 7f6ce661ab80 10 client.0 _ll_register_callbacks cb 0x562dc548a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:27.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.800+0000 7f6cd61a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc55794a0 0x562dc5700300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.800+0000 7f6cd61a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc55794a0 0x562dc5700300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59024/0 (socket says 172.21.15.19:59024) 2024-05-09T02:22:27.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.800+0000 7f6cd61a1700 1 -- 172.21.15.19:0/2191734450 learned_addr learned my addr 172.21.15.19:0/2191734450 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:27.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.800+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/2191734450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc56f6800 0x562dc5700d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.801+0000 7f6cd61a1700 1 -- 172.21.15.19:0/2191734450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc5663160 msgr2=0x562dc5700840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:27.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.801+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/2191734450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc5663160 0x562dc5700840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.801+0000 7f6cd61a1700 1 -- 172.21.15.19:0/2191734450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc55794a0 msgr2=0x562dc5700300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:27.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.801+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/2191734450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc55794a0 0x562dc5700300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.801+0000 7f6cd61a1700 1 -- 172.21.15.19:0/2191734450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dc5494490 con 0x562dc56f6800 2024-05-09T02:22:27.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.802+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/2191734450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc56f6800 0x562dc5700d80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x562dc56ffb80 tx=0x7f6ccc008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45149c80840c4c1a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:27.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.802+0000 7f6cb97fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:22:27.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.802+0000 7f6cb97fa700 1 -- 172.21.15.19:0/2191734450 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc012990 con 0x562dc56f6800 2024-05-09T02:22:27.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.802+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2191734450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ccc014c70 con 0x562dc56f6800 2024-05-09T02:22:27.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.802+0000 7f6cb97fa700 1 -- 172.21.15.19:0/2191734450 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f6ccc00f070 con 0x562dc56f6800 2024-05-09T02:22:27.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.802+0000 7f6cb97fa700 1 -- 172.21.15.19:0/2191734450 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6ccc012b30 con 0x562dc56f6800 2024-05-09T02:22:27.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.803+0000 7f6ce661ab80 10 client.26411 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:27.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.803+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2191734450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x562dc570dcb0 con 0x562dc56f6800 2024-05-09T02:22:27.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.803+0000 7f6cb97fa700 1 -- 172.21.15.19:0/2191734450 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ccc014c70 con 0x562dc56f6800 2024-05-09T02:22:27.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.803+0000 7f6cb97fa700 1 -- 172.21.15.19:0/2191734450 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f6ccc02f020 con 0x562dc56f6800 2024-05-09T02:22:27.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.805+0000 7f6ce661ab80 10 client.26411 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:27.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.805+0000 7f6ce661ab80 10 client.26411 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:22:27.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.805+0000 7f6ce661ab80 20 client.26411 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:27.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.805+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/2191734450 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x562dc5710540 0x562dc5712990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.805+0000 7f6ce661ab80 4 client.26411 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:22:27.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.805+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2191734450 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dc56269d0 con 0x562dc5710540 2024-05-09T02:22:27.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.811+0000 7f6cd519f700 1 --2- 172.21.15.19:0/2191734450 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x562dc5710540 0x562dc5712990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.814+0000 7f6cb97fa700 10 client.26411 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:22:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.814+0000 7f6cd519f700 1 --2- 172.21.15.19:0/2191734450 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x562dc5710540 0x562dc5712990 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5405f861d7d54f1d server_cookie=a8d1b4fa9c3d3a44 in_seq=0 out_seq=0 2024-05-09T02:22:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.815+0000 7f6cb97fa700 1 -- 172.21.15.19:0/2191734450 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562dc56269d0 con 0x562dc5710540 2024-05-09T02:22:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.815+0000 7f6cb97fa700 10 client.26411 handle_command_reply: tid=0 2024-05-09T02:22:27.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.816+0000 7f6ce661ab80 1 client.26411 shutdown 2024-05-09T02:22:27.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.818+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2191734450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc56f6800 msgr2=0x562dc5700d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:27.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.818+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/2191734450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc56f6800 0x562dc5700d80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x562dc56ffb80 tx=0x7f6ccc008040 comp rx=0 tx=0).stop 2024-05-09T02:22:27.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.821+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2191734450 shutdown_connections 2024-05-09T02:22:27.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.821+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/2191734450 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x562dc5710540 0x562dc5712990 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.821+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/2191734450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc56f6800 0x562dc5700d80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.821+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/2191734450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc55794a0 0x562dc5700300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.821+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/2191734450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc5663160 0x562dc5700840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.821+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2191734450 >> 172.21.15.19:0/2191734450 conn(0x562dc55d96a0 msgr2=0x562dc56f1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:27.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.821+0000 7f6cb97fa700 0 client.26411 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:22:27.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.824+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2191734450 shutdown_connections 2024-05-09T02:22:27.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.824+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2191734450 wait complete. 2024-05-09T02:22:27.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.824+0000 7f6ce661ab80 20 client.26411 trim_cache size 0 max 16384 2024-05-09T02:22:27.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.848+0000 7f6ce661ab80 1 Processor -- start 2024-05-09T02:22:27.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.848+0000 7f6ce661ab80 1 -- start start 2024-05-09T02:22:27.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.849+0000 7f6ce661ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc55794a0 0x562dc5700bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.849+0000 7f6ce661ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc5663160 0x562dc570c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.849+0000 7f6cd61a1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc55794a0 0x562dc5700bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.849+0000 7f6cd61a1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc55794a0 0x562dc5700bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:52732/0 (socket says 172.21.15.19:52732) 2024-05-09T02:22:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.849+0000 7f6cd61a1700 1 -- 172.21.15.19:0/4276473122 learned_addr learned my addr 172.21.15.19:0/4276473122 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.849+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/4276473122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc56f6800 0x562dc5718190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.849+0000 7f6ce661ab80 1 -- 172.21.15.19:0/4276473122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01a4480 con 0x562dc5663160 2024-05-09T02:22:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.850+0000 7f6ce661ab80 1 -- 172.21.15.19:0/4276473122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01ae550 con 0x562dc55794a0 2024-05-09T02:22:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.850+0000 7f6cd519f700 1 --2- 172.21.15.19:0/4276473122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc5663160 0x562dc570c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.850+0000 7f6ce661ab80 1 -- 172.21.15.19:0/4276473122 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01a4b50 con 0x562dc56f6800 2024-05-09T02:22:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.850+0000 7f6cd61a1700 1 -- 172.21.15.19:0/4276473122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc56f6800 msgr2=0x562dc5718190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.850+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/4276473122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc56f6800 0x562dc5718190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.850+0000 7f6cd61a1700 1 -- 172.21.15.19:0/4276473122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc5663160 msgr2=0x562dc570c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.850+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/4276473122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc5663160 0x562dc570c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.850+0000 7f6cd61a1700 1 -- 172.21.15.19:0/4276473122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dc57032e0 con 0x562dc55794a0 2024-05-09T02:22:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.850+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/4276473122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc55794a0 0x562dc5700bc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc000be0 tx=0x7f6ccc006df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a95d5653b83b14c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.850+0000 7f6cbb7fe700 1 -- 172.21.15.19:0/4276473122 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc036070 con 0x562dc55794a0 2024-05-09T02:22:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.851+0000 7f6cbb7fe700 1 -- 172.21.15.19:0/4276473122 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ccc014c40 con 0x562dc55794a0 2024-05-09T02:22:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.851+0000 7f6cbb7fe700 1 -- 172.21.15.19:0/4276473122 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc005a50 con 0x562dc55794a0 2024-05-09T02:22:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.852+0000 7f6ce661ab80 1 -- 172.21.15.19:0/4276473122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc55794a0 msgr2=0x562dc5700bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.852+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/4276473122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc55794a0 0x562dc5700bc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc000be0 tx=0x7f6ccc006df0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.852+0000 7f6ce661ab80 1 -- 172.21.15.19:0/4276473122 shutdown_connections 2024-05-09T02:22:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.852+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/4276473122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc56f6800 0x562dc5718190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.852+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/4276473122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc55794a0 0x562dc5700bc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.852+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/4276473122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc5663160 0x562dc570c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.852+0000 7f6ce661ab80 1 -- 172.21.15.19:0/4276473122 >> 172.21.15.19:0/4276473122 conn(0x562dc55d96a0 msgr2=0x562dc56febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.852+0000 7f6ce661ab80 1 -- 172.21.15.19:0/4276473122 shutdown_connections 2024-05-09T02:22:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.853+0000 7f6ce661ab80 1 -- 172.21.15.19:0/4276473122 wait complete. 2024-05-09T02:22:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.853+0000 7f6ce661ab80 1 -- start start 2024-05-09T02:22:27.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.853+0000 7f6ce661ab80 1 Processor -- start 2024-05-09T02:22:27.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.854+0000 7f6ce661ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc55794a0 0x562dc5708580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.854+0000 7f6ce661ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc5663160 0x562dc5708ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.854+0000 7f6ce661ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc56f6800 0x562dc570c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.854+0000 7f6ce661ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01ade80 con 0x562dc56f6800 2024-05-09T02:22:27.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.854+0000 7f6ce661ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01a5220 con 0x562dc5663160 2024-05-09T02:22:27.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.854+0000 7f6ce661ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01a2440 con 0x562dc55794a0 2024-05-09T02:22:27.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.854+0000 7f6ce661ab80 10 client.0 _ll_register_callbacks cb 0x562dc548a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:27.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.854+0000 7f6cd519f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc5663160 0x562dc5708ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.854+0000 7f6cd519f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc5663160 0x562dc5708ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:52742/0 (socket says 172.21.15.19:52742) 2024-05-09T02:22:27.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.854+0000 7f6cd519f700 1 -- 172.21.15.19:0/2496164265 learned_addr learned my addr 172.21.15.19:0/2496164265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:27.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.854+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/2496164265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc56f6800 0x562dc570c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.854+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/2496164265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc55794a0 0x562dc5708580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.855+0000 7f6cd519f700 1 -- 172.21.15.19:0/2496164265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc55794a0 msgr2=0x562dc5708580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:27.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.855+0000 7f6cd519f700 1 --2- 172.21.15.19:0/2496164265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc55794a0 0x562dc5708580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.855+0000 7f6cd519f700 1 -- 172.21.15.19:0/2496164265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc56f6800 msgr2=0x562dc570c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:27.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.855+0000 7f6cd519f700 1 --2- 172.21.15.19:0/2496164265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc56f6800 0x562dc570c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.855+0000 7f6cd519f700 1 -- 172.21.15.19:0/2496164265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dc570c7a0 con 0x562dc5663160 2024-05-09T02:22:27.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.855+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/2496164265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc56f6800 0x562dc570c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:22:27.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.855+0000 7f6cd519f700 1 --2- 172.21.15.19:0/2496164265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc5663160 0x562dc5708ac0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00047b0 tx=0x7f6cc000cd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd7f9a7712f2eb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:27.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.855+0000 7f6cb97fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:22:27.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.855+0000 7f6cb97fa700 1 -- 172.21.15.19:0/2496164265 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0017b40 con 0x562dc5663160 2024-05-09T02:22:27.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.855+0000 7f6cb97fa700 1 -- 172.21.15.19:0/2496164265 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f6cc0017ce0 con 0x562dc5663160 2024-05-09T02:22:27.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.855+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2496164265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ccc014c40 con 0x562dc5663160 2024-05-09T02:22:27.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.855+0000 7f6cb97fa700 1 -- 172.21.15.19:0/2496164265 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6cc0005a00 con 0x562dc5663160 2024-05-09T02:22:27.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.856+0000 7f6ce661ab80 10 client.26417 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:27.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.856+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2496164265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x562dc5717ef0 con 0x562dc5663160 2024-05-09T02:22:27.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.856+0000 7f6cb97fa700 1 -- 172.21.15.19:0/2496164265 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ccc014c40 con 0x562dc5663160 2024-05-09T02:22:27.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.856+0000 7f6cb97fa700 1 -- 172.21.15.19:0/2496164265 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f6cc0020020 con 0x562dc5663160 2024-05-09T02:22:27.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.856+0000 7f6ce661ab80 10 client.26417 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:27.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.856+0000 7f6ce661ab80 10 client.26417 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:22:27.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.856+0000 7f6ce661ab80 20 client.26417 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:27.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.856+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/2496164265 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x562dc577a1a0 0x562dc577c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:27.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.856+0000 7f6ce661ab80 4 client.26417 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:22:27.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.856+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2496164265 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562dc5620a70 con 0x562dc577a1a0 2024-05-09T02:22:27.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.856+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/2496164265 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x562dc577a1a0 0x562dc577c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:27.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.857+0000 7f6cb97fa700 10 client.26417 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:22:27.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.857+0000 7f6cd61a1700 1 --2- 172.21.15.19:0/2496164265 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x562dc577a1a0 0x562dc577c5c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=718733a500841964 server_cookie=cb3f8ac84465d6c8 in_seq=0 out_seq=0 2024-05-09T02:22:27.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.857+0000 7f6cb97fa700 1 -- 172.21.15.19:0/2496164265 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562dc5620a70 con 0x562dc577a1a0 2024-05-09T02:22:27.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.857+0000 7f6cb97fa700 10 client.26417 handle_command_reply: tid=0 2024-05-09T02:22:27.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.857+0000 7f6ce661ab80 1 client.26417 shutdown 2024-05-09T02:22:27.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.857+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2496164265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc5663160 msgr2=0x562dc5708ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:27.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.857+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/2496164265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc5663160 0x562dc5708ac0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00047b0 tx=0x7f6cc000cd60 comp rx=0 tx=0).stop 2024-05-09T02:22:27.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.857+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2496164265 shutdown_connections 2024-05-09T02:22:27.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.857+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/2496164265 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x562dc577a1a0 0x562dc577c5c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.857+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/2496164265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562dc56f6800 0x562dc570c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.857+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/2496164265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562dc55794a0 0x562dc5708580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.858+0000 7f6ce661ab80 1 --2- 172.21.15.19:0/2496164265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562dc5663160 0x562dc5708ac0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.858+0000 7f6cb97fa700 0 client.26417 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:22:27.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.858+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2496164265 >> 172.21.15.19:0/2496164265 conn(0x562dc55d96a0 msgr2=0x562dc56fb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:27.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.858+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2496164265 shutdown_connections 2024-05-09T02:22:27.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.858+0000 7f6ce661ab80 1 -- 172.21.15.19:0/2496164265 wait complete. 2024-05-09T02:22:27.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.858+0000 7f6ce661ab80 20 client.26417 trim_cache size 0 max 16384 2024-05-09T02:22:27.865 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:22:27.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.858+0000 7f6cd69a2700 1 -- 172.21.15.19:0/3317931535 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6cbc0b29d0 msgr2=0x7f6cbc0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:27.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.858+0000 7f6cd69a2700 1 --2- 172.21.15.19:0/3317931535 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6cbc0b29d0 0x7f6cbc0b4ee0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x562dc56f3090 tx=0x7f6cc8009040 comp rx=0 tx=0).stop 2024-05-09T02:22:27.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.858+0000 7f6cd69a2700 1 -- 172.21.15.19:0/3317931535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cd01032e0 msgr2=0x7f6cd010c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:27.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.858+0000 7f6cd69a2700 1 --2- 172.21.15.19:0/3317931535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cd01032e0 0x7f6cd010c450 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8000f80 tx=0x7f6cc8007a50 comp rx=0 tx=0).stop 2024-05-09T02:22:27.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.859+0000 7f6cd69a2700 1 -- 172.21.15.19:0/3317931535 shutdown_connections 2024-05-09T02:22:27.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.859+0000 7f6cd69a2700 1 --2- 172.21.15.19:0/3317931535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd0108560 0x7f6cd01ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.859+0000 7f6cd69a2700 1 --2- 172.21.15.19:0/3317931535 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6cbc0b29d0 0x7f6cbc0b4ee0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.859+0000 7f6cd69a2700 1 --2- 172.21.15.19:0/3317931535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cd01032e0 0x7f6cd010c450 secure :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8000f80 tx=0x7f6cc8007a50 comp rx=0 tx=0).stop 2024-05-09T02:22:27.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.859+0000 7f6cd69a2700 1 --2- 172.21.15.19:0/3317931535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd0107c10 0x7f6cd010c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:27.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.859+0000 7f6cd69a2700 1 -- 172.21.15.19:0/3317931535 >> 172.21.15.19:0/3317931535 conn(0x7f6cd0078ac0 msgr2=0x7f6cd0101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:27.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.859+0000 7f6cd69a2700 1 -- 172.21.15.19:0/3317931535 shutdown_connections 2024-05-09T02:22:27.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:27.859+0000 7f6cd69a2700 1 -- 172.21.15.19:0/3317931535 wait complete. 2024-05-09T02:22:28.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:28 smithi019 ceph-mon[30151]: pgmap v348: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.4 MiB/s wr, 511 op/s 2024-05-09T02:22:28.573 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T02:22:28.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:28 smithi059 ceph-mon[34781]: pgmap v348: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.4 MiB/s wr, 511 op/s 2024-05-09T02:22:28.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:28 smithi196 ceph-mon[34718]: pgmap v348: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.4 MiB/s wr, 511 op/s 2024-05-09T02:22:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:22:30.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:30 smithi059 ceph-mon[34781]: pgmap v349: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.2 MiB/s wr, 702 op/s 2024-05-09T02:22:30.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:30 smithi196 ceph-mon[34718]: pgmap v349: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.2 MiB/s wr, 702 op/s 2024-05-09T02:22:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:30 smithi019 ceph-mon[30151]: pgmap v349: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.2 MiB/s wr, 702 op/s 2024-05-09T02:22:31.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.974+0000 7fb4524cc700 1 -- 172.21.15.19:0/355507684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb44c103ba0 msgr2=0x7fb44c1080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:31.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.974+0000 7fb4524cc700 1 --2- 172.21.15.19:0/355507684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb44c103ba0 0x7fb44c1080d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb434009990 tx=0x7fb434009ca0 comp rx=0 tx=0).stop 2024-05-09T02:22:31.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.975+0000 7fb4524cc700 1 -- 172.21.15.19:0/355507684 shutdown_connections 2024-05-09T02:22:31.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.975+0000 7fb4524cc700 1 --2- 172.21.15.19:0/355507684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c1086a0 0x7fb44c108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:31.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.975+0000 7fb4524cc700 1 --2- 172.21.15.19:0/355507684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb44c103250 0x7fb44c103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:31.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.975+0000 7fb4524cc700 1 --2- 172.21.15.19:0/355507684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb44c103ba0 0x7fb44c1080d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:31.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.975+0000 7fb4524cc700 1 -- 172.21.15.19:0/355507684 >> 172.21.15.19:0/355507684 conn(0x7fb44c0fedc0 msgr2=0x7fb44c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:31.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.975+0000 7fb4524cc700 1 -- 172.21.15.19:0/355507684 shutdown_connections 2024-05-09T02:22:31.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.975+0000 7fb4524cc700 1 -- 172.21.15.19:0/355507684 wait complete. 2024-05-09T02:22:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.976+0000 7fb4524cc700 1 Processor -- start 2024-05-09T02:22:31.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.977+0000 7fb4524cc700 1 -- start start 2024-05-09T02:22:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.977+0000 7fb4524cc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c103250 0x7fb44c1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.977+0000 7fb4524cc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb44c103ba0 0x7fb44c1a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.977+0000 7fb4524cc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb44c1086a0 0x7fb44c1a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.977+0000 7fb4524cc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c11aa70 con 0x7fb44c103250 2024-05-09T02:22:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.977+0000 7fb4524cc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c11a8f0 con 0x7fb44c1086a0 2024-05-09T02:22:31.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.977+0000 7fb4524cc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c11abf0 con 0x7fb44c103ba0 2024-05-09T02:22:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.983+0000 7fb4437fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb44c103ba0 0x7fb44c1a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.983+0000 7fb4437fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb44c103ba0 0x7fb44c1a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:59344/0 (socket says 172.21.15.19:59344) 2024-05-09T02:22:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.983+0000 7fb4437fe700 1 -- 172.21.15.19:0/3436874529 learned_addr learned my addr 172.21.15.19:0/3436874529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:31.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.984+0000 7fb44bfff700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c103250 0x7fb44c1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:31.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.988+0000 7fb450a69700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb44c1086a0 0x7fb44c1a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:31.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.988+0000 7fb4437fe700 1 -- 172.21.15.19:0/3436874529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb44c1086a0 msgr2=0x7fb44c1a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:31.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.988+0000 7fb4437fe700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb44c1086a0 0x7fb44c1a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:31.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.988+0000 7fb4437fe700 1 -- 172.21.15.19:0/3436874529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c103250 msgr2=0x7fb44c1a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:31.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.988+0000 7fb4437fe700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c103250 0x7fb44c1a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:31.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.988+0000 7fb4437fe700 1 -- 172.21.15.19:0/3436874529 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44c1a62b0 con 0x7fb44c103ba0 2024-05-09T02:22:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.991+0000 7fb44bfff700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c103250 0x7fb44c1a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:22:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.991+0000 7fb450a69700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb44c1086a0 0x7fb44c1a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:22:31.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.991+0000 7fb4437fe700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb44c103ba0 0x7fb44c1a1980 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb434005910 tx=0x7fb43400b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:31.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.996+0000 7fb449ffb700 1 -- 172.21.15.19:0/3436874529 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43400fa90 con 0x7fb44c103ba0 2024-05-09T02:22:31.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.996+0000 7fb4524cc700 1 -- 172.21.15.19:0/3436874529 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44c1a6540 con 0x7fb44c103ba0 2024-05-09T02:22:32.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:31.996+0000 7fb4524cc700 1 -- 172.21.15.19:0/3436874529 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44c1a69c0 con 0x7fb44c103ba0 2024-05-09T02:22:32.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.001+0000 7fb460ee2b80 1 Processor -- start 2024-05-09T02:22:32.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.001+0000 7fb449ffb700 1 -- 172.21.15.19:0/3436874529 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb434004ce0 con 0x7fb44c103ba0 2024-05-09T02:22:32.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.001+0000 7fb449ffb700 1 -- 172.21.15.19:0/3436874529 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434005650 con 0x7fb44c103ba0 2024-05-09T02:22:32.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.003+0000 7fb460ee2b80 1 -- start start 2024-05-09T02:22:32.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.004+0000 7fb460ee2b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4bef3d00 0x557d4c0076d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:32.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.004+0000 7fb460ee2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4be794a0 0x557d4c007c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:32.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.004+0000 7fb460ee2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c00bde0 0x557d4c00c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:32.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.004+0000 7fb460ee2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d4bf190a0 con 0x557d4c00bde0 2024-05-09T02:22:32.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.004+0000 7fb460ee2b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557d4bebe9a0 con 0x557d4bef3d00 2024-05-09T02:22:32.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.004+0000 7fb460ee2b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557d4bf15060 con 0x557d4be794a0 2024-05-09T02:22:32.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.005+0000 7fb449ffb700 1 -- 172.21.15.19:0/3436874529 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb434029020 con 0x7fb44c103ba0 2024-05-09T02:22:32.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.006+0000 7fb449ffb700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb4380b2b40 0x7fb4380b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:32.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.007+0000 7fb450a69700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4bef3d00 0x557d4c0076d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:32.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.007+0000 7fb44bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4be794a0 0x557d4c007c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:32.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.007+0000 7fb44bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4be794a0 0x557d4c007c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:59352/0 (socket says 172.21.15.19:59352) 2024-05-09T02:22:32.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.007+0000 7fb450a69700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb4380b2b40 0x7fb4380b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:32.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.007+0000 7fb44bfff700 1 -- 172.21.15.19:0/194164538 learned_addr learned my addr 172.21.15.19:0/194164538 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:32.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.007+0000 7fb449ffb700 1 -- 172.21.15.19:0/3436874529 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fb4340a6fe0 con 0x7fb44c103ba0 2024-05-09T02:22:32.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.007+0000 7fb44bfff700 1 -- 172.21.15.19:0/194164538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4bef3d00 msgr2=0x557d4c0076d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:32.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.007+0000 7fb44bfff700 1 --2- 172.21.15.19:0/194164538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4bef3d00 0x557d4c0076d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.007+0000 7fb44bfff700 1 -- 172.21.15.19:0/194164538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c00bde0 msgr2=0x557d4c00c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:32.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.007+0000 7fb44bfff700 1 --2- 172.21.15.19:0/194164538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c00bde0 0x557d4c00c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.007+0000 7fb44bfff700 1 -- 172.21.15.19:0/194164538 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d4beaf500 con 0x557d4be794a0 2024-05-09T02:22:32.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.008+0000 7fb44bfff700 1 --2- 172.21.15.19:0/194164538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4be794a0 0x557d4c007c30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0094e0 tx=0x7fb43c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dff7b91a8163119 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:32.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.008+0000 7fb450a69700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb4380b2b40 0x7fb4380b5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb444014660 tx=0x7fb444009b00 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:32.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.008+0000 7fb4427fc700 1 -- 172.21.15.19:0/194164538 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c013750 con 0x557d4be794a0 2024-05-09T02:22:32.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.010+0000 7fb4427fc700 1 -- 172.21.15.19:0/194164538 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb43c013da0 con 0x557d4be794a0 2024-05-09T02:22:32.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.011+0000 7fb460ee2b80 1 -- 172.21.15.19:0/194164538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4be794a0 msgr2=0x557d4c007c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:32.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.011+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/194164538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4be794a0 0x557d4c007c30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0094e0 tx=0x7fb43c009920 comp rx=0 tx=0).stop 2024-05-09T02:22:32.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.011+0000 7fb460ee2b80 1 -- 172.21.15.19:0/194164538 shutdown_connections 2024-05-09T02:22:32.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.011+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/194164538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c00bde0 0x557d4c00c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.011+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/194164538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4bef3d00 0x557d4c0076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.011+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/194164538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4be794a0 0x557d4c007c30 secure :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0094e0 tx=0x7fb43c009920 comp rx=0 tx=0).stop 2024-05-09T02:22:32.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.011+0000 7fb460ee2b80 1 -- 172.21.15.19:0/194164538 >> 172.21.15.19:0/194164538 conn(0x557d4be7cf70 msgr2=0x557d4bff1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:32.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.013+0000 7fb460ee2b80 1 -- 172.21.15.19:0/194164538 shutdown_connections 2024-05-09T02:22:32.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.013+0000 7fb460ee2b80 1 -- 172.21.15.19:0/194164538 wait complete. 2024-05-09T02:22:32.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.014+0000 7fb460ee2b80 1 -- start start 2024-05-09T02:22:32.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.014+0000 7fb460ee2b80 1 Processor -- start 2024-05-09T02:22:32.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.014+0000 7fb460ee2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4bef3d00 0x557d4c0110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:32.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.014+0000 7fb460ee2b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4c00bde0 0x557d4c0115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:32.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.014+0000 7fb460ee2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c011b30 0x557d4c01d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:32.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.014+0000 7fb460ee2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d4c00cba0 con 0x557d4c011b30 2024-05-09T02:22:32.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.014+0000 7fb460ee2b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557d4beb3b20 con 0x557d4c00bde0 2024-05-09T02:22:32.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.014+0000 7fb460ee2b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557d4c014070 con 0x557d4bef3d00 2024-05-09T02:22:32.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.015+0000 7fb460ee2b80 10 client.0 _ll_register_callbacks cb 0x557d4bd8a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:32.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.015+0000 7fb4437fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4c00bde0 0x557d4c0115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:32.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.015+0000 7fb4437fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4c00bde0 0x557d4c0115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:52776/0 (socket says 172.21.15.19:52776) 2024-05-09T02:22:32.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.015+0000 7fb450a69700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c011b30 0x557d4c01d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:32.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.015+0000 7fb450a69700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c011b30 0x557d4c01d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59088/0 (socket says 172.21.15.19:59088) 2024-05-09T02:22:32.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.015+0000 7fb450a69700 1 -- 172.21.15.19:0/3109847701 learned_addr learned my addr 172.21.15.19:0/3109847701 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:32.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.016+0000 7fb450a69700 1 -- 172.21.15.19:0/3109847701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4bef3d00 msgr2=0x557d4c0110b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:32.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.016+0000 7fb450a69700 1 --2- 172.21.15.19:0/3109847701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4bef3d00 0x557d4c0110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.016+0000 7fb450a69700 1 -- 172.21.15.19:0/3109847701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4c00bde0 msgr2=0x557d4c0115f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:32.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.016+0000 7fb450a69700 1 --2- 172.21.15.19:0/3109847701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4c00bde0 0x557d4c0115f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.016+0000 7fb450a69700 1 -- 172.21.15.19:0/3109847701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d4bd94490 con 0x557d4c011b30 2024-05-09T02:22:32.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.016+0000 7fb450a69700 1 --2- 172.21.15.19:0/3109847701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c011b30 0x557d4c01d3b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb4440042b0 tx=0x7fb444019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9562f13ed044734 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:32.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.016+0000 7fb42b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:22:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.017+0000 7fb42b7fe700 1 -- 172.21.15.19:0/3109847701 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444024070 con 0x557d4c011b30 2024-05-09T02:22:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.017+0000 7fb42b7fe700 1 -- 172.21.15.19:0/3109847701 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fb444003de0 con 0x557d4c011b30 2024-05-09T02:22:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.017+0000 7fb460ee2b80 1 -- 172.21.15.19:0/3109847701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557d4bf26a80 con 0x557d4c011b30 2024-05-09T02:22:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.017+0000 7fb42b7fe700 1 -- 172.21.15.19:0/3109847701 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fb444030050 con 0x557d4c011b30 2024-05-09T02:22:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.017+0000 7fb460ee2b80 10 client.16698 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.017+0000 7fb460ee2b80 1 -- 172.21.15.19:0/3109847701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557d4c01f100 con 0x557d4c011b30 2024-05-09T02:22:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.017+0000 7fb42b7fe700 1 -- 172.21.15.19:0/3109847701 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557d4bf26a80 con 0x557d4c011b30 2024-05-09T02:22:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.018+0000 7fb42b7fe700 1 -- 172.21.15.19:0/3109847701 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fb4440204f0 con 0x557d4c011b30 2024-05-09T02:22:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.018+0000 7fb460ee2b80 10 client.16698 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.018+0000 7fb460ee2b80 10 client.16698 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:22:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.018+0000 7fb460ee2b80 20 client.16698 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.018+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/3109847701 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557d4c021760 0x557d4c023c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.018+0000 7fb460ee2b80 4 client.16698 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:22:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.018+0000 7fb460ee2b80 1 -- 172.21.15.19:0/3109847701 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d4bf20b20 con 0x557d4c021760 2024-05-09T02:22:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.018+0000 7fb44bfff700 1 --2- 172.21.15.19:0/3109847701 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557d4c021760 0x557d4c023c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.019+0000 7fb42b7fe700 10 client.16698 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:22:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.019+0000 7fb44bfff700 1 --2- 172.21.15.19:0/3109847701 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557d4c021760 0x557d4c023c50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=19841743f816e17b server_cookie=19ec3c98e14c2b78 in_seq=0 out_seq=0 2024-05-09T02:22:32.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.020+0000 7fb42b7fe700 1 -- 172.21.15.19:0/3109847701 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557d4bf20b20 con 0x557d4c021760 2024-05-09T02:22:32.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.020+0000 7fb42b7fe700 10 client.16698 handle_command_reply: tid=0 2024-05-09T02:22:32.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.020+0000 7fb460ee2b80 1 client.16698 shutdown 2024-05-09T02:22:32.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.020+0000 7fb460ee2b80 1 -- 172.21.15.19:0/3109847701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c011b30 msgr2=0x557d4c01d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.020+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/3109847701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c011b30 0x557d4c01d3b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb4440042b0 tx=0x7fb444019000 comp rx=0 tx=0).stop 2024-05-09T02:22:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.020+0000 7fb460ee2b80 1 -- 172.21.15.19:0/3109847701 shutdown_connections 2024-05-09T02:22:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.020+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/3109847701 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557d4c021760 0x557d4c023c50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.020+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/3109847701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c011b30 0x557d4c01d3b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.020+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/3109847701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4bef3d00 0x557d4c0110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.020+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/3109847701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4c00bde0 0x557d4c0115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.020+0000 7fb42b7fe700 0 client.16698 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:22:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.020+0000 7fb460ee2b80 1 -- 172.21.15.19:0/3109847701 >> 172.21.15.19:0/3109847701 conn(0x557d4be7cf70 msgr2=0x557d4bfeaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.020+0000 7fb460ee2b80 1 -- 172.21.15.19:0/3109847701 shutdown_connections 2024-05-09T02:22:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.020+0000 7fb460ee2b80 1 -- 172.21.15.19:0/3109847701 wait complete. 2024-05-09T02:22:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.021+0000 7fb460ee2b80 20 client.16698 trim_cache size 0 max 16384 2024-05-09T02:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.041+0000 7fb460ee2b80 1 Processor -- start 2024-05-09T02:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.042+0000 7fb460ee2b80 1 -- start start 2024-05-09T02:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.042+0000 7fb460ee2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4bef3d00 0x557d4bfeb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.042+0000 7fb460ee2b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4c00bde0 0x557d4c029410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.042+0000 7fb460ee2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c011b30 0x557d4c029950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.042+0000 7fb460ee2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c00d690 con 0x557d4c011b30 2024-05-09T02:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.042+0000 7fb460ee2b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c00f1d0 con 0x557d4c00bde0 2024-05-09T02:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.042+0000 7fb460ee2b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c00dd60 con 0x557d4bef3d00 2024-05-09T02:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.042+0000 7fb44bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4bef3d00 0x557d4bfeb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.042+0000 7fb450a69700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c011b30 0x557d4c029950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.042+0000 7fb4437fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4c00bde0 0x557d4c029410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.042+0000 7fb44bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4bef3d00 0x557d4bfeb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:59376/0 (socket says 172.21.15.19:59376) 2024-05-09T02:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.042+0000 7fb44bfff700 1 -- 172.21.15.19:0/1626737170 learned_addr learned my addr 172.21.15.19:0/1626737170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.043+0000 7fb4437fe700 1 -- 172.21.15.19:0/1626737170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4bef3d00 msgr2=0x557d4bfeb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.043+0000 7fb4437fe700 1 --2- 172.21.15.19:0/1626737170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4bef3d00 0x557d4bfeb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.043+0000 7fb4437fe700 1 -- 172.21.15.19:0/1626737170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c011b30 msgr2=0x557d4c029950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.043+0000 7fb4437fe700 1 --2- 172.21.15.19:0/1626737170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c011b30 0x557d4c029950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.043+0000 7fb4437fe700 1 -- 172.21.15.19:0/1626737170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d4bff2ee0 con 0x557d4c00bde0 2024-05-09T02:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.043+0000 7fb44bfff700 1 --2- 172.21.15.19:0/1626737170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4bef3d00 0x557d4bfeb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.043+0000 7fb450a69700 1 --2- 172.21.15.19:0/1626737170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c011b30 0x557d4c029950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.043+0000 7fb4437fe700 1 --2- 172.21.15.19:0/1626737170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4c00bde0 0x557d4c029410 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb4340e0dd0 tx=0x7fb4340afec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=717eca52b0307002 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.043+0000 7fb441ffb700 1 -- 172.21.15.19:0/1626737170 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4340a71e0 con 0x557d4c00bde0 2024-05-09T02:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.043+0000 7fb441ffb700 1 -- 172.21.15.19:0/1626737170 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb43401be50 con 0x557d4c00bde0 2024-05-09T02:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.043+0000 7fb441ffb700 1 -- 172.21.15.19:0/1626737170 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4340a7380 con 0x557d4c00bde0 2024-05-09T02:22:32.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.044+0000 7fb460ee2b80 1 -- 172.21.15.19:0/1626737170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4c00bde0 msgr2=0x557d4c029410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:32.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.044+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/1626737170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4c00bde0 0x557d4c029410 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb4340e0dd0 tx=0x7fb4340afec0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.044+0000 7fb460ee2b80 1 -- 172.21.15.19:0/1626737170 shutdown_connections 2024-05-09T02:22:32.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.044+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/1626737170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c011b30 0x557d4c029950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.044+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/1626737170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4bef3d00 0x557d4bfeb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.044+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/1626737170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4c00bde0 0x557d4c029410 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.044+0000 7fb460ee2b80 1 -- 172.21.15.19:0/1626737170 >> 172.21.15.19:0/1626737170 conn(0x557d4be7cf70 msgr2=0x557d4bfeaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:32.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.044+0000 7fb460ee2b80 1 -- 172.21.15.19:0/1626737170 shutdown_connections 2024-05-09T02:22:32.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.044+0000 7fb460ee2b80 1 -- 172.21.15.19:0/1626737170 wait complete. 2024-05-09T02:22:32.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.045+0000 7fb460ee2b80 1 -- start start 2024-05-09T02:22:32.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.045+0000 7fb460ee2b80 1 Processor -- start 2024-05-09T02:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.046+0000 7fb460ee2b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4bef3d00 0x557d4c0148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.047+0000 7fb460ee2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c00bde0 0x557d4c014e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.047+0000 7fb460ee2b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4c011b30 0x557d4c083ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.047+0000 7fb460ee2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c00eb00 con 0x557d4c00bde0 2024-05-09T02:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.047+0000 7fb44bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4bef3d00 0x557d4c0148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.047+0000 7fb44bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4bef3d00 0x557d4c0148d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:52792/0 (socket says 172.21.15.19:52792) 2024-05-09T02:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.047+0000 7fb44bfff700 1 -- 172.21.15.19:0/4146478697 learned_addr learned my addr 172.21.15.19:0/4146478697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.047+0000 7fb460ee2b80 1 -- 172.21.15.19:0/4146478697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c00e430 con 0x557d4bef3d00 2024-05-09T02:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.047+0000 7fb460ee2b80 1 -- 172.21.15.19:0/4146478697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c00b650 con 0x557d4c011b30 2024-05-09T02:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.047+0000 7fb450a69700 1 --2- 172.21.15.19:0/4146478697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4c011b30 0x557d4c083ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.047+0000 7fb460ee2b80 10 client.0 _ll_register_callbacks cb 0x557d4bd8a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.048+0000 7fb44bfff700 1 -- 172.21.15.19:0/4146478697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4c011b30 msgr2=0x557d4c083ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.048+0000 7fb44bfff700 1 --2- 172.21.15.19:0/4146478697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4c011b30 0x557d4c083ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.048+0000 7fb44bfff700 1 -- 172.21.15.19:0/4146478697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c00bde0 msgr2=0x557d4c014e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.048+0000 7fb44bfff700 1 --2- 172.21.15.19:0/4146478697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c00bde0 0x557d4c014e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.048+0000 7fb44bfff700 1 -- 172.21.15.19:0/4146478697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d4c084560 con 0x557d4bef3d00 2024-05-09T02:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.048+0000 7fb44bfff700 1 --2- 172.21.15.19:0/4146478697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4bef3d00 0x557d4c0148d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x557d4bff2a00 tx=0x7fb43c005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9832643ac7dd91b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.048+0000 7fb42b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.048+0000 7fb42b7fe700 1 -- 172.21.15.19:0/4146478697 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c003c10 con 0x557d4bef3d00 2024-05-09T02:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.048+0000 7fb460ee2b80 1 -- 172.21.15.19:0/4146478697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb43401be50 con 0x557d4bef3d00 2024-05-09T02:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.049+0000 7fb42b7fe700 1 -- 172.21.15.19:0/4146478697 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fb43c003db0 con 0x557d4bef3d00 2024-05-09T02:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.049+0000 7fb42b7fe700 1 -- 172.21.15.19:0/4146478697 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fb43c035050 con 0x557d4bef3d00 2024-05-09T02:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.049+0000 7fb460ee2b80 10 client.26432 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.049+0000 7fb460ee2b80 1 -- 172.21.15.19:0/4146478697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=58}) -- 0x557d4c08b6f0 con 0x557d4bef3d00 2024-05-09T02:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.049+0000 7fb42b7fe700 1 -- 172.21.15.19:0/4146478697 <== mon.2 v2:172.21.15.59:3300/0 4 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fb43c026ce0 con 0x557d4bef3d00 2024-05-09T02:22:32.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.049+0000 7fb460ee2b80 10 client.26432 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:32.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.049+0000 7fb460ee2b80 10 client.26432 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:22:32.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.049+0000 7fb460ee2b80 20 client.26432 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:32.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.049+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/4146478697 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557d4c08b960 0x557d4c08dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:32.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.049+0000 7fb460ee2b80 4 client.26432 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:22:32.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.049+0000 7fb460ee2b80 1 -- 172.21.15.19:0/4146478697 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb43401be50 con 0x557d4c08b960 2024-05-09T02:22:32.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.049+0000 7fb4437fe700 1 --2- 172.21.15.19:0/4146478697 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557d4c08b960 0x557d4c08dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.050+0000 7fb4437fe700 1 --2- 172.21.15.19:0/4146478697 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557d4c08b960 0x557d4c08dd80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7cef9251b5220a1a server_cookie=e7476049e0eb311d in_seq=0 out_seq=0 2024-05-09T02:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.050+0000 7fb42b7fe700 10 client.26432 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.050+0000 7fb42b7fe700 1 -- 172.21.15.19:0/4146478697 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb43401be50 con 0x557d4c08b960 2024-05-09T02:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.050+0000 7fb42b7fe700 10 client.26432 handle_command_reply: tid=0 2024-05-09T02:22:32.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.050+0000 7fb460ee2b80 1 client.26432 shutdown 2024-05-09T02:22:32.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.050+0000 7fb460ee2b80 1 -- 172.21.15.19:0/4146478697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4bef3d00 msgr2=0x557d4c0148d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:32.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.050+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/4146478697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4bef3d00 0x557d4c0148d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x557d4bff2a00 tx=0x7fb43c005b40 comp rx=0 tx=0).stop 2024-05-09T02:22:32.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.050+0000 7fb460ee2b80 1 -- 172.21.15.19:0/4146478697 shutdown_connections 2024-05-09T02:22:32.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.051+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/4146478697 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x557d4c08b960 0x557d4c08dd80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.051+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/4146478697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d4c011b30 0x557d4c083ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.051+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/4146478697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557d4bef3d00 0x557d4c0148d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.051+0000 7fb460ee2b80 1 --2- 172.21.15.19:0/4146478697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d4c00bde0 0x557d4c014e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.051+0000 7fb460ee2b80 1 -- 172.21.15.19:0/4146478697 >> 172.21.15.19:0/4146478697 conn(0x557d4be7cf70 msgr2=0x557d4bff53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:32.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.051+0000 7fb42b7fe700 0 client.26432 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:22:32.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.051+0000 7fb460ee2b80 1 -- 172.21.15.19:0/4146478697 shutdown_connections 2024-05-09T02:22:32.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.051+0000 7fb460ee2b80 1 -- 172.21.15.19:0/4146478697 wait complete. 2024-05-09T02:22:32.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.051+0000 7fb460ee2b80 20 client.26432 trim_cache size 0 max 16384 2024-05-09T02:22:32.058 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:22:32.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.052+0000 7fb4524cc700 1 -- 172.21.15.19:0/3436874529 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb4380b2b40 msgr2=0x7fb4380b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:32.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.052+0000 7fb4524cc700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb4380b2b40 0x7fb4380b5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb444014660 tx=0x7fb444009b00 comp rx=0 tx=0).stop 2024-05-09T02:22:32.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.052+0000 7fb4524cc700 1 -- 172.21.15.19:0/3436874529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb44c103ba0 msgr2=0x7fb44c1a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:32.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.052+0000 7fb4524cc700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb44c103ba0 0x7fb44c1a1980 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb434005910 tx=0x7fb43400b670 comp rx=0 tx=0).stop 2024-05-09T02:22:32.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.052+0000 7fb4524cc700 1 -- 172.21.15.19:0/3436874529 shutdown_connections 2024-05-09T02:22:32.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.052+0000 7fb4524cc700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb44c1086a0 0x7fb44c1a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.052+0000 7fb4524cc700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb4380b2b40 0x7fb4380b5050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.052+0000 7fb4524cc700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c103250 0x7fb44c1a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.052+0000 7fb4524cc700 1 --2- 172.21.15.19:0/3436874529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb44c103ba0 0x7fb44c1a1980 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:32.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.052+0000 7fb4524cc700 1 -- 172.21.15.19:0/3436874529 >> 172.21.15.19:0/3436874529 conn(0x7fb44c0fedc0 msgr2=0x7fb44c106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:32.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.053+0000 7fb4524cc700 1 -- 172.21.15.19:0/3436874529 shutdown_connections 2024-05-09T02:22:32.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:32.053+0000 7fb4524cc700 1 -- 172.21.15.19:0/3436874529 wait complete. 2024-05-09T02:22:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 650.0 s, 1499.8 tps, lat 66.200 ms stddev 71.671 2024-05-09T02:22:32.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:32 smithi059 ceph-mon[34781]: pgmap v350: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.6 MiB/s wr, 606 op/s 2024-05-09T02:22:32.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:32 smithi196 ceph-mon[34718]: pgmap v350: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.6 MiB/s wr, 606 op/s 2024-05-09T02:22:32.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:32 smithi019 ceph-mon[30151]: pgmap v350: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.6 MiB/s wr, 606 op/s 2024-05-09T02:22:32.915 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T02:22:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:22:34.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:34 smithi059 ceph-mon[34781]: pgmap v351: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.2 MiB/s wr, 548 op/s 2024-05-09T02:22:34.758 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:34 smithi019 ceph-mon[30151]: pgmap v351: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.2 MiB/s wr, 548 op/s 2024-05-09T02:22:34.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:34 smithi196 ceph-mon[34718]: pgmap v351: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.2 MiB/s wr, 548 op/s 2024-05-09T02:22:35.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.264+0000 7f65eea3e700 1 -- 172.21.15.19:0/925377503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65e81034c0 msgr2=0x7f65e81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:35.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.264+0000 7f65eea3e700 1 --2- 172.21.15.19:0/925377503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65e81034c0 0x7f65e81038d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f65d80099c0 tx=0x7f65d8009cd0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.264+0000 7f65eea3e700 1 -- 172.21.15.19:0/925377503 shutdown_connections 2024-05-09T02:22:35.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.264+0000 7f65eea3e700 1 --2- 172.21.15.19:0/925377503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e8108740 0x7f65e81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.264+0000 7f65eea3e700 1 --2- 172.21.15.19:0/925377503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65e81034c0 0x7f65e81038d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.264+0000 7f65eea3e700 1 --2- 172.21.15.19:0/925377503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65e8107df0 0x7f65e8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.264+0000 7f65eea3e700 1 -- 172.21.15.19:0/925377503 >> 172.21.15.19:0/925377503 conn(0x7f65e80fedf0 msgr2=0x7f65e8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:35.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.265+0000 7f65eea3e700 1 -- 172.21.15.19:0/925377503 shutdown_connections 2024-05-09T02:22:35.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.265+0000 7f65eea3e700 1 -- 172.21.15.19:0/925377503 wait complete. 2024-05-09T02:22:35.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.266+0000 7f65eea3e700 1 Processor -- start 2024-05-09T02:22:35.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.266+0000 7f65eea3e700 1 -- start start 2024-05-09T02:22:35.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.267+0000 7f65eea3e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65e81034c0 0x7f65e810c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.267+0000 7f65eea3e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65e8107df0 0x7f65e810ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.267+0000 7f65eea3e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e8108740 0x7f65e810cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.267+0000 7f65eea3e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65e8112370 con 0x7f65e81034c0 2024-05-09T02:22:35.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.267+0000 7f65eea3e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65e81121f0 con 0x7f65e8108740 2024-05-09T02:22:35.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.267+0000 7f65eea3e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65e81124f0 con 0x7f65e8107df0 2024-05-09T02:22:35.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.267+0000 7f65e7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65e81034c0 0x7f65e810c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.267+0000 7f65e7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65e81034c0 0x7f65e810c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59116/0 (socket says 172.21.15.19:59116) 2024-05-09T02:22:35.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.267+0000 7f65e7fff700 1 -- 172.21.15.19:0/1563101426 learned_addr learned my addr 172.21.15.19:0/1563101426 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:35.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.268+0000 7f65e7fff700 1 -- 172.21.15.19:0/1563101426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65e8107df0 msgr2=0x7f65e810ca20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T02:22:35.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.268+0000 7f65e7fff700 1 --2- 172.21.15.19:0/1563101426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65e8107df0 0x7f65e810ca20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.268+0000 7f65e7fff700 1 -- 172.21.15.19:0/1563101426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e8108740 msgr2=0x7f65e810cf60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T02:22:35.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.268+0000 7f65e7fff700 1 --2- 172.21.15.19:0/1563101426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e8108740 0x7f65e810cf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.268+0000 7f65e7fff700 1 -- 172.21.15.19:0/1563101426 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e81ad600 con 0x7f65e81034c0 2024-05-09T02:22:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.268+0000 7f65e7fff700 1 --2- 172.21.15.19:0/1563101426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65e81034c0 0x7f65e810c4e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f65d80098e0 tx=0x7f65d800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.269+0000 7f65e57fa700 1 -- 172.21.15.19:0/1563101426 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8009e60 con 0x7f65e81034c0 2024-05-09T02:22:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.269+0000 7f65e57fa700 1 -- 172.21.15.19:0/1563101426 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65d80054e0 con 0x7f65e81034c0 2024-05-09T02:22:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.269+0000 7f65e57fa700 1 -- 172.21.15.19:0/1563101426 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8004d20 con 0x7f65e81034c0 2024-05-09T02:22:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.269+0000 7f65eea3e700 1 -- 172.21.15.19:0/1563101426 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e81ad890 con 0x7f65e81034c0 2024-05-09T02:22:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.269+0000 7f65eea3e700 1 -- 172.21.15.19:0/1563101426 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e81adc90 con 0x7f65e81034c0 2024-05-09T02:22:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.276+0000 7f65e57fa700 1 -- 172.21.15.19:0/1563101426 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f65d802b440 con 0x7f65e81034c0 2024-05-09T02:22:35.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.277+0000 7f65fd454b80 1 Processor -- start 2024-05-09T02:22:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.277+0000 7f65fd454b80 1 -- start start 2024-05-09T02:22:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.277+0000 7f65fd454b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df013b0d00 0x55df014b9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.277+0000 7f65fd454b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df013364a0 0x55df014b9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.277+0000 7f65fd454b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df014b3400 0x55df014b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.277+0000 7f65fd454b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55df013d60a0 con 0x55df014b3400 2024-05-09T02:22:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.277+0000 7f65fd454b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55df0137b9a0 con 0x55df013364a0 2024-05-09T02:22:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.277+0000 7f65fd454b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df013d2060 con 0x55df013b0d00 2024-05-09T02:22:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.277+0000 7f65e7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df013364a0 0x55df014b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.277+0000 7f65e7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df013364a0 0x55df014b9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:52824/0 (socket says 172.21.15.19:52824) 2024-05-09T02:22:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.277+0000 7f65e7fff700 1 -- 172.21.15.19:0/4170808194 learned_addr learned my addr 172.21.15.19:0/4170808194 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.278+0000 7f65ecfdb700 1 --2- 172.21.15.19:0/4170808194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df013b0d00 0x55df014b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.278+0000 7f65e77fe700 1 --2- 172.21.15.19:0/4170808194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df014b3400 0x55df014b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.279+0000 7f65e7fff700 1 -- 172.21.15.19:0/4170808194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df013b0d00 msgr2=0x55df014b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.279+0000 7f65e7fff700 1 --2- 172.21.15.19:0/4170808194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df013b0d00 0x55df014b9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.279+0000 7f65e7fff700 1 -- 172.21.15.19:0/4170808194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df014b3400 msgr2=0x55df014b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.279+0000 7f65e7fff700 1 --2- 172.21.15.19:0/4170808194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df014b3400 0x55df014b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.279+0000 7f65e7fff700 1 -- 172.21.15.19:0/4170808194 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df0136c500 con 0x55df013364a0 2024-05-09T02:22:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.279+0000 7f65e57fa700 1 --2- 172.21.15.19:0/1563101426 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65d00b2b80 0x7f65d00b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.279+0000 7f65e7fff700 1 --2- 172.21.15.19:0/4170808194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df013364a0 0x55df014b9780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f65d80e0bc0 tx=0x7f65d80e9670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47001c0914620507 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.279+0000 7f65ecfdb700 1 --2- 172.21.15.19:0/1563101426 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65d00b2b80 0x7f65d00b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.279+0000 7f65e57fa700 1 -- 172.21.15.19:0/1563101426 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f65d80e3530 con 0x7f65e81034c0 2024-05-09T02:22:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.279+0000 7f65ce7fc700 1 -- 172.21.15.19:0/4170808194 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d80a6730 con 0x55df013364a0 2024-05-09T02:22:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.280+0000 7f65ce7fc700 1 -- 172.21.15.19:0/4170808194 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65d80aa380 con 0x55df013364a0 2024-05-09T02:22:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.280+0000 7f65ce7fc700 1 -- 172.21.15.19:0/4170808194 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d80a9fe0 con 0x55df013364a0 2024-05-09T02:22:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.281+0000 7f65fd454b80 1 -- 172.21.15.19:0/4170808194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df013364a0 msgr2=0x55df014b9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.281+0000 7f65fd454b80 1 --2- 172.21.15.19:0/4170808194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df013364a0 0x55df014b9780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f65d80e0bc0 tx=0x7f65d80e9670 comp rx=0 tx=0).stop 2024-05-09T02:22:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.281+0000 7f65ecfdb700 1 --2- 172.21.15.19:0/1563101426 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65d00b2b80 0x7f65d00b5090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f65e000f540 tx=0x7f65e000b040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.281+0000 7f65fd454b80 1 -- 172.21.15.19:0/4170808194 shutdown_connections 2024-05-09T02:22:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.281+0000 7f65fd454b80 1 --2- 172.21.15.19:0/4170808194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df014b3400 0x55df014b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.281+0000 7f65fd454b80 1 --2- 172.21.15.19:0/4170808194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df013b0d00 0x55df014b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.281+0000 7f65fd454b80 1 --2- 172.21.15.19:0/4170808194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df013364a0 0x55df014b9780 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.281+0000 7f65fd454b80 1 -- 172.21.15.19:0/4170808194 >> 172.21.15.19:0/4170808194 conn(0x55df01339f70 msgr2=0x55df014ae220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:35.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.282+0000 7f65fd454b80 1 -- 172.21.15.19:0/4170808194 shutdown_connections 2024-05-09T02:22:35.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.282+0000 7f65fd454b80 1 -- 172.21.15.19:0/4170808194 wait complete. 2024-05-09T02:22:35.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.282+0000 7f65fd454b80 1 -- start start 2024-05-09T02:22:35.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.283+0000 7f65fd454b80 1 Processor -- start 2024-05-09T02:22:35.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.284+0000 7f65fd454b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df013b0d00 0x55df014c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.284+0000 7f65fd454b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014b3400 0x55df014c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.284+0000 7f65fd454b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df014bcf30 0x55df014bd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.284+0000 7f65e7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df013b0d00 0x55df014c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.284+0000 7f65e7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df013b0d00 0x55df014c2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:52828/0 (socket says 172.21.15.19:52828) 2024-05-09T02:22:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.284+0000 7f65e77fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014b3400 0x55df014c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.284+0000 7f65e77fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014b3400 0x55df014c3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:59410/0 (socket says 172.21.15.19:59410) 2024-05-09T02:22:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.284+0000 7f65e7fff700 1 -- 172.21.15.19:0/469511574 learned_addr learned my addr 172.21.15.19:0/469511574 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.284+0000 7f65ecfdb700 1 --2- 172.21.15.19:0/469511574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df014bcf30 0x55df014bd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.284+0000 7f65fd454b80 1 -- 172.21.15.19:0/469511574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55df014b3fb0 con 0x55df014bcf30 2024-05-09T02:22:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.285+0000 7f65fd454b80 1 -- 172.21.15.19:0/469511574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55df01370b20 con 0x55df013b0d00 2024-05-09T02:22:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.285+0000 7f65fd454b80 1 -- 172.21.15.19:0/469511574 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df014b4130 con 0x55df014b3400 2024-05-09T02:22:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.285+0000 7f65fd454b80 10 client.0 _ll_register_callbacks cb 0x55df012476c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:35.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.285+0000 7f65e77fe700 1 -- 172.21.15.19:0/469511574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df013b0d00 msgr2=0x55df014c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:35.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.285+0000 7f65e77fe700 1 --2- 172.21.15.19:0/469511574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df013b0d00 0x55df014c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.285+0000 7f65e77fe700 1 -- 172.21.15.19:0/469511574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df014bcf30 msgr2=0x55df014bd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:35.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.285+0000 7f65e77fe700 1 --2- 172.21.15.19:0/469511574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df014bcf30 0x55df014bd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.285+0000 7f65e77fe700 1 -- 172.21.15.19:0/469511574 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df01251490 con 0x55df014b3400 2024-05-09T02:22:35.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.285+0000 7f65ecfdb700 1 --2- 172.21.15.19:0/469511574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df014bcf30 0x55df014bd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:22:35.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.285+0000 7f65fd454b80 1 -- 172.21.15.19:0/469511574 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65d80aa380 con 0x55df014b3400 2024-05-09T02:22:35.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.286+0000 7f65e77fe700 1 --2- 172.21.15.19:0/469511574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014b3400 0x55df014c3130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f65e8076b40 tx=0x7f65dc00bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=571b59542ca034e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:35.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.286+0000 7f65c77fe700 10 client.26473 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:22:35.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.288+0000 7f65c77fe700 1 -- 172.21.15.19:0/469511574 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc00b2c0 con 0x55df014b3400 2024-05-09T02:22:35.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.288+0000 7f65c77fe700 1 -- 172.21.15.19:0/469511574 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f65dc00dc60 con 0x55df014b3400 2024-05-09T02:22:35.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.288+0000 7f65c77fe700 1 -- 172.21.15.19:0/469511574 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f65dc01e450 con 0x55df014b3400 2024-05-09T02:22:35.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.288+0000 7f65fd454b80 10 client.26473 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:35.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.288+0000 7f65fd454b80 1 -- 172.21.15.19:0/469511574 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=58}) -- 0x55df014caba0 con 0x55df014b3400 2024-05-09T02:22:35.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.289+0000 7f65c77fe700 1 -- 172.21.15.19:0/469511574 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f65dc013070 con 0x55df014b3400 2024-05-09T02:22:35.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.289+0000 7f65fd454b80 10 client.26473 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:35.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.289+0000 7f65fd454b80 10 client.26473 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:22:35.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.289+0000 7f65fd454b80 20 client.26473 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:35.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.289+0000 7f65fd454b80 1 --2- 172.21.15.19:0/469511574 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55df014cd0d0 0x55df014cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.289+0000 7f65fd454b80 4 client.26473 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:22:35.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.289+0000 7f65fd454b80 1 -- 172.21.15.19:0/469511574 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df013e3a80 con 0x55df014cd0d0 2024-05-09T02:22:35.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.289+0000 7f65e7fff700 1 --2- 172.21.15.19:0/469511574 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55df014cd0d0 0x55df014cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.290+0000 7f65c77fe700 10 client.26473 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:22:35.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.290+0000 7f65e7fff700 1 --2- 172.21.15.19:0/469511574 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55df014cd0d0 0x55df014cf4f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7bc51a1323534b57 server_cookie=2fea55f343b92756 in_seq=0 out_seq=0 2024-05-09T02:22:35.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.290+0000 7f65c77fe700 1 -- 172.21.15.19:0/469511574 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55df013e3a80 con 0x55df014cd0d0 2024-05-09T02:22:35.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.290+0000 7f65c77fe700 10 client.26473 handle_command_reply: tid=0 2024-05-09T02:22:35.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.291+0000 7f65fd454b80 1 client.26473 shutdown 2024-05-09T02:22:35.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.291+0000 7f65fd454b80 1 -- 172.21.15.19:0/469511574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014b3400 msgr2=0x55df014c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:35.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.291+0000 7f65fd454b80 1 --2- 172.21.15.19:0/469511574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014b3400 0x55df014c3130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f65e8076b40 tx=0x7f65dc00bf90 comp rx=0 tx=0).stop 2024-05-09T02:22:35.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.291+0000 7f65fd454b80 1 -- 172.21.15.19:0/469511574 shutdown_connections 2024-05-09T02:22:35.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.291+0000 7f65fd454b80 1 --2- 172.21.15.19:0/469511574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df014bcf30 0x55df014bd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.291+0000 7f65fd454b80 1 --2- 172.21.15.19:0/469511574 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55df014cd0d0 0x55df014cf4f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.291+0000 7f65c77fe700 0 client.26473 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:22:35.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.291+0000 7f65fd454b80 1 --2- 172.21.15.19:0/469511574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df013b0d00 0x55df014c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.291+0000 7f65fd454b80 1 --2- 172.21.15.19:0/469511574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014b3400 0x55df014c3130 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.291+0000 7f65fd454b80 1 -- 172.21.15.19:0/469511574 >> 172.21.15.19:0/469511574 conn(0x55df01339f70 msgr2=0x55df014a7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:35.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.291+0000 7f65fd454b80 1 -- 172.21.15.19:0/469511574 shutdown_connections 2024-05-09T02:22:35.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.291+0000 7f65fd454b80 1 -- 172.21.15.19:0/469511574 wait complete. 2024-05-09T02:22:35.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.291+0000 7f65fd454b80 20 client.26473 trim_cache size 0 max 16384 2024-05-09T02:22:35.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.311+0000 7f65fd454b80 1 Processor -- start 2024-05-09T02:22:35.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.311+0000 7f65fd454b80 1 -- start start 2024-05-09T02:22:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.311+0000 7f65fd454b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df013b0d00 0x55df014d4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.311+0000 7f65fd454b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014b3400 0x55df014d5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.311+0000 7f65fd454b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df014bcf30 0x55df014adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.311+0000 7f65fd454b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65e81b5560 con 0x55df013b0d00 2024-05-09T02:22:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.311+0000 7f65fd454b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65e81b70a0 con 0x55df014bcf30 2024-05-09T02:22:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.311+0000 7f65fd454b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65e81b5c30 con 0x55df014b3400 2024-05-09T02:22:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.312+0000 7f65ecfdb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df014bcf30 0x55df014adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.312+0000 7f65ecfdb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df014bcf30 0x55df014adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:52834/0 (socket says 172.21.15.19:52834) 2024-05-09T02:22:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.312+0000 7f65ecfdb700 1 -- 172.21.15.19:0/1316268445 learned_addr learned my addr 172.21.15.19:0/1316268445 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.312+0000 7f65e77fe700 1 --2- 172.21.15.19:0/1316268445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014b3400 0x55df014d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.312+0000 7f65e7fff700 1 --2- 172.21.15.19:0/1316268445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df013b0d00 0x55df014d4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.312+0000 7f65ecfdb700 1 -- 172.21.15.19:0/1316268445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014b3400 msgr2=0x55df014d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.312+0000 7f65ecfdb700 1 --2- 172.21.15.19:0/1316268445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014b3400 0x55df014d5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.312+0000 7f65ecfdb700 1 -- 172.21.15.19:0/1316268445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df013b0d00 msgr2=0x55df014d4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.312+0000 7f65ecfdb700 1 --2- 172.21.15.19:0/1316268445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df013b0d00 0x55df014d4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.312+0000 7f65ecfdb700 1 -- 172.21.15.19:0/1316268445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df014bb4a0 con 0x55df014bcf30 2024-05-09T02:22:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.312+0000 7f65e7fff700 1 --2- 172.21.15.19:0/1316268445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df013b0d00 0x55df014d4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:22:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.313+0000 7f65ecfdb700 1 --2- 172.21.15.19:0/1316268445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df014bcf30 0x55df014adc80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f65e00086d0 tx=0x7f65e0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5886727de374f3d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.313+0000 7f65cdffb700 1 -- 172.21.15.19:0/1316268445 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0027070 con 0x55df014bcf30 2024-05-09T02:22:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.313+0000 7f65cdffb700 1 -- 172.21.15.19:0/1316268445 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e0004db0 con 0x55df014bcf30 2024-05-09T02:22:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.313+0000 7f65cdffb700 1 -- 172.21.15.19:0/1316268445 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e001b070 con 0x55df014bcf30 2024-05-09T02:22:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.314+0000 7f65fd454b80 1 -- 172.21.15.19:0/1316268445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df014bcf30 msgr2=0x55df014adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.314+0000 7f65fd454b80 1 --2- 172.21.15.19:0/1316268445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df014bcf30 0x55df014adc80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f65e00086d0 tx=0x7f65e0014000 comp rx=0 tx=0).stop 2024-05-09T02:22:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.314+0000 7f65fd454b80 1 -- 172.21.15.19:0/1316268445 shutdown_connections 2024-05-09T02:22:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.314+0000 7f65fd454b80 1 --2- 172.21.15.19:0/1316268445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df014bcf30 0x55df014adc80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.314+0000 7f65fd454b80 1 --2- 172.21.15.19:0/1316268445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df013b0d00 0x55df014d4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.314+0000 7f65fd454b80 1 --2- 172.21.15.19:0/1316268445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014b3400 0x55df014d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.314+0000 7f65fd454b80 1 -- 172.21.15.19:0/1316268445 >> 172.21.15.19:0/1316268445 conn(0x55df01339f70 msgr2=0x55df014bb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.314+0000 7f65fd454b80 1 -- 172.21.15.19:0/1316268445 shutdown_connections 2024-05-09T02:22:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.314+0000 7f65fd454b80 1 -- 172.21.15.19:0/1316268445 wait complete. 2024-05-09T02:22:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.314+0000 7f65fd454b80 1 -- start start 2024-05-09T02:22:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.315+0000 7f65fd454b80 1 Processor -- start 2024-05-09T02:22:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.315+0000 7f65fd454b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df013b0d00 0x55df014c9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.315+0000 7f65fd454b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df014b3400 0x55df014c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.315+0000 7f65fd454b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014bcf30 0x55df014c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.315+0000 7f65fd454b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65e81b69d0 con 0x55df013b0d00 2024-05-09T02:22:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.315+0000 7f65fd454b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65e81b6300 con 0x55df014b3400 2024-05-09T02:22:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.315+0000 7f65fd454b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65e81b3520 con 0x55df014bcf30 2024-05-09T02:22:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.315+0000 7f65fd454b80 10 client.0 _ll_register_callbacks cb 0x55df014b3dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.315+0000 7f65e7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df013b0d00 0x55df014c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.315+0000 7f65e7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df013b0d00 0x55df014c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59152/0 (socket says 172.21.15.19:59152) 2024-05-09T02:22:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.315+0000 7f65e7fff700 1 -- 172.21.15.19:0/4082164256 learned_addr learned my addr 172.21.15.19:0/4082164256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.315+0000 7f65e77fe700 1 --2- 172.21.15.19:0/4082164256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df014b3400 0x55df014c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.315+0000 7f65ecfdb700 1 --2- 172.21.15.19:0/4082164256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014bcf30 0x55df014c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65e77fe700 1 -- 172.21.15.19:0/4082164256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014bcf30 msgr2=0x55df014c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65e77fe700 1 --2- 172.21.15.19:0/4082164256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014bcf30 0x55df014c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65e77fe700 1 -- 172.21.15.19:0/4082164256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df013b0d00 msgr2=0x55df014c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65e77fe700 1 --2- 172.21.15.19:0/4082164256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df013b0d00 0x55df014c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65e77fe700 1 -- 172.21.15.19:0/4082164256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df014c7760 con 0x55df014b3400 2024-05-09T02:22:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65ecfdb700 1 --2- 172.21.15.19:0/4082164256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014bcf30 0x55df014c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:22:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65e7fff700 1 --2- 172.21.15.19:0/4082164256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df013b0d00 0x55df014c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:22:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65e77fe700 1 --2- 172.21.15.19:0/4082164256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df014b3400 0x55df014c69e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00d720 tx=0x7f65dc010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3229562627090fbc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:22:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65c77fe700 1 -- 172.21.15.19:0/4082164256 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc004020 con 0x55df014b3400 2024-05-09T02:22:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65fd454b80 1 -- 172.21.15.19:0/4082164256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e0004db0 con 0x55df014b3400 2024-05-09T02:22:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65c77fe700 1 -- 172.21.15.19:0/4082164256 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f65dc00f040 con 0x55df014b3400 2024-05-09T02:22:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65c77fe700 1 -- 172.21.15.19:0/4082164256 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f65dc034050 con 0x55df014b3400 2024-05-09T02:22:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65fd454b80 10 client.26447 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65fd454b80 1 -- 172.21.15.19:0/4082164256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55df015376f0 con 0x55df014b3400 2024-05-09T02:22:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.316+0000 7f65c77fe700 1 -- 172.21.15.19:0/4082164256 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e0004db0 con 0x55df014b3400 2024-05-09T02:22:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.317+0000 7f65c77fe700 1 -- 172.21.15.19:0/4082164256 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f65dc026070 con 0x55df014b3400 2024-05-09T02:22:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.317+0000 7f65fd454b80 10 client.26447 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.317+0000 7f65fd454b80 10 client.26447 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:22:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.317+0000 7f65fd454b80 20 client.26447 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.317+0000 7f65fd454b80 1 --2- 172.21.15.19:0/4082164256 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55df014cd0d0 0x55df01539e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.317+0000 7f65fd454b80 4 client.26447 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:22:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.317+0000 7f65fd454b80 1 -- 172.21.15.19:0/4082164256 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df013ddb20 con 0x55df014cd0d0 2024-05-09T02:22:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.317+0000 7f65e7fff700 1 --2- 172.21.15.19:0/4082164256 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55df014cd0d0 0x55df01539e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.317+0000 7f65c77fe700 10 client.26447 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:22:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65e7fff700 1 --2- 172.21.15.19:0/4082164256 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55df014cd0d0 0x55df01539e30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69d0dbabf6ca3881 server_cookie=955f246e5d277834 in_seq=0 out_seq=0 2024-05-09T02:22:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65c77fe700 1 -- 172.21.15.19:0/4082164256 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df013ddb20 con 0x55df014cd0d0 2024-05-09T02:22:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65c77fe700 10 client.26447 handle_command_reply: tid=0 2024-05-09T02:22:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65fd454b80 1 client.26447 shutdown 2024-05-09T02:22:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65fd454b80 1 -- 172.21.15.19:0/4082164256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df014b3400 msgr2=0x55df014c69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65fd454b80 1 --2- 172.21.15.19:0/4082164256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df014b3400 0x55df014c69e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00d720 tx=0x7f65dc010000 comp rx=0 tx=0).stop 2024-05-09T02:22:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65fd454b80 1 -- 172.21.15.19:0/4082164256 shutdown_connections 2024-05-09T02:22:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65fd454b80 1 --2- 172.21.15.19:0/4082164256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df014bcf30 0x55df014c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65fd454b80 1 --2- 172.21.15.19:0/4082164256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55df013b0d00 0x55df014c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65fd454b80 1 --2- 172.21.15.19:0/4082164256 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x55df014cd0d0 0x55df01539e30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65fd454b80 1 --2- 172.21.15.19:0/4082164256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df014b3400 0x55df014c69e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65fd454b80 1 -- 172.21.15.19:0/4082164256 >> 172.21.15.19:0/4082164256 conn(0x55df01339f70 msgr2=0x55df014adc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65c77fe700 0 client.26447 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:22:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65fd454b80 1 -- 172.21.15.19:0/4082164256 shutdown_connections 2024-05-09T02:22:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65fd454b80 1 -- 172.21.15.19:0/4082164256 wait complete. 2024-05-09T02:22:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.318+0000 7f65fd454b80 20 client.26447 trim_cache size 0 max 16384 2024-05-09T02:22:35.326 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:22:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.319+0000 7f65eea3e700 1 -- 172.21.15.19:0/1563101426 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65d00b2b80 msgr2=0x7f65d00b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.319+0000 7f65eea3e700 1 --2- 172.21.15.19:0/1563101426 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65d00b2b80 0x7f65d00b5090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f65e000f540 tx=0x7f65e000b040 comp rx=0 tx=0).stop 2024-05-09T02:22:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.319+0000 7f65eea3e700 1 -- 172.21.15.19:0/1563101426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65e81034c0 msgr2=0x7f65e810c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.319+0000 7f65eea3e700 1 --2- 172.21.15.19:0/1563101426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65e81034c0 0x7f65e810c4e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f65d80098e0 tx=0x7f65d800fab0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.319+0000 7f65eea3e700 1 -- 172.21.15.19:0/1563101426 shutdown_connections 2024-05-09T02:22:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.319+0000 7f65eea3e700 1 --2- 172.21.15.19:0/1563101426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e8108740 0x7f65e810cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.319+0000 7f65eea3e700 1 --2- 172.21.15.19:0/1563101426 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65d00b2b80 0x7f65d00b5090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.319+0000 7f65eea3e700 1 --2- 172.21.15.19:0/1563101426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65e81034c0 0x7f65e810c4e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.319+0000 7f65eea3e700 1 --2- 172.21.15.19:0/1563101426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65e8107df0 0x7f65e810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.319+0000 7f65eea3e700 1 -- 172.21.15.19:0/1563101426 >> 172.21.15.19:0/1563101426 conn(0x7f65e80fedf0 msgr2=0x7f65e8107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.319+0000 7f65eea3e700 1 -- 172.21.15.19:0/1563101426 shutdown_connections 2024-05-09T02:22:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:35.319+0000 7f65eea3e700 1 -- 172.21.15.19:0/1563101426 wait complete. 2024-05-09T02:22:36.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:36 smithi059 ceph-mon[34781]: pgmap v352: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.4 MiB/s wr, 799 op/s 2024-05-09T02:22:36.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:36 smithi196 ceph-mon[34718]: pgmap v352: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.4 MiB/s wr, 799 op/s 2024-05-09T02:22:36.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:36 smithi019 ceph-mon[30151]: pgmap v352: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.4 MiB/s wr, 799 op/s 2024-05-09T02:22:37.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 655.0 s, 1201.8 tps, lat 83.312 ms stddev 151.915 2024-05-09T02:22:38.155 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T02:22:38.537 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:22:38.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:38 smithi059 ceph-mon[34781]: pgmap v353: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 537 op/s 2024-05-09T02:22:38.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:38 smithi196 ceph-mon[34718]: pgmap v353: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 537 op/s 2024-05-09T02:22:38.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:38 smithi019 ceph-mon[30151]: pgmap v353: 97 pgs: 97 active+clean; 4.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 537 op/s 2024-05-09T02:22:40.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.666+0000 7f1aabf1c700 1 -- 172.21.15.19:0/3804845498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa4103ba0 msgr2=0x7f1aa41080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:40.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.666+0000 7f1aabf1c700 1 --2- 172.21.15.19:0/3804845498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa4103ba0 0x7f1aa41080d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009990 tx=0x7f1a94009ca0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.666+0000 7f1aabf1c700 1 -- 172.21.15.19:0/3804845498 shutdown_connections 2024-05-09T02:22:40.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.666+0000 7f1aabf1c700 1 --2- 172.21.15.19:0/3804845498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa41086a0 0x7f1aa4108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.666+0000 7f1aabf1c700 1 --2- 172.21.15.19:0/3804845498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1aa4103250 0x7f1aa4103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.666+0000 7f1aabf1c700 1 --2- 172.21.15.19:0/3804845498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa4103ba0 0x7f1aa41080d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.666+0000 7f1aabf1c700 1 -- 172.21.15.19:0/3804845498 >> 172.21.15.19:0/3804845498 conn(0x7f1aa40fedc0 msgr2=0x7f1aa41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:40.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.666+0000 7f1aabf1c700 1 -- 172.21.15.19:0/3804845498 shutdown_connections 2024-05-09T02:22:40.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.666+0000 7f1aabf1c700 1 -- 172.21.15.19:0/3804845498 wait complete. 2024-05-09T02:22:40.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.667+0000 7f1aabf1c700 1 Processor -- start 2024-05-09T02:22:40.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.667+0000 7f1aabf1c700 1 -- start start 2024-05-09T02:22:40.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.668+0000 7f1aabf1c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa4103250 0x7f1aa4117690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.668+0000 7f1aabf1c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1aa4103ba0 0x7f1aa41117b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.668+0000 7f1aabf1c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa41086a0 0x7f1aa4111cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.668+0000 7f1aa9cb8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa4103250 0x7f1aa4117690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.668+0000 7f1aabf1c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4109cb0 con 0x7f1aa4103250 2024-05-09T02:22:40.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.668+0000 7f1aa9cb8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa4103250 0x7f1aa4117690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36474/0 (socket says 172.21.15.19:36474) 2024-05-09T02:22:40.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.668+0000 7f1aaa4b9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa41086a0 0x7f1aa4111cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.668+0000 7f1aa9cb8700 1 -- 172.21.15.19:0/2425313063 learned_addr learned my addr 172.21.15.19:0/2425313063 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:40.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.668+0000 7f1aa94b7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1aa4103ba0 0x7f1aa41117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.668+0000 7f1aabf1c700 1 -- 172.21.15.19:0/2425313063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4109b30 con 0x7f1aa41086a0 2024-05-09T02:22:40.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.668+0000 7f1aabf1c700 1 -- 172.21.15.19:0/2425313063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4109e30 con 0x7f1aa4103ba0 2024-05-09T02:22:40.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.669+0000 7f1aaa4b9700 1 -- 172.21.15.19:0/2425313063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1aa4103ba0 msgr2=0x7f1aa41117b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:40.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.669+0000 7f1aaa4b9700 1 --2- 172.21.15.19:0/2425313063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1aa4103ba0 0x7f1aa41117b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.669+0000 7f1aaa4b9700 1 -- 172.21.15.19:0/2425313063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa4103250 msgr2=0x7f1aa4117690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:40.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.669+0000 7f1aaa4b9700 1 --2- 172.21.15.19:0/2425313063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa4103250 0x7f1aa4117690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.669+0000 7f1aaa4b9700 1 -- 172.21.15.19:0/2425313063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa41123f0 con 0x7f1aa41086a0 2024-05-09T02:22:40.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.669+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/2425313063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa4103250 0x7f1aa4117690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:22:40.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.669+0000 7f1aaa4b9700 1 --2- 172.21.15.19:0/2425313063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa41086a0 0x7f1aa4111cf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000d720 tx=0x7f1aa000da30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:40.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.670+0000 7f1a9affd700 1 -- 172.21.15.19:0/2425313063 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0016070 con 0x7f1aa41086a0 2024-05-09T02:22:40.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.670+0000 7f1aabf1c700 1 -- 172.21.15.19:0/2425313063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa41b61b0 con 0x7f1aa41086a0 2024-05-09T02:22:40.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.670+0000 7f1aabf1c700 1 -- 172.21.15.19:0/2425313063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa41b6610 con 0x7f1aa41086a0 2024-05-09T02:22:40.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.670+0000 7f1a9affd700 1 -- 172.21.15.19:0/2425313063 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa000f690 con 0x7f1aa41086a0 2024-05-09T02:22:40.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.670+0000 7f1a9affd700 1 -- 172.21.15.19:0/2425313063 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0016070 con 0x7f1aa41086a0 2024-05-09T02:22:40.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.674+0000 7f1a9affd700 1 -- 172.21.15.19:0/2425313063 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1aa0003840 con 0x7f1aa41086a0 2024-05-09T02:22:40.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.675+0000 7f1a9affd700 1 --2- 172.21.15.19:0/2425313063 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1a900b2c70 0x7f1a900b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.675+0000 7f1a9affd700 1 -- 172.21.15.19:0/2425313063 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f1aa00e18b0 con 0x7f1aa41086a0 2024-05-09T02:22:40.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.675+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/2425313063 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1a900b2c70 0x7f1a900b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.676+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/2425313063 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1a900b2c70 0x7f1a900b5180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009c10 tx=0x7f1a9c00b040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:40.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.676+0000 7f1aba932b80 1 Processor -- start 2024-05-09T02:22:40.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.676+0000 7f1aba932b80 1 -- start start 2024-05-09T02:22:40.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.677+0000 7f1aba932b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cec34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.677+0000 7f1aba932b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 0x55629cec3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.677+0000 7f1aba932b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 0x55629cebdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.677+0000 7f1aba932b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55629cddfff0 con 0x55629cebd6a0 2024-05-09T02:22:40.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.677+0000 7f1aba932b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55629cd7e7f0 con 0x55629ce2a160 2024-05-09T02:22:40.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.677+0000 7f1aa94b7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 0x55629cec3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.677+0000 7f1aba932b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55629cddbfb0 con 0x55629cd404a0 2024-05-09T02:22:40.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.677+0000 7f1aa94b7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 0x55629cec3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36396/0 (socket says 172.21.15.19:36396) 2024-05-09T02:22:40.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.677+0000 7f1aa94b7700 1 -- 172.21.15.19:0/1716427321 learned_addr learned my addr 172.21.15.19:0/1716427321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:40.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.678+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/1716427321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cec34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.678+0000 7f1aaa4b9700 1 --2- 172.21.15.19:0/1716427321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 0x55629cebdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.678+0000 7f1aa9cb8700 1 -- 172.21.15.19:0/1716427321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 msgr2=0x55629cec3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:40.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.678+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/1716427321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 0x55629cec3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.678+0000 7f1aa9cb8700 1 -- 172.21.15.19:0/1716427321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 msgr2=0x55629cebdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:40.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.678+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/1716427321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 0x55629cebdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.678+0000 7f1aa9cb8700 1 -- 172.21.15.19:0/1716427321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629cd76370 con 0x55629ce2a160 2024-05-09T02:22:40.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.678+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/1716427321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cec34f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c007c20 tx=0x7f1a9c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd72eea6935c4763 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:40.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.679+0000 7f1a8ffff700 1 -- 172.21.15.19:0/1716427321 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c027070 con 0x55629ce2a160 2024-05-09T02:22:40.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.679+0000 7f1a8ffff700 1 -- 172.21.15.19:0/1716427321 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9c01e040 con 0x55629ce2a160 2024-05-09T02:22:40.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.679+0000 7f1a8ffff700 1 -- 172.21.15.19:0/1716427321 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c017ce0 con 0x55629ce2a160 2024-05-09T02:22:40.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.680+0000 7f1aba932b80 1 -- 172.21.15.19:0/1716427321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 msgr2=0x55629cec34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:40.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.680+0000 7f1aba932b80 1 --2- 172.21.15.19:0/1716427321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cec34f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c007c20 tx=0x7f1a9c014000 comp rx=0 tx=0).stop 2024-05-09T02:22:40.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.680+0000 7f1aba932b80 1 -- 172.21.15.19:0/1716427321 shutdown_connections 2024-05-09T02:22:40.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.680+0000 7f1aba932b80 1 --2- 172.21.15.19:0/1716427321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 0x55629cebdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.680+0000 7f1aba932b80 1 --2- 172.21.15.19:0/1716427321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cec34f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.680+0000 7f1aba932b80 1 --2- 172.21.15.19:0/1716427321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 0x55629cec3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.680+0000 7f1aba932b80 1 -- 172.21.15.19:0/1716427321 >> 172.21.15.19:0/1716427321 conn(0x55629cda06a0 msgr2=0x55629ceb8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:40.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.680+0000 7f1aba932b80 1 -- 172.21.15.19:0/1716427321 shutdown_connections 2024-05-09T02:22:40.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.680+0000 7f1aba932b80 1 -- 172.21.15.19:0/1716427321 wait complete. 2024-05-09T02:22:40.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.680+0000 7f1aba932b80 1 -- start start 2024-05-09T02:22:40.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.680+0000 7f1aba932b80 1 Processor -- start 2024-05-09T02:22:40.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.681+0000 7f1aba932b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cd404a0 0x55629cec7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.681+0000 7f1aba932b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629ce2a160 0x55629cec76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.681+0000 7f1aba932b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629cebd6a0 0x55629cec7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.681+0000 7f1aba932b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55629cc5bba0 con 0x55629cd404a0 2024-05-09T02:22:40.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.681+0000 7f1aba932b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55629cd9a150 con 0x55629cebd6a0 2024-05-09T02:22:40.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.681+0000 7f1aba932b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55629ceca1d0 con 0x55629ce2a160 2024-05-09T02:22:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.681+0000 7f1aaa4b9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629cebd6a0 0x55629cec7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.681+0000 7f1aaa4b9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629cebd6a0 0x55629cec7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:38754/0 (socket says 172.21.15.19:38754) 2024-05-09T02:22:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.681+0000 7f1aaa4b9700 1 -- 172.21.15.19:0/2475957168 learned_addr learned my addr 172.21.15.19:0/2475957168 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.681+0000 7f1aa94b7700 1 --2- 172.21.15.19:0/2475957168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629ce2a160 0x55629cec76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.681+0000 7f1aba932b80 10 client.0 _ll_register_callbacks cb 0x55629cc516c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.681+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/2475957168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cd404a0 0x55629cec7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.682+0000 7f1aaa4b9700 1 -- 172.21.15.19:0/2475957168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629ce2a160 msgr2=0x55629cec76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.682+0000 7f1aaa4b9700 1 --2- 172.21.15.19:0/2475957168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629ce2a160 0x55629cec76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.682+0000 7f1aaa4b9700 1 -- 172.21.15.19:0/2475957168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cd404a0 msgr2=0x55629cec7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.682+0000 7f1aaa4b9700 1 --2- 172.21.15.19:0/2475957168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cd404a0 0x55629cec7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.682+0000 7f1aaa4b9700 1 -- 172.21.15.19:0/2475957168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55629cc5b490 con 0x55629cebd6a0 2024-05-09T02:22:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.682+0000 7f1aa94b7700 1 --2- 172.21.15.19:0/2475957168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629ce2a160 0x55629cec76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:22:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.682+0000 7f1aaa4b9700 1 --2- 172.21.15.19:0/2475957168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629cebd6a0 0x55629cec7be0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00a5d70 tx=0x7f1aa00a5040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72ddfd6b4cdb01a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.682+0000 7f1a8d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:22:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.682+0000 7f1a8d7fa700 1 -- 172.21.15.19:0/2475957168 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa00692f0 con 0x55629cebd6a0 2024-05-09T02:22:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.682+0000 7f1a8d7fa700 1 -- 172.21.15.19:0/2475957168 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f1aa0018310 con 0x55629cebd6a0 2024-05-09T02:22:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.683+0000 7f1aba932b80 1 -- 172.21.15.19:0/2475957168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a9c01e040 con 0x55629cebd6a0 2024-05-09T02:22:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.683+0000 7f1a8d7fa700 1 -- 172.21.15.19:0/2475957168 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1aa0018510 con 0x55629cebd6a0 2024-05-09T02:22:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.683+0000 7f1aba932b80 10 client.26459 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.683+0000 7f1aba932b80 1 -- 172.21.15.19:0/2475957168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55629ced4c30 con 0x55629cebd6a0 2024-05-09T02:22:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.684+0000 7f1a8d7fa700 1 -- 172.21.15.19:0/2475957168 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa00aceb0 con 0x55629cebd6a0 2024-05-09T02:22:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.684+0000 7f1a8d7fa700 1 -- 172.21.15.19:0/2475957168 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f1aa006f2a0 con 0x55629cebd6a0 2024-05-09T02:22:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.684+0000 7f1aba932b80 10 client.26459 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.684+0000 7f1aba932b80 10 client.26459 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:22:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.684+0000 7f1aba932b80 20 client.26459 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.684+0000 7f1aba932b80 1 --2- 172.21.15.19:0/2475957168 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55629ced7270 0x55629ced9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.684+0000 7f1aba932b80 4 client.26459 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:22:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.684+0000 7f1aba932b80 1 -- 172.21.15.19:0/2475957168 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1a9c01e040 con 0x55629ced7270 2024-05-09T02:22:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.685+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/2475957168 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55629ced7270 0x55629ced9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.685+0000 7f1a8d7fa700 10 client.26459 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:22:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.685+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/2475957168 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55629ced7270 0x55629ced9690 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b9999c9941a36f3 server_cookie=70d23f1d3313d15a in_seq=0 out_seq=0 2024-05-09T02:22:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.686+0000 7f1a8d7fa700 1 -- 172.21.15.19:0/2475957168 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f1a9c01e040 con 0x55629ced7270 2024-05-09T02:22:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.686+0000 7f1a8d7fa700 10 client.26459 handle_command_reply: tid=0 2024-05-09T02:22:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.686+0000 7f1aba932b80 1 client.26459 shutdown 2024-05-09T02:22:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.686+0000 7f1aba932b80 1 -- 172.21.15.19:0/2475957168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629cebd6a0 msgr2=0x55629cec7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.686+0000 7f1aba932b80 1 --2- 172.21.15.19:0/2475957168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629cebd6a0 0x55629cec7be0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00a5d70 tx=0x7f1aa00a5040 comp rx=0 tx=0).stop 2024-05-09T02:22:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.686+0000 7f1aba932b80 1 -- 172.21.15.19:0/2475957168 shutdown_connections 2024-05-09T02:22:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.686+0000 7f1aba932b80 1 --2- 172.21.15.19:0/2475957168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629cebd6a0 0x55629cec7be0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.686+0000 7f1aba932b80 1 --2- 172.21.15.19:0/2475957168 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55629ced7270 0x55629ced9690 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.686+0000 7f1aba932b80 1 --2- 172.21.15.19:0/2475957168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cd404a0 0x55629cec7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.686+0000 7f1aba932b80 1 --2- 172.21.15.19:0/2475957168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629ce2a160 0x55629cec76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.686+0000 7f1a8d7fa700 0 client.26459 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:22:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.686+0000 7f1aba932b80 1 -- 172.21.15.19:0/2475957168 >> 172.21.15.19:0/2475957168 conn(0x55629cda06a0 msgr2=0x55629ceb81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:40.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.686+0000 7f1aba932b80 1 -- 172.21.15.19:0/2475957168 shutdown_connections 2024-05-09T02:22:40.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.686+0000 7f1aba932b80 1 -- 172.21.15.19:0/2475957168 wait complete. 2024-05-09T02:22:40.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.686+0000 7f1aba932b80 20 client.26459 trim_cache size 0 max 16384 2024-05-09T02:22:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.707+0000 7f1aba932b80 1 Processor -- start 2024-05-09T02:22:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.707+0000 7f1aba932b80 1 -- start start 2024-05-09T02:22:40.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:40 smithi019 ceph-mon[30151]: pgmap v354: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 1.37k op/s 2024-05-09T02:22:40.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.707+0000 7f1aba932b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 0x55629cec4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.707+0000 7f1aba932b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cec2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.707+0000 7f1aba932b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 0x55629ced30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.707+0000 7f1aba932b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8800d760 con 0x55629cebd6a0 2024-05-09T02:22:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.707+0000 7f1aba932b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8800f2a0 con 0x55629ce2a160 2024-05-09T02:22:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.707+0000 7f1aba932b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8800de30 con 0x55629cd404a0 2024-05-09T02:22:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.707+0000 7f1aa94b7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cec2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.707+0000 7f1aa94b7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cec2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:38764/0 (socket says 172.21.15.19:38764) 2024-05-09T02:22:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.707+0000 7f1aa94b7700 1 -- 172.21.15.19:0/2960316440 learned_addr learned my addr 172.21.15.19:0/2960316440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.707+0000 7f1aaa4b9700 1 --2- 172.21.15.19:0/2960316440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 0x55629ced30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.707+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/2960316440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 0x55629cec4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.708+0000 7f1aa9cb8700 1 -- 172.21.15.19:0/2960316440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 msgr2=0x55629cec2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.708+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/2960316440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cec2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.708+0000 7f1aa9cb8700 1 -- 172.21.15.19:0/2960316440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 msgr2=0x55629ced30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.708+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/2960316440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 0x55629ced30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.708+0000 7f1aa9cb8700 1 -- 172.21.15.19:0/2960316440 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629cebe0a0 con 0x55629cd404a0 2024-05-09T02:22:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.708+0000 7f1aa94b7700 1 --2- 172.21.15.19:0/2960316440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cec2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:22:40.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.708+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/2960316440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 0x55629cec4e70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00efd0 tx=0x7f1a9c017740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bce60c3a52eb005 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:40.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.708+0000 7f1a8f7fe700 1 -- 172.21.15.19:0/2960316440 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c027040 con 0x55629cd404a0 2024-05-09T02:22:40.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.708+0000 7f1a8f7fe700 1 -- 172.21.15.19:0/2960316440 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9c032070 con 0x55629cd404a0 2024-05-09T02:22:40.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.708+0000 7f1a8f7fe700 1 -- 172.21.15.19:0/2960316440 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c0159d0 con 0x55629cd404a0 2024-05-09T02:22:40.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.709+0000 7f1aba932b80 1 -- 172.21.15.19:0/2960316440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 msgr2=0x55629cec4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:40.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.709+0000 7f1aba932b80 1 --2- 172.21.15.19:0/2960316440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 0x55629cec4e70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00efd0 tx=0x7f1a9c017740 comp rx=0 tx=0).stop 2024-05-09T02:22:40.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.709+0000 7f1aba932b80 1 -- 172.21.15.19:0/2960316440 shutdown_connections 2024-05-09T02:22:40.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.709+0000 7f1aba932b80 1 --2- 172.21.15.19:0/2960316440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 0x55629ced30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.709+0000 7f1aba932b80 1 --2- 172.21.15.19:0/2960316440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 0x55629cec4e70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.709+0000 7f1aba932b80 1 --2- 172.21.15.19:0/2960316440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cec2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.709+0000 7f1aba932b80 1 -- 172.21.15.19:0/2960316440 >> 172.21.15.19:0/2960316440 conn(0x55629cda06a0 msgr2=0x55629cec2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:40.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.710+0000 7f1aba932b80 1 -- 172.21.15.19:0/2960316440 shutdown_connections 2024-05-09T02:22:40.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.710+0000 7f1aba932b80 1 -- 172.21.15.19:0/2960316440 wait complete. 2024-05-09T02:22:40.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.710+0000 7f1aba932b80 1 -- start start 2024-05-09T02:22:40.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.710+0000 7f1aba932b80 1 Processor -- start 2024-05-09T02:22:40.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.711+0000 7f1aba932b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 0x55629cecfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.711+0000 7f1aba932b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cede420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.711+0000 7f1aba932b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 0x55629cede960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.711+0000 7f1aba932b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8800ebd0 con 0x55629cebd6a0 2024-05-09T02:22:40.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.711+0000 7f1aba932b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8800e500 con 0x55629ce2a160 2024-05-09T02:22:40.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.711+0000 7f1aba932b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8800b720 con 0x55629cd404a0 2024-05-09T02:22:40.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.711+0000 7f1aa94b7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cede420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.711+0000 7f1aa94b7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cede420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:38768/0 (socket says 172.21.15.19:38768) 2024-05-09T02:22:40.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.711+0000 7f1aa94b7700 1 -- 172.21.15.19:0/764007239 learned_addr learned my addr 172.21.15.19:0/764007239 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.711+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/764007239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 0x55629cecfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.711+0000 7f1aaa4b9700 1 --2- 172.21.15.19:0/764007239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 0x55629cede960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.711+0000 7f1aba932b80 10 client.0 _ll_register_callbacks cb 0x55629cc516c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.711+0000 7f1aa9cb8700 1 -- 172.21.15.19:0/764007239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 msgr2=0x55629cede420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.712+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/764007239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cede420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.712+0000 7f1aa9cb8700 1 -- 172.21.15.19:0/764007239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 msgr2=0x55629cede960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.712+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/764007239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 0x55629cede960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.712+0000 7f1aa9cb8700 1 -- 172.21.15.19:0/764007239 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55629cedf110 con 0x55629cd404a0 2024-05-09T02:22:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.712+0000 7f1aaa4b9700 1 --2- 172.21.15.19:0/764007239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 0x55629cede960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:22:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.712+0000 7f1aa9cb8700 1 --2- 172.21.15.19:0/764007239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 0x55629cecfa80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00ee80 tx=0x7f1a9c017440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1965fdda08624ab8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.712+0000 7f1a8d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:22:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.712+0000 7f1a8d7fa700 1 -- 172.21.15.19:0/764007239 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c027050 con 0x55629cd404a0 2024-05-09T02:22:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.712+0000 7f1a8d7fa700 1 -- 172.21.15.19:0/764007239 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f1a9c0179b0 con 0x55629cd404a0 2024-05-09T02:22:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.712+0000 7f1aba932b80 1 -- 172.21.15.19:0/764007239 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a9c032070 con 0x55629cd404a0 2024-05-09T02:22:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.712+0000 7f1a8d7fa700 1 -- 172.21.15.19:0/764007239 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1a9c017bb0 con 0x55629cd404a0 2024-05-09T02:22:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.712+0000 7f1aba932b80 10 client.26497 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.712+0000 7f1aba932b80 1 -- 172.21.15.19:0/764007239 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55629cf418f0 con 0x55629cd404a0 2024-05-09T02:22:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.713+0000 7f1a8d7fa700 1 -- 172.21.15.19:0/764007239 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9c01e040 con 0x55629cd404a0 2024-05-09T02:22:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.713+0000 7f1a8d7fa700 1 -- 172.21.15.19:0/764007239 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f1a9c04b570 con 0x55629cd404a0 2024-05-09T02:22:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.713+0000 7f1aba932b80 10 client.26497 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.713+0000 7f1aba932b80 10 client.26497 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:22:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.713+0000 7f1aba932b80 20 client.26497 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.713+0000 7f1aba932b80 1 --2- 172.21.15.19:0/764007239 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55629cf41dd0 0x55629cf441f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.713+0000 7f1aba932b80 4 client.26497 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:22:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.713+0000 7f1aba932b80 1 -- 172.21.15.19:0/764007239 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1a9c032070 con 0x55629cf41dd0 2024-05-09T02:22:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.713+0000 7f1aa94b7700 1 --2- 172.21.15.19:0/764007239 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55629cf41dd0 0x55629cf441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.714+0000 7f1a8d7fa700 10 client.26497 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:22:40.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.714+0000 7f1aa94b7700 1 --2- 172.21.15.19:0/764007239 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55629cf41dd0 0x55629cf441f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=860ef60b683d56e1 server_cookie=a964bff6855d5bb3 in_seq=0 out_seq=0 2024-05-09T02:22:40.719 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:40 smithi059 ceph-mon[34781]: pgmap v354: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 1.37k op/s 2024-05-09T02:22:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.715+0000 7f1a8d7fa700 1 -- 172.21.15.19:0/764007239 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1a9c032070 con 0x55629cf41dd0 2024-05-09T02:22:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.715+0000 7f1a8d7fa700 10 client.26497 handle_command_reply: tid=0 2024-05-09T02:22:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.715+0000 7f1aba932b80 1 client.26497 shutdown 2024-05-09T02:22:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.718+0000 7f1aba932b80 1 -- 172.21.15.19:0/764007239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 msgr2=0x55629cecfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.718+0000 7f1aba932b80 1 --2- 172.21.15.19:0/764007239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 0x55629cecfa80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00ee80 tx=0x7f1a9c017440 comp rx=0 tx=0).stop 2024-05-09T02:22:40.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.718+0000 7f1aba932b80 1 -- 172.21.15.19:0/764007239 shutdown_connections 2024-05-09T02:22:40.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.718+0000 7f1aba932b80 1 --2- 172.21.15.19:0/764007239 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55629cf41dd0 0x55629cf441f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.718+0000 7f1aba932b80 1 --2- 172.21.15.19:0/764007239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55629cebd6a0 0x55629cede960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.718+0000 7f1a8d7fa700 0 client.26497 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:22:40.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.718+0000 7f1aba932b80 1 --2- 172.21.15.19:0/764007239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55629cd404a0 0x55629cecfa80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.718+0000 7f1aba932b80 1 --2- 172.21.15.19:0/764007239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55629ce2a160 0x55629cede420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.718+0000 7f1aba932b80 1 -- 172.21.15.19:0/764007239 >> 172.21.15.19:0/764007239 conn(0x55629cda06a0 msgr2=0x55629cec0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:40.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.718+0000 7f1aba932b80 1 -- 172.21.15.19:0/764007239 shutdown_connections 2024-05-09T02:22:40.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.718+0000 7f1aba932b80 1 -- 172.21.15.19:0/764007239 wait complete. 2024-05-09T02:22:40.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.719+0000 7f1aba932b80 20 client.26497 trim_cache size 0 max 16384 2024-05-09T02:22:40.721 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:22:40.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.719+0000 7f1aabf1c700 1 -- 172.21.15.19:0/2425313063 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1a900b2c70 msgr2=0x7f1a900b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:40.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.719+0000 7f1aabf1c700 1 --2- 172.21.15.19:0/2425313063 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1a900b2c70 0x7f1a900b5180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009c10 tx=0x7f1a9c00b040 comp rx=0 tx=0).stop 2024-05-09T02:22:40.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.719+0000 7f1aabf1c700 1 -- 172.21.15.19:0/2425313063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa41086a0 msgr2=0x7f1aa4111cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:40.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.719+0000 7f1aabf1c700 1 --2- 172.21.15.19:0/2425313063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa41086a0 0x7f1aa4111cf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000d720 tx=0x7f1aa000da30 comp rx=0 tx=0).stop 2024-05-09T02:22:40.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.719+0000 7f1aabf1c700 1 -- 172.21.15.19:0/2425313063 shutdown_connections 2024-05-09T02:22:40.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.719+0000 7f1aabf1c700 1 --2- 172.21.15.19:0/2425313063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa41086a0 0x7f1aa4111cf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.719+0000 7f1aabf1c700 1 --2- 172.21.15.19:0/2425313063 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1a900b2c70 0x7f1a900b5180 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.719+0000 7f1aabf1c700 1 --2- 172.21.15.19:0/2425313063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa4103250 0x7f1aa4117690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.719+0000 7f1aabf1c700 1 --2- 172.21.15.19:0/2425313063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1aa4103ba0 0x7f1aa41117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:40.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.719+0000 7f1aabf1c700 1 -- 172.21.15.19:0/2425313063 >> 172.21.15.19:0/2425313063 conn(0x7f1aa40fedc0 msgr2=0x7f1aa41060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:40.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.720+0000 7f1aabf1c700 1 -- 172.21.15.19:0/2425313063 shutdown_connections 2024-05-09T02:22:40.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:40.720+0000 7f1aabf1c700 1 -- 172.21.15.19:0/2425313063 wait complete. 2024-05-09T02:22:40.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:40 smithi196 ceph-mon[34718]: pgmap v354: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 1.37k op/s 2024-05-09T02:22:41.540 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T02:22:41.927 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:22:42.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:42 smithi019 ceph-mon[30151]: pgmap v355: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 1.56k op/s 2024-05-09T02:22:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 660.0 s, 1802.4 tps, lat 55.948 ms stddev 60.315 2024-05-09T02:22:42.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:42 smithi059 ceph-mon[34781]: pgmap v355: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 1.56k op/s 2024-05-09T02:22:42.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:42 smithi196 ceph-mon[34718]: pgmap v355: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 1.56k op/s 2024-05-09T02:22:44.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.066+0000 7f98cf82e700 1 -- 172.21.15.19:0/3130287429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98c8108f20 msgr2=0x7f98c8109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:44.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.066+0000 7f98cf82e700 1 --2- 172.21.15.19:0/3130287429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98c8108f20 0x7f98c8109330 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f98b80099c0 tx=0x7f98b8009cd0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.066+0000 7f98cf82e700 1 -- 172.21.15.19:0/3130287429 shutdown_connections 2024-05-09T02:22:44.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.066+0000 7f98cf82e700 1 --2- 172.21.15.19:0/3130287429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c8109870 0x7f98c81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.066+0000 7f98cf82e700 1 --2- 172.21.15.19:0/3130287429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98c81045f0 0x7f98c8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.066+0000 7f98cf82e700 1 --2- 172.21.15.19:0/3130287429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98c8108f20 0x7f98c8109330 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.066+0000 7f98cf82e700 1 -- 172.21.15.19:0/3130287429 >> 172.21.15.19:0/3130287429 conn(0x7f98c80fffe0 msgr2=0x7f98c8102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.066+0000 7f98cf82e700 1 -- 172.21.15.19:0/3130287429 shutdown_connections 2024-05-09T02:22:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.066+0000 7f98cf82e700 1 -- 172.21.15.19:0/3130287429 wait complete. 2024-05-09T02:22:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.067+0000 7f98cf82e700 1 Processor -- start 2024-05-09T02:22:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.067+0000 7f98cf82e700 1 -- start start 2024-05-09T02:22:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.068+0000 7f98cf82e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81045f0 0x7f98c819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.068+0000 7f98cf82e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98c8108f20 0x7f98c819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.068+0000 7f98cd5ca700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81045f0 0x7f98c819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.068+0000 7f98cd5ca700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81045f0 0x7f98c819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:38784/0 (socket says 172.21.15.19:38784) 2024-05-09T02:22:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.068+0000 7f98cd5ca700 1 -- 172.21.15.19:0/612770139 learned_addr learned my addr 172.21.15.19:0/612770139 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.068+0000 7f98ccdc9700 1 --2- 172.21.15.19:0/612770139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98c8108f20 0x7f98c819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.068+0000 7f98cf82e700 1 --2- 172.21.15.19:0/612770139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98c8109870 0x7f98c819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.068+0000 7f98cf82e700 1 -- 172.21.15.19:0/612770139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98c8114480 con 0x7f98c8109870 2024-05-09T02:22:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.068+0000 7f98cf82e700 1 -- 172.21.15.19:0/612770139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98c8114300 con 0x7f98c81045f0 2024-05-09T02:22:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.068+0000 7f98cf82e700 1 -- 172.21.15.19:0/612770139 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98c8114600 con 0x7f98c8108f20 2024-05-09T02:22:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.069+0000 7f98cd5ca700 1 -- 172.21.15.19:0/612770139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98c8108f20 msgr2=0x7f98c819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.069+0000 7f98cd5ca700 1 --2- 172.21.15.19:0/612770139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98c8108f20 0x7f98c819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.069+0000 7f98cd5ca700 1 -- 172.21.15.19:0/612770139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98c8109870 msgr2=0x7f98c819f640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T02:22:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.069+0000 7f98cd5ca700 1 --2- 172.21.15.19:0/612770139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98c8109870 0x7f98c819f640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.069+0000 7f98cd5ca700 1 -- 172.21.15.19:0/612770139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c819fce0 con 0x7f98c81045f0 2024-05-09T02:22:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.069+0000 7f98cd5ca700 1 --2- 172.21.15.19:0/612770139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81045f0 0x7f98c819ae10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f98c400e9a0 tx=0x7f98c400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.070+0000 7f98be7fc700 1 -- 172.21.15.19:0/612770139 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c400cbd0 con 0x7f98c81045f0 2024-05-09T02:22:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.070+0000 7f98be7fc700 1 -- 172.21.15.19:0/612770139 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c400cd70 con 0x7f98c81045f0 2024-05-09T02:22:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.070+0000 7f98cf82e700 1 -- 172.21.15.19:0/612770139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c819ffd0 con 0x7f98c81045f0 2024-05-09T02:22:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.070+0000 7f98be7fc700 1 -- 172.21.15.19:0/612770139 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4018680 con 0x7f98c81045f0 2024-05-09T02:22:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.070+0000 7f98cf82e700 1 -- 172.21.15.19:0/612770139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c81a78f0 con 0x7f98c81045f0 2024-05-09T02:22:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.073+0000 7f98be7fc700 1 -- 172.21.15.19:0/612770139 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f98c4018820 con 0x7f98c81045f0 2024-05-09T02:22:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.074+0000 7f98be7fc700 1 --2- 172.21.15.19:0/612770139 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f98b40b2af0 0x7f98b40b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.074+0000 7f98de244b80 1 Processor -- start 2024-05-09T02:22:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.074+0000 7f98ccdc9700 1 --2- 172.21.15.19:0/612770139 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f98b40b2af0 0x7f98b40b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.074+0000 7f98de244b80 1 -- start start 2024-05-09T02:22:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.074+0000 7f98be7fc700 1 -- 172.21.15.19:0/612770139 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f98c40e2a40 con 0x7f98c81045f0 2024-05-09T02:22:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.074+0000 7f98de244b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc77b9160 0x557cc78524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.074+0000 7f98de244b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc76cf4a0 0x557cc7852a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.074+0000 7f98de244b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc784c6a0 0x557cc784cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.074+0000 7f98ccdc9700 1 --2- 172.21.15.19:0/612770139 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f98b40b2af0 0x7f98b40b5000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f98b8000c00 tx=0x7f98b801a040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.074+0000 7f98de244b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557cc776eff0 con 0x557cc77b9160 2024-05-09T02:22:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.075+0000 7f98cd5ca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc77b9160 0x557cc78524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.075+0000 7f98cd5ca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc77b9160 0x557cc78524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36520/0 (socket says 172.21.15.19:36520) 2024-05-09T02:22:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.075+0000 7f98cd5ca700 1 -- 172.21.15.19:0/13049473 learned_addr learned my addr 172.21.15.19:0/13049473 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.075+0000 7f98de244b80 1 -- 172.21.15.19:0/13049473 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557cc770d7f0 con 0x557cc76cf4a0 2024-05-09T02:22:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.075+0000 7f98cddcb700 1 --2- 172.21.15.19:0/13049473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc784c6a0 0x557cc784cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.075+0000 7f98de244b80 1 -- 172.21.15.19:0/13049473 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557cc776afb0 con 0x557cc784c6a0 2024-05-09T02:22:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.075+0000 7f98ccdc9700 1 --2- 172.21.15.19:0/13049473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc76cf4a0 0x557cc7852a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.075+0000 7f98cd5ca700 1 -- 172.21.15.19:0/13049473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc784c6a0 msgr2=0x557cc784cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.075+0000 7f98cd5ca700 1 --2- 172.21.15.19:0/13049473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc784c6a0 0x557cc784cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.075+0000 7f98cd5ca700 1 -- 172.21.15.19:0/13049473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc76cf4a0 msgr2=0x557cc7852a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.075+0000 7f98cd5ca700 1 --2- 172.21.15.19:0/13049473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc76cf4a0 0x557cc7852a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.075+0000 7f98cd5ca700 1 -- 172.21.15.19:0/13049473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc7705370 con 0x557cc77b9160 2024-05-09T02:22:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.075+0000 7f98cd5ca700 1 --2- 172.21.15.19:0/13049473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc77b9160 0x557cc78524f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f98c40a9c20 tx=0x7f98c4009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bfe2649053325 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.075+0000 7f98b37fe700 1 -- 172.21.15.19:0/13049473 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c40e0590 con 0x557cc77b9160 2024-05-09T02:22:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.076+0000 7f98b37fe700 1 -- 172.21.15.19:0/13049473 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c40a97a0 con 0x557cc77b9160 2024-05-09T02:22:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.076+0000 7f98b37fe700 1 -- 172.21.15.19:0/13049473 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c40a9ef0 con 0x557cc77b9160 2024-05-09T02:22:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.077+0000 7f98de244b80 1 -- 172.21.15.19:0/13049473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc77b9160 msgr2=0x557cc78524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.077+0000 7f98de244b80 1 --2- 172.21.15.19:0/13049473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc77b9160 0x557cc78524f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f98c40a9c20 tx=0x7f98c4009640 comp rx=0 tx=0).stop 2024-05-09T02:22:44.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.077+0000 7f98de244b80 1 -- 172.21.15.19:0/13049473 shutdown_connections 2024-05-09T02:22:44.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.077+0000 7f98de244b80 1 --2- 172.21.15.19:0/13049473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc784c6a0 0x557cc784cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.078+0000 7f98de244b80 1 --2- 172.21.15.19:0/13049473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc77b9160 0x557cc78524f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.078+0000 7f98de244b80 1 --2- 172.21.15.19:0/13049473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc76cf4a0 0x557cc7852a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.078+0000 7f98de244b80 1 -- 172.21.15.19:0/13049473 >> 172.21.15.19:0/13049473 conn(0x557cc772f6a0 msgr2=0x557cc7847400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:44.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.078+0000 7f98de244b80 1 -- 172.21.15.19:0/13049473 shutdown_connections 2024-05-09T02:22:44.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.078+0000 7f98de244b80 1 -- 172.21.15.19:0/13049473 wait complete. 2024-05-09T02:22:44.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.078+0000 7f98de244b80 1 -- start start 2024-05-09T02:22:44.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.078+0000 7f98de244b80 1 Processor -- start 2024-05-09T02:22:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.079+0000 7f98de244b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc76cf4a0 0x557cc7856260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.079+0000 7f98de244b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc77b9160 0x557cc78567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.079+0000 7f98de244b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc784c6a0 0x557cc7856ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.079+0000 7f98de244b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557cc75eaba0 con 0x557cc77b9160 2024-05-09T02:22:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.079+0000 7f98de244b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557cc7729150 con 0x557cc784c6a0 2024-05-09T02:22:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.079+0000 7f98de244b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557cc78591d0 con 0x557cc76cf4a0 2024-05-09T02:22:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.079+0000 7f98de244b80 10 client.0 _ll_register_callbacks cb 0x557cc75e06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.079+0000 7f98ccdc9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc77b9160 0x557cc78567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.079+0000 7f98cd5ca700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc76cf4a0 0x557cc7856260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.079+0000 7f98cddcb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc784c6a0 0x557cc7856ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.079+0000 7f98cd5ca700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc76cf4a0 0x557cc7856260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:36462/0 (socket says 172.21.15.19:36462) 2024-05-09T02:22:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.079+0000 7f98ccdc9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc77b9160 0x557cc78567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36532/0 (socket says 172.21.15.19:36532) 2024-05-09T02:22:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.079+0000 7f98cd5ca700 1 -- 172.21.15.19:0/3066115303 learned_addr learned my addr 172.21.15.19:0/3066115303 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.080+0000 7f98cddcb700 1 -- 172.21.15.19:0/3066115303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc76cf4a0 msgr2=0x557cc7856260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.080+0000 7f98cddcb700 1 --2- 172.21.15.19:0/3066115303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc76cf4a0 0x557cc7856260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.080+0000 7f98cddcb700 1 -- 172.21.15.19:0/3066115303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc77b9160 msgr2=0x557cc78567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.080+0000 7f98cddcb700 1 --2- 172.21.15.19:0/3066115303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc77b9160 0x557cc78567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.080+0000 7f98cddcb700 1 -- 172.21.15.19:0/3066115303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cc75ea490 con 0x557cc784c6a0 2024-05-09T02:22:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.080+0000 7f98cddcb700 1 --2- 172.21.15.19:0/3066115303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc784c6a0 0x557cc7856ce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f98c0011b80 tx=0x7f98c0011e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c37e676831777514 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.080+0000 7f98b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:22:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.080+0000 7f98b0ff9700 1 -- 172.21.15.19:0/3066115303 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c001a070 con 0x557cc784c6a0 2024-05-09T02:22:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.080+0000 7f98b0ff9700 1 -- 172.21.15.19:0/3066115303 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f98c0004db0 con 0x557cc784c6a0 2024-05-09T02:22:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.082+0000 7f98de244b80 1 -- 172.21.15.19:0/3066115303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c40a97a0 con 0x557cc784c6a0 2024-05-09T02:22:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.082+0000 7f98b0ff9700 1 -- 172.21.15.19:0/3066115303 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f98c00093a0 con 0x557cc784c6a0 2024-05-09T02:22:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.082+0000 7f98de244b80 10 client.26477 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.082+0000 7f98de244b80 1 -- 172.21.15.19:0/3066115303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557cc7863c90 con 0x557cc784c6a0 2024-05-09T02:22:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.083+0000 7f98b0ff9700 1 -- 172.21.15.19:0/3066115303 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c0004030 con 0x557cc784c6a0 2024-05-09T02:22:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.083+0000 7f98b0ff9700 1 -- 172.21.15.19:0/3066115303 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f98c001d020 con 0x557cc784c6a0 2024-05-09T02:22:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.083+0000 7f98de244b80 10 client.26477 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.083+0000 7f98de244b80 10 client.26477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:22:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.083+0000 7f98de244b80 20 client.26477 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.083+0000 7f98de244b80 1 --2- 172.21.15.19:0/3066115303 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557cc7866310 0x557cc7868730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.083+0000 7f98de244b80 4 client.26477 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:22:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.083+0000 7f98de244b80 1 -- 172.21.15.19:0/3066115303 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f98c40a97a0 con 0x557cc7866310 2024-05-09T02:22:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.083+0000 7f98cd5ca700 1 --2- 172.21.15.19:0/3066115303 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557cc7866310 0x557cc7868730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.084+0000 7f98b0ff9700 10 client.26477 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.084+0000 7f98cd5ca700 1 --2- 172.21.15.19:0/3066115303 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557cc7866310 0x557cc7868730 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de836863906acda4 server_cookie=6e52a4f02e4c36c8 in_seq=0 out_seq=0 2024-05-09T02:22:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.085+0000 7f98b0ff9700 1 -- 172.21.15.19:0/3066115303 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f98c40a97a0 con 0x557cc7866310 2024-05-09T02:22:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.085+0000 7f98b0ff9700 10 client.26477 handle_command_reply: tid=0 2024-05-09T02:22:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.085+0000 7f98de244b80 1 client.26477 shutdown 2024-05-09T02:22:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.085+0000 7f98de244b80 1 -- 172.21.15.19:0/3066115303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc784c6a0 msgr2=0x557cc7856ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.085+0000 7f98de244b80 1 --2- 172.21.15.19:0/3066115303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc784c6a0 0x557cc7856ce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f98c0011b80 tx=0x7f98c0011e90 comp rx=0 tx=0).stop 2024-05-09T02:22:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.085+0000 7f98de244b80 1 -- 172.21.15.19:0/3066115303 shutdown_connections 2024-05-09T02:22:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.085+0000 7f98de244b80 1 --2- 172.21.15.19:0/3066115303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc784c6a0 0x557cc7856ce0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.085+0000 7f98de244b80 1 --2- 172.21.15.19:0/3066115303 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557cc7866310 0x557cc7868730 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.085+0000 7f98b0ff9700 0 client.26477 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.085+0000 7f98de244b80 1 --2- 172.21.15.19:0/3066115303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc76cf4a0 0x557cc7856260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.085+0000 7f98de244b80 1 --2- 172.21.15.19:0/3066115303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc77b9160 0x557cc78567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.085+0000 7f98de244b80 1 -- 172.21.15.19:0/3066115303 >> 172.21.15.19:0/3066115303 conn(0x557cc772f6a0 msgr2=0x557cc78471c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.085+0000 7f98de244b80 1 -- 172.21.15.19:0/3066115303 shutdown_connections 2024-05-09T02:22:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.085+0000 7f98de244b80 1 -- 172.21.15.19:0/3066115303 wait complete. 2024-05-09T02:22:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.086+0000 7f98de244b80 20 client.26477 trim_cache size 0 max 16384 2024-05-09T02:22:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.107+0000 7f98de244b80 1 Processor -- start 2024-05-09T02:22:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.107+0000 7f98de244b80 1 -- start start 2024-05-09T02:22:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.107+0000 7f98de244b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc76cf4a0 0x557cc7854b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.107+0000 7f98de244b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc77b9160 0x557cc7855090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.107+0000 7f98de244b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc784c6a0 0x557cc786def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.107+0000 7f98de244b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81ace40 con 0x557cc784c6a0 2024-05-09T02:22:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.107+0000 7f98de244b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81ae980 con 0x557cc76cf4a0 2024-05-09T02:22:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.107+0000 7f98de244b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81ad510 con 0x557cc77b9160 2024-05-09T02:22:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.107+0000 7f98cd5ca700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc76cf4a0 0x557cc7854b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.107+0000 7f98cddcb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc784c6a0 0x557cc786def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.107+0000 7f98cd5ca700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc76cf4a0 0x557cc7854b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:38802/0 (socket says 172.21.15.19:38802) 2024-05-09T02:22:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.107+0000 7f98cd5ca700 1 -- 172.21.15.19:0/2455907325 learned_addr learned my addr 172.21.15.19:0/2455907325 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.107+0000 7f98ccdc9700 1 --2- 172.21.15.19:0/2455907325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc77b9160 0x557cc7855090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.108+0000 7f98cd5ca700 1 -- 172.21.15.19:0/2455907325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc77b9160 msgr2=0x557cc7855090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.108+0000 7f98cd5ca700 1 --2- 172.21.15.19:0/2455907325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc77b9160 0x557cc7855090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.108+0000 7f98cd5ca700 1 -- 172.21.15.19:0/2455907325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc784c6a0 msgr2=0x557cc786def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.108+0000 7f98cd5ca700 1 --2- 172.21.15.19:0/2455907325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc784c6a0 0x557cc786def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.108+0000 7f98cd5ca700 1 -- 172.21.15.19:0/2455907325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc7846f50 con 0x557cc76cf4a0 2024-05-09T02:22:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.108+0000 7f98ccdc9700 1 --2- 172.21.15.19:0/2455907325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc77b9160 0x557cc7855090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:22:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.108+0000 7f98cd5ca700 1 --2- 172.21.15.19:0/2455907325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc76cf4a0 0x557cc7854b50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f98c40a9990 tx=0x7f98c4010c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e808bbc60aa05961 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.108+0000 7f98b2ffd700 1 -- 172.21.15.19:0/2455907325 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c40aa390 con 0x557cc76cf4a0 2024-05-09T02:22:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.108+0000 7f98b2ffd700 1 -- 172.21.15.19:0/2455907325 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c40ae040 con 0x557cc76cf4a0 2024-05-09T02:22:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.108+0000 7f98b2ffd700 1 -- 172.21.15.19:0/2455907325 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c40a9ef0 con 0x557cc76cf4a0 2024-05-09T02:22:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.109+0000 7f98de244b80 1 -- 172.21.15.19:0/2455907325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc76cf4a0 msgr2=0x557cc7854b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.109+0000 7f98de244b80 1 --2- 172.21.15.19:0/2455907325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc76cf4a0 0x557cc7854b50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f98c40a9990 tx=0x7f98c4010c10 comp rx=0 tx=0).stop 2024-05-09T02:22:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.109+0000 7f98de244b80 1 -- 172.21.15.19:0/2455907325 shutdown_connections 2024-05-09T02:22:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.109+0000 7f98de244b80 1 --2- 172.21.15.19:0/2455907325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc784c6a0 0x557cc786def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.109+0000 7f98de244b80 1 --2- 172.21.15.19:0/2455907325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc76cf4a0 0x557cc7854b50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.109+0000 7f98de244b80 1 --2- 172.21.15.19:0/2455907325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc77b9160 0x557cc7855090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.109+0000 7f98de244b80 1 -- 172.21.15.19:0/2455907325 >> 172.21.15.19:0/2455907325 conn(0x557cc772f6a0 msgr2=0x557cc78524f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.109+0000 7f98de244b80 1 -- 172.21.15.19:0/2455907325 shutdown_connections 2024-05-09T02:22:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.109+0000 7f98de244b80 1 -- 172.21.15.19:0/2455907325 wait complete. 2024-05-09T02:22:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.109+0000 7f98de244b80 1 -- start start 2024-05-09T02:22:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.110+0000 7f98de244b80 1 Processor -- start 2024-05-09T02:22:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.110+0000 7f98de244b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc76cf4a0 0x557cc785e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.110+0000 7f98de244b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc77b9160 0x557cc786d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.110+0000 7f98de244b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc784c6a0 0x557cc786da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.110+0000 7f98de244b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81ae2b0 con 0x557cc76cf4a0 2024-05-09T02:22:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.110+0000 7f98de244b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81adbe0 con 0x557cc77b9160 2024-05-09T02:22:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.110+0000 7f98de244b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81aae20 con 0x557cc784c6a0 2024-05-09T02:22:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.110+0000 7f98de244b80 10 client.0 _ll_register_callbacks cb 0x557cc75e06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.110+0000 7f98ccdc9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc77b9160 0x557cc786d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.110+0000 7f98cd5ca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc76cf4a0 0x557cc785e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.110+0000 7f98ccdc9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc77b9160 0x557cc786d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:38818/0 (socket says 172.21.15.19:38818) 2024-05-09T02:22:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.110+0000 7f98ccdc9700 1 -- 172.21.15.19:0/592938789 learned_addr learned my addr 172.21.15.19:0/592938789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.110+0000 7f98cddcb700 1 --2- 172.21.15.19:0/592938789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc784c6a0 0x557cc786da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.112+0000 7f98cd5ca700 1 -- 172.21.15.19:0/592938789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc784c6a0 msgr2=0x557cc786da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.112+0000 7f98cd5ca700 1 --2- 172.21.15.19:0/592938789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc784c6a0 0x557cc786da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.112+0000 7f98cd5ca700 1 -- 172.21.15.19:0/592938789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc77b9160 msgr2=0x557cc786d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.112+0000 7f98cd5ca700 1 --2- 172.21.15.19:0/592938789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc77b9160 0x557cc786d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.112+0000 7f98cd5ca700 1 -- 172.21.15.19:0/592938789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cc786e080 con 0x557cc76cf4a0 2024-05-09T02:22:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.112+0000 7f98cd5ca700 1 --2- 172.21.15.19:0/592938789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc76cf4a0 0x557cc785e8d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f98c40a9990 tx=0x7f98c40a5c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed30b4e27cd411f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.112+0000 7f98b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:22:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.113+0000 7f98b0ff9700 1 -- 172.21.15.19:0/592938789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c40aa390 con 0x557cc76cf4a0 2024-05-09T02:22:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.113+0000 7f98b0ff9700 1 -- 172.21.15.19:0/592938789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f98c40152d0 con 0x557cc76cf4a0 2024-05-09T02:22:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.113+0000 7f98de244b80 1 -- 172.21.15.19:0/592938789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c40ae040 con 0x557cc76cf4a0 2024-05-09T02:22:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.114+0000 7f98b0ff9700 1 -- 172.21.15.19:0/592938789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f98c40154d0 con 0x557cc76cf4a0 2024-05-09T02:22:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.114+0000 7f98de244b80 10 client.16749 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.114+0000 7f98de244b80 1 -- 172.21.15.19:0/592938789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557cc78d08d0 con 0x557cc76cf4a0 2024-05-09T02:22:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.114+0000 7f98b0ff9700 1 -- 172.21.15.19:0/592938789 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c408f070 con 0x557cc76cf4a0 2024-05-09T02:22:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.114+0000 7f98b0ff9700 1 -- 172.21.15.19:0/592938789 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f98c40ca020 con 0x557cc76cf4a0 2024-05-09T02:22:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.114+0000 7f98de244b80 10 client.16749 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.114+0000 7f98de244b80 10 client.16749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:22:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.114+0000 7f98de244b80 20 client.16749 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.114+0000 7f98de244b80 1 --2- 172.21.15.19:0/592938789 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557cc7866310 0x557cc78d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.114+0000 7f98de244b80 4 client.16749 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:22:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.114+0000 7f98de244b80 1 -- 172.21.15.19:0/592938789 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f98c40ae040 con 0x557cc7866310 2024-05-09T02:22:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.114+0000 7f98ccdc9700 1 --2- 172.21.15.19:0/592938789 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557cc7866310 0x557cc78d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.115+0000 7f98b0ff9700 10 client.16749 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.115+0000 7f98ccdc9700 1 --2- 172.21.15.19:0/592938789 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557cc7866310 0x557cc78d2b00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f7c8f8f37790477 server_cookie=f7968a5f9a56ae34 in_seq=0 out_seq=0 2024-05-09T02:22:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.115+0000 7f98b0ff9700 1 -- 172.21.15.19:0/592938789 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f98c40ae040 con 0x557cc7866310 2024-05-09T02:22:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.115+0000 7f98b0ff9700 10 client.16749 handle_command_reply: tid=0 2024-05-09T02:22:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.115+0000 7f98de244b80 1 client.16749 shutdown 2024-05-09T02:22:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.116+0000 7f98de244b80 1 -- 172.21.15.19:0/592938789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc76cf4a0 msgr2=0x557cc785e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.116+0000 7f98de244b80 1 --2- 172.21.15.19:0/592938789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc76cf4a0 0x557cc785e8d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f98c40a9990 tx=0x7f98c40a5c80 comp rx=0 tx=0).stop 2024-05-09T02:22:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.116+0000 7f98de244b80 1 -- 172.21.15.19:0/592938789 shutdown_connections 2024-05-09T02:22:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.116+0000 7f98de244b80 1 --2- 172.21.15.19:0/592938789 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557cc7866310 0x557cc78d2b00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.116+0000 7f98de244b80 1 --2- 172.21.15.19:0/592938789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557cc784c6a0 0x557cc786da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.116+0000 7f98b0ff9700 0 client.16749 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.116+0000 7f98de244b80 1 --2- 172.21.15.19:0/592938789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cc76cf4a0 0x557cc785e8d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.116+0000 7f98de244b80 1 --2- 172.21.15.19:0/592938789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cc77b9160 0x557cc786d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.303 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:22:44.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.116+0000 7f98de244b80 1 -- 172.21.15.19:0/592938789 >> 172.21.15.19:0/592938789 conn(0x557cc772f6a0 msgr2=0x557cc78524f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:44.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.116+0000 7f98de244b80 1 -- 172.21.15.19:0/592938789 shutdown_connections 2024-05-09T02:22:44.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.116+0000 7f98de244b80 1 -- 172.21.15.19:0/592938789 wait complete. 2024-05-09T02:22:44.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.116+0000 7f98de244b80 20 client.16749 trim_cache size 0 max 16384 2024-05-09T02:22:44.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.117+0000 7f98cf82e700 1 -- 172.21.15.19:0/612770139 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f98b40b2af0 msgr2=0x7f98b40b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:44.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.117+0000 7f98cf82e700 1 --2- 172.21.15.19:0/612770139 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f98b40b2af0 0x7f98b40b5000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f98b8000c00 tx=0x7f98b801a040 comp rx=0 tx=0).stop 2024-05-09T02:22:44.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.117+0000 7f98cf82e700 1 -- 172.21.15.19:0/612770139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81045f0 msgr2=0x7f98c819ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:44.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.117+0000 7f98cf82e700 1 --2- 172.21.15.19:0/612770139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81045f0 0x7f98c819ae10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f98c400e9a0 tx=0x7f98c400ecb0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.117+0000 7f98cf82e700 1 -- 172.21.15.19:0/612770139 shutdown_connections 2024-05-09T02:22:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.117+0000 7f98cf82e700 1 --2- 172.21.15.19:0/612770139 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f98b40b2af0 0x7f98b40b5000 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.117+0000 7f98cf82e700 1 --2- 172.21.15.19:0/612770139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98c8109870 0x7f98c819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.117+0000 7f98cf82e700 1 --2- 172.21.15.19:0/612770139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81045f0 0x7f98c819ae10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.117+0000 7f98cf82e700 1 --2- 172.21.15.19:0/612770139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98c8108f20 0x7f98c819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.117+0000 7f98cf82e700 1 -- 172.21.15.19:0/612770139 >> 172.21.15.19:0/612770139 conn(0x7f98c80fffe0 msgr2=0x7f98c8106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.117+0000 7f98cf82e700 1 -- 172.21.15.19:0/612770139 shutdown_connections 2024-05-09T02:22:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:44.117+0000 7f98cf82e700 1 -- 172.21.15.19:0/612770139 wait complete. 2024-05-09T02:22:44.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:44 smithi196 ceph-mon[34718]: pgmap v356: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 1.47k op/s 2024-05-09T02:22:44.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:44 smithi019 ceph-mon[30151]: pgmap v356: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 1.47k op/s 2024-05-09T02:22:44.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:44 smithi059 ceph-mon[34781]: pgmap v356: 97 pgs: 97 active+clean; 4.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 1.47k op/s 2024-05-09T02:22:46.310 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T02:22:46.310 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T02:22:46.676 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:22:46.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:46 smithi196 ceph-mon[34718]: pgmap v357: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 18 MiB/s wr, 1.74k op/s 2024-05-09T02:22:46.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:46 smithi019 ceph-mon[30151]: pgmap v357: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 18 MiB/s wr, 1.74k op/s 2024-05-09T02:22:46.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:46 smithi059 ceph-mon[34781]: pgmap v357: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 18 MiB/s wr, 1.74k op/s 2024-05-09T02:22:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 665.0 s, 1741.4 tps, lat 57.197 ms stddev 64.452 2024-05-09T02:22:48.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:48 smithi196 ceph-mon[34718]: pgmap v358: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 1.48k op/s 2024-05-09T02:22:48.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:48 smithi019 ceph-mon[30151]: pgmap v358: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 1.48k op/s 2024-05-09T02:22:48.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:48 smithi059 ceph-mon[34781]: pgmap v358: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 1.48k op/s 2024-05-09T02:22:49.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.171+0000 7f0e88aca700 1 -- 172.21.15.19:0/1772457423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e841034e0 msgr2=0x7f0e841038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:49.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.171+0000 7f0e88aca700 1 --2- 172.21.15.19:0/1772457423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e841034e0 0x7f0e841038f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400dda0 tx=0x7f0e740084b0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.172+0000 7f0e88aca700 1 -- 172.21.15.19:0/1772457423 shutdown_connections 2024-05-09T02:22:49.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.172+0000 7f0e88aca700 1 --2- 172.21.15.19:0/1772457423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e84108760 0x7f0e841100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.172+0000 7f0e88aca700 1 --2- 172.21.15.19:0/1772457423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e841034e0 0x7f0e841038f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.172+0000 7f0e88aca700 1 --2- 172.21.15.19:0/1772457423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e84107e10 0x7f0e84108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.172+0000 7f0e88aca700 1 -- 172.21.15.19:0/1772457423 >> 172.21.15.19:0/1772457423 conn(0x7f0e840fef30 msgr2=0x7f0e84101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:49.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.172+0000 7f0e88aca700 1 -- 172.21.15.19:0/1772457423 shutdown_connections 2024-05-09T02:22:49.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.172+0000 7f0e88aca700 1 -- 172.21.15.19:0/1772457423 wait complete. 2024-05-09T02:22:49.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.173+0000 7f0e88aca700 1 Processor -- start 2024-05-09T02:22:49.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.174+0000 7f0e88aca700 1 -- start start 2024-05-09T02:22:49.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.174+0000 7f0e88aca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e841034e0 0x7f0e8419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.174+0000 7f0e88aca700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e84107e10 0x7f0e8419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.174+0000 7f0e88aca700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e84108760 0x7f0e84198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.174+0000 7f0e88aca700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841122d0 con 0x7f0e841034e0 2024-05-09T02:22:49.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.174+0000 7f0e88aca700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e84112150 con 0x7f0e84108760 2024-05-09T02:22:49.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.174+0000 7f0e88aca700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e84112450 con 0x7f0e84107e10 2024-05-09T02:22:49.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.174+0000 7f0e82d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e84108760 0x7f0e84198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.174+0000 7f0e82d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e84108760 0x7f0e84198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56904/0 (socket says 172.21.15.19:56904) 2024-05-09T02:22:49.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.174+0000 7f0e82d9d700 1 -- 172.21.15.19:0/3493612462 learned_addr learned my addr 172.21.15.19:0/3493612462 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:49.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.174+0000 7f0e8259c700 1 --2- 172.21.15.19:0/3493612462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e841034e0 0x7f0e8419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.174+0000 7f0e81d9b700 1 --2- 172.21.15.19:0/3493612462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e84107e10 0x7f0e8419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.175+0000 7f0e81d9b700 1 -- 172.21.15.19:0/3493612462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e84108760 msgr2=0x7f0e84198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:49.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.175+0000 7f0e81d9b700 1 --2- 172.21.15.19:0/3493612462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e84108760 0x7f0e84198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.175+0000 7f0e81d9b700 1 -- 172.21.15.19:0/3493612462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e841034e0 msgr2=0x7f0e8419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:49.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.176+0000 7f0e81d9b700 1 --2- 172.21.15.19:0/3493612462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e841034e0 0x7f0e8419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.176+0000 7f0e81d9b700 1 -- 172.21.15.19:0/3493612462 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e84199160 con 0x7f0e84107e10 2024-05-09T02:22:49.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.176+0000 7f0e8259c700 1 --2- 172.21.15.19:0/3493612462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e841034e0 0x7f0e8419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:22:49.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.176+0000 7f0e81d9b700 1 --2- 172.21.15.19:0/3493612462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e84107e10 0x7f0e8419edd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0e78016fb0 tx=0x7f0e780143a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:49.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.177+0000 7f0e737fe700 1 -- 172.21.15.19:0/3493612462 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78014c30 con 0x7f0e84107e10 2024-05-09T02:22:49.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.177+0000 7f0e737fe700 1 -- 172.21.15.19:0/3493612462 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e78014dd0 con 0x7f0e84107e10 2024-05-09T02:22:49.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.177+0000 7f0e88aca700 1 -- 172.21.15.19:0/3493612462 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e841993f0 con 0x7f0e84107e10 2024-05-09T02:22:49.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.177+0000 7f0e88aca700 1 -- 172.21.15.19:0/3493612462 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e84199870 con 0x7f0e84107e10 2024-05-09T02:22:49.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.178+0000 7f0e737fe700 1 -- 172.21.15.19:0/3493612462 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78011450 con 0x7f0e84107e10 2024-05-09T02:22:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.185+0000 7f0e737fe700 1 -- 172.21.15.19:0/3493612462 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0e7801a050 con 0x7f0e84107e10 2024-05-09T02:22:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.186+0000 7f0e974e0b80 1 Processor -- start 2024-05-09T02:22:49.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.187+0000 7f0e737fe700 1 --2- 172.21.15.19:0/3493612462 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0e6c0b2bc0 0x7f0e6c0b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.188+0000 7f0e737fe700 1 -- 172.21.15.19:0/3493612462 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f0e780e9da0 con 0x7f0e84107e10 2024-05-09T02:22:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.199+0000 7f0e974e0b80 1 -- start start 2024-05-09T02:22:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.200+0000 7f0e974e0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afea28d80 0x557afeb312c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.200+0000 7f0e974e0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afe9ae520 0x557afeb31800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.200+0000 7f0e974e0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afeb2b480 0x557afeb2b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.200+0000 7f0e974e0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557afea4e120 con 0x557afeb2b480 2024-05-09T02:22:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.200+0000 7f0e974e0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557afe9f3a20 con 0x557afea28d80 2024-05-09T02:22:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.200+0000 7f0e974e0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557afea4a0e0 con 0x557afe9ae520 2024-05-09T02:22:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.201+0000 7f0e82d9d700 1 --2- 172.21.15.19:0/3493612462 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0e6c0b2bc0 0x7f0e6c0b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.201+0000 7f0e82d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afeb2b480 0x557afeb2b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.201+0000 7f0e81d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afe9ae520 0x557afeb31800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.201+0000 7f0e82d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afeb2b480 0x557afeb2b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45180/0 (socket says 172.21.15.19:45180) 2024-05-09T02:22:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.201+0000 7f0e82d9d700 1 -- 172.21.15.19:0/407509549 learned_addr learned my addr 172.21.15.19:0/407509549 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.201+0000 7f0e8259c700 1 --2- 172.21.15.19:0/407509549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afea28d80 0x557afeb312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.202+0000 7f0e82d9d700 1 -- 172.21.15.19:0/407509549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afe9ae520 msgr2=0x557afeb31800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:49.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.202+0000 7f0e82d9d700 1 --2- 172.21.15.19:0/407509549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afe9ae520 0x557afeb31800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.202+0000 7f0e82d9d700 1 -- 172.21.15.19:0/407509549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afea28d80 msgr2=0x557afeb312c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:49.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.202+0000 7f0e82d9d700 1 --2- 172.21.15.19:0/407509549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afea28d80 0x557afeb312c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.202+0000 7f0e82d9d700 1 -- 172.21.15.19:0/407509549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557afe9e4580 con 0x557afeb2b480 2024-05-09T02:22:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.207+0000 7f0e82d9d700 1 --2- 172.21.15.19:0/407509549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afeb2b480 0x557afeb2b890 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c008ea0 tx=0x7f0e7c01a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55722a281ecf844 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:49.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.207+0000 7f0e82d9d700 1 --2- 172.21.15.19:0/3493612462 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0e6c0b2bc0 0x7f0e6c0b50d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c01aac0 tx=0x7f0e7c01f000 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:49.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.208+0000 7f0e70ff9700 1 -- 172.21.15.19:0/407509549 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c01ae40 con 0x557afeb2b480 2024-05-09T02:22:49.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.208+0000 7f0e70ff9700 1 -- 172.21.15.19:0/407509549 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e7c01f070 con 0x557afeb2b480 2024-05-09T02:22:49.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.209+0000 7f0e974e0b80 1 -- 172.21.15.19:0/407509549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afeb2b480 msgr2=0x557afeb2b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:49.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.209+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/407509549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afeb2b480 0x557afeb2b890 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c008ea0 tx=0x7f0e7c01a6c0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.215+0000 7f0e974e0b80 1 -- 172.21.15.19:0/407509549 shutdown_connections 2024-05-09T02:22:49.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.215+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/407509549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afeb2b480 0x557afeb2b890 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.215+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/407509549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afea28d80 0x557afeb312c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.215+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/407509549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afe9ae520 0x557afeb31800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.215+0000 7f0e974e0b80 1 -- 172.21.15.19:0/407509549 >> 172.21.15.19:0/407509549 conn(0x557afe9b1ff0 msgr2=0x557afeb262a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:49.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.217+0000 7f0e974e0b80 1 -- 172.21.15.19:0/407509549 shutdown_connections 2024-05-09T02:22:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.221+0000 7f0e974e0b80 1 -- 172.21.15.19:0/407509549 wait complete. 2024-05-09T02:22:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.221+0000 7f0e974e0b80 1 -- start start 2024-05-09T02:22:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.221+0000 7f0e974e0b80 1 Processor -- start 2024-05-09T02:22:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.222+0000 7f0e974e0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afe9ae520 0x557afeb35010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.222+0000 7f0e974e0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afea28d80 0x557afeb35550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.222+0000 7f0e974e0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afeb2b480 0x557afeb35a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.222+0000 7f0e974e0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557afeb2c170 con 0x557afea28d80 2024-05-09T02:22:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.222+0000 7f0e974e0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557afe9e8ba0 con 0x557afe9ae520 2024-05-09T02:22:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.222+0000 7f0e974e0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557afeb2c2f0 con 0x557afeb2b480 2024-05-09T02:22:49.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.222+0000 7f0e974e0b80 10 client.0 _ll_register_callbacks cb 0x557afe8bf740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:49.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.222+0000 7f0e82d9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afeb2b480 0x557afeb35a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.222+0000 7f0e82d9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afeb2b480 0x557afeb35a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:51048/0 (socket says 172.21.15.19:51048) 2024-05-09T02:22:49.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.222+0000 7f0e82d9d700 1 -- 172.21.15.19:0/116341292 learned_addr learned my addr 172.21.15.19:0/116341292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:49.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.223+0000 7f0e82d9d700 1 -- 172.21.15.19:0/116341292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afe9ae520 msgr2=0x557afeb35010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:22:49.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.223+0000 7f0e82d9d700 1 --2- 172.21.15.19:0/116341292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afe9ae520 0x557afeb35010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.223+0000 7f0e82d9d700 1 -- 172.21.15.19:0/116341292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afea28d80 msgr2=0x557afeb35550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:22:49.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.223+0000 7f0e82d9d700 1 --2- 172.21.15.19:0/116341292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afea28d80 0x557afeb35550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.223+0000 7f0e82d9d700 1 -- 172.21.15.19:0/116341292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557afe8c9510 con 0x557afeb2b480 2024-05-09T02:22:49.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.223+0000 7f0e82d9d700 1 --2- 172.21.15.19:0/116341292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afeb2b480 0x557afeb35a90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c025040 tx=0x7f0e7c01aef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=853e563e6a28fc23 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:49.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.223+0000 7f0e69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:22:49.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.223+0000 7f0e69ffb700 1 -- 172.21.15.19:0/116341292 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c030020 con 0x557afeb2b480 2024-05-09T02:22:49.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.223+0000 7f0e974e0b80 1 -- 172.21.15.19:0/116341292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e7c01f070 con 0x557afeb2b480 2024-05-09T02:22:49.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.224+0000 7f0e69ffb700 1 -- 172.21.15.19:0/116341292 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f0e7c008250 con 0x557afeb2b480 2024-05-09T02:22:49.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.224+0000 7f0e69ffb700 1 -- 172.21.15.19:0/116341292 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f0e7c03b050 con 0x557afeb2b480 2024-05-09T02:22:49.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.224+0000 7f0e974e0b80 10 client.26524 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:49.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.224+0000 7f0e974e0b80 1 -- 172.21.15.19:0/116341292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557afeb539d0 con 0x557afeb2b480 2024-05-09T02:22:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.224+0000 7f0e69ffb700 1 -- 172.21.15.19:0/116341292 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e7c01f070 con 0x557afeb2b480 2024-05-09T02:22:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.225+0000 7f0e69ffb700 1 -- 172.21.15.19:0/116341292 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f0e7c02c640 con 0x557afeb2b480 2024-05-09T02:22:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.225+0000 7f0e974e0b80 10 client.26524 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.225+0000 7f0e974e0b80 10 client.26524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:22:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.225+0000 7f0e974e0b80 20 client.26524 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.225+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/116341292 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557afeb56420 0x557afeb58910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.225+0000 7f0e974e0b80 4 client.26524 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:22:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.225+0000 7f0e974e0b80 1 -- 172.21.15.19:0/116341292 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557afea5bb00 con 0x557afeb56420 2024-05-09T02:22:49.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.226+0000 7f0e8259c700 1 --2- 172.21.15.19:0/116341292 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557afeb56420 0x557afeb58910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.231+0000 7f0e8259c700 1 --2- 172.21.15.19:0/116341292 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557afeb56420 0x557afeb58910 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83257afc96c7148e server_cookie=4cde5af780d4477d in_seq=0 out_seq=0 2024-05-09T02:22:49.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.234+0000 7f0e69ffb700 10 client.26524 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:49.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.234+0000 7f0e69ffb700 1 -- 172.21.15.19:0/116341292 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557afea5bb00 con 0x557afeb56420 2024-05-09T02:22:49.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.234+0000 7f0e69ffb700 10 client.26524 handle_command_reply: tid=0 2024-05-09T02:22:49.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.234+0000 7f0e974e0b80 1 client.26524 shutdown 2024-05-09T02:22:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.235+0000 7f0e974e0b80 1 -- 172.21.15.19:0/116341292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afeb2b480 msgr2=0x557afeb35a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.235+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/116341292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afeb2b480 0x557afeb35a90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c025040 tx=0x7f0e7c01aef0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.236+0000 7f0e974e0b80 1 -- 172.21.15.19:0/116341292 shutdown_connections 2024-05-09T02:22:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.236+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/116341292 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557afeb56420 0x557afeb58910 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.236+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/116341292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afeb2b480 0x557afeb35a90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.236+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/116341292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afe9ae520 0x557afeb35010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.236+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/116341292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afea28d80 0x557afeb35550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.236+0000 7f0e974e0b80 1 -- 172.21.15.19:0/116341292 >> 172.21.15.19:0/116341292 conn(0x557afe9b1ff0 msgr2=0x557afeb20070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.236+0000 7f0e69ffb700 0 client.26524 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:49.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.237+0000 7f0e974e0b80 1 -- 172.21.15.19:0/116341292 shutdown_connections 2024-05-09T02:22:49.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.239+0000 7f0e974e0b80 1 -- 172.21.15.19:0/116341292 wait complete. 2024-05-09T02:22:49.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.239+0000 7f0e974e0b80 20 client.26524 trim_cache size 0 max 16384 2024-05-09T02:22:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.282+0000 7f0e974e0b80 1 Processor -- start 2024-05-09T02:22:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.283+0000 7f0e974e0b80 1 -- start start 2024-05-09T02:22:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.283+0000 7f0e974e0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afe9ae520 0x557afeb20aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.283+0000 7f0e974e0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afea28d80 0x557afeb51f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.284+0000 7f0e8259c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afe9ae520 0x557afeb20aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.284+0000 7f0e8259c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afe9ae520 0x557afeb20aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:51054/0 (socket says 172.21.15.19:51054) 2024-05-09T02:22:49.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.284+0000 7f0e8259c700 1 -- 172.21.15.19:0/1868725723 learned_addr learned my addr 172.21.15.19:0/1868725723 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.284+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/1868725723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afeb2b480 0x557afeb35580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.284+0000 7f0e81d9b700 1 --2- 172.21.15.19:0/1868725723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afea28d80 0x557afeb51f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.284+0000 7f0e974e0b80 1 -- 172.21.15.19:0/1868725723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6400d7b0 con 0x557afea28d80 2024-05-09T02:22:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.285+0000 7f0e82d9d700 1 --2- 172.21.15.19:0/1868725723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afeb2b480 0x557afeb35580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.285+0000 7f0e974e0b80 1 -- 172.21.15.19:0/1868725723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6400f2f0 con 0x557afeb2b480 2024-05-09T02:22:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.285+0000 7f0e974e0b80 1 -- 172.21.15.19:0/1868725723 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6400de80 con 0x557afe9ae520 2024-05-09T02:22:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.287+0000 7f0e82d9d700 1 -- 172.21.15.19:0/1868725723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afe9ae520 msgr2=0x557afeb20aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.287+0000 7f0e82d9d700 1 --2- 172.21.15.19:0/1868725723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afe9ae520 0x557afeb20aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.287+0000 7f0e82d9d700 1 -- 172.21.15.19:0/1868725723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afea28d80 msgr2=0x557afeb51f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.287+0000 7f0e82d9d700 1 --2- 172.21.15.19:0/1868725723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afea28d80 0x557afeb51f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.287+0000 7f0e82d9d700 1 -- 172.21.15.19:0/1868725723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557afeb37f70 con 0x557afeb2b480 2024-05-09T02:22:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.288+0000 7f0e81d9b700 1 --2- 172.21.15.19:0/1868725723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afea28d80 0x557afeb51f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:22:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.288+0000 7f0e82d9d700 1 --2- 172.21.15.19:0/1868725723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afeb2b480 0x557afeb35580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c02ca90 tx=0x7f0e7c01a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fadcefad9286e41c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.288+0000 7f0e6bfff700 1 -- 172.21.15.19:0/1868725723 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c044070 con 0x557afeb2b480 2024-05-09T02:22:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.288+0000 7f0e6bfff700 1 -- 172.21.15.19:0/1868725723 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e7c004a30 con 0x557afeb2b480 2024-05-09T02:22:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.288+0000 7f0e6bfff700 1 -- 172.21.15.19:0/1868725723 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c030070 con 0x557afeb2b480 2024-05-09T02:22:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.292+0000 7f0e974e0b80 1 -- 172.21.15.19:0/1868725723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afeb2b480 msgr2=0x557afeb35580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.292+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/1868725723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afeb2b480 0x557afeb35580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c02ca90 tx=0x7f0e7c01a6c0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.295+0000 7f0e974e0b80 1 -- 172.21.15.19:0/1868725723 shutdown_connections 2024-05-09T02:22:49.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.295+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/1868725723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afeb2b480 0x557afeb35580 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.295+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/1868725723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afe9ae520 0x557afeb20aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.295+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/1868725723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afea28d80 0x557afeb51f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.295+0000 7f0e974e0b80 1 -- 172.21.15.19:0/1868725723 >> 172.21.15.19:0/1868725723 conn(0x557afe9b1ff0 msgr2=0x557afeb2f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:49.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.296+0000 7f0e974e0b80 1 -- 172.21.15.19:0/1868725723 shutdown_connections 2024-05-09T02:22:49.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.296+0000 7f0e974e0b80 1 -- 172.21.15.19:0/1868725723 wait complete. 2024-05-09T02:22:49.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.297+0000 7f0e974e0b80 1 -- start start 2024-05-09T02:22:49.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.297+0000 7f0e974e0b80 1 Processor -- start 2024-05-09T02:22:49.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.298+0000 7f0e974e0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afe9ae520 0x557afeb4e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.298+0000 7f0e974e0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afea28d80 0x557afeb4e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.298+0000 7f0e8259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afe9ae520 0x557afeb4e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.298+0000 7f0e8259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afe9ae520 0x557afeb4e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56948/0 (socket says 172.21.15.19:56948) 2024-05-09T02:22:49.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.298+0000 7f0e81d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afea28d80 0x557afeb4e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.298+0000 7f0e8259c700 1 -- 172.21.15.19:0/951521894 learned_addr learned my addr 172.21.15.19:0/951521894 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:49.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.298+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/951521894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afeb2b480 0x557afeb4b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.299+0000 7f0e974e0b80 1 -- 172.21.15.19:0/951521894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6400ec20 con 0x557afeb2b480 2024-05-09T02:22:49.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.299+0000 7f0e974e0b80 1 -- 172.21.15.19:0/951521894 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6400e550 con 0x557afe9ae520 2024-05-09T02:22:49.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.299+0000 7f0e974e0b80 1 -- 172.21.15.19:0/951521894 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6400b770 con 0x557afea28d80 2024-05-09T02:22:49.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.299+0000 7f0e974e0b80 10 client.0 _ll_register_callbacks cb 0x557afeb58df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:49.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.299+0000 7f0e82d9d700 1 --2- 172.21.15.19:0/951521894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afeb2b480 0x557afeb4b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.299+0000 7f0e81d9b700 1 -- 172.21.15.19:0/951521894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afe9ae520 msgr2=0x557afeb4e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:49.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.299+0000 7f0e81d9b700 1 --2- 172.21.15.19:0/951521894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afe9ae520 0x557afeb4e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.299+0000 7f0e81d9b700 1 -- 172.21.15.19:0/951521894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afeb2b480 msgr2=0x557afeb4b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:49.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.299+0000 7f0e81d9b700 1 --2- 172.21.15.19:0/951521894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afeb2b480 0x557afeb4b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.299+0000 7f0e81d9b700 1 -- 172.21.15.19:0/951521894 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557afeb31b70 con 0x557afea28d80 2024-05-09T02:22:49.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.300+0000 7f0e8259c700 1 --2- 172.21.15.19:0/951521894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afe9ae520 0x557afeb4e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:22:49.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.300+0000 7f0e81d9b700 1 --2- 172.21.15.19:0/951521894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afea28d80 0x557afeb4e830 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0e780e7460 tx=0x7f0e780b9ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37e59eeaa8b2ed91 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:49.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.300+0000 7f0e974e0b80 1 -- 172.21.15.19:0/951521894 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e7c004a30 con 0x557afea28d80 2024-05-09T02:22:49.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.300+0000 7f0e69ffb700 10 client.26530 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:22:49.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.301+0000 7f0e69ffb700 1 -- 172.21.15.19:0/951521894 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e780b16f0 con 0x557afea28d80 2024-05-09T02:22:49.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.301+0000 7f0e69ffb700 1 -- 172.21.15.19:0/951521894 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f0e78011800 con 0x557afea28d80 2024-05-09T02:22:49.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.301+0000 7f0e69ffb700 1 -- 172.21.15.19:0/951521894 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f0e780b0ff0 con 0x557afea28d80 2024-05-09T02:22:49.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.301+0000 7f0e974e0b80 10 client.26530 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:49.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.301+0000 7f0e974e0b80 1 -- 172.21.15.19:0/951521894 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=58}) -- 0x557afeb22a50 con 0x557afea28d80 2024-05-09T02:22:49.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.302+0000 7f0e69ffb700 1 -- 172.21.15.19:0/951521894 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f0e7801abc0 con 0x557afea28d80 2024-05-09T02:22:49.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.302+0000 7f0e974e0b80 10 client.26530 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:49.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.302+0000 7f0e974e0b80 10 client.26530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:22:49.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.302+0000 7f0e974e0b80 20 client.26530 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:49.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.302+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/951521894 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557afebc1680 0x557afebc3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:49.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.302+0000 7f0e974e0b80 4 client.26530 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T02:22:49.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.302+0000 7f0e974e0b80 1 -- 172.21.15.19:0/951521894 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f0e7c004a30 con 0x557afebc1680 2024-05-09T02:22:49.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.303+0000 7f0e8259c700 1 --2- 172.21.15.19:0/951521894 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557afebc1680 0x557afebc3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:49.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.304+0000 7f0e69ffb700 10 client.26530 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:49.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.304+0000 7f0e8259c700 1 --2- 172.21.15.19:0/951521894 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557afebc1680 0x557afebc3ae0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=639487dcca76c9a0 server_cookie=e00a51650aa5ee24 in_seq=0 out_seq=0 2024-05-09T02:22:49.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.306+0000 7f0e69ffb700 1 -- 172.21.15.19:0/951521894 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f0e7c004a30 con 0x557afebc1680 2024-05-09T02:22:49.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.306+0000 7f0e69ffb700 10 client.26530 handle_command_reply: tid=0 2024-05-09T02:22:49.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.306+0000 7f0e974e0b80 1 client.26530 shutdown 2024-05-09T02:22:49.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.306+0000 7f0e974e0b80 1 -- 172.21.15.19:0/951521894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afea28d80 msgr2=0x557afeb4e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:49.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.306+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/951521894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afea28d80 0x557afeb4e830 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0e780e7460 tx=0x7f0e780b9ca0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.307+0000 7f0e974e0b80 1 -- 172.21.15.19:0/951521894 shutdown_connections 2024-05-09T02:22:49.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.307+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/951521894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557afeb2b480 0x557afeb4b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.307+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/951521894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557afe9ae520 0x557afeb4e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.307+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/951521894 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557afebc1680 0x557afebc3ae0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.307+0000 7f0e974e0b80 1 --2- 172.21.15.19:0/951521894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557afea28d80 0x557afeb4e830 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.307+0000 7f0e974e0b80 1 -- 172.21.15.19:0/951521894 >> 172.21.15.19:0/951521894 conn(0x557afe9b1ff0 msgr2=0x557afeb2f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.307+0000 7f0e69ffb700 0 client.26530 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:49.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.309+0000 7f0e974e0b80 1 -- 172.21.15.19:0/951521894 shutdown_connections 2024-05-09T02:22:49.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.309+0000 7f0e974e0b80 1 -- 172.21.15.19:0/951521894 wait complete. 2024-05-09T02:22:49.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.313+0000 7f0e974e0b80 20 client.26530 trim_cache size 0 max 16384 2024-05-09T02:22:49.317 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:22:49.317 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T02:22:49.317 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "11b6b90f-a1e3-419c-9079-9dc62a30918a", 2024-05-09T02:22:49.317 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T02:22:49.317 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:22:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.315+0000 7f0e88aca700 1 -- 172.21.15.19:0/3493612462 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0e6c0b2bc0 msgr2=0x7f0e6c0b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.315+0000 7f0e88aca700 1 --2- 172.21.15.19:0/3493612462 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0e6c0b2bc0 0x7f0e6c0b50d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c01aac0 tx=0x7f0e7c01f000 comp rx=0 tx=0).stop 2024-05-09T02:22:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.315+0000 7f0e88aca700 1 -- 172.21.15.19:0/3493612462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e84107e10 msgr2=0x7f0e8419edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:49.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.315+0000 7f0e88aca700 1 --2- 172.21.15.19:0/3493612462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e84107e10 0x7f0e8419edd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0e78016fb0 tx=0x7f0e780143a0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.316+0000 7f0e88aca700 1 -- 172.21.15.19:0/3493612462 shutdown_connections 2024-05-09T02:22:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.316+0000 7f0e88aca700 1 --2- 172.21.15.19:0/3493612462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e84108760 0x7f0e84198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.316+0000 7f0e88aca700 1 --2- 172.21.15.19:0/3493612462 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0e6c0b2bc0 0x7f0e6c0b50d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.316+0000 7f0e88aca700 1 --2- 172.21.15.19:0/3493612462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e841034e0 0x7f0e8419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.316+0000 7f0e88aca700 1 --2- 172.21.15.19:0/3493612462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e84107e10 0x7f0e8419edd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.316+0000 7f0e88aca700 1 -- 172.21.15.19:0/3493612462 >> 172.21.15.19:0/3493612462 conn(0x7f0e840fef30 msgr2=0x7f0e841096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.316+0000 7f0e88aca700 1 -- 172.21.15.19:0/3493612462 shutdown_connections 2024-05-09T02:22:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:49.316+0000 7f0e88aca700 1 -- 172.21.15.19:0/3493612462 wait complete. 2024-05-09T02:22:49.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:49 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:22:49.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:49 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:22:49.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:49 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:22:50.137 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:22:50.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub queued for path: / 2024-05-09T02:22:50.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:22:50.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:22:50.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:22:50.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:22:50.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:22:50.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:22:50.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:22:50.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:22:50.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: pgmap v359: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 18 MiB/s wr, 1.65k op/s 2024-05-09T02:22:50.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:22:50.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:22:50.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:50 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:22:50.551 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:22:50.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub queued for path: / 2024-05-09T02:22:50.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:22:50.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:22:50.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:22:50.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:22:50.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:22:50.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:22:50.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:22:50.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:22:50.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: pgmap v359: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 18 MiB/s wr, 1.65k op/s 2024-05-09T02:22:50.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:22:50.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:22:50.797 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:50 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:22:50.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub queued for path: / 2024-05-09T02:22:50.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:22:50.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:22:50.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:22:50.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:22:50.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:22:50.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:22:50.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:22:50.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:22:50.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: pgmap v359: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 18 MiB/s wr, 1.65k op/s 2024-05-09T02:22:50.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:22:50.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:22:50.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:22:50.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:50 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:22:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 670.0 s, 1154.6 tps, lat 86.219 ms stddev 93.614 2024-05-09T02:22:52.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:52 smithi019 ceph-mon[30151]: pgmap v360: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.2 MiB/s wr, 898 op/s 2024-05-09T02:22:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.774+0000 7f5cd9f94700 1 -- 172.21.15.19:0/275180814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd410a930 msgr2=0x7f5cd410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.774+0000 7f5cd9f94700 1 --2- 172.21.15.19:0/275180814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd410a930 0x7f5cd410ad40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40098f0 tx=0x7f5cc4009c00 comp rx=0 tx=0).stop 2024-05-09T02:22:52.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.777+0000 7f5cd9f94700 1 -- 172.21.15.19:0/275180814 shutdown_connections 2024-05-09T02:22:52.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.777+0000 7f5cd9f94700 1 --2- 172.21.15.19:0/275180814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd411d930 0x7f5cd411fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.777+0000 7f5cd9f94700 1 --2- 172.21.15.19:0/275180814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd410a930 0x7f5cd410ad40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.777+0000 7f5cd9f94700 1 --2- 172.21.15.19:0/275180814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cd410b280 0x7f5cd411d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.777+0000 7f5cd9f94700 1 -- 172.21.15.19:0/275180814 >> 172.21.15.19:0/275180814 conn(0x7f5cd406f4d0 msgr2=0x7f5cd406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:52.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.780+0000 7f5cd9f94700 1 -- 172.21.15.19:0/275180814 shutdown_connections 2024-05-09T02:22:52.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.780+0000 7f5cd9f94700 1 -- 172.21.15.19:0/275180814 wait complete. 2024-05-09T02:22:52.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.782+0000 7f5cd9f94700 1 Processor -- start 2024-05-09T02:22:52.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.782+0000 7f5cd9f94700 1 -- start start 2024-05-09T02:22:52.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.783+0000 7f5cd9f94700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd410a930 0x7f5cd41a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.783+0000 7f5cd9f94700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cd410b280 0x7f5cd41aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.783+0000 7f5cd9f94700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd411d930 0x7f5cd41ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.783+0000 7f5cd9f94700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4123380 con 0x7f5cd410a930 2024-05-09T02:22:52.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.784+0000 7f5cd9f94700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4123200 con 0x7f5cd411d930 2024-05-09T02:22:52.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.784+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd411d930 0x7f5cd41ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.784+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd411d930 0x7f5cd41ae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56960/0 (socket says 172.21.15.19:56960) 2024-05-09T02:22:52.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.784+0000 7f5cd3fff700 1 -- 172.21.15.19:0/3005995639 learned_addr learned my addr 172.21.15.19:0/3005995639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:52.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.784+0000 7f5ccaffd700 1 --2- 172.21.15.19:0/3005995639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cd410b280 0x7f5cd41aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.784+0000 7f5cd37fe700 1 --2- 172.21.15.19:0/3005995639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd410a930 0x7f5cd41a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.784+0000 7f5cd9f94700 1 -- 172.21.15.19:0/3005995639 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4123500 con 0x7f5cd410b280 2024-05-09T02:22:52.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.786+0000 7f5cd3fff700 1 -- 172.21.15.19:0/3005995639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cd410b280 msgr2=0x7f5cd41aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:52.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.786+0000 7f5cd3fff700 1 --2- 172.21.15.19:0/3005995639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cd410b280 0x7f5cd41aa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.786+0000 7f5cd3fff700 1 -- 172.21.15.19:0/3005995639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd410a930 msgr2=0x7f5cd41a9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:52.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.786+0000 7f5cd3fff700 1 --2- 172.21.15.19:0/3005995639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd410a930 0x7f5cd41a9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.786+0000 7f5cd3fff700 1 -- 172.21.15.19:0/3005995639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd41aec40 con 0x7f5cd411d930 2024-05-09T02:22:52.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.786+0000 7f5cd3fff700 1 --2- 172.21.15.19:0/3005995639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd411d930 0x7f5cd41ae5a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc007fa0 tx=0x7f5ccc00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:52.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.791+0000 7f5cd17fa700 1 -- 172.21.15.19:0/3005995639 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc004bb0 con 0x7f5cd411d930 2024-05-09T02:22:52.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.791+0000 7f5cd17fa700 1 -- 172.21.15.19:0/3005995639 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ccc004d50 con 0x7f5cd411d930 2024-05-09T02:22:52.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.791+0000 7f5cd17fa700 1 -- 172.21.15.19:0/3005995639 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc005950 con 0x7f5cd411d930 2024-05-09T02:22:52.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:52 smithi196 ceph-mon[34718]: pgmap v360: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.2 MiB/s wr, 898 op/s 2024-05-09T02:22:52.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.791+0000 7f5cd9f94700 1 -- 172.21.15.19:0/3005995639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd41aef30 con 0x7f5cd411d930 2024-05-09T02:22:52.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.792+0000 7f5cd9f94700 1 -- 172.21.15.19:0/3005995639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd41b6850 con 0x7f5cd411d930 2024-05-09T02:22:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.796+0000 7f5cd17fa700 1 -- 172.21.15.19:0/3005995639 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5ccc01a020 con 0x7f5cd411d930 2024-05-09T02:22:52.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.797+0000 7f5cd17fa700 1 --2- 172.21.15.19:0/3005995639 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5cc00b2bc0 0x7f5cc00b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.797+0000 7f5cd17fa700 1 -- 172.21.15.19:0/3005995639 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f5ccc0e2b90 con 0x7f5cd411d930 2024-05-09T02:22:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.797+0000 7f5cd37fe700 1 --2- 172.21.15.19:0/3005995639 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5cc00b2bc0 0x7f5cc00b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.798+0000 7f5ce89aab80 1 Processor -- start 2024-05-09T02:22:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.798+0000 7f5ce89aab80 1 -- start start 2024-05-09T02:22:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.798+0000 7f5cd37fe700 1 --2- 172.21.15.19:0/3005995639 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5cc00b2bc0 0x7f5cc00b50d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40098c0 tx=0x7f5cc4019040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.799+0000 7f5ce89aab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef7c1160 0x562cef85a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.799+0000 7f5ce89aab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef6d74a0 0x562cef85aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.799+0000 7f5cd37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef7c1160 0x562cef85a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.799+0000 7f5cd37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef7c1160 0x562cef85a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45216/0 (socket says 172.21.15.19:45216) 2024-05-09T02:22:52.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.800+0000 7f5cd37fe700 1 -- 172.21.15.19:0/2360344886 learned_addr learned my addr 172.21.15.19:0/2360344886 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:52.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.800+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/2360344886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef8546a0 0x562cef854ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.800+0000 7f5ccaffd700 1 --2- 172.21.15.19:0/2360344886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef6d74a0 0x562cef85aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.800+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2360344886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562cef776ff0 con 0x562cef7c1160 2024-05-09T02:22:52.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.800+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2360344886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562cef7157f0 con 0x562cef6d74a0 2024-05-09T02:22:52.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.800+0000 7f5cd3fff700 1 --2- 172.21.15.19:0/2360344886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef8546a0 0x562cef854ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.801+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2360344886 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562cef772fb0 con 0x562cef8546a0 2024-05-09T02:22:52.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.801+0000 7f5cd37fe700 1 -- 172.21.15.19:0/2360344886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef8546a0 msgr2=0x562cef854ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:52.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.801+0000 7f5cd37fe700 1 --2- 172.21.15.19:0/2360344886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef8546a0 0x562cef854ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.801+0000 7f5cd37fe700 1 -- 172.21.15.19:0/2360344886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef6d74a0 msgr2=0x562cef85aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:52.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.801+0000 7f5cd37fe700 1 --2- 172.21.15.19:0/2360344886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef6d74a0 0x562cef85aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.801+0000 7f5cd37fe700 1 -- 172.21.15.19:0/2360344886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cef70d370 con 0x562cef7c1160 2024-05-09T02:22:52.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.801+0000 7f5cd3fff700 1 --2- 172.21.15.19:0/2360344886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef8546a0 0x562cef854ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:22:52.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.802+0000 7f5cd37fe700 1 --2- 172.21.15.19:0/2360344886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef7c1160 0x562cef85a4f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40098f0 tx=0x7f5cc401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e45da82df513e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:52.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.802+0000 7f5cc9ffb700 1 -- 172.21.15.19:0/2360344886 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4033070 con 0x562cef7c1160 2024-05-09T02:22:52.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.802+0000 7f5cc9ffb700 1 -- 172.21.15.19:0/2360344886 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cc4004690 con 0x562cef7c1160 2024-05-09T02:22:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.803+0000 7f5cc9ffb700 1 -- 172.21.15.19:0/2360344886 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc400f6c0 con 0x562cef7c1160 2024-05-09T02:22:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.803+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2360344886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef7c1160 msgr2=0x562cef85a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.803+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/2360344886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef7c1160 0x562cef85a4f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40098f0 tx=0x7f5cc401f000 comp rx=0 tx=0).stop 2024-05-09T02:22:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.804+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2360344886 shutdown_connections 2024-05-09T02:22:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.804+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/2360344886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef8546a0 0x562cef854ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.804+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/2360344886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef7c1160 0x562cef85a4f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.804+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/2360344886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef6d74a0 0x562cef85aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.804+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2360344886 >> 172.21.15.19:0/2360344886 conn(0x562cef7376a0 msgr2=0x562cef84f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.804+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2360344886 shutdown_connections 2024-05-09T02:22:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.804+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2360344886 wait complete. 2024-05-09T02:22:52.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.804+0000 7f5ce89aab80 1 -- start start 2024-05-09T02:22:52.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.804+0000 7f5ce89aab80 1 Processor -- start 2024-05-09T02:22:52.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5ce89aab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef6d74a0 0x562cef85e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5ce89aab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 0x562cef85e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5ce89aab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef8546a0 0x562cef85ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5ce89aab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562cef5f2ba0 con 0x562cef8546a0 2024-05-09T02:22:52.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5ce89aab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562cef731150 con 0x562cef7c1160 2024-05-09T02:22:52.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5ce89aab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562cef8611d0 con 0x562cef6d74a0 2024-05-09T02:22:52.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5ce89aab80 10 client.0 _ll_register_callbacks cb 0x562cef5e86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:52.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef8546a0 0x562cef85ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef8546a0 0x562cef85ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45224/0 (socket says 172.21.15.19:45224) 2024-05-09T02:22:52.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5cd3fff700 1 -- 172.21.15.19:0/1692628836 learned_addr learned my addr 172.21.15.19:0/1692628836 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:52.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5ccaffd700 1 --2- 172.21.15.19:0/1692628836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 0x562cef85e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5cd37fe700 1 --2- 172.21.15.19:0/1692628836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef6d74a0 0x562cef85e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5cd3fff700 1 -- 172.21.15.19:0/1692628836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef6d74a0 msgr2=0x562cef85e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:52.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5cd3fff700 1 --2- 172.21.15.19:0/1692628836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef6d74a0 0x562cef85e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5cd3fff700 1 -- 172.21.15.19:0/1692628836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 msgr2=0x562cef85e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:52.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5cd3fff700 1 --2- 172.21.15.19:0/1692628836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 0x562cef85e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.805+0000 7f5cd3fff700 1 -- 172.21.15.19:0/1692628836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cef5f2490 con 0x562cef8546a0 2024-05-09T02:22:52.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.806+0000 7f5cd3fff700 1 --2- 172.21.15.19:0/1692628836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef8546a0 0x562cef85ec20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0a9d70 tx=0x7f5ccc0e07d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae443469468e31d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:52.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.806+0000 7f5cb2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:22:52.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.806+0000 7f5cb2ffd700 1 -- 172.21.15.19:0/1692628836 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0a98f0 con 0x562cef8546a0 2024-05-09T02:22:52.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.806+0000 7f5cb2ffd700 1 -- 172.21.15.19:0/1692628836 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f5ccc0b29c0 con 0x562cef8546a0 2024-05-09T02:22:52.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.806+0000 7f5ce89aab80 1 -- 172.21.15.19:0/1692628836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cc4004690 con 0x562cef8546a0 2024-05-09T02:22:52.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.806+0000 7f5cb2ffd700 1 -- 172.21.15.19:0/1692628836 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f5ccc0aa4e0 con 0x562cef8546a0 2024-05-09T02:22:52.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.806+0000 7f5ce89aab80 10 client.16773 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:52.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.806+0000 7f5ce89aab80 1 -- 172.21.15.19:0/1692628836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x562cef86bc00 con 0x562cef8546a0 2024-05-09T02:22:52.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.806+0000 7f5cb2ffd700 1 -- 172.21.15.19:0/1692628836 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cc4004690 con 0x562cef8546a0 2024-05-09T02:22:52.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.807+0000 7f5cb2ffd700 1 -- 172.21.15.19:0/1692628836 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f5ccc00daf0 con 0x562cef8546a0 2024-05-09T02:22:52.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.807+0000 7f5ce89aab80 10 client.16773 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:52.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.807+0000 7f5ce89aab80 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:22:52.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.807+0000 7f5ce89aab80 20 client.16773 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:52.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.807+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/1692628836 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x562cef86e240 0x562cef870660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.807+0000 7f5ce89aab80 4 client.16773 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:22:52.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.807+0000 7f5ce89aab80 1 -- 172.21.15.19:0/1692628836 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cef7849d0 con 0x562cef86e240 2024-05-09T02:22:52.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.807+0000 7f5cd37fe700 1 --2- 172.21.15.19:0/1692628836 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x562cef86e240 0x562cef870660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.808+0000 7f5cd37fe700 1 --2- 172.21.15.19:0/1692628836 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x562cef86e240 0x562cef870660 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0f4f7e561ad9014 server_cookie=63dd4b9e9baba5ff in_seq=0 out_seq=0 2024-05-09T02:22:52.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.809+0000 7f5cb2ffd700 10 client.16773 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:52.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.809+0000 7f5cb2ffd700 1 -- 172.21.15.19:0/1692628836 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562cef7849d0 con 0x562cef86e240 2024-05-09T02:22:52.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.809+0000 7f5cb2ffd700 10 client.16773 handle_command_reply: tid=0 2024-05-09T02:22:52.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.809+0000 7f5ce89aab80 1 client.16773 shutdown 2024-05-09T02:22:52.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.809+0000 7f5ce89aab80 1 -- 172.21.15.19:0/1692628836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef8546a0 msgr2=0x562cef85ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:52.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.809+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/1692628836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef8546a0 0x562cef85ec20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0a9d70 tx=0x7f5ccc0e07d0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.809+0000 7f5ce89aab80 1 -- 172.21.15.19:0/1692628836 shutdown_connections 2024-05-09T02:22:52.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.809+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/1692628836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef8546a0 0x562cef85ec20 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.809+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/1692628836 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x562cef86e240 0x562cef870660 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.809+0000 7f5cb2ffd700 0 client.16773 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:52.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.809+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/1692628836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef6d74a0 0x562cef85e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.809+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/1692628836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 0x562cef85e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.809+0000 7f5ce89aab80 1 -- 172.21.15.19:0/1692628836 >> 172.21.15.19:0/1692628836 conn(0x562cef7376a0 msgr2=0x562cef84f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:52.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.810+0000 7f5ce89aab80 1 -- 172.21.15.19:0/1692628836 shutdown_connections 2024-05-09T02:22:52.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.810+0000 7f5ce89aab80 1 -- 172.21.15.19:0/1692628836 wait complete. 2024-05-09T02:22:52.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.810+0000 7f5ce89aab80 20 client.16773 trim_cache size 0 max 16384 2024-05-09T02:22:52.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.838+0000 7f5ce89aab80 1 Processor -- start 2024-05-09T02:22:52.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.838+0000 7f5ce89aab80 1 -- start start 2024-05-09T02:22:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5ce89aab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef6d74a0 0x562cef85ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5ce89aab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 0x562cef853da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5ce89aab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef8546a0 0x562cef85e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5ce89aab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd41bde70 con 0x562cef8546a0 2024-05-09T02:22:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5ce89aab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd41bf9b0 con 0x562cef7c1160 2024-05-09T02:22:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5ce89aab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd41be540 con 0x562cef6d74a0 2024-05-09T02:22:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5ccaffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 0x562cef853da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5ccaffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 0x562cef853da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56978/0 (socket says 172.21.15.19:56978) 2024-05-09T02:22:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5ccaffd700 1 -- 172.21.15.19:0/798031491 learned_addr learned my addr 172.21.15.19:0/798031491 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5cd3fff700 1 --2- 172.21.15.19:0/798031491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef8546a0 0x562cef85e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5cd37fe700 1 --2- 172.21.15.19:0/798031491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef6d74a0 0x562cef85ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5ccaffd700 1 -- 172.21.15.19:0/798031491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef6d74a0 msgr2=0x562cef85ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5ccaffd700 1 --2- 172.21.15.19:0/798031491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef6d74a0 0x562cef85ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5ccaffd700 1 -- 172.21.15.19:0/798031491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef8546a0 msgr2=0x562cef85e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:52.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5ccaffd700 1 --2- 172.21.15.19:0/798031491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef8546a0 0x562cef85e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.839+0000 7f5ccaffd700 1 -- 172.21.15.19:0/798031491 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cef860f30 con 0x562cef7c1160 2024-05-09T02:22:52.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.840+0000 7f5ccaffd700 1 --2- 172.21.15.19:0/798031491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 0x562cef853da0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009c30 tx=0x7f5cbc009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c25365f4d94aedfe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:52.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.840+0000 7f5cc97fa700 1 -- 172.21.15.19:0/798031491 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc022070 con 0x562cef7c1160 2024-05-09T02:22:52.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.840+0000 7f5cc97fa700 1 -- 172.21.15.19:0/798031491 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cbc014db0 con 0x562cef7c1160 2024-05-09T02:22:52.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.840+0000 7f5cc97fa700 1 -- 172.21.15.19:0/798031491 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc01de10 con 0x562cef7c1160 2024-05-09T02:22:52.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.841+0000 7f5ce89aab80 1 -- 172.21.15.19:0/798031491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 msgr2=0x562cef853da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:52.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.841+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/798031491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 0x562cef853da0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009c30 tx=0x7f5cbc009c60 comp rx=0 tx=0).stop 2024-05-09T02:22:52.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.841+0000 7f5ce89aab80 1 -- 172.21.15.19:0/798031491 shutdown_connections 2024-05-09T02:22:52.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.841+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/798031491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef8546a0 0x562cef85e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.841+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/798031491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef6d74a0 0x562cef85ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.841+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/798031491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 0x562cef853da0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.841+0000 7f5ce89aab80 1 -- 172.21.15.19:0/798031491 >> 172.21.15.19:0/798031491 conn(0x562cef7376a0 msgr2=0x562cef853690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:52.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.841+0000 7f5ce89aab80 1 -- 172.21.15.19:0/798031491 shutdown_connections 2024-05-09T02:22:52.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.841+0000 7f5ce89aab80 1 -- 172.21.15.19:0/798031491 wait complete. 2024-05-09T02:22:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.841+0000 7f5ce89aab80 1 -- start start 2024-05-09T02:22:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.842+0000 7f5ce89aab80 1 Processor -- start 2024-05-09T02:22:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.842+0000 7f5ce89aab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef6d74a0 0x562cef8757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.842+0000 7f5ce89aab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 0x562cef875d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.842+0000 7f5ce89aab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef8546a0 0x562cef876270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.842+0000 7f5ce89aab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd41bf2e0 con 0x562cef6d74a0 2024-05-09T02:22:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.842+0000 7f5ce89aab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd41bec10 con 0x562cef7c1160 2024-05-09T02:22:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.842+0000 7f5ce89aab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd41bbe30 con 0x562cef8546a0 2024-05-09T02:22:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.842+0000 7f5ce89aab80 10 client.0 _ll_register_callbacks cb 0x562cef5e86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:22:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.842+0000 7f5cd37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef6d74a0 0x562cef8757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.842+0000 7f5cd37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef6d74a0 0x562cef8757f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45248/0 (socket says 172.21.15.19:45248) 2024-05-09T02:22:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.842+0000 7f5cd37fe700 1 -- 172.21.15.19:0/2840658173 learned_addr learned my addr 172.21.15.19:0/2840658173 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:22:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.842+0000 7f5cd3fff700 1 --2- 172.21.15.19:0/2840658173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef8546a0 0x562cef876270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.843+0000 7f5ccaffd700 1 --2- 172.21.15.19:0/2840658173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 0x562cef875d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.843+0000 7f5cd37fe700 1 -- 172.21.15.19:0/2840658173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef8546a0 msgr2=0x562cef876270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:52.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.843+0000 7f5cd37fe700 1 --2- 172.21.15.19:0/2840658173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef8546a0 0x562cef876270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.843+0000 7f5cd37fe700 1 -- 172.21.15.19:0/2840658173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 msgr2=0x562cef875d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:22:52.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.843+0000 7f5cd37fe700 1 --2- 172.21.15.19:0/2840658173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 0x562cef875d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.843+0000 7f5cd37fe700 1 -- 172.21.15.19:0/2840658173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cef8e8a60 con 0x562cef6d74a0 2024-05-09T02:22:52.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.843+0000 7f5cd37fe700 1 --2- 172.21.15.19:0/2840658173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef6d74a0 0x562cef8757f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4005750 tx=0x7f5cc4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b92ec6887742a6b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:22:52.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.843+0000 7f5cb2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:22:52.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.844+0000 7f5cb2ffd700 1 -- 172.21.15.19:0/2840658173 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4033040 con 0x562cef6d74a0 2024-05-09T02:22:52.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.844+0000 7f5cb2ffd700 1 -- 172.21.15.19:0/2840658173 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f5cc4004aa0 con 0x562cef6d74a0 2024-05-09T02:22:52.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.845+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2840658173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cbc014db0 con 0x562cef6d74a0 2024-05-09T02:22:52.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.845+0000 7f5cb2ffd700 1 -- 172.21.15.19:0/2840658173 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f5cc400b980 con 0x562cef6d74a0 2024-05-09T02:22:52.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.845+0000 7f5ce89aab80 10 client.16779 fetch_fsmap learned FSMap version 58 2024-05-09T02:22:52.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.845+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2840658173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x562cef8e9cd0 con 0x562cef6d74a0 2024-05-09T02:22:52.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.846+0000 7f5cb2ffd700 1 -- 172.21.15.19:0/2840658173 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cc4034040 con 0x562cef6d74a0 2024-05-09T02:22:52.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.846+0000 7f5cb2ffd700 1 -- 172.21.15.19:0/2840658173 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f5cc400fb60 con 0x562cef6d74a0 2024-05-09T02:22:52.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.846+0000 7f5ce89aab80 10 client.16779 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:22:52.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.846+0000 7f5ce89aab80 10 client.16779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:22:52.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.846+0000 7f5ce89aab80 20 client.16779 populate_metadata read hostname 'smithi019' 2024-05-09T02:22:52.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.846+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/2840658173 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x562cef86e240 0x562cef8ebeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:22:52.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.846+0000 7f5ce89aab80 4 client.16779 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:22:52.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.846+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2840658173 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5cbc014db0 con 0x562cef86e240 2024-05-09T02:22:52.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.846+0000 7f5ccaffd700 1 --2- 172.21.15.19:0/2840658173 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x562cef86e240 0x562cef8ebeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:22:52.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.847+0000 7f5cb2ffd700 10 client.16779 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:52.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.847+0000 7f5ccaffd700 1 --2- 172.21.15.19:0/2840658173 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x562cef86e240 0x562cef8ebeb0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a7033eeef03a30a server_cookie=9a75d8cdf4047f87 in_seq=0 out_seq=0 2024-05-09T02:22:52.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.847+0000 7f5cb2ffd700 1 -- 172.21.15.19:0/2840658173 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f5cbc014db0 con 0x562cef86e240 2024-05-09T02:22:52.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.847+0000 7f5cb2ffd700 10 client.16779 handle_command_reply: tid=0 2024-05-09T02:22:52.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.847+0000 7f5ce89aab80 1 client.16779 shutdown 2024-05-09T02:22:52.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.848+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2840658173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef6d74a0 msgr2=0x562cef8757f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:52.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.848+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/2840658173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef6d74a0 0x562cef8757f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4005750 tx=0x7f5cc4005dc0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.848+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2840658173 shutdown_connections 2024-05-09T02:22:52.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.848+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/2840658173 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x562cef86e240 0x562cef8ebeb0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.848+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/2840658173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562cef8546a0 0x562cef876270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.848+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/2840658173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cef6d74a0 0x562cef8757f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.848+0000 7f5cb2ffd700 0 client.16779 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:22:52.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.848+0000 7f5ce89aab80 1 --2- 172.21.15.19:0/2840658173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562cef7c1160 0x562cef875d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.848+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2840658173 >> 172.21.15.19:0/2840658173 conn(0x562cef7376a0 msgr2=0x562cef85f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:52.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.848+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2840658173 shutdown_connections 2024-05-09T02:22:52.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.848+0000 7f5ce89aab80 1 -- 172.21.15.19:0/2840658173 wait complete. 2024-05-09T02:22:52.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.848+0000 7f5ce89aab80 20 client.16779 trim_cache size 0 max 16384 2024-05-09T02:22:52.855 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:22:52.855 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T02:22:52.856 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T02:22:52.856 INFO:teuthology.orchestra.run.smithi019.stdout: "11b6b90f-a1e3-419c-9079-9dc62a30918a": { 2024-05-09T02:22:52.856 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T02:22:52.856 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "11b6b90f-a1e3-419c-9079-9dc62a30918a", 2024-05-09T02:22:52.856 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T02:22:52.856 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:22:52.856 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:22:52.856 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:22:52.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.849+0000 7f5cd9f94700 1 -- 172.21.15.19:0/3005995639 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5cc00b2bc0 msgr2=0x7f5cc00b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:52.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.849+0000 7f5cd9f94700 1 --2- 172.21.15.19:0/3005995639 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5cc00b2bc0 0x7f5cc00b50d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40098c0 tx=0x7f5cc4019040 comp rx=0 tx=0).stop 2024-05-09T02:22:52.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.849+0000 7f5cd9f94700 1 -- 172.21.15.19:0/3005995639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd411d930 msgr2=0x7f5cd41ae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:22:52.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.849+0000 7f5cd9f94700 1 --2- 172.21.15.19:0/3005995639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd411d930 0x7f5cd41ae5a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc007fa0 tx=0x7f5ccc00d3b0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.849+0000 7f5cd9f94700 1 -- 172.21.15.19:0/3005995639 shutdown_connections 2024-05-09T02:22:52.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.849+0000 7f5cd9f94700 1 --2- 172.21.15.19:0/3005995639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd411d930 0x7f5cd41ae5a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.849+0000 7f5cd9f94700 1 --2- 172.21.15.19:0/3005995639 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5cc00b2bc0 0x7f5cc00b50d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.849+0000 7f5cd9f94700 1 --2- 172.21.15.19:0/3005995639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd410a930 0x7f5cd41a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.849+0000 7f5cd9f94700 1 --2- 172.21.15.19:0/3005995639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cd410b280 0x7f5cd41aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:22:52.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.849+0000 7f5cd9f94700 1 -- 172.21.15.19:0/3005995639 >> 172.21.15.19:0/3005995639 conn(0x7f5cd406f4d0 msgr2=0x7f5cd411b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:22:52.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.849+0000 7f5cd9f94700 1 -- 172.21.15.19:0/3005995639 shutdown_connections 2024-05-09T02:22:52.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:22:52.849+0000 7f5cd9f94700 1 -- 172.21.15.19:0/3005995639 wait complete. 2024-05-09T02:22:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:52 smithi059 ceph-mon[34781]: pgmap v360: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.2 MiB/s wr, 898 op/s 2024-05-09T02:22:53.665 INFO:tasks.cephfs.filesystem:scrub status for tag:11b6b90f-a1e3-419c-9079-9dc62a30918a - {'path': '/', 'tag': '11b6b90f-a1e3-419c-9079-9dc62a30918a', 'options': 'recursive,force'} 2024-05-09T02:22:54.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:54 smithi196 ceph-mon[34718]: pgmap v361: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-05-09T02:22:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:54 smithi019 ceph-mon[30151]: pgmap v361: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-05-09T02:22:54.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:54 smithi059 ceph-mon[34781]: pgmap v361: 97 pgs: 97 active+clean; 4.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-05-09T02:22:56.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:56 smithi196 ceph-mon[34718]: pgmap v362: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.9 MiB/s wr, 767 op/s 2024-05-09T02:22:56.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:56 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:22:56.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:56 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:22:56.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:56 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:22:56.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:56 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:22:56.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:56 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:22:56.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:56 smithi019 ceph-mon[30151]: pgmap v362: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.9 MiB/s wr, 767 op/s 2024-05-09T02:22:56.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:56 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:22:56.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:56 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:22:56.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:56 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:22:56.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:56 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:22:56.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:56 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:22:56.979 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:56 smithi059 ceph-mon[34781]: pgmap v362: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.9 MiB/s wr, 767 op/s 2024-05-09T02:22:56.979 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:56 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:22:56.979 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:56 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:22:56.980 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:56 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:22:56.980 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:56 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:22:56.980 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:56 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:22:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 675.0 s, 1128.6 tps, lat 88.616 ms stddev 101.788 2024-05-09T02:22:58.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:22:58 smithi196 ceph-mon[34718]: pgmap v363: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 499 op/s 2024-05-09T02:22:58.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:22:58 smithi019 ceph-mon[30151]: pgmap v363: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 499 op/s 2024-05-09T02:22:58.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:22:58 smithi059 ceph-mon[34781]: pgmap v363: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 499 op/s 2024-05-09T02:23:00.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:00 smithi196 ceph-mon[34718]: pgmap v364: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.8 MiB/s wr, 667 op/s 2024-05-09T02:23:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:00 smithi019 ceph-mon[30151]: pgmap v364: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.8 MiB/s wr, 667 op/s 2024-05-09T02:23:00.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:00 smithi059 ceph-mon[34781]: pgmap v364: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.8 MiB/s wr, 667 op/s 2024-05-09T02:23:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 680.0 s, 853.6 tps, lat 116.499 ms stddev 114.540 2024-05-09T02:23:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:02 smithi196 ceph-mon[34718]: pgmap v365: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.5 MiB/s wr, 609 op/s 2024-05-09T02:23:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:02 smithi019 ceph-mon[30151]: pgmap v365: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.5 MiB/s wr, 609 op/s 2024-05-09T02:23:02.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:02 smithi059 ceph-mon[34781]: pgmap v365: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.5 MiB/s wr, 609 op/s 2024-05-09T02:23:04.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:04 smithi196 ceph-mon[34718]: pgmap v366: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.6 MiB/s wr, 521 op/s 2024-05-09T02:23:04.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:04 smithi019 ceph-mon[30151]: pgmap v366: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.6 MiB/s wr, 521 op/s 2024-05-09T02:23:04.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:04 smithi059 ceph-mon[34781]: pgmap v366: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.6 MiB/s wr, 521 op/s 2024-05-09T02:23:06.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:06 smithi196 ceph-mon[34718]: pgmap v367: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 1.47k op/s 2024-05-09T02:23:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:06 smithi019 ceph-mon[30151]: pgmap v367: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 1.47k op/s 2024-05-09T02:23:06.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:06 smithi059 ceph-mon[34781]: pgmap v367: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 1.47k op/s 2024-05-09T02:23:07.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 685.0 s, 1252.8 tps, lat 79.957 ms stddev 186.148 2024-05-09T02:23:07.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:07 smithi019 ceph-mon[30151]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T02:23:07.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:07 smithi059 ceph-mon[34781]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T02:23:08.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:07 smithi196 ceph-mon[34718]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T02:23:08.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:08 smithi019 ceph-mon[30151]: pgmap v369: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-05-09T02:23:08.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:08 smithi059 ceph-mon[34781]: pgmap v369: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-05-09T02:23:09.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:08 smithi196 ceph-mon[34718]: pgmap v369: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-05-09T02:23:09.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:09 smithi019 ceph-mon[30151]: pgmap v370: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.64k op/s 2024-05-09T02:23:09.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:09 smithi059 ceph-mon[34781]: pgmap v370: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.64k op/s 2024-05-09T02:23:10.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:09 smithi196 ceph-mon[34718]: pgmap v370: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.64k op/s 2024-05-09T02:23:12.656 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 690.0 s, 1305.0 tps, lat 76.935 ms stddev 88.466 2024-05-09T02:23:12.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:12 smithi019 ceph-mon[30151]: pgmap v371: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.60k op/s 2024-05-09T02:23:12.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:12 smithi059 ceph-mon[34781]: pgmap v371: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.60k op/s 2024-05-09T02:23:12.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:12 smithi196 ceph-mon[34718]: pgmap v371: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.60k op/s 2024-05-09T02:23:14.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:14 smithi059 ceph-mon[34781]: pgmap v372: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.60k op/s 2024-05-09T02:23:14.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:14 smithi196 ceph-mon[34718]: pgmap v372: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.60k op/s 2024-05-09T02:23:14.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:14 smithi019 ceph-mon[30151]: pgmap v372: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.60k op/s 2024-05-09T02:23:15.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:15 smithi019 ceph-mon[30151]: osdmap e94: 12 total, 12 up, 12 in 2024-05-09T02:23:15.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:15 smithi059 ceph-mon[34781]: osdmap e94: 12 total, 12 up, 12 in 2024-05-09T02:23:16.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:15 smithi196 ceph-mon[34718]: osdmap e94: 12 total, 12 up, 12 in 2024-05-09T02:23:16.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:16 smithi019 ceph-mon[30151]: pgmap v374: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.7 MiB/s wr, 883 op/s 2024-05-09T02:23:16.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:16 smithi059 ceph-mon[34781]: pgmap v374: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.7 MiB/s wr, 883 op/s 2024-05-09T02:23:17.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:16 smithi196 ceph-mon[34718]: pgmap v374: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.7 MiB/s wr, 883 op/s 2024-05-09T02:23:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 695.0 s, 1302.6 tps, lat 77.082 ms stddev 81.607 2024-05-09T02:23:17.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:17 smithi019 ceph-mon[30151]: pgmap v375: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 782 op/s 2024-05-09T02:23:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:17 smithi059 ceph-mon[34781]: pgmap v375: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 782 op/s 2024-05-09T02:23:18.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:17 smithi196 ceph-mon[34718]: pgmap v375: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 782 op/s 2024-05-09T02:23:20.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:20 smithi059 ceph-mon[34781]: pgmap v376: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.0 MiB/s wr, 546 op/s 2024-05-09T02:23:20.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:20 smithi196 ceph-mon[34718]: pgmap v376: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.0 MiB/s wr, 546 op/s 2024-05-09T02:23:20.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:20 smithi019 ceph-mon[30151]: pgmap v376: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.0 MiB/s wr, 546 op/s 2024-05-09T02:23:22.657 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 700.0 s, 1378.8 tps, lat 71.834 ms stddev 75.522 2024-05-09T02:23:22.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:22 smithi059 ceph-mon[34781]: pgmap v377: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.9 MiB/s wr, 616 op/s 2024-05-09T02:23:22.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:22 smithi196 ceph-mon[34718]: pgmap v377: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.9 MiB/s wr, 616 op/s 2024-05-09T02:23:22.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:22 smithi019 ceph-mon[30151]: pgmap v377: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.9 MiB/s wr, 616 op/s 2024-05-09T02:23:23.666 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:23:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:23:24.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:24 smithi019 ceph-mon[30151]: pgmap v378: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.9 MiB/s wr, 616 op/s 2024-05-09T02:23:24.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:24 smithi059 ceph-mon[34781]: pgmap v378: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.9 MiB/s wr, 616 op/s 2024-05-09T02:23:24.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:24 smithi196 ceph-mon[34718]: pgmap v378: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.9 MiB/s wr, 616 op/s 2024-05-09T02:23:26.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.023+0000 7fb5affff700 1 -- 172.21.15.19:0/1901937743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5b0103080 msgr2=0x7fb5b0103490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.023+0000 7fb5affff700 1 --2- 172.21.15.19:0/1901937743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5b0103080 0x7fb5b0103490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00098f0 tx=0x7fb5a0009c00 comp rx=0 tx=0).stop 2024-05-09T02:23:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.024+0000 7fb5affff700 1 -- 172.21.15.19:0/1901937743 shutdown_connections 2024-05-09T02:23:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.024+0000 7fb5affff700 1 --2- 172.21.15.19:0/1901937743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b01084d0 0x7fb5b01089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.024+0000 7fb5affff700 1 --2- 172.21.15.19:0/1901937743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5b0103080 0x7fb5b0103490 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.024+0000 7fb5affff700 1 --2- 172.21.15.19:0/1901937743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5b01039d0 0x7fb5b0107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.024+0000 7fb5affff700 1 -- 172.21.15.19:0/1901937743 >> 172.21.15.19:0/1901937743 conn(0x7fb5b0078ac0 msgr2=0x7fb5b0078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.025+0000 7fb5affff700 1 -- 172.21.15.19:0/1901937743 shutdown_connections 2024-05-09T02:23:26.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.026+0000 7fb5affff700 1 -- 172.21.15.19:0/1901937743 wait complete. 2024-05-09T02:23:26.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.027+0000 7fb5affff700 1 Processor -- start 2024-05-09T02:23:26.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.027+0000 7fb5affff700 1 -- start start 2024-05-09T02:23:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.027+0000 7fb5affff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5b0103080 0x7fb5b0117390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.027+0000 7fb5affff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b01039d0 0x7fb5b0111550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.027+0000 7fb5affff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5b01084d0 0x7fb5b0111a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.027+0000 7fb5affff700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b01099a0 con 0x7fb5b0103080 2024-05-09T02:23:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.027+0000 7fb5affff700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b0109820 con 0x7fb5b01039d0 2024-05-09T02:23:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.027+0000 7fb5affff700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b0109b20 con 0x7fb5b01084d0 2024-05-09T02:23:26.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.028+0000 7fb5af7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5b01084d0 0x7fb5b0111a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:26.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.028+0000 7fb5af7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5b01084d0 0x7fb5b0111a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:46532/0 (socket says 172.21.15.19:46532) 2024-05-09T02:23:26.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.028+0000 7fb5af7fe700 1 -- 172.21.15.19:0/1530801057 learned_addr learned my addr 172.21.15.19:0/1530801057 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:26.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.030+0000 7fb5af7fe700 1 -- 172.21.15.19:0/1530801057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b01039d0 msgr2=0x7fb5b0111550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:26.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.030+0000 7fb5af7fe700 1 --2- 172.21.15.19:0/1530801057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b01039d0 0x7fb5b0111550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.030+0000 7fb5af7fe700 1 -- 172.21.15.19:0/1530801057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5b0103080 msgr2=0x7fb5b0117390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:26.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.030+0000 7fb5af7fe700 1 --2- 172.21.15.19:0/1530801057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5b0103080 0x7fb5b0117390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.030+0000 7fb5af7fe700 1 -- 172.21.15.19:0/1530801057 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b0112050 con 0x7fb5b01084d0 2024-05-09T02:23:26.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.036+0000 7fb5af7fe700 1 --2- 172.21.15.19:0/1530801057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5b01084d0 0x7fb5b0111a90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800ada0 tx=0x7fb5a800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:26.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.039+0000 7fb5b4927700 1 -- 172.21.15.19:0/1530801057 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8017020 con 0x7fb5b01084d0 2024-05-09T02:23:26.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.039+0000 7fb5b4927700 1 -- 172.21.15.19:0/1530801057 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5a800ce30 con 0x7fb5b01084d0 2024-05-09T02:23:26.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.040+0000 7fb5affff700 1 -- 172.21.15.19:0/1530801057 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b0112340 con 0x7fb5b01084d0 2024-05-09T02:23:26.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.040+0000 7fb5affff700 1 -- 172.21.15.19:0/1530801057 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b01b64b0 con 0x7fb5b01084d0 2024-05-09T02:23:26.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.040+0000 7fb5b4927700 1 -- 172.21.15.19:0/1530801057 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a80054c0 con 0x7fb5b01084d0 2024-05-09T02:23:26.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.044+0000 7fb5b4927700 1 -- 172.21.15.19:0/1530801057 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb5a8004ad0 con 0x7fb5b01084d0 2024-05-09T02:23:26.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.049+0000 7fb5b4927700 1 --2- 172.21.15.19:0/1530801057 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb5980b2b80 0x7fb5980b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.049+0000 7fb5b4927700 1 -- 172.21.15.19:0/1530801057 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fb5a80e1c80 con 0x7fb5b01084d0 2024-05-09T02:23:26.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.051+0000 7fb5c433fb80 1 Processor -- start 2024-05-09T02:23:26.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.052+0000 7fb5aeffd700 1 --2- 172.21.15.19:0/1530801057 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb5980b2b80 0x7fb5980b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:26.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.052+0000 7fb5aeffd700 1 --2- 172.21.15.19:0/1530801057 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb5980b2b80 0x7fb5980b5090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0009810 tx=0x7fb5a0005e40 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:26.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.052+0000 7fb5c433fb80 1 -- start start 2024-05-09T02:23:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.053+0000 7fb5c433fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5d67160 0x5651c5e00740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.053+0000 7fb5c433fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e00c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.053+0000 7fb5c433fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 0x5651c5dfac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.053+0000 7fb5c433fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5651c5d1cff0 con 0x5651c5c7d4a0 2024-05-09T02:23:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.053+0000 7fb5c433fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5651c5cbb7f0 con 0x5651c5d67160 2024-05-09T02:23:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.053+0000 7fb5c433fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5651c5d18fb0 con 0x5651c5dfa800 2024-05-09T02:23:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.053+0000 7fb5aeffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5d67160 0x5651c5e00740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.053+0000 7fb5aeffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5d67160 0x5651c5e00740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:55798/0 (socket says 172.21.15.19:55798) 2024-05-09T02:23:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.053+0000 7fb5aeffd700 1 -- 172.21.15.19:0/3443452524 learned_addr learned my addr 172.21.15.19:0/3443452524 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.053+0000 7fb5ae7fc700 1 --2- 172.21.15.19:0/3443452524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e00c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.054+0000 7fb5af7fe700 1 --2- 172.21.15.19:0/3443452524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 0x5651c5dfac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.054+0000 7fb5aeffd700 1 -- 172.21.15.19:0/3443452524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 msgr2=0x5651c5dfac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.054+0000 7fb5aeffd700 1 --2- 172.21.15.19:0/3443452524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 0x5651c5dfac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.054+0000 7fb5aeffd700 1 -- 172.21.15.19:0/3443452524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 msgr2=0x5651c5e00c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.054+0000 7fb5aeffd700 1 --2- 172.21.15.19:0/3443452524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e00c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.054+0000 7fb5aeffd700 1 -- 172.21.15.19:0/3443452524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651c5cb3370 con 0x5651c5d67160 2024-05-09T02:23:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.054+0000 7fb5af7fe700 1 --2- 172.21.15.19:0/3443452524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 0x5651c5dfac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:23:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.054+0000 7fb5aeffd700 1 --2- 172.21.15.19:0/3443452524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5d67160 0x5651c5e00740 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0016000 tx=0x7fb5a001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ead7ddbe58eb92f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.055+0000 7fb595ffb700 1 -- 172.21.15.19:0/3443452524 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0005070 con 0x5651c5d67160 2024-05-09T02:23:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.055+0000 7fb595ffb700 1 -- 172.21.15.19:0/3443452524 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5a0027070 con 0x5651c5d67160 2024-05-09T02:23:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.056+0000 7fb595ffb700 1 -- 172.21.15.19:0/3443452524 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a000e600 con 0x5651c5d67160 2024-05-09T02:23:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.056+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3443452524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5d67160 msgr2=0x5651c5e00740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.056+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3443452524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5d67160 0x5651c5e00740 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0016000 tx=0x7fb5a001b000 comp rx=0 tx=0).stop 2024-05-09T02:23:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.056+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3443452524 shutdown_connections 2024-05-09T02:23:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.056+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3443452524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 0x5651c5dfac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.056+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3443452524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5d67160 0x5651c5e00740 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.056+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3443452524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e00c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.056+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3443452524 >> 172.21.15.19:0/3443452524 conn(0x5651c5cdd6a0 msgr2=0x5651c5df56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.058+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3443452524 shutdown_connections 2024-05-09T02:23:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.058+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3443452524 wait complete. 2024-05-09T02:23:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.059+0000 7fb5c433fb80 1 -- start start 2024-05-09T02:23:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.059+0000 7fb5c433fb80 1 Processor -- start 2024-05-09T02:23:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.060+0000 7fb5c433fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e04300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.060+0000 7fb5c433fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5d67160 0x5651c5e04840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.060+0000 7fb5aeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e04300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.060+0000 7fb5c433fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5dfa800 0x5651c5e04d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.060+0000 7fb5aeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e04300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40166/0 (socket says 172.21.15.19:40166) 2024-05-09T02:23:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.060+0000 7fb5aeffd700 1 -- 172.21.15.19:0/3325725992 learned_addr learned my addr 172.21.15.19:0/3325725992 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.061+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3325725992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5651c5b98ba0 con 0x5651c5c7d4a0 2024-05-09T02:23:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.061+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3325725992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5651c5cd7150 con 0x5651c5dfa800 2024-05-09T02:23:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.061+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3325725992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5651c5e07420 con 0x5651c5d67160 2024-05-09T02:23:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.061+0000 7fb5c433fb80 10 client.0 _ll_register_callbacks cb 0x5651c5b8e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:26.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.061+0000 7fb5aeffd700 1 -- 172.21.15.19:0/3325725992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5d67160 msgr2=0x5651c5e04840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:26.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.061+0000 7fb5aeffd700 1 --2- 172.21.15.19:0/3325725992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5d67160 0x5651c5e04840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.061+0000 7fb5aeffd700 1 -- 172.21.15.19:0/3325725992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5dfa800 msgr2=0x5651c5e04d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:23:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.061+0000 7fb5aeffd700 1 --2- 172.21.15.19:0/3325725992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5dfa800 0x5651c5e04d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.061+0000 7fb5aeffd700 1 -- 172.21.15.19:0/3325725992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651c5b98490 con 0x5651c5c7d4a0 2024-05-09T02:23:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.062+0000 7fb5aeffd700 1 --2- 172.21.15.19:0/3325725992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e04300 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0003ea0 tx=0x7fb5a0003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219fd572826df063 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.062+0000 7fb58effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:23:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.062+0000 7fb58effd700 1 -- 172.21.15.19:0/3325725992 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a00054d0 con 0x5651c5c7d4a0 2024-05-09T02:23:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.062+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3325725992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5a0027070 con 0x5651c5c7d4a0 2024-05-09T02:23:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.062+0000 7fb58effd700 1 -- 172.21.15.19:0/3325725992 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fb5a0005670 con 0x5651c5c7d4a0 2024-05-09T02:23:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.063+0000 7fb58effd700 1 -- 172.21.15.19:0/3325725992 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fb5a000e600 con 0x5651c5c7d4a0 2024-05-09T02:23:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.064+0000 7fb5c433fb80 10 client.16788 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.064+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3325725992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5651c5e11b50 con 0x5651c5c7d4a0 2024-05-09T02:23:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.064+0000 7fb58effd700 1 -- 172.21.15.19:0/3325725992 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5a0027070 con 0x5651c5c7d4a0 2024-05-09T02:23:26.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.066+0000 7fb58effd700 1 -- 172.21.15.19:0/3325725992 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fb5a000baa0 con 0x5651c5c7d4a0 2024-05-09T02:23:26.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.066+0000 7fb5c433fb80 10 client.16788 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:26.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.066+0000 7fb5c433fb80 10 client.16788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:23:26.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.066+0000 7fb5c433fb80 20 client.16788 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:26.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.066+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3325725992 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5651c5e14070 0x5651c5e16520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.066+0000 7fb5c433fb80 4 client.16788 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:23:26.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.066+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3325725992 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651c5d2a9d0 con 0x5651c5e14070 2024-05-09T02:23:26.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.067+0000 7fb5ae7fc700 1 --2- 172.21.15.19:0/3325725992 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5651c5e14070 0x5651c5e16520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:26.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.069+0000 7fb5ae7fc700 1 --2- 172.21.15.19:0/3325725992 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5651c5e14070 0x5651c5e16520 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9316f9f3468e3d4f server_cookie=1e0a1342d6c53ad5 in_seq=0 out_seq=0 2024-05-09T02:23:26.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.069+0000 7fb58effd700 10 client.16788 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.070+0000 7fb58effd700 1 -- 172.21.15.19:0/3325725992 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5651c5d2a9d0 con 0x5651c5e14070 2024-05-09T02:23:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.070+0000 7fb58effd700 10 client.16788 handle_command_reply: tid=0 2024-05-09T02:23:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.070+0000 7fb5c433fb80 1 client.16788 shutdown 2024-05-09T02:23:26.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.071+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3325725992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 msgr2=0x5651c5e04300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:26.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.071+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3325725992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e04300 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0003ea0 tx=0x7fb5a0003990 comp rx=0 tx=0).stop 2024-05-09T02:23:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.074+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3325725992 shutdown_connections 2024-05-09T02:23:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.074+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3325725992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5dfa800 0x5651c5e04d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.074+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3325725992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e04300 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.074+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3325725992 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5651c5e14070 0x5651c5e16520 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.074+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3325725992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5d67160 0x5651c5e04840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.074+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3325725992 >> 172.21.15.19:0/3325725992 conn(0x5651c5cdd6a0 msgr2=0x5651c5df53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.075+0000 7fb58effd700 0 client.16788 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.075+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3325725992 shutdown_connections 2024-05-09T02:23:26.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.076+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3325725992 wait complete. 2024-05-09T02:23:26.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.076+0000 7fb5c433fb80 20 client.16788 trim_cache size 0 max 16384 2024-05-09T02:23:26.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.111+0000 7fb5c433fb80 1 Processor -- start 2024-05-09T02:23:26.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.111+0000 7fb5c433fb80 1 -- start start 2024-05-09T02:23:26.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.111+0000 7fb5c433fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5c7d4a0 0x5651c5e102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5c433fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5d67160 0x5651c5e1bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5c433fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 0x5651c5e1c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5c433fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c00d530 con 0x5651c5d67160 2024-05-09T02:23:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5c433fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c00f070 con 0x5651c5c7d4a0 2024-05-09T02:23:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5c433fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c00dc00 con 0x5651c5dfa800 2024-05-09T02:23:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5ae7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5d67160 0x5651c5e1bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5aeffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5c7d4a0 0x5651c5e102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5ae7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5d67160 0x5651c5e1bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40176/0 (socket says 172.21.15.19:40176) 2024-05-09T02:23:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5ae7fc700 1 -- 172.21.15.19:0/3019142874 learned_addr learned my addr 172.21.15.19:0/3019142874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5af7fe700 1 --2- 172.21.15.19:0/3019142874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 0x5651c5e1c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5ae7fc700 1 -- 172.21.15.19:0/3019142874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 msgr2=0x5651c5e1c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5ae7fc700 1 --2- 172.21.15.19:0/3019142874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 0x5651c5e1c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5ae7fc700 1 -- 172.21.15.19:0/3019142874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5c7d4a0 msgr2=0x5651c5e102f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5ae7fc700 1 --2- 172.21.15.19:0/3019142874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5c7d4a0 0x5651c5e102f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5ae7fc700 1 -- 172.21.15.19:0/3019142874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651c5df04d0 con 0x5651c5d67160 2024-05-09T02:23:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5aeffd700 1 --2- 172.21.15.19:0/3019142874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5c7d4a0 0x5651c5e102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:23:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.112+0000 7fb5ae7fc700 1 --2- 172.21.15.19:0/3019142874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5d67160 0x5651c5e1bd20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0108a20 tx=0x7fb5a4012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bace861d2e56a9d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:26.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.113+0000 7fb5957fa700 1 -- 172.21.15.19:0/3019142874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4012960 con 0x5651c5d67160 2024-05-09T02:23:26.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.113+0000 7fb5957fa700 1 -- 172.21.15.19:0/3019142874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5a401a3a0 con 0x5651c5d67160 2024-05-09T02:23:26.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.113+0000 7fb5957fa700 1 -- 172.21.15.19:0/3019142874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4010690 con 0x5651c5d67160 2024-05-09T02:23:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.114+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3019142874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5d67160 msgr2=0x5651c5e1bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.114+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3019142874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5d67160 0x5651c5e1bd20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0108a20 tx=0x7fb5a4012450 comp rx=0 tx=0).stop 2024-05-09T02:23:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.114+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3019142874 shutdown_connections 2024-05-09T02:23:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.114+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3019142874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 0x5651c5e1c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.114+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3019142874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5c7d4a0 0x5651c5e102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.114+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3019142874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5d67160 0x5651c5e1bd20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.114+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3019142874 >> 172.21.15.19:0/3019142874 conn(0x5651c5cdd6a0 msgr2=0x5651c5e02bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.114+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3019142874 shutdown_connections 2024-05-09T02:23:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.114+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3019142874 wait complete. 2024-05-09T02:23:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.114+0000 7fb5c433fb80 1 -- start start 2024-05-09T02:23:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.114+0000 7fb5c433fb80 1 Processor -- start 2024-05-09T02:23:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5c433fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e0c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5c433fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5d67160 0x5651c5e0cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5c433fb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 0x5651c5e1ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5c433fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c00e9a0 con 0x5651c5c7d4a0 2024-05-09T02:23:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5c433fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c00e2d0 con 0x5651c5d67160 2024-05-09T02:23:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5aeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e0c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5c433fb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c00b580 con 0x5651c5dfa800 2024-05-09T02:23:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5aeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e0c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40192/0 (socket says 172.21.15.19:40192) 2024-05-09T02:23:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5aeffd700 1 -- 172.21.15.19:0/3757989147 learned_addr learned my addr 172.21.15.19:0/3757989147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:26.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5c433fb80 10 client.0 _ll_register_callbacks cb 0x5651c5b8e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5af7fe700 1 --2- 172.21.15.19:0/3757989147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 0x5651c5e1ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5ae7fc700 1 --2- 172.21.15.19:0/3757989147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5d67160 0x5651c5e0cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5aeffd700 1 -- 172.21.15.19:0/3757989147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 msgr2=0x5651c5e1ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5aeffd700 1 --2- 172.21.15.19:0/3757989147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 0x5651c5e1ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5aeffd700 1 -- 172.21.15.19:0/3757989147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5d67160 msgr2=0x5651c5e0cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5aeffd700 1 --2- 172.21.15.19:0/3757989147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5d67160 0x5651c5e0cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5aeffd700 1 -- 172.21.15.19:0/3757989147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651c5e1b250 con 0x5651c5c7d4a0 2024-05-09T02:23:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5ae7fc700 1 --2- 172.21.15.19:0/3757989147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5d67160 0x5651c5e0cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:23:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb5aeffd700 1 --2- 172.21.15.19:0/3757989147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e0c670 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000c010 tx=0x7fb5a002c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657ed6fabeae0443 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.115+0000 7fb58effd700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:23:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.116+0000 7fb58effd700 1 -- 172.21.15.19:0/3757989147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a00056e0 con 0x5651c5c7d4a0 2024-05-09T02:23:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.116+0000 7fb58effd700 1 -- 172.21.15.19:0/3757989147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fb5a002c960 con 0x5651c5c7d4a0 2024-05-09T02:23:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.116+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3757989147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5a401a3a0 con 0x5651c5c7d4a0 2024-05-09T02:23:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.116+0000 7fb58effd700 1 -- 172.21.15.19:0/3757989147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fb5a004b050 con 0x5651c5c7d4a0 2024-05-09T02:23:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.116+0000 7fb5c433fb80 10 client.16794 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.116+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3757989147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5651c5e7e020 con 0x5651c5c7d4a0 2024-05-09T02:23:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.116+0000 7fb58effd700 1 -- 172.21.15.19:0/3757989147 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5a401a3a0 con 0x5651c5c7d4a0 2024-05-09T02:23:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.116+0000 7fb58effd700 1 -- 172.21.15.19:0/3757989147 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fb5a000b560 con 0x5651c5c7d4a0 2024-05-09T02:23:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.116+0000 7fb5c433fb80 10 client.16794 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.116+0000 7fb5c433fb80 10 client.16794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:23:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.116+0000 7fb5c433fb80 20 client.16794 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.116+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3757989147 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5651c5e7e690 0x5651c5e80ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.116+0000 7fb5c433fb80 4 client.16794 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:23:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.116+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3757989147 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651c5d24a70 con 0x5651c5e7e690 2024-05-09T02:23:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.117+0000 7fb5ae7fc700 1 --2- 172.21.15.19:0/3757989147 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5651c5e7e690 0x5651c5e80ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.117+0000 7fb58effd700 10 client.16794 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.118+0000 7fb5ae7fc700 1 --2- 172.21.15.19:0/3757989147 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5651c5e7e690 0x5651c5e80ab0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74b1d5a23bbba86b server_cookie=a76ed79d1e626351 in_seq=0 out_seq=0 2024-05-09T02:23:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.118+0000 7fb58effd700 1 -- 172.21.15.19:0/3757989147 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5651c5d24a70 con 0x5651c5e7e690 2024-05-09T02:23:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.118+0000 7fb58effd700 10 client.16794 handle_command_reply: tid=0 2024-05-09T02:23:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.119+0000 7fb5c433fb80 1 client.16794 shutdown 2024-05-09T02:23:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.119+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3757989147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 msgr2=0x5651c5e0c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.119+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3757989147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e0c670 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000c010 tx=0x7fb5a002c7a0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.120+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3757989147 shutdown_connections 2024-05-09T02:23:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.120+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3757989147 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5651c5e7e690 0x5651c5e80ab0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.120+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3757989147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c5dfa800 0x5651c5e1ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.120+0000 7fb58effd700 0 client.16794 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.121+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3757989147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651c5c7d4a0 0x5651c5e0c670 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.121+0000 7fb5c433fb80 1 --2- 172.21.15.19:0/3757989147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651c5d67160 0x5651c5e0cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.121+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3757989147 >> 172.21.15.19:0/3757989147 conn(0x5651c5cdd6a0 msgr2=0x5651c5df97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:26.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.121+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3757989147 shutdown_connections 2024-05-09T02:23:26.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.121+0000 7fb5c433fb80 1 -- 172.21.15.19:0/3757989147 wait complete. 2024-05-09T02:23:26.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.121+0000 7fb5c433fb80 20 client.16794 trim_cache size 0 max 16384 2024-05-09T02:23:26.127 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:23:26.127 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T02:23:26.127 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T02:23:26.127 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:23:26.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.122+0000 7fb5affff700 1 -- 172.21.15.19:0/1530801057 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb5980b2b80 msgr2=0x7fb5980b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:26.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.122+0000 7fb5affff700 1 --2- 172.21.15.19:0/1530801057 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb5980b2b80 0x7fb5980b5090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0009810 tx=0x7fb5a0005e40 comp rx=0 tx=0).stop 2024-05-09T02:23:26.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.122+0000 7fb5affff700 1 -- 172.21.15.19:0/1530801057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5b01084d0 msgr2=0x7fb5b0111a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:26.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.122+0000 7fb5affff700 1 --2- 172.21.15.19:0/1530801057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5b01084d0 0x7fb5b0111a90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800ada0 tx=0x7fb5a800c330 comp rx=0 tx=0).stop 2024-05-09T02:23:26.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.122+0000 7fb5affff700 1 -- 172.21.15.19:0/1530801057 shutdown_connections 2024-05-09T02:23:26.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.122+0000 7fb5affff700 1 --2- 172.21.15.19:0/1530801057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5b01084d0 0x7fb5b0111a90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.122+0000 7fb5affff700 1 --2- 172.21.15.19:0/1530801057 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fb5980b2b80 0x7fb5980b5090 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.122+0000 7fb5affff700 1 --2- 172.21.15.19:0/1530801057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5b0103080 0x7fb5b0117390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.122+0000 7fb5affff700 1 --2- 172.21.15.19:0/1530801057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b01039d0 0x7fb5b0111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:26.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.122+0000 7fb5affff700 1 -- 172.21.15.19:0/1530801057 >> 172.21.15.19:0/1530801057 conn(0x7fb5b0078ac0 msgr2=0x7fb5b0105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:26.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.122+0000 7fb5affff700 1 -- 172.21.15.19:0/1530801057 shutdown_connections 2024-05-09T02:23:26.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:26.122+0000 7fb5affff700 1 -- 172.21.15.19:0/1530801057 wait complete. 2024-05-09T02:23:26.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:26 smithi059 ceph-mon[34781]: pgmap v379: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.1 MiB/s wr, 670 op/s 2024-05-09T02:23:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:26 smithi019 ceph-mon[30151]: pgmap v379: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.1 MiB/s wr, 670 op/s 2024-05-09T02:23:26.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:26 smithi196 ceph-mon[34718]: pgmap v379: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.1 MiB/s wr, 670 op/s 2024-05-09T02:23:26.830 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T02:23:26.831 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:23:27.316 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:23:27.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 705.0 s, 1274.6 tps, lat 79.532 ms stddev 90.971 2024-05-09T02:23:28.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:28 smithi019 ceph-mon[30151]: pgmap v380: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 499 op/s 2024-05-09T02:23:28.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:28 smithi059 ceph-mon[34781]: pgmap v380: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 499 op/s 2024-05-09T02:23:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:28 smithi196 ceph-mon[34718]: pgmap v380: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 499 op/s 2024-05-09T02:23:29.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.429+0000 7f7371265700 1 -- 172.21.15.19:0/1641967994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f736c10f940 msgr2=0x7f736c10fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:29.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.429+0000 7f7371265700 1 --2- 172.21.15.19:0/1641967994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f736c10f940 0x7f736c10fd50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f736000afc0 tx=0x7f736000c650 comp rx=0 tx=0).stop 2024-05-09T02:23:29.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.429+0000 7f7371265700 1 -- 172.21.15.19:0/1641967994 shutdown_connections 2024-05-09T02:23:29.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.429+0000 7f7371265700 1 --2- 172.21.15.19:0/1641967994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c110290 0x7f736c117bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:29.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.429+0000 7f7371265700 1 --2- 172.21.15.19:0/1641967994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f736c10b010 0x7f736c10b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:29.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.429+0000 7f7371265700 1 --2- 172.21.15.19:0/1641967994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f736c10f940 0x7f736c10fd50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:29.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.429+0000 7f7371265700 1 -- 172.21.15.19:0/1641967994 >> 172.21.15.19:0/1641967994 conn(0x7f736c06f4d0 msgr2=0x7f736c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:29.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.431+0000 7f7371265700 1 -- 172.21.15.19:0/1641967994 shutdown_connections 2024-05-09T02:23:29.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.433+0000 7f7371265700 1 -- 172.21.15.19:0/1641967994 wait complete. 2024-05-09T02:23:29.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.434+0000 7f7371265700 1 Processor -- start 2024-05-09T02:23:29.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.435+0000 7f7371265700 1 -- start start 2024-05-09T02:23:29.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.435+0000 7f7371265700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f736c10b010 0x7f736c1a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:29.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.436+0000 7f7371265700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c10f940 0x7f736c1a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:29.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.436+0000 7f7371265700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f736c110290 0x7f736c1a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:29.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.436+0000 7f7371265700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f736c11aaf0 con 0x7f736c110290 2024-05-09T02:23:29.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.436+0000 7f7371265700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f736c11a970 con 0x7f736c10f940 2024-05-09T02:23:29.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.436+0000 7f736affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f736c10b010 0x7f736c1a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:29.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.436+0000 7f736affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f736c10b010 0x7f736c1a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:48902/0 (socket says 172.21.15.19:48902) 2024-05-09T02:23:29.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.436+0000 7f736affd700 1 -- 172.21.15.19:0/777981305 learned_addr learned my addr 172.21.15.19:0/777981305 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:29.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.436+0000 7f7371265700 1 -- 172.21.15.19:0/777981305 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f736c11ac70 con 0x7f736c10b010 2024-05-09T02:23:29.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.439+0000 7f736a7fc700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c10f940 0x7f736c1a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:29.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.439+0000 7f736b7fe700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f736c110290 0x7f736c1a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:29.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.440+0000 7f736a7fc700 1 -- 172.21.15.19:0/777981305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f736c10b010 msgr2=0x7f736c1a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:29.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.440+0000 7f736a7fc700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f736c10b010 0x7f736c1a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:29.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.440+0000 7f736a7fc700 1 -- 172.21.15.19:0/777981305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f736c110290 msgr2=0x7f736c1a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:29.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.440+0000 7f736a7fc700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f736c110290 0x7f736c1a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:29.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.440+0000 7f736a7fc700 1 -- 172.21.15.19:0/777981305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736c1a6350 con 0x7f736c10f940 2024-05-09T02:23:29.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.440+0000 7f736b7fe700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f736c110290 0x7f736c1a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:23:29.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.440+0000 7f736affd700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f736c10b010 0x7f736c1a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:23:29.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.442+0000 7f736a7fc700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c10f940 0x7f736c1a1a20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f736000ca90 tx=0x7f736000a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:29.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.443+0000 7f7353fff700 1 -- 172.21.15.19:0/777981305 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360003bd0 con 0x7f736c10f940 2024-05-09T02:23:29.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.443+0000 7f7353fff700 1 -- 172.21.15.19:0/777981305 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7360004220 con 0x7f736c10f940 2024-05-09T02:23:29.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.443+0000 7f7353fff700 1 -- 172.21.15.19:0/777981305 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360019420 con 0x7f736c10f940 2024-05-09T02:23:29.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.443+0000 7f7371265700 1 -- 172.21.15.19:0/777981305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f736c1a65e0 con 0x7f736c10f940 2024-05-09T02:23:29.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.443+0000 7f7371265700 1 -- 172.21.15.19:0/777981305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f736c1a6a10 con 0x7f736c10f940 2024-05-09T02:23:29.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.445+0000 7f7371265700 1 -- 172.21.15.19:0/777981305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f736c0512c0 con 0x7f736c10f940 2024-05-09T02:23:29.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.447+0000 7f7353fff700 1 -- 172.21.15.19:0/777981305 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7360011560 con 0x7f736c10f940 2024-05-09T02:23:29.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.449+0000 7f7353fff700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f73540b2c00 0x7f73540b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:29.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.449+0000 7f7353fff700 1 -- 172.21.15.19:0/777981305 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f73600e7320 con 0x7f736c10f940 2024-05-09T02:23:29.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.450+0000 7f736affd700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f73540b2c00 0x7f73540b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:29.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.452+0000 7f736affd700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f73540b2c00 0x7f73540b5110 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f735c005950 tx=0x7f735c0092a0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:29.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.454+0000 7f7353fff700 1 -- 172.21.15.19:0/777981305 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f73600aed80 con 0x7f736c10f940 2024-05-09T02:23:29.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.878+0000 7f7371265700 1 -- 172.21.15.19:0/777981305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f736c02e5f0 con 0x7f736c10f940 2024-05-09T02:23:29.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.882+0000 7f7353fff700 1 -- 172.21.15.19:0/777981305 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11917 (secure 0 0 0) 0x7f73600ae680 con 0x7f736c10f940 2024-05-09T02:23:29.885 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:23:29.885 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:22:01.074232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24457,24455,24473,24463,24484]},"id":1}]} 2024-05-09T02:23:29.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.888+0000 7f7371265700 1 -- 172.21.15.19:0/777981305 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f73540b2c00 msgr2=0x7f73540b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:29.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.888+0000 7f7371265700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f73540b2c00 0x7f73540b5110 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f735c005950 tx=0x7f735c0092a0 comp rx=0 tx=0).stop 2024-05-09T02:23:29.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.888+0000 7f7371265700 1 -- 172.21.15.19:0/777981305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c10f940 msgr2=0x7f736c1a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:29.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.888+0000 7f7371265700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c10f940 0x7f736c1a1a20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f736000ca90 tx=0x7f736000a610 comp rx=0 tx=0).stop 2024-05-09T02:23:29.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.888+0000 7f7371265700 1 -- 172.21.15.19:0/777981305 shutdown_connections 2024-05-09T02:23:29.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.888+0000 7f7371265700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f73540b2c00 0x7f73540b5110 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:29.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.888+0000 7f7371265700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f736c110290 0x7f736c1a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:29.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.888+0000 7f7371265700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f736c10b010 0x7f736c1a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:29.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.888+0000 7f7371265700 1 --2- 172.21.15.19:0/777981305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c10f940 0x7f736c1a1a20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:29.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.888+0000 7f7371265700 1 -- 172.21.15.19:0/777981305 >> 172.21.15.19:0/777981305 conn(0x7f736c06f4d0 msgr2=0x7f736c10da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:29.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.889+0000 7f7371265700 1 -- 172.21.15.19:0/777981305 shutdown_connections 2024-05-09T02:23:29.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:29.889+0000 7f7371265700 1 -- 172.21.15.19:0/777981305 wait complete. 2024-05-09T02:23:29.894 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-05-09T02:23:30.577 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T02:23:30.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:30 smithi196 ceph-mon[34718]: pgmap v381: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.0 MiB/s wr, 675 op/s 2024-05-09T02:23:30.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:30 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/777981305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:23:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:30 smithi019 ceph-mon[30151]: pgmap v381: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.0 MiB/s wr, 675 op/s 2024-05-09T02:23:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:30 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/777981305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:23:30.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:30 smithi059 ceph-mon[34781]: pgmap v381: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.0 MiB/s wr, 675 op/s 2024-05-09T02:23:30.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:30 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/777981305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:23:30.996 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:23:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 710.0 s, 1663.0 tps, lat 59.497 ms stddev 64.631 2024-05-09T02:23:32.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:32 smithi019 ceph-mon[30151]: pgmap v382: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.0 MiB/s wr, 666 op/s 2024-05-09T02:23:32.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:32 smithi059 ceph-mon[34781]: pgmap v382: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.0 MiB/s wr, 666 op/s 2024-05-09T02:23:33.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:32 smithi196 ceph-mon[34718]: pgmap v382: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.0 MiB/s wr, 666 op/s 2024-05-09T02:23:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.612+0000 7ff3b44db700 1 -- 172.21.15.19:0/2139054518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac1034c0 msgr2=0x7ff3ac1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.612+0000 7ff3b44db700 1 --2- 172.21.15.19:0/2139054518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac1034c0 0x7ff3ac1038d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80099c0 tx=0x7ff3a8009cd0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.612+0000 7ff3b44db700 1 -- 172.21.15.19:0/2139054518 shutdown_connections 2024-05-09T02:23:33.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.612+0000 7ff3b44db700 1 --2- 172.21.15.19:0/2139054518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ac108740 0x7ff3ac1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.612+0000 7ff3b44db700 1 --2- 172.21.15.19:0/2139054518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac1034c0 0x7ff3ac1038d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.612+0000 7ff3b44db700 1 --2- 172.21.15.19:0/2139054518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3ac107df0 0x7ff3ac108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.612+0000 7ff3b44db700 1 -- 172.21.15.19:0/2139054518 >> 172.21.15.19:0/2139054518 conn(0x7ff3ac0fedc0 msgr2=0x7ff3ac1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:33.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.615+0000 7ff3b44db700 1 -- 172.21.15.19:0/2139054518 shutdown_connections 2024-05-09T02:23:33.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.615+0000 7ff3b44db700 1 -- 172.21.15.19:0/2139054518 wait complete. 2024-05-09T02:23:33.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.616+0000 7ff3b44db700 1 Processor -- start 2024-05-09T02:23:33.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.616+0000 7ff3b44db700 1 -- start start 2024-05-09T02:23:33.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.616+0000 7ff3b44db700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ac1034c0 0x7ff3ac19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.616+0000 7ff3b44db700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac107df0 0x7ff3ac19c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.616+0000 7ff3b44db700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3ac108740 0x7ff3ac1a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.616+0000 7ff3b44db700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac113370 con 0x7ff3ac107df0 2024-05-09T02:23:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.616+0000 7ff3b44db700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1131f0 con 0x7ff3ac1034c0 2024-05-09T02:23:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.616+0000 7ff3b44db700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1134f0 con 0x7ff3ac108740 2024-05-09T02:23:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.616+0000 7ff3b1a76700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac107df0 0x7ff3ac19c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.616+0000 7ff3b1a76700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac107df0 0x7ff3ac19c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39684/0 (socket says 172.21.15.19:39684) 2024-05-09T02:23:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.616+0000 7ff3b1a76700 1 -- 172.21.15.19:0/2582452123 learned_addr learned my addr 172.21.15.19:0/2582452123 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.616+0000 7ff3b2a78700 1 --2- 172.21.15.19:0/2582452123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3ac108740 0x7ff3ac1a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.616+0000 7ff3b2277700 1 --2- 172.21.15.19:0/2582452123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ac1034c0 0x7ff3ac19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.617+0000 7ff3b2a78700 1 -- 172.21.15.19:0/2582452123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ac1034c0 msgr2=0x7ff3ac19bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.617+0000 7ff3b2a78700 1 --2- 172.21.15.19:0/2582452123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ac1034c0 0x7ff3ac19bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.617+0000 7ff3b2a78700 1 -- 172.21.15.19:0/2582452123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac107df0 msgr2=0x7ff3ac19c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.617+0000 7ff3b2a78700 1 --2- 172.21.15.19:0/2582452123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac107df0 0x7ff3ac19c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.617+0000 7ff3b2a78700 1 -- 172.21.15.19:0/2582452123 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ac19c9c0 con 0x7ff3ac108740 2024-05-09T02:23:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.617+0000 7ff3b2277700 1 --2- 172.21.15.19:0/2582452123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ac1034c0 0x7ff3ac19bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:23:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.617+0000 7ff3b2a78700 1 --2- 172.21.15.19:0/2582452123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3ac108740 0x7ff3ac1a6c50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400efc0 tx=0x7ff3a400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:33.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.617+0000 7ff3a37fe700 1 -- 172.21.15.19:0/2582452123 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40092e0 con 0x7ff3ac108740 2024-05-09T02:23:33.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.617+0000 7ff3b44db700 1 -- 172.21.15.19:0/2582452123 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ac1a7250 con 0x7ff3ac108740 2024-05-09T02:23:33.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.617+0000 7ff3a37fe700 1 -- 172.21.15.19:0/2582452123 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3a4004bb0 con 0x7ff3ac108740 2024-05-09T02:23:33.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.617+0000 7ff3a37fe700 1 -- 172.21.15.19:0/2582452123 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4004020 con 0x7ff3ac108740 2024-05-09T02:23:33.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.617+0000 7ff3b44db700 1 -- 172.21.15.19:0/2582452123 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ac1a7760 con 0x7ff3ac108740 2024-05-09T02:23:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.621+0000 7ff3a37fe700 1 -- 172.21.15.19:0/2582452123 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff3a401b030 con 0x7ff3ac108740 2024-05-09T02:23:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.622+0000 7ff3c2ef1b80 1 Processor -- start 2024-05-09T02:23:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.622+0000 7ff3a37fe700 1 --2- 172.21.15.19:0/2582452123 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff3980b29f0 0x7ff3980b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.622+0000 7ff3b1a76700 1 --2- 172.21.15.19:0/2582452123 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff3980b29f0 0x7ff3980b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.622+0000 7ff3c2ef1b80 1 -- start start 2024-05-09T02:23:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.622+0000 7ff3a37fe700 1 -- 172.21.15.19:0/2582452123 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7ff3a40e20b0 con 0x7ff3ac108740 2024-05-09T02:23:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3c2ef1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d91ad00 0x563c2da23240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3c2ef1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d8a04a0 0x563c2da23780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3c2ef1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da1d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3c2ef1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563c2d9400a0 con 0x563c2d8a04a0 2024-05-09T02:23:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3c2ef1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563c2d8e59a0 con 0x563c2d91ad00 2024-05-09T02:23:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3c2ef1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c2d93c060 con 0x563c2da1d400 2024-05-09T02:23:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3b1a76700 1 --2- 172.21.15.19:0/2582452123 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff3980b29f0 0x7ff3980b4f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac106550 tx=0x7ff39c009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3b2a78700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da1d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3b1a76700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d8a04a0 0x563c2da23780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3b2a78700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da1d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:48928/0 (socket says 172.21.15.19:48928) 2024-05-09T02:23:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3b2a78700 1 -- 172.21.15.19:0/3702336300 learned_addr learned my addr 172.21.15.19:0/3702336300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3b2277700 1 --2- 172.21.15.19:0/3702336300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d91ad00 0x563c2da23240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3b2a78700 1 -- 172.21.15.19:0/3702336300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d91ad00 msgr2=0x563c2da23240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3b2a78700 1 --2- 172.21.15.19:0/3702336300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d91ad00 0x563c2da23240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3b2a78700 1 -- 172.21.15.19:0/3702336300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d8a04a0 msgr2=0x563c2da23780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3b2a78700 1 --2- 172.21.15.19:0/3702336300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d8a04a0 0x563c2da23780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.623+0000 7ff3b2a78700 1 -- 172.21.15.19:0/3702336300 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c2d8d6500 con 0x563c2da1d400 2024-05-09T02:23:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.624+0000 7ff3b2a78700 1 --2- 172.21.15.19:0/3702336300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da1d810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40a5470 tx=0x7ff3a40a8f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63a3d6662413d9ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.624+0000 7ff3a0d47700 1 -- 172.21.15.19:0/3702336300 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40a93f0 con 0x563c2da1d400 2024-05-09T02:23:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.624+0000 7ff3a0d47700 1 -- 172.21.15.19:0/3702336300 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3a40ad7a0 con 0x563c2da1d400 2024-05-09T02:23:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.624+0000 7ff3a0d47700 1 -- 172.21.15.19:0/3702336300 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40a3b10 con 0x563c2da1d400 2024-05-09T02:23:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.625+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3702336300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 msgr2=0x563c2da1d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.625+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3702336300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da1d810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40a5470 tx=0x7ff3a40a8f00 comp rx=0 tx=0).stop 2024-05-09T02:23:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.625+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3702336300 shutdown_connections 2024-05-09T02:23:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.625+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3702336300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da1d810 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.625+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3702336300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d91ad00 0x563c2da23240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.625+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3702336300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d8a04a0 0x563c2da23780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.625+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3702336300 >> 172.21.15.19:0/3702336300 conn(0x563c2d8a3f70 msgr2=0x563c2da18220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.625+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3702336300 shutdown_connections 2024-05-09T02:23:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.625+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3702336300 wait complete. 2024-05-09T02:23:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.625+0000 7ff3c2ef1b80 1 -- start start 2024-05-09T02:23:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.625+0000 7ff3c2ef1b80 1 Processor -- start 2024-05-09T02:23:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3c2ef1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d8a04a0 0x563c2da27050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3c2ef1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 0x563c2da27590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3c2ef1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da27ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3c2ef1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563c2da1e100 con 0x563c2d91ad00 2024-05-09T02:23:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3c2ef1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563c2d8dab20 con 0x563c2d8a04a0 2024-05-09T02:23:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3c2ef1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c2da331d0 con 0x563c2da1d400 2024-05-09T02:23:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3c2ef1b80 10 client.0 _ll_register_callbacks cb 0x563c2d7b16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3b1a76700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 0x563c2da27590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3b2a78700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da27ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3b1a76700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 0x563c2da27590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39700/0 (socket says 172.21.15.19:39700) 2024-05-09T02:23:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3b1a76700 1 -- 172.21.15.19:0/3749897391 learned_addr learned my addr 172.21.15.19:0/3749897391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3b2277700 1 --2- 172.21.15.19:0/3749897391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d8a04a0 0x563c2da27050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3b2a78700 1 -- 172.21.15.19:0/3749897391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d8a04a0 msgr2=0x563c2da27050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3b2a78700 1 --2- 172.21.15.19:0/3749897391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d8a04a0 0x563c2da27050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3b2a78700 1 -- 172.21.15.19:0/3749897391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 msgr2=0x563c2da27590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3b2a78700 1 --2- 172.21.15.19:0/3749897391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 0x563c2da27590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.626+0000 7ff3b2a78700 1 -- 172.21.15.19:0/3749897391 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c2d7bb490 con 0x563c2da1d400 2024-05-09T02:23:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.627+0000 7ff3b2a78700 1 --2- 172.21.15.19:0/3749897391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da27ad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40a5470 tx=0x7ff3a40a9880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13351e073bb8da06 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.627+0000 7ff395ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:23:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.627+0000 7ff395ffb700 1 -- 172.21.15.19:0/3749897391 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40b21c0 con 0x563c2da1d400 2024-05-09T02:23:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.627+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3749897391 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3a40ad7a0 con 0x563c2da1d400 2024-05-09T02:23:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.627+0000 7ff395ffb700 1 -- 172.21.15.19:0/3749897391 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7ff3a40a8860 con 0x563c2da1d400 2024-05-09T02:23:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.627+0000 7ff395ffb700 1 -- 172.21.15.19:0/3749897391 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff3a40a93f0 con 0x563c2da1d400 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.627+0000 7ff3c2ef1b80 10 client.26575 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.627+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3749897391 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563c2da34b40 con 0x563c2da1d400 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.627+0000 7ff395ffb700 1 -- 172.21.15.19:0/3749897391 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3a40ad7a0 con 0x563c2da1d400 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.627+0000 7ff395ffb700 1 -- 172.21.15.19:0/3749897391 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7ff3a40ca070 con 0x563c2da1d400 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.627+0000 7ff3c2ef1b80 10 client.26575 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.627+0000 7ff3c2ef1b80 10 client.26575 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.627+0000 7ff3c2ef1b80 20 client.26575 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.628+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3749897391 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563c2da372e0 0x563c2da39730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.628+0000 7ff3c2ef1b80 4 client.26575 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.628+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3749897391 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c2d94da80 con 0x563c2da372e0 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.628+0000 7ff3b2277700 1 --2- 172.21.15.19:0/3749897391 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563c2da372e0 0x563c2da39730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.628+0000 7ff3b2277700 1 --2- 172.21.15.19:0/3749897391 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563c2da372e0 0x563c2da39730 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=934ec99865f1b23a server_cookie=ec3cb326d93c82bb in_seq=0 out_seq=0 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.628+0000 7ff395ffb700 10 client.26575 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.629+0000 7ff395ffb700 1 -- 172.21.15.19:0/3749897391 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563c2d94da80 con 0x563c2da372e0 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.629+0000 7ff395ffb700 10 client.26575 handle_command_reply: tid=0 2024-05-09T02:23:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.629+0000 7ff3c2ef1b80 1 client.26575 shutdown 2024-05-09T02:23:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.629+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3749897391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 msgr2=0x563c2da27ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.629+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3749897391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da27ad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40a5470 tx=0x7ff3a40a9880 comp rx=0 tx=0).stop 2024-05-09T02:23:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.629+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3749897391 shutdown_connections 2024-05-09T02:23:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.629+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3749897391 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563c2da372e0 0x563c2da39730 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.629+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3749897391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da27ad0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.629+0000 7ff395ffb700 0 client.26575 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:23:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.629+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3749897391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d8a04a0 0x563c2da27050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.629+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3749897391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 0x563c2da27590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.629+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3749897391 >> 172.21.15.19:0/3749897391 conn(0x563c2d8a3f70 msgr2=0x563c2da11ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.629+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3749897391 shutdown_connections 2024-05-09T02:23:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.629+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3749897391 wait complete. 2024-05-09T02:23:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.629+0000 7ff3c2ef1b80 20 client.26575 trim_cache size 0 max 16384 2024-05-09T02:23:33.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.657+0000 7ff3c2ef1b80 1 Processor -- start 2024-05-09T02:23:33.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.657+0000 7ff3c2ef1b80 1 -- start start 2024-05-09T02:23:33.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.657+0000 7ff3c2ef1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d8a04a0 0x563c2da294f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.657+0000 7ff3c2ef1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 0x563c2da12700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.657+0000 7ff3c2ef1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da3ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.657+0000 7ff3c2ef1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1accc0 con 0x563c2d91ad00 2024-05-09T02:23:33.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.657+0000 7ff3c2ef1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1ae800 con 0x563c2d8a04a0 2024-05-09T02:23:33.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.657+0000 7ff3c2ef1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1ad390 con 0x563c2da1d400 2024-05-09T02:23:33.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.662+0000 7ff3b1a76700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 0x563c2da12700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:33.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.662+0000 7ff3b1a76700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 0x563c2da12700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39706/0 (socket says 172.21.15.19:39706) 2024-05-09T02:23:33.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.662+0000 7ff3b1a76700 1 -- 172.21.15.19:0/2553280971 learned_addr learned my addr 172.21.15.19:0/2553280971 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.662+0000 7ff3b1a76700 1 -- 172.21.15.19:0/2553280971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 msgr2=0x563c2da3ef30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:23:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.662+0000 7ff3b1a76700 1 --2- 172.21.15.19:0/2553280971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da3ef30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.662+0000 7ff3b1a76700 1 -- 172.21.15.19:0/2553280971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d8a04a0 msgr2=0x563c2da294f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:23:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.662+0000 7ff3b1a76700 1 --2- 172.21.15.19:0/2553280971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d8a04a0 0x563c2da294f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.662+0000 7ff3b1a76700 1 -- 172.21.15.19:0/2553280971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c2da149d0 con 0x563c2d91ad00 2024-05-09T02:23:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.663+0000 7ff3b1a76700 1 --2- 172.21.15.19:0/2553280971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 0x563c2da12700 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009880 tx=0x7ff39c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a605b52ce96f03c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.663+0000 7ff397fff700 1 -- 172.21.15.19:0/2553280971 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c02d070 con 0x563c2d91ad00 2024-05-09T02:23:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.663+0000 7ff397fff700 1 -- 172.21.15.19:0/2553280971 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff39c003e20 con 0x563c2d91ad00 2024-05-09T02:23:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.665+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/2553280971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 msgr2=0x563c2da12700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.665+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/2553280971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 0x563c2da12700 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009880 tx=0x7ff39c014000 comp rx=0 tx=0).stop 2024-05-09T02:23:33.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.675+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/2553280971 shutdown_connections 2024-05-09T02:23:33.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.675+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/2553280971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da3ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.675+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/2553280971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d8a04a0 0x563c2da294f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.675+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/2553280971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 0x563c2da12700 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.675+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/2553280971 >> 172.21.15.19:0/2553280971 conn(0x563c2d8a3f70 msgr2=0x563c2da11ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:33.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.683+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/2553280971 shutdown_connections 2024-05-09T02:23:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.685+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/2553280971 wait complete. 2024-05-09T02:23:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.685+0000 7ff3c2ef1b80 1 -- start start 2024-05-09T02:23:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.685+0000 7ff3c2ef1b80 1 Processor -- start 2024-05-09T02:23:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.687+0000 7ff3c2ef1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d8a04a0 0x563c2da2c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.687+0000 7ff3c2ef1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 0x563c2da2c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.687+0000 7ff3c2ef1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da2cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.687+0000 7ff3c2ef1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1ae130 con 0x563c2d91ad00 2024-05-09T02:23:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.687+0000 7ff3c2ef1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1ada60 con 0x563c2d8a04a0 2024-05-09T02:23:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.687+0000 7ff3c2ef1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1aac80 con 0x563c2da1d400 2024-05-09T02:23:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.687+0000 7ff3c2ef1b80 10 client.0 _ll_register_callbacks cb 0x563c2da18880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.687+0000 7ff3b2a78700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da2cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.688+0000 7ff3b2a78700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da2cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:48954/0 (socket says 172.21.15.19:48954) 2024-05-09T02:23:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.688+0000 7ff3b2a78700 1 -- 172.21.15.19:0/3561449682 learned_addr learned my addr 172.21.15.19:0/3561449682 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.688+0000 7ff3b2a78700 1 -- 172.21.15.19:0/3561449682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d8a04a0 msgr2=0x563c2da2c100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:23:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.688+0000 7ff3b2a78700 1 --2- 172.21.15.19:0/3561449682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d8a04a0 0x563c2da2c100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.688+0000 7ff3b2a78700 1 -- 172.21.15.19:0/3561449682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 msgr2=0x563c2da2c640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:23:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.688+0000 7ff3b2a78700 1 --2- 172.21.15.19:0/3561449682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 0x563c2da2c640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.688+0000 7ff3b2a78700 1 -- 172.21.15.19:0/3561449682 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c2da11cc0 con 0x563c2da1d400 2024-05-09T02:23:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.689+0000 7ff3b2a78700 1 --2- 172.21.15.19:0/3561449682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da2cb80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40a6660 tx=0x7ff3a40a8c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2286270620c00072 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.691+0000 7ff395ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:23:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.691+0000 7ff395ffb700 1 -- 172.21.15.19:0/3561449682 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40b21c0 con 0x563c2da1d400 2024-05-09T02:23:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.691+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3561449682 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff39c003e20 con 0x563c2da1d400 2024-05-09T02:23:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.691+0000 7ff395ffb700 1 -- 172.21.15.19:0/3561449682 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7ff3a40a8860 con 0x563c2da1d400 2024-05-09T02:23:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.693+0000 7ff395ffb700 1 -- 172.21.15.19:0/3561449682 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff3a40ad7a0 con 0x563c2da1d400 2024-05-09T02:23:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.693+0000 7ff3c2ef1b80 10 client.26578 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:33.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.693+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3561449682 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563c2dac2a60 con 0x563c2da1d400 2024-05-09T02:23:33.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.694+0000 7ff395ffb700 1 -- 172.21.15.19:0/3561449682 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff39c003e20 con 0x563c2da1d400 2024-05-09T02:23:33.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.694+0000 7ff395ffb700 1 -- 172.21.15.19:0/3561449682 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7ff3a40ca050 con 0x563c2da1d400 2024-05-09T02:23:33.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.694+0000 7ff3c2ef1b80 10 client.26578 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:33.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.694+0000 7ff3c2ef1b80 10 client.26578 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:23:33.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.694+0000 7ff3c2ef1b80 20 client.26578 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:33.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.694+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3561449682 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563c2dac2ea0 0x563c2dac52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:33.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.694+0000 7ff3c2ef1b80 4 client.26578 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:23:33.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.694+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3561449682 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c2d947b20 con 0x563c2dac2ea0 2024-05-09T02:23:33.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.699+0000 7ff3b2277700 1 --2- 172.21.15.19:0/3561449682 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563c2dac2ea0 0x563c2dac52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.707+0000 7ff3b2277700 1 --2- 172.21.15.19:0/3561449682 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563c2dac2ea0 0x563c2dac52c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9b102dfc9a382cbc server_cookie=7f25cae197c015ff in_seq=0 out_seq=0 2024-05-09T02:23:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.708+0000 7ff395ffb700 10 client.26578 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:23:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.708+0000 7ff395ffb700 1 -- 172.21.15.19:0/3561449682 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c2d947b20 con 0x563c2dac2ea0 2024-05-09T02:23:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.708+0000 7ff395ffb700 10 client.26578 handle_command_reply: tid=0 2024-05-09T02:23:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.709+0000 7ff3c2ef1b80 1 client.26578 shutdown 2024-05-09T02:23:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.709+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3561449682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 msgr2=0x563c2da2cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.709+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3561449682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da2cb80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40a6660 tx=0x7ff3a40a8c10 comp rx=0 tx=0).stop 2024-05-09T02:23:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.709+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3561449682 shutdown_connections 2024-05-09T02:23:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.709+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3561449682 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x563c2dac2ea0 0x563c2dac52c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.709+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3561449682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c2da1d400 0x563c2da2cb80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.709+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3561449682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c2d8a04a0 0x563c2da2c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.709+0000 7ff3c2ef1b80 1 --2- 172.21.15.19:0/3561449682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c2d91ad00 0x563c2da2c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.709+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3561449682 >> 172.21.15.19:0/3561449682 conn(0x563c2d8a3f70 msgr2=0x563c2da1c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.709+0000 7ff395ffb700 0 client.26578 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:23:33.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.710+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3561449682 shutdown_connections 2024-05-09T02:23:33.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.711+0000 7ff3c2ef1b80 1 -- 172.21.15.19:0/3561449682 wait complete. 2024-05-09T02:23:33.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.711+0000 7ff3c2ef1b80 20 client.26578 trim_cache size 0 max 16384 2024-05-09T02:23:33.714 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:23:33.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.712+0000 7ff3b44db700 1 -- 172.21.15.19:0/2582452123 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff3980b29f0 msgr2=0x7ff3980b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:33.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.712+0000 7ff3b44db700 1 --2- 172.21.15.19:0/2582452123 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff3980b29f0 0x7ff3980b4f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac106550 tx=0x7ff39c009260 comp rx=0 tx=0).stop 2024-05-09T02:23:33.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.713+0000 7ff3b44db700 1 -- 172.21.15.19:0/2582452123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3ac108740 msgr2=0x7ff3ac1a6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:33.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.713+0000 7ff3b44db700 1 --2- 172.21.15.19:0/2582452123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3ac108740 0x7ff3ac1a6c50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400efc0 tx=0x7ff3a400c3a0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.713+0000 7ff3b44db700 1 -- 172.21.15.19:0/2582452123 shutdown_connections 2024-05-09T02:23:33.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.713+0000 7ff3b44db700 1 --2- 172.21.15.19:0/2582452123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3ac108740 0x7ff3ac1a6c50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.713+0000 7ff3b44db700 1 --2- 172.21.15.19:0/2582452123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ac1034c0 0x7ff3ac19bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.713+0000 7ff3b44db700 1 --2- 172.21.15.19:0/2582452123 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff3980b29f0 0x7ff3980b4f00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.713+0000 7ff3b44db700 1 --2- 172.21.15.19:0/2582452123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac107df0 0x7ff3ac19c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:33.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.713+0000 7ff3b44db700 1 -- 172.21.15.19:0/2582452123 >> 172.21.15.19:0/2582452123 conn(0x7ff3ac0fedc0 msgr2=0x7ff3ac101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:33.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.713+0000 7ff3b44db700 1 -- 172.21.15.19:0/2582452123 shutdown_connections 2024-05-09T02:23:33.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:33.713+0000 7ff3b44db700 1 -- 172.21.15.19:0/2582452123 wait complete. 2024-05-09T02:23:34.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:34 smithi019 ceph-mon[30151]: pgmap v383: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.7 MiB/s wr, 534 op/s 2024-05-09T02:23:34.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:34 smithi059 ceph-mon[34781]: pgmap v383: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.7 MiB/s wr, 534 op/s 2024-05-09T02:23:35.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:34 smithi196 ceph-mon[34718]: pgmap v383: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.7 MiB/s wr, 534 op/s 2024-05-09T02:23:35.175 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T02:23:35.502 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:23:36.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:36 smithi019 ceph-mon[30151]: pgmap v384: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.1 MiB/s wr, 785 op/s 2024-05-09T02:23:36.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:36 smithi196 ceph-mon[34718]: pgmap v384: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.1 MiB/s wr, 785 op/s 2024-05-09T02:23:36.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:36 smithi059 ceph-mon[34781]: pgmap v384: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.1 MiB/s wr, 785 op/s 2024-05-09T02:23:37.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 715.0 s, 1303.8 tps, lat 76.621 ms stddev 79.633 2024-05-09T02:23:38.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.643+0000 7faf8a587700 1 -- 172.21.15.19:0/1945509927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf84108740 msgr2=0x7faf841100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:38.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.644+0000 7faf8a587700 1 --2- 172.21.15.19:0/1945509927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf84108740 0x7faf841100a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00b0d0 tx=0x7faf7c00b3e0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.645+0000 7faf8a587700 1 -- 172.21.15.19:0/1945509927 shutdown_connections 2024-05-09T02:23:38.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.645+0000 7faf8a587700 1 --2- 172.21.15.19:0/1945509927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf84108740 0x7faf841100a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.645+0000 7faf8a587700 1 --2- 172.21.15.19:0/1945509927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf841034c0 0x7faf841038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.645+0000 7faf8a587700 1 --2- 172.21.15.19:0/1945509927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf84107df0 0x7faf84108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.645+0000 7faf8a587700 1 -- 172.21.15.19:0/1945509927 >> 172.21.15.19:0/1945509927 conn(0x7faf840fedc0 msgr2=0x7faf841011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:38.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.646+0000 7faf8a587700 1 -- 172.21.15.19:0/1945509927 shutdown_connections 2024-05-09T02:23:38.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.646+0000 7faf8a587700 1 -- 172.21.15.19:0/1945509927 wait complete. 2024-05-09T02:23:38.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.647+0000 7faf8a587700 1 Processor -- start 2024-05-09T02:23:38.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.647+0000 7faf8a587700 1 -- start start 2024-05-09T02:23:38.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.647+0000 7faf8a587700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf841034c0 0x7faf8419bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.647+0000 7faf8a587700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf84107df0 0x7faf8419c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.647+0000 7faf8a587700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf84108740 0x7faf841a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.647+0000 7faf8a587700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf841133c0 con 0x7faf841034c0 2024-05-09T02:23:38.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.647+0000 7faf8a587700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faf84113240 con 0x7faf84107df0 2024-05-09T02:23:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.647+0000 7faf8a587700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faf84113540 con 0x7faf84108740 2024-05-09T02:23:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.647+0000 7faf83fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf841034c0 0x7faf8419bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.647+0000 7faf83fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf841034c0 0x7faf8419bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58876/0 (socket says 172.21.15.19:58876) 2024-05-09T02:23:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.647+0000 7faf83fff700 1 -- 172.21.15.19:0/2472867533 learned_addr learned my addr 172.21.15.19:0/2472867533 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.647+0000 7faf7bfff700 1 --2- 172.21.15.19:0/2472867533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf84107df0 0x7faf8419c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.647+0000 7faf88b24700 1 --2- 172.21.15.19:0/2472867533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf84108740 0x7faf841a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.648+0000 7faf7bfff700 1 -- 172.21.15.19:0/2472867533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf84108740 msgr2=0x7faf841a6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.648+0000 7faf7bfff700 1 --2- 172.21.15.19:0/2472867533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf84108740 0x7faf841a6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.648+0000 7faf7bfff700 1 -- 172.21.15.19:0/2472867533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf841034c0 msgr2=0x7faf8419bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.648+0000 7faf7bfff700 1 --2- 172.21.15.19:0/2472867533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf841034c0 0x7faf8419bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.648+0000 7faf7bfff700 1 -- 172.21.15.19:0/2472867533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf8419ca10 con 0x7faf84107df0 2024-05-09T02:23:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.648+0000 7faf83fff700 1 --2- 172.21.15.19:0/2472867533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf841034c0 0x7faf8419bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:23:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.648+0000 7faf7bfff700 1 --2- 172.21.15.19:0/2472867533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf84107df0 0x7faf8419c160 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faf84076ab0 tx=0x7faf6c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.648+0000 7faf81ffb700 1 -- 172.21.15.19:0/2472867533 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c017070 con 0x7faf84107df0 2024-05-09T02:23:38.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.648+0000 7faf8a587700 1 -- 172.21.15.19:0/2472867533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf841a7300 con 0x7faf84107df0 2024-05-09T02:23:38.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.648+0000 7faf8a587700 1 -- 172.21.15.19:0/2472867533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf841a7840 con 0x7faf84107df0 2024-05-09T02:23:38.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.648+0000 7faf81ffb700 1 -- 172.21.15.19:0/2472867533 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf6c009de0 con 0x7faf84107df0 2024-05-09T02:23:38.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.648+0000 7faf81ffb700 1 -- 172.21.15.19:0/2472867533 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c010d90 con 0x7faf84107df0 2024-05-09T02:23:38.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.651+0000 7faf81ffb700 1 -- 172.21.15.19:0/2472867533 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faf6c01a070 con 0x7faf84107df0 2024-05-09T02:23:38.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.652+0000 7faf81ffb700 1 --2- 172.21.15.19:0/2472867533 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7faf700b2a70 0x7faf700b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.652+0000 7faf83fff700 1 --2- 172.21.15.19:0/2472867533 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7faf700b2a70 0x7faf700b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.652+0000 7faf98f9db80 1 Processor -- start 2024-05-09T02:23:38.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.652+0000 7faf98f9db80 1 -- start start 2024-05-09T02:23:38.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.653+0000 7faf98f9db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec056a7160 0x55ec05747c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.653+0000 7faf98f9db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec055bd4a0 0x55ec05748180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.653+0000 7faf98f9db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec0573a6a0 0x55ec0573aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.653+0000 7faf98f9db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ec0565cff0 con 0x55ec055bd4a0 2024-05-09T02:23:38.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.653+0000 7faf98f9db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ec055fb7f0 con 0x55ec0573a6a0 2024-05-09T02:23:38.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.653+0000 7faf98f9db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ec05658fb0 con 0x55ec056a7160 2024-05-09T02:23:38.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.653+0000 7faf83fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec056a7160 0x55ec05747c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.653+0000 7faf88b24700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec0573a6a0 0x55ec0573aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.653+0000 7faf83fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec056a7160 0x55ec05747c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:55184/0 (socket says 172.21.15.19:55184) 2024-05-09T02:23:38.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.653+0000 7faf83fff700 1 -- 172.21.15.19:0/596879991 learned_addr learned my addr 172.21.15.19:0/596879991 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:38.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.654+0000 7faf88b24700 1 -- 172.21.15.19:0/596879991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec056a7160 msgr2=0x55ec05747c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:38.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.654+0000 7faf88b24700 1 --2- 172.21.15.19:0/596879991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec056a7160 0x55ec05747c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.654+0000 7faf88b24700 1 -- 172.21.15.19:0/596879991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec055bd4a0 msgr2=0x55ec05748180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:38.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.654+0000 7faf7bfff700 1 --2- 172.21.15.19:0/596879991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec055bd4a0 0x55ec05748180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.654+0000 7faf81ffb700 1 -- 172.21.15.19:0/2472867533 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7faf6c0e1ed0 con 0x7faf84107df0 2024-05-09T02:23:38.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.654+0000 7faf88b24700 1 --2- 172.21.15.19:0/596879991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec055bd4a0 0x55ec05748180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.654+0000 7faf88b24700 1 -- 172.21.15.19:0/596879991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec055f3370 con 0x55ec0573a6a0 2024-05-09T02:23:38.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.654+0000 7faf83fff700 1 --2- 172.21.15.19:0/596879991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec056a7160 0x55ec05747c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:23:38.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.654+0000 7faf7bfff700 1 --2- 172.21.15.19:0/596879991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec055bd4a0 0x55ec05748180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:23:38.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.654+0000 7faf83fff700 1 --2- 172.21.15.19:0/2472867533 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7faf700b2a70 0x7faf700b4f80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faf74013f60 tx=0x7faf74013560 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:38.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.654+0000 7faf88b24700 1 --2- 172.21.15.19:0/596879991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec0573a6a0 0x55ec0573aab0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00b770 tx=0x7faf7c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ee1f06e0b5a79f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:38.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.654+0000 7faf7a7fc700 1 -- 172.21.15.19:0/596879991 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c020030 con 0x55ec0573a6a0 2024-05-09T02:23:38.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.654+0000 7faf7a7fc700 1 -- 172.21.15.19:0/596879991 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf7c0092e0 con 0x55ec0573a6a0 2024-05-09T02:23:38.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.655+0000 7faf7a7fc700 1 -- 172.21.15.19:0/596879991 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c007930 con 0x55ec0573a6a0 2024-05-09T02:23:38.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.655+0000 7faf98f9db80 1 -- 172.21.15.19:0/596879991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec0573a6a0 msgr2=0x55ec0573aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:38.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.655+0000 7faf98f9db80 1 --2- 172.21.15.19:0/596879991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec0573a6a0 0x55ec0573aab0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00b770 tx=0x7faf7c009b80 comp rx=0 tx=0).stop 2024-05-09T02:23:38.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.656+0000 7faf98f9db80 1 -- 172.21.15.19:0/596879991 shutdown_connections 2024-05-09T02:23:38.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.656+0000 7faf98f9db80 1 --2- 172.21.15.19:0/596879991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec0573a6a0 0x55ec0573aab0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.656+0000 7faf98f9db80 1 --2- 172.21.15.19:0/596879991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec056a7160 0x55ec05747c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.656+0000 7faf98f9db80 1 --2- 172.21.15.19:0/596879991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec055bd4a0 0x55ec05748180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.656+0000 7faf98f9db80 1 -- 172.21.15.19:0/596879991 >> 172.21.15.19:0/596879991 conn(0x55ec0561d6a0 msgr2=0x55ec05735660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:38.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.656+0000 7faf98f9db80 1 -- 172.21.15.19:0/596879991 shutdown_connections 2024-05-09T02:23:38.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.656+0000 7faf98f9db80 1 -- 172.21.15.19:0/596879991 wait complete. 2024-05-09T02:23:38.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.656+0000 7faf98f9db80 1 -- start start 2024-05-09T02:23:38.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.657+0000 7faf98f9db80 1 Processor -- start 2024-05-09T02:23:38.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.657+0000 7faf98f9db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec055bd4a0 0x55ec0574c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.657+0000 7faf98f9db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec056a7160 0x55ec0574cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.657+0000 7faf98f9db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 0x55ec0574d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.657+0000 7faf98f9db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ec054d8ba0 con 0x55ec0573a6a0 2024-05-09T02:23:38.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.657+0000 7faf98f9db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ec05617150 con 0x55ec055bd4a0 2024-05-09T02:23:38.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.657+0000 7faf98f9db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ec0573b4a0 con 0x55ec056a7160 2024-05-09T02:23:38.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.657+0000 7faf98f9db80 10 client.0 _ll_register_callbacks cb 0x55ec054ce6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:38.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.657+0000 7faf88b24700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 0x55ec0574d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.657+0000 7faf83fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec055bd4a0 0x55ec0574c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.657+0000 7faf88b24700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 0x55ec0574d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58898/0 (socket says 172.21.15.19:58898) 2024-05-09T02:23:38.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.657+0000 7faf88b24700 1 -- 172.21.15.19:0/714787846 learned_addr learned my addr 172.21.15.19:0/714787846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:38.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.657+0000 7faf7bfff700 1 --2- 172.21.15.19:0/714787846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec056a7160 0x55ec0574cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.658+0000 7faf7bfff700 1 -- 172.21.15.19:0/714787846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec055bd4a0 msgr2=0x55ec0574c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:38.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.658+0000 7faf7bfff700 1 --2- 172.21.15.19:0/714787846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec055bd4a0 0x55ec0574c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.658+0000 7faf7bfff700 1 -- 172.21.15.19:0/714787846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 msgr2=0x55ec0574d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:38.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.658+0000 7faf7bfff700 1 --2- 172.21.15.19:0/714787846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 0x55ec0574d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.658+0000 7faf7bfff700 1 -- 172.21.15.19:0/714787846 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec054d8490 con 0x55ec056a7160 2024-05-09T02:23:38.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.658+0000 7faf88b24700 1 --2- 172.21.15.19:0/714787846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 0x55ec0574d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:23:38.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.658+0000 7faf7bfff700 1 --2- 172.21.15.19:0/714787846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec056a7160 0x55ec0574cec0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0df590 tx=0x7faf6c010690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=660cef409e2cdb11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:38.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.658+0000 7faf637fe700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:23:38.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.658+0000 7faf637fe700 1 -- 172.21.15.19:0/714787846 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c021070 con 0x55ec056a7160 2024-05-09T02:23:38.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.658+0000 7faf637fe700 1 -- 172.21.15.19:0/714787846 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7faf6c024e60 con 0x55ec056a7160 2024-05-09T02:23:38.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.658+0000 7faf98f9db80 1 -- 172.21.15.19:0/714787846 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7c0092e0 con 0x55ec056a7160 2024-05-09T02:23:38.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.658+0000 7faf637fe700 1 -- 172.21.15.19:0/714787846 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7faf6c024c70 con 0x55ec056a7160 2024-05-09T02:23:38.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.658+0000 7faf98f9db80 10 client.26590 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:38.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.658+0000 7faf98f9db80 1 -- 172.21.15.19:0/714787846 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55ec0575a270 con 0x55ec056a7160 2024-05-09T02:23:38.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.659+0000 7faf637fe700 1 -- 172.21.15.19:0/714787846 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf7c0092e0 con 0x55ec056a7160 2024-05-09T02:23:38.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.659+0000 7faf637fe700 1 -- 172.21.15.19:0/714787846 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7faf6c0dfce0 con 0x55ec056a7160 2024-05-09T02:23:38.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.659+0000 7faf98f9db80 10 client.26590 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:38.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.659+0000 7faf98f9db80 10 client.26590 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:23:38.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.659+0000 7faf98f9db80 20 client.26590 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:38.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.659+0000 7faf98f9db80 1 --2- 172.21.15.19:0/714787846 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ec0575c800 0x55ec0575ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.659+0000 7faf98f9db80 4 client.26590 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:23:38.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.659+0000 7faf98f9db80 1 -- 172.21.15.19:0/714787846 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec0566a9d0 con 0x55ec0575c800 2024-05-09T02:23:38.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.659+0000 7faf83fff700 1 --2- 172.21.15.19:0/714787846 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ec0575c800 0x55ec0575ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.660+0000 7faf637fe700 10 client.26590 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:23:38.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.660+0000 7faf83fff700 1 --2- 172.21.15.19:0/714787846 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ec0575c800 0x55ec0575ec90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9eabd9494ddfda0c server_cookie=de6e2127a6d08236 in_seq=0 out_seq=0 2024-05-09T02:23:38.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.661+0000 7faf637fe700 1 -- 172.21.15.19:0/714787846 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ec0566a9d0 con 0x55ec0575c800 2024-05-09T02:23:38.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.661+0000 7faf637fe700 10 client.26590 handle_command_reply: tid=0 2024-05-09T02:23:38.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.661+0000 7faf98f9db80 1 client.26590 shutdown 2024-05-09T02:23:38.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.663+0000 7faf98f9db80 1 -- 172.21.15.19:0/714787846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec056a7160 msgr2=0x55ec0574cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:38.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.663+0000 7faf98f9db80 1 --2- 172.21.15.19:0/714787846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec056a7160 0x55ec0574cec0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0df590 tx=0x7faf6c010690 comp rx=0 tx=0).stop 2024-05-09T02:23:38.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.663+0000 7faf98f9db80 1 -- 172.21.15.19:0/714787846 shutdown_connections 2024-05-09T02:23:38.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.663+0000 7faf98f9db80 1 --2- 172.21.15.19:0/714787846 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ec0575c800 0x55ec0575ec90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.663+0000 7faf98f9db80 1 --2- 172.21.15.19:0/714787846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 0x55ec0574d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.663+0000 7faf98f9db80 1 --2- 172.21.15.19:0/714787846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec055bd4a0 0x55ec0574c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.663+0000 7faf98f9db80 1 --2- 172.21.15.19:0/714787846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec056a7160 0x55ec0574cec0 secure :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0df590 tx=0x7faf6c010690 comp rx=0 tx=0).stop 2024-05-09T02:23:38.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.663+0000 7faf98f9db80 1 -- 172.21.15.19:0/714787846 >> 172.21.15.19:0/714787846 conn(0x55ec0561d6a0 msgr2=0x55ec05735220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:38.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.664+0000 7faf637fe700 0 client.26590 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:23:38.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.664+0000 7faf98f9db80 1 -- 172.21.15.19:0/714787846 shutdown_connections 2024-05-09T02:23:38.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.664+0000 7faf98f9db80 1 -- 172.21.15.19:0/714787846 wait complete. 2024-05-09T02:23:38.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.664+0000 7faf98f9db80 20 client.26590 trim_cache size 0 max 16384 2024-05-09T02:23:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.693+0000 7faf98f9db80 1 Processor -- start 2024-05-09T02:23:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.694+0000 7faf98f9db80 1 -- start start 2024-05-09T02:23:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.694+0000 7faf98f9db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec055bd4a0 0x55ec057492e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.694+0000 7faf98f9db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 0x55ec0574d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.694+0000 7faf98f9db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec05758820 0x55ec05746ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.695+0000 7faf98f9db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf841acd10 con 0x55ec0573a6a0 2024-05-09T02:23:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.695+0000 7faf98f9db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faf841ae850 con 0x55ec055bd4a0 2024-05-09T02:23:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.695+0000 7faf98f9db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faf841ad3e0 con 0x55ec05758820 2024-05-09T02:23:38.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.696+0000 7faf88b24700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec05758820 0x55ec05746ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.696+0000 7faf88b24700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec05758820 0x55ec05746ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:55194/0 (socket says 172.21.15.19:55194) 2024-05-09T02:23:38.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.696+0000 7faf88b24700 1 -- 172.21.15.19:0/2361049355 learned_addr learned my addr 172.21.15.19:0/2361049355 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:38.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.698+0000 7faf83fff700 1 --2- 172.21.15.19:0/2361049355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec055bd4a0 0x55ec057492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.698+0000 7faf88b24700 1 -- 172.21.15.19:0/2361049355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec055bd4a0 msgr2=0x55ec057492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:38.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.698+0000 7faf88b24700 1 --2- 172.21.15.19:0/2361049355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec055bd4a0 0x55ec057492e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.698+0000 7faf88b24700 1 -- 172.21.15.19:0/2361049355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 msgr2=0x55ec0574d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:38.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.698+0000 7faf88b24700 1 --2- 172.21.15.19:0/2361049355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 0x55ec0574d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.698+0000 7faf88b24700 1 -- 172.21.15.19:0/2361049355 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec0574a990 con 0x55ec05758820 2024-05-09T02:23:38.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.698+0000 7faf88b24700 1 --2- 172.21.15.19:0/2361049355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec05758820 0x55ec05746ac0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faf7c004fa0 tx=0x7faf7c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5a4fdf2efa88f59 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:38.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.698+0000 7faf79ffb700 1 -- 172.21.15.19:0/2361049355 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c020070 con 0x55ec05758820 2024-05-09T02:23:38.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.699+0000 7faf79ffb700 1 -- 172.21.15.19:0/2361049355 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf7c029070 con 0x55ec05758820 2024-05-09T02:23:38.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.699+0000 7faf79ffb700 1 -- 172.21.15.19:0/2361049355 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c0052b0 con 0x55ec05758820 2024-05-09T02:23:38.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.700+0000 7faf98f9db80 1 -- 172.21.15.19:0/2361049355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec05758820 msgr2=0x55ec05746ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:38.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.700+0000 7faf98f9db80 1 --2- 172.21.15.19:0/2361049355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec05758820 0x55ec05746ac0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faf7c004fa0 tx=0x7faf7c000f40 comp rx=0 tx=0).stop 2024-05-09T02:23:38.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.700+0000 7faf98f9db80 1 -- 172.21.15.19:0/2361049355 shutdown_connections 2024-05-09T02:23:38.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.700+0000 7faf98f9db80 1 --2- 172.21.15.19:0/2361049355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec05758820 0x55ec05746ac0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.700+0000 7faf98f9db80 1 --2- 172.21.15.19:0/2361049355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec055bd4a0 0x55ec057492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.700+0000 7faf98f9db80 1 --2- 172.21.15.19:0/2361049355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 0x55ec0574d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.700+0000 7faf98f9db80 1 -- 172.21.15.19:0/2361049355 >> 172.21.15.19:0/2361049355 conn(0x55ec0561d6a0 msgr2=0x55ec05748bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:38.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.702+0000 7faf98f9db80 1 -- 172.21.15.19:0/2361049355 shutdown_connections 2024-05-09T02:23:38.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.702+0000 7faf98f9db80 1 -- 172.21.15.19:0/2361049355 wait complete. 2024-05-09T02:23:38.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.702+0000 7faf98f9db80 1 -- start start 2024-05-09T02:23:38.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.703+0000 7faf98f9db80 1 Processor -- start 2024-05-09T02:23:38.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.703+0000 7faf98f9db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec055bd4a0 0x55ec05754ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.703+0000 7faf98f9db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 0x55ec057550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.703+0000 7faf98f9db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec05758820 0x55ec057456d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.703+0000 7faf98f9db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf841ae180 con 0x55ec0573a6a0 2024-05-09T02:23:38.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.703+0000 7faf98f9db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faf841adab0 con 0x55ec05758820 2024-05-09T02:23:38.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.703+0000 7faf98f9db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faf841aacd0 con 0x55ec055bd4a0 2024-05-09T02:23:38.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.703+0000 7faf98f9db80 10 client.0 _ll_register_callbacks cb 0x55ec054ce6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:38.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.703+0000 7faf7bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 0x55ec057550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.703+0000 7faf7bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 0x55ec057550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58912/0 (socket says 172.21.15.19:58912) 2024-05-09T02:23:38.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.703+0000 7faf7bfff700 1 -- 172.21.15.19:0/197439099 learned_addr learned my addr 172.21.15.19:0/197439099 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:38.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.704+0000 7faf88b24700 1 --2- 172.21.15.19:0/197439099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec05758820 0x55ec057456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.704+0000 7faf83fff700 1 --2- 172.21.15.19:0/197439099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec055bd4a0 0x55ec05754ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.704+0000 7faf7bfff700 1 -- 172.21.15.19:0/197439099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec055bd4a0 msgr2=0x55ec05754ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.704+0000 7faf7bfff700 1 --2- 172.21.15.19:0/197439099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec055bd4a0 0x55ec05754ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.704+0000 7faf7bfff700 1 -- 172.21.15.19:0/197439099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec05758820 msgr2=0x55ec057456d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.704+0000 7faf7bfff700 1 --2- 172.21.15.19:0/197439099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec05758820 0x55ec057456d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.704+0000 7faf7bfff700 1 -- 172.21.15.19:0/197439099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec05745c90 con 0x55ec0573a6a0 2024-05-09T02:23:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.704+0000 7faf7bfff700 1 --2- 172.21.15.19:0/197439099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 0x55ec057550e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009520 tx=0x7faf6c00b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3750b56e4a2cfb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.704+0000 7faf637fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:23:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.705+0000 7faf637fe700 1 -- 172.21.15.19:0/197439099 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c021030 con 0x55ec0573a6a0 2024-05-09T02:23:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.705+0000 7faf98f9db80 1 -- 172.21.15.19:0/197439099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7c029070 con 0x55ec0573a6a0 2024-05-09T02:23:38.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.705+0000 7faf637fe700 1 -- 172.21.15.19:0/197439099 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7faf6c0a57f0 con 0x55ec0573a6a0 2024-05-09T02:23:38.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.705+0000 7faf637fe700 1 -- 172.21.15.19:0/197439099 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7faf6c0dfa20 con 0x55ec0573a6a0 2024-05-09T02:23:38.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.705+0000 7faf98f9db80 10 client.16824 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:38.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.705+0000 7faf98f9db80 1 -- 172.21.15.19:0/197439099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55ec05763d70 con 0x55ec0573a6a0 2024-05-09T02:23:38.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.706+0000 7faf637fe700 1 -- 172.21.15.19:0/197439099 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf7c029070 con 0x55ec0573a6a0 2024-05-09T02:23:38.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.706+0000 7faf637fe700 1 -- 172.21.15.19:0/197439099 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7faf6c0dfce0 con 0x55ec0573a6a0 2024-05-09T02:23:38.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.706+0000 7faf98f9db80 10 client.16824 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:38.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.706+0000 7faf98f9db80 10 client.16824 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:23:38.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.706+0000 7faf98f9db80 20 client.16824 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:38.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.706+0000 7faf98f9db80 1 --2- 172.21.15.19:0/197439099 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ec057d8060 0x55ec057da480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:38.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.706+0000 7faf98f9db80 4 client.16824 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:23:38.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.706+0000 7faf98f9db80 1 -- 172.21.15.19:0/197439099 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec05664a70 con 0x55ec057d8060 2024-05-09T02:23:38.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.708+0000 7faf83fff700 1 --2- 172.21.15.19:0/197439099 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ec057d8060 0x55ec057da480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:38.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.712+0000 7faf83fff700 1 --2- 172.21.15.19:0/197439099 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ec057d8060 0x55ec057da480 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a39a812f5ac71997 server_cookie=8e121d845659b32 in_seq=0 out_seq=0 2024-05-09T02:23:38.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.712+0000 7faf637fe700 10 client.16824 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:23:38.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.712+0000 7faf637fe700 1 -- 172.21.15.19:0/197439099 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec05664a70 con 0x55ec057d8060 2024-05-09T02:23:38.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.712+0000 7faf637fe700 10 client.16824 handle_command_reply: tid=0 2024-05-09T02:23:38.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.712+0000 7faf98f9db80 1 client.16824 shutdown 2024-05-09T02:23:38.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.712+0000 7faf98f9db80 1 -- 172.21.15.19:0/197439099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 msgr2=0x55ec057550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:38.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.712+0000 7faf98f9db80 1 --2- 172.21.15.19:0/197439099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 0x55ec057550e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009520 tx=0x7faf6c00b640 comp rx=0 tx=0).stop 2024-05-09T02:23:38.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.712+0000 7faf98f9db80 1 -- 172.21.15.19:0/197439099 shutdown_connections 2024-05-09T02:23:38.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.712+0000 7faf98f9db80 1 --2- 172.21.15.19:0/197439099 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55ec057d8060 0x55ec057da480 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.712+0000 7faf98f9db80 1 --2- 172.21.15.19:0/197439099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec05758820 0x55ec057456d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.712+0000 7faf98f9db80 1 --2- 172.21.15.19:0/197439099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec055bd4a0 0x55ec05754ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.712+0000 7faf98f9db80 1 --2- 172.21.15.19:0/197439099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec0573a6a0 0x55ec057550e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.712+0000 7faf98f9db80 1 -- 172.21.15.19:0/197439099 >> 172.21.15.19:0/197439099 conn(0x55ec0561d6a0 msgr2=0x55ec05746ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:38.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.712+0000 7faf637fe700 0 client.16824 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:23:38.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.713+0000 7faf98f9db80 1 -- 172.21.15.19:0/197439099 shutdown_connections 2024-05-09T02:23:38.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.713+0000 7faf98f9db80 1 -- 172.21.15.19:0/197439099 wait complete. 2024-05-09T02:23:38.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.713+0000 7faf98f9db80 20 client.16824 trim_cache size 0 max 16384 2024-05-09T02:23:38.717 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:23:38.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.714+0000 7faf8a587700 1 -- 172.21.15.19:0/2472867533 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7faf700b2a70 msgr2=0x7faf700b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:38.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.714+0000 7faf8a587700 1 --2- 172.21.15.19:0/2472867533 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7faf700b2a70 0x7faf700b4f80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faf74013f60 tx=0x7faf74013560 comp rx=0 tx=0).stop 2024-05-09T02:23:38.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.714+0000 7faf8a587700 1 -- 172.21.15.19:0/2472867533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf84107df0 msgr2=0x7faf8419c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:38.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.714+0000 7faf8a587700 1 --2- 172.21.15.19:0/2472867533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf84107df0 0x7faf8419c160 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faf84076ab0 tx=0x7faf6c00dc30 comp rx=0 tx=0).stop 2024-05-09T02:23:38.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.714+0000 7faf8a587700 1 -- 172.21.15.19:0/2472867533 shutdown_connections 2024-05-09T02:23:38.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.714+0000 7faf8a587700 1 --2- 172.21.15.19:0/2472867533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf84108740 0x7faf841a6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.714+0000 7faf8a587700 1 --2- 172.21.15.19:0/2472867533 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7faf700b2a70 0x7faf700b4f80 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faf74013f60 tx=0x7faf74013560 comp rx=0 tx=0).stop 2024-05-09T02:23:38.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.714+0000 7faf8a587700 1 --2- 172.21.15.19:0/2472867533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf841034c0 0x7faf8419bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:38.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.714+0000 7faf8a587700 1 --2- 172.21.15.19:0/2472867533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf84107df0 0x7faf8419c160 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faf84076ab0 tx=0x7faf6c00dc30 comp rx=0 tx=0).stop 2024-05-09T02:23:38.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.714+0000 7faf8a587700 1 -- 172.21.15.19:0/2472867533 >> 172.21.15.19:0/2472867533 conn(0x7faf840fedc0 msgr2=0x7faf84107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:38.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.715+0000 7faf8a587700 1 -- 172.21.15.19:0/2472867533 shutdown_connections 2024-05-09T02:23:38.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:38.715+0000 7faf8a587700 1 -- 172.21.15.19:0/2472867533 wait complete. 2024-05-09T02:23:38.734 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:38 smithi019 ceph-mon[30151]: pgmap v385: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.9 MiB/s wr, 536 op/s 2024-05-09T02:23:38.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:38 smithi059 ceph-mon[34781]: pgmap v385: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.9 MiB/s wr, 536 op/s 2024-05-09T02:23:39.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:38 smithi196 ceph-mon[34718]: pgmap v385: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.9 MiB/s wr, 536 op/s 2024-05-09T02:23:39.537 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T02:23:40.061 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:23:40.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:40 smithi019 ceph-mon[30151]: pgmap v386: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.4 MiB/s wr, 695 op/s 2024-05-09T02:23:40.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:40 smithi059 ceph-mon[34781]: pgmap v386: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.4 MiB/s wr, 695 op/s 2024-05-09T02:23:41.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:40 smithi196 ceph-mon[34718]: pgmap v386: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.4 MiB/s wr, 695 op/s 2024-05-09T02:23:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 720.0 s, 959.0 tps, lat 104.590 ms stddev 175.259 2024-05-09T02:23:42.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.687+0000 7fa0d7466700 1 -- 172.21.15.19:0/3697104131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0d010a930 msgr2=0x7fa0d010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:42.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.687+0000 7fa0d7466700 1 --2- 172.21.15.19:0/3697104131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0d010a930 0x7fa0d010ad40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009990 tx=0x7fa0cc009ca0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.688+0000 7fa0d7466700 1 -- 172.21.15.19:0/3697104131 shutdown_connections 2024-05-09T02:23:42.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.688+0000 7fa0d7466700 1 --2- 172.21.15.19:0/3697104131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d011d930 0x7fa0d011fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.688+0000 7fa0d7466700 1 --2- 172.21.15.19:0/3697104131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0d010a930 0x7fa0d010ad40 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.688+0000 7fa0d7466700 1 --2- 172.21.15.19:0/3697104131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0d010b280 0x7fa0d011d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.688+0000 7fa0d7466700 1 -- 172.21.15.19:0/3697104131 >> 172.21.15.19:0/3697104131 conn(0x7fa0d006f4d0 msgr2=0x7fa0d006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:42.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.690+0000 7fa0d7466700 1 -- 172.21.15.19:0/3697104131 shutdown_connections 2024-05-09T02:23:42.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.690+0000 7fa0d7466700 1 -- 172.21.15.19:0/3697104131 wait complete. 2024-05-09T02:23:42.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.691+0000 7fa0d7466700 1 Processor -- start 2024-05-09T02:23:42.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.691+0000 7fa0d7466700 1 -- start start 2024-05-09T02:23:42.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.692+0000 7fa0d7466700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0d010a930 0x7fa0d01af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.692+0000 7fa0d7466700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d010b280 0x7fa0d01afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.692+0000 7fa0d7466700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0d011d930 0x7fa0d01a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.692+0000 7fa0d7466700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01233a0 con 0x7fa0d011d930 2024-05-09T02:23:42.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.692+0000 7fa0d7466700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d0123220 con 0x7fa0d010a930 2024-05-09T02:23:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.692+0000 7fa0d7466700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d0123520 con 0x7fa0d010b280 2024-05-09T02:23:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.692+0000 7fa0d5202700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0d010a930 0x7fa0d01af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.692+0000 7fa0d5202700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0d010a930 0x7fa0d01af9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:55892/0 (socket says 172.21.15.19:55892) 2024-05-09T02:23:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.692+0000 7fa0d5202700 1 -- 172.21.15.19:0/103080848 learned_addr learned my addr 172.21.15.19:0/103080848 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.693+0000 7fa0d5a03700 1 --2- 172.21.15.19:0/103080848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0d011d930 0x7fa0d01a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.693+0000 7fa0d4a01700 1 --2- 172.21.15.19:0/103080848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d010b280 0x7fa0d01afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.695+0000 7fa0d5202700 1 -- 172.21.15.19:0/103080848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d010b280 msgr2=0x7fa0d01afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.695+0000 7fa0d5202700 1 --2- 172.21.15.19:0/103080848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d010b280 0x7fa0d01afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.695+0000 7fa0d5202700 1 -- 172.21.15.19:0/103080848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0d011d930 msgr2=0x7fa0d01a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.695+0000 7fa0d5202700 1 --2- 172.21.15.19:0/103080848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0d011d930 0x7fa0d01a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.695+0000 7fa0d5202700 1 -- 172.21.15.19:0/103080848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d01aa270 con 0x7fa0d010a930 2024-05-09T02:23:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.701+0000 7fa0d5202700 1 --2- 172.21.15.19:0/103080848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0d010a930 0x7fa0d01af9b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0057c0 tx=0x7fa0cc0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.705+0000 7fa0c67fc700 1 -- 172.21.15.19:0/103080848 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc026070 con 0x7fa0d010a930 2024-05-09T02:23:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.705+0000 7fa0d7466700 1 -- 172.21.15.19:0/103080848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d01aa500 con 0x7fa0d010a930 2024-05-09T02:23:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.705+0000 7fa0d7466700 1 -- 172.21.15.19:0/103080848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d01b6760 con 0x7fa0d010a930 2024-05-09T02:23:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.707+0000 7fa0c67fc700 1 -- 172.21.15.19:0/103080848 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0cc021410 con 0x7fa0d010a930 2024-05-09T02:23:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.707+0000 7fa0c67fc700 1 -- 172.21.15.19:0/103080848 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc017650 con 0x7fa0d010a930 2024-05-09T02:23:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.710+0000 7fa0c67fc700 1 -- 172.21.15.19:0/103080848 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa0cc009d10 con 0x7fa0d010a930 2024-05-09T02:23:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.712+0000 7fa0e5e7cb80 1 Processor -- start 2024-05-09T02:23:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.713+0000 7fa0c67fc700 1 --2- 172.21.15.19:0/103080848 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0bc0b2c00 0x7fa0bc0b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.713+0000 7fa0c67fc700 1 -- 172.21.15.19:0/103080848 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fa0cc0e3da0 con 0x7fa0d010a930 2024-05-09T02:23:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.714+0000 7fa0d5a03700 1 --2- 172.21.15.19:0/103080848 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0bc0b2c00 0x7fa0bc0b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.714+0000 7fa0e5e7cb80 1 -- start start 2024-05-09T02:23:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.714+0000 7fa0e5e7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95cead00 0x557f95df3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.714+0000 7fa0e5e7cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95c704a0 0x557f95df3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.714+0000 7fa0e5e7cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95ded400 0x557f95ded810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.714+0000 7fa0e5e7cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557f95d100a0 con 0x557f95cead00 2024-05-09T02:23:42.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.714+0000 7fa0e5e7cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557f95cb59a0 con 0x557f95c704a0 2024-05-09T02:23:42.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.714+0000 7fa0e5e7cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557f95d0c060 con 0x557f95ded400 2024-05-09T02:23:42.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.715+0000 7fa0d5202700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95cead00 0x557f95df3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.715+0000 7fa0d5202700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95cead00 0x557f95df3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58948/0 (socket says 172.21.15.19:58948) 2024-05-09T02:23:42.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.715+0000 7fa0d5202700 1 -- 172.21.15.19:0/3801068428 learned_addr learned my addr 172.21.15.19:0/3801068428 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:42.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.715+0000 7fa0d5202700 1 -- 172.21.15.19:0/3801068428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95ded400 msgr2=0x557f95ded810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:23:42.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.715+0000 7fa0d5202700 1 --2- 172.21.15.19:0/3801068428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95ded400 0x557f95ded810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.715+0000 7fa0d5202700 1 -- 172.21.15.19:0/3801068428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95c704a0 msgr2=0x557f95df3780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:23:42.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.715+0000 7fa0d5202700 1 --2- 172.21.15.19:0/3801068428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95c704a0 0x557f95df3780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.715+0000 7fa0d5202700 1 -- 172.21.15.19:0/3801068428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f95ca6500 con 0x557f95cead00 2024-05-09T02:23:42.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.715+0000 7fa0d5202700 1 --2- 172.21.15.19:0/3801068428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95cead00 0x557f95df3240 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0a7100 tx=0x7fa0cc0aaed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92ffac89536e932 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:42.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.717+0000 7fa0d5a03700 1 --2- 172.21.15.19:0/103080848 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0bc0b2c00 0x7fa0bc0b5110 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0112ba0 tx=0x7fa0c0008040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.725+0000 7fa0bb7fe700 1 -- 172.21.15.19:0/3801068428 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0b3e50 con 0x557f95cead00 2024-05-09T02:23:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.725+0000 7fa0bb7fe700 1 -- 172.21.15.19:0/3801068428 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0cc0af110 con 0x557f95cead00 2024-05-09T02:23:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.725+0000 7fa0bb7fe700 1 -- 172.21.15.19:0/3801068428 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0e1bb0 con 0x557f95cead00 2024-05-09T02:23:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.726+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/3801068428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95cead00 msgr2=0x557f95df3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.726+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/3801068428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95cead00 0x557f95df3240 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0a7100 tx=0x7fa0cc0aaed0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.726+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/3801068428 shutdown_connections 2024-05-09T02:23:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.726+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/3801068428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95ded400 0x557f95ded810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.726+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/3801068428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95cead00 0x557f95df3240 secure :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0a7100 tx=0x7fa0cc0aaed0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.726+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/3801068428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95c704a0 0x557f95df3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.726+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/3801068428 >> 172.21.15.19:0/3801068428 conn(0x557f95c73f70 msgr2=0x557f95de8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.727+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/3801068428 shutdown_connections 2024-05-09T02:23:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.727+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/3801068428 wait complete. 2024-05-09T02:23:42.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.728+0000 7fa0e5e7cb80 1 -- start start 2024-05-09T02:23:42.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.728+0000 7fa0e5e7cb80 1 Processor -- start 2024-05-09T02:23:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.729+0000 7fa0e5e7cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95c704a0 0x557f95df6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.730+0000 7fa0e5e7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95ded400 0x557f95df7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.730+0000 7fa0d5202700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95c704a0 0x557f95df6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.730+0000 7fa0d5202700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95c704a0 0x557f95df6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:55898/0 (socket says 172.21.15.19:55898) 2024-05-09T02:23:42.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.731+0000 7fa0d5202700 1 -- 172.21.15.19:0/2581033056 learned_addr learned my addr 172.21.15.19:0/2581033056 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:42.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.731+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/2581033056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95df7a70 0x557f95e25090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.731+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2581033056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557f95dedfb0 con 0x557f95ded400 2024-05-09T02:23:42.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.731+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2581033056 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557f95caab20 con 0x557f95c704a0 2024-05-09T02:23:42.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.731+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2581033056 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557f95dee130 con 0x557f95df7a70 2024-05-09T02:23:42.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.731+0000 7fa0d4a01700 1 --2- 172.21.15.19:0/2581033056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95ded400 0x557f95df7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.731+0000 7fa0e5e7cb80 10 client.0 _ll_register_callbacks cb 0x557f95b816c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:42.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.731+0000 7fa0d5a03700 1 --2- 172.21.15.19:0/2581033056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95df7a70 0x557f95e25090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.731+0000 7fa0d4a01700 1 -- 172.21.15.19:0/2581033056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95df7a70 msgr2=0x557f95e25090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.731+0000 7fa0d4a01700 1 --2- 172.21.15.19:0/2581033056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95df7a70 0x557f95e25090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.731+0000 7fa0d4a01700 1 -- 172.21.15.19:0/2581033056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95c704a0 msgr2=0x557f95df6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.731+0000 7fa0d4a01700 1 --2- 172.21.15.19:0/2581033056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95c704a0 0x557f95df6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.731+0000 7fa0d4a01700 1 -- 172.21.15.19:0/2581033056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f95b8b490 con 0x557f95ded400 2024-05-09T02:23:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.731+0000 7fa0d5202700 1 --2- 172.21.15.19:0/2581033056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95c704a0 0x557f95df6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:23:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.732+0000 7fa0d4a01700 1 --2- 172.21.15.19:0/2581033056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95ded400 0x557f95df7530 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa0d01ab270 tx=0x7fa0c8011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179d373b46b5c393 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.732+0000 7fa0b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:23:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.732+0000 7fa0b8ff9700 1 -- 172.21.15.19:0/2581033056 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8019020 con 0x557f95ded400 2024-05-09T02:23:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.732+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2581033056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0cc0af110 con 0x557f95ded400 2024-05-09T02:23:42.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.732+0000 7fa0b8ff9700 1 -- 172.21.15.19:0/2581033056 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fa0c8003c00 con 0x557f95ded400 2024-05-09T02:23:42.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.732+0000 7fa0b8ff9700 1 -- 172.21.15.19:0/2581033056 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa0c8024050 con 0x557f95ded400 2024-05-09T02:23:42.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.732+0000 7fa0e5e7cb80 10 client.16836 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:42.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.732+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2581033056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557f95e26e10 con 0x557f95ded400 2024-05-09T02:23:42.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.733+0000 7fa0b8ff9700 1 -- 172.21.15.19:0/2581033056 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0cc0af110 con 0x557f95ded400 2024-05-09T02:23:42.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.733+0000 7fa0b8ff9700 1 -- 172.21.15.19:0/2581033056 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fa0c80040b0 con 0x557f95ded400 2024-05-09T02:23:42.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.733+0000 7fa0e5e7cb80 10 client.16836 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:42.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.733+0000 7fa0e5e7cb80 10 client.16836 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:23:42.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.733+0000 7fa0e5e7cb80 20 client.16836 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:42.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.733+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/2581033056 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557f95e293a0 0x557f95e2b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.733+0000 7fa0e5e7cb80 4 client.16836 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:23:42.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.733+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2581033056 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f95d1da80 con 0x557f95e293a0 2024-05-09T02:23:42.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.734+0000 7fa0d5202700 1 --2- 172.21.15.19:0/2581033056 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557f95e293a0 0x557f95e2b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.736+0000 7fa0b8ff9700 10 client.16836 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:23:42.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.737+0000 7fa0d5202700 1 --2- 172.21.15.19:0/2581033056 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557f95e293a0 0x557f95e2b850 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=687333f524e51734 server_cookie=487e2fb106e2ad1f in_seq=0 out_seq=0 2024-05-09T02:23:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.738+0000 7fa0b8ff9700 1 -- 172.21.15.19:0/2581033056 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f95d1da80 con 0x557f95e293a0 2024-05-09T02:23:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.738+0000 7fa0b8ff9700 10 client.16836 handle_command_reply: tid=0 2024-05-09T02:23:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.738+0000 7fa0e5e7cb80 1 client.16836 shutdown 2024-05-09T02:23:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.738+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2581033056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95ded400 msgr2=0x557f95df7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.738+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/2581033056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95ded400 0x557f95df7530 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa0d01ab270 tx=0x7fa0c8011f50 comp rx=0 tx=0).stop 2024-05-09T02:23:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.738+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2581033056 shutdown_connections 2024-05-09T02:23:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.738+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/2581033056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95df7a70 0x557f95e25090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.738+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/2581033056 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557f95e293a0 0x557f95e2b850 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.738+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/2581033056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95c704a0 0x557f95df6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.738+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/2581033056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95ded400 0x557f95df7530 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.738+0000 7fa0b8ff9700 0 client.16836 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:23:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.738+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2581033056 >> 172.21.15.19:0/2581033056 conn(0x557f95c73f70 msgr2=0x557f95de1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.739+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2581033056 shutdown_connections 2024-05-09T02:23:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.739+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2581033056 wait complete. 2024-05-09T02:23:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.739+0000 7fa0e5e7cb80 20 client.16836 trim_cache size 0 max 16384 2024-05-09T02:23:42.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.762+0000 7fa0e5e7cb80 1 Processor -- start 2024-05-09T02:23:42.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.762+0000 7fa0e5e7cb80 1 -- start start 2024-05-09T02:23:42.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:42 smithi019 ceph-mon[30151]: pgmap v387: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.9 MiB/s wr, 615 op/s 2024-05-09T02:23:42.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.763+0000 7fa0e5e7cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95c704a0 0x557f95df5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.763+0000 7fa0e5e7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95ded400 0x557f95de2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.763+0000 7fa0d5202700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95c704a0 0x557f95df5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.763+0000 7fa0d5202700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95c704a0 0x557f95df5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:55222/0 (socket says 172.21.15.19:55222) 2024-05-09T02:23:42.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.763+0000 7fa0d5202700 1 -- 172.21.15.19:0/2784076688 learned_addr learned my addr 172.21.15.19:0/2784076688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:42.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.763+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/2784076688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95df7a70 0x557f95e310a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.763+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2784076688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b400d650 con 0x557f95ded400 2024-05-09T02:23:42.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.763+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2784076688 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b400f190 con 0x557f95df7a70 2024-05-09T02:23:42.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.763+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2784076688 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b400dd20 con 0x557f95c704a0 2024-05-09T02:23:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.764+0000 7fa0d5a03700 1 --2- 172.21.15.19:0/2784076688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95df7a70 0x557f95e310a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.764+0000 7fa0d4a01700 1 --2- 172.21.15.19:0/2784076688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95ded400 0x557f95de2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.764+0000 7fa0d5202700 1 -- 172.21.15.19:0/2784076688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95df7a70 msgr2=0x557f95e310a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.764+0000 7fa0d5202700 1 --2- 172.21.15.19:0/2784076688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95df7a70 0x557f95e310a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.764+0000 7fa0d5202700 1 -- 172.21.15.19:0/2784076688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95ded400 msgr2=0x557f95de2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.764+0000 7fa0d5202700 1 --2- 172.21.15.19:0/2784076688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95ded400 0x557f95de2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.764+0000 7fa0d5202700 1 -- 172.21.15.19:0/2784076688 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f95df8450 con 0x557f95c704a0 2024-05-09T02:23:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.765+0000 7fa0d5202700 1 --2- 172.21.15.19:0/2784076688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95c704a0 0x557f95df5f50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0ab240 tx=0x7fa0cc0aa9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14ed27a767d88409 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.765+0000 7fa0baffd700 1 -- 172.21.15.19:0/2784076688 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0af110 con 0x557f95c704a0 2024-05-09T02:23:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.765+0000 7fa0baffd700 1 -- 172.21.15.19:0/2784076688 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0cc0ab2e0 con 0x557f95c704a0 2024-05-09T02:23:42.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.766+0000 7fa0baffd700 1 -- 172.21.15.19:0/2784076688 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0aa540 con 0x557f95c704a0 2024-05-09T02:23:42.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.766+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2784076688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95c704a0 msgr2=0x557f95df5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:42.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.766+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/2784076688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95c704a0 0x557f95df5f50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0ab240 tx=0x7fa0cc0aa9d0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.766+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2784076688 shutdown_connections 2024-05-09T02:23:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.766+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/2784076688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95df7a70 0x557f95e310a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.766+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/2784076688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95c704a0 0x557f95df5f50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.766+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/2784076688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95ded400 0x557f95de2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.766+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2784076688 >> 172.21.15.19:0/2784076688 conn(0x557f95c73f70 msgr2=0x557f95de1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.766+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2784076688 shutdown_connections 2024-05-09T02:23:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.766+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/2784076688 wait complete. 2024-05-09T02:23:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.767+0000 7fa0e5e7cb80 1 -- start start 2024-05-09T02:23:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.767+0000 7fa0e5e7cb80 1 Processor -- start 2024-05-09T02:23:42.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.768+0000 7fa0e5e7cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95c704a0 0x557f95e215c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.768+0000 7fa0e5e7cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95ded400 0x557f95e24f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.768+0000 7fa0e5e7cb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95df7a70 0x557f95e25460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.768+0000 7fa0e5e7cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b400eac0 con 0x557f95c704a0 2024-05-09T02:23:42.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.768+0000 7fa0e5e7cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b400e3f0 con 0x557f95ded400 2024-05-09T02:23:42.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.768+0000 7fa0e5e7cb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b400b610 con 0x557f95df7a70 2024-05-09T02:23:42.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.768+0000 7fa0e5e7cb80 10 client.0 _ll_register_callbacks cb 0x557f95e2bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:42.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.768+0000 7fa0d5202700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95c704a0 0x557f95e215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.768+0000 7fa0d4a01700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95ded400 0x557f95e24f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.768+0000 7fa0d5202700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95c704a0 0x557f95e215c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58972/0 (socket says 172.21.15.19:58972) 2024-05-09T02:23:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.768+0000 7fa0d5202700 1 -- 172.21.15.19:0/1636889016 learned_addr learned my addr 172.21.15.19:0/1636889016 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.768+0000 7fa0d5a03700 1 --2- 172.21.15.19:0/1636889016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95df7a70 0x557f95e25460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.769+0000 7fa0d5202700 1 -- 172.21.15.19:0/1636889016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95df7a70 msgr2=0x557f95e25460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.769+0000 7fa0d5202700 1 --2- 172.21.15.19:0/1636889016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95df7a70 0x557f95e25460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.769+0000 7fa0d5202700 1 -- 172.21.15.19:0/1636889016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95ded400 msgr2=0x557f95e24f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.769+0000 7fa0d5202700 1 --2- 172.21.15.19:0/1636889016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95ded400 0x557f95e24f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.769+0000 7fa0d5202700 1 -- 172.21.15.19:0/1636889016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f95e25b00 con 0x557f95c704a0 2024-05-09T02:23:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.769+0000 7fa0d5a03700 1 --2- 172.21.15.19:0/1636889016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95df7a70 0x557f95e25460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:23:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.769+0000 7fa0d5202700 1 --2- 172.21.15.19:0/1636889016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95c704a0 0x557f95e215c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc005040 tx=0x7fa0cc0aab00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df910ac858752d45 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.769+0000 7fa0b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:23:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.769+0000 7fa0b8ff9700 1 -- 172.21.15.19:0/1636889016 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0050d0 con 0x557f95c704a0 2024-05-09T02:23:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.769+0000 7fa0b8ff9700 1 -- 172.21.15.19:0/1636889016 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fa0cc005270 con 0x557f95c704a0 2024-05-09T02:23:42.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.769+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/1636889016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0cc0ab2e0 con 0x557f95c704a0 2024-05-09T02:23:42.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.769+0000 7fa0b8ff9700 1 -- 172.21.15.19:0/1636889016 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa0cc0aa2e0 con 0x557f95c704a0 2024-05-09T02:23:42.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.769+0000 7fa0e5e7cb80 10 client.16842 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:42.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.769+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/1636889016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557f95e92cb0 con 0x557f95c704a0 2024-05-09T02:23:42.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.770+0000 7fa0b8ff9700 1 -- 172.21.15.19:0/1636889016 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0cc0ab2e0 con 0x557f95c704a0 2024-05-09T02:23:42.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.770+0000 7fa0b8ff9700 1 -- 172.21.15.19:0/1636889016 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fa0cc029030 con 0x557f95c704a0 2024-05-09T02:23:42.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.770+0000 7fa0e5e7cb80 10 client.16842 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:42.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.770+0000 7fa0e5e7cb80 10 client.16842 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:23:42.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.770+0000 7fa0e5e7cb80 20 client.16842 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:42.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.770+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/1636889016 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557f95e93000 0x557f95e95420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:42.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.770+0000 7fa0e5e7cb80 4 client.16842 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:23:42.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.770+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/1636889016 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f95d17b20 con 0x557f95e93000 2024-05-09T02:23:42.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.770+0000 7fa0d4a01700 1 --2- 172.21.15.19:0/1636889016 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557f95e93000 0x557f95e95420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:42.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.771+0000 7fa0b8ff9700 10 client.16842 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:23:42.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.771+0000 7fa0d4a01700 1 --2- 172.21.15.19:0/1636889016 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557f95e93000 0x557f95e95420 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f46fe6a8eb64f4b8 server_cookie=eabe5cc744846972 in_seq=0 out_seq=0 2024-05-09T02:23:42.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.772+0000 7fa0b8ff9700 1 -- 172.21.15.19:0/1636889016 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f95d17b20 con 0x557f95e93000 2024-05-09T02:23:42.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.772+0000 7fa0b8ff9700 10 client.16842 handle_command_reply: tid=0 2024-05-09T02:23:42.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.772+0000 7fa0e5e7cb80 1 client.16842 shutdown 2024-05-09T02:23:42.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.772+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/1636889016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95c704a0 msgr2=0x557f95e215c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:42.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.772+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/1636889016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95c704a0 0x557f95e215c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc005040 tx=0x7fa0cc0aab00 comp rx=0 tx=0).stop 2024-05-09T02:23:42.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.772+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/1636889016 shutdown_connections 2024-05-09T02:23:42.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.772+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/1636889016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f95df7a70 0x557f95e25460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.772+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/1636889016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f95c704a0 0x557f95e215c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.772+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/1636889016 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x557f95e93000 0x557f95e95420 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.772+0000 7fa0e5e7cb80 1 --2- 172.21.15.19:0/1636889016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f95ded400 0x557f95e24f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.772+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/1636889016 >> 172.21.15.19:0/1636889016 conn(0x557f95c73f70 msgr2=0x557f95df8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:42.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.772+0000 7fa0b8ff9700 0 client.16842 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:23:42.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.772+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/1636889016 shutdown_connections 2024-05-09T02:23:42.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.772+0000 7fa0e5e7cb80 1 -- 172.21.15.19:0/1636889016 wait complete. 2024-05-09T02:23:42.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.772+0000 7fa0e5e7cb80 20 client.16842 trim_cache size 0 max 16384 2024-05-09T02:23:42.779 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:23:42.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.773+0000 7fa0d7466700 1 -- 172.21.15.19:0/103080848 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0bc0b2c00 msgr2=0x7fa0bc0b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:42.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.773+0000 7fa0d7466700 1 --2- 172.21.15.19:0/103080848 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0bc0b2c00 0x7fa0bc0b5110 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0112ba0 tx=0x7fa0c0008040 comp rx=0 tx=0).stop 2024-05-09T02:23:42.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.773+0000 7fa0d7466700 1 -- 172.21.15.19:0/103080848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0d010a930 msgr2=0x7fa0d01af9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:42.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.773+0000 7fa0d7466700 1 --2- 172.21.15.19:0/103080848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0d010a930 0x7fa0d01af9b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0057c0 tx=0x7fa0cc0057f0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.773+0000 7fa0d7466700 1 -- 172.21.15.19:0/103080848 shutdown_connections 2024-05-09T02:23:42.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.773+0000 7fa0d7466700 1 --2- 172.21.15.19:0/103080848 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0bc0b2c00 0x7fa0bc0b5110 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.773+0000 7fa0d7466700 1 --2- 172.21.15.19:0/103080848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0d011d930 0x7fa0d01a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.773+0000 7fa0d7466700 1 --2- 172.21.15.19:0/103080848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0d010a930 0x7fa0d01af9b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.773+0000 7fa0d7466700 1 --2- 172.21.15.19:0/103080848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d010b280 0x7fa0d01afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:42.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.773+0000 7fa0d7466700 1 -- 172.21.15.19:0/103080848 >> 172.21.15.19:0/103080848 conn(0x7fa0d006f4d0 msgr2=0x7fa0d011b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:42.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.773+0000 7fa0d7466700 1 -- 172.21.15.19:0/103080848 shutdown_connections 2024-05-09T02:23:42.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:42.773+0000 7fa0d7466700 1 -- 172.21.15.19:0/103080848 wait complete. 2024-05-09T02:23:42.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:42 smithi059 ceph-mon[34781]: pgmap v387: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.9 MiB/s wr, 615 op/s 2024-05-09T02:23:43.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:42 smithi196 ceph-mon[34718]: pgmap v387: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.9 MiB/s wr, 615 op/s 2024-05-09T02:23:43.937 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T02:23:44.341 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:23:44.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:44 smithi019 ceph-mon[30151]: pgmap v388: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.9 MiB/s wr, 507 op/s 2024-05-09T02:23:44.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:44 smithi059 ceph-mon[34781]: pgmap v388: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.9 MiB/s wr, 507 op/s 2024-05-09T02:23:45.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:44 smithi196 ceph-mon[34718]: pgmap v388: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.9 MiB/s wr, 507 op/s 2024-05-09T02:23:45.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:45 smithi019 ceph-mon[30151]: pgmap v389: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 27 MiB/s wr, 1.86k op/s 2024-05-09T02:23:45.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:45 smithi059 ceph-mon[34781]: pgmap v389: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 27 MiB/s wr, 1.86k op/s 2024-05-09T02:23:46.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:45 smithi196 ceph-mon[34718]: pgmap v389: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 27 MiB/s wr, 1.86k op/s 2024-05-09T02:23:46.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.544+0000 7fae2b59e700 1 -- 172.21.15.19:0/3125069012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2c112740 msgr2=0x7fae2c110500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.544+0000 7fae2b59e700 1 --2- 172.21.15.19:0/3125069012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2c112740 0x7fae2c110500 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009990 tx=0x7fae1c009ca0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.546+0000 7fae2b59e700 1 -- 172.21.15.19:0/3125069012 shutdown_connections 2024-05-09T02:23:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.546+0000 7fae2b59e700 1 --2- 172.21.15.19:0/3125069012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae2c110fc0 0x7fae2c10ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.546+0000 7fae2b59e700 1 --2- 172.21.15.19:0/3125069012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2c112740 0x7fae2c110500 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.546+0000 7fae2b59e700 1 --2- 172.21.15.19:0/3125069012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae2c110ad0 0x7fae2c10a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.546+0000 7fae2b59e700 1 -- 172.21.15.19:0/3125069012 >> 172.21.15.19:0/3125069012 conn(0x7fae2c06f260 msgr2=0x7fae2c06f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:46.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.548+0000 7fae2b59e700 1 -- 172.21.15.19:0/3125069012 shutdown_connections 2024-05-09T02:23:46.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.549+0000 7fae2b59e700 1 -- 172.21.15.19:0/3125069012 wait complete. 2024-05-09T02:23:46.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.551+0000 7fae2b59e700 1 Processor -- start 2024-05-09T02:23:46.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.551+0000 7fae2b59e700 1 -- start start 2024-05-09T02:23:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.552+0000 7fae2b59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae2c110ad0 0x7fae2c120b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.552+0000 7fae2b59e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae2c110fc0 0x7fae2c119c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.553+0000 7fae2b59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2c112740 0x7fae2c11a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.553+0000 7fae2b59e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c123160 con 0x7fae2c110ad0 2024-05-09T02:23:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.553+0000 7fae2b59e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c122fe0 con 0x7fae2c112740 2024-05-09T02:23:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.553+0000 7fae2b59e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c1232e0 con 0x7fae2c110fc0 2024-05-09T02:23:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.553+0000 7fae29d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae2c110fc0 0x7fae2c119c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.553+0000 7fae29d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae2c110fc0 0x7fae2c119c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:55242/0 (socket says 172.21.15.19:55242) 2024-05-09T02:23:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.553+0000 7fae29d9b700 1 -- 172.21.15.19:0/3016649534 learned_addr learned my addr 172.21.15.19:0/3016649534 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.553+0000 7fae2ad9d700 1 --2- 172.21.15.19:0/3016649534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2c112740 0x7fae2c11a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.554+0000 7fae2ad9d700 1 -- 172.21.15.19:0/3016649534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae2c110fc0 msgr2=0x7fae2c119c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.554+0000 7fae2ad9d700 1 --2- 172.21.15.19:0/3016649534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae2c110fc0 0x7fae2c119c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.554+0000 7fae2ad9d700 1 -- 172.21.15.19:0/3016649534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae2c110ad0 msgr2=0x7fae2c120b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.554+0000 7fae2ad9d700 1 --2- 172.21.15.19:0/3016649534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae2c110ad0 0x7fae2c120b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.554+0000 7fae2ad9d700 1 -- 172.21.15.19:0/3016649534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2c11a840 con 0x7fae2c112740 2024-05-09T02:23:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.554+0000 7fae2ad9d700 1 --2- 172.21.15.19:0/3016649534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2c112740 0x7fae2c11a1a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae2400be20 tx=0x7fae24009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.554+0000 7fae1b7fe700 1 -- 172.21.15.19:0/3016649534 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24019070 con 0x7fae2c112740 2024-05-09T02:23:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.554+0000 7fae1b7fe700 1 -- 172.21.15.19:0/3016649534 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae2400cc10 con 0x7fae2c112740 2024-05-09T02:23:46.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.554+0000 7fae1b7fe700 1 -- 172.21.15.19:0/3016649534 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24012560 con 0x7fae2c112740 2024-05-09T02:23:46.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.555+0000 7fae2b59e700 1 -- 172.21.15.19:0/3016649534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2c11e660 con 0x7fae2c112740 2024-05-09T02:23:46.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.555+0000 7fae2b59e700 1 -- 172.21.15.19:0/3016649534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae2c11eaf0 con 0x7fae2c112740 2024-05-09T02:23:46.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.562+0000 7fae1b7fe700 1 -- 172.21.15.19:0/3016649534 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fae2401c070 con 0x7fae2c112740 2024-05-09T02:23:46.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.562+0000 7fae3f6f1b80 1 Processor -- start 2024-05-09T02:23:46.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.562+0000 7fae3f6f1b80 1 -- start start 2024-05-09T02:23:46.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.563+0000 7fae3f6f1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dd69160 0x55f64de02640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.563+0000 7fae3f6f1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dc7f4a0 0x55f64de02b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.563+0000 7fae3f6f1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64ddfc800 0x55f64ddfcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.563+0000 7fae29d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dd69160 0x55f64de02640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.563+0000 7fae3f6f1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f64dd1eff0 con 0x55f64dd69160 2024-05-09T02:23:46.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.563+0000 7fae3f6f1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f64dcbd7f0 con 0x55f64dc7f4a0 2024-05-09T02:23:46.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.563+0000 7fae3f6f1b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f64dd1afb0 con 0x55f64ddfc800 2024-05-09T02:23:46.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.563+0000 7fae1b7fe700 1 --2- 172.21.15.19:0/3016649534 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae140b2b80 0x7fae140b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.564+0000 7fae2a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dc7f4a0 0x55f64de02b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.564+0000 7fae29d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dd69160 0x55f64de02640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58996/0 (socket says 172.21.15.19:58996) 2024-05-09T02:23:46.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.564+0000 7fae29d9b700 1 -- 172.21.15.19:0/3573562289 learned_addr learned my addr 172.21.15.19:0/3573562289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:46.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.564+0000 7fae1b7fe700 1 -- 172.21.15.19:0/3016649534 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fae240e1a90 con 0x7fae2c112740 2024-05-09T02:23:46.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.564+0000 7fae2ad9d700 1 --2- 172.21.15.19:0/3016649534 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae140b2b80 0x7fae140b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.564+0000 7fae2a59c700 1 -- 172.21.15.19:0/3573562289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64ddfc800 msgr2=0x55f64ddfcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:46.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.564+0000 7fae2a59c700 1 --2- 172.21.15.19:0/3573562289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64ddfc800 0x55f64ddfcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.564+0000 7fae2a59c700 1 -- 172.21.15.19:0/3573562289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dd69160 msgr2=0x55f64de02640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.564+0000 7fae2a59c700 1 --2- 172.21.15.19:0/3573562289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dd69160 0x55f64de02640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.565+0000 7fae2a59c700 1 -- 172.21.15.19:0/3573562289 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f64dcb5370 con 0x55f64dc7f4a0 2024-05-09T02:23:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.565+0000 7fae29d9b700 1 --2- 172.21.15.19:0/3573562289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dd69160 0x55f64de02640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:23:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.565+0000 7fae2a59c700 1 --2- 172.21.15.19:0/3573562289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dc7f4a0 0x55f64de02b80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fae1c005b40 tx=0x7fae1c003810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff1dd4d3028b55d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.565+0000 7fae18ff9700 1 -- 172.21.15.19:0/3573562289 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c003b00 con 0x55f64dc7f4a0 2024-05-09T02:23:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.565+0000 7fae18ff9700 1 -- 172.21.15.19:0/3573562289 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae1c005580 con 0x55f64dc7f4a0 2024-05-09T02:23:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.565+0000 7fae18ff9700 1 -- 172.21.15.19:0/3573562289 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c00bd10 con 0x55f64dc7f4a0 2024-05-09T02:23:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.567+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3573562289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dc7f4a0 msgr2=0x55f64de02b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.567+0000 7fae2ad9d700 1 --2- 172.21.15.19:0/3016649534 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae140b2b80 0x7fae140b5090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae240a4ed0 tx=0x7fae2400e040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.567+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/3573562289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dc7f4a0 0x55f64de02b80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fae1c005b40 tx=0x7fae1c003810 comp rx=0 tx=0).stop 2024-05-09T02:23:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.567+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3573562289 shutdown_connections 2024-05-09T02:23:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.567+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/3573562289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64ddfc800 0x55f64ddfcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.567+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/3573562289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dd69160 0x55f64de02640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.567+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/3573562289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dc7f4a0 0x55f64de02b80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.567+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3573562289 >> 172.21.15.19:0/3573562289 conn(0x55f64dcdf6a0 msgr2=0x55f64ddf7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:46.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.567+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3573562289 shutdown_connections 2024-05-09T02:23:46.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.568+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3573562289 wait complete. 2024-05-09T02:23:46.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.568+0000 7fae3f6f1b80 1 -- start start 2024-05-09T02:23:46.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.569+0000 7fae3f6f1b80 1 Processor -- start 2024-05-09T02:23:46.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.570+0000 7fae3f6f1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dc7f4a0 0x55f64de06330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.570+0000 7fae3f6f1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64dd69160 0x55f64de06870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.570+0000 7fae29d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dc7f4a0 0x55f64de06330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.570+0000 7fae29d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dc7f4a0 0x55f64de06330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59010/0 (socket says 172.21.15.19:59010) 2024-05-09T02:23:46.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.570+0000 7fae3f6f1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64ddfc800 0x55f64de06db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.570+0000 7fae2a59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64dd69160 0x55f64de06870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.570+0000 7fae2a59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64dd69160 0x55f64de06870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:55262/0 (socket says 172.21.15.19:55262) 2024-05-09T02:23:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.570+0000 7fae29d9b700 1 -- 172.21.15.19:0/3951434687 learned_addr learned my addr 172.21.15.19:0/3951434687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.571+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3951434687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f64db9aba0 con 0x55f64dc7f4a0 2024-05-09T02:23:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.571+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3951434687 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f64dcd9150 con 0x55f64ddfc800 2024-05-09T02:23:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.571+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3951434687 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f64ddfd590 con 0x55f64dd69160 2024-05-09T02:23:46.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.571+0000 7fae3f6f1b80 10 client.0 _ll_register_callbacks cb 0x55f64db906c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:46.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.571+0000 7fae29d9b700 1 --2- 172.21.15.19:0/3951434687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64ddfc800 0x55f64de06db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.571+0000 7fae2a59c700 1 -- 172.21.15.19:0/3951434687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64ddfc800 msgr2=0x55f64de06db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:46.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.571+0000 7fae2a59c700 1 --2- 172.21.15.19:0/3951434687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64ddfc800 0x55f64de06db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.571+0000 7fae2a59c700 1 -- 172.21.15.19:0/3951434687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dc7f4a0 msgr2=0x55f64de06330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:46.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.572+0000 7fae2a59c700 1 --2- 172.21.15.19:0/3951434687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dc7f4a0 0x55f64de06330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.572+0000 7fae2a59c700 1 -- 172.21.15.19:0/3951434687 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f64db9a490 con 0x55f64dd69160 2024-05-09T02:23:46.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.572+0000 7fae29d9b700 1 --2- 172.21.15.19:0/3951434687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64ddfc800 0x55f64de06db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:23:46.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.572+0000 7fae2a59c700 1 --2- 172.21.15.19:0/3951434687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64dd69160 0x55f64de06870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0098e0 tx=0x7fae1c003d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d97dd62042f5f67a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:46.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.573+0000 7fae11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:23:46.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.573+0000 7fae11ffb700 1 -- 172.21.15.19:0/3951434687 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c02f060 con 0x55f64dd69160 2024-05-09T02:23:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.573+0000 7fae11ffb700 1 -- 172.21.15.19:0/3951434687 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fae1c026040 con 0x55f64dd69160 2024-05-09T02:23:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.574+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3951434687 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae1c005580 con 0x55f64dd69160 2024-05-09T02:23:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.574+0000 7fae11ffb700 1 -- 172.21.15.19:0/3951434687 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fae1c0095c0 con 0x55f64dd69160 2024-05-09T02:23:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.575+0000 7fae3f6f1b80 10 client.26614 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.575+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3951434687 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55f64de13c10 con 0x55f64dd69160 2024-05-09T02:23:46.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.575+0000 7fae11ffb700 1 -- 172.21.15.19:0/3951434687 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae1c005580 con 0x55f64dd69160 2024-05-09T02:23:46.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.575+0000 7fae11ffb700 1 -- 172.21.15.19:0/3951434687 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fae1c00fc20 con 0x55f64dd69160 2024-05-09T02:23:46.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.576+0000 7fae3f6f1b80 10 client.26614 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:46.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.576+0000 7fae3f6f1b80 10 client.26614 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:23:46.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.576+0000 7fae3f6f1b80 20 client.26614 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:46.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.576+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/3951434687 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55f64de160e0 0x55f64de18500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.576+0000 7fae3f6f1b80 4 client.26614 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:23:46.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.576+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3951434687 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f64dd2c9d0 con 0x55f64de160e0 2024-05-09T02:23:46.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.577+0000 7fae29d9b700 1 --2- 172.21.15.19:0/3951434687 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55f64de160e0 0x55f64de18500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.578+0000 7fae29d9b700 1 --2- 172.21.15.19:0/3951434687 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55f64de160e0 0x55f64de18500 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=791aa32423309a85 server_cookie=33732c1acedd48e1 in_seq=0 out_seq=0 2024-05-09T02:23:46.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.578+0000 7fae11ffb700 10 client.26614 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:23:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.580+0000 7fae11ffb700 1 -- 172.21.15.19:0/3951434687 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f64dd2c9d0 con 0x55f64de160e0 2024-05-09T02:23:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.580+0000 7fae11ffb700 10 client.26614 handle_command_reply: tid=0 2024-05-09T02:23:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.580+0000 7fae3f6f1b80 1 client.26614 shutdown 2024-05-09T02:23:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.580+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3951434687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64dd69160 msgr2=0x55f64de06870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.580+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/3951434687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64dd69160 0x55f64de06870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0098e0 tx=0x7fae1c003d10 comp rx=0 tx=0).stop 2024-05-09T02:23:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.581+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3951434687 shutdown_connections 2024-05-09T02:23:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.581+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/3951434687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64ddfc800 0x55f64de06db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.581+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/3951434687 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55f64de160e0 0x55f64de18500 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.581+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/3951434687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dc7f4a0 0x55f64de06330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.582+0000 7fae11ffb700 0 client.26614 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:23:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.582+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/3951434687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64dd69160 0x55f64de06870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.582+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3951434687 >> 172.21.15.19:0/3951434687 conn(0x55f64dcdf6a0 msgr2=0x55f64ddf7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.582+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3951434687 shutdown_connections 2024-05-09T02:23:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.582+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3951434687 wait complete. 2024-05-09T02:23:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.582+0000 7fae3f6f1b80 20 client.26614 trim_cache size 0 max 16384 2024-05-09T02:23:46.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.606+0000 7fae3f6f1b80 1 Processor -- start 2024-05-09T02:23:46.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.607+0000 7fae3f6f1b80 1 -- start start 2024-05-09T02:23:46.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.608+0000 7fae3f6f1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dc7f4a0 0x55f64de12500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.608+0000 7fae3f6f1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dd69160 0x55f64de05c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.608+0000 7fae3f6f1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64ddfc800 0x55f64de00710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.608+0000 7fae29d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dc7f4a0 0x55f64de12500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.608+0000 7fae29d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dc7f4a0 0x55f64de12500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59024/0 (socket says 172.21.15.19:59024) 2024-05-09T02:23:46.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.608+0000 7fae29d9b700 1 -- 172.21.15.19:0/3251350421 learned_addr learned my addr 172.21.15.19:0/3251350421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:46.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.608+0000 7fae2a59c700 1 --2- 172.21.15.19:0/3251350421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dd69160 0x55f64de05c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.608+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3251350421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c1c6010 con 0x55f64dc7f4a0 2024-05-09T02:23:46.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.608+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3251350421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c1c7b50 con 0x55f64dd69160 2024-05-09T02:23:46.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.608+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3251350421 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c1c66e0 con 0x55f64ddfc800 2024-05-09T02:23:46.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.608+0000 7fae29d9b700 1 --2- 172.21.15.19:0/3251350421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64ddfc800 0x55f64de00710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.609+0000 7fae29d9b700 1 -- 172.21.15.19:0/3251350421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64ddfc800 msgr2=0x55f64de00710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:46.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.609+0000 7fae29d9b700 1 --2- 172.21.15.19:0/3251350421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64ddfc800 0x55f64de00710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.609+0000 7fae29d9b700 1 -- 172.21.15.19:0/3251350421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dd69160 msgr2=0x55f64de05c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:46.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.609+0000 7fae29d9b700 1 --2- 172.21.15.19:0/3251350421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dd69160 0x55f64de05c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.609+0000 7fae29d9b700 1 -- 172.21.15.19:0/3251350421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f64ddf2410 con 0x55f64dc7f4a0 2024-05-09T02:23:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.609+0000 7fae29d9b700 1 --2- 172.21.15.19:0/3251350421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dc7f4a0 0x55f64de12500 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fae2000adb0 tx=0x7fae2000aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37b9a8981d2c3b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.609+0000 7fae13fff700 1 -- 172.21.15.19:0/3251350421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20004a30 con 0x55f64dc7f4a0 2024-05-09T02:23:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.609+0000 7fae13fff700 1 -- 172.21.15.19:0/3251350421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae2001c040 con 0x55f64dc7f4a0 2024-05-09T02:23:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.609+0000 7fae13fff700 1 -- 172.21.15.19:0/3251350421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20012890 con 0x55f64dc7f4a0 2024-05-09T02:23:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.610+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3251350421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dc7f4a0 msgr2=0x55f64de12500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.610+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/3251350421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dc7f4a0 0x55f64de12500 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fae2000adb0 tx=0x7fae2000aa70 comp rx=0 tx=0).stop 2024-05-09T02:23:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.610+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3251350421 shutdown_connections 2024-05-09T02:23:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.610+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/3251350421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64ddfc800 0x55f64de00710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.610+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/3251350421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64dc7f4a0 0x55f64de12500 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.610+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/3251350421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dd69160 0x55f64de05c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.610+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3251350421 >> 172.21.15.19:0/3251350421 conn(0x55f64dcdf6a0 msgr2=0x55f64de00000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.611+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3251350421 shutdown_connections 2024-05-09T02:23:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.611+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/3251350421 wait complete. 2024-05-09T02:23:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.611+0000 7fae3f6f1b80 1 -- start start 2024-05-09T02:23:46.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.611+0000 7fae3f6f1b80 1 Processor -- start 2024-05-09T02:23:46.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.612+0000 7fae3f6f1b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64dc7f4a0 0x55f64de187c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.612+0000 7fae3f6f1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dd69160 0x55f64de18d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.612+0000 7fae3f6f1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ddfc800 0x55f64de7e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.612+0000 7fae3f6f1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c1c7480 con 0x55f64ddfc800 2024-05-09T02:23:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.612+0000 7fae29d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64dc7f4a0 0x55f64de187c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.612+0000 7fae29d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64dc7f4a0 0x55f64de187c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:55286/0 (socket says 172.21.15.19:55286) 2024-05-09T02:23:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.612+0000 7fae29d9b700 1 -- 172.21.15.19:0/776096428 learned_addr learned my addr 172.21.15.19:0/776096428 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.612+0000 7fae2a59c700 1 --2- 172.21.15.19:0/776096428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dd69160 0x55f64de18d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.612+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/776096428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c1c6db0 con 0x55f64dd69160 2024-05-09T02:23:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.612+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/776096428 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c1c3fd0 con 0x55f64dc7f4a0 2024-05-09T02:23:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.612+0000 7fae29d9b700 1 --2- 172.21.15.19:0/776096428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ddfc800 0x55f64de7e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.612+0000 7fae3f6f1b80 10 client.0 _ll_register_callbacks cb 0x55f64db906c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.613+0000 7fae2a59c700 1 -- 172.21.15.19:0/776096428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64dc7f4a0 msgr2=0x55f64de187c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.613+0000 7fae2a59c700 1 --2- 172.21.15.19:0/776096428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64dc7f4a0 0x55f64de187c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.613+0000 7fae2a59c700 1 -- 172.21.15.19:0/776096428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ddfc800 msgr2=0x55f64de7e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.613+0000 7fae2a59c700 1 --2- 172.21.15.19:0/776096428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ddfc800 0x55f64de7e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.613+0000 7fae2a59c700 1 -- 172.21.15.19:0/776096428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f64de7eb60 con 0x55f64dd69160 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.613+0000 7fae29d9b700 1 --2- 172.21.15.19:0/776096428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64dc7f4a0 0x55f64de187c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.613+0000 7fae2a59c700 1 --2- 172.21.15.19:0/776096428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dd69160 0x55f64de18d00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009990 tx=0x7fae1c005ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fefb270e39d1f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.613+0000 7fae11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.613+0000 7fae11ffb700 1 -- 172.21.15.19:0/776096428 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c02f030 con 0x55f64dd69160 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.613+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/776096428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae2001c040 con 0x55f64dd69160 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.614+0000 7fae11ffb700 1 -- 172.21.15.19:0/776096428 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fae1c026040 con 0x55f64dd69160 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.614+0000 7fae11ffb700 1 -- 172.21.15.19:0/776096428 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fae1c00bb60 con 0x55f64dd69160 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.614+0000 7fae3f6f1b80 10 client.26585 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.614+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/776096428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55f64de7fc70 con 0x55f64dd69160 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.615+0000 7fae11ffb700 1 -- 172.21.15.19:0/776096428 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae2001c040 con 0x55f64dd69160 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.615+0000 7fae11ffb700 1 -- 172.21.15.19:0/776096428 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fae1c02a480 con 0x55f64dd69160 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.615+0000 7fae3f6f1b80 10 client.26585 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.615+0000 7fae3f6f1b80 10 client.26585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.615+0000 7fae3f6f1b80 20 client.26585 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.615+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/776096428 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55f64de160e0 0x55f64de822a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.615+0000 7fae3f6f1b80 4 client.26585 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:23:46.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.615+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/776096428 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f64dd26a70 con 0x55f64de160e0 2024-05-09T02:23:46.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.615+0000 7fae29d9b700 1 --2- 172.21.15.19:0/776096428 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55f64de160e0 0x55f64de822a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:46.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.616+0000 7fae11ffb700 10 client.26585 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:23:46.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.616+0000 7fae29d9b700 1 --2- 172.21.15.19:0/776096428 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55f64de160e0 0x55f64de822a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3b0267b94a01e4c server_cookie=ed2768ed381960e7 in_seq=0 out_seq=0 2024-05-09T02:23:46.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.616+0000 7fae11ffb700 1 -- 172.21.15.19:0/776096428 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f64dd26a70 con 0x55f64de160e0 2024-05-09T02:23:46.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.616+0000 7fae11ffb700 10 client.26585 handle_command_reply: tid=0 2024-05-09T02:23:46.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.616+0000 7fae3f6f1b80 1 client.26585 shutdown 2024-05-09T02:23:46.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.616+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/776096428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dd69160 msgr2=0x55f64de18d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:46.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.616+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/776096428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dd69160 0x55f64de18d00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009990 tx=0x7fae1c005ee0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.616+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/776096428 shutdown_connections 2024-05-09T02:23:46.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.616+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/776096428 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55f64de160e0 0x55f64de822a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.616+0000 7fae11ffb700 0 client.26585 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:23:46.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.616+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/776096428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f64ddfc800 0x55f64de7e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.616+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/776096428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f64dc7f4a0 0x55f64de187c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.616+0000 7fae3f6f1b80 1 --2- 172.21.15.19:0/776096428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f64dd69160 0x55f64de18d00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.616+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/776096428 >> 172.21.15.19:0/776096428 conn(0x55f64dcdf6a0 msgr2=0x55f64de01c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:46.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.617+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/776096428 shutdown_connections 2024-05-09T02:23:46.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.617+0000 7fae3f6f1b80 1 -- 172.21.15.19:0/776096428 wait complete. 2024-05-09T02:23:46.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.617+0000 7fae3f6f1b80 20 client.26585 trim_cache size 0 max 16384 2024-05-09T02:23:46.623 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:23:46.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.617+0000 7fae2b59e700 1 -- 172.21.15.19:0/3016649534 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae140b2b80 msgr2=0x7fae140b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:46.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.617+0000 7fae2b59e700 1 --2- 172.21.15.19:0/3016649534 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae140b2b80 0x7fae140b5090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae240a4ed0 tx=0x7fae2400e040 comp rx=0 tx=0).stop 2024-05-09T02:23:46.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.617+0000 7fae2b59e700 1 -- 172.21.15.19:0/3016649534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2c112740 msgr2=0x7fae2c11a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:46.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.617+0000 7fae2b59e700 1 --2- 172.21.15.19:0/3016649534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2c112740 0x7fae2c11a1a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae2400be20 tx=0x7fae24009ed0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.618+0000 7fae2b59e700 1 -- 172.21.15.19:0/3016649534 shutdown_connections 2024-05-09T02:23:46.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.618+0000 7fae2b59e700 1 --2- 172.21.15.19:0/3016649534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2c112740 0x7fae2c11a1a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.618+0000 7fae2b59e700 1 --2- 172.21.15.19:0/3016649534 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae140b2b80 0x7fae140b5090 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.618+0000 7fae2b59e700 1 --2- 172.21.15.19:0/3016649534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae2c110ad0 0x7fae2c120b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.618+0000 7fae2b59e700 1 --2- 172.21.15.19:0/3016649534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae2c110fc0 0x7fae2c119c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:46.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.618+0000 7fae2b59e700 1 -- 172.21.15.19:0/3016649534 >> 172.21.15.19:0/3016649534 conn(0x7fae2c06f260 msgr2=0x7fae2c10c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:46.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.618+0000 7fae2b59e700 1 -- 172.21.15.19:0/3016649534 shutdown_connections 2024-05-09T02:23:46.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:46.618+0000 7fae2b59e700 1 -- 172.21.15.19:0/3016649534 wait complete. 2024-05-09T02:23:47.405 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T02:23:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 725.0 s, 1235.4 tps, lat 81.262 ms stddev 104.094 2024-05-09T02:23:47.813 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:23:48.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:48 smithi019 ceph-mon[30151]: pgmap v390: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 25 MiB/s wr, 1.61k op/s 2024-05-09T02:23:48.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:48 smithi059 ceph-mon[34781]: pgmap v390: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 25 MiB/s wr, 1.61k op/s 2024-05-09T02:23:48.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:48 smithi196 ceph-mon[34718]: pgmap v390: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 25 MiB/s wr, 1.61k op/s 2024-05-09T02:23:49.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.915+0000 7f03a959a700 1 -- 172.21.15.19:0/4003676692 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a40092e0 con 0x7f03ac109870 2024-05-09T02:23:49.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.917+0000 7f03b0ba4700 1 -- 172.21.15.19:0/4003676692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ac109870 msgr2=0x7f03ac1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:49.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.917+0000 7f03b0ba4700 1 --2- 172.21.15.19:0/4003676692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ac109870 0x7f03ac1111d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f03a400b0d0 tx=0x7f03a400b3e0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.917+0000 7f03b0ba4700 1 -- 172.21.15.19:0/4003676692 shutdown_connections 2024-05-09T02:23:49.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.917+0000 7f03b0ba4700 1 --2- 172.21.15.19:0/4003676692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ac109870 0x7f03ac1111d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.917+0000 7f03b0ba4700 1 --2- 172.21.15.19:0/4003676692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac1045f0 0x7f03ac104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.917+0000 7f03b0ba4700 1 --2- 172.21.15.19:0/4003676692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03ac108f20 0x7f03ac109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.917+0000 7f03b0ba4700 1 -- 172.21.15.19:0/4003676692 >> 172.21.15.19:0/4003676692 conn(0x7f03ac100040 msgr2=0x7f03ac102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.917+0000 7f03b0ba4700 1 -- 172.21.15.19:0/4003676692 shutdown_connections 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.918+0000 7f03b0ba4700 1 -- 172.21.15.19:0/4003676692 wait complete. 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.919+0000 7f03b0ba4700 1 Processor -- start 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.919+0000 7f03b0ba4700 1 -- start start 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.919+0000 7f03b0ba4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac1045f0 0x7f03ac077660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.919+0000 7f03b0ba4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03ac108f20 0x7f03ac075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.919+0000 7f03b0ba4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ac109870 0x7f03ac0760a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.919+0000 7f03aa59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac1045f0 0x7f03ac077660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.919+0000 7f03b0ba4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1144a0 con 0x7f03ac109870 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.919+0000 7f03b0ba4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac114320 con 0x7f03ac1045f0 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.919+0000 7f03b0ba4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac114620 con 0x7f03ac108f20 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.919+0000 7f03aa59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac1045f0 0x7f03ac077660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56740/0 (socket says 172.21.15.19:56740) 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.919+0000 7f03aa59c700 1 -- 172.21.15.19:0/1195265449 learned_addr learned my addr 172.21.15.19:0/1195265449 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.919+0000 7f03a9d9b700 1 --2- 172.21.15.19:0/1195265449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03ac108f20 0x7f03ac075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.920+0000 7f03aad9d700 1 --2- 172.21.15.19:0/1195265449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ac109870 0x7f03ac0760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.920+0000 7f03a9d9b700 1 -- 172.21.15.19:0/1195265449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac1045f0 msgr2=0x7f03ac077660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:49.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.920+0000 7f03a9d9b700 1 --2- 172.21.15.19:0/1195265449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac1045f0 0x7f03ac077660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.920+0000 7f03a9d9b700 1 -- 172.21.15.19:0/1195265449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ac109870 msgr2=0x7f03ac0760a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:49.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.920+0000 7f03a9d9b700 1 --2- 172.21.15.19:0/1195265449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ac109870 0x7f03ac0760a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.920+0000 7f03a9d9b700 1 -- 172.21.15.19:0/1195265449 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac0768d0 con 0x7f03ac108f20 2024-05-09T02:23:49.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.920+0000 7f03a9d9b700 1 --2- 172.21.15.19:0/1195265449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03ac108f20 0x7f03ac075ac0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f03a000e9c0 tx=0x7f03a000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:49.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.920+0000 7f039b7fe700 1 -- 172.21.15.19:0/1195265449 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a000cbd0 con 0x7f03ac108f20 2024-05-09T02:23:49.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.920+0000 7f03b0ba4700 1 -- 172.21.15.19:0/1195265449 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac1ad860 con 0x7f03ac108f20 2024-05-09T02:23:49.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.920+0000 7f039b7fe700 1 -- 172.21.15.19:0/1195265449 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03a000cd70 con 0x7f03ac108f20 2024-05-09T02:23:49.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.920+0000 7f039b7fe700 1 -- 172.21.15.19:0/1195265449 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0018680 con 0x7f03ac108f20 2024-05-09T02:23:49.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.920+0000 7f03b0ba4700 1 -- 172.21.15.19:0/1195265449 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac1add70 con 0x7f03ac108f20 2024-05-09T02:23:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.925+0000 7f039b7fe700 1 -- 172.21.15.19:0/1195265449 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f03a0018820 con 0x7f03ac108f20 2024-05-09T02:23:49.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.927+0000 7f039b7fe700 1 --2- 172.21.15.19:0/1195265449 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f03940b29f0 0x7f03940b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:49.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.927+0000 7f039b7fe700 1 -- 172.21.15.19:0/1195265449 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f03a00e27f0 con 0x7f03ac108f20 2024-05-09T02:23:49.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.927+0000 7f03bf5bab80 1 Processor -- start 2024-05-09T02:23:49.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.928+0000 7f03aa59c700 1 --2- 172.21.15.19:0/1195265449 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f03940b29f0 0x7f03940b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.928+0000 7f03bf5bab80 1 -- start start 2024-05-09T02:23:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.928+0000 7f03aa59c700 1 --2- 172.21.15.19:0/1195265449 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f03940b29f0 0x7f03940b4f00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f039c009d20 tx=0x7f039c009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.928+0000 7f03bf5bab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804560d00 0x559804669240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.928+0000 7f03bf5bab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x559804669780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.928+0000 7f03bf5bab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 0x559804663810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.929+0000 7f03aa59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804560d00 0x559804669240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.929+0000 7f03a9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x559804669780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.929+0000 7f03bf5bab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5598045860a0 con 0x5598044e64a0 2024-05-09T02:23:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.929+0000 7f03aa59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804560d00 0x559804669240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:51706/0 (socket says 172.21.15.19:51706) 2024-05-09T02:23:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.929+0000 7f03aa59c700 1 -- 172.21.15.19:0/2984835199 learned_addr learned my addr 172.21.15.19:0/2984835199 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.929+0000 7f03bf5bab80 1 -- 172.21.15.19:0/2984835199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55980452b9a0 con 0x559804663400 2024-05-09T02:23:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.929+0000 7f03bf5bab80 1 -- 172.21.15.19:0/2984835199 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559804582060 con 0x559804560d00 2024-05-09T02:23:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.929+0000 7f03aad9d700 1 --2- 172.21.15.19:0/2984835199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 0x559804663810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.929+0000 7f03aad9d700 1 -- 172.21.15.19:0/2984835199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804560d00 msgr2=0x559804669240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.929+0000 7f03aad9d700 1 --2- 172.21.15.19:0/2984835199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804560d00 0x559804669240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.929+0000 7f03aad9d700 1 -- 172.21.15.19:0/2984835199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 msgr2=0x559804669780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.929+0000 7f03aad9d700 1 --2- 172.21.15.19:0/2984835199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x559804669780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.929+0000 7f03aad9d700 1 -- 172.21.15.19:0/2984835199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980451c500 con 0x559804663400 2024-05-09T02:23:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.929+0000 7f03aad9d700 1 --2- 172.21.15.19:0/2984835199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 0x559804663810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009fd0 tx=0x7f03a4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e12aa141e6dada88 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.929+0000 7f0398ff9700 1 -- 172.21.15.19:0/2984835199 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a40092e0 con 0x559804663400 2024-05-09T02:23:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.930+0000 7f0398ff9700 1 -- 172.21.15.19:0/2984835199 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03a4003a00 con 0x559804663400 2024-05-09T02:23:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.930+0000 7f0398ff9700 1 -- 172.21.15.19:0/2984835199 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4007c00 con 0x559804663400 2024-05-09T02:23:49.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.930+0000 7f03bf5bab80 1 -- 172.21.15.19:0/2984835199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 msgr2=0x559804663810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:49.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.930+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/2984835199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 0x559804663810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009fd0 tx=0x7f03a4009780 comp rx=0 tx=0).stop 2024-05-09T02:23:49.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.931+0000 7f03bf5bab80 1 -- 172.21.15.19:0/2984835199 shutdown_connections 2024-05-09T02:23:49.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.931+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/2984835199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 0x559804663810 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.931+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/2984835199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804560d00 0x559804669240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.931+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/2984835199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x559804669780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.931+0000 7f03bf5bab80 1 -- 172.21.15.19:0/2984835199 >> 172.21.15.19:0/2984835199 conn(0x5598044e9f70 msgr2=0x55980465e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:49.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.932+0000 7f03bf5bab80 1 -- 172.21.15.19:0/2984835199 shutdown_connections 2024-05-09T02:23:49.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.932+0000 7f03bf5bab80 1 -- 172.21.15.19:0/2984835199 wait complete. 2024-05-09T02:23:49.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.933+0000 7f03bf5bab80 1 -- start start 2024-05-09T02:23:49.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.933+0000 7f03bf5bab80 1 Processor -- start 2024-05-09T02:23:49.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.934+0000 7f03bf5bab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x55980466d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:49.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.934+0000 7f03bf5bab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804560d00 0x55980466d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:49.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.934+0000 7f03aa59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x55980466d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:49.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.934+0000 7f03aa59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x55980466d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57052/0 (socket says 172.21.15.19:57052) 2024-05-09T02:23:49.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.934+0000 7f03aa59c700 1 -- 172.21.15.19:0/4236244694 learned_addr learned my addr 172.21.15.19:0/4236244694 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:49.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.934+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804663400 0x55980466daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:49.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.934+0000 7f03bf5bab80 1 -- 172.21.15.19:0/4236244694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559804664100 con 0x5598044e64a0 2024-05-09T02:23:49.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.934+0000 7f03bf5bab80 1 -- 172.21.15.19:0/4236244694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559804520b20 con 0x559804560d00 2024-05-09T02:23:49.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.934+0000 7f03bf5bab80 1 -- 172.21.15.19:0/4236244694 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5598046791d0 con 0x559804663400 2024-05-09T02:23:49.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.934+0000 7f03a9d9b700 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804560d00 0x55980466d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:49.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.934+0000 7f03aad9d700 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804663400 0x55980466daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:49.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.934+0000 7f03bf5bab80 10 client.0 _ll_register_callbacks cb 0x5598043f76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:49.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.935+0000 7f03aa59c700 1 -- 172.21.15.19:0/4236244694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804663400 msgr2=0x55980466daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:49.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.935+0000 7f03aa59c700 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804663400 0x55980466daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.935+0000 7f03aa59c700 1 -- 172.21.15.19:0/4236244694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804560d00 msgr2=0x55980466d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:49.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.935+0000 7f03aa59c700 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804560d00 0x55980466d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.935+0000 7f03aa59c700 1 -- 172.21.15.19:0/4236244694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559804401490 con 0x5598044e64a0 2024-05-09T02:23:49.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.935+0000 7f03aad9d700 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804663400 0x55980466daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:23:49.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.935+0000 7f03a9d9b700 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804560d00 0x55980466d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:23:49.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.936+0000 7f03aa59c700 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x55980466d020 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f039c01ece0 tx=0x7f039c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9268ab46e1e9393f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:49.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.936+0000 7f0391ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:23:49.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.936+0000 7f0391ffb700 1 -- 172.21.15.19:0/4236244694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c01e260 con 0x5598044e64a0 2024-05-09T02:23:49.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.936+0000 7f0391ffb700 1 -- 172.21.15.19:0/4236244694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f039c00e8f0 con 0x5598044e64a0 2024-05-09T02:23:49.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.936+0000 7f03bf5bab80 1 -- 172.21.15.19:0/4236244694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03a4003a00 con 0x5598044e64a0 2024-05-09T02:23:49.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.936+0000 7f0391ffb700 1 -- 172.21.15.19:0/4236244694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f039c00eaf0 con 0x5598044e64a0 2024-05-09T02:23:49.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.936+0000 7f03bf5bab80 10 client.16866 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:49.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.936+0000 7f03bf5bab80 1 -- 172.21.15.19:0/4236244694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55980467a980 con 0x5598044e64a0 2024-05-09T02:23:49.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.937+0000 7f0391ffb700 1 -- 172.21.15.19:0/4236244694 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03a4003a00 con 0x5598044e64a0 2024-05-09T02:23:49.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.937+0000 7f0391ffb700 1 -- 172.21.15.19:0/4236244694 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f039c025070 con 0x5598044e64a0 2024-05-09T02:23:49.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.937+0000 7f03bf5bab80 10 client.16866 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:49.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.937+0000 7f03bf5bab80 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:23:49.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.937+0000 7f03bf5bab80 20 client.16866 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:49.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.937+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55980467cf60 0x55980467f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:49.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.937+0000 7f03bf5bab80 4 client.16866 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:23:49.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.937+0000 7f03bf5bab80 1 -- 172.21.15.19:0/4236244694 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559804593a80 con 0x55980467cf60 2024-05-09T02:23:49.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.937+0000 7f03a9d9b700 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55980467cf60 0x55980467f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:49.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.938+0000 7f0391ffb700 10 client.16866 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:49.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.938+0000 7f03a9d9b700 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55980467cf60 0x55980467f3a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc7cac37c82d3eb8 server_cookie=a86f57beaa189287 in_seq=0 out_seq=0 2024-05-09T02:23:49.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.938+0000 7f0391ffb700 1 -- 172.21.15.19:0/4236244694 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f03a008ac70 con 0x55980467cf60 2024-05-09T02:23:49.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.938+0000 7f0391ffb700 10 client.16866 handle_command_reply: tid=0 2024-05-09T02:23:49.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.939+0000 7f03bf5bab80 1 client.16866 shutdown 2024-05-09T02:23:49.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.939+0000 7f03bf5bab80 1 -- 172.21.15.19:0/4236244694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 msgr2=0x55980466d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:49.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.939+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x55980466d020 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f039c01ece0 tx=0x7f039c0047c0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.940+0000 7f03bf5bab80 1 -- 172.21.15.19:0/4236244694 shutdown_connections 2024-05-09T02:23:49.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.940+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55980467cf60 0x55980467f3a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.940+0000 7f0391ffb700 0 client.16866 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:49.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.940+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804663400 0x55980466daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.940+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x55980466d020 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.940+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/4236244694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804560d00 0x55980466d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.940+0000 7f03bf5bab80 1 -- 172.21.15.19:0/4236244694 >> 172.21.15.19:0/4236244694 conn(0x5598044e9f70 msgr2=0x559804657ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:49.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.940+0000 7f03bf5bab80 1 -- 172.21.15.19:0/4236244694 shutdown_connections 2024-05-09T02:23:49.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.940+0000 7f03bf5bab80 1 -- 172.21.15.19:0/4236244694 wait complete. 2024-05-09T02:23:49.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.941+0000 7f03bf5bab80 20 client.16866 trim_cache size 0 max 16384 2024-05-09T02:23:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.965+0000 7f03bf5bab80 1 Processor -- start 2024-05-09T02:23:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.969+0000 7f03bf5bab80 1 -- start start 2024-05-09T02:23:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.969+0000 7f03bf5bab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x55980466f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.969+0000 7f03bf5bab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804560d00 0x55980466c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.969+0000 7f03bf5bab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 0x559804684b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.969+0000 7f03bf5bab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1b5420 con 0x5598044e64a0 2024-05-09T02:23:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.969+0000 7f03bf5bab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1b6f60 con 0x559804663400 2024-05-09T02:23:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.969+0000 7f03bf5bab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1b5af0 con 0x559804560d00 2024-05-09T02:23:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.973+0000 7f03aa59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x55980466f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.973+0000 7f03aa59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x55980466f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57060/0 (socket says 172.21.15.19:57060) 2024-05-09T02:23:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.973+0000 7f03aa59c700 1 -- 172.21.15.19:0/1803040323 learned_addr learned my addr 172.21.15.19:0/1803040323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.974+0000 7f03aad9d700 1 --2- 172.21.15.19:0/1803040323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 0x559804684b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.976+0000 7f03a9d9b700 1 --2- 172.21.15.19:0/1803040323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804560d00 0x55980466c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:49.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.980+0000 7f03aa59c700 1 -- 172.21.15.19:0/1803040323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804560d00 msgr2=0x55980466c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:49.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.980+0000 7f03aa59c700 1 --2- 172.21.15.19:0/1803040323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804560d00 0x55980466c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.980+0000 7f03aa59c700 1 -- 172.21.15.19:0/1803040323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 msgr2=0x559804684b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:49.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.980+0000 7f03aa59c700 1 --2- 172.21.15.19:0/1803040323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 0x559804684b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.980+0000 7f03aa59c700 1 -- 172.21.15.19:0/1803040323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980466cdf0 con 0x5598044e64a0 2024-05-09T02:23:49.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.980+0000 7f03a9d9b700 1 --2- 172.21.15.19:0/1803040323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804560d00 0x55980466c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:23:49.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.980+0000 7f03aa59c700 1 --2- 172.21.15.19:0/1803040323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x55980466f380 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f039c016040 tx=0x7f039c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123b99aec4df4df1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:49.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.981+0000 7f0393fff700 1 -- 172.21.15.19:0/1803040323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c01e260 con 0x5598044e64a0 2024-05-09T02:23:49.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.981+0000 7f0393fff700 1 -- 172.21.15.19:0/1803040323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f039c036070 con 0x5598044e64a0 2024-05-09T02:23:49.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.981+0000 7f0393fff700 1 -- 172.21.15.19:0/1803040323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c031870 con 0x5598044e64a0 2024-05-09T02:23:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.984+0000 7f03bf5bab80 1 -- 172.21.15.19:0/1803040323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 msgr2=0x55980466f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.984+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/1803040323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x55980466f380 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f039c016040 tx=0x7f039c01ec70 comp rx=0 tx=0).stop 2024-05-09T02:23:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.990+0000 7f03bf5bab80 1 -- 172.21.15.19:0/1803040323 shutdown_connections 2024-05-09T02:23:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.990+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/1803040323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 0x559804684b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.990+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/1803040323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598044e64a0 0x55980466f380 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.990+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/1803040323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559804560d00 0x55980466c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.990+0000 7f03bf5bab80 1 -- 172.21.15.19:0/1803040323 >> 172.21.15.19:0/1803040323 conn(0x5598044e9f70 msgr2=0x55980466b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.997+0000 7f03bf5bab80 1 -- 172.21.15.19:0/1803040323 shutdown_connections 2024-05-09T02:23:50.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.997+0000 7f03bf5bab80 1 -- 172.21.15.19:0/1803040323 wait complete. 2024-05-09T02:23:50.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.998+0000 7f03bf5bab80 1 -- start start 2024-05-09T02:23:50.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:49.998+0000 7f03bf5bab80 1 Processor -- start 2024-05-09T02:23:50.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.000+0000 7f03bf5bab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598044e64a0 0x559804675130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:50.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.000+0000 7f03bf5bab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559804560d00 0x559804675670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:50.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.000+0000 7f03bf5bab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 0x5598046e26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:50.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.000+0000 7f03bf5bab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1b6890 con 0x559804560d00 2024-05-09T02:23:50.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.000+0000 7f03bf5bab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1b61c0 con 0x559804663400 2024-05-09T02:23:50.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.000+0000 7f03bf5bab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1b33e0 con 0x5598044e64a0 2024-05-09T02:23:50.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.000+0000 7f03bf5bab80 10 client.0 _ll_register_callbacks cb 0x5598043f76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:50.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.001+0000 7f03a9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559804560d00 0x559804675670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:50.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.001+0000 7f03a9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559804560d00 0x559804675670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57070/0 (socket says 172.21.15.19:57070) 2024-05-09T02:23:50.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.001+0000 7f03a9d9b700 1 -- 172.21.15.19:0/1354744048 learned_addr learned my addr 172.21.15.19:0/1354744048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:50.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.001+0000 7f03aad9d700 1 --2- 172.21.15.19:0/1354744048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 0x5598046e26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:50.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.001+0000 7f03aa59c700 1 --2- 172.21.15.19:0/1354744048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598044e64a0 0x559804675130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:50.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.001+0000 7f03a9d9b700 1 -- 172.21.15.19:0/1354744048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598044e64a0 msgr2=0x559804675130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:50.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.001+0000 7f03a9d9b700 1 --2- 172.21.15.19:0/1354744048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598044e64a0 0x559804675130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:50.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.001+0000 7f03a9d9b700 1 -- 172.21.15.19:0/1354744048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 msgr2=0x5598046e26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:50.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.001+0000 7f03a9d9b700 1 --2- 172.21.15.19:0/1354744048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 0x5598046e26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:50.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.001+0000 7f03a9d9b700 1 -- 172.21.15.19:0/1354744048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598046e2c30 con 0x559804560d00 2024-05-09T02:23:50.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.002+0000 7f03a9d9b700 1 --2- 172.21.15.19:0/1354744048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559804560d00 0x559804675670 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f03a00e0510 tx=0x7f03a000efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebf048b428d319d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:50.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.002+0000 7f0391ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:23:50.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.002+0000 7f0391ffb700 1 -- 172.21.15.19:0/1354744048 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a00b2240 con 0x559804560d00 2024-05-09T02:23:50.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.003+0000 7f0391ffb700 1 -- 172.21.15.19:0/1354744048 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f03a00a6090 con 0x559804560d00 2024-05-09T02:23:50.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.003+0000 7f03bf5bab80 1 -- 172.21.15.19:0/1354744048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f039c036070 con 0x559804560d00 2024-05-09T02:23:50.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.003+0000 7f0391ffb700 1 -- 172.21.15.19:0/1354744048 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f03a00a95b0 con 0x559804560d00 2024-05-09T02:23:50.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.003+0000 7f03bf5bab80 10 client.16872 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:50.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.003+0000 7f03bf5bab80 1 -- 172.21.15.19:0/1354744048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55980466ea50 con 0x559804560d00 2024-05-09T02:23:50.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.005+0000 7f0391ffb700 1 -- 172.21.15.19:0/1354744048 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f039c036070 con 0x559804560d00 2024-05-09T02:23:50.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.005+0000 7f0391ffb700 1 -- 172.21.15.19:0/1354744048 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f03a00b2470 con 0x559804560d00 2024-05-09T02:23:50.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.006+0000 7f03bf5bab80 10 client.16872 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:50.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.006+0000 7f03bf5bab80 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:23:50.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.006+0000 7f03bf5bab80 20 client.16872 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:50.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.006+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/1354744048 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55980467cf60 0x5598046fa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:50.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.006+0000 7f03bf5bab80 4 client.16872 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:23:50.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.006+0000 7f03bf5bab80 1 -- 172.21.15.19:0/1354744048 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55980458db20 con 0x55980467cf60 2024-05-09T02:23:50.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.007+0000 7f03aa59c700 1 --2- 172.21.15.19:0/1354744048 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55980467cf60 0x5598046fa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:50.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.009+0000 7f03aa59c700 1 --2- 172.21.15.19:0/1354744048 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55980467cf60 0x5598046fa730 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf8a87a8e17a173a server_cookie=80a6c64f87b250fd in_seq=0 out_seq=0 2024-05-09T02:23:50.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.009+0000 7f0391ffb700 10 client.16872 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:50.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.009+0000 7f0391ffb700 1 -- 172.21.15.19:0/1354744048 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55980458db20 con 0x55980467cf60 2024-05-09T02:23:50.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.010+0000 7f0391ffb700 10 client.16872 handle_command_reply: tid=0 2024-05-09T02:23:50.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.010+0000 7f03bf5bab80 1 client.16872 shutdown 2024-05-09T02:23:50.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.015+0000 7f03bf5bab80 1 -- 172.21.15.19:0/1354744048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559804560d00 msgr2=0x559804675670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:50.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.015+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/1354744048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559804560d00 0x559804675670 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f03a00e0510 tx=0x7f03a000efd0 comp rx=0 tx=0).stop 2024-05-09T02:23:50.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.018+0000 7f03bf5bab80 1 -- 172.21.15.19:0/1354744048 shutdown_connections 2024-05-09T02:23:50.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.018+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/1354744048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559804663400 0x5598046e26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:50.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.018+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/1354744048 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55980467cf60 0x5598046fa730 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:50.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.018+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/1354744048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598044e64a0 0x559804675130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:50.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.018+0000 7f03bf5bab80 1 --2- 172.21.15.19:0/1354744048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559804560d00 0x559804675670 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:50.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.018+0000 7f0391ffb700 0 client.16872 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:50.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.018+0000 7f03bf5bab80 1 -- 172.21.15.19:0/1354744048 >> 172.21.15.19:0/1354744048 conn(0x5598044e9f70 msgr2=0x55980466b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:50.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.019+0000 7f03bf5bab80 1 -- 172.21.15.19:0/1354744048 shutdown_connections 2024-05-09T02:23:50.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.019+0000 7f03bf5bab80 1 -- 172.21.15.19:0/1354744048 wait complete. 2024-05-09T02:23:50.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.020+0000 7f03bf5bab80 20 client.16872 trim_cache size 0 max 16384 2024-05-09T02:23:50.023 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:23:50.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.021+0000 7f03b0ba4700 1 -- 172.21.15.19:0/1195265449 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f03940b29f0 msgr2=0x7f03940b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:50.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.021+0000 7f03b0ba4700 1 --2- 172.21.15.19:0/1195265449 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f03940b29f0 0x7f03940b4f00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f039c009d20 tx=0x7f039c009260 comp rx=0 tx=0).stop 2024-05-09T02:23:50.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.021+0000 7f03b0ba4700 1 -- 172.21.15.19:0/1195265449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03ac108f20 msgr2=0x7f03ac075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:50.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.021+0000 7f03b0ba4700 1 --2- 172.21.15.19:0/1195265449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03ac108f20 0x7f03ac075ac0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f03a000e9c0 tx=0x7f03a000ecd0 comp rx=0 tx=0).stop 2024-05-09T02:23:50.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.024+0000 7f03b0ba4700 1 -- 172.21.15.19:0/1195265449 shutdown_connections 2024-05-09T02:23:50.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.024+0000 7f03b0ba4700 1 --2- 172.21.15.19:0/1195265449 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f03940b29f0 0x7f03940b4f00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:50.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.024+0000 7f03b0ba4700 1 --2- 172.21.15.19:0/1195265449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ac109870 0x7f03ac0760a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:50.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.024+0000 7f03b0ba4700 1 --2- 172.21.15.19:0/1195265449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac1045f0 0x7f03ac077660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:50.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.024+0000 7f03b0ba4700 1 --2- 172.21.15.19:0/1195265449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03ac108f20 0x7f03ac075ac0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:50.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.024+0000 7f03b0ba4700 1 -- 172.21.15.19:0/1195265449 >> 172.21.15.19:0/1195265449 conn(0x7f03ac100040 msgr2=0x7f03ac108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:50.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.027+0000 7f03b0ba4700 1 -- 172.21.15.19:0/1195265449 shutdown_connections 2024-05-09T02:23:50.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:50.027+0000 7f03b0ba4700 1 -- 172.21.15.19:0/1195265449 wait complete. 2024-05-09T02:23:50.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:50 smithi059 ceph-mon[34781]: pgmap v391: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 29 MiB/s wr, 1.78k op/s 2024-05-09T02:23:50.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:50 smithi196 ceph-mon[34718]: pgmap v391: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 29 MiB/s wr, 1.78k op/s 2024-05-09T02:23:50.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:50 smithi019 ceph-mon[30151]: pgmap v391: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 29 MiB/s wr, 1.78k op/s 2024-05-09T02:23:51.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:51 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:23:51.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:51 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:23:51.902 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T02:23:51.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T02:23:51.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:51 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:23:52.259 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:23:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 730.0 s, 1671.0 tps, lat 59.972 ms stddev 71.490 2024-05-09T02:23:52.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:52 smithi019 ceph-mon[30151]: pgmap v392: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 30 MiB/s wr, 1.74k op/s 2024-05-09T02:23:52.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:52 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:23:52.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:52 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:23:52.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:52 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:23:52.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:52 smithi196 ceph-mon[34718]: pgmap v392: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 30 MiB/s wr, 1.74k op/s 2024-05-09T02:23:52.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:52 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:23:52.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:52 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:23:52.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:52 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:23:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:52 smithi059 ceph-mon[34781]: pgmap v392: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 30 MiB/s wr, 1.74k op/s 2024-05-09T02:23:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:52 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:23:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:52 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:23:52.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:52 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:23:54.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.410+0000 7f417920e700 1 -- 172.21.15.19:0/1852036456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f417410b280 msgr2=0x7f41741198b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:54.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.410+0000 7f417920e700 1 --2- 172.21.15.19:0/1852036456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f417410b280 0x7f41741198b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4168009990 tx=0x7f4168009ca0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.411+0000 7f417920e700 1 -- 172.21.15.19:0/1852036456 shutdown_connections 2024-05-09T02:23:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.411+0000 7f417920e700 1 --2- 172.21.15.19:0/1852036456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f417410b280 0x7f41741198b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.411+0000 7f417920e700 1 --2- 172.21.15.19:0/1852036456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4174111bd0 0x7f4174111fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.411+0000 7f417920e700 1 --2- 172.21.15.19:0/1852036456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417410a930 0x7f417410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.411+0000 7f417920e700 1 -- 172.21.15.19:0/1852036456 >> 172.21.15.19:0/1852036456 conn(0x7f417406f4d0 msgr2=0x7f417406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:54.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.413+0000 7f417920e700 1 -- 172.21.15.19:0/1852036456 shutdown_connections 2024-05-09T02:23:54.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.414+0000 7f417920e700 1 -- 172.21.15.19:0/1852036456 wait complete. 2024-05-09T02:23:54.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.416+0000 7f417920e700 1 Processor -- start 2024-05-09T02:23:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.416+0000 7f417920e700 1 -- start start 2024-05-09T02:23:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.417+0000 7f417920e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417410a930 0x7f41741a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.417+0000 7f417920e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f417410b280 0x7f41741a27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.417+0000 7f417920e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4174111bd0 0x7f41741ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.417+0000 7f417920e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f417411cdc0 con 0x7f417410b280 2024-05-09T02:23:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.417+0000 7f417920e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f417411cc40 con 0x7f417410a930 2024-05-09T02:23:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.417+0000 7f417920e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f417411cf40 con 0x7f4174111bd0 2024-05-09T02:23:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.417+0000 7f4172d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417410a930 0x7f41741a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.417+0000 7f4172d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417410a930 0x7f41741a2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56796/0 (socket says 172.21.15.19:56796) 2024-05-09T02:23:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.417+0000 7f4172d9d700 1 -- 172.21.15.19:0/3166148896 learned_addr learned my addr 172.21.15.19:0/3166148896 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.417+0000 7f417259c700 1 --2- 172.21.15.19:0/3166148896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f417410b280 0x7f41741a27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.417+0000 7f417359e700 1 --2- 172.21.15.19:0/3166148896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4174111bd0 0x7f41741ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.418+0000 7f4172d9d700 1 -- 172.21.15.19:0/3166148896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4174111bd0 msgr2=0x7f41741ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:54.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.418+0000 7f4172d9d700 1 --2- 172.21.15.19:0/3166148896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4174111bd0 0x7f41741ad310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.418+0000 7f4172d9d700 1 -- 172.21.15.19:0/3166148896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f417410b280 msgr2=0x7f41741a27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:54.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.418+0000 7f4172d9d700 1 --2- 172.21.15.19:0/3166148896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f417410b280 0x7f41741a27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.418+0000 7f4172d9d700 1 -- 172.21.15.19:0/3166148896 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41741ad850 con 0x7f417410a930 2024-05-09T02:23:54.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.418+0000 7f417259c700 1 --2- 172.21.15.19:0/3166148896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f417410b280 0x7f41741a27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:23:54.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.418+0000 7f4172d9d700 1 --2- 172.21.15.19:0/3166148896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417410a930 0x7f41741a2290 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f416400b4d0 tx=0x7f416400b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:54.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.419+0000 7f415bfff700 1 -- 172.21.15.19:0/3166148896 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164015020 con 0x7f417410a930 2024-05-09T02:23:54.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.419+0000 7f417920e700 1 -- 172.21.15.19:0/3166148896 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41741adb40 con 0x7f417410a930 2024-05-09T02:23:54.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.419+0000 7f417920e700 1 -- 172.21.15.19:0/3166148896 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41741ae080 con 0x7f417410a930 2024-05-09T02:23:54.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.419+0000 7f415bfff700 1 -- 172.21.15.19:0/3166148896 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4164011ce0 con 0x7f417410a930 2024-05-09T02:23:54.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.419+0000 7f415bfff700 1 -- 172.21.15.19:0/3166148896 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164010c10 con 0x7f417410a930 2024-05-09T02:23:54.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.423+0000 7f4187c24b80 1 Processor -- start 2024-05-09T02:23:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.423+0000 7f4187c24b80 1 -- start start 2024-05-09T02:23:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.424+0000 7f4187c24b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a98d80 0x5614f3ba12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.424+0000 7f4187c24b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a1e520 0x5614f3ba1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.424+0000 7f4187c24b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3b9b480 0x5614f3b9b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.424+0000 7f4187c24b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5614f3abe120 con 0x5614f3a1e520 2024-05-09T02:23:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.424+0000 7f4187c24b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5614f3a63a20 con 0x5614f3a98d80 2024-05-09T02:23:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.424+0000 7f4187c24b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5614f3aba0e0 con 0x5614f3b9b480 2024-05-09T02:23:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.424+0000 7f417359e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a98d80 0x5614f3ba12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.424+0000 7f415bfff700 1 -- 172.21.15.19:0/3166148896 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4164018020 con 0x7f417410a930 2024-05-09T02:23:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.424+0000 7f417359e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a98d80 0x5614f3ba12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56812/0 (socket says 172.21.15.19:56812) 2024-05-09T02:23:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.424+0000 7f417359e700 1 -- 172.21.15.19:0/1300684760 learned_addr learned my addr 172.21.15.19:0/1300684760 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.424+0000 7f417259c700 1 --2- 172.21.15.19:0/1300684760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3b9b480 0x5614f3b9b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.425+0000 7f4172d9d700 1 --2- 172.21.15.19:0/1300684760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a1e520 0x5614f3ba1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.425+0000 7f417359e700 1 -- 172.21.15.19:0/1300684760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3b9b480 msgr2=0x5614f3b9b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.425+0000 7f417359e700 1 --2- 172.21.15.19:0/1300684760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3b9b480 0x5614f3b9b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.425+0000 7f417359e700 1 -- 172.21.15.19:0/1300684760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a1e520 msgr2=0x5614f3ba1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.425+0000 7f417359e700 1 --2- 172.21.15.19:0/1300684760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a1e520 0x5614f3ba1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.425+0000 7f417359e700 1 -- 172.21.15.19:0/1300684760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f3a54580 con 0x5614f3a98d80 2024-05-09T02:23:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.425+0000 7f4172d9d700 1 --2- 172.21.15.19:0/1300684760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a1e520 0x5614f3ba1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:23:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.426+0000 7f415bfff700 1 --2- 172.21.15.19:0/3166148896 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f415c0b2a20 0x7f415c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.426+0000 7f417359e700 1 --2- 172.21.15.19:0/1300684760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a98d80 0x5614f3ba12c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4168009ff0 tx=0x7f416800b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3666ff90ee8dce04 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.426+0000 7f4172d9d700 1 --2- 172.21.15.19:0/3166148896 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f415c0b2a20 0x7f415c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.426+0000 7f41597fa700 1 -- 172.21.15.19:0/1300684760 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168003bb0 con 0x5614f3a98d80 2024-05-09T02:23:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.426+0000 7f415bfff700 1 -- 172.21.15.19:0/3166148896 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f41640e1ac0 con 0x7f417410a930 2024-05-09T02:23:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.426+0000 7f41597fa700 1 -- 172.21.15.19:0/1300684760 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f416800bdb0 con 0x5614f3a98d80 2024-05-09T02:23:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.426+0000 7f41597fa700 1 -- 172.21.15.19:0/1300684760 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416800f460 con 0x5614f3a98d80 2024-05-09T02:23:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.427+0000 7f4187c24b80 1 -- 172.21.15.19:0/1300684760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a98d80 msgr2=0x5614f3ba12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.427+0000 7f4172d9d700 1 --2- 172.21.15.19:0/3166148896 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f415c0b2a20 0x7f415c0b4f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f41640df3a0 tx=0x7f4164009300 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.427+0000 7f4187c24b80 1 --2- 172.21.15.19:0/1300684760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a98d80 0x5614f3ba12c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4168009ff0 tx=0x7f416800b560 comp rx=0 tx=0).stop 2024-05-09T02:23:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.427+0000 7f4187c24b80 1 -- 172.21.15.19:0/1300684760 shutdown_connections 2024-05-09T02:23:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.427+0000 7f4187c24b80 1 --2- 172.21.15.19:0/1300684760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3b9b480 0x5614f3b9b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.427+0000 7f4187c24b80 1 --2- 172.21.15.19:0/1300684760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a98d80 0x5614f3ba12c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.427+0000 7f4187c24b80 1 --2- 172.21.15.19:0/1300684760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a1e520 0x5614f3ba1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.427+0000 7f4187c24b80 1 -- 172.21.15.19:0/1300684760 >> 172.21.15.19:0/1300684760 conn(0x5614f3a21ff0 msgr2=0x5614f3b962a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.428+0000 7f4187c24b80 1 -- 172.21.15.19:0/1300684760 shutdown_connections 2024-05-09T02:23:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.428+0000 7f4187c24b80 1 -- 172.21.15.19:0/1300684760 wait complete. 2024-05-09T02:23:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.428+0000 7f4187c24b80 1 -- start start 2024-05-09T02:23:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.428+0000 7f4187c24b80 1 Processor -- start 2024-05-09T02:23:54.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f4187c24b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a1e520 0x5614f3ba4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f4187c24b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a98d80 0x5614f3ba54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f4187c24b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3b9b480 0x5614f3ba5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f4187c24b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5614f3b9c1d0 con 0x5614f3a1e520 2024-05-09T02:23:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f4187c24b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5614f3a58ba0 con 0x5614f3a98d80 2024-05-09T02:23:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f4187c24b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5614f3ba80a0 con 0x5614f3b9b480 2024-05-09T02:23:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f4187c24b80 10 client.0 _ll_register_callbacks cb 0x5614f392f740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f417359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a1e520 0x5614f3ba4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f417359e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a1e520 0x5614f3ba4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57088/0 (socket says 172.21.15.19:57088) 2024-05-09T02:23:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f417359e700 1 -- 172.21.15.19:0/759181789 learned_addr learned my addr 172.21.15.19:0/759181789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f417259c700 1 --2- 172.21.15.19:0/759181789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a98d80 0x5614f3ba54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f417359e700 1 --2- 172.21.15.19:0/759181789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3b9b480 0x5614f3ba5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f417359e700 1 -- 172.21.15.19:0/759181789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3b9b480 msgr2=0x5614f3ba5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f417359e700 1 --2- 172.21.15.19:0/759181789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3b9b480 0x5614f3ba5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f417359e700 1 -- 172.21.15.19:0/759181789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a98d80 msgr2=0x5614f3ba54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f417359e700 1 --2- 172.21.15.19:0/759181789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a98d80 0x5614f3ba54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f417359e700 1 -- 172.21.15.19:0/759181789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614f3939510 con 0x5614f3a1e520 2024-05-09T02:23:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f417359e700 1 --2- 172.21.15.19:0/759181789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a1e520 0x5614f3ba4f80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f41680098e0 tx=0x7f4168005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6b40844d083a7f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.429+0000 7f4156ff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:23:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.430+0000 7f4156ff5700 1 -- 172.21.15.19:0/759181789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168050040 con 0x5614f3a1e520 2024-05-09T02:23:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.430+0000 7f4187c24b80 1 -- 172.21.15.19:0/759181789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f416800bdb0 con 0x5614f3a1e520 2024-05-09T02:23:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.430+0000 7f4156ff5700 1 -- 172.21.15.19:0/759181789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f416800f870 con 0x5614f3a1e520 2024-05-09T02:23:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.430+0000 7f4156ff5700 1 -- 172.21.15.19:0/759181789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f416805b050 con 0x5614f3a1e520 2024-05-09T02:23:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.430+0000 7f4187c24b80 10 client.16881 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.430+0000 7f4187c24b80 1 -- 172.21.15.19:0/759181789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5614f3bb28c0 con 0x5614f3a1e520 2024-05-09T02:23:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.431+0000 7f4156ff5700 1 -- 172.21.15.19:0/759181789 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f416800bdb0 con 0x5614f3a1e520 2024-05-09T02:23:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.431+0000 7f4156ff5700 1 -- 172.21.15.19:0/759181789 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f4168021020 con 0x5614f3a1e520 2024-05-09T02:23:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.431+0000 7f4187c24b80 10 client.16881 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.431+0000 7f4187c24b80 10 client.16881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:23:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.431+0000 7f4187c24b80 20 client.16881 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.431+0000 7f4187c24b80 1 --2- 172.21.15.19:0/759181789 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5614f3bb5030 0x5614f3bb74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.431+0000 7f4187c24b80 4 client.16881 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:23:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.431+0000 7f4187c24b80 1 -- 172.21.15.19:0/759181789 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614f3acbb00 con 0x5614f3bb5030 2024-05-09T02:23:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.431+0000 7f417259c700 1 --2- 172.21.15.19:0/759181789 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5614f3bb5030 0x5614f3bb74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.432+0000 7f4156ff5700 10 client.16881 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.432+0000 7f417259c700 1 --2- 172.21.15.19:0/759181789 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5614f3bb5030 0x5614f3bb74a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ee6a34174371d60 server_cookie=e128913cd80faa14 in_seq=0 out_seq=0 2024-05-09T02:23:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.433+0000 7f4156ff5700 1 -- 172.21.15.19:0/759181789 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5614f3acbb00 con 0x5614f3bb5030 2024-05-09T02:23:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.433+0000 7f4156ff5700 10 client.16881 handle_command_reply: tid=0 2024-05-09T02:23:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.433+0000 7f4187c24b80 1 client.16881 shutdown 2024-05-09T02:23:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.433+0000 7f4187c24b80 1 -- 172.21.15.19:0/759181789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a1e520 msgr2=0x5614f3ba4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.433+0000 7f4187c24b80 1 --2- 172.21.15.19:0/759181789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a1e520 0x5614f3ba4f80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f41680098e0 tx=0x7f4168005440 comp rx=0 tx=0).stop 2024-05-09T02:23:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.433+0000 7f4187c24b80 1 -- 172.21.15.19:0/759181789 shutdown_connections 2024-05-09T02:23:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.433+0000 7f4187c24b80 1 --2- 172.21.15.19:0/759181789 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5614f3bb5030 0x5614f3bb74a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.433+0000 7f4187c24b80 1 --2- 172.21.15.19:0/759181789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3b9b480 0x5614f3ba5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.433+0000 7f4156ff5700 0 client.16881 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.433+0000 7f4187c24b80 1 --2- 172.21.15.19:0/759181789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a1e520 0x5614f3ba4f80 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.433+0000 7f4187c24b80 1 --2- 172.21.15.19:0/759181789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a98d80 0x5614f3ba54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.433+0000 7f4187c24b80 1 -- 172.21.15.19:0/759181789 >> 172.21.15.19:0/759181789 conn(0x5614f3a21ff0 msgr2=0x5614f3b90070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.434+0000 7f4187c24b80 1 -- 172.21.15.19:0/759181789 shutdown_connections 2024-05-09T02:23:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.434+0000 7f4187c24b80 1 -- 172.21.15.19:0/759181789 wait complete. 2024-05-09T02:23:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.434+0000 7f4187c24b80 20 client.16881 trim_cache size 0 max 16384 2024-05-09T02:23:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.464+0000 7f4187c24b80 1 Processor -- start 2024-05-09T02:23:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.464+0000 7f4187c24b80 1 -- start start 2024-05-09T02:23:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.464+0000 7f4187c24b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a1e520 0x5614f3b90aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.464+0000 7f4187c24b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a98d80 0x5614f3bbcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.464+0000 7f4187c24b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3b9b480 0x5614f3bbd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.464+0000 7f4187c24b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f416000d690 con 0x5614f3a98d80 2024-05-09T02:23:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.464+0000 7f4187c24b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f416000f1d0 con 0x5614f3a1e520 2024-05-09T02:23:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.464+0000 7f4187c24b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f416000dd60 con 0x5614f3b9b480 2024-05-09T02:23:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.465+0000 7f417259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a98d80 0x5614f3bbcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.465+0000 7f417259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a98d80 0x5614f3bbcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57100/0 (socket says 172.21.15.19:57100) 2024-05-09T02:23:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.465+0000 7f417259c700 1 -- 172.21.15.19:0/3754477457 learned_addr learned my addr 172.21.15.19:0/3754477457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.465+0000 7f417359e700 1 --2- 172.21.15.19:0/3754477457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a1e520 0x5614f3b90aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.465+0000 7f417359e700 1 --2- 172.21.15.19:0/3754477457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3b9b480 0x5614f3bbd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.465+0000 7f417359e700 1 -- 172.21.15.19:0/3754477457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3b9b480 msgr2=0x5614f3bbd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.465+0000 7f417359e700 1 --2- 172.21.15.19:0/3754477457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3b9b480 0x5614f3bbd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.465+0000 7f417359e700 1 -- 172.21.15.19:0/3754477457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a98d80 msgr2=0x5614f3bbcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.465+0000 7f417359e700 1 --2- 172.21.15.19:0/3754477457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a98d80 0x5614f3bbcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.465+0000 7f417359e700 1 -- 172.21.15.19:0/3754477457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f3ba7820 con 0x5614f3a1e520 2024-05-09T02:23:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.466+0000 7f417359e700 1 --2- 172.21.15.19:0/3754477457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a1e520 0x5614f3b90aa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4168001eb0 tx=0x7f4168028040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4b674e81a95988 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.466+0000 7f4158ff9700 1 -- 172.21.15.19:0/3754477457 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168023640 con 0x5614f3a1e520 2024-05-09T02:23:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.466+0000 7f4158ff9700 1 -- 172.21.15.19:0/3754477457 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f416800bd90 con 0x5614f3a1e520 2024-05-09T02:23:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.466+0000 7f4158ff9700 1 -- 172.21.15.19:0/3754477457 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416800fd80 con 0x5614f3a1e520 2024-05-09T02:23:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.468+0000 7f4187c24b80 1 -- 172.21.15.19:0/3754477457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a1e520 msgr2=0x5614f3b90aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.468+0000 7f4187c24b80 1 --2- 172.21.15.19:0/3754477457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a1e520 0x5614f3b90aa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4168001eb0 tx=0x7f4168028040 comp rx=0 tx=0).stop 2024-05-09T02:23:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.468+0000 7f4187c24b80 1 -- 172.21.15.19:0/3754477457 shutdown_connections 2024-05-09T02:23:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.468+0000 7f4187c24b80 1 --2- 172.21.15.19:0/3754477457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3b9b480 0x5614f3bbd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.468+0000 7f4187c24b80 1 --2- 172.21.15.19:0/3754477457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a1e520 0x5614f3b90aa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.468+0000 7f4187c24b80 1 --2- 172.21.15.19:0/3754477457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3a98d80 0x5614f3bbcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.468+0000 7f4187c24b80 1 -- 172.21.15.19:0/3754477457 >> 172.21.15.19:0/3754477457 conn(0x5614f3a21ff0 msgr2=0x5614f3b9f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.468+0000 7f4187c24b80 1 -- 172.21.15.19:0/3754477457 shutdown_connections 2024-05-09T02:23:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.468+0000 7f4187c24b80 1 -- 172.21.15.19:0/3754477457 wait complete. 2024-05-09T02:23:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.468+0000 7f4187c24b80 1 -- start start 2024-05-09T02:23:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.468+0000 7f4187c24b80 1 Processor -- start 2024-05-09T02:23:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.469+0000 7f4187c24b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a1e520 0x5614f3ba2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.469+0000 7f4187c24b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3a98d80 0x5614f3ba2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.469+0000 7f4187c24b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3b9b480 0x5614f3ba2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.469+0000 7f4187c24b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f416000eb00 con 0x5614f3b9b480 2024-05-09T02:23:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.469+0000 7f4187c24b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f416000e430 con 0x5614f3a1e520 2024-05-09T02:23:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.469+0000 7f4187c24b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f416000b650 con 0x5614f3a98d80 2024-05-09T02:23:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.469+0000 7f417259c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3a98d80 0x5614f3ba2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.469+0000 7f4187c24b80 10 client.0 _ll_register_callbacks cb 0x5614f392f740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.469+0000 7f417259c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3a98d80 0x5614f3ba2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:51786/0 (socket says 172.21.15.19:51786) 2024-05-09T02:23:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.469+0000 7f417259c700 1 -- 172.21.15.19:0/3179969858 learned_addr learned my addr 172.21.15.19:0/3179969858 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.469+0000 7f417359e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a1e520 0x5614f3ba2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.469+0000 7f417359e700 1 --2- 172.21.15.19:0/3179969858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3b9b480 0x5614f3ba2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.470+0000 7f417259c700 1 -- 172.21.15.19:0/3179969858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a1e520 msgr2=0x5614f3ba2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.470+0000 7f417259c700 1 --2- 172.21.15.19:0/3179969858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a1e520 0x5614f3ba2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.470+0000 7f417259c700 1 -- 172.21.15.19:0/3179969858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3b9b480 msgr2=0x5614f3ba2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.470+0000 7f417259c700 1 --2- 172.21.15.19:0/3179969858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3b9b480 0x5614f3ba2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.470+0000 7f417259c700 1 -- 172.21.15.19:0/3179969858 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614f3c1eab0 con 0x5614f3a98d80 2024-05-09T02:23:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.470+0000 7f417259c700 1 --2- 172.21.15.19:0/3179969858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3a98d80 0x5614f3ba2660 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f417410e6f0 tx=0x7f416c00ae90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=123b62f228ccc44e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.470+0000 7f4156ff5700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:23:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.472+0000 7f4156ff5700 1 -- 172.21.15.19:0/3179969858 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c02a060 con 0x5614f3a98d80 2024-05-09T02:23:54.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:54 smithi019 ceph-mon[30151]: pgmap v393: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 29 MiB/s wr, 1.64k op/s 2024-05-09T02:23:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.472+0000 7f4156ff5700 1 -- 172.21.15.19:0/3179969858 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f416c01d560 con 0x5614f3a98d80 2024-05-09T02:23:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.472+0000 7f4187c24b80 1 -- 172.21.15.19:0/3179969858 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f416800bd90 con 0x5614f3a98d80 2024-05-09T02:23:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.474+0000 7f4156ff5700 1 -- 172.21.15.19:0/3179969858 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f416c01d760 con 0x5614f3a98d80 2024-05-09T02:23:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.474+0000 7f4187c24b80 10 client.26653 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.474+0000 7f4187c24b80 1 -- 172.21.15.19:0/3179969858 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5614f3c1fd90 con 0x5614f3a98d80 2024-05-09T02:23:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.474+0000 7f4156ff5700 1 -- 172.21.15.19:0/3179969858 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f416800bd90 con 0x5614f3a98d80 2024-05-09T02:23:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.474+0000 7f4156ff5700 1 -- 172.21.15.19:0/3179969858 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f416c024030 con 0x5614f3a98d80 2024-05-09T02:23:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.475+0000 7f4187c24b80 10 client.26653 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.475+0000 7f4187c24b80 10 client.26653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:23:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.475+0000 7f4187c24b80 20 client.26653 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.475+0000 7f4187c24b80 1 --2- 172.21.15.19:0/3179969858 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5614f3c204b0 0x5614f3c228d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.475+0000 7f4187c24b80 4 client.26653 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T02:23:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.475+0000 7f4187c24b80 1 -- 172.21.15.19:0/3179969858 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5614f3ac5ba0 con 0x5614f3c204b0 2024-05-09T02:23:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.475+0000 7f417359e700 1 --2- 172.21.15.19:0/3179969858 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5614f3c204b0 0x5614f3c228d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:54.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.477+0000 7f417359e700 1 --2- 172.21.15.19:0/3179969858 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5614f3c204b0 0x5614f3c228d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=917f4edf342b4a3f server_cookie=5d4afdedd79f9717 in_seq=0 out_seq=0 2024-05-09T02:23:54.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.477+0000 7f4156ff5700 10 client.26653 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:54.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.478+0000 7f4156ff5700 1 -- 172.21.15.19:0/3179969858 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5614f3ac5ba0 con 0x5614f3c204b0 2024-05-09T02:23:54.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.478+0000 7f4156ff5700 10 client.26653 handle_command_reply: tid=0 2024-05-09T02:23:54.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.478+0000 7f4187c24b80 1 client.26653 shutdown 2024-05-09T02:23:54.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.478+0000 7f4187c24b80 1 -- 172.21.15.19:0/3179969858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3a98d80 msgr2=0x5614f3ba2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:54.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.478+0000 7f4187c24b80 1 --2- 172.21.15.19:0/3179969858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3a98d80 0x5614f3ba2660 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f417410e6f0 tx=0x7f416c00ae90 comp rx=0 tx=0).stop 2024-05-09T02:23:54.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.478+0000 7f4187c24b80 1 -- 172.21.15.19:0/3179969858 shutdown_connections 2024-05-09T02:23:54.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.478+0000 7f4187c24b80 1 --2- 172.21.15.19:0/3179969858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614f3b9b480 0x5614f3ba2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.478+0000 7f4187c24b80 1 --2- 172.21.15.19:0/3179969858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f3a1e520 0x5614f3ba2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.478+0000 7f4187c24b80 1 --2- 172.21.15.19:0/3179969858 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5614f3c204b0 0x5614f3c228d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.478+0000 7f4187c24b80 1 --2- 172.21.15.19:0/3179969858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f3a98d80 0x5614f3ba2660 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.478+0000 7f4187c24b80 1 -- 172.21.15.19:0/3179969858 >> 172.21.15.19:0/3179969858 conn(0x5614f3a21ff0 msgr2=0x5614f3b9a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:54.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.478+0000 7f4156ff5700 0 client.26653 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:54.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.478+0000 7f4187c24b80 1 -- 172.21.15.19:0/3179969858 shutdown_connections 2024-05-09T02:23:54.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.478+0000 7f4187c24b80 1 -- 172.21.15.19:0/3179969858 wait complete. 2024-05-09T02:23:54.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.478+0000 7f4187c24b80 20 client.26653 trim_cache size 0 max 16384 2024-05-09T02:23:54.482 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:23:54.482 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T02:23:54.482 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "edb599a0-4b60-4780-9387-fbd95ee94731", 2024-05-09T02:23:54.483 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T02:23:54.483 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:23:54.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.479+0000 7f417920e700 1 -- 172.21.15.19:0/3166148896 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f415c0b2a20 msgr2=0x7f415c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:54.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.479+0000 7f417920e700 1 --2- 172.21.15.19:0/3166148896 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f415c0b2a20 0x7f415c0b4f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f41640df3a0 tx=0x7f4164009300 comp rx=0 tx=0).stop 2024-05-09T02:23:54.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.479+0000 7f417920e700 1 -- 172.21.15.19:0/3166148896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417410a930 msgr2=0x7f41741a2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.479+0000 7f417920e700 1 --2- 172.21.15.19:0/3166148896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417410a930 0x7f41741a2290 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f416400b4d0 tx=0x7f416400b7e0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.479+0000 7f417920e700 1 -- 172.21.15.19:0/3166148896 shutdown_connections 2024-05-09T02:23:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.479+0000 7f417920e700 1 --2- 172.21.15.19:0/3166148896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4174111bd0 0x7f41741ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.479+0000 7f417920e700 1 --2- 172.21.15.19:0/3166148896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417410a930 0x7f41741a2290 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.479+0000 7f417920e700 1 --2- 172.21.15.19:0/3166148896 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f415c0b2a20 0x7f415c0b4f30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.479+0000 7f417920e700 1 --2- 172.21.15.19:0/3166148896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f417410b280 0x7f41741a27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.479+0000 7f417920e700 1 -- 172.21.15.19:0/3166148896 >> 172.21.15.19:0/3166148896 conn(0x7f417406f4d0 msgr2=0x7f4174110270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.479+0000 7f417920e700 1 -- 172.21.15.19:0/3166148896 shutdown_connections 2024-05-09T02:23:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:54.480+0000 7f417920e700 1 -- 172.21.15.19:0/3166148896 wait complete. 2024-05-09T02:23:54.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:54 smithi196 ceph-mon[34718]: pgmap v393: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 29 MiB/s wr, 1.64k op/s 2024-05-09T02:23:54.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:54 smithi059 ceph-mon[34781]: pgmap v393: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 29 MiB/s wr, 1.64k op/s 2024-05-09T02:23:55.162 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:23:55.569 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:23:55.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub queued for path: / 2024-05-09T02:23:55.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:23:55.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:23:55.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:23:55.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:23:55.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:23:55.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:23:55.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:23:55.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:23:55.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.796 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:55 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub queued for path: / 2024-05-09T02:23:55.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:23:55.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:23:55.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:23:55.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:23:55.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:23:55.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:23:55.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:23:55.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:23:55.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:55 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub queued for path: / 2024-05-09T02:23:55.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:23:55.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:23:55.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:23:55.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:23:55.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:23:55.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:23:55.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:23:55.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:23:55.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:55.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:55 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:23:56.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:56 smithi196 ceph-mon[34718]: pgmap v394: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 35 MiB/s wr, 1.90k op/s 2024-05-09T02:23:56.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:56 smithi019 ceph-mon[30151]: pgmap v394: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 35 MiB/s wr, 1.90k op/s 2024-05-09T02:23:56.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:56 smithi059 ceph-mon[34781]: pgmap v394: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 35 MiB/s wr, 1.90k op/s 2024-05-09T02:23:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 735.0 s, 1741.2 tps, lat 57.027 ms stddev 71.301 2024-05-09T02:23:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.741+0000 7fa07d2fc700 1 -- 172.21.15.19:0/71694579 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068003c00 con 0x7fa07810a930 2024-05-09T02:23:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.742+0000 7fa080562700 1 -- 172.21.15.19:0/71694579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07810a930 msgr2=0x7fa07810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.742+0000 7fa080562700 1 --2- 172.21.15.19:0/71694579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07810a930 0x7fa07810ad40 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa068009990 tx=0x7fa068009ca0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.742+0000 7fa080562700 1 -- 172.21.15.19:0/71694579 shutdown_connections 2024-05-09T02:23:57.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.742+0000 7fa080562700 1 --2- 172.21.15.19:0/71694579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa07811d930 0x7fa07811fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.742+0000 7fa080562700 1 --2- 172.21.15.19:0/71694579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07810a930 0x7fa07810ad40 secure :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa068009990 tx=0x7fa068009ca0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.742+0000 7fa080562700 1 --2- 172.21.15.19:0/71694579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa07810b280 0x7fa07811d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.742+0000 7fa080562700 1 -- 172.21.15.19:0/71694579 >> 172.21.15.19:0/71694579 conn(0x7fa07806f4d0 msgr2=0x7fa07806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:57.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.742+0000 7fa080562700 1 -- 172.21.15.19:0/71694579 shutdown_connections 2024-05-09T02:23:57.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.742+0000 7fa080562700 1 -- 172.21.15.19:0/71694579 wait complete. 2024-05-09T02:23:57.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.743+0000 7fa080562700 1 Processor -- start 2024-05-09T02:23:57.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.743+0000 7fa080562700 1 -- start start 2024-05-09T02:23:57.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.744+0000 7fa080562700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa07810b280 0x7fa0781a9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.744+0000 7fa080562700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa07811d930 0x7fa0781aa530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.744+0000 7fa080562700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0781ae700 0x7fa0781aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.744+0000 7fa080562700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa078123440 con 0x7fa0781ae700 2024-05-09T02:23:57.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.744+0000 7fa080562700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa0781232c0 con 0x7fa07811d930 2024-05-09T02:23:57.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.744+0000 7fa080562700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0781235c0 con 0x7fa07810b280 2024-05-09T02:23:57.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.744+0000 7fa07eaff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0781ae700 0x7fa0781aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.744+0000 7fa07dafd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa07811d930 0x7fa0781aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.745+0000 7fa07e2fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa07810b280 0x7fa0781a9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.745+0000 7fa07e2fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa07810b280 0x7fa0781a9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:38944/0 (socket says 172.21.15.19:38944) 2024-05-09T02:23:57.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.745+0000 7fa07e2fe700 1 -- 172.21.15.19:0/555105816 learned_addr learned my addr 172.21.15.19:0/555105816 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:57.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.745+0000 7fa07eaff700 1 -- 172.21.15.19:0/555105816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa07810b280 msgr2=0x7fa0781a9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:57.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.745+0000 7fa07eaff700 1 --2- 172.21.15.19:0/555105816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa07810b280 0x7fa0781a9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.745+0000 7fa07eaff700 1 -- 172.21.15.19:0/555105816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa07811d930 msgr2=0x7fa0781aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:57.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.745+0000 7fa07eaff700 1 --2- 172.21.15.19:0/555105816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa07811d930 0x7fa0781aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.745+0000 7fa07eaff700 1 -- 172.21.15.19:0/555105816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0781af250 con 0x7fa0781ae700 2024-05-09T02:23:57.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.746+0000 7fa07dafd700 1 --2- 172.21.15.19:0/555105816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa07811d930 0x7fa0781aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:23:57.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.746+0000 7fa07eaff700 1 --2- 172.21.15.19:0/555105816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0781ae700 0x7fa0781aeb10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa07000bdf0 tx=0x7fa070009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.748+0000 7fa06f7fe700 1 -- 172.21.15.19:0/555105816 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070019070 con 0x7fa0781ae700 2024-05-09T02:23:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.748+0000 7fa06f7fe700 1 -- 172.21.15.19:0/555105816 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa07000cc90 con 0x7fa0781ae700 2024-05-09T02:23:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.748+0000 7fa06f7fe700 1 -- 172.21.15.19:0/555105816 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070012600 con 0x7fa0781ae700 2024-05-09T02:23:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.748+0000 7fa080562700 1 -- 172.21.15.19:0/555105816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0781af540 con 0x7fa0781ae700 2024-05-09T02:23:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.748+0000 7fa080562700 1 -- 172.21.15.19:0/555105816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa07807bd40 con 0x7fa0781ae700 2024-05-09T02:23:57.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.756+0000 7fa06f7fe700 1 -- 172.21.15.19:0/555105816 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa0700127a0 con 0x7fa0781ae700 2024-05-09T02:23:57.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.757+0000 7fa06f7fe700 1 --2- 172.21.15.19:0/555105816 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0640b2b80 0x7fa0640b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.757+0000 7fa06f7fe700 1 -- 172.21.15.19:0/555105816 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fa0700e1990 con 0x7fa0781ae700 2024-05-09T02:23:57.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.757+0000 7fa08ef78b80 1 Processor -- start 2024-05-09T02:23:57.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.758+0000 7fa07e2fe700 1 --2- 172.21.15.19:0/555105816 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0640b2b80 0x7fa0640b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.758+0000 7fa08ef78b80 1 -- start start 2024-05-09T02:23:57.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.758+0000 7fa07e2fe700 1 --2- 172.21.15.19:0/555105816 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0640b2b80 0x7fa0640b5090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa068009750 tx=0x7fa06801a040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:57.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.759+0000 7fa08ef78b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1655d00 0x559bc175e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.759+0000 7fa08ef78b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc15db4a0 0x559bc175e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.759+0000 7fa08ef78b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc1758400 0x559bc1758810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.759+0000 7fa08ef78b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559bc167b0a0 con 0x559bc1758400 2024-05-09T02:23:57.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.759+0000 7fa08ef78b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559bc16209a0 con 0x559bc1655d00 2024-05-09T02:23:57.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.759+0000 7fa08ef78b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559bc1677060 con 0x559bc15db4a0 2024-05-09T02:23:57.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.759+0000 7fa07eaff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc1758400 0x559bc1758810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.759+0000 7fa07eaff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc1758400 0x559bc1758810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39380/0 (socket says 172.21.15.19:39380) 2024-05-09T02:23:57.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.759+0000 7fa07eaff700 1 -- 172.21.15.19:0/1691835836 learned_addr learned my addr 172.21.15.19:0/1691835836 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:57.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.760+0000 7fa07dafd700 1 --2- 172.21.15.19:0/1691835836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc15db4a0 0x559bc175e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.760+0000 7fa07e2fe700 1 --2- 172.21.15.19:0/1691835836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1655d00 0x559bc175e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.760+0000 7fa07eaff700 1 -- 172.21.15.19:0/1691835836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc15db4a0 msgr2=0x559bc175e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.760+0000 7fa07eaff700 1 --2- 172.21.15.19:0/1691835836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc15db4a0 0x559bc175e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.760+0000 7fa07eaff700 1 -- 172.21.15.19:0/1691835836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1655d00 msgr2=0x559bc175e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.760+0000 7fa07eaff700 1 --2- 172.21.15.19:0/1691835836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1655d00 0x559bc175e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.760+0000 7fa07eaff700 1 -- 172.21.15.19:0/1691835836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bc1611500 con 0x559bc1758400 2024-05-09T02:23:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.760+0000 7fa07eaff700 1 --2- 172.21.15.19:0/1691835836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc1758400 0x559bc1758810 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa0700a4dd0 tx=0x7fa0700a8bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b76804673e7827a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.761+0000 7fa06cff9700 1 -- 172.21.15.19:0/1691835836 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0700ad100 con 0x559bc1758400 2024-05-09T02:23:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.761+0000 7fa06cff9700 1 -- 172.21.15.19:0/1691835836 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0700b1bb0 con 0x559bc1758400 2024-05-09T02:23:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.761+0000 7fa06cff9700 1 -- 172.21.15.19:0/1691835836 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0700b1980 con 0x559bc1758400 2024-05-09T02:23:57.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.762+0000 7fa08ef78b80 1 -- 172.21.15.19:0/1691835836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc1758400 msgr2=0x559bc1758810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:57.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.762+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/1691835836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc1758400 0x559bc1758810 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa0700a4dd0 tx=0x7fa0700a8bd0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.763+0000 7fa08ef78b80 1 -- 172.21.15.19:0/1691835836 shutdown_connections 2024-05-09T02:23:57.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.763+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/1691835836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc1758400 0x559bc1758810 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.763+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/1691835836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1655d00 0x559bc175e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.763+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/1691835836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc15db4a0 0x559bc175e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.763+0000 7fa08ef78b80 1 -- 172.21.15.19:0/1691835836 >> 172.21.15.19:0/1691835836 conn(0x559bc15def70 msgr2=0x559bc1753220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:57.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.763+0000 7fa08ef78b80 1 -- 172.21.15.19:0/1691835836 shutdown_connections 2024-05-09T02:23:57.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.764+0000 7fa08ef78b80 1 -- 172.21.15.19:0/1691835836 wait complete. 2024-05-09T02:23:57.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.765+0000 7fa08ef78b80 1 -- start start 2024-05-09T02:23:57.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.765+0000 7fa08ef78b80 1 Processor -- start 2024-05-09T02:23:57.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.766+0000 7fa08ef78b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc15db4a0 0x559bc1761f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.766+0000 7fa08ef78b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1655d00 0x559bc17624a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.766+0000 7fa07e2fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc15db4a0 0x559bc1761f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.766+0000 7fa07e2fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc15db4a0 0x559bc1761f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39388/0 (socket says 172.21.15.19:39388) 2024-05-09T02:23:57.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.766+0000 7fa07e2fe700 1 -- 172.21.15.19:0/3543830887 learned_addr learned my addr 172.21.15.19:0/3543830887 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:57.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.766+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/3543830887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc1758400 0x559bc17629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.766+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3543830887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559bc17591b0 con 0x559bc15db4a0 2024-05-09T02:23:57.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.766+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3543830887 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559bc1615b20 con 0x559bc1655d00 2024-05-09T02:23:57.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.766+0000 7fa07dafd700 1 --2- 172.21.15.19:0/3543830887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1655d00 0x559bc17624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.766+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3543830887 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559bc1765020 con 0x559bc1758400 2024-05-09T02:23:57.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.766+0000 7fa07eaff700 1 --2- 172.21.15.19:0/3543830887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc1758400 0x559bc17629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.767+0000 7fa07e2fe700 1 -- 172.21.15.19:0/3543830887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc1758400 msgr2=0x559bc17629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.767+0000 7fa07e2fe700 1 --2- 172.21.15.19:0/3543830887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc1758400 0x559bc17629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.767+0000 7fa07e2fe700 1 -- 172.21.15.19:0/3543830887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1655d00 msgr2=0x559bc17624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.767+0000 7fa07e2fe700 1 --2- 172.21.15.19:0/3543830887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1655d00 0x559bc17624a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.767+0000 7fa07e2fe700 1 -- 172.21.15.19:0/3543830887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bc14f6490 con 0x559bc15db4a0 2024-05-09T02:23:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.767+0000 7fa07dafd700 1 --2- 172.21.15.19:0/3543830887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1655d00 0x559bc17624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:23:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.767+0000 7fa08ef78b80 10 client.0 _ll_register_callbacks cb 0x559bc14ec6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.767+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3543830887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0700b1bb0 con 0x559bc15db4a0 2024-05-09T02:23:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.767+0000 7fa07e2fe700 1 --2- 172.21.15.19:0/3543830887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc15db4a0 0x559bc1761f60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa068003ab0 tx=0x7fa068009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f799708f1a447f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.767+0000 7fa061ffb700 10 client.16899 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:23:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.767+0000 7fa061ffb700 1 -- 172.21.15.19:0/3543830887 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068032060 con 0x559bc15db4a0 2024-05-09T02:23:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.768+0000 7fa061ffb700 1 -- 172.21.15.19:0/3543830887 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fa068005480 con 0x559bc15db4a0 2024-05-09T02:23:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.768+0000 7fa061ffb700 1 -- 172.21.15.19:0/3543830887 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa0680056c0 con 0x559bc15db4a0 2024-05-09T02:23:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.768+0000 7fa08ef78b80 10 client.16899 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:57.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.768+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3543830887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=58}) -- 0x559bc176f790 con 0x559bc15db4a0 2024-05-09T02:23:57.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.768+0000 7fa061ffb700 1 -- 172.21.15.19:0/3543830887 <== mon.0 v2:172.21.15.19:3300/0 4 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fa06800f460 con 0x559bc15db4a0 2024-05-09T02:23:57.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.768+0000 7fa08ef78b80 10 client.16899 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:57.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.768+0000 7fa08ef78b80 10 client.16899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:23:57.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.768+0000 7fa08ef78b80 20 client.16899 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:57.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.768+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/3543830887 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x559bc1771cb0 0x559bc17741a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.768+0000 7fa08ef78b80 4 client.16899 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:23:57.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.768+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3543830887 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bc1688a80 con 0x559bc1771cb0 2024-05-09T02:23:57.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.769+0000 7fa07dafd700 1 --2- 172.21.15.19:0/3543830887 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x559bc1771cb0 0x559bc17741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.770+0000 7fa061ffb700 10 client.16899 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.770+0000 7fa07dafd700 1 --2- 172.21.15.19:0/3543830887 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x559bc1771cb0 0x559bc17741a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d727acfd7f0126f6 server_cookie=843c9e123e6b3a63 in_seq=0 out_seq=0 2024-05-09T02:23:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.771+0000 7fa061ffb700 1 -- 172.21.15.19:0/3543830887 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559bc1688a80 con 0x559bc1771cb0 2024-05-09T02:23:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.771+0000 7fa061ffb700 10 client.16899 handle_command_reply: tid=0 2024-05-09T02:23:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.771+0000 7fa08ef78b80 1 client.16899 shutdown 2024-05-09T02:23:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.771+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3543830887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc15db4a0 msgr2=0x559bc1761f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.771+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/3543830887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc15db4a0 0x559bc1761f60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa068003ab0 tx=0x7fa068009310 comp rx=0 tx=0).stop 2024-05-09T02:23:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.771+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3543830887 shutdown_connections 2024-05-09T02:23:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.771+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/3543830887 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x559bc1771cb0 0x559bc17741a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.771+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/3543830887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc1758400 0x559bc17629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.771+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/3543830887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc15db4a0 0x559bc1761f60 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.771+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/3543830887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1655d00 0x559bc17624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.771+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3543830887 >> 172.21.15.19:0/3543830887 conn(0x559bc15def70 msgr2=0x559bc174cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.772+0000 7fa061ffb700 0 client.16899 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.773+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3543830887 shutdown_connections 2024-05-09T02:23:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.773+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3543830887 wait complete. 2024-05-09T02:23:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.773+0000 7fa08ef78b80 20 client.16899 trim_cache size 0 max 16384 2024-05-09T02:23:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.814+0000 7fa08ef78b80 1 Processor -- start 2024-05-09T02:23:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.814+0000 7fa08ef78b80 1 -- start start 2024-05-09T02:23:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.815+0000 7fa08ef78b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc15db4a0 0x559bc1761280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.815+0000 7fa08ef78b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc1655d00 0x559bc176def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.815+0000 7fa08ef78b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1758400 0x559bc1779960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.815+0000 7fa08ef78b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c00d610 con 0x559bc15db4a0 2024-05-09T02:23:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.815+0000 7fa08ef78b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c00f150 con 0x559bc1758400 2024-05-09T02:23:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.815+0000 7fa07dafd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc1655d00 0x559bc176def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.815+0000 7fa08ef78b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c00dce0 con 0x559bc1655d00 2024-05-09T02:23:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.815+0000 7fa07dafd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc1655d00 0x559bc176def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:38972/0 (socket says 172.21.15.19:38972) 2024-05-09T02:23:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.815+0000 7fa07dafd700 1 -- 172.21.15.19:0/3527700286 learned_addr learned my addr 172.21.15.19:0/3527700286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.816+0000 7fa07eaff700 1 --2- 172.21.15.19:0/3527700286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1758400 0x559bc1779960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.816+0000 7fa07e2fe700 1 --2- 172.21.15.19:0/3527700286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc15db4a0 0x559bc1761280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.816+0000 7fa07dafd700 1 -- 172.21.15.19:0/3527700286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1758400 msgr2=0x559bc1779960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.816+0000 7fa07dafd700 1 --2- 172.21.15.19:0/3527700286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1758400 0x559bc1779960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.816+0000 7fa07dafd700 1 -- 172.21.15.19:0/3527700286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc15db4a0 msgr2=0x559bc1761280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.816+0000 7fa07dafd700 1 --2- 172.21.15.19:0/3527700286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc15db4a0 0x559bc1761280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.816+0000 7fa07dafd700 1 -- 172.21.15.19:0/3527700286 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bc17746d0 con 0x559bc1655d00 2024-05-09T02:23:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.817+0000 7fa07dafd700 1 --2- 172.21.15.19:0/3527700286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc1655d00 0x559bc176def0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa078120050 tx=0x7fa074008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=637e7e657d402824 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.817+0000 7fa063fff700 1 -- 172.21.15.19:0/3527700286 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074016b80 con 0x559bc1655d00 2024-05-09T02:23:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.817+0000 7fa063fff700 1 -- 172.21.15.19:0/3527700286 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa074016d20 con 0x559bc1655d00 2024-05-09T02:23:57.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.819+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3527700286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc1655d00 msgr2=0x559bc176def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:57.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.819+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/3527700286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc1655d00 0x559bc176def0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa078120050 tx=0x7fa074008040 comp rx=0 tx=0).stop 2024-05-09T02:23:57.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.820+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3527700286 shutdown_connections 2024-05-09T02:23:57.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.820+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/3527700286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1758400 0x559bc1779960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.820+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/3527700286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc15db4a0 0x559bc1761280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.820+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/3527700286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc1655d00 0x559bc176def0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.820+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3527700286 >> 172.21.15.19:0/3527700286 conn(0x559bc15def70 msgr2=0x559bc175d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:57.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.820+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3527700286 shutdown_connections 2024-05-09T02:23:57.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.820+0000 7fa08ef78b80 1 -- 172.21.15.19:0/3527700286 wait complete. 2024-05-09T02:23:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.821+0000 7fa08ef78b80 1 -- start start 2024-05-09T02:23:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.821+0000 7fa08ef78b80 1 Processor -- start 2024-05-09T02:23:57.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.821+0000 7fa08ef78b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc15db4a0 0x559bc176a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.822+0000 7fa08ef78b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc1655d00 0x559bc1778d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.822+0000 7fa08ef78b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1758400 0x559bc1779290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.822+0000 7fa08ef78b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c00ea80 con 0x559bc1655d00 2024-05-09T02:23:57.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.822+0000 7fa08ef78b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c00e3b0 con 0x559bc1758400 2024-05-09T02:23:57.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.822+0000 7fa08ef78b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c00b5d0 con 0x559bc15db4a0 2024-05-09T02:23:57.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.822+0000 7fa08ef78b80 10 client.0 _ll_register_callbacks cb 0x559bc14ec6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:23:57.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.822+0000 7fa07e2fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc15db4a0 0x559bc176a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.822+0000 7fa07e2fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc15db4a0 0x559bc176a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:38984/0 (socket says 172.21.15.19:38984) 2024-05-09T02:23:57.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.822+0000 7fa07e2fe700 1 -- 172.21.15.19:0/2551467368 learned_addr learned my addr 172.21.15.19:0/2551467368 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:23:57.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.824+0000 7fa07dafd700 1 --2- 172.21.15.19:0/2551467368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc1655d00 0x559bc1778d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.826+0000 7fa07eaff700 1 --2- 172.21.15.19:0/2551467368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1758400 0x559bc1779290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.828+0000 7fa07eaff700 1 -- 172.21.15.19:0/2551467368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc15db4a0 msgr2=0x559bc176a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:57.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.828+0000 7fa07eaff700 1 --2- 172.21.15.19:0/2551467368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc15db4a0 0x559bc176a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.828+0000 7fa07eaff700 1 -- 172.21.15.19:0/2551467368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc1655d00 msgr2=0x559bc1778d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:23:57.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.828+0000 7fa07eaff700 1 --2- 172.21.15.19:0/2551467368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc1655d00 0x559bc1778d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.828+0000 7fa07eaff700 1 -- 172.21.15.19:0/2551467368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bc1779850 con 0x559bc1758400 2024-05-09T02:23:57.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.828+0000 7fa07eaff700 1 --2- 172.21.15.19:0/2551467368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1758400 0x559bc1779290 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa0700a4dd0 tx=0x7fa0700a87d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28bbe063ea243ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:23:57.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.828+0000 7fa061ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:23:57.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.829+0000 7fa061ffb700 1 -- 172.21.15.19:0/2551467368 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0700a82e0 con 0x559bc1758400 2024-05-09T02:23:57.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.829+0000 7fa061ffb700 1 -- 172.21.15.19:0/2551467368 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fa0700a5340 con 0x559bc1758400 2024-05-09T02:23:57.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.829+0000 7fa08ef78b80 1 -- 172.21.15.19:0/2551467368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559bc1682b20 con 0x559bc1758400 2024-05-09T02:23:57.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.829+0000 7fa061ffb700 1 -- 172.21.15.19:0/2551467368 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa0700b1970 con 0x559bc1758400 2024-05-09T02:23:57.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.829+0000 7fa08ef78b80 10 client.26630 fetch_fsmap learned FSMap version 58 2024-05-09T02:23:57.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.829+0000 7fa08ef78b80 1 -- 172.21.15.19:0/2551467368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x559bc17dc2a0 con 0x559bc1758400 2024-05-09T02:23:57.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.830+0000 7fa061ffb700 1 -- 172.21.15.19:0/2551467368 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559bc1682b20 con 0x559bc1758400 2024-05-09T02:23:57.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.830+0000 7fa061ffb700 1 -- 172.21.15.19:0/2551467368 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fa0700c2020 con 0x559bc1758400 2024-05-09T02:23:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.830+0000 7fa08ef78b80 10 client.26630 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:23:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.830+0000 7fa08ef78b80 10 client.26630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:23:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.830+0000 7fa08ef78b80 20 client.26630 populate_metadata read hostname 'smithi019' 2024-05-09T02:23:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.830+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/2551467368 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x559bc17dc4d0 0x559bc17de8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:23:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.830+0000 7fa08ef78b80 4 client.26630 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:23:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.830+0000 7fa08ef78b80 1 -- 172.21.15.19:0/2551467368 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559bc14f6970 con 0x559bc17dc4d0 2024-05-09T02:23:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.831+0000 7fa07e2fe700 1 --2- 172.21.15.19:0/2551467368 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x559bc17dc4d0 0x559bc17de8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:23:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.831+0000 7fa061ffb700 10 client.26630 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:57.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.844+0000 7fa07e2fe700 1 --2- 172.21.15.19:0/2551467368 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x559bc17dc4d0 0x559bc17de8f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9861d5ca1cd8d51 server_cookie=6e22a05b8e13ecd in_seq=0 out_seq=0 2024-05-09T02:23:57.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.845+0000 7fa061ffb700 1 -- 172.21.15.19:0/2551467368 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559bc14f6970 con 0x559bc17dc4d0 2024-05-09T02:23:57.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.845+0000 7fa061ffb700 10 client.26630 handle_command_reply: tid=0 2024-05-09T02:23:57.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.845+0000 7fa08ef78b80 1 client.26630 shutdown 2024-05-09T02:23:57.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.845+0000 7fa08ef78b80 1 -- 172.21.15.19:0/2551467368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1758400 msgr2=0x559bc1779290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:57.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.845+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/2551467368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1758400 0x559bc1779290 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa0700a4dd0 tx=0x7fa0700a87d0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.846+0000 7fa08ef78b80 1 -- 172.21.15.19:0/2551467368 shutdown_connections 2024-05-09T02:23:57.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.846+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/2551467368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bc1758400 0x559bc1779290 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.846+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/2551467368 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x559bc17dc4d0 0x559bc17de8f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.846+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/2551467368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559bc15db4a0 0x559bc176a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.846+0000 7fa08ef78b80 1 --2- 172.21.15.19:0/2551467368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bc1655d00 0x559bc1778d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.846+0000 7fa08ef78b80 1 -- 172.21.15.19:0/2551467368 >> 172.21.15.19:0/2551467368 conn(0x559bc15def70 msgr2=0x559bc174cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:57.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.846+0000 7fa061ffb700 0 client.26630 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:23:57.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.846+0000 7fa08ef78b80 1 -- 172.21.15.19:0/2551467368 shutdown_connections 2024-05-09T02:23:57.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.846+0000 7fa08ef78b80 1 -- 172.21.15.19:0/2551467368 wait complete. 2024-05-09T02:23:57.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.853+0000 7fa08ef78b80 20 client.26630 trim_cache size 0 max 16384 2024-05-09T02:23:57.858 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:23:57.858 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T02:23:57.859 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T02:23:57.859 INFO:teuthology.orchestra.run.smithi019.stdout: "edb599a0-4b60-4780-9387-fbd95ee94731": { 2024-05-09T02:23:57.859 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T02:23:57.859 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "edb599a0-4b60-4780-9387-fbd95ee94731", 2024-05-09T02:23:57.859 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T02:23:57.859 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:23:57.859 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:23:57.859 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:23:57.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.856+0000 7fa080562700 1 -- 172.21.15.19:0/555105816 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0640b2b80 msgr2=0x7fa0640b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:57.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.856+0000 7fa080562700 1 --2- 172.21.15.19:0/555105816 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0640b2b80 0x7fa0640b5090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa068009750 tx=0x7fa06801a040 comp rx=0 tx=0).stop 2024-05-09T02:23:57.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.856+0000 7fa080562700 1 -- 172.21.15.19:0/555105816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0781ae700 msgr2=0x7fa0781aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:23:57.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.856+0000 7fa080562700 1 --2- 172.21.15.19:0/555105816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0781ae700 0x7fa0781aeb10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa07000bdf0 tx=0x7fa070009ed0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.857+0000 7fa080562700 1 -- 172.21.15.19:0/555105816 shutdown_connections 2024-05-09T02:23:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.857+0000 7fa080562700 1 --2- 172.21.15.19:0/555105816 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0640b2b80 0x7fa0640b5090 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa068009750 tx=0x7fa06801a040 comp rx=0 tx=0).stop 2024-05-09T02:23:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.857+0000 7fa080562700 1 --2- 172.21.15.19:0/555105816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0781ae700 0x7fa0781aeb10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.857+0000 7fa080562700 1 --2- 172.21.15.19:0/555105816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa07810b280 0x7fa0781a9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.857+0000 7fa080562700 1 --2- 172.21.15.19:0/555105816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa07811d930 0x7fa0781aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:23:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.857+0000 7fa080562700 1 -- 172.21.15.19:0/555105816 >> 172.21.15.19:0/555105816 conn(0x7fa07806f4d0 msgr2=0x7fa07811b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:23:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.857+0000 7fa080562700 1 -- 172.21.15.19:0/555105816 shutdown_connections 2024-05-09T02:23:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:23:57.857+0000 7fa080562700 1 -- 172.21.15.19:0/555105816 wait complete. 2024-05-09T02:23:58.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:23:58 smithi196 ceph-mon[34718]: pgmap v395: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 551 op/s 2024-05-09T02:23:58.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:23:58 smithi019 ceph-mon[30151]: pgmap v395: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 551 op/s 2024-05-09T02:23:58.901 INFO:tasks.cephfs.filesystem:scrub status for tag:edb599a0-4b60-4780-9387-fbd95ee94731 - {'path': '/', 'tag': 'edb599a0-4b60-4780-9387-fbd95ee94731', 'options': 'recursive,force'} 2024-05-09T02:23:58.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:23:58 smithi059 ceph-mon[34781]: pgmap v395: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 551 op/s 2024-05-09T02:24:00.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:00 smithi196 ceph-mon[34718]: pgmap v396: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 735 op/s 2024-05-09T02:24:00.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:00 smithi019 ceph-mon[30151]: pgmap v396: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 735 op/s 2024-05-09T02:24:00.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:00 smithi059 ceph-mon[34781]: pgmap v396: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 735 op/s 2024-05-09T02:24:01.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:01 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:24:01.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:01 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:24:01.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:01 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:24:01.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:01 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:24:01.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:01 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:24:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:01 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:24:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:01 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:24:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:01 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:24:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:01 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:24:01.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:01 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:24:01.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:01 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:24:01.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:01 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:24:01.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:01 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:24:01.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:01 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:24:01.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:01 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:24:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 740.0 s, 954.0 tps, lat 100.356 ms stddev 127.000 2024-05-09T02:24:02.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:02 smithi196 ceph-mon[34718]: pgmap v397: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 15 MiB/s wr, 714 op/s 2024-05-09T02:24:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:02 smithi019 ceph-mon[30151]: pgmap v397: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 15 MiB/s wr, 714 op/s 2024-05-09T02:24:02.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:02 smithi059 ceph-mon[34781]: pgmap v397: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 15 MiB/s wr, 714 op/s 2024-05-09T02:24:04.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:04 smithi196 ceph-mon[34718]: pgmap v398: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 595 op/s 2024-05-09T02:24:04.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:04 smithi019 ceph-mon[30151]: pgmap v398: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 595 op/s 2024-05-09T02:24:04.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:04 smithi059 ceph-mon[34781]: pgmap v398: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 595 op/s 2024-05-09T02:24:06.768 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:06 smithi059 ceph-mon[34781]: pgmap v399: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 15 MiB/s wr, 761 op/s 2024-05-09T02:24:06.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:06 smithi196 ceph-mon[34718]: pgmap v399: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 15 MiB/s wr, 761 op/s 2024-05-09T02:24:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:06 smithi019 ceph-mon[30151]: pgmap v399: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 15 MiB/s wr, 761 op/s 2024-05-09T02:24:07.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 745.0 s, 198.4 tps, lat 348.492 ms stddev 214.527 2024-05-09T02:24:07.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:07 smithi196 ceph-mon[34718]: osdmap e95: 12 total, 12 up, 12 in 2024-05-09T02:24:07.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:07 smithi019 ceph-mon[30151]: osdmap e95: 12 total, 12 up, 12 in 2024-05-09T02:24:07.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:07 smithi059 ceph-mon[34781]: osdmap e95: 12 total, 12 up, 12 in 2024-05-09T02:24:08.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:08 smithi196 ceph-mon[34718]: pgmap v401: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 11 MiB/s wr, 595 op/s 2024-05-09T02:24:08.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:08 smithi019 ceph-mon[30151]: pgmap v401: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 11 MiB/s wr, 595 op/s 2024-05-09T02:24:08.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:08 smithi059 ceph-mon[34781]: pgmap v401: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 11 MiB/s wr, 595 op/s 2024-05-09T02:24:10.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:10 smithi196 ceph-mon[34718]: pgmap v402: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-05-09T02:24:10.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:10 smithi019 ceph-mon[30151]: pgmap v402: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-05-09T02:24:10.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:10 smithi059 ceph-mon[34781]: pgmap v402: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-05-09T02:24:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 750.0 s, 1699.8 tps, lat 79.640 ms stddev 221.547 2024-05-09T02:24:12.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:12 smithi019 ceph-mon[30151]: pgmap v403: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 MiB/s wr, 1.52k op/s 2024-05-09T02:24:12.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:12 smithi059 ceph-mon[34781]: pgmap v403: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 MiB/s wr, 1.52k op/s 2024-05-09T02:24:13.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:12 smithi196 ceph-mon[34718]: pgmap v403: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 MiB/s wr, 1.52k op/s 2024-05-09T02:24:14.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:14 smithi019 ceph-mon[30151]: pgmap v404: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 MiB/s wr, 1.52k op/s 2024-05-09T02:24:14.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:14 smithi059 ceph-mon[34781]: pgmap v404: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 MiB/s wr, 1.52k op/s 2024-05-09T02:24:15.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:14 smithi196 ceph-mon[34718]: pgmap v404: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 MiB/s wr, 1.52k op/s 2024-05-09T02:24:15.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:15 smithi059 ceph-mon[34781]: osdmap e96: 12 total, 12 up, 12 in 2024-05-09T02:24:16.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:15 smithi196 ceph-mon[34718]: osdmap e96: 12 total, 12 up, 12 in 2024-05-09T02:24:16.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:15 smithi019 ceph-mon[30151]: osdmap e96: 12 total, 12 up, 12 in 2024-05-09T02:24:16.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:16 smithi059 ceph-mon[34781]: pgmap v406: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 32 MiB/s wr, 1.89k op/s 2024-05-09T02:24:17.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:16 smithi196 ceph-mon[34718]: pgmap v406: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 32 MiB/s wr, 1.89k op/s 2024-05-09T02:24:17.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:16 smithi019 ceph-mon[30151]: pgmap v406: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 32 MiB/s wr, 1.89k op/s 2024-05-09T02:24:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 755.0 s, 1570.8 tps, lat 62.880 ms stddev 75.527 2024-05-09T02:24:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:17 smithi059 ceph-mon[34781]: pgmap v407: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 28 MiB/s wr, 1.68k op/s 2024-05-09T02:24:18.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:17 smithi196 ceph-mon[34718]: pgmap v407: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 28 MiB/s wr, 1.68k op/s 2024-05-09T02:24:18.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:17 smithi019 ceph-mon[30151]: pgmap v407: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 28 MiB/s wr, 1.68k op/s 2024-05-09T02:24:20.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:20 smithi059 ceph-mon[34781]: pgmap v408: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 19 MiB/s wr, 1.11k op/s 2024-05-09T02:24:20.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:20 smithi196 ceph-mon[34718]: pgmap v408: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 19 MiB/s wr, 1.11k op/s 2024-05-09T02:24:20.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:20 smithi019 ceph-mon[30151]: pgmap v408: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 19 MiB/s wr, 1.11k op/s 2024-05-09T02:24:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 760.0 s, 1587.6 tps, lat 63.667 ms stddev 72.226 2024-05-09T02:24:22.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:22 smithi196 ceph-mon[34718]: pgmap v409: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 655 op/s 2024-05-09T02:24:22.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:22 smithi019 ceph-mon[30151]: pgmap v409: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 655 op/s 2024-05-09T02:24:22.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:22 smithi059 ceph-mon[34781]: pgmap v409: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 655 op/s 2024-05-09T02:24:24.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:24 smithi196 ceph-mon[34718]: pgmap v410: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 655 op/s 2024-05-09T02:24:24.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:24 smithi019 ceph-mon[30151]: pgmap v410: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 655 op/s 2024-05-09T02:24:24.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:24 smithi059 ceph-mon[34781]: pgmap v410: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 655 op/s 2024-05-09T02:24:26.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:26 smithi196 ceph-mon[34718]: pgmap v411: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 656 op/s 2024-05-09T02:24:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:26 smithi019 ceph-mon[30151]: pgmap v411: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 656 op/s 2024-05-09T02:24:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:26 smithi059 ceph-mon[34781]: pgmap v411: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 656 op/s 2024-05-09T02:24:27.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 765.0 s, 1567.6 tps, lat 64.060 ms stddev 79.626 2024-05-09T02:24:28.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:28 smithi196 ceph-mon[34718]: pgmap v412: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.6 MiB/s wr, 525 op/s 2024-05-09T02:24:28.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:28 smithi019 ceph-mon[30151]: pgmap v412: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.6 MiB/s wr, 525 op/s 2024-05-09T02:24:28.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:24:28.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:28 smithi059 ceph-mon[34781]: pgmap v412: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.6 MiB/s wr, 525 op/s 2024-05-09T02:24:29.315 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:24:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:30 smithi196 ceph-mon[34718]: pgmap v413: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 10 MiB/s wr, 698 op/s 2024-05-09T02:24:30.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:30 smithi019 ceph-mon[30151]: pgmap v413: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 10 MiB/s wr, 698 op/s 2024-05-09T02:24:30.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:30 smithi059 ceph-mon[34781]: pgmap v413: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 10 MiB/s wr, 698 op/s 2024-05-09T02:24:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.395+0000 7f95fdf9f700 1 -- 172.21.15.19:0/1524414924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f8105be0 msgr2=0x7f95f810e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.395+0000 7f95fdf9f700 1 --2- 172.21.15.19:0/1524414924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f8105be0 0x7f95f810e130 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009990 tx=0x7f95e4009ca0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.395+0000 7f95fdf9f700 1 -- 172.21.15.19:0/1524414924 shutdown_connections 2024-05-09T02:24:31.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.395+0000 7f95fdf9f700 1 --2- 172.21.15.19:0/1524414924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f810e700 0x7f95f8110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.395+0000 7f95fdf9f700 1 --2- 172.21.15.19:0/1524414924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95f8105290 0x7f95f81056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.395+0000 7f95fdf9f700 1 --2- 172.21.15.19:0/1524414924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f8105be0 0x7f95f810e130 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.395+0000 7f95fdf9f700 1 -- 172.21.15.19:0/1524414924 >> 172.21.15.19:0/1524414924 conn(0x7f95f8078ac0 msgr2=0x7f95f8078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:31.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.395+0000 7f95fdf9f700 1 -- 172.21.15.19:0/1524414924 shutdown_connections 2024-05-09T02:24:31.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.396+0000 7f95fdf9f700 1 -- 172.21.15.19:0/1524414924 wait complete. 2024-05-09T02:24:31.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.397+0000 7f95fdf9f700 1 Processor -- start 2024-05-09T02:24:31.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.397+0000 7f95fdf9f700 1 -- start start 2024-05-09T02:24:31.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.397+0000 7f95fdf9f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f8105290 0x7f95f81a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.397+0000 7f95fdf9f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f8105be0 0x7f95f81a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.397+0000 7f95fdf9f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95f810e700 0x7f95f819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.397+0000 7f95fdf9f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95f81142b0 con 0x7f95f8105290 2024-05-09T02:24:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.397+0000 7f95fdf9f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f95f8114130 con 0x7f95f8105be0 2024-05-09T02:24:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.397+0000 7f95fdf9f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95f8114430 con 0x7f95f810e700 2024-05-09T02:24:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.397+0000 7f95fcf9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f8105290 0x7f95f81a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.397+0000 7f95fcf9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f8105290 0x7f95f81a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48582/0 (socket says 172.21.15.19:48582) 2024-05-09T02:24:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.397+0000 7f95f7fff700 1 --2- 172.21.15.19:0/637215191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f8105be0 0x7f95f81a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.397+0000 7f95fcf9d700 1 -- 172.21.15.19:0/637215191 learned_addr learned my addr 172.21.15.19:0/637215191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.398+0000 7f95fd79e700 1 --2- 172.21.15.19:0/637215191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95f810e700 0x7f95f819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.398+0000 7f95f7fff700 1 -- 172.21.15.19:0/637215191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95f810e700 msgr2=0x7f95f819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.398+0000 7f95f7fff700 1 --2- 172.21.15.19:0/637215191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95f810e700 0x7f95f819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.398+0000 7f95f7fff700 1 -- 172.21.15.19:0/637215191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f8105290 msgr2=0x7f95f81a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.398+0000 7f95f7fff700 1 --2- 172.21.15.19:0/637215191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f8105290 0x7f95f81a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.398+0000 7f95f7fff700 1 -- 172.21.15.19:0/637215191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f819b260 con 0x7f95f8105be0 2024-05-09T02:24:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.398+0000 7f95fd79e700 1 --2- 172.21.15.19:0/637215191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95f810e700 0x7f95f819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:24:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.398+0000 7f95f7fff700 1 --2- 172.21.15.19:0/637215191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f8105be0 0x7f95f81a0ee0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f95e4000c00 tx=0x7f95e40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.398+0000 7f95f5ffb700 1 -- 172.21.15.19:0/637215191 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e402f070 con 0x7f95f8105be0 2024-05-09T02:24:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.398+0000 7f95f5ffb700 1 -- 172.21.15.19:0/637215191 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e402a470 con 0x7f95f8105be0 2024-05-09T02:24:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.398+0000 7f95f5ffb700 1 -- 172.21.15.19:0/637215191 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e400e640 con 0x7f95f8105be0 2024-05-09T02:24:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.398+0000 7f95fdf9f700 1 -- 172.21.15.19:0/637215191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f819b4f0 con 0x7f95f8105be0 2024-05-09T02:24:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.399+0000 7f95fdf9f700 1 -- 172.21.15.19:0/637215191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f819b980 con 0x7f95f8105be0 2024-05-09T02:24:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.403+0000 7f95f5ffb700 1 -- 172.21.15.19:0/637215191 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f95e4032070 con 0x7f95f8105be0 2024-05-09T02:24:31.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.404+0000 7f960dc17b80 1 Processor -- start 2024-05-09T02:24:31.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.404+0000 7f95f5ffb700 1 --2- 172.21.15.19:0/637215191 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f95e80b2a70 0x7f95e80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.404+0000 7f960dc17b80 1 -- start start 2024-05-09T02:24:31.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.405+0000 7f95fd79e700 1 --2- 172.21.15.19:0/637215191 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f95e80b2a70 0x7f95e80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.405+0000 7f95f5ffb700 1 -- 172.21.15.19:0/637215191 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f95e40ec1f0 con 0x7f95f8105be0 2024-05-09T02:24:31.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.405+0000 7f960dc17b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29a27160 0x556b29ac0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.405+0000 7f960dc17b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2993d4a0 0x556b29ac0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.405+0000 7f960dc17b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29aba800 0x556b29abac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.405+0000 7f960dc17b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556b299dcff0 con 0x556b29aba800 2024-05-09T02:24:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.405+0000 7f960dc17b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556b2997b7f0 con 0x556b29a27160 2024-05-09T02:24:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.405+0000 7f960dc17b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556b299d8fb0 con 0x556b2993d4a0 2024-05-09T02:24:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.405+0000 7f95f7fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2993d4a0 0x556b29ac0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.405+0000 7f95f7fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2993d4a0 0x556b29ac0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:42164/0 (socket says 172.21.15.19:42164) 2024-05-09T02:24:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.405+0000 7f95fcf9d700 1 --2- 172.21.15.19:0/3621052427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29a27160 0x556b29ac0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.405+0000 7f95f7fff700 1 -- 172.21.15.19:0/3621052427 learned_addr learned my addr 172.21.15.19:0/3621052427 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.406+0000 7f95fd79e700 1 --2- 172.21.15.19:0/637215191 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f95e80b2a70 0x7f95e80b4f80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f95f0003e30 tx=0x7f95f000b040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.406+0000 7f95fd79e700 1 --2- 172.21.15.19:0/3621052427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29aba800 0x556b29abac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.406+0000 7f95f7fff700 1 -- 172.21.15.19:0/3621052427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29a27160 msgr2=0x556b29ac0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.406+0000 7f95f7fff700 1 --2- 172.21.15.19:0/3621052427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29a27160 0x556b29ac0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.406+0000 7f95f7fff700 1 -- 172.21.15.19:0/3621052427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29aba800 msgr2=0x556b29abac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.406+0000 7f95f7fff700 1 --2- 172.21.15.19:0/3621052427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29aba800 0x556b29abac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.406+0000 7f95f7fff700 1 -- 172.21.15.19:0/3621052427 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b29973370 con 0x556b2993d4a0 2024-05-09T02:24:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.406+0000 7f95fcf9d700 1 --2- 172.21.15.19:0/3621052427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29a27160 0x556b29ac0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:24:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.406+0000 7f95fd79e700 1 --2- 172.21.15.19:0/3621052427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29aba800 0x556b29abac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:24:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.406+0000 7f95f7fff700 1 --2- 172.21.15.19:0/3621052427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2993d4a0 0x556b29ac0b80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f95e40af550 tx=0x7f95e40b3060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56928cac845a4ec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.406+0000 7f95e2ffd700 1 -- 172.21.15.19:0/3621052427 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e40b7880 con 0x556b2993d4a0 2024-05-09T02:24:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.406+0000 7f95e2ffd700 1 -- 172.21.15.19:0/3621052427 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e40bc2a0 con 0x556b2993d4a0 2024-05-09T02:24:31.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.406+0000 7f95e2ffd700 1 -- 172.21.15.19:0/3621052427 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e40b3930 con 0x556b2993d4a0 2024-05-09T02:24:31.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.407+0000 7f960dc17b80 1 -- 172.21.15.19:0/3621052427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2993d4a0 msgr2=0x556b29ac0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:31.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.407+0000 7f960dc17b80 1 --2- 172.21.15.19:0/3621052427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2993d4a0 0x556b29ac0b80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f95e40af550 tx=0x7f95e40b3060 comp rx=0 tx=0).stop 2024-05-09T02:24:31.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.407+0000 7f960dc17b80 1 -- 172.21.15.19:0/3621052427 shutdown_connections 2024-05-09T02:24:31.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.407+0000 7f960dc17b80 1 --2- 172.21.15.19:0/3621052427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29aba800 0x556b29abac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.407+0000 7f960dc17b80 1 --2- 172.21.15.19:0/3621052427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29a27160 0x556b29ac0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.407+0000 7f960dc17b80 1 --2- 172.21.15.19:0/3621052427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2993d4a0 0x556b29ac0b80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.407+0000 7f960dc17b80 1 -- 172.21.15.19:0/3621052427 >> 172.21.15.19:0/3621052427 conn(0x556b2999d6a0 msgr2=0x556b29ab5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:31.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.407+0000 7f960dc17b80 1 -- 172.21.15.19:0/3621052427 shutdown_connections 2024-05-09T02:24:31.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.408+0000 7f960dc17b80 1 -- 172.21.15.19:0/3621052427 wait complete. 2024-05-09T02:24:31.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.408+0000 7f960dc17b80 1 -- start start 2024-05-09T02:24:31.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.408+0000 7f960dc17b80 1 Processor -- start 2024-05-09T02:24:31.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.409+0000 7f960dc17b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2993d4a0 0x556b29ac9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.409+0000 7f960dc17b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29a27160 0x556b29aca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.409+0000 7f960dc17b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29aba800 0x556b29ac41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.409+0000 7f95f7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29a27160 0x556b29aca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.409+0000 7f95f7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29a27160 0x556b29aca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56732/0 (socket says 172.21.15.19:56732) 2024-05-09T02:24:31.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.409+0000 7f95fd79e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29aba800 0x556b29ac41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.409+0000 7f95fd79e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29aba800 0x556b29ac41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48610/0 (socket says 172.21.15.19:48610) 2024-05-09T02:24:31.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.409+0000 7f95f7fff700 1 -- 172.21.15.19:0/1023884762 learned_addr learned my addr 172.21.15.19:0/1023884762 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:31.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.409+0000 7f960dc17b80 1 -- 172.21.15.19:0/1023884762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556b29858ba0 con 0x556b29aba800 2024-05-09T02:24:31.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.409+0000 7f960dc17b80 1 -- 172.21.15.19:0/1023884762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556b29997150 con 0x556b29a27160 2024-05-09T02:24:31.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.409+0000 7f960dc17b80 1 -- 172.21.15.19:0/1023884762 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556b29abb3b0 con 0x556b2993d4a0 2024-05-09T02:24:31.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.409+0000 7f960dc17b80 10 client.0 _ll_register_callbacks cb 0x556b2984e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:24:31.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.410+0000 7f95fcf9d700 1 --2- 172.21.15.19:0/1023884762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2993d4a0 0x556b29ac9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.410+0000 7f95f7fff700 1 -- 172.21.15.19:0/1023884762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2993d4a0 msgr2=0x556b29ac9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:31.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.410+0000 7f95f7fff700 1 --2- 172.21.15.19:0/1023884762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2993d4a0 0x556b29ac9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.410+0000 7f95f7fff700 1 -- 172.21.15.19:0/1023884762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29aba800 msgr2=0x556b29ac41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:31.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.410+0000 7f95f7fff700 1 --2- 172.21.15.19:0/1023884762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29aba800 0x556b29ac41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.410+0000 7f95f7fff700 1 -- 172.21.15.19:0/1023884762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b29858490 con 0x556b29a27160 2024-05-09T02:24:31.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.411+0000 7f95f7fff700 1 --2- 172.21.15.19:0/1023884762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29a27160 0x556b29aca2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f95e40af500 tx=0x7f95e40b3960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e35632ead2fc729c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:31.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.411+0000 7f95d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:24:31.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.411+0000 7f95d7fff700 1 -- 172.21.15.19:0/1023884762 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e40b7880 con 0x556b29a27160 2024-05-09T02:24:31.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.411+0000 7f95d7fff700 1 -- 172.21.15.19:0/1023884762 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f95e40afac0 con 0x556b29a27160 2024-05-09T02:24:31.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.411+0000 7f960dc17b80 1 -- 172.21.15.19:0/1023884762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e40bc2a0 con 0x556b29a27160 2024-05-09T02:24:31.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.411+0000 7f95d7fff700 1 -- 172.21.15.19:0/1023884762 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f95e40bc020 con 0x556b29a27160 2024-05-09T02:24:31.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.411+0000 7f960dc17b80 10 client.26642 fetch_fsmap learned FSMap version 58 2024-05-09T02:24:31.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.411+0000 7f960dc17b80 1 -- 172.21.15.19:0/1023884762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556b29ad17c0 con 0x556b29a27160 2024-05-09T02:24:31.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.411+0000 7f95d7fff700 1 -- 172.21.15.19:0/1023884762 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e40bc2a0 con 0x556b29a27160 2024-05-09T02:24:31.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.412+0000 7f95d7fff700 1 -- 172.21.15.19:0/1023884762 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f95e40bab70 con 0x556b29a27160 2024-05-09T02:24:31.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.412+0000 7f960dc17b80 10 client.26642 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:24:31.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.412+0000 7f960dc17b80 10 client.26642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:24:31.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.412+0000 7f960dc17b80 20 client.26642 populate_metadata read hostname 'smithi019' 2024-05-09T02:24:31.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.412+0000 7f960dc17b80 1 --2- 172.21.15.19:0/1023884762 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556b29ad3c90 0x556b29ad60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.412+0000 7f960dc17b80 4 client.26642 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:24:31.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.412+0000 7f960dc17b80 1 -- 172.21.15.19:0/1023884762 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b299ea9d0 con 0x556b29ad3c90 2024-05-09T02:24:31.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.412+0000 7f95fcf9d700 1 --2- 172.21.15.19:0/1023884762 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556b29ad3c90 0x556b29ad60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.413+0000 7f95d7fff700 10 client.26642 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:24:31.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.413+0000 7f95fcf9d700 1 --2- 172.21.15.19:0/1023884762 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556b29ad3c90 0x556b29ad60b0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd25b7f31a0be829 server_cookie=51b473cef56dc13e in_seq=0 out_seq=0 2024-05-09T02:24:31.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.414+0000 7f95d7fff700 1 -- 172.21.15.19:0/1023884762 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556b299ea9d0 con 0x556b29ad3c90 2024-05-09T02:24:31.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.414+0000 7f95d7fff700 10 client.26642 handle_command_reply: tid=0 2024-05-09T02:24:31.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.414+0000 7f960dc17b80 1 client.26642 shutdown 2024-05-09T02:24:31.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.414+0000 7f960dc17b80 1 -- 172.21.15.19:0/1023884762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29a27160 msgr2=0x556b29aca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:31.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.414+0000 7f960dc17b80 1 --2- 172.21.15.19:0/1023884762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29a27160 0x556b29aca2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f95e40af500 tx=0x7f95e40b3960 comp rx=0 tx=0).stop 2024-05-09T02:24:31.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.414+0000 7f960dc17b80 1 -- 172.21.15.19:0/1023884762 shutdown_connections 2024-05-09T02:24:31.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.414+0000 7f960dc17b80 1 --2- 172.21.15.19:0/1023884762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29aba800 0x556b29ac41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.414+0000 7f960dc17b80 1 --2- 172.21.15.19:0/1023884762 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556b29ad3c90 0x556b29ad60b0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.414+0000 7f960dc17b80 1 --2- 172.21.15.19:0/1023884762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b2993d4a0 0x556b29ac9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.414+0000 7f95d7fff700 0 client.26642 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:24:31.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.414+0000 7f960dc17b80 1 --2- 172.21.15.19:0/1023884762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29a27160 0x556b29aca2f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.414+0000 7f960dc17b80 1 -- 172.21.15.19:0/1023884762 >> 172.21.15.19:0/1023884762 conn(0x556b2999d6a0 msgr2=0x556b29ab5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:31.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.414+0000 7f960dc17b80 1 -- 172.21.15.19:0/1023884762 shutdown_connections 2024-05-09T02:24:31.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.414+0000 7f960dc17b80 1 -- 172.21.15.19:0/1023884762 wait complete. 2024-05-09T02:24:31.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.414+0000 7f960dc17b80 20 client.26642 trim_cache size 0 max 16384 2024-05-09T02:24:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.453+0000 7f960dc17b80 1 Processor -- start 2024-05-09T02:24:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.455+0000 7f960dc17b80 1 -- start start 2024-05-09T02:24:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.456+0000 7f960dc17b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b2993d4a0 0x556b29abe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.456+0000 7f960dc17b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b29a27160 0x556b29abf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.456+0000 7f960dc17b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29aba800 0x556b29abf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.456+0000 7f95fcf9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b2993d4a0 0x556b29abe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.456+0000 7f95fcf9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b2993d4a0 0x556b29abe710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48618/0 (socket says 172.21.15.19:48618) 2024-05-09T02:24:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.456+0000 7f95fcf9d700 1 -- 172.21.15.19:0/3562086384 learned_addr learned my addr 172.21.15.19:0/3562086384 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.456+0000 7f960dc17b80 1 -- 172.21.15.19:0/3562086384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95f81acb00 con 0x556b2993d4a0 2024-05-09T02:24:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.456+0000 7f960dc17b80 1 -- 172.21.15.19:0/3562086384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f95f81ae640 con 0x556b29aba800 2024-05-09T02:24:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.456+0000 7f960dc17b80 1 -- 172.21.15.19:0/3562086384 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95f81ad1d0 con 0x556b29a27160 2024-05-09T02:24:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.456+0000 7f95f7fff700 1 --2- 172.21.15.19:0/3562086384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b29a27160 0x556b29abf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.460+0000 7f95fd79e700 1 --2- 172.21.15.19:0/3562086384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29aba800 0x556b29abf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.463+0000 7f95f7fff700 1 -- 172.21.15.19:0/3562086384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29aba800 msgr2=0x556b29abf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:31.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.463+0000 7f95f7fff700 1 --2- 172.21.15.19:0/3562086384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29aba800 0x556b29abf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.463+0000 7f95f7fff700 1 -- 172.21.15.19:0/3562086384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b2993d4a0 msgr2=0x556b29abe710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:31.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.463+0000 7f95f7fff700 1 --2- 172.21.15.19:0/3562086384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b2993d4a0 0x556b29abe710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.463+0000 7f95f7fff700 1 -- 172.21.15.19:0/3562086384 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b29ab0410 con 0x556b29a27160 2024-05-09T02:24:31.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.467+0000 7f95f7fff700 1 --2- 172.21.15.19:0/3562086384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b29a27160 0x556b29abf2a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009350 tx=0x7f95e402ac30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2255ae4fc4f7b7ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:31.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.470+0000 7f95e27fc700 1 -- 172.21.15.19:0/3562086384 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e40bc2a0 con 0x556b29a27160 2024-05-09T02:24:31.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.471+0000 7f95e27fc700 1 -- 172.21.15.19:0/3562086384 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e40afac0 con 0x556b29a27160 2024-05-09T02:24:31.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.471+0000 7f95e27fc700 1 -- 172.21.15.19:0/3562086384 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e40bab70 con 0x556b29a27160 2024-05-09T02:24:31.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.473+0000 7f960dc17b80 1 -- 172.21.15.19:0/3562086384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b29a27160 msgr2=0x556b29abf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:31.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.473+0000 7f960dc17b80 1 --2- 172.21.15.19:0/3562086384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b29a27160 0x556b29abf2a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009350 tx=0x7f95e402ac30 comp rx=0 tx=0).stop 2024-05-09T02:24:31.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.474+0000 7f960dc17b80 1 -- 172.21.15.19:0/3562086384 shutdown_connections 2024-05-09T02:24:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.474+0000 7f960dc17b80 1 --2- 172.21.15.19:0/3562086384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b29aba800 0x556b29abf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.474+0000 7f960dc17b80 1 --2- 172.21.15.19:0/3562086384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b2993d4a0 0x556b29abe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.474+0000 7f960dc17b80 1 --2- 172.21.15.19:0/3562086384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b29a27160 0x556b29abf2a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.474+0000 7f960dc17b80 1 -- 172.21.15.19:0/3562086384 >> 172.21.15.19:0/3562086384 conn(0x556b2999d6a0 msgr2=0x556b29abe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:31.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.475+0000 7f960dc17b80 1 -- 172.21.15.19:0/3562086384 shutdown_connections 2024-05-09T02:24:31.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.475+0000 7f960dc17b80 1 -- 172.21.15.19:0/3562086384 wait complete. 2024-05-09T02:24:31.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.476+0000 7f960dc17b80 1 -- start start 2024-05-09T02:24:31.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.476+0000 7f960dc17b80 1 Processor -- start 2024-05-09T02:24:31.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.477+0000 7f960dc17b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b2993d4a0 0x556b29b32370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.477+0000 7f960dc17b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29a27160 0x556b29b328b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.477+0000 7f960dc17b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b29aba800 0x556b29b3d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.477+0000 7f960dc17b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95f81adf70 con 0x556b29a27160 2024-05-09T02:24:31.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.477+0000 7f960dc17b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f95f81ad8a0 con 0x556b2993d4a0 2024-05-09T02:24:31.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.477+0000 7f960dc17b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95f81aaae0 con 0x556b29aba800 2024-05-09T02:24:31.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.477+0000 7f960dc17b80 10 client.0 _ll_register_callbacks cb 0x556b2984e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:24:31.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.478+0000 7f95fd79e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b29aba800 0x556b29b3d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.478+0000 7f95fd79e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b29aba800 0x556b29b3d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:42184/0 (socket says 172.21.15.19:42184) 2024-05-09T02:24:31.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.478+0000 7f95fd79e700 1 -- 172.21.15.19:0/3295917141 learned_addr learned my addr 172.21.15.19:0/3295917141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:31.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.478+0000 7f95f7fff700 1 --2- 172.21.15.19:0/3295917141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29a27160 0x556b29b328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.478+0000 7f95fcf9d700 1 --2- 172.21.15.19:0/3295917141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b2993d4a0 0x556b29b32370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.480+0000 7f95fd79e700 1 -- 172.21.15.19:0/3295917141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b2993d4a0 msgr2=0x556b29b32370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:31.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.480+0000 7f95fd79e700 1 --2- 172.21.15.19:0/3295917141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b2993d4a0 0x556b29b32370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.480+0000 7f95fd79e700 1 -- 172.21.15.19:0/3295917141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29a27160 msgr2=0x556b29b328b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:31.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.480+0000 7f95fd79e700 1 --2- 172.21.15.19:0/3295917141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29a27160 0x556b29b328b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.480+0000 7f95fd79e700 1 -- 172.21.15.19:0/3295917141 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b29abec50 con 0x556b29aba800 2024-05-09T02:24:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.481+0000 7f95fd79e700 1 --2- 172.21.15.19:0/3295917141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b29aba800 0x556b29b3d1f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f95f000ef70 tx=0x7f95f0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac4a33df4600d960 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.481+0000 7f95d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:24:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.481+0000 7f95d7fff700 1 -- 172.21.15.19:0/3295917141 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0017e60 con 0x556b29aba800 2024-05-09T02:24:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.481+0000 7f95d7fff700 1 -- 172.21.15.19:0/3295917141 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f95f0009020 con 0x556b29aba800 2024-05-09T02:24:31.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.482+0000 7f960dc17b80 1 -- 172.21.15.19:0/3295917141 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e40afac0 con 0x556b29aba800 2024-05-09T02:24:31.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.487+0000 7f95d7fff700 1 -- 172.21.15.19:0/3295917141 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f95f0009220 con 0x556b29aba800 2024-05-09T02:24:31.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.488+0000 7f960dc17b80 10 client.26686 fetch_fsmap learned FSMap version 58 2024-05-09T02:24:31.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.488+0000 7f960dc17b80 1 -- 172.21.15.19:0/3295917141 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556b29b3e7b0 con 0x556b29aba800 2024-05-09T02:24:31.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.489+0000 7f95d7fff700 1 -- 172.21.15.19:0/3295917141 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e40afac0 con 0x556b29aba800 2024-05-09T02:24:31.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.489+0000 7f95d7fff700 1 -- 172.21.15.19:0/3295917141 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f95f0020070 con 0x556b29aba800 2024-05-09T02:24:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.489+0000 7f960dc17b80 10 client.26686 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:24:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.489+0000 7f960dc17b80 10 client.26686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:24:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.489+0000 7f960dc17b80 20 client.26686 populate_metadata read hostname 'smithi019' 2024-05-09T02:24:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.489+0000 7f960dc17b80 1 --2- 172.21.15.19:0/3295917141 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556b29ad3c90 0x556b29b40f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.489+0000 7f960dc17b80 4 client.26686 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:24:31.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.490+0000 7f960dc17b80 1 -- 172.21.15.19:0/3295917141 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b299e4a70 con 0x556b29ad3c90 2024-05-09T02:24:31.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.491+0000 7f95fcf9d700 1 --2- 172.21.15.19:0/3295917141 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556b29ad3c90 0x556b29b40f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:31.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.493+0000 7f95fcf9d700 1 --2- 172.21.15.19:0/3295917141 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556b29ad3c90 0x556b29b40f10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0dc2661e74a52ff server_cookie=b56ca7fb0b60c6d7 in_seq=0 out_seq=0 2024-05-09T02:24:31.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.493+0000 7f95d7fff700 10 client.26686 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:24:31.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.493+0000 7f95d7fff700 1 -- 172.21.15.19:0/3295917141 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556b299e4a70 con 0x556b29ad3c90 2024-05-09T02:24:31.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.493+0000 7f95d7fff700 10 client.26686 handle_command_reply: tid=0 2024-05-09T02:24:31.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.493+0000 7f960dc17b80 1 client.26686 shutdown 2024-05-09T02:24:31.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.494+0000 7f960dc17b80 1 -- 172.21.15.19:0/3295917141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b29aba800 msgr2=0x556b29b3d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:31.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.494+0000 7f960dc17b80 1 --2- 172.21.15.19:0/3295917141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b29aba800 0x556b29b3d1f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f95f000ef70 tx=0x7f95f0004ad0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.496+0000 7f960dc17b80 1 -- 172.21.15.19:0/3295917141 shutdown_connections 2024-05-09T02:24:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.496+0000 7f960dc17b80 1 --2- 172.21.15.19:0/3295917141 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556b29ad3c90 0x556b29b40f10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.496+0000 7f960dc17b80 1 --2- 172.21.15.19:0/3295917141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b29aba800 0x556b29b3d1f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.496+0000 7f960dc17b80 1 --2- 172.21.15.19:0/3295917141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556b2993d4a0 0x556b29b32370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.496+0000 7f960dc17b80 1 --2- 172.21.15.19:0/3295917141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b29a27160 0x556b29b328b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.496+0000 7f95d7fff700 0 client.26686 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:24:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.496+0000 7f960dc17b80 1 -- 172.21.15.19:0/3295917141 >> 172.21.15.19:0/3295917141 conn(0x556b2999d6a0 msgr2=0x556b29ac3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:31.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.497+0000 7f960dc17b80 1 -- 172.21.15.19:0/3295917141 shutdown_connections 2024-05-09T02:24:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.497+0000 7f960dc17b80 1 -- 172.21.15.19:0/3295917141 wait complete. 2024-05-09T02:24:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.497+0000 7f960dc17b80 20 client.26686 trim_cache size 0 max 16384 2024-05-09T02:24:31.502 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:24:31.502 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T02:24:31.502 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T02:24:31.502 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:24:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.499+0000 7f95fdf9f700 1 -- 172.21.15.19:0/637215191 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f95e80b2a70 msgr2=0x7f95e80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.499+0000 7f95fdf9f700 1 --2- 172.21.15.19:0/637215191 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f95e80b2a70 0x7f95e80b4f80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f95f0003e30 tx=0x7f95f000b040 comp rx=0 tx=0).stop 2024-05-09T02:24:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.499+0000 7f95fdf9f700 1 -- 172.21.15.19:0/637215191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f8105be0 msgr2=0x7f95f81a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.499+0000 7f95fdf9f700 1 --2- 172.21.15.19:0/637215191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f8105be0 0x7f95f81a0ee0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f95e4000c00 tx=0x7f95e40047b0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.499+0000 7f95fdf9f700 1 -- 172.21.15.19:0/637215191 shutdown_connections 2024-05-09T02:24:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.499+0000 7f95fdf9f700 1 --2- 172.21.15.19:0/637215191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95f810e700 0x7f95f819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.499+0000 7f95fdf9f700 1 --2- 172.21.15.19:0/637215191 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f95e80b2a70 0x7f95e80b4f80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.499+0000 7f95fdf9f700 1 --2- 172.21.15.19:0/637215191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f8105290 0x7f95f81a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.499+0000 7f95fdf9f700 1 --2- 172.21.15.19:0/637215191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f8105be0 0x7f95f81a0ee0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:31.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.499+0000 7f95fdf9f700 1 -- 172.21.15.19:0/637215191 >> 172.21.15.19:0/637215191 conn(0x7f95f8078ac0 msgr2=0x7f95f8100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:31.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.499+0000 7f95fdf9f700 1 -- 172.21.15.19:0/637215191 shutdown_connections 2024-05-09T02:24:31.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:31.499+0000 7f95fdf9f700 1 -- 172.21.15.19:0/637215191 wait complete. 2024-05-09T02:24:32.495 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T02:24:32.495 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:24:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 770.0 s, 1227.2 tps, lat 81.533 ms stddev 129.483 2024-05-09T02:24:32.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:32 smithi196 ceph-mon[34718]: pgmap v414: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 687 op/s 2024-05-09T02:24:32.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:32 smithi019 ceph-mon[30151]: pgmap v414: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 687 op/s 2024-05-09T02:24:32.912 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:24:32.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:32 smithi059 ceph-mon[34781]: pgmap v414: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 687 op/s 2024-05-09T02:24:34.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:34 smithi196 ceph-mon[34718]: pgmap v415: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.6 MiB/s wr, 587 op/s 2024-05-09T02:24:34.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:34 smithi019 ceph-mon[30151]: pgmap v415: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.6 MiB/s wr, 587 op/s 2024-05-09T02:24:34.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:34 smithi059 ceph-mon[34781]: pgmap v415: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.6 MiB/s wr, 587 op/s 2024-05-09T02:24:35.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.394+0000 7f1a77a89700 1 -- 172.21.15.19:0/1240822580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7010b280 msgr2=0x7f1a7011d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:35.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.394+0000 7f1a77a89700 1 --2- 172.21.15.19:0/1240822580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7010b280 0x7f1a7011d3f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1a6001a9a0 tx=0x7f1a6001acb0 comp rx=0 tx=0).stop 2024-05-09T02:24:35.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.396+0000 7f1a77a89700 1 -- 172.21.15.19:0/1240822580 shutdown_connections 2024-05-09T02:24:35.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.396+0000 7f1a77a89700 1 --2- 172.21.15.19:0/1240822580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7011d930 0x7f1a7011fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:35.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.396+0000 7f1a77a89700 1 --2- 172.21.15.19:0/1240822580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a7010a930 0x7f1a7010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:35.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.396+0000 7f1a77a89700 1 --2- 172.21.15.19:0/1240822580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7010b280 0x7f1a7011d3f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:35.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.396+0000 7f1a77a89700 1 -- 172.21.15.19:0/1240822580 >> 172.21.15.19:0/1240822580 conn(0x7f1a7006f4d0 msgr2=0x7f1a7006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:35.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.397+0000 7f1a77a89700 1 -- 172.21.15.19:0/1240822580 shutdown_connections 2024-05-09T02:24:35.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.398+0000 7f1a77a89700 1 -- 172.21.15.19:0/1240822580 wait complete. 2024-05-09T02:24:35.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.399+0000 7f1a77a89700 1 Processor -- start 2024-05-09T02:24:35.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.399+0000 7f1a77a89700 1 -- start start 2024-05-09T02:24:35.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.400+0000 7f1a77a89700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a7010a930 0x7f1a7011d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:35.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.400+0000 7f1a77a89700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7010b280 0x7f1a70119330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:35.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.400+0000 7f1a77a89700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7011d930 0x7f1a70119870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:35.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.400+0000 7f1a77a89700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a701154b0 con 0x7f1a7011d930 2024-05-09T02:24:35.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.400+0000 7f1a77a89700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a70115330 con 0x7f1a7010b280 2024-05-09T02:24:35.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.400+0000 7f1a77a89700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a70115630 con 0x7f1a7010a930 2024-05-09T02:24:35.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.400+0000 7f1a75024700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7010b280 0x7f1a70119330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:35.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.400+0000 7f1a75024700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7010b280 0x7f1a70119330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:56784/0 (socket says 172.21.15.19:56784) 2024-05-09T02:24:35.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.400+0000 7f1a75024700 1 -- 172.21.15.19:0/2176888208 learned_addr learned my addr 172.21.15.19:0/2176888208 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:35.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.401+0000 7f1a76026700 1 --2- 172.21.15.19:0/2176888208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7011d930 0x7f1a70119870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:35.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.401+0000 7f1a75024700 1 -- 172.21.15.19:0/2176888208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a7010a930 msgr2=0x7f1a7011d1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T02:24:35.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.401+0000 7f1a75024700 1 --2- 172.21.15.19:0/2176888208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a7010a930 0x7f1a7011d1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:35.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.401+0000 7f1a75024700 1 -- 172.21.15.19:0/2176888208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7011d930 msgr2=0x7f1a70119870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:35.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.401+0000 7f1a75024700 1 --2- 172.21.15.19:0/2176888208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7011d930 0x7f1a70119870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:35.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.401+0000 7f1a75024700 1 -- 172.21.15.19:0/2176888208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7011a000 con 0x7f1a7010b280 2024-05-09T02:24:35.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.401+0000 7f1a76026700 1 --2- 172.21.15.19:0/2176888208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7011d930 0x7f1a70119870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:24:35.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.401+0000 7f1a75024700 1 --2- 172.21.15.19:0/2176888208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7010b280 0x7f1a70119330 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1a60004430 tx=0x7f1a60004460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:35.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.402+0000 7f1a66ffd700 1 -- 172.21.15.19:0/2176888208 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a60036070 con 0x7f1a7010b280 2024-05-09T02:24:35.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.402+0000 7f1a77a89700 1 -- 172.21.15.19:0/2176888208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a701ad8c0 con 0x7f1a7010b280 2024-05-09T02:24:35.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.402+0000 7f1a66ffd700 1 -- 172.21.15.19:0/2176888208 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a60028ab0 con 0x7f1a7010b280 2024-05-09T02:24:35.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.402+0000 7f1a66ffd700 1 -- 172.21.15.19:0/2176888208 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a600317e0 con 0x7f1a7010b280 2024-05-09T02:24:35.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.402+0000 7f1a77a89700 1 -- 172.21.15.19:0/2176888208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a701add70 con 0x7f1a7010b280 2024-05-09T02:24:35.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.403+0000 7f1a77a89700 1 -- 172.21.15.19:0/2176888208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a7010c070 con 0x7f1a7010b280 2024-05-09T02:24:35.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.408+0000 7f1a66ffd700 1 -- 172.21.15.19:0/2176888208 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1a60003830 con 0x7f1a7010b280 2024-05-09T02:24:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.408+0000 7f1a66ffd700 1 --2- 172.21.15.19:0/2176888208 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1a5c0b2ab0 0x7f1a5c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.409+0000 7f1a66ffd700 1 -- 172.21.15.19:0/2176888208 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f1a600f4040 con 0x7f1a7010b280 2024-05-09T02:24:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.409+0000 7f1a75825700 1 --2- 172.21.15.19:0/2176888208 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1a5c0b2ab0 0x7f1a5c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.409+0000 7f1a75825700 1 --2- 172.21.15.19:0/2176888208 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1a5c0b2ab0 0x7f1a5c0b4fc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009710 tx=0x7f1a6c006c60 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:35.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.410+0000 7f1a66ffd700 1 -- 172.21.15.19:0/2176888208 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1a60024070 con 0x7f1a7010b280 2024-05-09T02:24:35.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.822+0000 7f1a77a89700 1 -- 172.21.15.19:0/2176888208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a701ae000 con 0x7f1a7010b280 2024-05-09T02:24:35.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.822+0000 7f1a66ffd700 1 -- 172.21.15.19:0/2176888208 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11917 (secure 0 0 0) 0x7f1a600bbb00 con 0x7f1a7010b280 2024-05-09T02:24:35.825 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:24:35.825 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:22:01.074232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24457,24455,24473,24463,24484]},"id":1}]} 2024-05-09T02:24:35.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.825+0000 7f1a77a89700 1 -- 172.21.15.19:0/2176888208 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1a5c0b2ab0 msgr2=0x7f1a5c0b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:35.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.825+0000 7f1a77a89700 1 --2- 172.21.15.19:0/2176888208 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1a5c0b2ab0 0x7f1a5c0b4fc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009710 tx=0x7f1a6c006c60 comp rx=0 tx=0).stop 2024-05-09T02:24:35.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.825+0000 7f1a77a89700 1 -- 172.21.15.19:0/2176888208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7010b280 msgr2=0x7f1a70119330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:35.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.825+0000 7f1a77a89700 1 --2- 172.21.15.19:0/2176888208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7010b280 0x7f1a70119330 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1a60004430 tx=0x7f1a60004460 comp rx=0 tx=0).stop 2024-05-09T02:24:35.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.825+0000 7f1a77a89700 1 -- 172.21.15.19:0/2176888208 shutdown_connections 2024-05-09T02:24:35.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.825+0000 7f1a77a89700 1 --2- 172.21.15.19:0/2176888208 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1a5c0b2ab0 0x7f1a5c0b4fc0 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009710 tx=0x7f1a6c006c60 comp rx=0 tx=0).stop 2024-05-09T02:24:35.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.825+0000 7f1a77a89700 1 --2- 172.21.15.19:0/2176888208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7011d930 0x7f1a70119870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:35.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.825+0000 7f1a77a89700 1 --2- 172.21.15.19:0/2176888208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a7010a930 0x7f1a7011d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:35.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.825+0000 7f1a77a89700 1 --2- 172.21.15.19:0/2176888208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7010b280 0x7f1a70119330 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:35.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.825+0000 7f1a77a89700 1 -- 172.21.15.19:0/2176888208 >> 172.21.15.19:0/2176888208 conn(0x7f1a7006f4d0 msgr2=0x7f1a7011de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:35.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.826+0000 7f1a77a89700 1 -- 172.21.15.19:0/2176888208 shutdown_connections 2024-05-09T02:24:35.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:35.826+0000 7f1a77a89700 1 -- 172.21.15.19:0/2176888208 wait complete. 2024-05-09T02:24:35.830 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-05-09T02:24:36.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:36 smithi196 ceph-mon[34718]: pgmap v416: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 24 MiB/s wr, 1.82k op/s 2024-05-09T02:24:36.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:36 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2176888208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:24:36.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:36 smithi019 ceph-mon[30151]: pgmap v416: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 24 MiB/s wr, 1.82k op/s 2024-05-09T02:24:36.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:36 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2176888208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:24:36.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:36 smithi059 ceph-mon[34781]: pgmap v416: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 24 MiB/s wr, 1.82k op/s 2024-05-09T02:24:36.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:36 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2176888208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:24:37.043 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T02:24:37.372 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:24:37.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 775.0 s, 1535.6 tps, lat 64.736 ms stddev 78.430 2024-05-09T02:24:38.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:38 smithi196 ceph-mon[34718]: pgmap v417: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 20 MiB/s wr, 1.54k op/s 2024-05-09T02:24:38.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:38 smithi019 ceph-mon[30151]: pgmap v417: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 20 MiB/s wr, 1.54k op/s 2024-05-09T02:24:38.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:38 smithi059 ceph-mon[34781]: pgmap v417: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 20 MiB/s wr, 1.54k op/s 2024-05-09T02:24:40.584 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:40 smithi019 ceph-mon[30151]: pgmap v418: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 22 MiB/s wr, 1.69k op/s 2024-05-09T02:24:40.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:40 smithi196 ceph-mon[34718]: pgmap v418: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 22 MiB/s wr, 1.69k op/s 2024-05-09T02:24:40.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:40 smithi059 ceph-mon[34781]: pgmap v418: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 22 MiB/s wr, 1.69k op/s 2024-05-09T02:24:41.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.067+0000 7fc5d352e700 1 -- 172.21.15.19:0/1643202880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc10a930 msgr2=0x7fc5cc10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:41.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.067+0000 7fc5d352e700 1 --2- 172.21.15.19:0/1643202880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc10a930 0x7fc5cc10ad40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009990 tx=0x7fc5c8009ca0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.072+0000 7fc5d352e700 1 -- 172.21.15.19:0/1643202880 shutdown_connections 2024-05-09T02:24:41.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.072+0000 7fc5d352e700 1 --2- 172.21.15.19:0/1643202880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5cc11d930 0x7fc5cc11fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.072+0000 7fc5d352e700 1 --2- 172.21.15.19:0/1643202880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc10a930 0x7fc5cc10ad40 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.072+0000 7fc5d352e700 1 --2- 172.21.15.19:0/1643202880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5cc10b280 0x7fc5cc11d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.072+0000 7fc5d352e700 1 -- 172.21.15.19:0/1643202880 >> 172.21.15.19:0/1643202880 conn(0x7fc5cc06f4d0 msgr2=0x7fc5cc06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:41.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.076+0000 7fc5d352e700 1 -- 172.21.15.19:0/1643202880 shutdown_connections 2024-05-09T02:24:41.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.076+0000 7fc5d352e700 1 -- 172.21.15.19:0/1643202880 wait complete. 2024-05-09T02:24:41.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.078+0000 7fc5d352e700 1 Processor -- start 2024-05-09T02:24:41.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.081+0000 7fc5d352e700 1 -- start start 2024-05-09T02:24:41.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.082+0000 7fc5d352e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5cc10a930 0x7fc5cc1af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.082+0000 7fc5d352e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5cc10b280 0x7fc5cc1afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.082+0000 7fc5d352e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc11d930 0x7fc5cc1a9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.082+0000 7fc5d352e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc123330 con 0x7fc5cc11d930 2024-05-09T02:24:41.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.082+0000 7fc5d352e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc1231b0 con 0x7fc5cc10a930 2024-05-09T02:24:41.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.082+0000 7fc5d352e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc1234b0 con 0x7fc5cc10b280 2024-05-09T02:24:41.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.088+0000 7fc5d0ac9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5cc10b280 0x7fc5cc1afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.088+0000 7fc5d0ac9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5cc10b280 0x7fc5cc1afd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:51334/0 (socket says 172.21.15.19:51334) 2024-05-09T02:24:41.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.088+0000 7fc5d0ac9700 1 -- 172.21.15.19:0/4285801859 learned_addr learned my addr 172.21.15.19:0/4285801859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:41.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.088+0000 7fc5d12ca700 1 --2- 172.21.15.19:0/4285801859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5cc10a930 0x7fc5cc1af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.088+0000 7fc5d1acb700 1 --2- 172.21.15.19:0/4285801859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc11d930 0x7fc5cc1a9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.092+0000 7fc5d1acb700 1 -- 172.21.15.19:0/4285801859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5cc10b280 msgr2=0x7fc5cc1afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:41.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.092+0000 7fc5d1acb700 1 --2- 172.21.15.19:0/4285801859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5cc10b280 0x7fc5cc1afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.092+0000 7fc5d1acb700 1 -- 172.21.15.19:0/4285801859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5cc10a930 msgr2=0x7fc5cc1af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:41.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.092+0000 7fc5d1acb700 1 --2- 172.21.15.19:0/4285801859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5cc10a930 0x7fc5cc1af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.092+0000 7fc5d1acb700 1 -- 172.21.15.19:0/4285801859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc1aa0e0 con 0x7fc5cc11d930 2024-05-09T02:24:41.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.096+0000 7fc5d12ca700 1 --2- 172.21.15.19:0/4285801859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5cc10a930 0x7fc5cc1af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:24:41.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.097+0000 7fc5d1acb700 1 --2- 172.21.15.19:0/4285801859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc11d930 0x7fc5cc1a9a30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4007ee0 tx=0x7fc5c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:41.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.098+0000 7fc5c27fc700 1 -- 172.21.15.19:0/4285801859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c400ddd0 con 0x7fc5cc11d930 2024-05-09T02:24:41.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.098+0000 7fc5c27fc700 1 -- 172.21.15.19:0/4285801859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c4009e20 con 0x7fc5cc11d930 2024-05-09T02:24:41.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.098+0000 7fc5c27fc700 1 -- 172.21.15.19:0/4285801859 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4015400 con 0x7fc5cc11d930 2024-05-09T02:24:41.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.101+0000 7fc5d352e700 1 -- 172.21.15.19:0/4285801859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc1aa3d0 con 0x7fc5cc11d930 2024-05-09T02:24:41.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.101+0000 7fc5d352e700 1 -- 172.21.15.19:0/4285801859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc1b66a0 con 0x7fc5cc11d930 2024-05-09T02:24:41.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.106+0000 7fc5e1f44b80 1 Processor -- start 2024-05-09T02:24:41.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.109+0000 7fc5e1f44b80 1 -- start start 2024-05-09T02:24:41.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.110+0000 7fc5c27fc700 1 -- 172.21.15.19:0/4285801859 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc5c40155a0 con 0x7fc5cc11d930 2024-05-09T02:24:41.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.112+0000 7fc5e1f44b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcfd4d00 0x5640dd0dd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.112+0000 7fc5e1f44b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dcf5a4a0 0x5640dd0dd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.112+0000 7fc5e1f44b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dd0d7400 0x5640dd0d7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.112+0000 7fc5e1f44b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5640dcffa0a0 con 0x5640dcfd4d00 2024-05-09T02:24:41.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.112+0000 7fc5e1f44b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5640dcf9f9a0 con 0x5640dd0d7400 2024-05-09T02:24:41.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.112+0000 7fc5e1f44b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5640dcff6060 con 0x5640dcf5a4a0 2024-05-09T02:24:41.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.112+0000 7fc5c27fc700 1 --2- 172.21.15.19:0/4285801859 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc5b80b2c00 0x7fc5b80b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.113+0000 7fc5c27fc700 1 -- 172.21.15.19:0/4285801859 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fc5c40e2340 con 0x7fc5cc11d930 2024-05-09T02:24:41.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.113+0000 7fc5d12ca700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dcf5a4a0 0x5640dd0dd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.113+0000 7fc5d12ca700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dcf5a4a0 0x5640dd0dd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:51344/0 (socket says 172.21.15.19:51344) 2024-05-09T02:24:41.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.113+0000 7fc5d12ca700 1 -- 172.21.15.19:0/141443441 learned_addr learned my addr 172.21.15.19:0/141443441 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:41.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.113+0000 7fc5d0ac9700 1 --2- 172.21.15.19:0/141443441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dd0d7400 0x5640dd0d7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.115+0000 7fc5d0ac9700 1 -- 172.21.15.19:0/141443441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dcf5a4a0 msgr2=0x5640dd0dd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:41.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.115+0000 7fc5d0ac9700 1 --2- 172.21.15.19:0/141443441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dcf5a4a0 0x5640dd0dd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.115+0000 7fc5d0ac9700 1 -- 172.21.15.19:0/141443441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcfd4d00 msgr2=0x5640dd0dd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:41.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.115+0000 7fc5d0ac9700 1 --2- 172.21.15.19:0/141443441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcfd4d00 0x5640dd0dd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.115+0000 7fc5d0ac9700 1 -- 172.21.15.19:0/141443441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640dcf90500 con 0x5640dd0d7400 2024-05-09T02:24:41.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.115+0000 7fc5d1acb700 1 --2- 172.21.15.19:0/4285801859 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc5b80b2c00 0x7fc5b80b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.115+0000 7fc5d12ca700 1 --2- 172.21.15.19:0/141443441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dcf5a4a0 0x5640dd0dd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:24:41.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.116+0000 7fc5d0ac9700 1 --2- 172.21.15.19:0/141443441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dd0d7400 0x5640dd0d7810 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009880 tx=0x7fc5bc011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d476bfa02e683f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:41.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.117+0000 7fc5b77fe700 1 -- 172.21.15.19:0/141443441 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc011bd0 con 0x5640dd0d7400 2024-05-09T02:24:41.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.117+0000 7fc5b77fe700 1 -- 172.21.15.19:0/141443441 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5bc004500 con 0x5640dd0d7400 2024-05-09T02:24:41.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.118+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/141443441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dd0d7400 msgr2=0x5640dd0d7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.118+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/141443441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dd0d7400 0x5640dd0d7810 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009880 tx=0x7fc5bc011720 comp rx=0 tx=0).stop 2024-05-09T02:24:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.118+0000 7fc5d1acb700 1 --2- 172.21.15.19:0/4285801859 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc5b80b2c00 0x7fc5b80b5110 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40a6800 tx=0x7fc5c400b040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.119+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/141443441 shutdown_connections 2024-05-09T02:24:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.119+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/141443441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dd0d7400 0x5640dd0d7810 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.119+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/141443441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcfd4d00 0x5640dd0dd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.119+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/141443441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dcf5a4a0 0x5640dd0dd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.119+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/141443441 >> 172.21.15.19:0/141443441 conn(0x5640dcf5df70 msgr2=0x5640dd0d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:41.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.122+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/141443441 shutdown_connections 2024-05-09T02:24:41.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.122+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/141443441 wait complete. 2024-05-09T02:24:41.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.123+0000 7fc5e1f44b80 1 -- start start 2024-05-09T02:24:41.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.123+0000 7fc5e1f44b80 1 Processor -- start 2024-05-09T02:24:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.124+0000 7fc5e1f44b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 0x5640dd0e0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.124+0000 7fc5e1f44b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dcfd4d00 0x5640dd0e14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.124+0000 7fc5e1f44b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dd0d7400 0x5640dd0e1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.124+0000 7fc5e1f44b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5640dd0d81b0 con 0x5640dcf5a4a0 2024-05-09T02:24:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.124+0000 7fc5e1f44b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5640dcf94b20 con 0x5640dd0d7400 2024-05-09T02:24:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.124+0000 7fc5e1f44b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5640dd0e4020 con 0x5640dcfd4d00 2024-05-09T02:24:41.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.124+0000 7fc5e1f44b80 10 client.0 _ll_register_callbacks cb 0x5640dce6b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:24:41.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.127+0000 7fc5d0ac9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dd0d7400 0x5640dd0e1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.127+0000 7fc5d0ac9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 0x5640dd0e0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.127+0000 7fc5d0ac9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 0x5640dd0e0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55182/0 (socket says 172.21.15.19:55182) 2024-05-09T02:24:41.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.127+0000 7fc5d0ac9700 1 -- 172.21.15.19:0/591733204 learned_addr learned my addr 172.21.15.19:0/591733204 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:41.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.127+0000 7fc5d12ca700 1 --2- 172.21.15.19:0/591733204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dcfd4d00 0x5640dd0e14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.128+0000 7fc5d0ac9700 1 -- 172.21.15.19:0/591733204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dcfd4d00 msgr2=0x5640dd0e14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.128+0000 7fc5d0ac9700 1 --2- 172.21.15.19:0/591733204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dcfd4d00 0x5640dd0e14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.128+0000 7fc5d0ac9700 1 -- 172.21.15.19:0/591733204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 msgr2=0x5640dd0e0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.128+0000 7fc5d0ac9700 1 --2- 172.21.15.19:0/591733204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 0x5640dd0e0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.128+0000 7fc5d0ac9700 1 -- 172.21.15.19:0/591733204 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640dce75490 con 0x5640dd0d7400 2024-05-09T02:24:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.129+0000 7fc5d0ac9700 1 --2- 172.21.15.19:0/591733204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dd0d7400 0x5640dd0e1a10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00a010 tx=0x7fc5bc013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=802f5cfe4a492a73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:41.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.129+0000 7fc5b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:24:41.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.129+0000 7fc5b4ff9700 1 -- 172.21.15.19:0/591733204 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc020040 con 0x5640dd0d7400 2024-05-09T02:24:41.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.129+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/591733204 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5bc004500 con 0x5640dd0d7400 2024-05-09T02:24:41.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.129+0000 7fc5b4ff9700 1 -- 172.21.15.19:0/591733204 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fc5bc01a620 con 0x5640dd0d7400 2024-05-09T02:24:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.130+0000 7fc5b4ff9700 1 -- 172.21.15.19:0/591733204 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fc5bc01a820 con 0x5640dd0d7400 2024-05-09T02:24:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.130+0000 7fc5e1f44b80 10 client.26663 fetch_fsmap learned FSMap version 58 2024-05-09T02:24:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.131+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/591733204 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5640dd0eea70 con 0x5640dd0d7400 2024-05-09T02:24:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.132+0000 7fc5b4ff9700 1 -- 172.21.15.19:0/591733204 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5bc004500 con 0x5640dd0d7400 2024-05-09T02:24:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.132+0000 7fc5b4ff9700 1 -- 172.21.15.19:0/591733204 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fc5bc032920 con 0x5640dd0d7400 2024-05-09T02:24:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.132+0000 7fc5e1f44b80 10 client.26663 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:24:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.132+0000 7fc5e1f44b80 10 client.26663 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:24:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.132+0000 7fc5e1f44b80 20 client.26663 populate_metadata read hostname 'smithi019' 2024-05-09T02:24:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.132+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/591733204 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5640dd0f13d0 0x5640dd0f3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.132+0000 7fc5e1f44b80 4 client.26663 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:24:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.132+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/591733204 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640dd007a80 con 0x5640dd0f13d0 2024-05-09T02:24:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.135+0000 7fc5d12ca700 1 --2- 172.21.15.19:0/591733204 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5640dd0f13d0 0x5640dd0f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.140+0000 7fc5d12ca700 1 --2- 172.21.15.19:0/591733204 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5640dd0f13d0 0x5640dd0f3820 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c410ded9ceed10f9 server_cookie=60b45610bc2cfdb5 in_seq=0 out_seq=0 2024-05-09T02:24:41.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.141+0000 7fc5b4ff9700 10 client.26663 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:24:41.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.142+0000 7fc5b4ff9700 1 -- 172.21.15.19:0/591733204 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5640dd007a80 con 0x5640dd0f13d0 2024-05-09T02:24:41.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.142+0000 7fc5b4ff9700 10 client.26663 handle_command_reply: tid=0 2024-05-09T02:24:41.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.142+0000 7fc5e1f44b80 1 client.26663 shutdown 2024-05-09T02:24:41.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.145+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/591733204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dd0d7400 msgr2=0x5640dd0e1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:41.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.145+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/591733204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dd0d7400 0x5640dd0e1a10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00a010 tx=0x7fc5bc013040 comp rx=0 tx=0).stop 2024-05-09T02:24:41.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.145+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/591733204 shutdown_connections 2024-05-09T02:24:41.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.145+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/591733204 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5640dd0f13d0 0x5640dd0f3820 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.145+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/591733204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dd0d7400 0x5640dd0e1a10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.145+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/591733204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 0x5640dd0e0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.145+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/591733204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dcfd4d00 0x5640dd0e14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.145+0000 7fc5b4ff9700 0 client.26663 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:24:41.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.145+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/591733204 >> 172.21.15.19:0/591733204 conn(0x5640dcf5df70 msgr2=0x5640dd0cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:41.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.147+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/591733204 shutdown_connections 2024-05-09T02:24:41.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.148+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/591733204 wait complete. 2024-05-09T02:24:41.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.151+0000 7fc5e1f44b80 20 client.26663 trim_cache size 0 max 16384 2024-05-09T02:24:41.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.180+0000 7fc5e1f44b80 1 Processor -- start 2024-05-09T02:24:41.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.183+0000 7fc5e1f44b80 1 -- start start 2024-05-09T02:24:41.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.183+0000 7fc5e1f44b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 0x5640dd0e3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.183+0000 7fc5e1f44b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dcfd4d00 0x5640dd0ecef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.183+0000 7fc5e1f44b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dd0d7400 0x5640dd0e1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.183+0000 7fc5e1f44b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b000d740 con 0x5640dcf5a4a0 2024-05-09T02:24:41.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.183+0000 7fc5e1f44b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b000f280 con 0x5640dcfd4d00 2024-05-09T02:24:41.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.183+0000 7fc5e1f44b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b000de10 con 0x5640dd0d7400 2024-05-09T02:24:41.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.185+0000 7fc5d12ca700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dcfd4d00 0x5640dd0ecef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.187+0000 7fc5d12ca700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dcfd4d00 0x5640dd0ecef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:43934/0 (socket says 172.21.15.19:43934) 2024-05-09T02:24:41.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.187+0000 7fc5d12ca700 1 -- 172.21.15.19:0/975295753 learned_addr learned my addr 172.21.15.19:0/975295753 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:41.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.187+0000 7fc5d0ac9700 1 --2- 172.21.15.19:0/975295753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 0x5640dd0e3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.187+0000 7fc5d0ac9700 1 --2- 172.21.15.19:0/975295753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dd0d7400 0x5640dd0e1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.187+0000 7fc5d12ca700 1 -- 172.21.15.19:0/975295753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dd0d7400 msgr2=0x5640dd0e1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:41.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.187+0000 7fc5d12ca700 1 --2- 172.21.15.19:0/975295753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dd0d7400 0x5640dd0e1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.187+0000 7fc5d12ca700 1 -- 172.21.15.19:0/975295753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 msgr2=0x5640dd0e3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:41.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.187+0000 7fc5d12ca700 1 --2- 172.21.15.19:0/975295753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 0x5640dd0e3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.187+0000 7fc5d12ca700 1 -- 172.21.15.19:0/975295753 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640dd0d3180 con 0x5640dcfd4d00 2024-05-09T02:24:41.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.188+0000 7fc5d12ca700 1 --2- 172.21.15.19:0/975295753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dcfd4d00 0x5640dd0ecef0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8006010 tx=0x7fc5c8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81b59e7a6e9a8ddb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:41.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.188+0000 7fc5b6ffd700 1 -- 172.21.15.19:0/975295753 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80052b0 con 0x5640dcfd4d00 2024-05-09T02:24:41.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.189+0000 7fc5b6ffd700 1 -- 172.21.15.19:0/975295753 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c8005450 con 0x5640dcfd4d00 2024-05-09T02:24:41.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.189+0000 7fc5b6ffd700 1 -- 172.21.15.19:0/975295753 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8027380 con 0x5640dcfd4d00 2024-05-09T02:24:41.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.192+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/975295753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dcfd4d00 msgr2=0x5640dd0ecef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:41.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.192+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/975295753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dcfd4d00 0x5640dd0ecef0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8006010 tx=0x7fc5c8009310 comp rx=0 tx=0).stop 2024-05-09T02:24:41.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.193+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/975295753 shutdown_connections 2024-05-09T02:24:41.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.193+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/975295753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dd0d7400 0x5640dd0e1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.193+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/975295753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 0x5640dd0e3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.193+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/975295753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dcfd4d00 0x5640dd0ecef0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.193+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/975295753 >> 172.21.15.19:0/975295753 conn(0x5640dcf5df70 msgr2=0x5640dd0d1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.193+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/975295753 shutdown_connections 2024-05-09T02:24:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.194+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/975295753 wait complete. 2024-05-09T02:24:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.194+0000 7fc5e1f44b80 1 -- start start 2024-05-09T02:24:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.195+0000 7fc5e1f44b80 1 Processor -- start 2024-05-09T02:24:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.196+0000 7fc5e1f44b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 0x5640dd0e9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.196+0000 7fc5e1f44b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dcfd4d00 0x5640dd0e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.196+0000 7fc5d0ac9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 0x5640dd0e9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.196+0000 7fc5e1f44b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dd0d7400 0x5640dd0ecc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.196+0000 7fc5e1f44b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b000ebb0 con 0x5640dcf5a4a0 2024-05-09T02:24:41.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.197+0000 7fc5d0ac9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 0x5640dd0e9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55190/0 (socket says 172.21.15.19:55190) 2024-05-09T02:24:41.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.197+0000 7fc5d0ac9700 1 -- 172.21.15.19:0/1975893010 learned_addr learned my addr 172.21.15.19:0/1975893010 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:41.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.196+0000 7fc5e1f44b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b000e4e0 con 0x5640dcfd4d00 2024-05-09T02:24:41.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.197+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/1975893010 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b000b700 con 0x5640dd0d7400 2024-05-09T02:24:41.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.197+0000 7fc5e1f44b80 10 client.0 _ll_register_callbacks cb 0x5640dce6b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:24:41.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.198+0000 7fc5d12ca700 1 --2- 172.21.15.19:0/1975893010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dcfd4d00 0x5640dd0e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.198+0000 7fc5d0ac9700 1 --2- 172.21.15.19:0/1975893010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dd0d7400 0x5640dd0ecc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.199+0000 7fc5d0ac9700 1 -- 172.21.15.19:0/1975893010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dd0d7400 msgr2=0x5640dd0ecc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:41.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.199+0000 7fc5d0ac9700 1 --2- 172.21.15.19:0/1975893010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dd0d7400 0x5640dd0ecc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.199+0000 7fc5d0ac9700 1 -- 172.21.15.19:0/1975893010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dcfd4d00 msgr2=0x5640dd0e97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:41.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.199+0000 7fc5d0ac9700 1 --2- 172.21.15.19:0/1975893010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dcfd4d00 0x5640dd0e97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.199+0000 7fc5d0ac9700 1 -- 172.21.15.19:0/1975893010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640dd0ed360 con 0x5640dcf5a4a0 2024-05-09T02:24:41.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.200+0000 7fc5d0ac9700 1 --2- 172.21.15.19:0/1975893010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 0x5640dd0e9270 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc032d40 tx=0x7fc5bc003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3570e2e336bcba61 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:41.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.200+0000 7fc5b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:24:41.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.201+0000 7fc5b4ff9700 1 -- 172.21.15.19:0/1975893010 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc003f20 con 0x5640dcf5a4a0 2024-05-09T02:24:41.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.201+0000 7fc5b4ff9700 1 -- 172.21.15.19:0/1975893010 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fc5bc01b420 con 0x5640dcf5a4a0 2024-05-09T02:24:41.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.201+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/1975893010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c8005450 con 0x5640dcf5a4a0 2024-05-09T02:24:41.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.204+0000 7fc5b4ff9700 1 -- 172.21.15.19:0/1975893010 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fc5bc0040c0 con 0x5640dcf5a4a0 2024-05-09T02:24:41.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.207+0000 7fc5e1f44b80 10 client.16941 fetch_fsmap learned FSMap version 58 2024-05-09T02:24:41.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.207+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/1975893010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5640dd157670 con 0x5640dcf5a4a0 2024-05-09T02:24:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.208+0000 7fc5b4ff9700 1 -- 172.21.15.19:0/1975893010 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c8005450 con 0x5640dcf5a4a0 2024-05-09T02:24:41.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.209+0000 7fc5b4ff9700 1 -- 172.21.15.19:0/1975893010 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fc5bc046020 con 0x5640dcf5a4a0 2024-05-09T02:24:41.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.209+0000 7fc5e1f44b80 10 client.16941 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:24:41.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.209+0000 7fc5e1f44b80 10 client.16941 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:24:41.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.209+0000 7fc5e1f44b80 20 client.16941 populate_metadata read hostname 'smithi019' 2024-05-09T02:24:41.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.209+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/1975893010 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5640dd0f8190 0x5640dd0f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:41.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.209+0000 7fc5e1f44b80 4 client.16941 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:24:41.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.210+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/1975893010 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640dd001b20 con 0x5640dd0f8190 2024-05-09T02:24:41.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.213+0000 7fc5d12ca700 1 --2- 172.21.15.19:0/1975893010 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5640dd0f8190 0x5640dd0f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.213+0000 7fc5b4ff9700 10 client.16941 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:24:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.214+0000 7fc5d12ca700 1 --2- 172.21.15.19:0/1975893010 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5640dd0f8190 0x5640dd0f85a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f72882092c4442e8 server_cookie=5358bf10816e085c in_seq=0 out_seq=0 2024-05-09T02:24:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.216+0000 7fc5b4ff9700 1 -- 172.21.15.19:0/1975893010 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640dd001b20 con 0x5640dd0f8190 2024-05-09T02:24:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.216+0000 7fc5b4ff9700 10 client.16941 handle_command_reply: tid=0 2024-05-09T02:24:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.216+0000 7fc5e1f44b80 1 client.16941 shutdown 2024-05-09T02:24:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.217+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/1975893010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 msgr2=0x5640dd0e9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.217+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/1975893010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 0x5640dd0e9270 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc032d40 tx=0x7fc5bc003b60 comp rx=0 tx=0).stop 2024-05-09T02:24:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.217+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/1975893010 shutdown_connections 2024-05-09T02:24:41.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.217+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/1975893010 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x5640dd0f8190 0x5640dd0f85a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.217+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/1975893010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dd0d7400 0x5640dd0ecc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.221 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:24:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.217+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/1975893010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5640dcf5a4a0 0x5640dd0e9270 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.217+0000 7fc5e1f44b80 1 --2- 172.21.15.19:0/1975893010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640dcfd4d00 0x5640dd0e97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.217+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/1975893010 >> 172.21.15.19:0/1975893010 conn(0x5640dcf5df70 msgr2=0x5640dd0dacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.217+0000 7fc5b4ff9700 0 client.16941 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:24:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.217+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/1975893010 shutdown_connections 2024-05-09T02:24:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.217+0000 7fc5e1f44b80 1 -- 172.21.15.19:0/1975893010 wait complete. 2024-05-09T02:24:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.217+0000 7fc5e1f44b80 20 client.16941 trim_cache size 0 max 16384 2024-05-09T02:24:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.218+0000 7fc5d352e700 1 -- 172.21.15.19:0/4285801859 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc5b80b2c00 msgr2=0x7fc5b80b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.218+0000 7fc5d352e700 1 --2- 172.21.15.19:0/4285801859 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc5b80b2c00 0x7fc5b80b5110 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40a6800 tx=0x7fc5c400b040 comp rx=0 tx=0).stop 2024-05-09T02:24:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.218+0000 7fc5d352e700 1 -- 172.21.15.19:0/4285801859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc11d930 msgr2=0x7fc5cc1a9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.218+0000 7fc5d352e700 1 --2- 172.21.15.19:0/4285801859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc11d930 0x7fc5cc1a9a30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4007ee0 tx=0x7fc5c400d3b0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.218+0000 7fc5d352e700 1 -- 172.21.15.19:0/4285801859 shutdown_connections 2024-05-09T02:24:41.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.218+0000 7fc5d352e700 1 --2- 172.21.15.19:0/4285801859 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc5b80b2c00 0x7fc5b80b5110 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.218+0000 7fc5d352e700 1 --2- 172.21.15.19:0/4285801859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc11d930 0x7fc5cc1a9a30 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.218+0000 7fc5d352e700 1 --2- 172.21.15.19:0/4285801859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5cc10a930 0x7fc5cc1af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.218+0000 7fc5d352e700 1 --2- 172.21.15.19:0/4285801859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5cc10b280 0x7fc5cc1afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:41.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.218+0000 7fc5d352e700 1 -- 172.21.15.19:0/4285801859 >> 172.21.15.19:0/4285801859 conn(0x7fc5cc06f4d0 msgr2=0x7fc5cc11b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:41.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.219+0000 7fc5d352e700 1 -- 172.21.15.19:0/4285801859 shutdown_connections 2024-05-09T02:24:41.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:41.219+0000 7fc5d352e700 1 -- 172.21.15.19:0/4285801859 wait complete. 2024-05-09T02:24:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 780.0 s, 1446.0 tps, lat 69.713 ms stddev 94.946 2024-05-09T02:24:42.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:42 smithi196 ceph-mon[34718]: pgmap v419: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 21 MiB/s wr, 1.62k op/s 2024-05-09T02:24:42.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:42 smithi019 ceph-mon[30151]: pgmap v419: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 21 MiB/s wr, 1.62k op/s 2024-05-09T02:24:42.937 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T02:24:42.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:42 smithi059 ceph-mon[34781]: pgmap v419: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 21 MiB/s wr, 1.62k op/s 2024-05-09T02:24:43.310 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:24:44.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:44 smithi196 ceph-mon[34718]: pgmap v420: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 19 MiB/s wr, 1.49k op/s 2024-05-09T02:24:44.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:44 smithi019 ceph-mon[30151]: pgmap v420: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 19 MiB/s wr, 1.49k op/s 2024-05-09T02:24:44.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:44 smithi059 ceph-mon[34781]: pgmap v420: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 19 MiB/s wr, 1.49k op/s 2024-05-09T02:24:45.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.468+0000 7fdd477fe700 1 -- 172.21.15.19:0/1822133737 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34017400 con 0x7fdd40098790 2024-05-09T02:24:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.468+0000 7fdd4d94e700 1 -- 172.21.15.19:0/1822133737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd40098790 msgr2=0x7fdd40098ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.468+0000 7fdd4d94e700 1 --2- 172.21.15.19:0/1822133737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd40098790 0x7fdd40098ba0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd340098f0 tx=0x7fdd34009c00 comp rx=0 tx=0).stop 2024-05-09T02:24:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.468+0000 7fdd4d94e700 1 -- 172.21.15.19:0/1822133737 shutdown_connections 2024-05-09T02:24:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.468+0000 7fdd4d94e700 1 --2- 172.21.15.19:0/1822133737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd400990e0 0x7fdd400a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.468+0000 7fdd4d94e700 1 --2- 172.21.15.19:0/1822133737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd4009e7c0 0x7fdd4009ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.468+0000 7fdd4d94e700 1 --2- 172.21.15.19:0/1822133737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd40098790 0x7fdd40098ba0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.468+0000 7fdd4d94e700 1 -- 172.21.15.19:0/1822133737 >> 172.21.15.19:0/1822133737 conn(0x7fdd400942b0 msgr2=0x7fdd40096700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.468+0000 7fdd4d94e700 1 -- 172.21.15.19:0/1822133737 shutdown_connections 2024-05-09T02:24:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.468+0000 7fdd4d94e700 1 -- 172.21.15.19:0/1822133737 wait complete. 2024-05-09T02:24:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.469+0000 7fdd4d94e700 1 Processor -- start 2024-05-09T02:24:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.469+0000 7fdd4d94e700 1 -- start start 2024-05-09T02:24:45.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd4d94e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd40098790 0x7fdd4012f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd4d94e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd400990e0 0x7fdd4012f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd4d94e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd4009e7c0 0x7fdd40133b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd4d94e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdd400a8960 con 0x7fdd400990e0 2024-05-09T02:24:45.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd4d94e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdd400a87e0 con 0x7fdd40098790 2024-05-09T02:24:45.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd4c94c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd40098790 0x7fdd4012f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd4d94e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdd400a8ae0 con 0x7fdd4009e7c0 2024-05-09T02:24:45.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd47fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd400990e0 0x7fdd4012f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd47fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd400990e0 0x7fdd4012f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55202/0 (socket says 172.21.15.19:55202) 2024-05-09T02:24:45.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd47fff700 1 -- 172.21.15.19:0/2494321530 learned_addr learned my addr 172.21.15.19:0/2494321530 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:45.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd4d14d700 1 --2- 172.21.15.19:0/2494321530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd4009e7c0 0x7fdd40133b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd4d14d700 1 -- 172.21.15.19:0/2494321530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd40098790 msgr2=0x7fdd4012f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:45.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd4d14d700 1 --2- 172.21.15.19:0/2494321530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd40098790 0x7fdd4012f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd4d14d700 1 -- 172.21.15.19:0/2494321530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd400990e0 msgr2=0x7fdd4012f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:45.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd4d14d700 1 --2- 172.21.15.19:0/2494321530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd400990e0 0x7fdd4012f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.470+0000 7fdd4d14d700 1 -- 172.21.15.19:0/2494321530 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd40134200 con 0x7fdd4009e7c0 2024-05-09T02:24:45.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.471+0000 7fdd4d14d700 1 --2- 172.21.15.19:0/2494321530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd4009e7c0 0x7fdd40133b60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdd48069820 tx=0x7fdd4806bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:45.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.471+0000 7fdd45ffb700 1 -- 172.21.15.19:0/2494321530 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48084070 con 0x7fdd4009e7c0 2024-05-09T02:24:45.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.471+0000 7fdd45ffb700 1 -- 172.21.15.19:0/2494321530 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd48080be0 con 0x7fdd4009e7c0 2024-05-09T02:24:45.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.471+0000 7fdd45ffb700 1 -- 172.21.15.19:0/2494321530 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4807fc30 con 0x7fdd4009e7c0 2024-05-09T02:24:45.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.471+0000 7fdd4d94e700 1 -- 172.21.15.19:0/2494321530 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd401344f0 con 0x7fdd4009e7c0 2024-05-09T02:24:45.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.472+0000 7fdd4d94e700 1 -- 172.21.15.19:0/2494321530 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd4013be10 con 0x7fdd4009e7c0 2024-05-09T02:24:45.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.475+0000 7fdd45ffb700 1 -- 172.21.15.19:0/2494321530 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdd48087070 con 0x7fdd4009e7c0 2024-05-09T02:24:45.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.476+0000 7fdd45ffb700 1 --2- 172.21.15.19:0/2494321530 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fdd380b2bc0 0x7fdd380b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.476+0000 7fdd45ffb700 1 -- 172.21.15.19:0/2494321530 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fdd481502b0 con 0x7fdd4009e7c0 2024-05-09T02:24:45.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.477+0000 7fdd4c94c700 1 --2- 172.21.15.19:0/2494321530 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fdd380b2bc0 0x7fdd380b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.477+0000 7fdd4c94c700 1 --2- 172.21.15.19:0/2494321530 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fdd380b2bc0 0x7fdd380b50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdd400998d0 tx=0x7fdd3c009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:45.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.478+0000 7fdd5d5c6b80 1 Processor -- start 2024-05-09T02:24:45.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.478+0000 7fdd5d5c6b80 1 -- start start 2024-05-09T02:24:45.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd5d5c6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557278465d00 0x557278568940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd5d5c6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572783eb4a0 0x557278568e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd5d5c6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55727857ad50 0x55727857d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd5d5c6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55727848b0a0 con 0x5572783eb4a0 2024-05-09T02:24:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd5d5c6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5572784309a0 con 0x55727857ad50 2024-05-09T02:24:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd5d5c6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557278487060 con 0x557278465d00 2024-05-09T02:24:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd47fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572783eb4a0 0x557278568e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd4c94c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557278465d00 0x557278568940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd4c94c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557278465d00 0x557278568940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:51394/0 (socket says 172.21.15.19:51394) 2024-05-09T02:24:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd4c94c700 1 -- 172.21.15.19:0/1586998819 learned_addr learned my addr 172.21.15.19:0/1586998819 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd47fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572783eb4a0 0x557278568e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55214/0 (socket says 172.21.15.19:55214) 2024-05-09T02:24:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd4d14d700 1 --2- 172.21.15.19:0/1586998819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55727857ad50 0x55727857d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd4c94c700 1 -- 172.21.15.19:0/1586998819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55727857ad50 msgr2=0x55727857d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd4c94c700 1 --2- 172.21.15.19:0/1586998819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55727857ad50 0x55727857d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd4c94c700 1 -- 172.21.15.19:0/1586998819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572783eb4a0 msgr2=0x557278568e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd4c94c700 1 --2- 172.21.15.19:0/1586998819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572783eb4a0 0x557278568e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd4c94c700 1 -- 172.21.15.19:0/1586998819 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557278421500 con 0x557278465d00 2024-05-09T02:24:45.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.479+0000 7fdd47fff700 1 --2- 172.21.15.19:0/1586998819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572783eb4a0 0x557278568e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:24:45.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.480+0000 7fdd4c94c700 1 --2- 172.21.15.19:0/1586998819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557278465d00 0x557278568940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c011d40 tx=0x7fdd3c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1ec7ef3f35758d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:45.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.480+0000 7fdd32ffd700 1 -- 172.21.15.19:0/1586998819 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c011d70 con 0x557278465d00 2024-05-09T02:24:45.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.480+0000 7fdd32ffd700 1 -- 172.21.15.19:0/1586998819 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3c004020 con 0x557278465d00 2024-05-09T02:24:45.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.480+0000 7fdd32ffd700 1 -- 172.21.15.19:0/1586998819 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c003bc0 con 0x557278465d00 2024-05-09T02:24:45.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.481+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/1586998819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557278465d00 msgr2=0x557278568940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:45.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.481+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/1586998819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557278465d00 0x557278568940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c011d40 tx=0x7fdd3c014000 comp rx=0 tx=0).stop 2024-05-09T02:24:45.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.481+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/1586998819 shutdown_connections 2024-05-09T02:24:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.481+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/1586998819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55727857ad50 0x55727857d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.481+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/1586998819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557278465d00 0x557278568940 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.481+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/1586998819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572783eb4a0 0x557278568e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.481+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/1586998819 >> 172.21.15.19:0/1586998819 conn(0x5572783eef70 msgr2=0x557278563760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.481+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/1586998819 shutdown_connections 2024-05-09T02:24:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.481+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/1586998819 wait complete. 2024-05-09T02:24:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.481+0000 7fdd5d5c6b80 1 -- start start 2024-05-09T02:24:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.482+0000 7fdd5d5c6b80 1 Processor -- start 2024-05-09T02:24:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.482+0000 7fdd5d5c6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572783eb4a0 0x557278577d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.482+0000 7fdd5d5c6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557278465d00 0x557278578260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.482+0000 7fdd5d5c6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55727857ad50 0x5572785787a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.483+0000 7fdd5d5c6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55727857d770 con 0x5572783eb4a0 2024-05-09T02:24:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.483+0000 7fdd5d5c6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557278425b20 con 0x557278465d00 2024-05-09T02:24:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.483+0000 7fdd5d5c6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55727857d8f0 con 0x55727857ad50 2024-05-09T02:24:45.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.483+0000 7fdd4c94c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572783eb4a0 0x557278577d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.483+0000 7fdd5d5c6b80 10 client.0 _ll_register_callbacks cb 0x5572782fc6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:24:45.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.483+0000 7fdd47fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557278465d00 0x557278578260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.483+0000 7fdd47fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557278465d00 0x557278578260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:43972/0 (socket says 172.21.15.19:43972) 2024-05-09T02:24:45.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.483+0000 7fdd47fff700 1 -- 172.21.15.19:0/3622410333 learned_addr learned my addr 172.21.15.19:0/3622410333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:45.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.483+0000 7fdd4c94c700 1 --2- 172.21.15.19:0/3622410333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572783eb4a0 0x557278577d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55222/0 (socket says 172.21.15.19:55222) 2024-05-09T02:24:45.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.483+0000 7fdd4d14d700 1 --2- 172.21.15.19:0/3622410333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55727857ad50 0x5572785787a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.483+0000 7fdd47fff700 1 -- 172.21.15.19:0/3622410333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55727857ad50 msgr2=0x5572785787a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:45.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.483+0000 7fdd47fff700 1 --2- 172.21.15.19:0/3622410333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55727857ad50 0x5572785787a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.483+0000 7fdd47fff700 1 -- 172.21.15.19:0/3622410333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572783eb4a0 msgr2=0x557278577d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:45.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.483+0000 7fdd47fff700 1 --2- 172.21.15.19:0/3622410333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572783eb4a0 0x557278577d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.483+0000 7fdd47fff700 1 -- 172.21.15.19:0/3622410333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557278306490 con 0x557278465d00 2024-05-09T02:24:45.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.484+0000 7fdd47fff700 1 --2- 172.21.15.19:0/3622410333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557278465d00 0x557278578260 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400c010 tx=0x7fdd3400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddaa4249b3267b41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:45.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.484+0000 7fdd27fff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:24:45.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.484+0000 7fdd27fff700 1 -- 172.21.15.19:0/3622410333 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34005040 con 0x557278465d00 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.484+0000 7fdd27fff700 1 -- 172.21.15.19:0/3622410333 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fdd340051e0 con 0x557278465d00 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.484+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/3622410333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd3c004020 con 0x557278465d00 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.484+0000 7fdd27fff700 1 -- 172.21.15.19:0/3622410333 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fdd34051050 con 0x557278465d00 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.484+0000 7fdd5d5c6b80 10 client.26678 fetch_fsmap learned FSMap version 58 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.484+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/3622410333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55727856bc60 con 0x557278465d00 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.484+0000 7fdd27fff700 1 -- 172.21.15.19:0/3622410333 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd340053e0 con 0x557278465d00 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.484+0000 7fdd27fff700 1 -- 172.21.15.19:0/3622410333 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fdd3404c030 con 0x557278465d00 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.485+0000 7fdd5d5c6b80 10 client.26678 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.485+0000 7fdd5d5c6b80 10 client.26678 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.485+0000 7fdd5d5c6b80 20 client.26678 populate_metadata read hostname 'smithi019' 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.485+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/3622410333 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55727856e290 0x5572785706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.485+0000 7fdd5d5c6b80 4 client.26678 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.485+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/3622410333 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdd3c004020 con 0x55727856e290 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.485+0000 7fdd4c94c700 1 --2- 172.21.15.19:0/3622410333 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55727856e290 0x5572785706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.486+0000 7fdd4c94c700 1 --2- 172.21.15.19:0/3622410333 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55727856e290 0x5572785706e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e3880a54e29ff573 server_cookie=f20c3dbb3b24951e in_seq=0 out_seq=0 2024-05-09T02:24:45.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.486+0000 7fdd27fff700 10 client.26678 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:24:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.486+0000 7fdd27fff700 1 -- 172.21.15.19:0/3622410333 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fdd3c004020 con 0x55727856e290 2024-05-09T02:24:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.486+0000 7fdd27fff700 10 client.26678 handle_command_reply: tid=0 2024-05-09T02:24:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.486+0000 7fdd5d5c6b80 1 client.26678 shutdown 2024-05-09T02:24:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.487+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/3622410333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557278465d00 msgr2=0x557278578260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.487+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/3622410333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557278465d00 0x557278578260 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400c010 tx=0x7fdd3400f690 comp rx=0 tx=0).stop 2024-05-09T02:24:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.487+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/3622410333 shutdown_connections 2024-05-09T02:24:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.487+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/3622410333 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55727856e290 0x5572785706e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.487+0000 7fdd27fff700 0 client.26678 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:24:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.487+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/3622410333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55727857ad50 0x5572785787a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.487+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/3622410333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5572783eb4a0 0x557278577d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.487+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/3622410333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557278465d00 0x557278578260 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.487+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/3622410333 >> 172.21.15.19:0/3622410333 conn(0x5572783eef70 msgr2=0x557278563250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.487+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/3622410333 shutdown_connections 2024-05-09T02:24:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.487+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/3622410333 wait complete. 2024-05-09T02:24:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.488+0000 7fdd5d5c6b80 20 client.26678 trim_cache size 0 max 16384 2024-05-09T02:24:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.511+0000 7fdd5d5c6b80 1 Processor -- start 2024-05-09T02:24:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.511+0000 7fdd5d5c6b80 1 -- start start 2024-05-09T02:24:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.512+0000 7fdd5d5c6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572783eb4a0 0x5572785735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.512+0000 7fdd5d5c6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557278465d00 0x55727859cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.512+0000 7fdd5d5c6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55727856e290 0x55727859d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.512+0000 7fdd5d5c6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4013ef40 con 0x55727856e290 2024-05-09T02:24:45.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.512+0000 7fdd5d5c6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdd40140a80 con 0x557278465d00 2024-05-09T02:24:45.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.512+0000 7fdd5d5c6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4013f610 con 0x5572783eb4a0 2024-05-09T02:24:45.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.512+0000 7fdd4c94c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572783eb4a0 0x5572785735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.512+0000 7fdd4c94c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572783eb4a0 0x5572785735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:51424/0 (socket says 172.21.15.19:51424) 2024-05-09T02:24:45.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.512+0000 7fdd4c94c700 1 -- 172.21.15.19:0/4067757956 learned_addr learned my addr 172.21.15.19:0/4067757956 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:45.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.512+0000 7fdd4d14d700 1 --2- 172.21.15.19:0/4067757956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55727856e290 0x55727859d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.512+0000 7fdd47fff700 1 --2- 172.21.15.19:0/4067757956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557278465d00 0x55727859cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.512+0000 7fdd4c94c700 1 -- 172.21.15.19:0/4067757956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557278465d00 msgr2=0x55727859cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.512+0000 7fdd4c94c700 1 --2- 172.21.15.19:0/4067757956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557278465d00 0x55727859cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.513+0000 7fdd4c94c700 1 -- 172.21.15.19:0/4067757956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55727856e290 msgr2=0x55727859d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.513+0000 7fdd4c94c700 1 --2- 172.21.15.19:0/4067757956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55727856e290 0x55727859d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.513+0000 7fdd4c94c700 1 -- 172.21.15.19:0/4067757956 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557278564ee0 con 0x5572783eb4a0 2024-05-09T02:24:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.513+0000 7fdd4d14d700 1 --2- 172.21.15.19:0/4067757956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55727856e290 0x55727859d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:24:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.513+0000 7fdd4c94c700 1 --2- 172.21.15.19:0/4067757956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572783eb4a0 0x5572785735d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c003790 tx=0x7fdd3c0037c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e306abd489df23ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.513+0000 7fdd327fc700 1 -- 172.21.15.19:0/4067757956 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c00ee20 con 0x5572783eb4a0 2024-05-09T02:24:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.513+0000 7fdd327fc700 1 -- 172.21.15.19:0/4067757956 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3c003830 con 0x5572783eb4a0 2024-05-09T02:24:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.513+0000 7fdd327fc700 1 -- 172.21.15.19:0/4067757956 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c01e7a0 con 0x5572783eb4a0 2024-05-09T02:24:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.514+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/4067757956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572783eb4a0 msgr2=0x5572785735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.514+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/4067757956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572783eb4a0 0x5572785735d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c003790 tx=0x7fdd3c0037c0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.514+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/4067757956 shutdown_connections 2024-05-09T02:24:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.514+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/4067757956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55727856e290 0x55727859d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.514+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/4067757956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572783eb4a0 0x5572785735d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.514+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/4067757956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557278465d00 0x55727859cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.514+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/4067757956 >> 172.21.15.19:0/4067757956 conn(0x5572783eef70 msgr2=0x557278572ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.514+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/4067757956 shutdown_connections 2024-05-09T02:24:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.514+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/4067757956 wait complete. 2024-05-09T02:24:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.515+0000 7fdd5d5c6b80 1 -- start start 2024-05-09T02:24:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.515+0000 7fdd5d5c6b80 1 Processor -- start 2024-05-09T02:24:45.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd5d5c6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572783eb4a0 0x557278580a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd5d5c6b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557278465d00 0x557278580fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd5d5c6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55727856e290 0x5572785fc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd5d5c6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdd401403b0 con 0x55727856e290 2024-05-09T02:24:45.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd5d5c6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4013fce0 con 0x5572783eb4a0 2024-05-09T02:24:45.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd5d5c6b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4000d2c0 con 0x557278465d00 2024-05-09T02:24:45.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd4d14d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55727856e290 0x5572785fc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd4c94c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572783eb4a0 0x557278580a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd4c94c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572783eb4a0 0x557278580a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:44000/0 (socket says 172.21.15.19:44000) 2024-05-09T02:24:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd4c94c700 1 -- 172.21.15.19:0/1954096773 learned_addr learned my addr 172.21.15.19:0/1954096773 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd5d5c6b80 10 client.0 _ll_register_callbacks cb 0x55727856aa60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:24:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd47fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557278465d00 0x557278580fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd4c94c700 1 -- 172.21.15.19:0/1954096773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557278465d00 msgr2=0x557278580fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd4c94c700 1 --2- 172.21.15.19:0/1954096773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557278465d00 0x557278580fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd4c94c700 1 -- 172.21.15.19:0/1954096773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55727856e290 msgr2=0x5572785fc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd4c94c700 1 --2- 172.21.15.19:0/1954096773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55727856e290 0x5572785fc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.516+0000 7fdd4c94c700 1 -- 172.21.15.19:0/1954096773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572785817e0 con 0x5572783eb4a0 2024-05-09T02:24:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.517+0000 7fdd4c94c700 1 --2- 172.21.15.19:0/1954096773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572783eb4a0 0x557278580a80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0036c0 tx=0x7fdd3c00eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c5f843f144a1e35 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.517+0000 7fdd27fff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:24:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.517+0000 7fdd27fff700 1 -- 172.21.15.19:0/1954096773 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c003df0 con 0x5572783eb4a0 2024-05-09T02:24:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.517+0000 7fdd27fff700 1 -- 172.21.15.19:0/1954096773 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fdd3c0050b0 con 0x5572783eb4a0 2024-05-09T02:24:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.517+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/1954096773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd3c003830 con 0x5572783eb4a0 2024-05-09T02:24:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.517+0000 7fdd27fff700 1 -- 172.21.15.19:0/1954096773 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fdd3c0052b0 con 0x5572783eb4a0 2024-05-09T02:24:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.517+0000 7fdd5d5c6b80 10 client.26681 fetch_fsmap learned FSMap version 58 2024-05-09T02:24:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.517+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/1954096773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5572785fdd10 con 0x5572783eb4a0 2024-05-09T02:24:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.518+0000 7fdd27fff700 1 -- 172.21.15.19:0/1954096773 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3c003830 con 0x5572783eb4a0 2024-05-09T02:24:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.518+0000 7fdd27fff700 1 -- 172.21.15.19:0/1954096773 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fdd3c01e420 con 0x5572783eb4a0 2024-05-09T02:24:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.518+0000 7fdd5d5c6b80 10 client.26681 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:24:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.518+0000 7fdd5d5c6b80 10 client.26681 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:24:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.518+0000 7fdd5d5c6b80 20 client.26681 populate_metadata read hostname 'smithi019' 2024-05-09T02:24:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.519+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/1954096773 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55727857ad50 0x557278600050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:45.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.519+0000 7fdd5d5c6b80 4 client.26681 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:24:45.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.519+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/1954096773 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557278498a80 con 0x55727857ad50 2024-05-09T02:24:45.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.519+0000 7fdd47fff700 1 --2- 172.21.15.19:0/1954096773 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55727857ad50 0x557278600050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:45.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd47fff700 1 --2- 172.21.15.19:0/1954096773 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55727857ad50 0x557278600050 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc6899d1bcf65f47 server_cookie=e1b9873f2fda0ee2 in_seq=0 out_seq=0 2024-05-09T02:24:45.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd27fff700 10 client.26681 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:24:45.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd27fff700 1 -- 172.21.15.19:0/1954096773 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557278498a80 con 0x55727857ad50 2024-05-09T02:24:45.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd27fff700 10 client.26681 handle_command_reply: tid=0 2024-05-09T02:24:45.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd5d5c6b80 1 client.26681 shutdown 2024-05-09T02:24:45.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/1954096773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572783eb4a0 msgr2=0x557278580a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:45.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/1954096773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572783eb4a0 0x557278580a80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0036c0 tx=0x7fdd3c00eea0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/1954096773 shutdown_connections 2024-05-09T02:24:45.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/1954096773 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55727857ad50 0x557278600050 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/1954096773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55727856e290 0x5572785fc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/1954096773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572783eb4a0 0x557278580a80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd5d5c6b80 1 --2- 172.21.15.19:0/1954096773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557278465d00 0x557278580fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/1954096773 >> 172.21.15.19:0/1954096773 conn(0x5572783eef70 msgr2=0x557278563130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:45.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd27fff700 0 client.26681 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:24:45.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/1954096773 shutdown_connections 2024-05-09T02:24:45.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.520+0000 7fdd5d5c6b80 1 -- 172.21.15.19:0/1954096773 wait complete. 2024-05-09T02:24:45.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.521+0000 7fdd5d5c6b80 20 client.26681 trim_cache size 0 max 16384 2024-05-09T02:24:45.526 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:24:45.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.521+0000 7fdd4d94e700 1 -- 172.21.15.19:0/2494321530 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fdd380b2bc0 msgr2=0x7fdd380b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:45.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.521+0000 7fdd4d94e700 1 --2- 172.21.15.19:0/2494321530 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fdd380b2bc0 0x7fdd380b50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdd400998d0 tx=0x7fdd3c009260 comp rx=0 tx=0).stop 2024-05-09T02:24:45.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.521+0000 7fdd4d94e700 1 -- 172.21.15.19:0/2494321530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd4009e7c0 msgr2=0x7fdd40133b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:45.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.521+0000 7fdd4d94e700 1 --2- 172.21.15.19:0/2494321530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd4009e7c0 0x7fdd40133b60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdd48069820 tx=0x7fdd4806bc20 comp rx=0 tx=0).stop 2024-05-09T02:24:45.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.521+0000 7fdd4d94e700 1 -- 172.21.15.19:0/2494321530 shutdown_connections 2024-05-09T02:24:45.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.521+0000 7fdd4d94e700 1 --2- 172.21.15.19:0/2494321530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd4009e7c0 0x7fdd40133b60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.521+0000 7fdd4d94e700 1 --2- 172.21.15.19:0/2494321530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd40098790 0x7fdd4012f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.521+0000 7fdd4d94e700 1 --2- 172.21.15.19:0/2494321530 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fdd380b2bc0 0x7fdd380b50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.521+0000 7fdd4d94e700 1 --2- 172.21.15.19:0/2494321530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd400990e0 0x7fdd4012f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:45.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.522+0000 7fdd4d94e700 1 -- 172.21.15.19:0/2494321530 >> 172.21.15.19:0/2494321530 conn(0x7fdd400942b0 msgr2=0x7fdd4009cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:45.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.522+0000 7fdd4d94e700 1 -- 172.21.15.19:0/2494321530 shutdown_connections 2024-05-09T02:24:45.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:45.522+0000 7fdd4d94e700 1 -- 172.21.15.19:0/2494321530 wait complete. 2024-05-09T02:24:46.360 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T02:24:46.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:46 smithi019 ceph-mon[30151]: pgmap v421: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 26 MiB/s wr, 2.15k op/s 2024-05-09T02:24:46.764 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:24:46.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:46 smithi196 ceph-mon[34718]: pgmap v421: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 26 MiB/s wr, 2.15k op/s 2024-05-09T02:24:46.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:46 smithi059 ceph-mon[34781]: pgmap v421: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 26 MiB/s wr, 2.15k op/s 2024-05-09T02:24:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 785.0 s, 1489.6 tps, lat 66.746 ms stddev 78.964 2024-05-09T02:24:48.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:48 smithi019 ceph-mon[30151]: pgmap v422: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 1.08k op/s 2024-05-09T02:24:48.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.768+0000 7f7590fe6700 1 -- 172.21.15.19:0/1998367907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f758c104420 msgr2=0x7f758c104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:48.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.768+0000 7f7590fe6700 1 --2- 172.21.15.19:0/1998367907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f758c104420 0x7f758c104830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f757c01a9a0 tx=0x7f757c01acb0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.768+0000 7f7590fe6700 1 -- 172.21.15.19:0/1998367907 shutdown_connections 2024-05-09T02:24:48.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.768+0000 7f7590fe6700 1 --2- 172.21.15.19:0/1998367907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f758c1096a0 0x7f758c111000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.768+0000 7f7590fe6700 1 --2- 172.21.15.19:0/1998367907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f758c104420 0x7f758c104830 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.768+0000 7f7590fe6700 1 --2- 172.21.15.19:0/1998367907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f758c108d50 0x7f758c109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.768+0000 7f7590fe6700 1 -- 172.21.15.19:0/1998367907 >> 172.21.15.19:0/1998367907 conn(0x7f758c07acb0 msgr2=0x7f758c07b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:48.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.768+0000 7f7590fe6700 1 -- 172.21.15.19:0/1998367907 shutdown_connections 2024-05-09T02:24:48.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.768+0000 7f7590fe6700 1 -- 172.21.15.19:0/1998367907 wait complete. 2024-05-09T02:24:48.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.769+0000 7f7590fe6700 1 Processor -- start 2024-05-09T02:24:48.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.770+0000 7f7590fe6700 1 -- start start 2024-05-09T02:24:48.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.770+0000 7f7590fe6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f758c104420 0x7f758c110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.770+0000 7f7590fe6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f758c108d50 0x7f758c110fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.770+0000 7f7590fe6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f758c1096a0 0x7f758c10ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.770+0000 7f7590fe6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f758c06f100 con 0x7f758c104420 2024-05-09T02:24:48.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.770+0000 7f7590fe6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f758c06ef80 con 0x7f758c108d50 2024-05-09T02:24:48.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.771+0000 7f7590fe6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f758c06f280 con 0x7f758c1096a0 2024-05-09T02:24:48.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.771+0000 7f758bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f758c1096a0 0x7f758c10ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.771+0000 7f758bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f758c1096a0 0x7f758c10ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40624/0 (socket says 172.21.15.19:40624) 2024-05-09T02:24:48.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.771+0000 7f758bfff700 1 -- 172.21.15.19:0/330755747 learned_addr learned my addr 172.21.15.19:0/330755747 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:48.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.771+0000 7f758b7fe700 1 --2- 172.21.15.19:0/330755747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f758c104420 0x7f758c110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.771+0000 7f758affd700 1 --2- 172.21.15.19:0/330755747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f758c108d50 0x7f758c110fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.771+0000 7f758bfff700 1 -- 172.21.15.19:0/330755747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f758c108d50 msgr2=0x7f758c110fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:48.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.771+0000 7f758bfff700 1 --2- 172.21.15.19:0/330755747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f758c108d50 0x7f758c110fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.771+0000 7f758bfff700 1 -- 172.21.15.19:0/330755747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f758c104420 msgr2=0x7f758c110a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:48.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.771+0000 7f758bfff700 1 --2- 172.21.15.19:0/330755747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f758c104420 0x7f758c110a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.771+0000 7f758bfff700 1 -- 172.21.15.19:0/330755747 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c10b560 con 0x7f758c1096a0 2024-05-09T02:24:48.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.772+0000 7f758affd700 1 --2- 172.21.15.19:0/330755747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f758c108d50 0x7f758c110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:24:48.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.772+0000 7f758bfff700 1 --2- 172.21.15.19:0/330755747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f758c1096a0 0x7f758c10ad30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f758000e940 tx=0x7f758000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:48.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.772+0000 7f7588ff9700 1 -- 172.21.15.19:0/330755747 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758000cbd0 con 0x7f758c1096a0 2024-05-09T02:24:48.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.772+0000 7f7588ff9700 1 -- 172.21.15.19:0/330755747 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f758000ee90 con 0x7f758c1096a0 2024-05-09T02:24:48.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.772+0000 7f7588ff9700 1 -- 172.21.15.19:0/330755747 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580005590 con 0x7f758c1096a0 2024-05-09T02:24:48.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.772+0000 7f7590fe6700 1 -- 172.21.15.19:0/330755747 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758c10b850 con 0x7f758c1096a0 2024-05-09T02:24:48.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.772+0000 7f7590fe6700 1 -- 172.21.15.19:0/330755747 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f758c1a52c0 con 0x7f758c1096a0 2024-05-09T02:24:48.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.777+0000 7f75a0c5eb80 1 Processor -- start 2024-05-09T02:24:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.778+0000 7f7588ff9700 1 -- 172.21.15.19:0/330755747 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7580004750 con 0x7f758c1096a0 2024-05-09T02:24:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.779+0000 7f75a0c5eb80 1 -- start start 2024-05-09T02:24:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.779+0000 7f75a0c5eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cccb71160 0x559cccc0a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.779+0000 7f75a0c5eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ccca874a0 0x559cccc0ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.779+0000 7f75a0c5eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 0x559cccc04c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.779+0000 7f75a0c5eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559cccb26ff0 con 0x559cccb71160 2024-05-09T02:24:48.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.779+0000 7f75a0c5eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559cccac57f0 con 0x559ccca874a0 2024-05-09T02:24:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.779+0000 7f75a0c5eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559cccb22fb0 con 0x559cccc04800 2024-05-09T02:24:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.779+0000 7f758b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ccca874a0 0x559cccc0ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.779+0000 7f758b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ccca874a0 0x559cccc0ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:39848/0 (socket says 172.21.15.19:39848) 2024-05-09T02:24:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.779+0000 7f758b7fe700 1 -- 172.21.15.19:0/3546010934 learned_addr learned my addr 172.21.15.19:0/3546010934 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.779+0000 7f758affd700 1 --2- 172.21.15.19:0/3546010934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cccb71160 0x559cccc0a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.780+0000 7f758affd700 1 --2- 172.21.15.19:0/3546010934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 0x559cccc04c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.780+0000 7f758b7fe700 1 -- 172.21.15.19:0/3546010934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 msgr2=0x559cccc04c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.780+0000 7f758b7fe700 1 --2- 172.21.15.19:0/3546010934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 0x559cccc04c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.780+0000 7f758b7fe700 1 -- 172.21.15.19:0/3546010934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cccb71160 msgr2=0x559cccc0a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.780+0000 7f758b7fe700 1 --2- 172.21.15.19:0/3546010934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cccb71160 0x559cccc0a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.780+0000 7f758b7fe700 1 -- 172.21.15.19:0/3546010934 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cccabd370 con 0x559ccca874a0 2024-05-09T02:24:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.780+0000 7f758affd700 1 --2- 172.21.15.19:0/3546010934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 0x559cccc04c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:24:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.780+0000 7f7588ff9700 1 --2- 172.21.15.19:0/330755747 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f75740b2ab0 0x7f75740b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.780+0000 7f758b7fe700 1 --2- 172.21.15.19:0/3546010934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ccca874a0 0x559cccc0ac80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f757c01a690 tx=0x7f757c01a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a55030d4d22d91af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.780+0000 7f7588ff9700 1 -- 172.21.15.19:0/330755747 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f75800e2810 con 0x7f758c1096a0 2024-05-09T02:24:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.780+0000 7f7571ffb700 1 -- 172.21.15.19:0/3546010934 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c0039a0 con 0x559ccca874a0 2024-05-09T02:24:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.780+0000 7f7571ffb700 1 -- 172.21.15.19:0/3546010934 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f757c003b40 con 0x559ccca874a0 2024-05-09T02:24:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.780+0000 7f7571ffb700 1 -- 172.21.15.19:0/3546010934 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c0283f0 con 0x559ccca874a0 2024-05-09T02:24:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.781+0000 7f758affd700 1 --2- 172.21.15.19:0/330755747 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f75740b2ab0 0x7f75740b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.781+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/3546010934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ccca874a0 msgr2=0x559cccc0ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.781+0000 7f758affd700 1 --2- 172.21.15.19:0/330755747 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f75740b2ab0 0x7f75740b4fc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f75840060b0 tx=0x7f7584009340 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.781+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/3546010934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ccca874a0 0x559cccc0ac80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f757c01a690 tx=0x7f757c01a6c0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.782+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/3546010934 shutdown_connections 2024-05-09T02:24:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.782+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/3546010934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 0x559cccc04c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.782+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/3546010934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cccb71160 0x559cccc0a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.782+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/3546010934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ccca874a0 0x559cccc0ac80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.782+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/3546010934 >> 172.21.15.19:0/3546010934 conn(0x559cccae76a0 msgr2=0x559cccbff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.782+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/3546010934 shutdown_connections 2024-05-09T02:24:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.782+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/3546010934 wait complete. 2024-05-09T02:24:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.782+0000 7f75a0c5eb80 1 -- start start 2024-05-09T02:24:48.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.783+0000 7f75a0c5eb80 1 Processor -- start 2024-05-09T02:24:48.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.783+0000 7f75a0c5eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ccca874a0 0x559cccc0e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.783+0000 7f75a0c5eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cccb71160 0x559cccc0e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.783+0000 7f75a0c5eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 0x559cccc0ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.783+0000 7f75a0c5eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559ccc9a2ba0 con 0x559cccb71160 2024-05-09T02:24:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.783+0000 7f75a0c5eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559cccae1150 con 0x559ccca874a0 2024-05-09T02:24:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.783+0000 7f75a0c5eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559cccc05660 con 0x559cccc04800 2024-05-09T02:24:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.783+0000 7f75a0c5eb80 10 client.0 _ll_register_callbacks cb 0x559ccc9986c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:24:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.784+0000 7f758b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ccca874a0 0x559cccc0e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.784+0000 7f758affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cccb71160 0x559cccc0e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.784+0000 7f758b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ccca874a0 0x559cccc0e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:39856/0 (socket says 172.21.15.19:39856) 2024-05-09T02:24:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.784+0000 7f758b7fe700 1 -- 172.21.15.19:0/2678638231 learned_addr learned my addr 172.21.15.19:0/2678638231 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.784+0000 7f758bfff700 1 --2- 172.21.15.19:0/2678638231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 0x559cccc0ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.784+0000 7f758b7fe700 1 -- 172.21.15.19:0/2678638231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 msgr2=0x559cccc0ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.784+0000 7f758b7fe700 1 --2- 172.21.15.19:0/2678638231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 0x559cccc0ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.784+0000 7f758b7fe700 1 -- 172.21.15.19:0/2678638231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cccb71160 msgr2=0x559cccc0e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.784+0000 7f758b7fe700 1 --2- 172.21.15.19:0/2678638231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cccb71160 0x559cccc0e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.784+0000 7f758b7fe700 1 -- 172.21.15.19:0/2678638231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ccc9a2490 con 0x559ccca874a0 2024-05-09T02:24:48.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.784+0000 7f758bfff700 1 --2- 172.21.15.19:0/2678638231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 0x559cccc0ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:24:48.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.784+0000 7f758b7fe700 1 --2- 172.21.15.19:0/2678638231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ccca874a0 0x559cccc0e3c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f757c01a8f0 tx=0x7f757c005300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=342c427ebad2ee87 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:48.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.784+0000 7f756affd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:24:48.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.785+0000 7f756affd700 1 -- 172.21.15.19:0/2678638231 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c0039a0 con 0x559ccca874a0 2024-05-09T02:24:48.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.785+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2678638231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f757c003b40 con 0x559ccca874a0 2024-05-09T02:24:48.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.785+0000 7f756affd700 1 -- 172.21.15.19:0/2678638231 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f757c033440 con 0x559ccca874a0 2024-05-09T02:24:48.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.785+0000 7f756affd700 1 -- 172.21.15.19:0/2678638231 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f757c033640 con 0x559ccca874a0 2024-05-09T02:24:48.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.785+0000 7f75a0c5eb80 10 client.26690 fetch_fsmap learned FSMap version 58 2024-05-09T02:24:48.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.785+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2678638231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x559cccc1ba80 con 0x559ccca874a0 2024-05-09T02:24:48.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.785+0000 7f756affd700 1 -- 172.21.15.19:0/2678638231 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f757c003b40 con 0x559ccca874a0 2024-05-09T02:24:48.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.785+0000 7f756affd700 1 -- 172.21.15.19:0/2678638231 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f757c031020 con 0x559ccca874a0 2024-05-09T02:24:48.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.785+0000 7f75a0c5eb80 10 client.26690 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:24:48.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.785+0000 7f75a0c5eb80 10 client.26690 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:24:48.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.786+0000 7f75a0c5eb80 20 client.26690 populate_metadata read hostname 'smithi019' 2024-05-09T02:24:48.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.786+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2678638231 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559cccc1df50 0x559cccc20370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.786+0000 7f75a0c5eb80 4 client.26690 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:24:48.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.786+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2678638231 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cccb349d0 con 0x559cccc1df50 2024-05-09T02:24:48.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.786+0000 7f758affd700 1 --2- 172.21.15.19:0/2678638231 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559cccc1df50 0x559cccc20370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.788+0000 7f758affd700 1 --2- 172.21.15.19:0/2678638231 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559cccc1df50 0x559cccc20370 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bbc30edd9700b3f6 server_cookie=59452e16b49679d1 in_seq=0 out_seq=0 2024-05-09T02:24:48.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.788+0000 7f756affd700 10 client.26690 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:24:48.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.788+0000 7f756affd700 1 -- 172.21.15.19:0/2678638231 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559cccb349d0 con 0x559cccc1df50 2024-05-09T02:24:48.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.788+0000 7f756affd700 10 client.26690 handle_command_reply: tid=0 2024-05-09T02:24:48.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.788+0000 7f75a0c5eb80 1 client.26690 shutdown 2024-05-09T02:24:48.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.789+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2678638231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ccca874a0 msgr2=0x559cccc0e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:48.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.789+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2678638231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ccca874a0 0x559cccc0e3c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f757c01a8f0 tx=0x7f757c005300 comp rx=0 tx=0).stop 2024-05-09T02:24:48.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.789+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2678638231 shutdown_connections 2024-05-09T02:24:48.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.789+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2678638231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 0x559cccc0ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.789+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2678638231 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559cccc1df50 0x559cccc20370 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.789+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2678638231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ccca874a0 0x559cccc0e3c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.789+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2678638231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cccb71160 0x559cccc0e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.789+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2678638231 >> 172.21.15.19:0/2678638231 conn(0x559cccae76a0 msgr2=0x559cccbff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:48.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.789+0000 7f756affd700 0 client.26690 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:24:48.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.789+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2678638231 shutdown_connections 2024-05-09T02:24:48.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.789+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2678638231 wait complete. 2024-05-09T02:24:48.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.790+0000 7f75a0c5eb80 20 client.26690 trim_cache size 0 max 16384 2024-05-09T02:24:48.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:48 smithi196 ceph-mon[34718]: pgmap v422: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 1.08k op/s 2024-05-09T02:24:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.807+0000 7f75a0c5eb80 1 Processor -- start 2024-05-09T02:24:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.807+0000 7f75a0c5eb80 1 -- start start 2024-05-09T02:24:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.808+0000 7f75a0c5eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ccca874a0 0x559cccc1a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.808+0000 7f75a0c5eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cccb71160 0x559cccc09bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.808+0000 7f75a0c5eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 0x559cccc0ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.808+0000 7f75a0c5eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f758c1aca80 con 0x559ccca874a0 2024-05-09T02:24:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.808+0000 7f75a0c5eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f758c1ae5c0 con 0x559cccb71160 2024-05-09T02:24:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.808+0000 7f758b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ccca874a0 0x559cccc1a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.808+0000 7f758b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ccca874a0 0x559cccc1a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37964/0 (socket says 172.21.15.19:37964) 2024-05-09T02:24:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.808+0000 7f758b7fe700 1 -- 172.21.15.19:0/2292391984 learned_addr learned my addr 172.21.15.19:0/2292391984 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.808+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2292391984 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f758c1ad150 con 0x559cccc04800 2024-05-09T02:24:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.808+0000 7f758affd700 1 --2- 172.21.15.19:0/2292391984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cccb71160 0x559cccc09bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.808+0000 7f758bfff700 1 --2- 172.21.15.19:0/2292391984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 0x559cccc0ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.809+0000 7f758bfff700 1 -- 172.21.15.19:0/2292391984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cccb71160 msgr2=0x559cccc09bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.809+0000 7f758bfff700 1 --2- 172.21.15.19:0/2292391984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cccb71160 0x559cccc09bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.809+0000 7f758bfff700 1 -- 172.21.15.19:0/2292391984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ccca874a0 msgr2=0x559cccc1a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.809+0000 7f758bfff700 1 --2- 172.21.15.19:0/2292391984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ccca874a0 0x559cccc1a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.809+0000 7f758bfff700 1 -- 172.21.15.19:0/2292391984 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cccc005e0 con 0x559cccc04800 2024-05-09T02:24:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.809+0000 7f758bfff700 1 --2- 172.21.15.19:0/2292391984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 0x559cccc0ec80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f75800dff30 tx=0x7f75800a9410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e633dcc714fd306 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.809+0000 7f75717fa700 1 -- 172.21.15.19:0/2292391984 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758000cd70 con 0x559cccc04800 2024-05-09T02:24:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.810+0000 7f75717fa700 1 -- 172.21.15.19:0/2292391984 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75800b2260 con 0x559cccc04800 2024-05-09T02:24:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.810+0000 7f75717fa700 1 -- 172.21.15.19:0/2292391984 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75800a9f40 con 0x559cccc04800 2024-05-09T02:24:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.811+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2292391984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 msgr2=0x559cccc0ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.811+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2292391984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 0x559cccc0ec80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f75800dff30 tx=0x7f75800a9410 comp rx=0 tx=0).stop 2024-05-09T02:24:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.811+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2292391984 shutdown_connections 2024-05-09T02:24:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.811+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2292391984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cccc04800 0x559cccc0ec80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.811+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2292391984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559ccca874a0 0x559cccc1a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.811+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2292391984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cccb71160 0x559cccc09bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.811+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2292391984 >> 172.21.15.19:0/2292391984 conn(0x559cccae76a0 msgr2=0x559cccc094b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.811+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2292391984 shutdown_connections 2024-05-09T02:24:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.811+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2292391984 wait complete. 2024-05-09T02:24:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.812+0000 7f75a0c5eb80 1 -- start start 2024-05-09T02:24:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.812+0000 7f75a0c5eb80 1 Processor -- start 2024-05-09T02:24:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.812+0000 7f75a0c5eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ccca874a0 0x559cccc1ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f75a0c5eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cccb71160 0x559cccc204a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f75a0c5eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cccc04800 0x559cccc209e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f75a0c5eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f758c1adef0 con 0x559cccb71160 2024-05-09T02:24:48.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f75a0c5eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f758c1ad820 con 0x559cccc04800 2024-05-09T02:24:48.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f75a0c5eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f758c1aaa40 con 0x559ccca874a0 2024-05-09T02:24:48.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f75a0c5eb80 10 client.0 _ll_register_callbacks cb 0x559ccc9986c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:24:48.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f758b7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ccca874a0 0x559cccc1ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f758affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cccb71160 0x559cccc204a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f758b7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ccca874a0 0x559cccc1ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40664/0 (socket says 172.21.15.19:40664) 2024-05-09T02:24:48.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f758b7fe700 1 -- 172.21.15.19:0/2338946736 learned_addr learned my addr 172.21.15.19:0/2338946736 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:48.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f758bfff700 1 --2- 172.21.15.19:0/2338946736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cccc04800 0x559cccc209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f758b7fe700 1 -- 172.21.15.19:0/2338946736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cccc04800 msgr2=0x559cccc209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:48.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f758b7fe700 1 --2- 172.21.15.19:0/2338946736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cccc04800 0x559cccc209e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f758b7fe700 1 -- 172.21.15.19:0/2338946736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cccb71160 msgr2=0x559cccc204a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:48.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f758b7fe700 1 --2- 172.21.15.19:0/2338946736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cccb71160 0x559cccc204a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f758b7fe700 1 -- 172.21.15.19:0/2338946736 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cccc86680 con 0x559ccca874a0 2024-05-09T02:24:48.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f758b7fe700 1 --2- 172.21.15.19:0/2338946736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ccca874a0 0x559cccc1ff60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f757c0446b0 tx=0x7f757c0447c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccc5a33c53ac8786 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:48.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f756affd700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:24:48.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.813+0000 7f756affd700 1 -- 172.21.15.19:0/2338946736 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c0039a0 con 0x559ccca874a0 2024-05-09T02:24:48.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.814+0000 7f756affd700 1 -- 172.21.15.19:0/2338946736 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f757c003b40 con 0x559ccca874a0 2024-05-09T02:24:48.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.814+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2338946736 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559cccb2ea70 con 0x559ccca874a0 2024-05-09T02:24:48.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.814+0000 7f756affd700 1 -- 172.21.15.19:0/2338946736 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f757c044bd0 con 0x559ccca874a0 2024-05-09T02:24:48.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.814+0000 7f75a0c5eb80 10 client.26737 fetch_fsmap learned FSMap version 58 2024-05-09T02:24:48.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.814+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2338946736 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x559cccc878e0 con 0x559ccca874a0 2024-05-09T02:24:48.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.814+0000 7f756affd700 1 -- 172.21.15.19:0/2338946736 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f757c044d70 con 0x559ccca874a0 2024-05-09T02:24:48.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.814+0000 7f756affd700 1 -- 172.21.15.19:0/2338946736 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f757c031040 con 0x559ccca874a0 2024-05-09T02:24:48.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.814+0000 7f75a0c5eb80 10 client.26737 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:24:48.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.814+0000 7f75a0c5eb80 10 client.26737 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:24:48.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.814+0000 7f75a0c5eb80 20 client.26737 populate_metadata read hostname 'smithi019' 2024-05-09T02:24:48.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.814+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2338946736 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559cccc87da0 0x559cccc8a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:48.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.814+0000 7f75a0c5eb80 4 client.26737 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:24:48.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.814+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2338946736 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559cccb2ea70 con 0x559cccc87da0 2024-05-09T02:24:48.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.815+0000 7f758affd700 1 --2- 172.21.15.19:0/2338946736 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559cccc87da0 0x559cccc8a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:48.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.815+0000 7f756affd700 10 client.26737 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:24:48.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.815+0000 7f758affd700 1 --2- 172.21.15.19:0/2338946736 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559cccc87da0 0x559cccc8a1c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bfda2ac318f538ff server_cookie=bdefdb32a48345c3 in_seq=0 out_seq=0 2024-05-09T02:24:48.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.815+0000 7f756affd700 1 -- 172.21.15.19:0/2338946736 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559cccb2ea70 con 0x559cccc87da0 2024-05-09T02:24:48.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.815+0000 7f756affd700 10 client.26737 handle_command_reply: tid=0 2024-05-09T02:24:48.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.816+0000 7f75a0c5eb80 1 client.26737 shutdown 2024-05-09T02:24:48.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.816+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2338946736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ccca874a0 msgr2=0x559cccc1ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:48.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.816+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2338946736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ccca874a0 0x559cccc1ff60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f757c0446b0 tx=0x7f757c0447c0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.816+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2338946736 shutdown_connections 2024-05-09T02:24:48.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.816+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2338946736 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x559cccc87da0 0x559cccc8a1c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.816+0000 7f756affd700 0 client.26737 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:24:48.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.816+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2338946736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cccc04800 0x559cccc209e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.816+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2338946736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ccca874a0 0x559cccc1ff60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.816+0000 7f75a0c5eb80 1 --2- 172.21.15.19:0/2338946736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cccb71160 0x559cccc204a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.816+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2338946736 >> 172.21.15.19:0/2338946736 conn(0x559cccae76a0 msgr2=0x559cccc08050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:48.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.816+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2338946736 shutdown_connections 2024-05-09T02:24:48.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.816+0000 7f75a0c5eb80 1 -- 172.21.15.19:0/2338946736 wait complete. 2024-05-09T02:24:48.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.816+0000 7f75a0c5eb80 20 client.26737 trim_cache size 0 max 16384 2024-05-09T02:24:48.822 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:24:48.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.817+0000 7f7590fe6700 1 -- 172.21.15.19:0/330755747 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f75740b2ab0 msgr2=0x7f75740b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:48.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.817+0000 7f7590fe6700 1 --2- 172.21.15.19:0/330755747 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f75740b2ab0 0x7f75740b4fc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f75840060b0 tx=0x7f7584009340 comp rx=0 tx=0).stop 2024-05-09T02:24:48.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.817+0000 7f7590fe6700 1 -- 172.21.15.19:0/330755747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f758c1096a0 msgr2=0x7f758c10ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:48.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.817+0000 7f7590fe6700 1 --2- 172.21.15.19:0/330755747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f758c1096a0 0x7f758c10ad30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f758000e940 tx=0x7f758000ec50 comp rx=0 tx=0).stop 2024-05-09T02:24:48.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.817+0000 7f7590fe6700 1 -- 172.21.15.19:0/330755747 shutdown_connections 2024-05-09T02:24:48.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.817+0000 7f7590fe6700 1 --2- 172.21.15.19:0/330755747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f758c1096a0 0x7f758c10ad30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.817+0000 7f7590fe6700 1 --2- 172.21.15.19:0/330755747 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f75740b2ab0 0x7f75740b4fc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.817+0000 7f7590fe6700 1 --2- 172.21.15.19:0/330755747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f758c104420 0x7f758c110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.817+0000 7f7590fe6700 1 --2- 172.21.15.19:0/330755747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f758c108d50 0x7f758c110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.817+0000 7f7590fe6700 1 -- 172.21.15.19:0/330755747 >> 172.21.15.19:0/330755747 conn(0x7f758c07acb0 msgr2=0x7f758c1034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.818+0000 7f7590fe6700 1 -- 172.21.15.19:0/330755747 shutdown_connections 2024-05-09T02:24:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:48.819+0000 7f7590fe6700 1 -- 172.21.15.19:0/330755747 wait complete. 2024-05-09T02:24:48.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:48 smithi059 ceph-mon[34781]: pgmap v422: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 1.08k op/s 2024-05-09T02:24:50.298 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T02:24:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:24:50.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:50 smithi196 ceph-mon[34718]: pgmap v423: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 1.19k op/s 2024-05-09T02:24:50.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:50 smithi019 ceph-mon[30151]: pgmap v423: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 1.19k op/s 2024-05-09T02:24:50.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:50 smithi059 ceph-mon[34781]: pgmap v423: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 1.19k op/s 2024-05-09T02:24:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 790.0 s, 1337.1 tps, lat 74.227 ms stddev 80.741 2024-05-09T02:24:52.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.695+0000 7f13ec1e8700 1 -- 172.21.15.19:0/2045477891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13e41034e0 msgr2=0x7f13e41038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.695+0000 7f13ec1e8700 1 --2- 172.21.15.19:0/2045477891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13e41034e0 0x7f13e41038f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f13d40099c0 tx=0x7f13d4009cd0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.695+0000 7f13ec1e8700 1 -- 172.21.15.19:0/2045477891 shutdown_connections 2024-05-09T02:24:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.695+0000 7f13ec1e8700 1 --2- 172.21.15.19:0/2045477891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13e4108760 0x7f13e41100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.695+0000 7f13ec1e8700 1 --2- 172.21.15.19:0/2045477891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13e41034e0 0x7f13e41038f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.695+0000 7f13ec1e8700 1 --2- 172.21.15.19:0/2045477891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e4107e10 0x7f13e4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.695+0000 7f13ec1e8700 1 -- 172.21.15.19:0/2045477891 >> 172.21.15.19:0/2045477891 conn(0x7f13e40fef30 msgr2=0x7f13e4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:52.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.696+0000 7f13ec1e8700 1 -- 172.21.15.19:0/2045477891 shutdown_connections 2024-05-09T02:24:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.696+0000 7f13ec1e8700 1 -- 172.21.15.19:0/2045477891 wait complete. 2024-05-09T02:24:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.696+0000 7f13ec1e8700 1 Processor -- start 2024-05-09T02:24:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.697+0000 7f13ec1e8700 1 -- start start 2024-05-09T02:24:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.697+0000 7f13ec1e8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e41034e0 0x7f13e419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.697+0000 7f13ec1e8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13e4107e10 0x7f13e419c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.697+0000 7f13ec1e8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13e4108760 0x7f13e419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.697+0000 7f13ec1e8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13e41134b0 con 0x7f13e41034e0 2024-05-09T02:24:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.697+0000 7f13ec1e8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4113330 con 0x7f13e4107e10 2024-05-09T02:24:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.697+0000 7f13ec1e8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4113630 con 0x7f13e4108760 2024-05-09T02:24:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.697+0000 7f13ea785700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13e4108760 0x7f13e419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.697+0000 7f13e9f84700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e41034e0 0x7f13e419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.697+0000 7f13ea785700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13e4108760 0x7f13e419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40672/0 (socket says 172.21.15.19:40672) 2024-05-09T02:24:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.697+0000 7f13ea785700 1 -- 172.21.15.19:0/1299348072 learned_addr learned my addr 172.21.15.19:0/1299348072 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.697+0000 7f13e9783700 1 --2- 172.21.15.19:0/1299348072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13e4107e10 0x7f13e419c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.698+0000 7f13ea785700 1 -- 172.21.15.19:0/1299348072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13e4107e10 msgr2=0x7f13e419c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.698+0000 7f13ea785700 1 --2- 172.21.15.19:0/1299348072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13e4107e10 0x7f13e419c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.698+0000 7f13ea785700 1 -- 172.21.15.19:0/1299348072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e41034e0 msgr2=0x7f13e419bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.698+0000 7f13ea785700 1 --2- 172.21.15.19:0/1299348072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e41034e0 0x7f13e419bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.698+0000 7f13ea785700 1 -- 172.21.15.19:0/1299348072 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e4199a80 con 0x7f13e4108760 2024-05-09T02:24:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.698+0000 7f13ea785700 1 --2- 172.21.15.19:0/1299348072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13e4108760 0x7f13e419c630 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00b970 tx=0x7f13dc00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.699+0000 7f13daffd700 1 -- 172.21.15.19:0/1299348072 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc019070 con 0x7f13e4108760 2024-05-09T02:24:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.699+0000 7f13ec1e8700 1 -- 172.21.15.19:0/1299348072 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e4199d70 con 0x7f13e4108760 2024-05-09T02:24:52.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.700+0000 7f13ec1e8700 1 -- 172.21.15.19:0/1299348072 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e419a2b0 con 0x7f13e4108760 2024-05-09T02:24:52.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.701+0000 7f13daffd700 1 -- 172.21.15.19:0/1299348072 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13dc00cde0 con 0x7f13e4108760 2024-05-09T02:24:52.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.702+0000 7f13daffd700 1 -- 172.21.15.19:0/1299348072 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc007370 con 0x7f13e4108760 2024-05-09T02:24:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.704+0000 7f13daffd700 1 -- 172.21.15.19:0/1299348072 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f13dc01c070 con 0x7f13e4108760 2024-05-09T02:24:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.707+0000 7f13daffd700 1 --2- 172.21.15.19:0/1299348072 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f13d00b2b20 0x7f13d00b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.707+0000 7f13daffd700 1 -- 172.21.15.19:0/1299348072 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f13dc0e1840 con 0x7f13e4108760 2024-05-09T02:24:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.708+0000 7f13fabfeb80 1 Processor -- start 2024-05-09T02:24:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.708+0000 7f13e9f84700 1 --2- 172.21.15.19:0/1299348072 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f13d00b2b20 0x7f13d00b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.708+0000 7f13fabfeb80 1 -- start start 2024-05-09T02:24:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.708+0000 7f13fabfeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335add160 0x55d335b764f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.708+0000 7f13fabfeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3359f34a0 0x55d335b76a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.708+0000 7f13fabfeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d335b706a0 0x55d335b70ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.708+0000 7f13fabfeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d335a92ff0 con 0x55d335add160 2024-05-09T02:24:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.708+0000 7f13fabfeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d335a317f0 con 0x55d335b706a0 2024-05-09T02:24:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.708+0000 7f13fabfeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d335a8efb0 con 0x55d3359f34a0 2024-05-09T02:24:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.708+0000 7f13e9783700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3359f34a0 0x55d335b76a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.708+0000 7f13e9783700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3359f34a0 0x55d335b76a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40686/0 (socket says 172.21.15.19:40686) 2024-05-09T02:24:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.708+0000 7f13e9783700 1 -- 172.21.15.19:0/8677709 learned_addr learned my addr 172.21.15.19:0/8677709 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.709+0000 7f13ea785700 1 --2- 172.21.15.19:0/8677709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d335b706a0 0x55d335b70ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.709+0000 7f13ea785700 1 -- 172.21.15.19:0/8677709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3359f34a0 msgr2=0x55d335b76a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.709+0000 7f13ea785700 1 --2- 172.21.15.19:0/8677709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3359f34a0 0x55d335b76a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.709+0000 7f13ea785700 1 -- 172.21.15.19:0/8677709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335add160 msgr2=0x55d335b764f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.709+0000 7f13ea785700 1 --2- 172.21.15.19:0/8677709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335add160 0x55d335b764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.709+0000 7f13e9f84700 1 --2- 172.21.15.19:0/1299348072 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f13d00b2b20 0x7f13d00b5030 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f13d4003800 tx=0x7f13d4009f10 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.709+0000 7f13ea785700 1 -- 172.21.15.19:0/8677709 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d335a29370 con 0x55d335b706a0 2024-05-09T02:24:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.709+0000 7f13ea785700 1 --2- 172.21.15.19:0/8677709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d335b706a0 0x55d335b70ad0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0df010 tx=0x7f13dc0a97a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d4b5ff489e54078 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.709+0000 7f13cffff700 1 -- 172.21.15.19:0/8677709 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc0adfc0 con 0x55d335b706a0 2024-05-09T02:24:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.709+0000 7f13cffff700 1 -- 172.21.15.19:0/8677709 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13dc0c9e50 con 0x55d335b706a0 2024-05-09T02:24:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.709+0000 7f13cffff700 1 -- 172.21.15.19:0/8677709 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc0a8e70 con 0x55d335b706a0 2024-05-09T02:24:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.710+0000 7f13fabfeb80 1 -- 172.21.15.19:0/8677709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d335b706a0 msgr2=0x55d335b70ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.710+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/8677709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d335b706a0 0x55d335b70ad0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0df010 tx=0x7f13dc0a97a0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.710+0000 7f13fabfeb80 1 -- 172.21.15.19:0/8677709 shutdown_connections 2024-05-09T02:24:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.710+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/8677709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d335b706a0 0x55d335b70ad0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.710+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/8677709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335add160 0x55d335b764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.711+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/8677709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3359f34a0 0x55d335b76a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.711+0000 7f13fabfeb80 1 -- 172.21.15.19:0/8677709 >> 172.21.15.19:0/8677709 conn(0x55d335a536a0 msgr2=0x55d335b6b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.711+0000 7f13fabfeb80 1 -- 172.21.15.19:0/8677709 shutdown_connections 2024-05-09T02:24:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.711+0000 7f13fabfeb80 1 -- 172.21.15.19:0/8677709 wait complete. 2024-05-09T02:24:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.711+0000 7f13fabfeb80 1 -- start start 2024-05-09T02:24:52.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.712+0000 7f13fabfeb80 1 Processor -- start 2024-05-09T02:24:52.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.713+0000 7f13fabfeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3359f34a0 0x55d335b7a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.713+0000 7f13fabfeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d335add160 0x55d335b7a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.713+0000 7f13fabfeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335b706a0 0x55d335b7acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.713+0000 7f13e9783700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d335add160 0x55d335b7a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:52.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.713+0000 7f13e9783700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d335add160 0x55d335b7a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40700/0 (socket says 172.21.15.19:40700) 2024-05-09T02:24:52.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.713+0000 7f13e9783700 1 -- 172.21.15.19:0/1553327212 learned_addr learned my addr 172.21.15.19:0/1553327212 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:52.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.713+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1553327212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d33590eba0 con 0x55d335b706a0 2024-05-09T02:24:52.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.713+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1553327212 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d335a4d150 con 0x55d3359f34a0 2024-05-09T02:24:52.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.713+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1553327212 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d335b7d1d0 con 0x55d335add160 2024-05-09T02:24:52.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.714+0000 7f13fabfeb80 10 client.0 _ll_register_callbacks cb 0x55d3359046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:24:52.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.714+0000 7f13e9783700 1 -- 172.21.15.19:0/1553327212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3359f34a0 msgr2=0x55d335b7a230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:24:52.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.714+0000 7f13e9783700 1 --2- 172.21.15.19:0/1553327212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3359f34a0 0x55d335b7a230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.714+0000 7f13e9783700 1 -- 172.21.15.19:0/1553327212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335b706a0 msgr2=0x55d335b7acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:52.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.714+0000 7f13e9783700 1 --2- 172.21.15.19:0/1553327212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335b706a0 0x55d335b7acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.714+0000 7f13e9783700 1 -- 172.21.15.19:0/1553327212 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d33590e490 con 0x55d335add160 2024-05-09T02:24:52.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.714+0000 7f13e9783700 1 --2- 172.21.15.19:0/1553327212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d335add160 0x55d335b7a770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f13e000bf50 tx=0x7f13e0011770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9595e420f1133985 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:52.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.714+0000 7f13cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:24:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.715+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1553327212 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13dc0c9e50 con 0x55d335add160 2024-05-09T02:24:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.715+0000 7f13cd7fa700 1 -- 172.21.15.19:0/1553327212 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0012450 con 0x55d335add160 2024-05-09T02:24:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.715+0000 7f13cd7fa700 1 -- 172.21.15.19:0/1553327212 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f13e00125f0 con 0x55d335add160 2024-05-09T02:24:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.716+0000 7f13cd7fa700 1 -- 172.21.15.19:0/1553327212 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f13e00127f0 con 0x55d335add160 2024-05-09T02:24:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.716+0000 7f13fabfeb80 10 client.26749 fetch_fsmap learned FSMap version 58 2024-05-09T02:24:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.716+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1553327212 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=58}) -- 0x55d335b901f0 con 0x55d335add160 2024-05-09T02:24:52.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.716+0000 7f13cd7fa700 1 -- 172.21.15.19:0/1553327212 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f13e0012990 con 0x55d335add160 2024-05-09T02:24:52.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.716+0000 7f13fabfeb80 10 client.26749 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:24:52.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.716+0000 7f13fabfeb80 10 client.26749 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:24:52.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.716+0000 7f13fabfeb80 20 client.26749 populate_metadata read hostname 'smithi019' 2024-05-09T02:24:52.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.716+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/1553327212 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d335b92b60 0x55d335b95050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.716+0000 7f13fabfeb80 4 client.26749 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:24:52.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.716+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1553327212 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f13dc0c9e50 con 0x55d335b92b60 2024-05-09T02:24:52.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.717+0000 7f13e9f84700 1 --2- 172.21.15.19:0/1553327212 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d335b92b60 0x55d335b95050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:52.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.717+0000 7f13e9f84700 1 --2- 172.21.15.19:0/1553327212 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d335b92b60 0x55d335b95050 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9bb7e9531f9197e9 server_cookie=e97640d57f5fbfd3 in_seq=0 out_seq=0 2024-05-09T02:24:52.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.717+0000 7f13cd7fa700 10 client.26749 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:24:52.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.718+0000 7f13cd7fa700 1 -- 172.21.15.19:0/1553327212 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f13dc0c9e50 con 0x55d335b92b60 2024-05-09T02:24:52.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.718+0000 7f13cd7fa700 10 client.26749 handle_command_reply: tid=0 2024-05-09T02:24:52.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.718+0000 7f13fabfeb80 1 client.26749 shutdown 2024-05-09T02:24:52.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.718+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1553327212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d335add160 msgr2=0x55d335b7a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:52.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.718+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/1553327212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d335add160 0x55d335b7a770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f13e000bf50 tx=0x7f13e0011770 comp rx=0 tx=0).stop 2024-05-09T02:24:52.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.719+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1553327212 shutdown_connections 2024-05-09T02:24:52.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.719+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/1553327212 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d335b92b60 0x55d335b95050 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.719+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/1553327212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335b706a0 0x55d335b7acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.719+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/1553327212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3359f34a0 0x55d335b7a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.719+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/1553327212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d335add160 0x55d335b7a770 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.719+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1553327212 >> 172.21.15.19:0/1553327212 conn(0x55d335a536a0 msgr2=0x55d335b6b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.719+0000 7f13cd7fa700 0 client.26749 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:24:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.719+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1553327212 shutdown_connections 2024-05-09T02:24:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.719+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1553327212 wait complete. 2024-05-09T02:24:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.720+0000 7f13fabfeb80 20 client.26749 trim_cache size 0 max 16384 2024-05-09T02:24:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.739+0000 7f13fabfeb80 1 Processor -- start 2024-05-09T02:24:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.739+0000 7f13fabfeb80 1 -- start start 2024-05-09T02:24:52.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:52 smithi019 ceph-mon[30151]: pgmap v424: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 13 MiB/s wr, 1.18k op/s 2024-05-09T02:24:52.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:52 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:24:52.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.739+0000 7f13fabfeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3359f34a0 0x55d335b77e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.739+0000 7f13fabfeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335add160 0x55d335b8ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.739+0000 7f13fabfeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d335b706a0 0x55d335b7a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.739+0000 7f13fabfeb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13c800d790 con 0x55d335add160 2024-05-09T02:24:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.739+0000 7f13fabfeb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13c800f2d0 con 0x55d3359f34a0 2024-05-09T02:24:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.739+0000 7f13fabfeb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f13c800de60 con 0x55d335b706a0 2024-05-09T02:24:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.742+0000 7f13e9783700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335add160 0x55d335b8ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.742+0000 7f13e9783700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335add160 0x55d335b8ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38022/0 (socket says 172.21.15.19:38022) 2024-05-09T02:24:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.742+0000 7f13e9783700 1 -- 172.21.15.19:0/1448914732 learned_addr learned my addr 172.21.15.19:0/1448914732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.742+0000 7f13e9f84700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3359f34a0 0x55d335b77e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.742+0000 7f13ea785700 1 --2- 172.21.15.19:0/1448914732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d335b706a0 0x55d335b7a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.742+0000 7f13ea785700 1 -- 172.21.15.19:0/1448914732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3359f34a0 msgr2=0x55d335b77e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.742+0000 7f13ea785700 1 --2- 172.21.15.19:0/1448914732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3359f34a0 0x55d335b77e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.742+0000 7f13ea785700 1 -- 172.21.15.19:0/1448914732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335add160 msgr2=0x55d335b8ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.742+0000 7f13ea785700 1 --2- 172.21.15.19:0/1448914732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335add160 0x55d335b8ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.742+0000 7f13ea785700 1 -- 172.21.15.19:0/1448914732 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d335b71640 con 0x55d335b706a0 2024-05-09T02:24:52.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.742+0000 7f13e9783700 1 --2- 172.21.15.19:0/1448914732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335add160 0x55d335b8ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:24:52.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.743+0000 7f13ea785700 1 --2- 172.21.15.19:0/1448914732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d335b706a0 0x55d335b7a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0a5c40 tx=0x7f13dc00bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ccf5d8ff0d21ad4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.743+0000 7f13cf7fe700 1 -- 172.21.15.19:0/1448914732 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc0a99e0 con 0x55d335b706a0 2024-05-09T02:24:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.743+0000 7f13cf7fe700 1 -- 172.21.15.19:0/1448914732 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13dc0c9e50 con 0x55d335b706a0 2024-05-09T02:24:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.743+0000 7f13cf7fe700 1 -- 172.21.15.19:0/1448914732 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc0a8e70 con 0x55d335b706a0 2024-05-09T02:24:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.744+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1448914732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d335b706a0 msgr2=0x55d335b7a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.744+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/1448914732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d335b706a0 0x55d335b7a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0a5c40 tx=0x7f13dc00bf50 comp rx=0 tx=0).stop 2024-05-09T02:24:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.745+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1448914732 shutdown_connections 2024-05-09T02:24:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.745+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/1448914732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d335b706a0 0x55d335b7a7a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.745+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/1448914732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3359f34a0 0x55d335b77e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.745+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/1448914732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335add160 0x55d335b8ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.745+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1448914732 >> 172.21.15.19:0/1448914732 conn(0x55d335a536a0 msgr2=0x55d335b78b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.745+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1448914732 shutdown_connections 2024-05-09T02:24:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.745+0000 7f13fabfeb80 1 -- 172.21.15.19:0/1448914732 wait complete. 2024-05-09T02:24:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.745+0000 7f13fabfeb80 1 -- start start 2024-05-09T02:24:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.746+0000 7f13fabfeb80 1 Processor -- start 2024-05-09T02:24:52.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.746+0000 7f13fabfeb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3359f34a0 0x55d335b95390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.747+0000 7f13fabfeb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d335add160 0x55d335b958d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.747+0000 7f13fabfeb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335b706a0 0x55d335b97760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.747+0000 7f13e9783700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d335add160 0x55d335b958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:52.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.747+0000 7f13e9783700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d335add160 0x55d335b958d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:39926/0 (socket says 172.21.15.19:39926) 2024-05-09T02:24:52.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.747+0000 7f13e9f84700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3359f34a0 0x55d335b95390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:52.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.747+0000 7f13e9f84700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3359f34a0 0x55d335b95390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40722/0 (socket says 172.21.15.19:40722) 2024-05-09T02:24:52.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.747+0000 7f13e9783700 1 -- 172.21.15.19:0/3423801209 learned_addr learned my addr 172.21.15.19:0/3423801209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:52.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.747+0000 7f13fabfeb80 1 -- 172.21.15.19:0/3423801209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13c800ec00 con 0x55d335b706a0 2024-05-09T02:24:52.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.747+0000 7f13ea785700 1 --2- 172.21.15.19:0/3423801209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335b706a0 0x55d335b97760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:52.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.747+0000 7f13fabfeb80 1 -- 172.21.15.19:0/3423801209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13c800e530 con 0x55d335add160 2024-05-09T02:24:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.747+0000 7f13fabfeb80 1 -- 172.21.15.19:0/3423801209 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f13c800b750 con 0x55d3359f34a0 2024-05-09T02:24:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.747+0000 7f13fabfeb80 10 client.0 _ll_register_callbacks cb 0x55d3359046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:24:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.748+0000 7f13e9f84700 1 -- 172.21.15.19:0/3423801209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d335add160 msgr2=0x55d335b958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.748+0000 7f13e9f84700 1 --2- 172.21.15.19:0/3423801209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d335add160 0x55d335b958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.748+0000 7f13e9f84700 1 -- 172.21.15.19:0/3423801209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335b706a0 msgr2=0x55d335b97760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.748+0000 7f13e9f84700 1 --2- 172.21.15.19:0/3423801209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335b706a0 0x55d335b97760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.748+0000 7f13e9f84700 1 -- 172.21.15.19:0/3423801209 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d335b64f60 con 0x55d3359f34a0 2024-05-09T02:24:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.748+0000 7f13e9f84700 1 --2- 172.21.15.19:0/3423801209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3359f34a0 0x55d335b95390 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009990 tx=0x7f13d400b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a284f3c5832f3b7c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.748+0000 7f13cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:24:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.748+0000 7f13cd7fa700 1 -- 172.21.15.19:0/3423801209 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4005690 con 0x55d3359f34a0 2024-05-09T02:24:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.749+0000 7f13cd7fa700 1 -- 172.21.15.19:0/3423801209 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f13d400b9a0 con 0x55d3359f34a0 2024-05-09T02:24:52.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.749+0000 7f13fabfeb80 1 -- 172.21.15.19:0/3423801209 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13dc0c9e50 con 0x55d3359f34a0 2024-05-09T02:24:52.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.749+0000 7f13cd7fa700 1 -- 172.21.15.19:0/3423801209 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f13d403d050 con 0x55d3359f34a0 2024-05-09T02:24:52.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.749+0000 7f13fabfeb80 10 client.26755 fetch_fsmap learned FSMap version 58 2024-05-09T02:24:52.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.749+0000 7f13fabfeb80 1 -- 172.21.15.19:0/3423801209 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55d335bfcaa0 con 0x55d3359f34a0 2024-05-09T02:24:52.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.749+0000 7f13cd7fa700 1 -- 172.21.15.19:0/3423801209 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13d4032020 con 0x55d3359f34a0 2024-05-09T02:24:52.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.749+0000 7f13cd7fa700 1 -- 172.21.15.19:0/3423801209 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f13d402ece0 con 0x55d3359f34a0 2024-05-09T02:24:52.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.750+0000 7f13fabfeb80 10 client.26755 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:24:52.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.750+0000 7f13fabfeb80 10 client.26755 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:24:52.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.750+0000 7f13fabfeb80 20 client.26755 populate_metadata read hostname 'smithi019' 2024-05-09T02:24:52.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.750+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/3423801209 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d335bfd200 0x55d335bff620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:52.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.750+0000 7f13fabfeb80 4 client.26755 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:24:52.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.750+0000 7f13fabfeb80 1 -- 172.21.15.19:0/3423801209 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f13dc0c9e50 con 0x55d335bfd200 2024-05-09T02:24:52.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.750+0000 7f13e9783700 1 --2- 172.21.15.19:0/3423801209 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d335bfd200 0x55d335bff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:52.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.750+0000 7f13cd7fa700 10 client.26755 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:24:52.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.750+0000 7f13e9783700 1 --2- 172.21.15.19:0/3423801209 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d335bfd200 0x55d335bff620 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38fd1e1e7f950e12 server_cookie=45c4d099e1a42708 in_seq=0 out_seq=0 2024-05-09T02:24:52.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.751+0000 7f13cd7fa700 1 -- 172.21.15.19:0/3423801209 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f13dc0c9e50 con 0x55d335bfd200 2024-05-09T02:24:52.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.751+0000 7f13cd7fa700 10 client.26755 handle_command_reply: tid=0 2024-05-09T02:24:52.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.751+0000 7f13fabfeb80 1 client.26755 shutdown 2024-05-09T02:24:52.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.751+0000 7f13fabfeb80 1 -- 172.21.15.19:0/3423801209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3359f34a0 msgr2=0x55d335b95390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:52.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.751+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/3423801209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3359f34a0 0x55d335b95390 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009990 tx=0x7f13d400b540 comp rx=0 tx=0).stop 2024-05-09T02:24:52.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.751+0000 7f13fabfeb80 1 -- 172.21.15.19:0/3423801209 shutdown_connections 2024-05-09T02:24:52.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.751+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/3423801209 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55d335bfd200 0x55d335bff620 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.751+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/3423801209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d335b706a0 0x55d335b97760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.751+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/3423801209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3359f34a0 0x55d335b95390 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.751+0000 7f13fabfeb80 1 --2- 172.21.15.19:0/3423801209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d335add160 0x55d335b958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.751+0000 7f13fabfeb80 1 -- 172.21.15.19:0/3423801209 >> 172.21.15.19:0/3423801209 conn(0x55d335a536a0 msgr2=0x55d335b73e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:52.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.751+0000 7f13cd7fa700 0 client.26755 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:24:52.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.755+0000 7f13fabfeb80 1 -- 172.21.15.19:0/3423801209 shutdown_connections 2024-05-09T02:24:52.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.755+0000 7f13fabfeb80 1 -- 172.21.15.19:0/3423801209 wait complete. 2024-05-09T02:24:52.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.755+0000 7f13fabfeb80 20 client.26755 trim_cache size 0 max 16384 2024-05-09T02:24:52.759 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:24:52.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.756+0000 7f13ec1e8700 1 -- 172.21.15.19:0/1299348072 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f13d00b2b20 msgr2=0x7f13d00b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:52.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.756+0000 7f13ec1e8700 1 --2- 172.21.15.19:0/1299348072 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f13d00b2b20 0x7f13d00b5030 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f13d4003800 tx=0x7f13d4009f10 comp rx=0 tx=0).stop 2024-05-09T02:24:52.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.756+0000 7f13ec1e8700 1 -- 172.21.15.19:0/1299348072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13e4108760 msgr2=0x7f13e419c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:52.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.756+0000 7f13ec1e8700 1 --2- 172.21.15.19:0/1299348072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13e4108760 0x7f13e419c630 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00b970 tx=0x7f13dc00bc80 comp rx=0 tx=0).stop 2024-05-09T02:24:52.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.756+0000 7f13ec1e8700 1 -- 172.21.15.19:0/1299348072 shutdown_connections 2024-05-09T02:24:52.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.756+0000 7f13ec1e8700 1 --2- 172.21.15.19:0/1299348072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13e4108760 0x7f13e419c630 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.756+0000 7f13ec1e8700 1 --2- 172.21.15.19:0/1299348072 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f13d00b2b20 0x7f13d00b5030 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.756+0000 7f13ec1e8700 1 --2- 172.21.15.19:0/1299348072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e41034e0 0x7f13e419bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.756+0000 7f13ec1e8700 1 --2- 172.21.15.19:0/1299348072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13e4107e10 0x7f13e419c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:52.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.756+0000 7f13ec1e8700 1 -- 172.21.15.19:0/1299348072 >> 172.21.15.19:0/1299348072 conn(0x7f13e40fef30 msgr2=0x7f13e41073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:52.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.757+0000 7f13ec1e8700 1 -- 172.21.15.19:0/1299348072 shutdown_connections 2024-05-09T02:24:52.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:52.757+0000 7f13ec1e8700 1 -- 172.21.15.19:0/1299348072 wait complete. 2024-05-09T02:24:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:52 smithi059 ceph-mon[34781]: pgmap v424: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 13 MiB/s wr, 1.18k op/s 2024-05-09T02:24:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:52 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:24:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:52 smithi196 ceph-mon[34718]: pgmap v424: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 13 MiB/s wr, 1.18k op/s 2024-05-09T02:24:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:52 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:24:53.576 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T02:24:53.825 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:53 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:24:53.825 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:53 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:24:53.825 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:53 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:24:53.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:53 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:24:53.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:53 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:24:53.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:53 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:24:53.990 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:24:54.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:53 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:24:54.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:53 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:24:54.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:53 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:24:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:54 smithi019 ceph-mon[30151]: pgmap v425: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 1.07k op/s 2024-05-09T02:24:54.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:54 smithi059 ceph-mon[34781]: pgmap v425: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 1.07k op/s 2024-05-09T02:24:55.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:54 smithi196 ceph-mon[34718]: pgmap v425: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 1.07k op/s 2024-05-09T02:24:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.911+0000 7f80cbdf7700 1 -- 172.21.15.19:0/3219739454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80c410e8d0 msgr2=0x7f80c4110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.911+0000 7f80cbdf7700 1 --2- 172.21.15.19:0/3219739454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80c410e8d0 0x7f80c4110db0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0098f0 tx=0x7f80bc009c00 comp rx=0 tx=0).stop 2024-05-09T02:24:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.911+0000 7f80cbdf7700 1 -- 172.21.15.19:0/3219739454 shutdown_connections 2024-05-09T02:24:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.911+0000 7f80cbdf7700 1 --2- 172.21.15.19:0/3219739454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80c410e8d0 0x7f80c4110db0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.911+0000 7f80cbdf7700 1 --2- 172.21.15.19:0/3219739454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c4105460 0x7f80c4105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.911+0000 7f80cbdf7700 1 --2- 172.21.15.19:0/3219739454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80c4105db0 0x7f80c410e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.911+0000 7f80cbdf7700 1 -- 172.21.15.19:0/3219739454 >> 172.21.15.19:0/3219739454 conn(0x7f80c40fedc0 msgr2=0x7f80c41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.911+0000 7f80cbdf7700 1 -- 172.21.15.19:0/3219739454 shutdown_connections 2024-05-09T02:24:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.911+0000 7f80cbdf7700 1 -- 172.21.15.19:0/3219739454 wait complete. 2024-05-09T02:24:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.912+0000 7f80cbdf7700 1 Processor -- start 2024-05-09T02:24:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.912+0000 7f80cbdf7700 1 -- start start 2024-05-09T02:24:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.912+0000 7f80cbdf7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80c4105460 0x7f80c419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.912+0000 7f80cbdf7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c4105db0 0x7f80c419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.912+0000 7f80cbdf7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80c410e8d0 0x7f80c419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.913+0000 7f80cbdf7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80c4114480 con 0x7f80c4105db0 2024-05-09T02:24:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.913+0000 7f80cbdf7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f80c4114300 con 0x7f80c4105460 2024-05-09T02:24:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.913+0000 7f80cbdf7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80c4114600 con 0x7f80c410e8d0 2024-05-09T02:24:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.913+0000 7f80c9b93700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80c4105460 0x7f80c419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.913+0000 7f80c9b93700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80c4105460 0x7f80c419ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:39958/0 (socket says 172.21.15.19:39958) 2024-05-09T02:24:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.913+0000 7f80c9b93700 1 -- 172.21.15.19:0/503570119 learned_addr learned my addr 172.21.15.19:0/503570119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.913+0000 7f80c9392700 1 --2- 172.21.15.19:0/503570119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c4105db0 0x7f80c419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.913+0000 7f80ca394700 1 --2- 172.21.15.19:0/503570119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80c410e8d0 0x7f80c419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.913+0000 7f80c9b93700 1 -- 172.21.15.19:0/503570119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80c410e8d0 msgr2=0x7f80c419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.913+0000 7f80c9b93700 1 --2- 172.21.15.19:0/503570119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80c410e8d0 0x7f80c419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.913+0000 7f80c9b93700 1 -- 172.21.15.19:0/503570119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c4105db0 msgr2=0x7f80c419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.913+0000 7f80c9b93700 1 --2- 172.21.15.19:0/503570119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c4105db0 0x7f80c419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.913+0000 7f80c9b93700 1 -- 172.21.15.19:0/503570119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c419fd40 con 0x7f80c4105460 2024-05-09T02:24:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.913+0000 7f80c9b93700 1 --2- 172.21.15.19:0/503570119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80c4105460 0x7f80c419ae70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f80c000c0c0 tx=0x7f80c000c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.914+0000 7f80baffd700 1 -- 172.21.15.19:0/503570119 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c00092e0 con 0x7f80c4105460 2024-05-09T02:24:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.914+0000 7f80cbdf7700 1 -- 172.21.15.19:0/503570119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c419ffd0 con 0x7f80c4105460 2024-05-09T02:24:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.914+0000 7f80cbdf7700 1 -- 172.21.15.19:0/503570119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c41a7950 con 0x7f80c4105460 2024-05-09T02:24:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.914+0000 7f80baffd700 1 -- 172.21.15.19:0/503570119 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c00075c0 con 0x7f80c4105460 2024-05-09T02:24:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.914+0000 7f80baffd700 1 -- 172.21.15.19:0/503570119 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c00146e0 con 0x7f80c4105460 2024-05-09T02:24:55.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.920+0000 7f80baffd700 1 -- 172.21.15.19:0/503570119 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f80c001b020 con 0x7f80c4105460 2024-05-09T02:24:55.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.921+0000 7f80baffd700 1 --2- 172.21.15.19:0/503570119 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f80b00b2aa0 0x7f80b00b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:55.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.922+0000 7f80baffd700 1 -- 172.21.15.19:0/503570119 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f80c00e3280 con 0x7f80c4105460 2024-05-09T02:24:55.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.923+0000 7f80da80db80 1 Processor -- start 2024-05-09T02:24:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.925+0000 7f80da80db80 1 -- start start 2024-05-09T02:24:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.925+0000 7f80da80db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6962d00 0x5617f6a692a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.925+0000 7f80da80db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 0x5617f6a65420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.925+0000 7f80da80db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6a65a20 0x5617f6a7a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.925+0000 7f80da80db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5617f69880a0 con 0x5617f68e84a0 2024-05-09T02:24:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.925+0000 7f80da80db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5617f692d9a0 con 0x5617f6a65a20 2024-05-09T02:24:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.925+0000 7f80da80db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5617f6984060 con 0x5617f6962d00 2024-05-09T02:24:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.926+0000 7f80c9392700 1 --2- 172.21.15.19:0/503570119 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f80b00b2aa0 0x7f80b00b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.930+0000 7f80c9392700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 0x5617f6a65420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.930+0000 7f80c9392700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 0x5617f6a65420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38060/0 (socket says 172.21.15.19:38060) 2024-05-09T02:24:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.930+0000 7f80c9392700 1 -- 172.21.15.19:0/4289183785 learned_addr learned my addr 172.21.15.19:0/4289183785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.931+0000 7f80c9b93700 1 --2- 172.21.15.19:0/4289183785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6962d00 0x5617f6a692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.931+0000 7f80ca394700 1 --2- 172.21.15.19:0/4289183785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6a65a20 0x5617f6a7a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.934+0000 7f80c9b93700 1 -- 172.21.15.19:0/4289183785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6a65a20 msgr2=0x5617f6a7a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.934+0000 7f80c9b93700 1 --2- 172.21.15.19:0/4289183785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6a65a20 0x5617f6a7a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.934+0000 7f80c9b93700 1 -- 172.21.15.19:0/4289183785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 msgr2=0x5617f6a65420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.934+0000 7f80c9b93700 1 --2- 172.21.15.19:0/4289183785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 0x5617f6a65420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.934+0000 7f80c9b93700 1 -- 172.21.15.19:0/4289183785 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617f691e500 con 0x5617f6962d00 2024-05-09T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.934+0000 7f80c9392700 1 --2- 172.21.15.19:0/503570119 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f80b00b2aa0 0x7f80b00b4fb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f80b40128e0 tx=0x7f80b4007af0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.938+0000 7f80c9b93700 1 --2- 172.21.15.19:0/4289183785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6962d00 0x5617f6a692a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f80c00e09a0 tx=0x7f80c00a9d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1827ee116aa1d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.940+0000 7f80affff700 1 -- 172.21.15.19:0/4289183785 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c00aa530 con 0x5617f6962d00 2024-05-09T02:24:55.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.940+0000 7f80affff700 1 -- 172.21.15.19:0/4289183785 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c00ae8e0 con 0x5617f6962d00 2024-05-09T02:24:55.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.940+0000 7f80affff700 1 -- 172.21.15.19:0/4289183785 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c00a5ea0 con 0x5617f6962d00 2024-05-09T02:24:55.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.942+0000 7f80da80db80 1 -- 172.21.15.19:0/4289183785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6962d00 msgr2=0x5617f6a692a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:55.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.942+0000 7f80da80db80 1 --2- 172.21.15.19:0/4289183785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6962d00 0x5617f6a692a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f80c00e09a0 tx=0x7f80c00a9d70 comp rx=0 tx=0).stop 2024-05-09T02:24:55.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.945+0000 7f80da80db80 1 -- 172.21.15.19:0/4289183785 shutdown_connections 2024-05-09T02:24:55.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.945+0000 7f80da80db80 1 --2- 172.21.15.19:0/4289183785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6a65a20 0x5617f6a7a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.945+0000 7f80da80db80 1 --2- 172.21.15.19:0/4289183785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6962d00 0x5617f6a692a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.945+0000 7f80da80db80 1 --2- 172.21.15.19:0/4289183785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 0x5617f6a65420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.945+0000 7f80da80db80 1 -- 172.21.15.19:0/4289183785 >> 172.21.15.19:0/4289183785 conn(0x5617f68ebf70 msgr2=0x5617f6a60220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.947+0000 7f80da80db80 1 -- 172.21.15.19:0/4289183785 shutdown_connections 2024-05-09T02:24:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.950+0000 7f80da80db80 1 -- 172.21.15.19:0/4289183785 wait complete. 2024-05-09T02:24:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.950+0000 7f80da80db80 1 -- start start 2024-05-09T02:24:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.951+0000 7f80da80db80 1 Processor -- start 2024-05-09T02:24:55.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.951+0000 7f80da80db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f68e84a0 0x5617f6a73e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:55.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.951+0000 7f80da80db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6962d00 0x5617f6a743c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:55.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.951+0000 7f80da80db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f6a65a20 0x5617f6a79b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:55.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.951+0000 7f80da80db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5617f6a69810 con 0x5617f6a65a20 2024-05-09T02:24:55.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.951+0000 7f80da80db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5617f6922b20 con 0x5617f6962d00 2024-05-09T02:24:55.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.951+0000 7f80da80db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5617f6a69990 con 0x5617f68e84a0 2024-05-09T02:24:55.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.951+0000 7f80da80db80 10 client.0 _ll_register_callbacks cb 0x5617f67f96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:24:55.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.952+0000 7f80c9b93700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f68e84a0 0x5617f6a73e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.953+0000 7f80c9b93700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f68e84a0 0x5617f6a73e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40764/0 (socket says 172.21.15.19:40764) 2024-05-09T02:24:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.953+0000 7f80c9b93700 1 -- 172.21.15.19:0/1613561797 learned_addr learned my addr 172.21.15.19:0/1613561797 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.953+0000 7f80ca394700 1 --2- 172.21.15.19:0/1613561797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f6a65a20 0x5617f6a79b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.953+0000 7f80c9392700 1 --2- 172.21.15.19:0/1613561797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6962d00 0x5617f6a743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.954+0000 7f80c9b93700 1 -- 172.21.15.19:0/1613561797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6962d00 msgr2=0x5617f6a743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.954+0000 7f80c9b93700 1 --2- 172.21.15.19:0/1613561797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6962d00 0x5617f6a743c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.954+0000 7f80c9b93700 1 -- 172.21.15.19:0/1613561797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f6a65a20 msgr2=0x5617f6a79b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.954+0000 7f80c9b93700 1 --2- 172.21.15.19:0/1613561797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f6a65a20 0x5617f6a79b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.954+0000 7f80c9b93700 1 -- 172.21.15.19:0/1613561797 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617f6803490 con 0x5617f68e84a0 2024-05-09T02:24:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.956+0000 7f80c9b93700 1 --2- 172.21.15.19:0/1613561797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f68e84a0 0x5617f6a73e20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f80c00e0a00 tx=0x7f80c0005ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b38c6929406967dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.958+0000 7f80ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:24:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.958+0000 7f80ad7fa700 1 -- 172.21.15.19:0/1613561797 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c00c3070 con 0x5617f68e84a0 2024-05-09T02:24:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.958+0000 7f80ad7fa700 1 -- 172.21.15.19:0/1613561797 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f80c00a99a0 con 0x5617f68e84a0 2024-05-09T02:24:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.960+0000 7f80da80db80 1 -- 172.21.15.19:0/1613561797 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80c00ae8e0 con 0x5617f68e84a0 2024-05-09T02:24:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.961+0000 7f80ad7fa700 1 -- 172.21.15.19:0/1613561797 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f80c00aa530 con 0x5617f68e84a0 2024-05-09T02:24:55.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.962+0000 7f80da80db80 10 client.26767 fetch_fsmap learned FSMap version 58 2024-05-09T02:24:55.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.962+0000 7f80da80db80 1 -- 172.21.15.19:0/1613561797 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5617f6a69810 con 0x5617f68e84a0 2024-05-09T02:24:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.963+0000 7f80ad7fa700 1 -- 172.21.15.19:0/1613561797 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c00ae8e0 con 0x5617f68e84a0 2024-05-09T02:24:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.964+0000 7f80ad7fa700 1 -- 172.21.15.19:0/1613561797 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f80c00afa70 con 0x5617f68e84a0 2024-05-09T02:24:55.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.964+0000 7f80da80db80 10 client.26767 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:24:55.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.964+0000 7f80da80db80 10 client.26767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:24:55.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.964+0000 7f80da80db80 20 client.26767 populate_metadata read hostname 'smithi019' 2024-05-09T02:24:55.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.964+0000 7f80da80db80 1 --2- 172.21.15.19:0/1613561797 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5617f6a8fdc0 0x5617f6a921e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:55.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.964+0000 7f80da80db80 4 client.26767 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:24:55.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.964+0000 7f80da80db80 1 -- 172.21.15.19:0/1613561797 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617f6995a80 con 0x5617f6a8fdc0 2024-05-09T02:24:55.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.965+0000 7f80c9392700 1 --2- 172.21.15.19:0/1613561797 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5617f6a8fdc0 0x5617f6a921e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:55.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.966+0000 7f80c9392700 1 --2- 172.21.15.19:0/1613561797 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5617f6a8fdc0 0x5617f6a921e0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe34248629fa830 server_cookie=8a588f78cf067bc8 in_seq=0 out_seq=0 2024-05-09T02:24:55.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.966+0000 7f80ad7fa700 10 client.26767 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:24:55.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.967+0000 7f80ad7fa700 1 -- 172.21.15.19:0/1613561797 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5617f6995a80 con 0x5617f6a8fdc0 2024-05-09T02:24:55.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.967+0000 7f80ad7fa700 10 client.26767 handle_command_reply: tid=0 2024-05-09T02:24:55.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.967+0000 7f80da80db80 1 client.26767 shutdown 2024-05-09T02:24:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.967+0000 7f80da80db80 1 -- 172.21.15.19:0/1613561797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f68e84a0 msgr2=0x5617f6a73e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.967+0000 7f80da80db80 1 --2- 172.21.15.19:0/1613561797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f68e84a0 0x5617f6a73e20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f80c00e0a00 tx=0x7f80c0005ae0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.968+0000 7f80da80db80 1 -- 172.21.15.19:0/1613561797 shutdown_connections 2024-05-09T02:24:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.968+0000 7f80da80db80 1 --2- 172.21.15.19:0/1613561797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f6a65a20 0x5617f6a79b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.968+0000 7f80da80db80 1 --2- 172.21.15.19:0/1613561797 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5617f6a8fdc0 0x5617f6a921e0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.968+0000 7f80da80db80 1 --2- 172.21.15.19:0/1613561797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f68e84a0 0x5617f6a73e20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.968+0000 7f80da80db80 1 --2- 172.21.15.19:0/1613561797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6962d00 0x5617f6a743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.968+0000 7f80da80db80 1 -- 172.21.15.19:0/1613561797 >> 172.21.15.19:0/1613561797 conn(0x5617f68ebf70 msgr2=0x5617f6a623e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:55.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.968+0000 7f80ad7fa700 0 client.26767 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:24:55.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.968+0000 7f80da80db80 1 -- 172.21.15.19:0/1613561797 shutdown_connections 2024-05-09T02:24:55.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.968+0000 7f80da80db80 1 -- 172.21.15.19:0/1613561797 wait complete. 2024-05-09T02:24:55.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.968+0000 7f80da80db80 20 client.26767 trim_cache size 0 max 16384 2024-05-09T02:24:55.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.992+0000 7f80da80db80 1 Processor -- start 2024-05-09T02:24:55.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.992+0000 7f80da80db80 1 -- start start 2024-05-09T02:24:55.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.993+0000 7f80da80db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 0x5617f6a65400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:55.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.993+0000 7f80da80db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6962d00 0x5617f6a94df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.993+0000 7f80da80db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6a65a20 0x5617f6a95330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.993+0000 7f80da80db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80a800d6c0 con 0x5617f68e84a0 2024-05-09T02:24:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.993+0000 7f80da80db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f80a800f200 con 0x5617f6962d00 2024-05-09T02:24:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.993+0000 7f80da80db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80a800dd90 con 0x5617f6a65a20 2024-05-09T02:24:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.994+0000 7f80c9392700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6962d00 0x5617f6a94df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.994+0000 7f80ca394700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6a65a20 0x5617f6a95330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.994+0000 7f80ca394700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6a65a20 0x5617f6a95330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40772/0 (socket says 172.21.15.19:40772) 2024-05-09T02:24:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.994+0000 7f80ca394700 1 -- 172.21.15.19:0/2035231626 learned_addr learned my addr 172.21.15.19:0/2035231626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.994+0000 7f80c9b93700 1 --2- 172.21.15.19:0/2035231626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 0x5617f6a65400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.995+0000 7f80ca394700 1 -- 172.21.15.19:0/2035231626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6962d00 msgr2=0x5617f6a94df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.995+0000 7f80ca394700 1 --2- 172.21.15.19:0/2035231626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6962d00 0x5617f6a94df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.995+0000 7f80ca394700 1 -- 172.21.15.19:0/2035231626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 msgr2=0x5617f6a65400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.995+0000 7f80ca394700 1 --2- 172.21.15.19:0/2035231626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 0x5617f6a65400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.995+0000 7f80ca394700 1 -- 172.21.15.19:0/2035231626 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617f6a5c9d0 con 0x5617f6a65a20 2024-05-09T02:24:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.995+0000 7f80ca394700 1 --2- 172.21.15.19:0/2035231626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6a65a20 0x5617f6a95330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009230 tx=0x7f80bc00fc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3a49ae0a8d86bda server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.995+0000 7f80af7fe700 1 -- 172.21.15.19:0/2035231626 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00f660 con 0x5617f6a65a20 2024-05-09T02:24:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.996+0000 7f80af7fe700 1 -- 172.21.15.19:0/2035231626 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80bc00f800 con 0x5617f6a65a20 2024-05-09T02:24:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.996+0000 7f80af7fe700 1 -- 172.21.15.19:0/2035231626 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc003970 con 0x5617f6a65a20 2024-05-09T02:24:56.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.997+0000 7f80da80db80 1 -- 172.21.15.19:0/2035231626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6a65a20 msgr2=0x5617f6a95330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:56.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.997+0000 7f80da80db80 1 --2- 172.21.15.19:0/2035231626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6a65a20 0x5617f6a95330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009230 tx=0x7f80bc00fc70 comp rx=0 tx=0).stop 2024-05-09T02:24:56.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.997+0000 7f80da80db80 1 -- 172.21.15.19:0/2035231626 shutdown_connections 2024-05-09T02:24:56.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.997+0000 7f80da80db80 1 --2- 172.21.15.19:0/2035231626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6a65a20 0x5617f6a95330 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:56.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.997+0000 7f80da80db80 1 --2- 172.21.15.19:0/2035231626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 0x5617f6a65400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.997+0000 7f80da80db80 1 --2- 172.21.15.19:0/2035231626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6962d00 0x5617f6a94df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.997+0000 7f80da80db80 1 -- 172.21.15.19:0/2035231626 >> 172.21.15.19:0/2035231626 conn(0x5617f68ebf70 msgr2=0x5617f6a6db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.997+0000 7f80da80db80 1 -- 172.21.15.19:0/2035231626 shutdown_connections 2024-05-09T02:24:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.997+0000 7f80da80db80 1 -- 172.21.15.19:0/2035231626 wait complete. 2024-05-09T02:24:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.998+0000 7f80da80db80 1 -- start start 2024-05-09T02:24:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.998+0000 7f80da80db80 1 Processor -- start 2024-05-09T02:24:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.998+0000 7f80da80db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 0x5617f6a92260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.998+0000 7f80da80db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6962d00 0x5617f6a927a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80c9b93700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 0x5617f6a92260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80da80db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6a65a20 0x5617f6af22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80c9b93700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 0x5617f6a92260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38096/0 (socket says 172.21.15.19:38096) 2024-05-09T02:24:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80da80db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80a800eb30 con 0x5617f68e84a0 2024-05-09T02:24:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80c9b93700 1 -- 172.21.15.19:0/4234020502 learned_addr learned my addr 172.21.15.19:0/4234020502 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:24:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80da80db80 1 -- 172.21.15.19:0/4234020502 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f80a800e460 con 0x5617f6a65a20 2024-05-09T02:24:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80da80db80 1 -- 172.21.15.19:0/4234020502 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80a800b680 con 0x5617f6962d00 2024-05-09T02:24:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80da80db80 10 client.0 _ll_register_callbacks cb 0x5617f67f96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:24:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80ca394700 1 --2- 172.21.15.19:0/4234020502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6a65a20 0x5617f6af22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80c9392700 1 --2- 172.21.15.19:0/4234020502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6962d00 0x5617f6a927a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80c9b93700 1 -- 172.21.15.19:0/4234020502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6962d00 msgr2=0x5617f6a927a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80c9b93700 1 --2- 172.21.15.19:0/4234020502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6962d00 0x5617f6a927a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80c9b93700 1 -- 172.21.15.19:0/4234020502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6a65a20 msgr2=0x5617f6af22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:24:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80c9b93700 1 --2- 172.21.15.19:0/4234020502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6a65a20 0x5617f6af22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80c9b93700 1 -- 172.21.15.19:0/4234020502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617f6af2840 con 0x5617f68e84a0 2024-05-09T02:24:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80ca394700 1 --2- 172.21.15.19:0/4234020502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6a65a20 0x5617f6af22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:24:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:55.999+0000 7f80c9392700 1 --2- 172.21.15.19:0/4234020502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6962d00 0x5617f6a927a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:24:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.000+0000 7f80c9b93700 1 --2- 172.21.15.19:0/4234020502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 0x5617f6a92260 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f80c0021000 tx=0x7f80c0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa43f0151babc4a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:24:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.000+0000 7f80ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:24:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.000+0000 7f80ad7fa700 1 -- 172.21.15.19:0/4234020502 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c00c3070 con 0x5617f68e84a0 2024-05-09T02:24:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.000+0000 7f80ad7fa700 1 -- 172.21.15.19:0/4234020502 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f80c00a9200 con 0x5617f68e84a0 2024-05-09T02:24:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.000+0000 7f80da80db80 1 -- 172.21.15.19:0/4234020502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80bc00f800 con 0x5617f68e84a0 2024-05-09T02:24:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.000+0000 7f80ad7fa700 1 -- 172.21.15.19:0/4234020502 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f80c00ae8e0 con 0x5617f68e84a0 2024-05-09T02:24:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.000+0000 7f80da80db80 10 client.17007 fetch_fsmap learned FSMap version 58 2024-05-09T02:24:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.000+0000 7f80da80db80 1 -- 172.21.15.19:0/4234020502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5617f6af9b40 con 0x5617f68e84a0 2024-05-09T02:24:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.000+0000 7f80ad7fa700 1 -- 172.21.15.19:0/4234020502 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80bc00f800 con 0x5617f68e84a0 2024-05-09T02:24:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.001+0000 7f80ad7fa700 1 -- 172.21.15.19:0/4234020502 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f80c000c9d0 con 0x5617f68e84a0 2024-05-09T02:24:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.001+0000 7f80da80db80 10 client.17007 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:24:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.001+0000 7f80da80db80 10 client.17007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:24:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.001+0000 7f80da80db80 20 client.17007 populate_metadata read hostname 'smithi019' 2024-05-09T02:24:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.001+0000 7f80da80db80 1 --2- 172.21.15.19:0/4234020502 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5617f6af9f50 0x5617f6afc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:24:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.001+0000 7f80da80db80 4 client.17007 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:24:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.001+0000 7f80da80db80 1 -- 172.21.15.19:0/4234020502 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5617f698fb20 con 0x5617f6af9f50 2024-05-09T02:24:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.001+0000 7f80c9392700 1 --2- 172.21.15.19:0/4234020502 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5617f6af9f50 0x5617f6afc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:24:56.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.002+0000 7f80c9392700 1 --2- 172.21.15.19:0/4234020502 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5617f6af9f50 0x5617f6afc370 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76301446c447f300 server_cookie=7fa5e124c82acee0 in_seq=0 out_seq=0 2024-05-09T02:24:56.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.002+0000 7f80ad7fa700 10 client.17007 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:24:56.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.002+0000 7f80ad7fa700 1 -- 172.21.15.19:0/4234020502 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617f698fb20 con 0x5617f6af9f50 2024-05-09T02:24:56.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.002+0000 7f80ad7fa700 10 client.17007 handle_command_reply: tid=0 2024-05-09T02:24:56.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.003+0000 7f80da80db80 1 client.17007 shutdown 2024-05-09T02:24:56.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.003+0000 7f80da80db80 1 -- 172.21.15.19:0/4234020502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 msgr2=0x5617f6a92260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:56.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.003+0000 7f80da80db80 1 --2- 172.21.15.19:0/4234020502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 0x5617f6a92260 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f80c0021000 tx=0x7f80c0005b50 comp rx=0 tx=0).stop 2024-05-09T02:24:56.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.003+0000 7f80da80db80 1 -- 172.21.15.19:0/4234020502 shutdown_connections 2024-05-09T02:24:56.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.003+0000 7f80da80db80 1 --2- 172.21.15.19:0/4234020502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617f6a65a20 0x5617f6af22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:56.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.003+0000 7f80da80db80 1 --2- 172.21.15.19:0/4234020502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617f68e84a0 0x5617f6a92260 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:56.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.003+0000 7f80da80db80 1 --2- 172.21.15.19:0/4234020502 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5617f6af9f50 0x5617f6afc370 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:56.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.003+0000 7f80ad7fa700 0 client.17007 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:24:56.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.003+0000 7f80da80db80 1 --2- 172.21.15.19:0/4234020502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617f6962d00 0x5617f6a927a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:56.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.003+0000 7f80da80db80 1 -- 172.21.15.19:0/4234020502 >> 172.21.15.19:0/4234020502 conn(0x5617f68ebf70 msgr2=0x5617f6a78b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:56.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.004+0000 7f80da80db80 1 -- 172.21.15.19:0/4234020502 shutdown_connections 2024-05-09T02:24:56.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.004+0000 7f80da80db80 1 -- 172.21.15.19:0/4234020502 wait complete. 2024-05-09T02:24:56.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.004+0000 7f80da80db80 20 client.17007 trim_cache size 0 max 16384 2024-05-09T02:24:56.010 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:24:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.004+0000 7f80cbdf7700 1 -- 172.21.15.19:0/503570119 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f80b00b2aa0 msgr2=0x7f80b00b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.004+0000 7f80cbdf7700 1 --2- 172.21.15.19:0/503570119 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f80b00b2aa0 0x7f80b00b4fb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f80b40128e0 tx=0x7f80b4007af0 comp rx=0 tx=0).stop 2024-05-09T02:24:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.004+0000 7f80cbdf7700 1 -- 172.21.15.19:0/503570119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80c4105460 msgr2=0x7f80c419ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:24:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.004+0000 7f80cbdf7700 1 --2- 172.21.15.19:0/503570119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80c4105460 0x7f80c419ae70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f80c000c0c0 tx=0x7f80c000c3d0 comp rx=0 tx=0).stop 2024-05-09T02:24:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.005+0000 7f80cbdf7700 1 -- 172.21.15.19:0/503570119 shutdown_connections 2024-05-09T02:24:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.005+0000 7f80cbdf7700 1 --2- 172.21.15.19:0/503570119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80c410e8d0 0x7f80c419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.005+0000 7f80cbdf7700 1 --2- 172.21.15.19:0/503570119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80c4105460 0x7f80c419ae70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.005+0000 7f80cbdf7700 1 --2- 172.21.15.19:0/503570119 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f80b00b2aa0 0x7f80b00b4fb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.005+0000 7f80cbdf7700 1 --2- 172.21.15.19:0/503570119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80c4105db0 0x7f80c419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:24:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.005+0000 7f80cbdf7700 1 -- 172.21.15.19:0/503570119 >> 172.21.15.19:0/503570119 conn(0x7f80c40fedc0 msgr2=0x7f80c40ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:24:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.005+0000 7f80cbdf7700 1 -- 172.21.15.19:0/503570119 shutdown_connections 2024-05-09T02:24:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:24:56.005+0000 7f80cbdf7700 1 -- 172.21.15.19:0/503570119 wait complete. 2024-05-09T02:24:56.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:56 smithi019 ceph-mon[30151]: pgmap v426: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 1.29k op/s 2024-05-09T02:24:56.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:56 smithi059 ceph-mon[34781]: pgmap v426: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 1.29k op/s 2024-05-09T02:24:57.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:56 smithi196 ceph-mon[34718]: pgmap v426: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 1.29k op/s 2024-05-09T02:24:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 795.0 s, 1369.7 tps, lat 73.038 ms stddev 80.477 2024-05-09T02:24:57.786 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T02:24:57.787 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T02:24:58.162 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:24:58.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:24:58 smithi019 ceph-mon[30151]: pgmap v427: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.7 MiB/s wr, 657 op/s 2024-05-09T02:24:58.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:24:58 smithi059 ceph-mon[34781]: pgmap v427: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.7 MiB/s wr, 657 op/s 2024-05-09T02:24:59.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:24:58 smithi196 ceph-mon[34718]: pgmap v427: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.7 MiB/s wr, 657 op/s 2024-05-09T02:25:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.795+0000 7f05ef59e700 1 -- 172.21.15.19:0/2415175517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f01107e0 msgr2=0x7f05f0110bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.795+0000 7f05ef59e700 1 --2- 172.21.15.19:0/2415175517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f01107e0 0x7f05f0110bf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009990 tx=0x7f05e0009ca0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.796+0000 7f05ef59e700 1 -- 172.21.15.19:0/2415175517 shutdown_connections 2024-05-09T02:25:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.796+0000 7f05ef59e700 1 --2- 172.21.15.19:0/2415175517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05f010b150 0x7f05f0117680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.796+0000 7f05ef59e700 1 --2- 172.21.15.19:0/2415175517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f01107e0 0x7f05f0110bf0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.796+0000 7f05ef59e700 1 --2- 172.21.15.19:0/2415175517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05f010a800 0x7f05f010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.796+0000 7f05ef59e700 1 -- 172.21.15.19:0/2415175517 >> 172.21.15.19:0/2415175517 conn(0x7f05f0076b00 msgr2=0x7f05f0076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.796+0000 7f05ef59e700 1 -- 172.21.15.19:0/2415175517 shutdown_connections 2024-05-09T02:25:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.796+0000 7f05ef59e700 1 -- 172.21.15.19:0/2415175517 wait complete. 2024-05-09T02:25:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.797+0000 7f05ef59e700 1 Processor -- start 2024-05-09T02:25:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.798+0000 7f05ef59e700 1 -- start start 2024-05-09T02:25:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.798+0000 7f05ef59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05f010a800 0x7f05f01a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.798+0000 7f05ef59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f010b150 0x7f05f01a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.798+0000 7f05ef59e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05f01107e0 0x7f05f01a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.798+0000 7f05ef59e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05f011a970 con 0x7f05f010b150 2024-05-09T02:25:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.798+0000 7f05edd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f010b150 0x7f05f01a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.799+0000 7f05ee59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05f010a800 0x7f05f01a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.799+0000 7f05edd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f010b150 0x7f05f01a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40740/0 (socket says 172.21.15.19:40740) 2024-05-09T02:25:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.799+0000 7f05edd9b700 1 -- 172.21.15.19:0/704861861 learned_addr learned my addr 172.21.15.19:0/704861861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.799+0000 7f05ef59e700 1 -- 172.21.15.19:0/704861861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f05f011a7f0 con 0x7f05f010a800 2024-05-09T02:25:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.799+0000 7f05eed9d700 1 --2- 172.21.15.19:0/704861861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05f01107e0 0x7f05f01a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.799+0000 7f05ef59e700 1 -- 172.21.15.19:0/704861861 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05f011aaf0 con 0x7f05f01107e0 2024-05-09T02:25:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.799+0000 7f05edd9b700 1 -- 172.21.15.19:0/704861861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05f01107e0 msgr2=0x7f05f01a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.799+0000 7f05edd9b700 1 --2- 172.21.15.19:0/704861861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05f01107e0 0x7f05f01a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.799+0000 7f05edd9b700 1 -- 172.21.15.19:0/704861861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05f010a800 msgr2=0x7f05f01a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.799+0000 7f05edd9b700 1 --2- 172.21.15.19:0/704861861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05f010a800 0x7f05f01a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.799+0000 7f05edd9b700 1 -- 172.21.15.19:0/704861861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f01a6170 con 0x7f05f010b150 2024-05-09T02:25:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.799+0000 7f05ee59c700 1 --2- 172.21.15.19:0/704861861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05f010a800 0x7f05f01a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:25:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.799+0000 7f05edd9b700 1 --2- 172.21.15.19:0/704861861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f010b150 0x7f05f01a1840 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f05e400d980 tx=0x7f05e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.800+0000 7f05df7fe700 1 -- 172.21.15.19:0/704861861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4017070 con 0x7f05f010b150 2024-05-09T02:25:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.800+0000 7f05df7fe700 1 -- 172.21.15.19:0/704861861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e400fd40 con 0x7f05f010b150 2024-05-09T02:25:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.800+0000 7f05ef59e700 1 -- 172.21.15.19:0/704861861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f01a6460 con 0x7f05f010b150 2024-05-09T02:25:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.800+0000 7f05df7fe700 1 -- 172.21.15.19:0/704861861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e400e3b0 con 0x7f05f010b150 2024-05-09T02:25:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.800+0000 7f05ef59e700 1 -- 172.21.15.19:0/704861861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f01add80 con 0x7f05f010b150 2024-05-09T02:25:00.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.805+0000 7f06039d0b80 1 Processor -- start 2024-05-09T02:25:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f06039d0b80 1 -- start start 2024-05-09T02:25:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f05df7fe700 1 -- 172.21.15.19:0/704861861 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f05e4010610 con 0x7f05f010b150 2024-05-09T02:25:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f06039d0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5804ddd80 0x55d5805e6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f06039d0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d580463520 0x55d5805e6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f06039d0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5805e0660 0x55d5805e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f06039d0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d580503120 con 0x55d5805e0660 2024-05-09T02:25:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f06039d0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d5804a8a20 con 0x55d5804ddd80 2024-05-09T02:25:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f06039d0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d5804ff0e0 con 0x55d580463520 2024-05-09T02:25:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f05ee59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d580463520 0x55d5805e6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f05ee59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d580463520 0x55d5805e6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40208/0 (socket says 172.21.15.19:40208) 2024-05-09T02:25:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f05ee59c700 1 -- 172.21.15.19:0/1758920647 learned_addr learned my addr 172.21.15.19:0/1758920647 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f05eed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5804ddd80 0x55d5805e6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f05edd9b700 1 --2- 172.21.15.19:0/1758920647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5805e0660 0x55d5805e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f05ee59c700 1 -- 172.21.15.19:0/1758920647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5804ddd80 msgr2=0x55d5805e6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f05ee59c700 1 --2- 172.21.15.19:0/1758920647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5804ddd80 0x55d5805e6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.806+0000 7f05ee59c700 1 -- 172.21.15.19:0/1758920647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5805e0660 msgr2=0x55d5805e0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.807+0000 7f05ee59c700 1 --2- 172.21.15.19:0/1758920647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5805e0660 0x55d5805e0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.807+0000 7f05ee59c700 1 -- 172.21.15.19:0/1758920647 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d580499580 con 0x55d580463520 2024-05-09T02:25:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.807+0000 7f05ee59c700 1 --2- 172.21.15.19:0/1758920647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d580463520 0x55d5805e6a50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f05e0015000 tx=0x7f05e0009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5e5a01eb5533fcc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.807+0000 7f05df7fe700 1 --2- 172.21.15.19:0/704861861 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f05d80b29f0 0x7f05d80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.807+0000 7f05dcff9700 1 -- 172.21.15.19:0/1758920647 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0009580 con 0x55d580463520 2024-05-09T02:25:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.807+0000 7f05eed9d700 1 --2- 172.21.15.19:0/704861861 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f05d80b29f0 0x7f05d80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.807+0000 7f05dcff9700 1 -- 172.21.15.19:0/1758920647 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e0004ce0 con 0x55d580463520 2024-05-09T02:25:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.807+0000 7f05dcff9700 1 -- 172.21.15.19:0/1758920647 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e000f460 con 0x55d580463520 2024-05-09T02:25:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.807+0000 7f05df7fe700 1 -- 172.21.15.19:0/704861861 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f05e40e20c0 con 0x7f05f010b150 2024-05-09T02:25:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.809+0000 7f06039d0b80 1 -- 172.21.15.19:0/1758920647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d580463520 msgr2=0x55d5805e6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.809+0000 7f06039d0b80 1 --2- 172.21.15.19:0/1758920647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d580463520 0x55d5805e6a50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f05e0015000 tx=0x7f05e0009510 comp rx=0 tx=0).stop 2024-05-09T02:25:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.809+0000 7f05eed9d700 1 --2- 172.21.15.19:0/704861861 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f05d80b29f0 0x7f05d80b4f00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55d5805e1850 tx=0x7f05e800b040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.809+0000 7f06039d0b80 1 -- 172.21.15.19:0/1758920647 shutdown_connections 2024-05-09T02:25:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.809+0000 7f06039d0b80 1 --2- 172.21.15.19:0/1758920647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5805e0660 0x55d5805e0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.809+0000 7f06039d0b80 1 --2- 172.21.15.19:0/1758920647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5804ddd80 0x55d5805e6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.809+0000 7f06039d0b80 1 --2- 172.21.15.19:0/1758920647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d580463520 0x55d5805e6a50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.809+0000 7f06039d0b80 1 -- 172.21.15.19:0/1758920647 >> 172.21.15.19:0/1758920647 conn(0x55d580466ff0 msgr2=0x55d5805db860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.809+0000 7f06039d0b80 1 -- 172.21.15.19:0/1758920647 shutdown_connections 2024-05-09T02:25:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.809+0000 7f06039d0b80 1 -- 172.21.15.19:0/1758920647 wait complete. 2024-05-09T02:25:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.810+0000 7f06039d0b80 1 -- start start 2024-05-09T02:25:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.810+0000 7f06039d0b80 1 Processor -- start 2024-05-09T02:25:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.810+0000 7f06039d0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 0x55d5805ea190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.810+0000 7f06039d0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5804ddd80 0x55d5805ea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.810+0000 7f06039d0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5805e0660 0x55d5805eac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.810+0000 7f06039d0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d5805e13b0 con 0x55d5804ddd80 2024-05-09T02:25:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.810+0000 7f06039d0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d58049dba0 con 0x55d580463520 2024-05-09T02:25:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.810+0000 7f06039d0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d5805ed280 con 0x55d5805e0660 2024-05-09T02:25:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f05edd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5804ddd80 0x55d5805ea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f05edd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5804ddd80 0x55d5805ea6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40764/0 (socket says 172.21.15.19:40764) 2024-05-09T02:25:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f05edd9b700 1 -- 172.21.15.19:0/429592573 learned_addr learned my addr 172.21.15.19:0/429592573 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f05eed9d700 1 --2- 172.21.15.19:0/429592573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5805e0660 0x55d5805eac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f05ee59c700 1 --2- 172.21.15.19:0/429592573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 0x55d5805ea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f06039d0b80 10 client.0 _ll_register_callbacks cb 0x55d580374740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:25:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f05ee59c700 1 -- 172.21.15.19:0/429592573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5805e0660 msgr2=0x55d5805eac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f05ee59c700 1 --2- 172.21.15.19:0/429592573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5805e0660 0x55d5805eac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f05ee59c700 1 -- 172.21.15.19:0/429592573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5804ddd80 msgr2=0x55d5805ea6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f05ee59c700 1 --2- 172.21.15.19:0/429592573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5804ddd80 0x55d5805ea6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f05ee59c700 1 -- 172.21.15.19:0/429592573 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d58037e510 con 0x55d580463520 2024-05-09T02:25:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f05eed9d700 1 --2- 172.21.15.19:0/429592573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5805e0660 0x55d5805eac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:25:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f06039d0b80 1 -- 172.21.15.19:0/429592573 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05e0004ce0 con 0x55d580463520 2024-05-09T02:25:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f05ee59c700 1 --2- 172.21.15.19:0/429592573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 0x55d5805ea190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f05e0000c00 tx=0x7f05e0004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3ed2ac25f72a979 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f05d5ffb700 10 client.26729 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:25:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f05d5ffb700 1 -- 172.21.15.19:0/429592573 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e005a050 con 0x55d580463520 2024-05-09T02:25:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.811+0000 7f05d5ffb700 1 -- 172.21.15.19:0/429592573 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f05e0009580 con 0x55d580463520 2024-05-09T02:25:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.812+0000 7f05d5ffb700 1 -- 172.21.15.19:0/429592573 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f05e0050070 con 0x55d580463520 2024-05-09T02:25:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.812+0000 7f06039d0b80 10 client.26729 fetch_fsmap learned FSMap version 58 2024-05-09T02:25:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.812+0000 7f06039d0b80 1 -- 172.21.15.19:0/429592573 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=58}) -- 0x55d5805f7810 con 0x55d580463520 2024-05-09T02:25:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.812+0000 7f05d5ffb700 1 -- 172.21.15.19:0/429592573 <== mon.2 v2:172.21.15.59:3300/0 4 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f05e000f460 con 0x55d580463520 2024-05-09T02:25:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.812+0000 7f06039d0b80 10 client.26729 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:25:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.812+0000 7f06039d0b80 10 client.26729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:25:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.812+0000 7f06039d0b80 20 client.26729 populate_metadata read hostname 'smithi019' 2024-05-09T02:25:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.812+0000 7f06039d0b80 1 --2- 172.21.15.19:0/429592573 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d5805f9ce0 0x55d5805fc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.812+0000 7f06039d0b80 4 client.26729 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:25:00.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.812+0000 7f06039d0b80 1 -- 172.21.15.19:0/429592573 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d580510b00 con 0x55d5805f9ce0 2024-05-09T02:25:00.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.813+0000 7f05edd9b700 1 --2- 172.21.15.19:0/429592573 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d5805f9ce0 0x55d5805fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.813+0000 7f05edd9b700 1 --2- 172.21.15.19:0/429592573 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d5805f9ce0 0x55d5805fc100 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e0842a148a1a7f server_cookie=40c0c6ab5d643772 in_seq=0 out_seq=0 2024-05-09T02:25:00.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.813+0000 7f05d5ffb700 10 client.26729 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:25:00.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.814+0000 7f05d5ffb700 1 -- 172.21.15.19:0/429592573 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d580510b00 con 0x55d5805f9ce0 2024-05-09T02:25:00.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.814+0000 7f05d5ffb700 10 client.26729 handle_command_reply: tid=0 2024-05-09T02:25:00.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.814+0000 7f06039d0b80 1 client.26729 shutdown 2024-05-09T02:25:00.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.814+0000 7f06039d0b80 1 -- 172.21.15.19:0/429592573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 msgr2=0x55d5805ea190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:00.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.814+0000 7f06039d0b80 1 --2- 172.21.15.19:0/429592573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 0x55d5805ea190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f05e0000c00 tx=0x7f05e0004a60 comp rx=0 tx=0).stop 2024-05-09T02:25:00.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.814+0000 7f06039d0b80 1 -- 172.21.15.19:0/429592573 shutdown_connections 2024-05-09T02:25:00.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.814+0000 7f06039d0b80 1 --2- 172.21.15.19:0/429592573 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d5805f9ce0 0x55d5805fc100 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.814+0000 7f05d5ffb700 0 client.26729 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:25:00.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.814+0000 7f06039d0b80 1 --2- 172.21.15.19:0/429592573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5805e0660 0x55d5805eac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.814+0000 7f06039d0b80 1 --2- 172.21.15.19:0/429592573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 0x55d5805ea190 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.814+0000 7f06039d0b80 1 --2- 172.21.15.19:0/429592573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5804ddd80 0x55d5805ea6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.814+0000 7f06039d0b80 1 -- 172.21.15.19:0/429592573 >> 172.21.15.19:0/429592573 conn(0x55d580466ff0 msgr2=0x55d5805db350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.815+0000 7f06039d0b80 1 -- 172.21.15.19:0/429592573 shutdown_connections 2024-05-09T02:25:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.815+0000 7f06039d0b80 1 -- 172.21.15.19:0/429592573 wait complete. 2024-05-09T02:25:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.815+0000 7f06039d0b80 20 client.26729 trim_cache size 0 max 16384 2024-05-09T02:25:00.840 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:00 smithi019 ceph-mon[30151]: pgmap v428: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.0 MiB/s wr, 596 op/s 2024-05-09T02:25:00.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.837+0000 7f06039d0b80 1 Processor -- start 2024-05-09T02:25:00.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.838+0000 7f06039d0b80 1 -- start start 2024-05-09T02:25:00.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.838+0000 7f06039d0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 0x55d5805e4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.838+0000 7f06039d0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5804ddd80 0x55d5805eaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.838+0000 7f06039d0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5805e0660 0x55d5805f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.838+0000 7f06039d0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05f00745f0 con 0x55d5804ddd80 2024-05-09T02:25:00.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.838+0000 7f06039d0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01ae6a0 con 0x55d580463520 2024-05-09T02:25:00.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.838+0000 7f06039d0b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05f0074cc0 con 0x55d5805e0660 2024-05-09T02:25:00.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.838+0000 7f05edd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5804ddd80 0x55d5805eaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.838+0000 7f05edd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5804ddd80 0x55d5805eaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40768/0 (socket says 172.21.15.19:40768) 2024-05-09T02:25:00.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.838+0000 7f05edd9b700 1 -- 172.21.15.19:0/2721811761 learned_addr learned my addr 172.21.15.19:0/2721811761 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:00.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.838+0000 7f05ee59c700 1 --2- 172.21.15.19:0/2721811761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 0x55d5805e4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.838+0000 7f05eed9d700 1 --2- 172.21.15.19:0/2721811761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5805e0660 0x55d5805f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.839+0000 7f05edd9b700 1 -- 172.21.15.19:0/2721811761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5805e0660 msgr2=0x55d5805f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:00.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.839+0000 7f05edd9b700 1 --2- 172.21.15.19:0/2721811761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5805e0660 0x55d5805f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.839+0000 7f05edd9b700 1 -- 172.21.15.19:0/2721811761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 msgr2=0x55d5805e4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:00.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.839+0000 7f05edd9b700 1 --2- 172.21.15.19:0/2721811761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 0x55d5805e4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.839+0000 7f05edd9b700 1 -- 172.21.15.19:0/2721811761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5805d7590 con 0x55d5804ddd80 2024-05-09T02:25:00.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.839+0000 7f05ee59c700 1 --2- 172.21.15.19:0/2721811761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 0x55d5805e4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:25:00.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.839+0000 7f05edd9b700 1 --2- 172.21.15.19:0/2721811761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5804ddd80 0x55d5805eaa50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f05e40a9300 tx=0x7f05e40109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a023175a3f1a7f77 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:00.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.839+0000 7f05d7fff700 1 -- 172.21.15.19:0/2721811761 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4020040 con 0x55d5804ddd80 2024-05-09T02:25:00.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.839+0000 7f05d7fff700 1 -- 172.21.15.19:0/2721811761 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e40a8e80 con 0x55d5804ddd80 2024-05-09T02:25:00.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.839+0000 7f05d7fff700 1 -- 172.21.15.19:0/2721811761 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e40a97d0 con 0x55d5804ddd80 2024-05-09T02:25:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.840+0000 7f06039d0b80 1 -- 172.21.15.19:0/2721811761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5804ddd80 msgr2=0x55d5805eaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.840+0000 7f06039d0b80 1 --2- 172.21.15.19:0/2721811761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5804ddd80 0x55d5805eaa50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f05e40a9300 tx=0x7f05e40109b0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.840+0000 7f06039d0b80 1 -- 172.21.15.19:0/2721811761 shutdown_connections 2024-05-09T02:25:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.840+0000 7f06039d0b80 1 --2- 172.21.15.19:0/2721811761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5805e0660 0x55d5805f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.840+0000 7f06039d0b80 1 --2- 172.21.15.19:0/2721811761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 0x55d5805e4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.840+0000 7f06039d0b80 1 --2- 172.21.15.19:0/2721811761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5804ddd80 0x55d5805eaa50 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.840+0000 7f06039d0b80 1 -- 172.21.15.19:0/2721811761 >> 172.21.15.19:0/2721811761 conn(0x55d580466ff0 msgr2=0x55d5805dfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.840+0000 7f06039d0b80 1 -- 172.21.15.19:0/2721811761 shutdown_connections 2024-05-09T02:25:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.840+0000 7f06039d0b80 1 -- 172.21.15.19:0/2721811761 wait complete. 2024-05-09T02:25:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.841+0000 7f06039d0b80 1 -- start start 2024-05-09T02:25:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.841+0000 7f06039d0b80 1 Processor -- start 2024-05-09T02:25:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.841+0000 7f06039d0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 0x55d5805fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.841+0000 7f06039d0b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5804ddd80 0x55d5805fe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.841+0000 7f06039d0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5805e0660 0x55d5805fe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.841+0000 7f05edd9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5804ddd80 0x55d5805fe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.841+0000 7f05edd9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5804ddd80 0x55d5805fe1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40226/0 (socket says 172.21.15.19:40226) 2024-05-09T02:25:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.841+0000 7f05edd9b700 1 -- 172.21.15.19:0/3392580061 learned_addr learned my addr 172.21.15.19:0/3392580061 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f05ee59c700 1 --2- 172.21.15.19:0/3392580061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 0x55d5805fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f05eed9d700 1 --2- 172.21.15.19:0/3392580061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5805e0660 0x55d5805fe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f06039d0b80 1 -- 172.21.15.19:0/3392580061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01adfd0 con 0x55d5805e0660 2024-05-09T02:25:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f06039d0b80 1 -- 172.21.15.19:0/3392580061 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f05f0075390 con 0x55d580463520 2024-05-09T02:25:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f06039d0b80 1 -- 172.21.15.19:0/3392580061 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05f00725b0 con 0x55d5804ddd80 2024-05-09T02:25:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f05edd9b700 1 -- 172.21.15.19:0/3392580061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 msgr2=0x55d5805fdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f05edd9b700 1 --2- 172.21.15.19:0/3392580061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 0x55d5805fdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f05edd9b700 1 -- 172.21.15.19:0/3392580061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5805e0660 msgr2=0x55d5805fe710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:00.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f05edd9b700 1 --2- 172.21.15.19:0/3392580061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5805e0660 0x55d5805fe710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f05edd9b700 1 -- 172.21.15.19:0/3392580061 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d580664240 con 0x55d5804ddd80 2024-05-09T02:25:00.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f05ee59c700 1 --2- 172.21.15.19:0/3392580061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 0x55d5805fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:25:00.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f06039d0b80 10 client.0 _ll_register_callbacks cb 0x55d580374740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:25:00.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f06039d0b80 1 -- 172.21.15.19:0/3392580061 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05e40a8e80 con 0x55d5804ddd80 2024-05-09T02:25:00.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f05edd9b700 1 --2- 172.21.15.19:0/3392580061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5804ddd80 0x55d5805fe1d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f05e40df780 tx=0x7f05e40a9860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4948de0378eb169 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:00.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f05d5ffb700 10 client.26791 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:25:00.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f05d5ffb700 1 -- 172.21.15.19:0/3392580061 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4020030 con 0x55d5804ddd80 2024-05-09T02:25:00.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.842+0000 7f05d5ffb700 1 -- 172.21.15.19:0/3392580061 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f05e40b1f50 con 0x55d5804ddd80 2024-05-09T02:25:00.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.843+0000 7f05d5ffb700 1 -- 172.21.15.19:0/3392580061 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f05e40ad720 con 0x55d5804ddd80 2024-05-09T02:25:00.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.843+0000 7f06039d0b80 10 client.26791 fetch_fsmap learned FSMap version 58 2024-05-09T02:25:00.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.843+0000 7f06039d0b80 1 -- 172.21.15.19:0/3392580061 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=58}) -- 0x55d5806651f0 con 0x55d5804ddd80 2024-05-09T02:25:00.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.843+0000 7f05d5ffb700 1 -- 172.21.15.19:0/3392580061 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f05e40e40f0 con 0x55d5804ddd80 2024-05-09T02:25:00.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.843+0000 7f06039d0b80 10 client.26791 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:25:00.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.843+0000 7f06039d0b80 10 client.26791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:25:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.843+0000 7f06039d0b80 20 client.26791 populate_metadata read hostname 'smithi019' 2024-05-09T02:25:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.843+0000 7f06039d0b80 1 --2- 172.21.15.19:0/3392580061 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d5805f9ce0 0x55d580667840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.843+0000 7f06039d0b80 4 client.26791 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T02:25:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.843+0000 7f06039d0b80 1 -- 172.21.15.19:0/3392580061 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d58050aba0 con 0x55d5805f9ce0 2024-05-09T02:25:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.843+0000 7f05ee59c700 1 --2- 172.21.15.19:0/3392580061 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d5805f9ce0 0x55d580667840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.844+0000 7f05ee59c700 1 --2- 172.21.15.19:0/3392580061 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d5805f9ce0 0x55d580667840 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=259152df396abb4b server_cookie=7f7d24b063f4b789 in_seq=0 out_seq=0 2024-05-09T02:25:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.844+0000 7f05d5ffb700 10 client.26791 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:25:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.845+0000 7f05d5ffb700 1 -- 172.21.15.19:0/3392580061 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d58050aba0 con 0x55d5805f9ce0 2024-05-09T02:25:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.845+0000 7f05d5ffb700 10 client.26791 handle_command_reply: tid=0 2024-05-09T02:25:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.845+0000 7f06039d0b80 1 client.26791 shutdown 2024-05-09T02:25:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.845+0000 7f06039d0b80 1 -- 172.21.15.19:0/3392580061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5804ddd80 msgr2=0x55d5805fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.845+0000 7f06039d0b80 1 --2- 172.21.15.19:0/3392580061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5804ddd80 0x55d5805fe1d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f05e40df780 tx=0x7f05e40a9860 comp rx=0 tx=0).stop 2024-05-09T02:25:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.845+0000 7f06039d0b80 1 -- 172.21.15.19:0/3392580061 shutdown_connections 2024-05-09T02:25:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.845+0000 7f06039d0b80 1 --2- 172.21.15.19:0/3392580061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d5805e0660 0x55d5805fe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.845+0000 7f06039d0b80 1 --2- 172.21.15.19:0/3392580061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d580463520 0x55d5805fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.845+0000 7f06039d0b80 1 --2- 172.21.15.19:0/3392580061 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55d5805f9ce0 0x55d580667840 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.845+0000 7f06039d0b80 1 --2- 172.21.15.19:0/3392580061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5804ddd80 0x55d5805fe1d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.845+0000 7f06039d0b80 1 -- 172.21.15.19:0/3392580061 >> 172.21.15.19:0/3392580061 conn(0x55d580466ff0 msgr2=0x55d5805eb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.845+0000 7f05d5ffb700 0 client.26791 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:25:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.845+0000 7f06039d0b80 1 -- 172.21.15.19:0/3392580061 shutdown_connections 2024-05-09T02:25:00.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.846+0000 7f06039d0b80 1 -- 172.21.15.19:0/3392580061 wait complete. 2024-05-09T02:25:00.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.846+0000 7f06039d0b80 20 client.26791 trim_cache size 0 max 16384 2024-05-09T02:25:00.854 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:25:00.854 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T02:25:00.854 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "d54fdff6-e10b-4d59-b00c-5e9ca0152916", 2024-05-09T02:25:00.854 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T02:25:00.854 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:25:00.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.847+0000 7f05ef59e700 1 -- 172.21.15.19:0/704861861 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f05d80b29f0 msgr2=0x7f05d80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:00.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.847+0000 7f05ef59e700 1 --2- 172.21.15.19:0/704861861 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f05d80b29f0 0x7f05d80b4f00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55d5805e1850 tx=0x7f05e800b040 comp rx=0 tx=0).stop 2024-05-09T02:25:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.847+0000 7f05ef59e700 1 -- 172.21.15.19:0/704861861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f010b150 msgr2=0x7f05f01a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.847+0000 7f05ef59e700 1 --2- 172.21.15.19:0/704861861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f010b150 0x7f05f01a1840 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f05e400d980 tx=0x7f05e400dc90 comp rx=0 tx=0).stop 2024-05-09T02:25:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.847+0000 7f05ef59e700 1 -- 172.21.15.19:0/704861861 shutdown_connections 2024-05-09T02:25:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.847+0000 7f05ef59e700 1 --2- 172.21.15.19:0/704861861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05f01107e0 0x7f05f01a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.847+0000 7f05ef59e700 1 --2- 172.21.15.19:0/704861861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05f010a800 0x7f05f01a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.847+0000 7f05ef59e700 1 --2- 172.21.15.19:0/704861861 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f05d80b29f0 0x7f05d80b4f00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.847+0000 7f05ef59e700 1 --2- 172.21.15.19:0/704861861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f010b150 0x7f05f01a1840 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.847+0000 7f05ef59e700 1 -- 172.21.15.19:0/704861861 >> 172.21.15.19:0/704861861 conn(0x7f05f0076b00 msgr2=0x7f05f01091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.847+0000 7f05ef59e700 1 -- 172.21.15.19:0/704861861 shutdown_connections 2024-05-09T02:25:00.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:00.847+0000 7f05ef59e700 1 -- 172.21.15.19:0/704861861 wait complete. 2024-05-09T02:25:00.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:00 smithi059 ceph-mon[34781]: pgmap v428: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.0 MiB/s wr, 596 op/s 2024-05-09T02:25:01.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:00 smithi196 ceph-mon[34718]: pgmap v428: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.0 MiB/s wr, 596 op/s 2024-05-09T02:25:01.644 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:25:01.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub queued for path: / 2024-05-09T02:25:01.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:01.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:25:01.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:25:01.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:01.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:25:01.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:25:01.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:25:01.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:25:01.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:01.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:01.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:01.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:25:01.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:01.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:25:01.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:25:01.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:01.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:25:01.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:01.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:01.968 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:01 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub queued for path: / 2024-05-09T02:25:02.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:25:02.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:25:02.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:25:02.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:25:02.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:25:02.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:25:02.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.051 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.051 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:25:02.051 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.051 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:25:02.051 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:25:02.051 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.051 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:25:02.051 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.051 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.051 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:01 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub queued for path: / 2024-05-09T02:25:02.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:25:02.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:25:02.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:25:02.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:25:02.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:25:02.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:25:02.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:25:02.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:25:02.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:25:02.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:25:02.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:01 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.133 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:25:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 800.0 s, 1027.0 tps, lat 98.473 ms stddev 134.067 2024-05-09T02:25:02.887 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:02 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.887 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:02 smithi019 ceph-mon[30151]: pgmap v429: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.4 MiB/s wr, 653 op/s 2024-05-09T02:25:02.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:02 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:02.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:02 smithi059 ceph-mon[34781]: pgmap v429: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.4 MiB/s wr, 653 op/s 2024-05-09T02:25:03.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:02 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:25:03.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:02 smithi196 ceph-mon[34718]: pgmap v429: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.4 MiB/s wr, 653 op/s 2024-05-09T02:25:03.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:03 smithi019 ceph-mon[30151]: pgmap v430: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 514 op/s 2024-05-09T02:25:03.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:03 smithi059 ceph-mon[34781]: pgmap v430: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 514 op/s 2024-05-09T02:25:04.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:03 smithi196 ceph-mon[34718]: pgmap v430: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 514 op/s 2024-05-09T02:25:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.732+0000 7fae9b718700 1 -- 172.21.15.19:0/1597375763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94110c30 msgr2=0x7fae94113050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.732+0000 7fae9b718700 1 --2- 172.21.15.19:0/1597375763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94110c30 0x7fae94113050 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00b0a0 tx=0x7fae8c00b3b0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.732+0000 7fae9b718700 1 -- 172.21.15.19:0/1597375763 shutdown_connections 2024-05-09T02:25:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.732+0000 7fae9b718700 1 --2- 172.21.15.19:0/1597375763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94110c30 0x7fae94113050 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.732+0000 7fae9b718700 1 --2- 172.21.15.19:0/1597375763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae94103080 0x7fae94103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.732+0000 7fae9b718700 1 --2- 172.21.15.19:0/1597375763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae94103a60 0x7fae941106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.732+0000 7fae9b718700 1 -- 172.21.15.19:0/1597375763 >> 172.21.15.19:0/1597375763 conn(0x7fae940fedc0 msgr2=0x7fae941011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.733+0000 7fae9b718700 1 -- 172.21.15.19:0/1597375763 shutdown_connections 2024-05-09T02:25:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.733+0000 7fae9b718700 1 -- 172.21.15.19:0/1597375763 wait complete. 2024-05-09T02:25:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.734+0000 7fae9b718700 1 Processor -- start 2024-05-09T02:25:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.734+0000 7fae9b718700 1 -- start start 2024-05-09T02:25:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.735+0000 7fae9b718700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94103080 0x7fae94107930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.735+0000 7fae9b718700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae94103a60 0x7fae94107e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.735+0000 7fae994b4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94103080 0x7fae94107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:04.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.735+0000 7fae994b4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94103080 0x7fae94107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40800/0 (socket says 172.21.15.19:40800) 2024-05-09T02:25:04.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.735+0000 7fae994b4700 1 -- 172.21.15.19:0/671188962 learned_addr learned my addr 172.21.15.19:0/671188962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:04.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.735+0000 7fae98cb3700 1 --2- 172.21.15.19:0/671188962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae94103a60 0x7fae94107e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:04.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.735+0000 7fae9b718700 1 --2- 172.21.15.19:0/671188962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae94110c30 0x7fae941083b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.736+0000 7fae9b718700 1 -- 172.21.15.19:0/671188962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae94107170 con 0x7fae94103080 2024-05-09T02:25:04.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.736+0000 7fae99cb5700 1 --2- 172.21.15.19:0/671188962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae94110c30 0x7fae941083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:04.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.736+0000 7fae9b718700 1 -- 172.21.15.19:0/671188962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae94106ff0 con 0x7fae94103a60 2024-05-09T02:25:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.736+0000 7fae9b718700 1 -- 172.21.15.19:0/671188962 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae941072f0 con 0x7fae94110c30 2024-05-09T02:25:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.736+0000 7fae994b4700 1 -- 172.21.15.19:0/671188962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae94110c30 msgr2=0x7fae941083b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.736+0000 7fae994b4700 1 --2- 172.21.15.19:0/671188962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae94110c30 0x7fae941083b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.736+0000 7fae994b4700 1 -- 172.21.15.19:0/671188962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae94103a60 msgr2=0x7fae94107e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.736+0000 7fae994b4700 1 --2- 172.21.15.19:0/671188962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae94103a60 0x7fae94107e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.736+0000 7fae994b4700 1 -- 172.21.15.19:0/671188962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9410c320 con 0x7fae94103080 2024-05-09T02:25:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.737+0000 7fae994b4700 1 --2- 172.21.15.19:0/671188962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94103080 0x7fae94107930 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fae9000d950 tx=0x7fae9000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.737+0000 7fae8a7fc700 1 -- 172.21.15.19:0/671188962 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90016070 con 0x7fae94103080 2024-05-09T02:25:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.737+0000 7fae8a7fc700 1 -- 172.21.15.19:0/671188962 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9000fd40 con 0x7fae94103080 2024-05-09T02:25:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.737+0000 7fae9b718700 1 -- 172.21.15.19:0/671188962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9410c610 con 0x7fae94103080 2024-05-09T02:25:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.737+0000 7fae9b718700 1 -- 172.21.15.19:0/671188962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae9410cba0 con 0x7fae94103080 2024-05-09T02:25:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.737+0000 7fae8a7fc700 1 -- 172.21.15.19:0/671188962 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9000e3b0 con 0x7fae94103080 2024-05-09T02:25:04.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.742+0000 7fae8a7fc700 1 -- 172.21.15.19:0/671188962 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fae90019070 con 0x7fae94103080 2024-05-09T02:25:04.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.743+0000 7faeaa12eb80 1 Processor -- start 2024-05-09T02:25:04.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.743+0000 7faeaa12eb80 1 -- start start 2024-05-09T02:25:04.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.743+0000 7faeaa12eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a4d8d00 0x55626a5e1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.743+0000 7faeaa12eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a45e4a0 0x55626a5e1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.743+0000 7faeaa12eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a5db400 0x55626a5db810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.743+0000 7faeaa12eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55626a4fe0a0 con 0x55626a45e4a0 2024-05-09T02:25:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.743+0000 7faeaa12eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55626a4a39a0 con 0x55626a5db400 2024-05-09T02:25:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.743+0000 7faeaa12eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55626a4fa060 con 0x55626a4d8d00 2024-05-09T02:25:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.744+0000 7fae99cb5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a4d8d00 0x55626a5e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.744+0000 7fae99cb5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a4d8d00 0x55626a5e1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40252/0 (socket says 172.21.15.19:40252) 2024-05-09T02:25:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.744+0000 7fae99cb5700 1 -- 172.21.15.19:0/3003053731 learned_addr learned my addr 172.21.15.19:0/3003053731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.744+0000 7fae98cb3700 1 --2- 172.21.15.19:0/3003053731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a5db400 0x55626a5db810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.744+0000 7fae99cb5700 1 -- 172.21.15.19:0/3003053731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a5db400 msgr2=0x55626a5db810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:04.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.744+0000 7fae99cb5700 1 --2- 172.21.15.19:0/3003053731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a5db400 0x55626a5db810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.744+0000 7fae99cb5700 1 -- 172.21.15.19:0/3003053731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a45e4a0 msgr2=0x55626a5e1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.744+0000 7fae99cb5700 1 --2- 172.21.15.19:0/3003053731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a45e4a0 0x55626a5e1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.744+0000 7fae99cb5700 1 -- 172.21.15.19:0/3003053731 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55626a494500 con 0x55626a4d8d00 2024-05-09T02:25:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.744+0000 7fae99cb5700 1 --2- 172.21.15.19:0/3003053731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a4d8d00 0x55626a5e1240 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00d010 tx=0x7fae8c009390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b1225673c633440 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:04.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.745+0000 7fae7f7fe700 1 -- 172.21.15.19:0/3003053731 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c00bcb0 con 0x55626a4d8d00 2024-05-09T02:25:04.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.745+0000 7fae7f7fe700 1 -- 172.21.15.19:0/3003053731 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae8c0075c0 con 0x55626a4d8d00 2024-05-09T02:25:04.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.745+0000 7fae7f7fe700 1 -- 172.21.15.19:0/3003053731 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0052a0 con 0x55626a4d8d00 2024-05-09T02:25:04.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.746+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3003053731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a4d8d00 msgr2=0x55626a5e1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:04.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.746+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/3003053731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a4d8d00 0x55626a5e1240 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00d010 tx=0x7fae8c009390 comp rx=0 tx=0).stop 2024-05-09T02:25:04.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.746+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3003053731 shutdown_connections 2024-05-09T02:25:04.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.746+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/3003053731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a5db400 0x55626a5db810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.746+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/3003053731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a4d8d00 0x55626a5e1240 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.746+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/3003053731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a45e4a0 0x55626a5e1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.746+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3003053731 >> 172.21.15.19:0/3003053731 conn(0x55626a461f70 msgr2=0x55626a5d6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:04.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.749+0000 7fae8a7fc700 1 --2- 172.21.15.19:0/671188962 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae800b2aa0 0x7fae800b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.750+0000 7fae8a7fc700 1 -- 172.21.15.19:0/671188962 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fae900e2480 con 0x7fae94103080 2024-05-09T02:25:04.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.751+0000 7fae98cb3700 1 --2- 172.21.15.19:0/671188962 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae800b2aa0 0x7fae800b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:04.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.754+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3003053731 shutdown_connections 2024-05-09T02:25:04.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.757+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3003053731 wait complete. 2024-05-09T02:25:04.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.757+0000 7faeaa12eb80 1 -- start start 2024-05-09T02:25:04.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.757+0000 7faeaa12eb80 1 Processor -- start 2024-05-09T02:25:04.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.758+0000 7fae98cb3700 1 --2- 172.21.15.19:0/671188962 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae800b2aa0 0x7fae800b4fb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fae94104a80 tx=0x7fae84009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:04.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.758+0000 7faeaa12eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a45e4a0 0x55626a5e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.759+0000 7faeaa12eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a4d8d00 0x55626a5e5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.759+0000 7faeaa12eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a5db400 0x55626a5e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.759+0000 7fae98cb3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a4d8d00 0x55626a5e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:04.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.759+0000 7fae98cb3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a4d8d00 0x55626a5e5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40812/0 (socket says 172.21.15.19:40812) 2024-05-09T02:25:04.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.759+0000 7fae98cb3700 1 -- 172.21.15.19:0/3633573238 learned_addr learned my addr 172.21.15.19:0/3633573238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:04.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.759+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3633573238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55626a5dc100 con 0x55626a4d8d00 2024-05-09T02:25:04.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.759+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3633573238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55626a498b20 con 0x55626a45e4a0 2024-05-09T02:25:04.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.759+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3633573238 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55626a5f1280 con 0x55626a5db400 2024-05-09T02:25:04.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.759+0000 7fae99cb5700 1 --2- 172.21.15.19:0/3633573238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a5db400 0x55626a5e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:04.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.760+0000 7fae994b4700 1 --2- 172.21.15.19:0/3633573238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a45e4a0 0x55626a5e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:04.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.760+0000 7faeaa12eb80 10 client.0 _ll_register_callbacks cb 0x55626a36f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:25:04.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.760+0000 7fae99cb5700 1 -- 172.21.15.19:0/3633573238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a45e4a0 msgr2=0x55626a5e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:04.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.760+0000 7fae99cb5700 1 --2- 172.21.15.19:0/3633573238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a45e4a0 0x55626a5e5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.760+0000 7fae99cb5700 1 -- 172.21.15.19:0/3633573238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a4d8d00 msgr2=0x55626a5e5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:04.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.760+0000 7fae99cb5700 1 --2- 172.21.15.19:0/3633573238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a4d8d00 0x55626a5e5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.760+0000 7fae99cb5700 1 -- 172.21.15.19:0/3633573238 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55626a379490 con 0x55626a5db400 2024-05-09T02:25:04.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.766+0000 7fae99cb5700 1 --2- 172.21.15.19:0/3633573238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a5db400 0x55626a5e5ba0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00b780 tx=0x7fae8c004d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62970822c8d3aed8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:04.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.767+0000 7fae7cff9700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:25:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.771+0000 7fae7cff9700 1 -- 172.21.15.19:0/3633573238 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c007980 con 0x55626a5db400 2024-05-09T02:25:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.771+0000 7fae7cff9700 1 -- 172.21.15.19:0/3633573238 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fae8c007b20 con 0x55626a5db400 2024-05-09T02:25:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.772+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3633573238 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae8c0075c0 con 0x55626a5db400 2024-05-09T02:25:04.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.776+0000 7fae7cff9700 1 -- 172.21.15.19:0/3633573238 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fae8c02f470 con 0x55626a5db400 2024-05-09T02:25:04.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.779+0000 7faeaa12eb80 10 client.26803 fetch_fsmap learned FSMap version 58 2024-05-09T02:25:04.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.779+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3633573238 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55626a5f29d0 con 0x55626a5db400 2024-05-09T02:25:04.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.784+0000 7fae7cff9700 1 -- 172.21.15.19:0/3633573238 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae8c0075c0 con 0x55626a5db400 2024-05-09T02:25:04.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.784+0000 7fae7cff9700 1 -- 172.21.15.19:0/3633573238 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fae8c011070 con 0x55626a5db400 2024-05-09T02:25:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.785+0000 7faeaa12eb80 10 client.26803 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:25:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.785+0000 7faeaa12eb80 10 client.26803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:25:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.785+0000 7faeaa12eb80 20 client.26803 populate_metadata read hostname 'smithi019' 2024-05-09T02:25:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.785+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/3633573238 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55626a5f4ea0 0x55626a5f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.786+0000 7faeaa12eb80 4 client.26803 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:25:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.786+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3633573238 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55626a50ba80 con 0x55626a5f4ea0 2024-05-09T02:25:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.789+0000 7fae994b4700 1 --2- 172.21.15.19:0/3633573238 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55626a5f4ea0 0x55626a5f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.795+0000 7fae994b4700 1 --2- 172.21.15.19:0/3633573238 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55626a5f4ea0 0x55626a5f72c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f300076ac1d1b153 server_cookie=62062c517e7830da in_seq=0 out_seq=0 2024-05-09T02:25:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.796+0000 7fae7cff9700 10 client.26803 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:25:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.796+0000 7fae7cff9700 1 -- 172.21.15.19:0/3633573238 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55626a50ba80 con 0x55626a5f4ea0 2024-05-09T02:25:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.796+0000 7fae7cff9700 10 client.26803 handle_command_reply: tid=0 2024-05-09T02:25:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.796+0000 7faeaa12eb80 1 client.26803 shutdown 2024-05-09T02:25:04.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.799+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3633573238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a5db400 msgr2=0x55626a5e5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:04.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.799+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/3633573238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a5db400 0x55626a5e5ba0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00b780 tx=0x7fae8c004d70 comp rx=0 tx=0).stop 2024-05-09T02:25:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.802+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3633573238 shutdown_connections 2024-05-09T02:25:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.802+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/3633573238 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55626a5f4ea0 0x55626a5f72c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.802+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/3633573238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a5db400 0x55626a5e5ba0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.802+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/3633573238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a45e4a0 0x55626a5e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.802+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/3633573238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a4d8d00 0x55626a5e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.802+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3633573238 >> 172.21.15.19:0/3633573238 conn(0x55626a461f70 msgr2=0x55626a5cfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.802+0000 7fae7cff9700 0 client.26803 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:25:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.803+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3633573238 shutdown_connections 2024-05-09T02:25:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.803+0000 7faeaa12eb80 1 -- 172.21.15.19:0/3633573238 wait complete. 2024-05-09T02:25:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.804+0000 7faeaa12eb80 20 client.26803 trim_cache size 0 max 16384 2024-05-09T02:25:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.845+0000 7faeaa12eb80 1 Processor -- start 2024-05-09T02:25:04.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.846+0000 7faeaa12eb80 1 -- start start 2024-05-09T02:25:04.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.847+0000 7faeaa12eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a45e4a0 0x55626a5e5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.847+0000 7faeaa12eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a4d8d00 0x55626a5da3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.847+0000 7faeaa12eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a5db400 0x55626a5da930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.847+0000 7faeaa12eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae941bdd80 con 0x55626a5db400 2024-05-09T02:25:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.847+0000 7faeaa12eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae941bf8c0 con 0x55626a4d8d00 2024-05-09T02:25:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.847+0000 7faeaa12eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae941be450 con 0x55626a45e4a0 2024-05-09T02:25:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.847+0000 7fae98cb3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a4d8d00 0x55626a5da3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.847+0000 7fae98cb3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a4d8d00 0x55626a5da3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:55630/0 (socket says 172.21.15.19:55630) 2024-05-09T02:25:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.847+0000 7fae98cb3700 1 -- 172.21.15.19:0/1946869190 learned_addr learned my addr 172.21.15.19:0/1946869190 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:04.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.848+0000 7fae994b4700 1 --2- 172.21.15.19:0/1946869190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a45e4a0 0x55626a5e5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:04.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.848+0000 7fae98cb3700 1 -- 172.21.15.19:0/1946869190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a45e4a0 msgr2=0x55626a5e5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:04.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.848+0000 7fae98cb3700 1 --2- 172.21.15.19:0/1946869190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a45e4a0 0x55626a5e5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.848+0000 7fae98cb3700 1 -- 172.21.15.19:0/1946869190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a5db400 msgr2=0x55626a5da930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:04.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.848+0000 7fae98cb3700 1 --2- 172.21.15.19:0/1946869190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a5db400 0x55626a5da930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.848+0000 7fae98cb3700 1 -- 172.21.15.19:0/1946869190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55626a5cfd40 con 0x55626a4d8d00 2024-05-09T02:25:04.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.848+0000 7fae98cb3700 1 --2- 172.21.15.19:0/1946869190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a4d8d00 0x55626a5da3f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fae84012e10 tx=0x7fae84015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea7db18a9a32f433 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:04.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.848+0000 7fae7effd700 1 -- 172.21.15.19:0/1946869190 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84010c00 con 0x55626a4d8d00 2024-05-09T02:25:04.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.849+0000 7fae7effd700 1 -- 172.21.15.19:0/1946869190 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae84010da0 con 0x55626a4d8d00 2024-05-09T02:25:04.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.850+0000 7fae7effd700 1 -- 172.21.15.19:0/1946869190 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84005680 con 0x55626a4d8d00 2024-05-09T02:25:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.851+0000 7faeaa12eb80 1 -- 172.21.15.19:0/1946869190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a4d8d00 msgr2=0x55626a5da3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.851+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/1946869190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a4d8d00 0x55626a5da3f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fae84012e10 tx=0x7fae84015000 comp rx=0 tx=0).stop 2024-05-09T02:25:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.851+0000 7faeaa12eb80 1 -- 172.21.15.19:0/1946869190 shutdown_connections 2024-05-09T02:25:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.851+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/1946869190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a5db400 0x55626a5da930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.851+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/1946869190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a45e4a0 0x55626a5e5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.851+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/1946869190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a4d8d00 0x55626a5da3f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.851+0000 7faeaa12eb80 1 -- 172.21.15.19:0/1946869190 >> 172.21.15.19:0/1946869190 conn(0x55626a461f70 msgr2=0x55626a5e3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.854+0000 7faeaa12eb80 1 -- 172.21.15.19:0/1946869190 shutdown_connections 2024-05-09T02:25:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.854+0000 7faeaa12eb80 1 -- 172.21.15.19:0/1946869190 wait complete. 2024-05-09T02:25:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.855+0000 7faeaa12eb80 1 -- start start 2024-05-09T02:25:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.855+0000 7faeaa12eb80 1 Processor -- start 2024-05-09T02:25:04.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.856+0000 7faeaa12eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a45e4a0 0x55626a653580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.856+0000 7faeaa12eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a4d8d00 0x55626a653ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.856+0000 7faeaa12eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a5db400 0x55626a65e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.856+0000 7faeaa12eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae941bf1f0 con 0x55626a45e4a0 2024-05-09T02:25:04.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.856+0000 7faeaa12eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae941beb20 con 0x55626a5db400 2024-05-09T02:25:04.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.856+0000 7faeaa12eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae941bbd40 con 0x55626a4d8d00 2024-05-09T02:25:04.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.856+0000 7faeaa12eb80 10 client.0 _ll_register_callbacks cb 0x55626a36f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:25:04.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.857+0000 7fae98cb3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a4d8d00 0x55626a653ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:04.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.857+0000 7fae98cb3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a4d8d00 0x55626a653ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:40268/0 (socket says 172.21.15.19:40268) 2024-05-09T02:25:04.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.857+0000 7fae98cb3700 1 -- 172.21.15.19:0/2678886054 learned_addr learned my addr 172.21.15.19:0/2678886054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:04.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.857+0000 7fae99cb5700 1 --2- 172.21.15.19:0/2678886054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a5db400 0x55626a65e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:04.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.857+0000 7fae98cb3700 1 -- 172.21.15.19:0/2678886054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a5db400 msgr2=0x55626a65e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:04.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.857+0000 7fae98cb3700 1 --2- 172.21.15.19:0/2678886054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a5db400 0x55626a65e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.857+0000 7fae98cb3700 1 -- 172.21.15.19:0/2678886054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a45e4a0 msgr2=0x55626a653580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:25:04.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.857+0000 7fae98cb3700 1 --2- 172.21.15.19:0/2678886054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a45e4a0 0x55626a653580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.857+0000 7fae98cb3700 1 -- 172.21.15.19:0/2678886054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55626a66f950 con 0x55626a4d8d00 2024-05-09T02:25:04.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.858+0000 7fae98cb3700 1 --2- 172.21.15.19:0/2678886054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a4d8d00 0x55626a653ac0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fae84012de0 tx=0x7fae84004a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d81f66020a49f0c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:04.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.858+0000 7fae7cff9700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:25:04.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.858+0000 7fae7cff9700 1 -- 172.21.15.19:0/2678886054 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84010c00 con 0x55626a4d8d00 2024-05-09T02:25:04.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.858+0000 7faeaa12eb80 1 -- 172.21.15.19:0/2678886054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae84010da0 con 0x55626a4d8d00 2024-05-09T02:25:04.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.858+0000 7fae7cff9700 1 -- 172.21.15.19:0/2678886054 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fae84004030 con 0x55626a4d8d00 2024-05-09T02:25:04.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.859+0000 7fae7cff9700 1 -- 172.21.15.19:0/2678886054 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fae84035050 con 0x55626a4d8d00 2024-05-09T02:25:04.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.862+0000 7faeaa12eb80 10 client.26809 fetch_fsmap learned FSMap version 58 2024-05-09T02:25:04.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.862+0000 7faeaa12eb80 1 -- 172.21.15.19:0/2678886054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55626a670b50 con 0x55626a4d8d00 2024-05-09T02:25:04.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.862+0000 7fae7cff9700 1 -- 172.21.15.19:0/2678886054 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae84010da0 con 0x55626a4d8d00 2024-05-09T02:25:04.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.862+0000 7fae7cff9700 1 -- 172.21.15.19:0/2678886054 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fae84019040 con 0x55626a4d8d00 2024-05-09T02:25:04.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.863+0000 7faeaa12eb80 10 client.26809 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:25:04.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.863+0000 7faeaa12eb80 10 client.26809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:25:04.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.863+0000 7faeaa12eb80 20 client.26809 populate_metadata read hostname 'smithi019' 2024-05-09T02:25:04.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.863+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/2678886054 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55626a671110 0x55626a673530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:04.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.863+0000 7faeaa12eb80 4 client.26809 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:25:04.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.863+0000 7faeaa12eb80 1 -- 172.21.15.19:0/2678886054 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55626a505b20 con 0x55626a671110 2024-05-09T02:25:04.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.864+0000 7fae994b4700 1 --2- 172.21.15.19:0/2678886054 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55626a671110 0x55626a673530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:04.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.865+0000 7fae994b4700 1 --2- 172.21.15.19:0/2678886054 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55626a671110 0x55626a673530 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b90e1c1b3678fd76 server_cookie=37214da3fa5c1941 in_seq=0 out_seq=0 2024-05-09T02:25:04.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.865+0000 7fae7cff9700 10 client.26809 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:25:04.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.866+0000 7fae7cff9700 1 -- 172.21.15.19:0/2678886054 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55626a505b20 con 0x55626a671110 2024-05-09T02:25:04.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.866+0000 7fae7cff9700 10 client.26809 handle_command_reply: tid=0 2024-05-09T02:25:04.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.866+0000 7faeaa12eb80 1 client.26809 shutdown 2024-05-09T02:25:04.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.867+0000 7faeaa12eb80 1 -- 172.21.15.19:0/2678886054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a4d8d00 msgr2=0x55626a653ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:04.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.867+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/2678886054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a4d8d00 0x55626a653ac0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fae84012de0 tx=0x7fae84004a00 comp rx=0 tx=0).stop 2024-05-09T02:25:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.867+0000 7faeaa12eb80 1 -- 172.21.15.19:0/2678886054 shutdown_connections 2024-05-09T02:25:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.867+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/2678886054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55626a5db400 0x55626a65e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.867+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/2678886054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55626a45e4a0 0x55626a653580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.867+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/2678886054 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x55626a671110 0x55626a673530 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.867+0000 7faeaa12eb80 1 --2- 172.21.15.19:0/2678886054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55626a4d8d00 0x55626a653ac0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.867+0000 7faeaa12eb80 1 -- 172.21.15.19:0/2678886054 >> 172.21.15.19:0/2678886054 conn(0x55626a461f70 msgr2=0x55626a5decd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.867+0000 7fae7cff9700 0 client.26809 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:25:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.867+0000 7faeaa12eb80 1 -- 172.21.15.19:0/2678886054 shutdown_connections 2024-05-09T02:25:04.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.870+0000 7faeaa12eb80 1 -- 172.21.15.19:0/2678886054 wait complete. 2024-05-09T02:25:04.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.871+0000 7faeaa12eb80 20 client.26809 trim_cache size 0 max 16384 2024-05-09T02:25:04.874 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:25:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T02:25:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T02:25:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: "d54fdff6-e10b-4d59-b00c-5e9ca0152916": { 2024-05-09T02:25:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T02:25:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "d54fdff6-e10b-4d59-b00c-5e9ca0152916", 2024-05-09T02:25:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T02:25:04.875 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:25:04.875 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:25:04.875 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:25:04.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.872+0000 7fae9b718700 1 -- 172.21.15.19:0/671188962 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae800b2aa0 msgr2=0x7fae800b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:04.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.872+0000 7fae9b718700 1 --2- 172.21.15.19:0/671188962 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae800b2aa0 0x7fae800b4fb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fae94104a80 tx=0x7fae84009260 comp rx=0 tx=0).stop 2024-05-09T02:25:04.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.872+0000 7fae9b718700 1 -- 172.21.15.19:0/671188962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94103080 msgr2=0x7fae94107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:04.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.872+0000 7fae9b718700 1 --2- 172.21.15.19:0/671188962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94103080 0x7fae94107930 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fae9000d950 tx=0x7fae9000dc60 comp rx=0 tx=0).stop 2024-05-09T02:25:04.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.872+0000 7fae9b718700 1 -- 172.21.15.19:0/671188962 shutdown_connections 2024-05-09T02:25:04.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.872+0000 7fae9b718700 1 --2- 172.21.15.19:0/671188962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae94110c30 0x7fae941083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.872+0000 7fae9b718700 1 --2- 172.21.15.19:0/671188962 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fae800b2aa0 0x7fae800b4fb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.872+0000 7fae9b718700 1 --2- 172.21.15.19:0/671188962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae94103080 0x7fae94107930 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.872+0000 7fae9b718700 1 --2- 172.21.15.19:0/671188962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae94103a60 0x7fae94107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:04.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.872+0000 7fae9b718700 1 -- 172.21.15.19:0/671188962 >> 172.21.15.19:0/671188962 conn(0x7fae940fedc0 msgr2=0x7fae94112790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:04.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.873+0000 7fae9b718700 1 -- 172.21.15.19:0/671188962 shutdown_connections 2024-05-09T02:25:04.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:04.873+0000 7fae9b718700 1 -- 172.21.15.19:0/671188962 wait complete. 2024-05-09T02:25:06.747 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:06 smithi196 ceph-mon[34718]: pgmap v431: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 15 MiB/s wr, 1.38k op/s 2024-05-09T02:25:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:06 smithi019 ceph-mon[30151]: pgmap v431: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 15 MiB/s wr, 1.38k op/s 2024-05-09T02:25:06.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:06 smithi059 ceph-mon[34781]: pgmap v431: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 15 MiB/s wr, 1.38k op/s 2024-05-09T02:25:07.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 805.0 s, 915.2 tps, lat 108.076 ms stddev 153.140 2024-05-09T02:25:07.693 INFO:tasks.cephfs.filesystem:scrub status for tag:d54fdff6-e10b-4d59-b00c-5e9ca0152916 - {'path': '/', 'tag': 'd54fdff6-e10b-4d59-b00c-5e9ca0152916', 'options': 'recursive,force'} 2024-05-09T02:25:07.717 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:07 smithi059 ceph-mon[34781]: osdmap e97: 12 total, 12 up, 12 in 2024-05-09T02:25:07.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:07 smithi196 ceph-mon[34718]: osdmap e97: 12 total, 12 up, 12 in 2024-05-09T02:25:07.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:07 smithi019 ceph-mon[30151]: osdmap e97: 12 total, 12 up, 12 in 2024-05-09T02:25:08.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:08 smithi059 ceph-mon[34781]: pgmap v433: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 1.57k op/s 2024-05-09T02:25:08.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:08 smithi196 ceph-mon[34718]: pgmap v433: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 1.57k op/s 2024-05-09T02:25:08.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:08 smithi019 ceph-mon[30151]: pgmap v433: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 1.57k op/s 2024-05-09T02:25:10.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:10 smithi196 ceph-mon[34718]: pgmap v434: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 16 MiB/s wr, 1.58k op/s 2024-05-09T02:25:10.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:10 smithi019 ceph-mon[30151]: pgmap v434: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 16 MiB/s wr, 1.58k op/s 2024-05-09T02:25:10.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:10 smithi059 ceph-mon[34781]: pgmap v434: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 16 MiB/s wr, 1.58k op/s 2024-05-09T02:25:11.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:11 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:25:11.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:11 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:25:11.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:11 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:25:11.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:11 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:25:11.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:11 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:25:11.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:11 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:25:11.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:11 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:25:11.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:11 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:25:11.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:11 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:25:11.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:11 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:25:11.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:11 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:25:11.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:11 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:25:11.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:11 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:25:11.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:11 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:25:11.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:11 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:25:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 810.0 s, 1127.4 tps, lat 89.002 ms stddev 102.528 2024-05-09T02:25:12.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:12 smithi196 ceph-mon[34718]: pgmap v435: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-05-09T02:25:12.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:12 smithi019 ceph-mon[30151]: pgmap v435: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-05-09T02:25:12.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:12 smithi059 ceph-mon[34781]: pgmap v435: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-05-09T02:25:14.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:14 smithi019 ceph-mon[30151]: pgmap v436: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-05-09T02:25:14.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:14 smithi059 ceph-mon[34781]: pgmap v436: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-05-09T02:25:15.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:14 smithi196 ceph-mon[34718]: pgmap v436: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-05-09T02:25:15.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:15 smithi059 ceph-mon[34781]: osdmap e98: 12 total, 12 up, 12 in 2024-05-09T02:25:16.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:15 smithi196 ceph-mon[34718]: osdmap e98: 12 total, 12 up, 12 in 2024-05-09T02:25:16.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:15 smithi019 ceph-mon[30151]: osdmap e98: 12 total, 12 up, 12 in 2024-05-09T02:25:16.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:16 smithi059 ceph-mon[34781]: pgmap v438: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-05-09T02:25:17.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:16 smithi196 ceph-mon[34718]: pgmap v438: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-05-09T02:25:17.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:16 smithi019 ceph-mon[30151]: pgmap v438: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-05-09T02:25:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 815.0 s, 1269.6 tps, lat 79.051 ms stddev 84.851 2024-05-09T02:25:18.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:18 smithi059 ceph-mon[34781]: pgmap v439: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.1 MiB/s wr, 589 op/s 2024-05-09T02:25:19.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:18 smithi196 ceph-mon[34718]: pgmap v439: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.1 MiB/s wr, 589 op/s 2024-05-09T02:25:19.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:18 smithi019 ceph-mon[30151]: pgmap v439: 97 pgs: 97 active+clean; 4.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.1 MiB/s wr, 589 op/s 2024-05-09T02:25:19.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:19 smithi059 ceph-mon[34781]: pgmap v440: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.6 MiB/s wr, 615 op/s 2024-05-09T02:25:20.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:19 smithi196 ceph-mon[34718]: pgmap v440: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.6 MiB/s wr, 615 op/s 2024-05-09T02:25:20.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:19 smithi019 ceph-mon[30151]: pgmap v440: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.6 MiB/s wr, 615 op/s 2024-05-09T02:25:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 820.0 s, 1182.4 tps, lat 84.479 ms stddev 92.984 2024-05-09T02:25:22.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:22 smithi196 ceph-mon[34718]: pgmap v441: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.1 MiB/s wr, 601 op/s 2024-05-09T02:25:22.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:22 smithi019 ceph-mon[30151]: pgmap v441: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.1 MiB/s wr, 601 op/s 2024-05-09T02:25:22.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:22 smithi059 ceph-mon[34781]: pgmap v441: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.1 MiB/s wr, 601 op/s 2024-05-09T02:25:24.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:24 smithi196 ceph-mon[34718]: pgmap v442: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.1 MiB/s wr, 601 op/s 2024-05-09T02:25:24.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:24 smithi019 ceph-mon[30151]: pgmap v442: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.1 MiB/s wr, 601 op/s 2024-05-09T02:25:24.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:24 smithi059 ceph-mon[34781]: pgmap v442: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.1 MiB/s wr, 601 op/s 2024-05-09T02:25:26.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:26 smithi196 ceph-mon[34718]: pgmap v443: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.3 MiB/s wr, 655 op/s 2024-05-09T02:25:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:26 smithi019 ceph-mon[30151]: pgmap v443: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.3 MiB/s wr, 655 op/s 2024-05-09T02:25:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:26 smithi059 ceph-mon[34781]: pgmap v443: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.3 MiB/s wr, 655 op/s 2024-05-09T02:25:27.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 825.0 s, 1537.4 tps, lat 65.159 ms stddev 80.127 2024-05-09T02:25:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:28 smithi196 ceph-mon[34718]: pgmap v444: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.6 MiB/s wr, 547 op/s 2024-05-09T02:25:28.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:28 smithi019 ceph-mon[30151]: pgmap v444: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.6 MiB/s wr, 547 op/s 2024-05-09T02:25:28.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:28 smithi059 ceph-mon[34781]: pgmap v444: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.6 MiB/s wr, 547 op/s 2024-05-09T02:25:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:30 smithi196 ceph-mon[34718]: pgmap v445: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 647 op/s 2024-05-09T02:25:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:30 smithi019 ceph-mon[30151]: pgmap v445: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 647 op/s 2024-05-09T02:25:30.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:30 smithi059 ceph-mon[34781]: pgmap v445: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 647 op/s 2024-05-09T02:25:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 830.0 s, 1557.4 tps, lat 64.149 ms stddev 77.132 2024-05-09T02:25:32.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:32 smithi196 ceph-mon[34718]: pgmap v446: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.9 MiB/s wr, 650 op/s 2024-05-09T02:25:32.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:32 smithi019 ceph-mon[30151]: pgmap v446: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.9 MiB/s wr, 650 op/s 2024-05-09T02:25:32.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:32 smithi059 ceph-mon[34781]: pgmap v446: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.9 MiB/s wr, 650 op/s 2024-05-09T02:25:34.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:34 smithi196 ceph-mon[34718]: pgmap v447: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.5 MiB/s wr, 523 op/s 2024-05-09T02:25:34.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:34 smithi019 ceph-mon[30151]: pgmap v447: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.5 MiB/s wr, 523 op/s 2024-05-09T02:25:34.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:34 smithi059 ceph-mon[34781]: pgmap v447: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.5 MiB/s wr, 523 op/s 2024-05-09T02:25:36.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:36 smithi196 ceph-mon[34718]: pgmap v448: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.3 MiB/s wr, 779 op/s 2024-05-09T02:25:36.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:36 smithi019 ceph-mon[30151]: pgmap v448: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.3 MiB/s wr, 779 op/s 2024-05-09T02:25:36.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:36 smithi059 ceph-mon[34781]: pgmap v448: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.3 MiB/s wr, 779 op/s 2024-05-09T02:25:37.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 835.0 s, 1335.4 tps, lat 74.471 ms stddev 83.143 2024-05-09T02:25:37.694 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:25:38.265 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:25:38.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:38 smithi019 ceph-mon[30151]: pgmap v449: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.2 MiB/s wr, 600 op/s 2024-05-09T02:25:38.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:38 smithi196 ceph-mon[34718]: pgmap v449: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.2 MiB/s wr, 600 op/s 2024-05-09T02:25:38.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:38 smithi059 ceph-mon[34781]: pgmap v449: 97 pgs: 97 active+clean; 4.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.2 MiB/s wr, 600 op/s 2024-05-09T02:25:40.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:40 smithi196 ceph-mon[34718]: pgmap v450: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.3 MiB/s wr, 614 op/s 2024-05-09T02:25:40.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.793+0000 7fef1b59e700 1 -- 172.21.15.19:0/4195586066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef1c0764b0 msgr2=0x7fef1c114c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.793+0000 7fef1b59e700 1 --2- 172.21.15.19:0/4195586066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef1c0764b0 0x7fef1c114c40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fef100098f0 tx=0x7fef10009c00 comp rx=0 tx=0).stop 2024-05-09T02:25:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.794+0000 7fef1b59e700 1 -- 172.21.15.19:0/4195586066 shutdown_connections 2024-05-09T02:25:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.795+0000 7fef1b59e700 1 --2- 172.21.15.19:0/4195586066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef1c115180 0x7fef1c1175a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.795+0000 7fef1b59e700 1 --2- 172.21.15.19:0/4195586066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef1c075b60 0x7fef1c075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.795+0000 7fef1b59e700 1 --2- 172.21.15.19:0/4195586066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef1c0764b0 0x7fef1c114c40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.795+0000 7fef1b59e700 1 -- 172.21.15.19:0/4195586066 >> 172.21.15.19:0/4195586066 conn(0x7fef1c079d10 msgr2=0x7fef1c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:40.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:40 smithi019 ceph-mon[30151]: pgmap v450: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.3 MiB/s wr, 614 op/s 2024-05-09T02:25:40.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.797+0000 7fef1b59e700 1 -- 172.21.15.19:0/4195586066 shutdown_connections 2024-05-09T02:25:40.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.798+0000 7fef1b59e700 1 -- 172.21.15.19:0/4195586066 wait complete. 2024-05-09T02:25:40.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.799+0000 7fef1b59e700 1 Processor -- start 2024-05-09T02:25:40.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.821+0000 7fef1b59e700 1 -- start start 2024-05-09T02:25:40.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.821+0000 7fef1b59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef1c075b60 0x7fef1c0752d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.821+0000 7fef1b59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef1c115180 0x7fef1c0733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.821+0000 7fef1b59e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef1c073900 0x7fef1c073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.821+0000 7fef1b59e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c06d6b0 con 0x7fef1c115180 2024-05-09T02:25:40.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.821+0000 7fef1b59e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c06d530 con 0x7fef1c075b60 2024-05-09T02:25:40.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.821+0000 7fef1b59e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c06d830 con 0x7fef1c073900 2024-05-09T02:25:40.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.824+0000 7fef1a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef1c075b60 0x7fef1c0752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:40.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.824+0000 7fef1a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef1c075b60 0x7fef1c0752d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:52886/0 (socket says 172.21.15.19:52886) 2024-05-09T02:25:40.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.824+0000 7fef1a59c700 1 -- 172.21.15.19:0/1997387376 learned_addr learned my addr 172.21.15.19:0/1997387376 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:40.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.824+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/1997387376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef1c073900 0x7fef1c073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.830+0000 7fef1a59c700 1 -- 172.21.15.19:0/1997387376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef1c073900 msgr2=0x7fef1c073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.830+0000 7fef1a59c700 1 --2- 172.21.15.19:0/1997387376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef1c073900 0x7fef1c073d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.830+0000 7fef1a59c700 1 -- 172.21.15.19:0/1997387376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef1c115180 msgr2=0x7fef1c0733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.830+0000 7fef1a59c700 1 --2- 172.21.15.19:0/1997387376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef1c115180 0x7fef1c0733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.830+0000 7fef1a59c700 1 -- 172.21.15.19:0/1997387376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1c112500 con 0x7fef1c075b60 2024-05-09T02:25:40.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.834+0000 7fef1a59c700 1 --2- 172.21.15.19:0/1997387376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef1c075b60 0x7fef1c0752d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00e9c0 tx=0x7fef0c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:40.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.836+0000 7fef0b7fe700 1 -- 172.21.15.19:0/1997387376 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00cbd0 con 0x7fef1c075b60 2024-05-09T02:25:40.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.836+0000 7fef1b59e700 1 -- 172.21.15.19:0/1997387376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef1c1127f0 con 0x7fef1c075b60 2024-05-09T02:25:40.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.836+0000 7fef1b59e700 1 -- 172.21.15.19:0/1997387376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef1c112d80 con 0x7fef1c075b60 2024-05-09T02:25:40.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.842+0000 7fef2f38ab80 1 Processor -- start 2024-05-09T02:25:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.842+0000 7fef2f38ab80 1 -- start start 2024-05-09T02:25:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.843+0000 7fef2f38ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626697f6d00 0x5626698ff490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.843+0000 7fef2f38ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266977c4a0 0x5626698ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.843+0000 7fef2f38ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626698f95e0 0x5626698f99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.843+0000 7fef2f38ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56266981c0a0 con 0x56266977c4a0 2024-05-09T02:25:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.843+0000 7fef2f38ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5626697c19a0 con 0x5626697f6d00 2024-05-09T02:25:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.843+0000 7fef2f38ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562669818060 con 0x5626698f95e0 2024-05-09T02:25:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.843+0000 7fef1ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626697f6d00 0x5626698ff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.844+0000 7fef1ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626697f6d00 0x5626698ff490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:52900/0 (socket says 172.21.15.19:52900) 2024-05-09T02:25:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.844+0000 7fef1ad9d700 1 -- 172.21.15.19:0/1719957319 learned_addr learned my addr 172.21.15.19:0/1719957319 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.844+0000 7fef0b7fe700 1 -- 172.21.15.19:0/1997387376 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef0c00cd70 con 0x7fef1c075b60 2024-05-09T02:25:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.844+0000 7fef0b7fe700 1 -- 172.21.15.19:0/1997387376 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00cbd0 con 0x7fef1c075b60 2024-05-09T02:25:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.845+0000 7fef1a59c700 1 --2- 172.21.15.19:0/1719957319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266977c4a0 0x5626698ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:40.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.845+0000 7fef1ad9d700 1 -- 172.21.15.19:0/1719957319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626698f95e0 msgr2=0x5626698f99f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:25:40.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.845+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/1719957319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626698f95e0 0x5626698f99f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.845+0000 7fef1ad9d700 1 -- 172.21.15.19:0/1719957319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266977c4a0 msgr2=0x5626698ff9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:40.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.845+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/1719957319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266977c4a0 0x5626698ff9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.845+0000 7fef1ad9d700 1 -- 172.21.15.19:0/1719957319 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626697b2500 con 0x5626697f6d00 2024-05-09T02:25:40.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.846+0000 7fef1a59c700 1 --2- 172.21.15.19:0/1719957319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266977c4a0 0x5626698ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:25:40.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.850+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/1719957319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626697f6d00 0x5626698ff490 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fef14011680 tx=0x7fef14009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a47e60ce38289f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:40.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.851+0000 7fef08ff9700 1 -- 172.21.15.19:0/1719957319 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14019030 con 0x5626697f6d00 2024-05-09T02:25:40.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.853+0000 7fef08ff9700 1 -- 172.21.15.19:0/1719957319 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef140093a0 con 0x5626697f6d00 2024-05-09T02:25:40.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.853+0000 7fef0b7fe700 1 -- 172.21.15.19:0/1997387376 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fef0c005390 con 0x7fef1c075b60 2024-05-09T02:25:40.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.855+0000 7fef0b7fe700 1 --2- 172.21.15.19:0/1997387376 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fef040b2eb0 0x7fef040b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.855+0000 7fef2f38ab80 1 -- 172.21.15.19:0/1719957319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626697f6d00 msgr2=0x5626698ff490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:40.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.855+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/1719957319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626697f6d00 0x5626698ff490 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fef14011680 tx=0x7fef14009920 comp rx=0 tx=0).stop 2024-05-09T02:25:40.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.855+0000 7fef2f38ab80 1 -- 172.21.15.19:0/1719957319 shutdown_connections 2024-05-09T02:25:40.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.855+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/1719957319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626698f95e0 0x5626698f99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.855+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/1719957319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626697f6d00 0x5626698ff490 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.855+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/1719957319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266977c4a0 0x5626698ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.855+0000 7fef2f38ab80 1 -- 172.21.15.19:0/1719957319 >> 172.21.15.19:0/1719957319 conn(0x56266977ff70 msgr2=0x5626698f47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:40.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.855+0000 7fef0b7fe700 1 -- 172.21.15.19:0/1997387376 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fef0c00c9d0 con 0x7fef1c075b60 2024-05-09T02:25:40.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.858+0000 7fef1a59c700 1 --2- 172.21.15.19:0/1997387376 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fef040b2eb0 0x7fef040b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:40.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.858+0000 7fef2f38ab80 1 -- 172.21.15.19:0/1719957319 shutdown_connections 2024-05-09T02:25:40.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.862+0000 7fef2f38ab80 1 -- 172.21.15.19:0/1719957319 wait complete. 2024-05-09T02:25:40.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.863+0000 7fef2f38ab80 1 -- start start 2024-05-09T02:25:40.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.863+0000 7fef1a59c700 1 --2- 172.21.15.19:0/1997387376 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fef040b2eb0 0x7fef040b53c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0dfa70 tx=0x7fef0c00efb0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:40.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.863+0000 7fef2f38ab80 1 Processor -- start 2024-05-09T02:25:40.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.864+0000 7fef2f38ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56266977c4a0 0x562669905a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.864+0000 7fef2f38ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626698f95e0 0x562669902cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.864+0000 7fef2f38ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562669903210 0x562669903620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.864+0000 7fef2f38ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5626698fa3e0 con 0x562669903210 2024-05-09T02:25:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.864+0000 7fef2f38ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5626697b6b20 con 0x5626698f95e0 2024-05-09T02:25:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.864+0000 7fef2f38ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562669906200 con 0x56266977c4a0 2024-05-09T02:25:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.864+0000 7fef2f38ab80 10 client.0 _ll_register_callbacks cb 0x56266968d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:25:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.867+0000 7fef1ad9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56266977c4a0 0x562669905a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.867+0000 7fef1ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562669903210 0x562669903620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.867+0000 7fef1ad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562669903210 0x562669903620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45992/0 (socket says 172.21.15.19:45992) 2024-05-09T02:25:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.867+0000 7fef1ad9d700 1 -- 172.21.15.19:0/2673480996 learned_addr learned my addr 172.21.15.19:0/2673480996 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.867+0000 7fef19d9b700 1 --2- 172.21.15.19:0/2673480996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626698f95e0 0x562669902cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:40.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.872+0000 7fef1ad9d700 1 -- 172.21.15.19:0/2673480996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56266977c4a0 msgr2=0x562669905a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:40.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.872+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/2673480996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56266977c4a0 0x562669905a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.872+0000 7fef1ad9d700 1 -- 172.21.15.19:0/2673480996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626698f95e0 msgr2=0x562669902cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:40.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.872+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/2673480996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626698f95e0 0x562669902cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.872+0000 7fef1ad9d700 1 -- 172.21.15.19:0/2673480996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562669697490 con 0x562669903210 2024-05-09T02:25:40.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.872+0000 7fef19d9b700 1 --2- 172.21.15.19:0/2673480996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626698f95e0 0x562669902cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:25:40.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.877+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/2673480996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562669903210 0x562669903620 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x562669902550 tx=0x7fef14011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3311b2dcd52b27 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:40.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.877+0000 7fef01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:25:40.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.881+0000 7fef01ffb700 1 -- 172.21.15.19:0/2673480996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14005050 con 0x562669903210 2024-05-09T02:25:40.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.881+0000 7fef2f38ab80 1 -- 172.21.15.19:0/2673480996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562669829a80 con 0x562669903210 2024-05-09T02:25:40.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.882+0000 7fef01ffb700 1 -- 172.21.15.19:0/2673480996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fef140051f0 con 0x562669903210 2024-05-09T02:25:40.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.882+0000 7fef01ffb700 1 -- 172.21.15.19:0/2673480996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fef140053f0 con 0x562669903210 2024-05-09T02:25:40.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.884+0000 7fef2f38ab80 10 client.17040 fetch_fsmap learned FSMap version 58 2024-05-09T02:25:40.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.884+0000 7fef2f38ab80 1 -- 172.21.15.19:0/2673480996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x562669910cd0 con 0x562669903210 2024-05-09T02:25:40.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.885+0000 7fef01ffb700 1 -- 172.21.15.19:0/2673480996 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562669829a80 con 0x562669903210 2024-05-09T02:25:40.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.885+0000 7fef01ffb700 1 -- 172.21.15.19:0/2673480996 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fef14015a30 con 0x562669903210 2024-05-09T02:25:40.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.887+0000 7fef2f38ab80 10 client.17040 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:25:40.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.887+0000 7fef2f38ab80 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:25:40.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.887+0000 7fef2f38ab80 20 client.17040 populate_metadata read hostname 'smithi019' 2024-05-09T02:25:40.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.887+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/2673480996 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5626699132b0 0x5626699156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.887+0000 7fef2f38ab80 4 client.17040 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:25:40.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.887+0000 7fef2f38ab80 1 -- 172.21.15.19:0/2673480996 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562669823b20 con 0x5626699132b0 2024-05-09T02:25:40.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.889+0000 7fef19d9b700 1 --2- 172.21.15.19:0/2673480996 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5626699132b0 0x5626699156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:40.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.895+0000 7fef01ffb700 10 client.17040 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:25:40.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.895+0000 7fef19d9b700 1 --2- 172.21.15.19:0/2673480996 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5626699132b0 0x5626699156d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceb74faa0ec469ab server_cookie=e600c32160160899 in_seq=0 out_seq=0 2024-05-09T02:25:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.902+0000 7fef01ffb700 1 -- 172.21.15.19:0/2673480996 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562669823b20 con 0x5626699132b0 2024-05-09T02:25:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.902+0000 7fef01ffb700 10 client.17040 handle_command_reply: tid=0 2024-05-09T02:25:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.902+0000 7fef2f38ab80 1 client.17040 shutdown 2024-05-09T02:25:40.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.906+0000 7fef2f38ab80 1 -- 172.21.15.19:0/2673480996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562669903210 msgr2=0x562669903620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:40.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.906+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/2673480996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562669903210 0x562669903620 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x562669902550 tx=0x7fef14011fd0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.906+0000 7fef2f38ab80 1 -- 172.21.15.19:0/2673480996 shutdown_connections 2024-05-09T02:25:40.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.906+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/2673480996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562669903210 0x562669903620 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.906+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/2673480996 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x5626699132b0 0x5626699156d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.906+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/2673480996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56266977c4a0 0x562669905a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.906+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/2673480996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626698f95e0 0x562669902cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.906+0000 7fef2f38ab80 1 -- 172.21.15.19:0/2673480996 >> 172.21.15.19:0/2673480996 conn(0x56266977ff70 msgr2=0x5626698f42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:40.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.906+0000 7fef01ffb700 0 client.17040 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:25:40.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.912+0000 7fef2f38ab80 1 -- 172.21.15.19:0/2673480996 shutdown_connections 2024-05-09T02:25:40.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.912+0000 7fef2f38ab80 1 -- 172.21.15.19:0/2673480996 wait complete. 2024-05-09T02:25:40.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.912+0000 7fef2f38ab80 20 client.17040 trim_cache size 0 max 16384 2024-05-09T02:25:40.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.958+0000 7fef2f38ab80 1 Processor -- start 2024-05-09T02:25:40.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.959+0000 7fef2f38ab80 1 -- start start 2024-05-09T02:25:40.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.959+0000 7fef2f38ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56266977c4a0 0x56266990f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.959+0000 7fef2f38ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626698f95e0 0x5626698f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.959+0000 7fef2f38ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562669903210 0x5626699004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.959+0000 7fef2f38ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feefc00d810 con 0x5626698f95e0 2024-05-09T02:25:40.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.959+0000 7fef2f38ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7feefc00f350 con 0x562669903210 2024-05-09T02:25:40.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.959+0000 7fef2f38ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feefc00dee0 con 0x56266977c4a0 2024-05-09T02:25:40.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:40 smithi059 ceph-mon[34781]: pgmap v450: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.3 MiB/s wr, 614 op/s 2024-05-09T02:25:40.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.965+0000 7fef19d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626698f95e0 0x5626698f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:40.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.965+0000 7fef19d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626698f95e0 0x5626698f8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46006/0 (socket says 172.21.15.19:46006) 2024-05-09T02:25:40.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.965+0000 7fef19d9b700 1 -- 172.21.15.19:0/66139297 learned_addr learned my addr 172.21.15.19:0/66139297 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:40.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.967+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/66139297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56266977c4a0 0x56266990f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:40.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.967+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/66139297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562669903210 0x5626699004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:40.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.969+0000 7fef1ad9d700 1 -- 172.21.15.19:0/66139297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56266977c4a0 msgr2=0x56266990f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:40.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.969+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/66139297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56266977c4a0 0x56266990f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.969+0000 7fef1ad9d700 1 -- 172.21.15.19:0/66139297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626698f95e0 msgr2=0x5626698f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:40.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.969+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/66139297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626698f95e0 0x5626698f8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.969+0000 7fef1ad9d700 1 -- 172.21.15.19:0/66139297 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626697f6f40 con 0x562669903210 2024-05-09T02:25:40.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.972+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/66139297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562669903210 0x5626699004a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fef14002230 tx=0x7fef14008a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36e20f258b4735 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:40.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.974+0000 7fef03fff700 1 -- 172.21.15.19:0/66139297 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14019040 con 0x562669903210 2024-05-09T02:25:40.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.976+0000 7fef03fff700 1 -- 172.21.15.19:0/66139297 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef14005050 con 0x562669903210 2024-05-09T02:25:40.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.976+0000 7fef03fff700 1 -- 172.21.15.19:0/66139297 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14005e60 con 0x562669903210 2024-05-09T02:25:40.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.978+0000 7fef2f38ab80 1 -- 172.21.15.19:0/66139297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562669903210 msgr2=0x5626699004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:40.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.978+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/66139297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562669903210 0x5626699004a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fef14002230 tx=0x7fef14008a80 comp rx=0 tx=0).stop 2024-05-09T02:25:40.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.983+0000 7fef2f38ab80 1 -- 172.21.15.19:0/66139297 shutdown_connections 2024-05-09T02:25:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.983+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/66139297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562669903210 0x5626699004a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.983+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/66139297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56266977c4a0 0x56266990f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.983+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/66139297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626698f95e0 0x5626698f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.983+0000 7fef2f38ab80 1 -- 172.21.15.19:0/66139297 >> 172.21.15.19:0/66139297 conn(0x56266977ff70 msgr2=0x5626698f83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.985+0000 7fef2f38ab80 1 -- 172.21.15.19:0/66139297 shutdown_connections 2024-05-09T02:25:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.985+0000 7fef2f38ab80 1 -- 172.21.15.19:0/66139297 wait complete. 2024-05-09T02:25:40.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.986+0000 7fef2f38ab80 1 -- start start 2024-05-09T02:25:40.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.986+0000 7fef2f38ab80 1 Processor -- start 2024-05-09T02:25:40.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.987+0000 7fef2f38ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56266977c4a0 0x562669915f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.987+0000 7fef2f38ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626698f95e0 0x5626699164a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.987+0000 7fef2f38ab80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562669903210 0x5626699169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:40.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.987+0000 7fef2f38ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feefc00ec80 con 0x5626698f95e0 2024-05-09T02:25:40.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.987+0000 7fef2f38ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7feefc00e5b0 con 0x56266977c4a0 2024-05-09T02:25:40.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.987+0000 7fef2f38ab80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feefc00b7d0 con 0x562669903210 2024-05-09T02:25:40.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.987+0000 7fef2f38ab80 10 client.0 _ll_register_callbacks cb 0x56266968d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:25:40.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.987+0000 7fef1ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56266977c4a0 0x562669915f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:40.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.988+0000 7fef1ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56266977c4a0 0x562669915f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:52934/0 (socket says 172.21.15.19:52934) 2024-05-09T02:25:40.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.988+0000 7fef1ad9d700 1 -- 172.21.15.19:0/565614497 learned_addr learned my addr 172.21.15.19:0/565614497 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:40.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.988+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/565614497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562669903210 0x5626699169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:40.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.988+0000 7fef1ad9d700 1 -- 172.21.15.19:0/565614497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562669903210 msgr2=0x5626699169e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:40.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.988+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/565614497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562669903210 0x5626699169e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.988+0000 7fef1ad9d700 1 -- 172.21.15.19:0/565614497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626698f95e0 msgr2=0x5626699164a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:40.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.988+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/565614497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626698f95e0 0x5626699164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:40.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.989+0000 7fef1ad9d700 1 -- 172.21.15.19:0/565614497 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626698f09d0 con 0x56266977c4a0 2024-05-09T02:25:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.992+0000 7fef1ad9d700 1 --2- 172.21.15.19:0/565614497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56266977c4a0 0x562669915f60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fef14015e30 tx=0x7fef140094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d16998f191cd319a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.992+0000 7fef01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:25:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.993+0000 7fef01ffb700 1 -- 172.21.15.19:0/565614497 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14019070 con 0x56266977c4a0 2024-05-09T02:25:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.993+0000 7fef2f38ab80 1 -- 172.21.15.19:0/565614497 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef14005050 con 0x56266977c4a0 2024-05-09T02:25:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:40.993+0000 7fef01ffb700 1 -- 172.21.15.19:0/565614497 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fef14008c90 con 0x56266977c4a0 2024-05-09T02:25:41.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.000+0000 7fef01ffb700 1 -- 172.21.15.19:0/565614497 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fef14005c00 con 0x56266977c4a0 2024-05-09T02:25:41.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.000+0000 7fef2f38ab80 10 client.26765 fetch_fsmap learned FSMap version 58 2024-05-09T02:25:41.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.000+0000 7fef2f38ab80 1 -- 172.21.15.19:0/565614497 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56266997d8d0 con 0x56266977c4a0 2024-05-09T02:25:41.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.000+0000 7fef01ffb700 1 -- 172.21.15.19:0/565614497 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef14005050 con 0x56266977c4a0 2024-05-09T02:25:41.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.000+0000 7fef01ffb700 1 -- 172.21.15.19:0/565614497 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fef14008180 con 0x56266977c4a0 2024-05-09T02:25:41.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.000+0000 7fef2f38ab80 10 client.26765 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:25:41.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.000+0000 7fef2f38ab80 10 client.26765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:25:41.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.000+0000 7fef2f38ab80 20 client.26765 populate_metadata read hostname 'smithi019' 2024-05-09T02:25:41.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.000+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/565614497 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56266997dce0 0x562669980100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:41.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.001+0000 7fef2f38ab80 4 client.26765 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:25:41.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.001+0000 7fef2f38ab80 1 -- 172.21.15.19:0/565614497 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562669697970 con 0x56266997dce0 2024-05-09T02:25:41.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.006+0000 7fef19d9b700 1 --2- 172.21.15.19:0/565614497 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56266997dce0 0x562669980100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.017+0000 7fef01ffb700 10 client.26765 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:25:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.017+0000 7fef19d9b700 1 --2- 172.21.15.19:0/565614497 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56266997dce0 0x562669980100 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c86e720af44bf643 server_cookie=981367a9e3aa5cad in_seq=0 out_seq=0 2024-05-09T02:25:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.018+0000 7fef01ffb700 1 -- 172.21.15.19:0/565614497 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562669697970 con 0x56266997dce0 2024-05-09T02:25:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.018+0000 7fef01ffb700 10 client.26765 handle_command_reply: tid=0 2024-05-09T02:25:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.018+0000 7fef2f38ab80 1 client.26765 shutdown 2024-05-09T02:25:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.022+0000 7fef2f38ab80 1 -- 172.21.15.19:0/565614497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56266977c4a0 msgr2=0x562669915f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.022+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/565614497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56266977c4a0 0x562669915f60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fef14015e30 tx=0x7fef140094b0 comp rx=0 tx=0).stop 2024-05-09T02:25:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.022+0000 7fef2f38ab80 1 -- 172.21.15.19:0/565614497 shutdown_connections 2024-05-09T02:25:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.022+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/565614497 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56266997dce0 0x562669980100 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.022+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/565614497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562669903210 0x5626699169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.022+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/565614497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56266977c4a0 0x562669915f60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.022+0000 7fef2f38ab80 1 --2- 172.21.15.19:0/565614497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626698f95e0 0x5626699164a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.022+0000 7fef2f38ab80 1 -- 172.21.15.19:0/565614497 >> 172.21.15.19:0/565614497 conn(0x56266977ff70 msgr2=0x5626698f3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.022+0000 7fef01ffb700 0 client.26765 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:25:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.026+0000 7fef2f38ab80 1 -- 172.21.15.19:0/565614497 shutdown_connections 2024-05-09T02:25:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.026+0000 7fef2f38ab80 1 -- 172.21.15.19:0/565614497 wait complete. 2024-05-09T02:25:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.026+0000 7fef2f38ab80 20 client.26765 trim_cache size 0 max 16384 2024-05-09T02:25:41.029 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:25:41.029 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T02:25:41.029 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T02:25:41.029 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:25:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.030+0000 7fef1b59e700 1 -- 172.21.15.19:0/1997387376 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fef040b2eb0 msgr2=0x7fef040b53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.030+0000 7fef1b59e700 1 --2- 172.21.15.19:0/1997387376 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fef040b2eb0 0x7fef040b53c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0dfa70 tx=0x7fef0c00efb0 comp rx=0 tx=0).stop 2024-05-09T02:25:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.030+0000 7fef1b59e700 1 -- 172.21.15.19:0/1997387376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef1c075b60 msgr2=0x7fef1c0752d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.030+0000 7fef1b59e700 1 --2- 172.21.15.19:0/1997387376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef1c075b60 0x7fef1c0752d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00e9c0 tx=0x7fef0c00ecd0 comp rx=0 tx=0).stop 2024-05-09T02:25:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.030+0000 7fef1b59e700 1 -- 172.21.15.19:0/1997387376 shutdown_connections 2024-05-09T02:25:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.030+0000 7fef1b59e700 1 --2- 172.21.15.19:0/1997387376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef1c073900 0x7fef1c073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.030+0000 7fef1b59e700 1 --2- 172.21.15.19:0/1997387376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef1c075b60 0x7fef1c0752d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.030+0000 7fef1b59e700 1 --2- 172.21.15.19:0/1997387376 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fef040b2eb0 0x7fef040b53c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.030+0000 7fef1b59e700 1 --2- 172.21.15.19:0/1997387376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef1c115180 0x7fef1c0733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.030+0000 7fef1b59e700 1 -- 172.21.15.19:0/1997387376 >> 172.21.15.19:0/1997387376 conn(0x7fef1c079d10 msgr2=0x7fef1c102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.031+0000 7fef1b59e700 1 -- 172.21.15.19:0/1997387376 shutdown_connections 2024-05-09T02:25:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:41.032+0000 7fef1b59e700 1 -- 172.21.15.19:0/1997387376 wait complete. 2024-05-09T02:25:42.591 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T02:25:42.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:25:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 840.0 s, 1159.2 tps, lat 86.767 ms stddev 142.310 2024-05-09T02:25:42.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:42 smithi196 ceph-mon[34718]: pgmap v451: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.5 MiB/s wr, 651 op/s 2024-05-09T02:25:42.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:42 smithi019 ceph-mon[30151]: pgmap v451: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.5 MiB/s wr, 651 op/s 2024-05-09T02:25:42.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:42 smithi059 ceph-mon[34781]: pgmap v451: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.5 MiB/s wr, 651 op/s 2024-05-09T02:25:43.079 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:25:44.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:44 smithi196 ceph-mon[34718]: pgmap v452: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.2 MiB/s wr, 524 op/s 2024-05-09T02:25:44.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:44 smithi019 ceph-mon[30151]: pgmap v452: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.2 MiB/s wr, 524 op/s 2024-05-09T02:25:44.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:44 smithi059 ceph-mon[34781]: pgmap v452: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.2 MiB/s wr, 524 op/s 2024-05-09T02:25:46.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.342+0000 7f87466b9700 1 -- 172.21.15.19:0/1038803736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f874010b010 msgr2=0x7f874010b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:46.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.342+0000 7f87466b9700 1 --2- 172.21.15.19:0/1038803736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f874010b010 0x7f874010b420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f87300099c0 tx=0x7f8730009cd0 comp rx=0 tx=0).stop 2024-05-09T02:25:46.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.347+0000 7f87466b9700 1 -- 172.21.15.19:0/1038803736 shutdown_connections 2024-05-09T02:25:46.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.347+0000 7f87466b9700 1 --2- 172.21.15.19:0/1038803736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8740110290 0x7f8740117bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:46.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.347+0000 7f87466b9700 1 --2- 172.21.15.19:0/1038803736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f874010b010 0x7f874010b420 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:46.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.347+0000 7f87466b9700 1 --2- 172.21.15.19:0/1038803736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f874010f940 0x7f874010fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:46.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.347+0000 7f87466b9700 1 -- 172.21.15.19:0/1038803736 >> 172.21.15.19:0/1038803736 conn(0x7f874006f4d0 msgr2=0x7f874006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:46.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.353+0000 7f87466b9700 1 -- 172.21.15.19:0/1038803736 shutdown_connections 2024-05-09T02:25:46.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.353+0000 7f87466b9700 1 -- 172.21.15.19:0/1038803736 wait complete. 2024-05-09T02:25:46.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.355+0000 7f87466b9700 1 Processor -- start 2024-05-09T02:25:46.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.355+0000 7f87466b9700 1 -- start start 2024-05-09T02:25:46.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.356+0000 7f87466b9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f874010b010 0x7f87401a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:46.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.356+0000 7f87466b9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874010f940 0x7f87401a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:46.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.357+0000 7f873f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874010f940 0x7f87401a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:46.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.357+0000 7f873f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874010f940 0x7f87401a1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46024/0 (socket says 172.21.15.19:46024) 2024-05-09T02:25:46.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.357+0000 7f87466b9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8740110290 0x7f87401a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:46.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.357+0000 7f873f7fe700 1 -- 172.21.15.19:0/1894250060 learned_addr learned my addr 172.21.15.19:0/1894250060 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:46.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.357+0000 7f873ffff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f874010b010 0x7f87401a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:46.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.358+0000 7f8744c56700 1 --2- 172.21.15.19:0/1894250060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8740110290 0x7f87401a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:46.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.358+0000 7f87466b9700 1 -- 172.21.15.19:0/1894250060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f874011ac20 con 0x7f874010f940 2024-05-09T02:25:46.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.358+0000 7f87466b9700 1 -- 172.21.15.19:0/1894250060 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f874011aaa0 con 0x7f8740110290 2024-05-09T02:25:46.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.358+0000 7f87466b9700 1 -- 172.21.15.19:0/1894250060 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f874011ada0 con 0x7f874010b010 2024-05-09T02:25:46.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.359+0000 7f8744c56700 1 -- 172.21.15.19:0/1894250060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f874010b010 msgr2=0x7f87401a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:46.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.359+0000 7f8744c56700 1 --2- 172.21.15.19:0/1894250060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f874010b010 0x7f87401a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:46.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.359+0000 7f8744c56700 1 -- 172.21.15.19:0/1894250060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874010f940 msgr2=0x7f87401a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:46.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.359+0000 7f8744c56700 1 --2- 172.21.15.19:0/1894250060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874010f940 0x7f87401a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:46.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.359+0000 7f8744c56700 1 -- 172.21.15.19:0/1894250060 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87401a64e0 con 0x7f8740110290 2024-05-09T02:25:46.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.359+0000 7f873ffff700 1 --2- 172.21.15.19:0/1894250060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f874010b010 0x7f87401a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:25:46.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.359+0000 7f8744c56700 1 --2- 172.21.15.19:0/1894250060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8740110290 0x7f87401a5e40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f873800be00 tx=0x7f8738009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:46.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.360+0000 7f873d7fa700 1 -- 172.21.15.19:0/1894250060 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8738019070 con 0x7f8740110290 2024-05-09T02:25:46.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.360+0000 7f873d7fa700 1 -- 172.21.15.19:0/1894250060 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f873800cc10 con 0x7f8740110290 2024-05-09T02:25:46.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.360+0000 7f873d7fa700 1 -- 172.21.15.19:0/1894250060 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87380125a0 con 0x7f8740110290 2024-05-09T02:25:46.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.360+0000 7f87466b9700 1 -- 172.21.15.19:0/1894250060 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87401a67d0 con 0x7f8740110290 2024-05-09T02:25:46.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.361+0000 7f87466b9700 1 -- 172.21.15.19:0/1894250060 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87401a6c60 con 0x7f8740110290 2024-05-09T02:25:46.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.362+0000 7f87466b9700 1 -- 172.21.15.19:0/1894250060 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87400512c0 con 0x7f8740110290 2024-05-09T02:25:46.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.364+0000 7f873d7fa700 1 -- 172.21.15.19:0/1894250060 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f873801c070 con 0x7f8740110290 2024-05-09T02:25:46.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.366+0000 7f873d7fa700 1 --2- 172.21.15.19:0/1894250060 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f87280b4e10 0x7f87280b7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:46.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.366+0000 7f873d7fa700 1 -- 172.21.15.19:0/1894250060 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f87380e19e0 con 0x7f8740110290 2024-05-09T02:25:46.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.366+0000 7f873ffff700 1 --2- 172.21.15.19:0/1894250060 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f87280b4e10 0x7f87280b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.366+0000 7f873ffff700 1 --2- 172.21.15.19:0/1894250060 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f87280b4e10 0x7f87280b7320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8730000c00 tx=0x7f873001a040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:46.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.370+0000 7f873d7fa700 1 -- 172.21.15.19:0/1894250060 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87380a9440 con 0x7f8740110290 2024-05-09T02:25:46.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.790+0000 7f87466b9700 1 -- 172.21.15.19:0/1894250060 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f874002e5f0 con 0x7f8740110290 2024-05-09T02:25:46.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.791+0000 7f873d7fa700 1 -- 172.21.15.19:0/1894250060 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11917 (secure 0 0 0) 0x7f87380a8d40 con 0x7f8740110290 2024-05-09T02:25:46.794 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:25:46.794 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:22:01.074232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24457,24455,24473,24463,24484]},"id":1}]} 2024-05-09T02:25:46.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.795+0000 7f87466b9700 1 -- 172.21.15.19:0/1894250060 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f87280b4e10 msgr2=0x7f87280b7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:46.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.795+0000 7f87466b9700 1 --2- 172.21.15.19:0/1894250060 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f87280b4e10 0x7f87280b7320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8730000c00 tx=0x7f873001a040 comp rx=0 tx=0).stop 2024-05-09T02:25:46.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.795+0000 7f87466b9700 1 -- 172.21.15.19:0/1894250060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8740110290 msgr2=0x7f87401a5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:46.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.795+0000 7f87466b9700 1 --2- 172.21.15.19:0/1894250060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8740110290 0x7f87401a5e40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f873800be00 tx=0x7f8738009ed0 comp rx=0 tx=0).stop 2024-05-09T02:25:46.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.795+0000 7f87466b9700 1 -- 172.21.15.19:0/1894250060 shutdown_connections 2024-05-09T02:25:46.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.795+0000 7f87466b9700 1 --2- 172.21.15.19:0/1894250060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8740110290 0x7f87401a5e40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:46.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.795+0000 7f87466b9700 1 --2- 172.21.15.19:0/1894250060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f874010b010 0x7f87401a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:46.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.795+0000 7f87466b9700 1 --2- 172.21.15.19:0/1894250060 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f87280b4e10 0x7f87280b7320 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:46.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.795+0000 7f87466b9700 1 --2- 172.21.15.19:0/1894250060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874010f940 0x7f87401a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:46.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.795+0000 7f87466b9700 1 -- 172.21.15.19:0/1894250060 >> 172.21.15.19:0/1894250060 conn(0x7f874006f4d0 msgr2=0x7f874010da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:46.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:46 smithi019 ceph-mon[30151]: pgmap v453: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 18 MiB/s wr, 1.55k op/s 2024-05-09T02:25:46.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.795+0000 7f87466b9700 1 -- 172.21.15.19:0/1894250060 shutdown_connections 2024-05-09T02:25:46.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:46.795+0000 7f87466b9700 1 -- 172.21.15.19:0/1894250060 wait complete. 2024-05-09T02:25:46.799 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-05-09T02:25:46.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:46 smithi059 ceph-mon[34781]: pgmap v453: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 18 MiB/s wr, 1.55k op/s 2024-05-09T02:25:47.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:46 smithi196 ceph-mon[34718]: pgmap v453: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 18 MiB/s wr, 1.55k op/s 2024-05-09T02:25:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 845.0 s, 1535.2 tps, lat 65.426 ms stddev 73.761 2024-05-09T02:25:47.757 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:47 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1894250060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:25:47.812 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T02:25:47.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:47 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1894250060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:25:48.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:47 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1894250060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:25:48.205 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:25:48.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:48 smithi019 ceph-mon[30151]: pgmap v454: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 1.52k op/s 2024-05-09T02:25:48.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:48 smithi059 ceph-mon[34781]: pgmap v454: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 1.52k op/s 2024-05-09T02:25:49.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:48 smithi196 ceph-mon[34718]: pgmap v454: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 1.52k op/s 2024-05-09T02:25:50.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.618+0000 7fe002977700 1 -- 172.21.15.19:0/878445481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdffc105e50 msgr2=0x7fdffc10e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:50.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.618+0000 7fe002977700 1 --2- 172.21.15.19:0/878445481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdffc105e50 0x7fdffc10e3a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009990 tx=0x7fdfe4009ca0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:50 smithi019 ceph-mon[30151]: pgmap v455: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 1.52k op/s 2024-05-09T02:25:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.618+0000 7fe002977700 1 -- 172.21.15.19:0/878445481 shutdown_connections 2024-05-09T02:25:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.618+0000 7fe002977700 1 --2- 172.21.15.19:0/878445481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdffc10e8e0 0x7fdffc110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.618+0000 7fe002977700 1 --2- 172.21.15.19:0/878445481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdffc105500 0x7fdffc105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.618+0000 7fe002977700 1 --2- 172.21.15.19:0/878445481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdffc105e50 0x7fdffc10e3a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.618+0000 7fe002977700 1 -- 172.21.15.19:0/878445481 >> 172.21.15.19:0/878445481 conn(0x7fdffc0fedc0 msgr2=0x7fdffc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.618+0000 7fe002977700 1 -- 172.21.15.19:0/878445481 shutdown_connections 2024-05-09T02:25:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.618+0000 7fe002977700 1 -- 172.21.15.19:0/878445481 wait complete. 2024-05-09T02:25:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.619+0000 7fe002977700 1 Processor -- start 2024-05-09T02:25:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.619+0000 7fe002977700 1 -- start start 2024-05-09T02:25:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.619+0000 7fe002977700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdffc105500 0x7fdffc19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.619+0000 7fe002977700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdffc105e50 0x7fdffc19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.619+0000 7fe002977700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdffc10e8e0 0x7fdffc19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.619+0000 7fe002977700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc114440 con 0x7fdffc105500 2024-05-09T02:25:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.619+0000 7fe002977700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1142c0 con 0x7fdffc10e8e0 2024-05-09T02:25:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.619+0000 7fe002977700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1145c0 con 0x7fdffc105e50 2024-05-09T02:25:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fdffbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdffc105500 0x7fdffc19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.619+0000 7fdff3fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdffc105e50 0x7fdffc19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fe000f14700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdffc10e8e0 0x7fdffc19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fdffbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdffc105500 0x7fdffc19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40642/0 (socket says 172.21.15.19:40642) 2024-05-09T02:25:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fdff3fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdffc105e50 0x7fdffc19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:35304/0 (socket says 172.21.15.19:35304) 2024-05-09T02:25:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fdffbfff700 1 -- 172.21.15.19:0/361086513 learned_addr learned my addr 172.21.15.19:0/361086513 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fdffbfff700 1 -- 172.21.15.19:0/361086513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdffc105e50 msgr2=0x7fdffc19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fdffbfff700 1 --2- 172.21.15.19:0/361086513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdffc105e50 0x7fdffc19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fdffbfff700 1 -- 172.21.15.19:0/361086513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdffc10e8e0 msgr2=0x7fdffc19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fdffbfff700 1 --2- 172.21.15.19:0/361086513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdffc10e8e0 0x7fdffc19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fdffbfff700 1 -- 172.21.15.19:0/361086513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdffc19fc80 con 0x7fdffc105500 2024-05-09T02:25:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fdffbfff700 1 --2- 172.21.15.19:0/361086513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdffc105500 0x7fdffc19adb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00e9a0 tx=0x7fdfec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fdff9ffb700 1 -- 172.21.15.19:0/361086513 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec00ca10 con 0x7fdffc105500 2024-05-09T02:25:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fe002977700 1 -- 172.21.15.19:0/361086513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdffc19ff70 con 0x7fdffc105500 2024-05-09T02:25:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fdff9ffb700 1 -- 172.21.15.19:0/361086513 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfec00cbb0 con 0x7fdffc105500 2024-05-09T02:25:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fdff9ffb700 1 -- 172.21.15.19:0/361086513 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec018680 con 0x7fdffc105500 2024-05-09T02:25:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.620+0000 7fe002977700 1 -- 172.21.15.19:0/361086513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdffc1a7890 con 0x7fdffc105500 2024-05-09T02:25:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.626+0000 7fdff9ffb700 1 -- 172.21.15.19:0/361086513 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdfec010430 con 0x7fdffc105500 2024-05-09T02:25:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.626+0000 7fe01138db80 1 Processor -- start 2024-05-09T02:25:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.626+0000 7fe01138db80 1 -- start start 2024-05-09T02:25:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.626+0000 7fe01138db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3a83160 0x557ab3b1c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.626+0000 7fe01138db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 0x557ab3b1ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.626+0000 7fe01138db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3b166a0 0x557ab3b16ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fe01138db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557ab3a38ff0 con 0x557ab39994a0 2024-05-09T02:25:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fe01138db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557ab39d77f0 con 0x557ab3a83160 2024-05-09T02:25:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fe01138db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557ab3a34fb0 con 0x557ab3b166a0 2024-05-09T02:25:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fe000f14700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3a83160 0x557ab3b1c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fdff3fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3b166a0 0x557ab3b16ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fe000f14700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3a83160 0x557ab3b1c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:49150/0 (socket says 172.21.15.19:49150) 2024-05-09T02:25:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fe000f14700 1 -- 172.21.15.19:0/2509978289 learned_addr learned my addr 172.21.15.19:0/2509978289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fdffbfff700 1 --2- 172.21.15.19:0/2509978289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 0x557ab3b1ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fe000f14700 1 -- 172.21.15.19:0/2509978289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3b166a0 msgr2=0x557ab3b16ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fe000f14700 1 --2- 172.21.15.19:0/2509978289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3b166a0 0x557ab3b16ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fe000f14700 1 -- 172.21.15.19:0/2509978289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 msgr2=0x557ab3b1ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fe000f14700 1 --2- 172.21.15.19:0/2509978289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 0x557ab3b1ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fe000f14700 1 -- 172.21.15.19:0/2509978289 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ab39cf370 con 0x557ab3a83160 2024-05-09T02:25:50.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fdff3fff700 1 --2- 172.21.15.19:0/2509978289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3b166a0 0x557ab3b16ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:25:50.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fdffbfff700 1 --2- 172.21.15.19:0/2509978289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 0x557ab3b1ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:25:50.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.627+0000 7fdff9ffb700 1 --2- 172.21.15.19:0/361086513 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fdfe80b29f0 0x7fdfe80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.628+0000 7fe000f14700 1 --2- 172.21.15.19:0/2509978289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3a83160 0x557ab3b1c4f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdff4011680 tx=0x7fdff4009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0d40df5a0736c09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:50.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.628+0000 7fdffbfff700 1 --2- 172.21.15.19:0/361086513 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fdfe80b29f0 0x7fdfe80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.628+0000 7fdff9ffb700 1 -- 172.21.15.19:0/361086513 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fdfec0e27a0 con 0x7fdffc105500 2024-05-09T02:25:50.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.628+0000 7fdff27fc700 1 -- 172.21.15.19:0/2509978289 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4019030 con 0x557ab3a83160 2024-05-09T02:25:50.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.628+0000 7fdff27fc700 1 -- 172.21.15.19:0/2509978289 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdff40092e0 con 0x557ab3a83160 2024-05-09T02:25:50.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.628+0000 7fdff27fc700 1 -- 172.21.15.19:0/2509978289 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4007500 con 0x557ab3a83160 2024-05-09T02:25:50.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.629+0000 7fe01138db80 1 -- 172.21.15.19:0/2509978289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3a83160 msgr2=0x557ab3b1c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:50.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.629+0000 7fdffbfff700 1 --2- 172.21.15.19:0/361086513 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fdfe80b29f0 0x7fdfe80b4f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0dff70 tx=0x7fdfec009790 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:50.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.629+0000 7fe01138db80 1 --2- 172.21.15.19:0/2509978289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3a83160 0x557ab3b1c4f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdff4011680 tx=0x7fdff4009920 comp rx=0 tx=0).stop 2024-05-09T02:25:50.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.629+0000 7fe01138db80 1 -- 172.21.15.19:0/2509978289 shutdown_connections 2024-05-09T02:25:50.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.629+0000 7fe01138db80 1 --2- 172.21.15.19:0/2509978289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3b166a0 0x557ab3b16ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.629+0000 7fe01138db80 1 --2- 172.21.15.19:0/2509978289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3a83160 0x557ab3b1c4f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.629+0000 7fe01138db80 1 --2- 172.21.15.19:0/2509978289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 0x557ab3b1ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.629+0000 7fe01138db80 1 -- 172.21.15.19:0/2509978289 >> 172.21.15.19:0/2509978289 conn(0x557ab39f96a0 msgr2=0x557ab3b11400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.629+0000 7fe01138db80 1 -- 172.21.15.19:0/2509978289 shutdown_connections 2024-05-09T02:25:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.629+0000 7fe01138db80 1 -- 172.21.15.19:0/2509978289 wait complete. 2024-05-09T02:25:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.629+0000 7fe01138db80 1 -- start start 2024-05-09T02:25:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.629+0000 7fe01138db80 1 Processor -- start 2024-05-09T02:25:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe01138db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab39994a0 0x557ab3b20280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe01138db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3a83160 0x557ab3b207c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe01138db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab3b166a0 0x557ab3b20d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe01138db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557ab38b4ba0 con 0x557ab3b166a0 2024-05-09T02:25:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe01138db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557ab39f3150 con 0x557ab39994a0 2024-05-09T02:25:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe01138db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557ab3b275e0 con 0x557ab3a83160 2024-05-09T02:25:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe01138db80 10 client.0 _ll_register_callbacks cb 0x557ab38aa6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:25:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe000f14700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab3b166a0 0x557ab3b20d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe000f14700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab3b166a0 0x557ab3b20d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40658/0 (socket says 172.21.15.19:40658) 2024-05-09T02:25:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe000f14700 1 -- 172.21.15.19:0/394940968 learned_addr learned my addr 172.21.15.19:0/394940968 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fdff3fff700 1 --2- 172.21.15.19:0/394940968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3a83160 0x557ab3b207c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe000f14700 1 --2- 172.21.15.19:0/394940968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab39994a0 0x557ab3b20280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe000f14700 1 -- 172.21.15.19:0/394940968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3a83160 msgr2=0x557ab3b207c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe000f14700 1 --2- 172.21.15.19:0/394940968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3a83160 0x557ab3b207c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe000f14700 1 -- 172.21.15.19:0/394940968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab39994a0 msgr2=0x557ab3b20280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe000f14700 1 --2- 172.21.15.19:0/394940968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab39994a0 0x557ab3b20280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fe000f14700 1 -- 172.21.15.19:0/394940968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ab38b4490 con 0x557ab3b166a0 2024-05-09T02:25:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.630+0000 7fdff3fff700 1 --2- 172.21.15.19:0/394940968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3a83160 0x557ab3b207c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:25:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.631+0000 7fe000f14700 1 --2- 172.21.15.19:0/394940968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab3b166a0 0x557ab3b20d00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x557ab3b1fb00 tx=0x7fdff4009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30364079b2e1931 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.631+0000 7fdfdb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:25:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.631+0000 7fdfdb7fe700 1 -- 172.21.15.19:0/394940968 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4003e20 con 0x557ab3b166a0 2024-05-09T02:25:50.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.631+0000 7fdfdb7fe700 1 -- 172.21.15.19:0/394940968 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fdff4003fc0 con 0x557ab3b166a0 2024-05-09T02:25:50.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.631+0000 7fe01138db80 1 -- 172.21.15.19:0/394940968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdff40092e0 con 0x557ab3b166a0 2024-05-09T02:25:50.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.631+0000 7fdfdb7fe700 1 -- 172.21.15.19:0/394940968 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fdff4007500 con 0x557ab3b166a0 2024-05-09T02:25:50.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.631+0000 7fe01138db80 10 client.17058 fetch_fsmap learned FSMap version 58 2024-05-09T02:25:50.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.631+0000 7fe01138db80 1 -- 172.21.15.19:0/394940968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557ab3b32070 con 0x557ab3b166a0 2024-05-09T02:25:50.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.632+0000 7fdfdb7fe700 1 -- 172.21.15.19:0/394940968 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdff40092e0 con 0x557ab3b166a0 2024-05-09T02:25:50.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.633+0000 7fdfdb7fe700 1 -- 172.21.15.19:0/394940968 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fdff4015740 con 0x557ab3b166a0 2024-05-09T02:25:50.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.633+0000 7fe01138db80 10 client.17058 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:25:50.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.633+0000 7fe01138db80 10 client.17058 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:25:50.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.633+0000 7fe01138db80 20 client.17058 populate_metadata read hostname 'smithi019' 2024-05-09T02:25:50.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.633+0000 7fe01138db80 1 --2- 172.21.15.19:0/394940968 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x557ab3b348b0 0x557ab3b36d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.633+0000 7fe01138db80 4 client.17058 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:25:50.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.633+0000 7fe01138db80 1 -- 172.21.15.19:0/394940968 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ab3a469d0 con 0x557ab3b348b0 2024-05-09T02:25:50.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.634+0000 7fdff3fff700 1 --2- 172.21.15.19:0/394940968 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x557ab3b348b0 0x557ab3b36d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.634+0000 7fdff3fff700 1 --2- 172.21.15.19:0/394940968 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x557ab3b348b0 0x557ab3b36d10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2e16361eb01df2e3 server_cookie=2c9945dc530eb849 in_seq=0 out_seq=0 2024-05-09T02:25:50.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.634+0000 7fdfdb7fe700 10 client.17058 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:25:50.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.635+0000 7fdfdb7fe700 1 -- 172.21.15.19:0/394940968 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557ab3a469d0 con 0x557ab3b348b0 2024-05-09T02:25:50.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.635+0000 7fdfdb7fe700 10 client.17058 handle_command_reply: tid=0 2024-05-09T02:25:50.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.635+0000 7fe01138db80 1 client.17058 shutdown 2024-05-09T02:25:50.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.635+0000 7fe01138db80 1 -- 172.21.15.19:0/394940968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab3b166a0 msgr2=0x557ab3b20d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:50.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.635+0000 7fe01138db80 1 --2- 172.21.15.19:0/394940968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab3b166a0 0x557ab3b20d00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x557ab3b1fb00 tx=0x7fdff4009ac0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.635+0000 7fe01138db80 1 -- 172.21.15.19:0/394940968 shutdown_connections 2024-05-09T02:25:50.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.635+0000 7fe01138db80 1 --2- 172.21.15.19:0/394940968 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x557ab3b348b0 0x557ab3b36d10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.635+0000 7fe01138db80 1 --2- 172.21.15.19:0/394940968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab3b166a0 0x557ab3b20d00 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.635+0000 7fe01138db80 1 --2- 172.21.15.19:0/394940968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab39994a0 0x557ab3b20280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.635+0000 7fe01138db80 1 --2- 172.21.15.19:0/394940968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3a83160 0x557ab3b207c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.635+0000 7fe01138db80 1 -- 172.21.15.19:0/394940968 >> 172.21.15.19:0/394940968 conn(0x557ab39f96a0 msgr2=0x557ab3b111c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:50.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.635+0000 7fdfdb7fe700 0 client.17058 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:25:50.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.636+0000 7fe01138db80 1 -- 172.21.15.19:0/394940968 shutdown_connections 2024-05-09T02:25:50.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.636+0000 7fe01138db80 1 -- 172.21.15.19:0/394940968 wait complete. 2024-05-09T02:25:50.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.636+0000 7fe01138db80 20 client.17058 trim_cache size 0 max 16384 2024-05-09T02:25:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.661+0000 7fe01138db80 1 Processor -- start 2024-05-09T02:25:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.662+0000 7fe01138db80 1 -- start start 2024-05-09T02:25:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.662+0000 7fe01138db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 0x557ab3b1df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.662+0000 7fe01138db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3a83160 0x557ab3b1f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.662+0000 7fe000f14700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 0x557ab3b1df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.662+0000 7fe000f14700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 0x557ab3b1df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40668/0 (socket says 172.21.15.19:40668) 2024-05-09T02:25:50.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.662+0000 7fe000f14700 1 -- 172.21.15.19:0/3855127863 learned_addr learned my addr 172.21.15.19:0/3855127863 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:50.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.662+0000 7fe01138db80 1 --2- 172.21.15.19:0/3855127863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3b166a0 0x557ab3b27000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.662+0000 7fe01138db80 1 -- 172.21.15.19:0/3855127863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1acdb0 con 0x557ab39994a0 2024-05-09T02:25:50.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.662+0000 7fe01138db80 1 -- 172.21.15.19:0/3855127863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1ae8f0 con 0x557ab3b166a0 2024-05-09T02:25:50.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.662+0000 7fe01138db80 1 -- 172.21.15.19:0/3855127863 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1ad480 con 0x557ab3a83160 2024-05-09T02:25:50.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.665+0000 7fdff3fff700 1 --2- 172.21.15.19:0/3855127863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3a83160 0x557ab3b1f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.666+0000 7fe000f14700 1 --2- 172.21.15.19:0/3855127863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3b166a0 0x557ab3b27000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.672+0000 7fdff3fff700 1 -- 172.21.15.19:0/3855127863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3b166a0 msgr2=0x557ab3b27000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.672+0000 7fdff3fff700 1 --2- 172.21.15.19:0/3855127863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3b166a0 0x557ab3b27000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.672+0000 7fdff3fff700 1 -- 172.21.15.19:0/3855127863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 msgr2=0x557ab3b1df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.672+0000 7fdff3fff700 1 --2- 172.21.15.19:0/3855127863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 0x557ab3b1df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.672+0000 7fdff3fff700 1 -- 172.21.15.19:0/3855127863 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ab3b0af60 con 0x557ab3a83160 2024-05-09T02:25:50.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.673+0000 7fe000f14700 1 --2- 172.21.15.19:0/3855127863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3b166a0 0x557ab3b27000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:25:50.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.676+0000 7fdff3fff700 1 --2- 172.21.15.19:0/3855127863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3a83160 0x557ab3b1f5a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40059d0 tx=0x7fdfe4005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac456f3dd4019fd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:50.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.678+0000 7fdff1ffb700 1 -- 172.21.15.19:0/3855127863 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400bd00 con 0x557ab3a83160 2024-05-09T02:25:50.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.678+0000 7fdff1ffb700 1 -- 172.21.15.19:0/3855127863 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfe402d070 con 0x557ab3a83160 2024-05-09T02:25:50.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.678+0000 7fdff1ffb700 1 -- 172.21.15.19:0/3855127863 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40284c0 con 0x557ab3a83160 2024-05-09T02:25:50.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.680+0000 7fe01138db80 1 -- 172.21.15.19:0/3855127863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3a83160 msgr2=0x557ab3b1f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:50.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.680+0000 7fe01138db80 1 --2- 172.21.15.19:0/3855127863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3a83160 0x557ab3b1f5a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40059d0 tx=0x7fdfe4005040 comp rx=0 tx=0).stop 2024-05-09T02:25:50.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.683+0000 7fe01138db80 1 -- 172.21.15.19:0/3855127863 shutdown_connections 2024-05-09T02:25:50.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.683+0000 7fe01138db80 1 --2- 172.21.15.19:0/3855127863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3b166a0 0x557ab3b27000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.683+0000 7fe01138db80 1 --2- 172.21.15.19:0/3855127863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 0x557ab3b1df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.683+0000 7fe01138db80 1 --2- 172.21.15.19:0/3855127863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3a83160 0x557ab3b1f5a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.683+0000 7fe01138db80 1 -- 172.21.15.19:0/3855127863 >> 172.21.15.19:0/3855127863 conn(0x557ab39f96a0 msgr2=0x557ab3b19e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:50.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.686+0000 7fe01138db80 1 -- 172.21.15.19:0/3855127863 shutdown_connections 2024-05-09T02:25:50.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.688+0000 7fe01138db80 1 -- 172.21.15.19:0/3855127863 wait complete. 2024-05-09T02:25:50.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.689+0000 7fe01138db80 1 -- start start 2024-05-09T02:25:50.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.689+0000 7fe01138db80 1 Processor -- start 2024-05-09T02:25:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.691+0000 7fe01138db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 0x557ab3b309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.691+0000 7fe01138db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3a83160 0x557ab3b2c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.691+0000 7fe000f14700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 0x557ab3b309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.691+0000 7fe01138db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3b166a0 0x557ab3b2cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.691+0000 7fe01138db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1ae220 con 0x557ab39994a0 2024-05-09T02:25:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.691+0000 7fe01138db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1adb50 con 0x557ab3a83160 2024-05-09T02:25:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.691+0000 7fe01138db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1aad70 con 0x557ab3b166a0 2024-05-09T02:25:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.691+0000 7fdff3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3a83160 0x557ab3b2c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.691+0000 7fdff3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3a83160 0x557ab3b2c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:49180/0 (socket says 172.21.15.19:49180) 2024-05-09T02:25:50.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.691+0000 7fdff3fff700 1 -- 172.21.15.19:0/4156411135 learned_addr learned my addr 172.21.15.19:0/4156411135 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:50.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.691+0000 7fe01138db80 10 client.0 _ll_register_callbacks cb 0x557ab38aa6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:25:50.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.693+0000 7fe000f14700 1 --2- 172.21.15.19:0/4156411135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3b166a0 0x557ab3b2cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.695+0000 7fe000f14700 1 -- 172.21.15.19:0/4156411135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3a83160 msgr2=0x557ab3b2c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.695+0000 7fe000f14700 1 --2- 172.21.15.19:0/4156411135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3a83160 0x557ab3b2c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.695+0000 7fe000f14700 1 -- 172.21.15.19:0/4156411135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 msgr2=0x557ab3b309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.695+0000 7fe000f14700 1 --2- 172.21.15.19:0/4156411135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 0x557ab3b309c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.695+0000 7fe000f14700 1 -- 172.21.15.19:0/4156411135 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ab3b3c110 con 0x557ab3b166a0 2024-05-09T02:25:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.699+0000 7fe000f14700 1 --2- 172.21.15.19:0/4156411135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3b166a0 0x557ab3b2cb10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x557ab3b31290 tx=0x7fdff4009ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fec2cee9544b314 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.699+0000 7fdfdb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:25:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.700+0000 7fdfdb7fe700 1 -- 172.21.15.19:0/4156411135 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4019040 con 0x557ab3b166a0 2024-05-09T02:25:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.700+0000 7fdfdb7fe700 1 -- 172.21.15.19:0/4156411135 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fdff4005c60 con 0x557ab3b166a0 2024-05-09T02:25:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.701+0000 7fe01138db80 1 -- 172.21.15.19:0/4156411135 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfe402d070 con 0x557ab3b166a0 2024-05-09T02:25:50.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.703+0000 7fdfdb7fe700 1 -- 172.21.15.19:0/4156411135 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fdff4005e60 con 0x557ab3b166a0 2024-05-09T02:25:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.706+0000 7fe01138db80 10 client.26848 fetch_fsmap learned FSMap version 58 2024-05-09T02:25:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.706+0000 7fe01138db80 1 -- 172.21.15.19:0/4156411135 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557ab3b992e0 con 0x557ab3b166a0 2024-05-09T02:25:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.707+0000 7fdfdb7fe700 1 -- 172.21.15.19:0/4156411135 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdff4036070 con 0x557ab3b166a0 2024-05-09T02:25:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.707+0000 7fdfdb7fe700 1 -- 172.21.15.19:0/4156411135 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fdff403a030 con 0x557ab3b166a0 2024-05-09T02:25:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.708+0000 7fe01138db80 10 client.26848 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:25:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.708+0000 7fe01138db80 10 client.26848 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:25:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.708+0000 7fe01138db80 20 client.26848 populate_metadata read hostname 'smithi019' 2024-05-09T02:25:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.708+0000 7fe01138db80 1 --2- 172.21.15.19:0/4156411135 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x557ab3b99970 0x557ab3b9bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.708+0000 7fe01138db80 4 client.26848 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:25:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.708+0000 7fe01138db80 1 -- 172.21.15.19:0/4156411135 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fdfe402d070 con 0x557ab3b99970 2024-05-09T02:25:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.708+0000 7fdff3fff700 1 --2- 172.21.15.19:0/4156411135 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x557ab3b99970 0x557ab3b9bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:50.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.714+0000 7fdff3fff700 1 --2- 172.21.15.19:0/4156411135 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x557ab3b99970 0x557ab3b9bd90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=70418997fc5c4954 server_cookie=c3ef10af78a7fab3 in_seq=0 out_seq=0 2024-05-09T02:25:50.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.714+0000 7fdfdb7fe700 10 client.26848 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:25:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.718+0000 7fdfdb7fe700 1 -- 172.21.15.19:0/4156411135 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fdfe402d070 con 0x557ab3b99970 2024-05-09T02:25:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.718+0000 7fdfdb7fe700 10 client.26848 handle_command_reply: tid=0 2024-05-09T02:25:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.719+0000 7fe01138db80 1 client.26848 shutdown 2024-05-09T02:25:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.721+0000 7fe01138db80 1 -- 172.21.15.19:0/4156411135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3b166a0 msgr2=0x557ab3b2cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.721+0000 7fe01138db80 1 --2- 172.21.15.19:0/4156411135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3b166a0 0x557ab3b2cb10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x557ab3b31290 tx=0x7fdff4009ac0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.723+0000 7fe01138db80 1 -- 172.21.15.19:0/4156411135 shutdown_connections 2024-05-09T02:25:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.723+0000 7fe01138db80 1 --2- 172.21.15.19:0/4156411135 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x557ab3b99970 0x557ab3b9bd90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.723+0000 7fe01138db80 1 --2- 172.21.15.19:0/4156411135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ab3b166a0 0x557ab3b2cb10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.723+0000 7fe01138db80 1 --2- 172.21.15.19:0/4156411135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557ab39994a0 0x557ab3b309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.723+0000 7fe01138db80 1 --2- 172.21.15.19:0/4156411135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab3a83160 0x557ab3b2c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.724+0000 7fe01138db80 1 -- 172.21.15.19:0/4156411135 >> 172.21.15.19:0/4156411135 conn(0x557ab39f96a0 msgr2=0x557ab3b19e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.724+0000 7fdfdb7fe700 0 client.26848 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:25:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.724+0000 7fe01138db80 1 -- 172.21.15.19:0/4156411135 shutdown_connections 2024-05-09T02:25:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.725+0000 7fe01138db80 1 -- 172.21.15.19:0/4156411135 wait complete. 2024-05-09T02:25:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.725+0000 7fe01138db80 20 client.26848 trim_cache size 0 max 16384 2024-05-09T02:25:50.728 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:25:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.727+0000 7fe002977700 1 -- 172.21.15.19:0/361086513 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fdfe80b29f0 msgr2=0x7fdfe80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.727+0000 7fe002977700 1 --2- 172.21.15.19:0/361086513 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fdfe80b29f0 0x7fdfe80b4f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0dff70 tx=0x7fdfec009790 comp rx=0 tx=0).stop 2024-05-09T02:25:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.727+0000 7fe002977700 1 -- 172.21.15.19:0/361086513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdffc105500 msgr2=0x7fdffc19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.727+0000 7fe002977700 1 --2- 172.21.15.19:0/361086513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdffc105500 0x7fdffc19adb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00e9a0 tx=0x7fdfec00ecb0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.727+0000 7fe002977700 1 -- 172.21.15.19:0/361086513 shutdown_connections 2024-05-09T02:25:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.727+0000 7fe002977700 1 --2- 172.21.15.19:0/361086513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdffc10e8e0 0x7fdffc19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.727+0000 7fe002977700 1 --2- 172.21.15.19:0/361086513 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fdfe80b29f0 0x7fdfe80b4f00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.727+0000 7fe002977700 1 --2- 172.21.15.19:0/361086513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdffc105500 0x7fdffc19adb0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.727+0000 7fe002977700 1 --2- 172.21.15.19:0/361086513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdffc105e50 0x7fdffc19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.727+0000 7fe002977700 1 -- 172.21.15.19:0/361086513 >> 172.21.15.19:0/361086513 conn(0x7fdffc0fedc0 msgr2=0x7fdffc0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.727+0000 7fe002977700 1 -- 172.21.15.19:0/361086513 shutdown_connections 2024-05-09T02:25:50.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:50.727+0000 7fe002977700 1 -- 172.21.15.19:0/361086513 wait complete. 2024-05-09T02:25:50.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:50 smithi059 ceph-mon[34781]: pgmap v455: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 1.52k op/s 2024-05-09T02:25:51.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:50 smithi196 ceph-mon[34718]: pgmap v455: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 1.52k op/s 2024-05-09T02:25:51.821 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T02:25:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:25:52.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:52 smithi019 ceph-mon[30151]: pgmap v456: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 19 MiB/s wr, 1.71k op/s 2024-05-09T02:25:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 850.0 s, 1333.6 tps, lat 71.013 ms stddev 78.705 2024-05-09T02:25:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:52 smithi059 ceph-mon[34781]: pgmap v456: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 19 MiB/s wr, 1.71k op/s 2024-05-09T02:25:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:52 smithi196 ceph-mon[34718]: pgmap v456: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 19 MiB/s wr, 1.71k op/s 2024-05-09T02:25:53.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:53 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:25:53.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:53 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:25:54.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:53 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:25:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.271+0000 7f11e7447700 1 -- 172.21.15.19:0/1036184371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e01054e0 msgr2=0x7f11e01058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.271+0000 7f11e7447700 1 --2- 172.21.15.19:0/1036184371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e01054e0 0x7f11e01058f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009990 tx=0x7f11dc009ca0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.271+0000 7f11e7447700 1 -- 172.21.15.19:0/1036184371 shutdown_connections 2024-05-09T02:25:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.271+0000 7f11e7447700 1 --2- 172.21.15.19:0/1036184371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e010e910 0x7f11e0110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.271+0000 7f11e7447700 1 --2- 172.21.15.19:0/1036184371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e01054e0 0x7f11e01058f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.271+0000 7f11e7447700 1 --2- 172.21.15.19:0/1036184371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e0105e30 0x7f11e010e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.271+0000 7f11e7447700 1 -- 172.21.15.19:0/1036184371 >> 172.21.15.19:0/1036184371 conn(0x7f11e00fef30 msgr2=0x7f11e0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.271+0000 7f11e7447700 1 -- 172.21.15.19:0/1036184371 shutdown_connections 2024-05-09T02:25:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.271+0000 7f11e7447700 1 -- 172.21.15.19:0/1036184371 wait complete. 2024-05-09T02:25:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.271+0000 7f11e7447700 1 Processor -- start 2024-05-09T02:25:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.271+0000 7f11e7447700 1 -- start start 2024-05-09T02:25:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e7447700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e01054e0 0x7f11e01031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e7447700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e0105e30 0x7f11e0103710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e7447700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e010e910 0x7f11e0103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e7447700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0114460 con 0x7f11e010e910 2024-05-09T02:25:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e7447700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01142e0 con 0x7f11e01054e0 2024-05-09T02:25:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e7447700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01145e0 con 0x7f11e0105e30 2024-05-09T02:25:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e59e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e010e910 0x7f11e0103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e51e3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e01054e0 0x7f11e01031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e59e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e010e910 0x7f11e0103c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40710/0 (socket says 172.21.15.19:40710) 2024-05-09T02:25:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e59e4700 1 -- 172.21.15.19:0/2033365556 learned_addr learned my addr 172.21.15.19:0/2033365556 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e49e2700 1 --2- 172.21.15.19:0/2033365556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e0105e30 0x7f11e0103710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e51e3700 1 -- 172.21.15.19:0/2033365556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e0105e30 msgr2=0x7f11e0103710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e51e3700 1 --2- 172.21.15.19:0/2033365556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e0105e30 0x7f11e0103710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e51e3700 1 -- 172.21.15.19:0/2033365556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e010e910 msgr2=0x7f11e0103c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e51e3700 1 --2- 172.21.15.19:0/2033365556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e010e910 0x7f11e0103c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.272+0000 7f11e51e3700 1 -- 172.21.15.19:0/2033365556 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e01a91e0 con 0x7f11e01054e0 2024-05-09T02:25:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.273+0000 7f11e51e3700 1 --2- 172.21.15.19:0/2033365556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e01054e0 0x7f11e01031d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0059d0 tx=0x7f11dc0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.273+0000 7f11d67fc700 1 -- 172.21.15.19:0/2033365556 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc027070 con 0x7f11e01054e0 2024-05-09T02:25:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.273+0000 7f11d67fc700 1 -- 172.21.15.19:0/2033365556 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11dc022470 con 0x7f11e01054e0 2024-05-09T02:25:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.273+0000 7f11d67fc700 1 -- 172.21.15.19:0/2033365556 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc017650 con 0x7f11e01054e0 2024-05-09T02:25:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.273+0000 7f11e7447700 1 -- 172.21.15.19:0/2033365556 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e01a9470 con 0x7f11e01054e0 2024-05-09T02:25:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.273+0000 7f11e7447700 1 -- 172.21.15.19:0/2033365556 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e01a98d0 con 0x7f11e01054e0 2024-05-09T02:25:54.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.276+0000 7f11d67fc700 1 -- 172.21.15.19:0/2033365556 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f11dc02a070 con 0x7f11e01054e0 2024-05-09T02:25:54.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.277+0000 7f11d67fc700 1 --2- 172.21.15.19:0/2033365556 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f11cc0b2a70 0x7f11cc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.277+0000 7f11e49e2700 1 --2- 172.21.15.19:0/2033365556 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f11cc0b2a70 0x7f11cc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.277+0000 7f11d67fc700 1 -- 172.21.15.19:0/2033365556 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f11dc0e3360 con 0x7f11e01054e0 2024-05-09T02:25:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.278+0000 7f11f5e5db80 1 Processor -- start 2024-05-09T02:25:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.278+0000 7f11f5e5db80 1 -- start start 2024-05-09T02:25:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.278+0000 7f11e49e2700 1 --2- 172.21.15.19:0/2033365556 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f11cc0b2a70 0x7f11cc0b4f80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f11e0102450 tx=0x7f11d0009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.278+0000 7f11f5e5db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356d6f160 0x55c356e084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.278+0000 7f11f5e5db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e08a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.278+0000 7f11f5e5db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356e026a0 0x55c356e02ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.278+0000 7f11f5e5db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c356d24ff0 con 0x55c356c854a0 2024-05-09T02:25:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.278+0000 7f11f5e5db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c356cc37f0 con 0x55c356e026a0 2024-05-09T02:25:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.278+0000 7f11f5e5db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c356d20fb0 con 0x55c356d6f160 2024-05-09T02:25:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.278+0000 7f11e49e2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e08a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.278+0000 7f11e49e2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e08a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40720/0 (socket says 172.21.15.19:40720) 2024-05-09T02:25:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.278+0000 7f11e49e2700 1 -- 172.21.15.19:0/2533617156 learned_addr learned my addr 172.21.15.19:0/2533617156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.278+0000 7f11e51e3700 1 --2- 172.21.15.19:0/2533617156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356d6f160 0x55c356e084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.278+0000 7f11e59e4700 1 --2- 172.21.15.19:0/2533617156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356e026a0 0x55c356e02ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.279+0000 7f11e51e3700 1 -- 172.21.15.19:0/2533617156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356e026a0 msgr2=0x55c356e02ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.279+0000 7f11e51e3700 1 --2- 172.21.15.19:0/2533617156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356e026a0 0x55c356e02ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.279+0000 7f11e51e3700 1 -- 172.21.15.19:0/2533617156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 msgr2=0x55c356e08a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.279+0000 7f11e51e3700 1 --2- 172.21.15.19:0/2533617156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e08a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.279+0000 7f11e51e3700 1 -- 172.21.15.19:0/2533617156 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c356cbb370 con 0x55c356d6f160 2024-05-09T02:25:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.279+0000 7f11e51e3700 1 --2- 172.21.15.19:0/2533617156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356d6f160 0x55c356e084f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0e0a20 tx=0x7f11dc0a9f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e1cb89306c24caf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.279+0000 7f11cb7fe700 1 -- 172.21.15.19:0/2533617156 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0e2030 con 0x55c356d6f160 2024-05-09T02:25:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.279+0000 7f11cb7fe700 1 -- 172.21.15.19:0/2533617156 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11dc0aa5b0 con 0x55c356d6f160 2024-05-09T02:25:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.279+0000 7f11cb7fe700 1 -- 172.21.15.19:0/2533617156 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc022940 con 0x55c356d6f160 2024-05-09T02:25:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.280+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2533617156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356d6f160 msgr2=0x55c356e084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.280+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2533617156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356d6f160 0x55c356e084f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0e0a20 tx=0x7f11dc0a9f80 comp rx=0 tx=0).stop 2024-05-09T02:25:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.280+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2533617156 shutdown_connections 2024-05-09T02:25:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.280+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2533617156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356e026a0 0x55c356e02ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.280+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2533617156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356d6f160 0x55c356e084f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.280+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2533617156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e08a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.280+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2533617156 >> 172.21.15.19:0/2533617156 conn(0x55c356ce56a0 msgr2=0x55c356dfd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.280+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2533617156 shutdown_connections 2024-05-09T02:25:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.280+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2533617156 wait complete. 2024-05-09T02:25:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.281+0000 7f11f5e5db80 1 -- start start 2024-05-09T02:25:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.281+0000 7f11f5e5db80 1 Processor -- start 2024-05-09T02:25:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.281+0000 7f11f5e5db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e11d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.281+0000 7f11f5e5db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356d6f160 0x55c356e12290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.281+0000 7f11f5e5db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 0x55c356e0c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.281+0000 7f11f5e5db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c356ba0ba0 con 0x55c356c854a0 2024-05-09T02:25:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.281+0000 7f11f5e5db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c356cdf150 con 0x55c356d6f160 2024-05-09T02:25:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.281+0000 7f11f5e5db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c356e03360 con 0x55c356e026a0 2024-05-09T02:25:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.281+0000 7f11f5e5db80 10 client.0 _ll_register_callbacks cb 0x55c356b966c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:25:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.281+0000 7f11e51e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e11d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.281+0000 7f11e51e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e11d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40730/0 (socket says 172.21.15.19:40730) 2024-05-09T02:25:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.281+0000 7f11e51e3700 1 -- 172.21.15.19:0/2076687809 learned_addr learned my addr 172.21.15.19:0/2076687809 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.282+0000 7f11e59e4700 1 --2- 172.21.15.19:0/2076687809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 0x55c356e0c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.282+0000 7f11e49e2700 1 --2- 172.21.15.19:0/2076687809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356d6f160 0x55c356e12290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.282+0000 7f11e51e3700 1 -- 172.21.15.19:0/2076687809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 msgr2=0x55c356e0c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.282+0000 7f11e51e3700 1 --2- 172.21.15.19:0/2076687809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 0x55c356e0c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.282+0000 7f11e51e3700 1 -- 172.21.15.19:0/2076687809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356d6f160 msgr2=0x55c356e12290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.282+0000 7f11e51e3700 1 --2- 172.21.15.19:0/2076687809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356d6f160 0x55c356e12290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.282+0000 7f11e51e3700 1 -- 172.21.15.19:0/2076687809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c356ba0490 con 0x55c356c854a0 2024-05-09T02:25:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.282+0000 7f11e51e3700 1 --2- 172.21.15.19:0/2076687809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e11d50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0e0fa0 tx=0x7f11dc02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b7a70f4b94fb00 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.282+0000 7f11c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:25:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.282+0000 7f11c8ff9700 1 -- 172.21.15.19:0/2076687809 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0e2070 con 0x55c356c854a0 2024-05-09T02:25:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.282+0000 7f11c8ff9700 1 -- 172.21.15.19:0/2076687809 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f11dc0aaa20 con 0x55c356c854a0 2024-05-09T02:25:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.282+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2076687809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11dc0aa5b0 con 0x55c356c854a0 2024-05-09T02:25:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.282+0000 7f11c8ff9700 1 -- 172.21.15.19:0/2076687809 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f11dc0ae960 con 0x55c356c854a0 2024-05-09T02:25:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.282+0000 7f11f5e5db80 10 client.17076 fetch_fsmap learned FSMap version 58 2024-05-09T02:25:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.282+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2076687809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55c356e19970 con 0x55c356c854a0 2024-05-09T02:25:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.283+0000 7f11c8ff9700 1 -- 172.21.15.19:0/2076687809 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11dc0aa5b0 con 0x55c356c854a0 2024-05-09T02:25:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.283+0000 7f11c8ff9700 1 -- 172.21.15.19:0/2076687809 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f11dc0b1ce0 con 0x55c356c854a0 2024-05-09T02:25:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.283+0000 7f11f5e5db80 10 client.17076 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:25:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.283+0000 7f11f5e5db80 10 client.17076 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:25:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.283+0000 7f11f5e5db80 20 client.17076 populate_metadata read hostname 'smithi019' 2024-05-09T02:25:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.283+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2076687809 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55c356e1be40 0x55c356e1e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.283+0000 7f11f5e5db80 4 client.17076 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:25:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.283+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2076687809 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c356d329d0 con 0x55c356e1be40 2024-05-09T02:25:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.283+0000 7f11e49e2700 1 --2- 172.21.15.19:0/2076687809 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55c356e1be40 0x55c356e1e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.284+0000 7f11c8ff9700 10 client.17076 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:25:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.284+0000 7f11e49e2700 1 --2- 172.21.15.19:0/2076687809 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55c356e1be40 0x55c356e1e260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7b0e4fb9e51accf server_cookie=9ac3c93eec98a5b in_seq=0 out_seq=0 2024-05-09T02:25:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.284+0000 7f11c8ff9700 1 -- 172.21.15.19:0/2076687809 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c356d329d0 con 0x55c356e1be40 2024-05-09T02:25:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.284+0000 7f11c8ff9700 10 client.17076 handle_command_reply: tid=0 2024-05-09T02:25:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.285+0000 7f11f5e5db80 1 client.17076 shutdown 2024-05-09T02:25:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.285+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2076687809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 msgr2=0x55c356e11d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.285+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2076687809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e11d50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0e0fa0 tx=0x7f11dc02f000 comp rx=0 tx=0).stop 2024-05-09T02:25:54.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.285+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2076687809 shutdown_connections 2024-05-09T02:25:54.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.285+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2076687809 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55c356e1be40 0x55c356e1e260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.285+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2076687809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 0x55c356e0c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.285+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2076687809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e11d50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.285+0000 7f11c8ff9700 0 client.17076 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:25:54.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.285+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2076687809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356d6f160 0x55c356e12290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.285+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2076687809 >> 172.21.15.19:0/2076687809 conn(0x55c356ce56a0 msgr2=0x55c356dfd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:54.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.285+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2076687809 shutdown_connections 2024-05-09T02:25:54.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.285+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2076687809 wait complete. 2024-05-09T02:25:54.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.285+0000 7f11f5e5db80 20 client.17076 trim_cache size 0 max 16384 2024-05-09T02:25:54.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.300+0000 7f11f5e5db80 1 Processor -- start 2024-05-09T02:25:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.300+0000 7f11f5e5db80 1 -- start start 2024-05-09T02:25:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.300+0000 7f11f5e5db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e06860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.300+0000 7f11f5e5db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356d6f160 0x55c356e0c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.300+0000 7f11f5e5db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 0x55c356e04e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.300+0000 7f11f5e5db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01b10b0 con 0x55c356c854a0 2024-05-09T02:25:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.300+0000 7f11f5e5db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01b2bf0 con 0x55c356d6f160 2024-05-09T02:25:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.300+0000 7f11f5e5db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01b1780 con 0x55c356e026a0 2024-05-09T02:25:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.300+0000 7f11e51e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e06860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.301+0000 7f11e51e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e06860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40736/0 (socket says 172.21.15.19:40736) 2024-05-09T02:25:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.301+0000 7f11e51e3700 1 -- 172.21.15.19:0/4086502593 learned_addr learned my addr 172.21.15.19:0/4086502593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.301+0000 7f11e59e4700 1 --2- 172.21.15.19:0/4086502593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 0x55c356e04e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.301+0000 7f11e49e2700 1 --2- 172.21.15.19:0/4086502593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356d6f160 0x55c356e0c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.301+0000 7f11e51e3700 1 -- 172.21.15.19:0/4086502593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 msgr2=0x55c356e04e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.301+0000 7f11e51e3700 1 --2- 172.21.15.19:0/4086502593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 0x55c356e04e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.301+0000 7f11e51e3700 1 -- 172.21.15.19:0/4086502593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356d6f160 msgr2=0x55c356e0c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.301+0000 7f11e51e3700 1 --2- 172.21.15.19:0/4086502593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356d6f160 0x55c356e0c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.301+0000 7f11e51e3700 1 -- 172.21.15.19:0/4086502593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c356dfed00 con 0x55c356c854a0 2024-05-09T02:25:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.301+0000 7f11e59e4700 1 --2- 172.21.15.19:0/4086502593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 0x55c356e04e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:25:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.301+0000 7f11e51e3700 1 --2- 172.21.15.19:0/4086502593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e06860 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f11dc000c00 tx=0x7f11dc022ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4849cfe2aaba1b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.301+0000 7f11caffd700 1 -- 172.21.15.19:0/4086502593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0e2040 con 0x55c356c854a0 2024-05-09T02:25:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.301+0000 7f11caffd700 1 -- 172.21.15.19:0/4086502593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11dc0b3410 con 0x55c356c854a0 2024-05-09T02:25:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.301+0000 7f11caffd700 1 -- 172.21.15.19:0/4086502593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0b2f80 con 0x55c356c854a0 2024-05-09T02:25:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.302+0000 7f11f5e5db80 1 -- 172.21.15.19:0/4086502593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 msgr2=0x55c356e06860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.302+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/4086502593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e06860 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f11dc000c00 tx=0x7f11dc022ce0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.302+0000 7f11f5e5db80 1 -- 172.21.15.19:0/4086502593 shutdown_connections 2024-05-09T02:25:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.302+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/4086502593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 0x55c356e04e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.302+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/4086502593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356c854a0 0x55c356e06860 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.302+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/4086502593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356d6f160 0x55c356e0c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.302+0000 7f11f5e5db80 1 -- 172.21.15.19:0/4086502593 >> 172.21.15.19:0/4086502593 conn(0x55c356ce56a0 msgr2=0x55c356e0ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.302+0000 7f11f5e5db80 1 -- 172.21.15.19:0/4086502593 shutdown_connections 2024-05-09T02:25:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.302+0000 7f11f5e5db80 1 -- 172.21.15.19:0/4086502593 wait complete. 2024-05-09T02:25:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.303+0000 7f11f5e5db80 1 -- start start 2024-05-09T02:25:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.303+0000 7f11f5e5db80 1 Processor -- start 2024-05-09T02:25:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.303+0000 7f11f5e5db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356c854a0 0x55c356e18540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.303+0000 7f11f5e5db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356d6f160 0x55c356e138a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.303+0000 7f11f5e5db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 0x55c356e13de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.303+0000 7f11f5e5db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01b2520 con 0x55c356d6f160 2024-05-09T02:25:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.303+0000 7f11f5e5db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01b1e50 con 0x55c356c854a0 2024-05-09T02:25:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.303+0000 7f11f5e5db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01af070 con 0x55c356e026a0 2024-05-09T02:25:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.303+0000 7f11f5e5db80 10 client.0 _ll_register_callbacks cb 0x55c356b966c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:25:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.303+0000 7f11e49e2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356d6f160 0x55c356e138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.303+0000 7f11e49e2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356d6f160 0x55c356e138a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40742/0 (socket says 172.21.15.19:40742) 2024-05-09T02:25:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.303+0000 7f11e49e2700 1 -- 172.21.15.19:0/2411506444 learned_addr learned my addr 172.21.15.19:0/2411506444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.303+0000 7f11e51e3700 1 --2- 172.21.15.19:0/2411506444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356c854a0 0x55c356e18540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.303+0000 7f11e59e4700 1 --2- 172.21.15.19:0/2411506444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 0x55c356e13de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.304+0000 7f11e49e2700 1 -- 172.21.15.19:0/2411506444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 msgr2=0x55c356e13de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.304+0000 7f11e49e2700 1 --2- 172.21.15.19:0/2411506444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 0x55c356e13de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.304+0000 7f11e49e2700 1 -- 172.21.15.19:0/2411506444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356c854a0 msgr2=0x55c356e18540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.304+0000 7f11e49e2700 1 --2- 172.21.15.19:0/2411506444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356c854a0 0x55c356e18540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.304+0000 7f11e49e2700 1 -- 172.21.15.19:0/2411506444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c356e143d0 con 0x55c356d6f160 2024-05-09T02:25:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.304+0000 7f11e51e3700 1 --2- 172.21.15.19:0/2411506444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356c854a0 0x55c356e18540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:25:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.304+0000 7f11e49e2700 1 --2- 172.21.15.19:0/2411506444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356d6f160 0x55c356e138a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f11d0011fd0 tx=0x7f11d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e6d0ba678b92c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.304+0000 7f11c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:25:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.304+0000 7f11c8ff9700 1 -- 172.21.15.19:0/2411506444 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d001dc90 con 0x55c356d6f160 2024-05-09T02:25:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.304+0000 7f11c8ff9700 1 -- 172.21.15.19:0/2411506444 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f11d0004030 con 0x55c356d6f160 2024-05-09T02:25:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.304+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2411506444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11dc0b3410 con 0x55c356d6f160 2024-05-09T02:25:54.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.304+0000 7f11c8ff9700 1 -- 172.21.15.19:0/2411506444 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f11d003a050 con 0x55c356d6f160 2024-05-09T02:25:54.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.304+0000 7f11f5e5db80 10 client.17082 fetch_fsmap learned FSMap version 58 2024-05-09T02:25:54.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.304+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2411506444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55c356e862c0 con 0x55c356d6f160 2024-05-09T02:25:54.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.304+0000 7f11c8ff9700 1 -- 172.21.15.19:0/2411506444 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11dc0b3410 con 0x55c356d6f160 2024-05-09T02:25:54.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.305+0000 7f11c8ff9700 1 -- 172.21.15.19:0/2411506444 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f11d0003680 con 0x55c356d6f160 2024-05-09T02:25:54.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.305+0000 7f11f5e5db80 10 client.17082 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:25:54.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.305+0000 7f11f5e5db80 10 client.17082 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:25:54.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.305+0000 7f11f5e5db80 20 client.17082 populate_metadata read hostname 'smithi019' 2024-05-09T02:25:54.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.305+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2411506444 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55c356e86a00 0x55c356e88e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:54.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.305+0000 7f11f5e5db80 4 client.17082 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:25:54.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.305+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2411506444 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c356d2ca70 con 0x55c356e86a00 2024-05-09T02:25:54.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.305+0000 7f11e51e3700 1 --2- 172.21.15.19:0/2411506444 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55c356e86a00 0x55c356e88e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:54.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.305+0000 7f11c8ff9700 10 client.17082 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:25:54.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.305+0000 7f11e51e3700 1 --2- 172.21.15.19:0/2411506444 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55c356e86a00 0x55c356e88e20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3a3fa01a65529635 server_cookie=7129ae9b0a678620 in_seq=0 out_seq=0 2024-05-09T02:25:54.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11c8ff9700 1 -- 172.21.15.19:0/2411506444 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c356d2ca70 con 0x55c356e86a00 2024-05-09T02:25:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11c8ff9700 10 client.17082 handle_command_reply: tid=0 2024-05-09T02:25:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11f5e5db80 1 client.17082 shutdown 2024-05-09T02:25:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2411506444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356d6f160 msgr2=0x55c356e138a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2411506444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356d6f160 0x55c356e138a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f11d0011fd0 tx=0x7f11d00047c0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2411506444 shutdown_connections 2024-05-09T02:25:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2411506444 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x55c356e86a00 0x55c356e88e20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11c8ff9700 0 client.17082 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:25:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2411506444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c356e026a0 0x55c356e13de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2411506444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c356c854a0 0x55c356e18540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11f5e5db80 1 --2- 172.21.15.19:0/2411506444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c356d6f160 0x55c356e138a0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2411506444 >> 172.21.15.19:0/2411506444 conn(0x55c356ce56a0 msgr2=0x55c356e07240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2411506444 shutdown_connections 2024-05-09T02:25:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11f5e5db80 1 -- 172.21.15.19:0/2411506444 wait complete. 2024-05-09T02:25:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11f5e5db80 20 client.17082 trim_cache size 0 max 16384 2024-05-09T02:25:54.315 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:25:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11e7447700 1 -- 172.21.15.19:0/2033365556 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f11cc0b2a70 msgr2=0x7f11cc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.306+0000 7f11e7447700 1 --2- 172.21.15.19:0/2033365556 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f11cc0b2a70 0x7f11cc0b4f80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f11e0102450 tx=0x7f11d0009260 comp rx=0 tx=0).stop 2024-05-09T02:25:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.307+0000 7f11e7447700 1 -- 172.21.15.19:0/2033365556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e01054e0 msgr2=0x7f11e01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.307+0000 7f11e7447700 1 --2- 172.21.15.19:0/2033365556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e01054e0 0x7f11e01031d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0059d0 tx=0x7f11dc0047b0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.307+0000 7f11e7447700 1 -- 172.21.15.19:0/2033365556 shutdown_connections 2024-05-09T02:25:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.307+0000 7f11e7447700 1 --2- 172.21.15.19:0/2033365556 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f11cc0b2a70 0x7f11cc0b4f80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.307+0000 7f11e7447700 1 --2- 172.21.15.19:0/2033365556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e010e910 0x7f11e0103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.307+0000 7f11e7447700 1 --2- 172.21.15.19:0/2033365556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e01054e0 0x7f11e01031d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.307+0000 7f11e7447700 1 --2- 172.21.15.19:0/2033365556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e0105e30 0x7f11e0103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.307+0000 7f11e7447700 1 -- 172.21.15.19:0/2033365556 >> 172.21.15.19:0/2033365556 conn(0x7f11e00fef30 msgr2=0x7f11e00ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.307+0000 7f11e7447700 1 -- 172.21.15.19:0/2033365556 shutdown_connections 2024-05-09T02:25:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:54.307+0000 7f11e7447700 1 -- 172.21.15.19:0/2033365556 wait complete. 2024-05-09T02:25:54.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:54 smithi059 ceph-mon[34781]: pgmap v457: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 18 MiB/s wr, 1.53k op/s 2024-05-09T02:25:54.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:54 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:25:54.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:54 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:25:54.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:54 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:25:55.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:54 smithi196 ceph-mon[34718]: pgmap v457: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 18 MiB/s wr, 1.53k op/s 2024-05-09T02:25:55.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:54 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:25:55.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:54 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:25:55.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:54 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:25:55.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:54 smithi019 ceph-mon[30151]: pgmap v457: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 18 MiB/s wr, 1.53k op/s 2024-05-09T02:25:55.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:54 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:25:55.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:54 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:25:55.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:54 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:25:55.248 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T02:25:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:25:56.900 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:56 smithi019 ceph-mon[30151]: pgmap v458: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 1.67k op/s 2024-05-09T02:25:56.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:56 smithi059 ceph-mon[34781]: pgmap v458: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 1.67k op/s 2024-05-09T02:25:57.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:56 smithi196 ceph-mon[34718]: pgmap v458: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 1.67k op/s 2024-05-09T02:25:57.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.399+0000 7f2e45465700 1 -- 172.21.15.19:0/2638122937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e40107e10 msgr2=0x7f2e40108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:57.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.399+0000 7f2e45465700 1 --2- 172.21.15.19:0/2638122937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e40107e10 0x7f2e40108220 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2e340099c0 tx=0x7f2e34009cd0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.400+0000 7f2e45465700 1 -- 172.21.15.19:0/2638122937 shutdown_connections 2024-05-09T02:25:57.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.400+0000 7f2e45465700 1 --2- 172.21.15.19:0/2638122937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e40108760 0x7f2e401100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.400+0000 7f2e45465700 1 --2- 172.21.15.19:0/2638122937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e401034e0 0x7f2e401038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.400+0000 7f2e45465700 1 --2- 172.21.15.19:0/2638122937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e40107e10 0x7f2e40108220 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.400+0000 7f2e45465700 1 -- 172.21.15.19:0/2638122937 >> 172.21.15.19:0/2638122937 conn(0x7f2e400fef30 msgr2=0x7f2e40101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:57.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.401+0000 7f2e45465700 1 -- 172.21.15.19:0/2638122937 shutdown_connections 2024-05-09T02:25:57.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.401+0000 7f2e45465700 1 -- 172.21.15.19:0/2638122937 wait complete. 2024-05-09T02:25:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.402+0000 7f2e45465700 1 Processor -- start 2024-05-09T02:25:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.402+0000 7f2e45465700 1 -- start start 2024-05-09T02:25:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.402+0000 7f2e45465700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e401034e0 0x7f2e40075240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.402+0000 7f2e45465700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e40107e10 0x7f2e4006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.402+0000 7f2e45465700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e40108760 0x7f2e4006d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.403+0000 7f2e45465700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40112300 con 0x7f2e401034e0 2024-05-09T02:25:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.403+0000 7f2e45465700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40112180 con 0x7f2e40108760 2024-05-09T02:25:57.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.403+0000 7f2e45465700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40112480 con 0x7f2e40107e10 2024-05-09T02:25:57.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.403+0000 7f2e3f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e40108760 0x7f2e4006d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:57.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.403+0000 7f2e3f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e40108760 0x7f2e4006d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:36676/0 (socket says 172.21.15.19:36676) 2024-05-09T02:25:57.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.403+0000 7f2e3f7fe700 1 -- 172.21.15.19:0/3152318115 learned_addr learned my addr 172.21.15.19:0/3152318115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:57.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.403+0000 7f2e3effd700 1 --2- 172.21.15.19:0/3152318115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e401034e0 0x7f2e40075240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:57.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.406+0000 7f2e3e7fc700 1 --2- 172.21.15.19:0/3152318115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e40107e10 0x7f2e4006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.407+0000 7f2e3effd700 1 -- 172.21.15.19:0/3152318115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e40107e10 msgr2=0x7f2e4006d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.407+0000 7f2e3effd700 1 --2- 172.21.15.19:0/3152318115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e40107e10 0x7f2e4006d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.407+0000 7f2e3effd700 1 -- 172.21.15.19:0/3152318115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e40108760 msgr2=0x7f2e4006d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.407+0000 7f2e3effd700 1 --2- 172.21.15.19:0/3152318115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e40108760 0x7f2e4006d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.407+0000 7f2e3effd700 1 -- 172.21.15.19:0/3152318115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4006def0 con 0x7f2e401034e0 2024-05-09T02:25:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.412+0000 7f2e3effd700 1 --2- 172.21.15.19:0/3152318115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e401034e0 0x7f2e40075240 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000be10 tx=0x7f2e3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.415+0000 7f2e27fff700 1 -- 172.21.15.19:0/3152318115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30018070 con 0x7f2e401034e0 2024-05-09T02:25:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.415+0000 7f2e27fff700 1 -- 172.21.15.19:0/3152318115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e30012e10 con 0x7f2e401034e0 2024-05-09T02:25:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.415+0000 7f2e27fff700 1 -- 172.21.15.19:0/3152318115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3001c660 con 0x7f2e401034e0 2024-05-09T02:25:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.416+0000 7f2e45465700 1 -- 172.21.15.19:0/3152318115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e401987b0 con 0x7f2e401034e0 2024-05-09T02:25:57.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.416+0000 7f2e45465700 1 -- 172.21.15.19:0/3152318115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e40198ca0 con 0x7f2e401034e0 2024-05-09T02:25:57.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.421+0000 7f2e53e7bb80 1 Processor -- start 2024-05-09T02:25:57.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.421+0000 7f2e53e7bb80 1 -- start start 2024-05-09T02:25:57.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.422+0000 7f2e53e7bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba5905160 0x556ba599e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.422+0000 7f2e53e7bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba581b4a0 0x556ba599ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.422+0000 7f2e53e7bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5998880 0x556ba5998c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.422+0000 7f2e3f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba5905160 0x556ba599e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:57.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.422+0000 7f2e3f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba5905160 0x556ba599e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:33048/0 (socket says 172.21.15.19:33048) 2024-05-09T02:25:57.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.422+0000 7f2e3f7fe700 1 -- 172.21.15.19:0/3043560145 learned_addr learned my addr 172.21.15.19:0/3043560145 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:57.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.423+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3043560145 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556ba58baff0 con 0x556ba5998880 2024-05-09T02:25:57.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.423+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3043560145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556ba58597f0 con 0x556ba581b4a0 2024-05-09T02:25:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.423+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3043560145 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556ba58b6fb0 con 0x556ba5905160 2024-05-09T02:25:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.423+0000 7f2e3f7fe700 1 -- 172.21.15.19:0/3043560145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba581b4a0 msgr2=0x556ba599ec00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:25:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.423+0000 7f2e3f7fe700 1 --2- 172.21.15.19:0/3043560145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba581b4a0 0x556ba599ec00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.423+0000 7f2e3f7fe700 1 -- 172.21.15.19:0/3043560145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5998880 msgr2=0x556ba5998c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.423+0000 7f2e3f7fe700 1 --2- 172.21.15.19:0/3043560145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5998880 0x556ba5998c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.423+0000 7f2e3f7fe700 1 -- 172.21.15.19:0/3043560145 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba5851370 con 0x556ba5905160 2024-05-09T02:25:57.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.424+0000 7f2e3f7fe700 1 --2- 172.21.15.19:0/3043560145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba5905160 0x556ba599e660 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009ef0 tx=0x7f2e38009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fd23aff26f6e32f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:57.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.426+0000 7f2e27fff700 1 -- 172.21.15.19:0/3152318115 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2e30009850 con 0x7f2e401034e0 2024-05-09T02:25:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.427+0000 7f2e24ff9700 1 -- 172.21.15.19:0/3043560145 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3803a030 con 0x556ba5905160 2024-05-09T02:25:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.427+0000 7f2e24ff9700 1 -- 172.21.15.19:0/3043560145 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e38007e40 con 0x556ba5905160 2024-05-09T02:25:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.427+0000 7f2e27fff700 1 --2- 172.21.15.19:0/3152318115 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2e280b2a60 0x7f2e280b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.427+0000 7f2e27fff700 1 -- 172.21.15.19:0/3152318115 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f2e300e5fb0 con 0x7f2e401034e0 2024-05-09T02:25:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.428+0000 7f2e3effd700 1 --2- 172.21.15.19:0/3152318115 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2e280b2a60 0x7f2e280b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.428+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3043560145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba5905160 msgr2=0x556ba599e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.428+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/3043560145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba5905160 0x556ba599e660 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009ef0 tx=0x7f2e38009230 comp rx=0 tx=0).stop 2024-05-09T02:25:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.428+0000 7f2e3effd700 1 --2- 172.21.15.19:0/3152318115 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2e280b2a60 0x7f2e280b4f70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x556ba5994e40 tx=0x7f2e30000bc0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:57.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.429+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3043560145 shutdown_connections 2024-05-09T02:25:57.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.429+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/3043560145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5998880 0x556ba5998c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.429+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/3043560145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba5905160 0x556ba599e660 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.429+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/3043560145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba581b4a0 0x556ba599ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.429+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3043560145 >> 172.21.15.19:0/3043560145 conn(0x556ba587b6a0 msgr2=0x556ba5993a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:57.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.432+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3043560145 shutdown_connections 2024-05-09T02:25:57.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.433+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3043560145 wait complete. 2024-05-09T02:25:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.433+0000 7f2e53e7bb80 1 -- start start 2024-05-09T02:25:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.433+0000 7f2e53e7bb80 1 Processor -- start 2024-05-09T02:25:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.434+0000 7f2e53e7bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba59a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.434+0000 7f2e53e7bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5905160 0x556ba59a2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.434+0000 7f2e53e7bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5998880 0x556ba59a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.434+0000 7f2e3f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba59a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.434+0000 7f2e3f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba59a4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:33052/0 (socket says 172.21.15.19:33052) 2024-05-09T02:25:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.434+0000 7f2e3e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5905160 0x556ba59a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.434+0000 7f2e3e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5905160 0x556ba59a2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:36690/0 (socket says 172.21.15.19:36690) 2024-05-09T02:25:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.434+0000 7f2e3f7fe700 1 -- 172.21.15.19:0/3588506396 learned_addr learned my addr 172.21.15.19:0/3588506396 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.434+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3588506396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556ba5736ba0 con 0x556ba5998880 2024-05-09T02:25:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.434+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3588506396 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556ba5875150 con 0x556ba5905160 2024-05-09T02:25:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.434+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3588506396 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556ba59996c0 con 0x556ba581b4a0 2024-05-09T02:25:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.435+0000 7f2e53e7bb80 10 client.0 _ll_register_callbacks cb 0x556ba572c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:25:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.436+0000 7f2e3f7fe700 1 -- 172.21.15.19:0/3588506396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5905160 msgr2=0x556ba59a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.436+0000 7f2e3f7fe700 1 --2- 172.21.15.19:0/3588506396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5905160 0x556ba59a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.436+0000 7f2e3f7fe700 1 -- 172.21.15.19:0/3588506396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5998880 msgr2=0x556ba59a2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.436+0000 7f2e3f7fe700 1 --2- 172.21.15.19:0/3588506396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5998880 0x556ba59a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.436+0000 7f2e3f7fe700 1 -- 172.21.15.19:0/3588506396 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba5736490 con 0x556ba581b4a0 2024-05-09T02:25:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.436+0000 7f2e3f7fe700 1 --2- 172.21.15.19:0/3588506396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba59a4c80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e38036830 tx=0x7f2e38036860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eabea7f46b126d1a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.437+0000 7f2e22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:25:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.440+0000 7f2e22ff5700 1 -- 172.21.15.19:0/3588506396 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38005440 con 0x556ba581b4a0 2024-05-09T02:25:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.440+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3588506396 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556ba58c89d0 con 0x556ba581b4a0 2024-05-09T02:25:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.440+0000 7f2e22ff5700 1 -- 172.21.15.19:0/3588506396 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f2e380055e0 con 0x556ba581b4a0 2024-05-09T02:25:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.441+0000 7f2e22ff5700 1 -- 172.21.15.19:0/3588506396 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f2e380057e0 con 0x556ba581b4a0 2024-05-09T02:25:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.442+0000 7f2e53e7bb80 10 client.26878 fetch_fsmap learned FSMap version 58 2024-05-09T02:25:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.442+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3588506396 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556ba59b8130 con 0x556ba581b4a0 2024-05-09T02:25:57.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.445+0000 7f2e22ff5700 1 -- 172.21.15.19:0/3588506396 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556ba58c89d0 con 0x556ba581b4a0 2024-05-09T02:25:57.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.445+0000 7f2e22ff5700 1 -- 172.21.15.19:0/3588506396 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f2e38051070 con 0x556ba581b4a0 2024-05-09T02:25:57.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.445+0000 7f2e53e7bb80 10 client.26878 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:25:57.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.446+0000 7f2e53e7bb80 10 client.26878 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:25:57.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.446+0000 7f2e53e7bb80 20 client.26878 populate_metadata read hostname 'smithi019' 2024-05-09T02:25:57.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.446+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/3588506396 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556ba59ba650 0x556ba59bca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.446+0000 7f2e53e7bb80 4 client.26878 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:25:57.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.446+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3588506396 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ba58c2a70 con 0x556ba59ba650 2024-05-09T02:25:57.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.446+0000 7f2e3e7fc700 1 --2- 172.21.15.19:0/3588506396 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556ba59ba650 0x556ba59bca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:57.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.447+0000 7f2e3e7fc700 1 --2- 172.21.15.19:0/3588506396 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556ba59ba650 0x556ba59bca70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91e9c856cf1acf5d server_cookie=cb6f376ab784e572 in_seq=0 out_seq=0 2024-05-09T02:25:57.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.447+0000 7f2e22ff5700 10 client.26878 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:25:57.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.449+0000 7f2e22ff5700 1 -- 172.21.15.19:0/3588506396 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556ba58c2a70 con 0x556ba59ba650 2024-05-09T02:25:57.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.450+0000 7f2e22ff5700 10 client.26878 handle_command_reply: tid=0 2024-05-09T02:25:57.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.450+0000 7f2e53e7bb80 1 client.26878 shutdown 2024-05-09T02:25:57.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.453+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3588506396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 msgr2=0x556ba59a4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:57.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.453+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/3588506396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba59a4c80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e38036830 tx=0x7f2e38036860 comp rx=0 tx=0).stop 2024-05-09T02:25:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.456+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3588506396 shutdown_connections 2024-05-09T02:25:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.456+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/3588506396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5998880 0x556ba59a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.457+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/3588506396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba59a4c80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.457+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/3588506396 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556ba59ba650 0x556ba59bca70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.457+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/3588506396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5905160 0x556ba59a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.457+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3588506396 >> 172.21.15.19:0/3588506396 conn(0x556ba587b6a0 msgr2=0x556ba5993550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.457+0000 7f2e22ff5700 0 client.26878 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:25:57.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.461+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3588506396 shutdown_connections 2024-05-09T02:25:57.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.461+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/3588506396 wait complete. 2024-05-09T02:25:57.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.462+0000 7f2e53e7bb80 20 client.26878 trim_cache size 0 max 16384 2024-05-09T02:25:57.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.489+0000 7f2e53e7bb80 1 Processor -- start 2024-05-09T02:25:57.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.489+0000 7f2e53e7bb80 1 -- start start 2024-05-09T02:25:57.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.489+0000 7f2e53e7bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba59a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.490+0000 7f2e53e7bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5905160 0x556ba5993a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.490+0000 7f2e53e7bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5998880 0x556ba599fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.490+0000 7f2e53e7bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401a0690 con 0x556ba5998880 2024-05-09T02:25:57.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.490+0000 7f2e53e7bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401a21d0 con 0x556ba5905160 2024-05-09T02:25:57.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.490+0000 7f2e53e7bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401a0d60 con 0x556ba581b4a0 2024-05-09T02:25:57.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.490+0000 7f2e3f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba59a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:57.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.490+0000 7f2e3f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba59a2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:33062/0 (socket says 172.21.15.19:33062) 2024-05-09T02:25:57.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.490+0000 7f2e3f7fe700 1 -- 172.21.15.19:0/1033484300 learned_addr learned my addr 172.21.15.19:0/1033484300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:57.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.490+0000 7f2e3f7fe700 1 --2- 172.21.15.19:0/1033484300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5998880 0x556ba599fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:57.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.490+0000 7f2e3e7fc700 1 --2- 172.21.15.19:0/1033484300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5905160 0x556ba5993a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:57.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.492+0000 7f2e3e7fc700 1 -- 172.21.15.19:0/1033484300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 msgr2=0x556ba59a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:57.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.492+0000 7f2e3e7fc700 1 --2- 172.21.15.19:0/1033484300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba59a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.492+0000 7f2e3e7fc700 1 -- 172.21.15.19:0/1033484300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5998880 msgr2=0x556ba599fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:57.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.492+0000 7f2e3e7fc700 1 --2- 172.21.15.19:0/1033484300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5998880 0x556ba599fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.492+0000 7f2e3e7fc700 1 -- 172.21.15.19:0/1033484300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba59754e0 con 0x556ba5905160 2024-05-09T02:25:57.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.492+0000 7f2e3e7fc700 1 --2- 172.21.15.19:0/1033484300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5905160 0x556ba5993a50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009990 tx=0x7f2e34000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb538bdd06bbe5ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:57.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.493+0000 7f2e247f8700 1 -- 172.21.15.19:0/1033484300 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e340052e0 con 0x556ba5905160 2024-05-09T02:25:57.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.493+0000 7f2e247f8700 1 -- 172.21.15.19:0/1033484300 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e34003d10 con 0x556ba5905160 2024-05-09T02:25:57.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.493+0000 7f2e247f8700 1 -- 172.21.15.19:0/1033484300 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400bca0 con 0x556ba5905160 2024-05-09T02:25:57.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.495+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/1033484300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5905160 msgr2=0x556ba5993a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:57.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.495+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/1033484300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5905160 0x556ba5993a50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009990 tx=0x7f2e34000bc0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.496+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/1033484300 shutdown_connections 2024-05-09T02:25:57.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.496+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/1033484300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5998880 0x556ba599fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.496+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/1033484300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba59a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.496+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/1033484300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5905160 0x556ba5993a50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.496+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/1033484300 >> 172.21.15.19:0/1033484300 conn(0x556ba587b6a0 msgr2=0x556ba599f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:57.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.496+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/1033484300 shutdown_connections 2024-05-09T02:25:57.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.496+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/1033484300 wait complete. 2024-05-09T02:25:57.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.496+0000 7f2e53e7bb80 1 -- start start 2024-05-09T02:25:57.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.497+0000 7f2e53e7bb80 1 Processor -- start 2024-05-09T02:25:57.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.497+0000 7f2e53e7bb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba5992a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.497+0000 7f2e53e7bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5905160 0x556ba5992f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.497+0000 7f2e53e7bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5998880 0x556ba59934c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.497+0000 7f2e53e7bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401a1b00 con 0x556ba5905160 2024-05-09T02:25:57.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.497+0000 7f2e53e7bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401a1430 con 0x556ba5998880 2024-05-09T02:25:57.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.497+0000 7f2e53e7bb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4019e650 con 0x556ba581b4a0 2024-05-09T02:25:57.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.498+0000 7f2e53e7bb80 10 client.0 _ll_register_callbacks cb 0x556ba572c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:25:57.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.498+0000 7f2e3f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba5992a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:57.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.498+0000 7f2e3f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba5992a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:33064/0 (socket says 172.21.15.19:33064) 2024-05-09T02:25:57.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.498+0000 7f2e3f7fe700 1 -- 172.21.15.19:0/2091214720 learned_addr learned my addr 172.21.15.19:0/2091214720 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:25:57.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.498+0000 7f2e3e7fc700 1 --2- 172.21.15.19:0/2091214720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5905160 0x556ba5992f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:57.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.499+0000 7f2e3f7fe700 1 --2- 172.21.15.19:0/2091214720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5998880 0x556ba59934c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:57.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.499+0000 7f2e3f7fe700 1 -- 172.21.15.19:0/2091214720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5998880 msgr2=0x556ba59934c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:57.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.499+0000 7f2e3f7fe700 1 --2- 172.21.15.19:0/2091214720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5998880 0x556ba59934c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.499+0000 7f2e3f7fe700 1 -- 172.21.15.19:0/2091214720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5905160 msgr2=0x556ba5992f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:25:57.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.499+0000 7f2e3f7fe700 1 --2- 172.21.15.19:0/2091214720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5905160 0x556ba5992f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.499+0000 7f2e3f7fe700 1 -- 172.21.15.19:0/2091214720 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba5a22b90 con 0x556ba581b4a0 2024-05-09T02:25:57.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.500+0000 7f2e3f7fe700 1 --2- 172.21.15.19:0/2091214720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba5992a40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2e38004e40 tx=0x7f2e38036d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3872995af59aa281 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:25:57.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.500+0000 7f2e22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:25:57.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.501+0000 7f2e22ff5700 1 -- 172.21.15.19:0/2091214720 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38057070 con 0x556ba581b4a0 2024-05-09T02:25:57.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.501+0000 7f2e22ff5700 1 -- 172.21.15.19:0/2091214720 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f2e380051c0 con 0x556ba581b4a0 2024-05-09T02:25:57.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.501+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/2091214720 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e34003d10 con 0x556ba581b4a0 2024-05-09T02:25:57.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.501+0000 7f2e22ff5700 1 -- 172.21.15.19:0/2091214720 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f2e38065050 con 0x556ba581b4a0 2024-05-09T02:25:57.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.501+0000 7f2e53e7bb80 10 client.26884 fetch_fsmap learned FSMap version 58 2024-05-09T02:25:57.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.501+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/2091214720 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556ba5a23ce0 con 0x556ba581b4a0 2024-05-09T02:25:57.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.502+0000 7f2e22ff5700 1 -- 172.21.15.19:0/2091214720 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e38058040 con 0x556ba581b4a0 2024-05-09T02:25:57.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.502+0000 7f2e22ff5700 1 -- 172.21.15.19:0/2091214720 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f2e38051040 con 0x556ba581b4a0 2024-05-09T02:25:57.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.502+0000 7f2e53e7bb80 10 client.26884 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:25:57.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.502+0000 7f2e53e7bb80 10 client.26884 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:25:57.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.502+0000 7f2e53e7bb80 20 client.26884 populate_metadata read hostname 'smithi019' 2024-05-09T02:25:57.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.502+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/2091214720 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556ba5a24300 0x556ba5a26720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:25:57.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.502+0000 7f2e53e7bb80 4 client.26884 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:25:57.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.502+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/2091214720 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2e34003d10 con 0x556ba5a24300 2024-05-09T02:25:57.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.502+0000 7f2e3e7fc700 1 --2- 172.21.15.19:0/2091214720 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556ba5a24300 0x556ba5a26720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:25:57.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.503+0000 7f2e3e7fc700 1 --2- 172.21.15.19:0/2091214720 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556ba5a24300 0x556ba5a26720 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13fdd77f2ca69dde server_cookie=b4f0bcca7e34f8d3 in_seq=0 out_seq=0 2024-05-09T02:25:57.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.503+0000 7f2e22ff5700 10 client.26884 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:25:57.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.503+0000 7f2e22ff5700 1 -- 172.21.15.19:0/2091214720 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2e34003d10 con 0x556ba5a24300 2024-05-09T02:25:57.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.503+0000 7f2e22ff5700 10 client.26884 handle_command_reply: tid=0 2024-05-09T02:25:57.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.503+0000 7f2e53e7bb80 1 client.26884 shutdown 2024-05-09T02:25:57.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.503+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/2091214720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 msgr2=0x556ba5992a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:57.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.503+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/2091214720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba5992a40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2e38004e40 tx=0x7f2e38036d60 comp rx=0 tx=0).stop 2024-05-09T02:25:57.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.504+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/2091214720 shutdown_connections 2024-05-09T02:25:57.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.504+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/2091214720 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x556ba5a24300 0x556ba5a26720 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.504+0000 7f2e22ff5700 0 client.26884 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:25:57.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.504+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/2091214720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba5998880 0x556ba59934c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.504+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/2091214720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba581b4a0 0x556ba5992a40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.504+0000 7f2e53e7bb80 1 --2- 172.21.15.19:0/2091214720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ba5905160 0x556ba5992f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.504+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/2091214720 >> 172.21.15.19:0/2091214720 conn(0x556ba587b6a0 msgr2=0x556ba59a08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.505+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/2091214720 shutdown_connections 2024-05-09T02:25:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.505+0000 7f2e53e7bb80 1 -- 172.21.15.19:0/2091214720 wait complete. 2024-05-09T02:25:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.505+0000 7f2e53e7bb80 20 client.26884 trim_cache size 0 max 16384 2024-05-09T02:25:57.508 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:25:57.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.506+0000 7f2e25ffb700 1 -- 172.21.15.19:0/3152318115 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2e280b2a60 msgr2=0x7f2e280b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:57.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.506+0000 7f2e25ffb700 1 --2- 172.21.15.19:0/3152318115 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2e280b2a60 0x7f2e280b4f70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x556ba5994e40 tx=0x7f2e30000bc0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.506+0000 7f2e25ffb700 1 -- 172.21.15.19:0/3152318115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e401034e0 msgr2=0x7f2e40075240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:25:57.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.506+0000 7f2e25ffb700 1 --2- 172.21.15.19:0/3152318115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e401034e0 0x7f2e40075240 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000be10 tx=0x7f2e3000dc30 comp rx=0 tx=0).stop 2024-05-09T02:25:57.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.506+0000 7f2e25ffb700 1 -- 172.21.15.19:0/3152318115 shutdown_connections 2024-05-09T02:25:57.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.507+0000 7f2e25ffb700 1 --2- 172.21.15.19:0/3152318115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e40108760 0x7f2e4006d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.507+0000 7f2e25ffb700 1 --2- 172.21.15.19:0/3152318115 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2e280b2a60 0x7f2e280b4f70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.507+0000 7f2e25ffb700 1 --2- 172.21.15.19:0/3152318115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e401034e0 0x7f2e40075240 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.507+0000 7f2e25ffb700 1 --2- 172.21.15.19:0/3152318115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e40107e10 0x7f2e4006d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:25:57.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.507+0000 7f2e25ffb700 1 -- 172.21.15.19:0/3152318115 >> 172.21.15.19:0/3152318115 conn(0x7f2e400fef30 msgr2=0x7f2e40107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:25:57.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.507+0000 7f2e25ffb700 1 -- 172.21.15.19:0/3152318115 shutdown_connections 2024-05-09T02:25:57.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:25:57.507+0000 7f2e25ffb700 1 -- 172.21.15.19:0/3152318115 wait complete. 2024-05-09T02:25:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 855.0 s, 357.0 tps, lat 293.133 ms stddev 1036.891 2024-05-09T02:25:58.302 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T02:25:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:25:58.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:25:58 smithi019 ceph-mon[30151]: pgmap v459: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.8 MiB/s wr, 683 op/s 2024-05-09T02:25:58.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:25:58 smithi059 ceph-mon[34781]: pgmap v459: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.8 MiB/s wr, 683 op/s 2024-05-09T02:25:59.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:25:58 smithi196 ceph-mon[34718]: pgmap v459: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.8 MiB/s wr, 683 op/s 2024-05-09T02:26:00.849 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:00 smithi019 ceph-mon[30151]: pgmap v460: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.4 MiB/s wr, 477 op/s 2024-05-09T02:26:00.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.898+0000 7ff6acccd700 1 -- 172.21.15.19:0/3416219839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a8103130 msgr2=0x7ff6a8111120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:00.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.898+0000 7ff6acccd700 1 --2- 172.21.15.19:0/3416219839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a8103130 0x7ff6a8111120 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000b0a0 tx=0x7ff6a000b3b0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.898+0000 7ff6acccd700 1 -- 172.21.15.19:0/3416219839 shutdown_connections 2024-05-09T02:26:00.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.898+0000 7ff6acccd700 1 --2- 172.21.15.19:0/3416219839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a8103130 0x7ff6a8111120 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.898+0000 7ff6acccd700 1 --2- 172.21.15.19:0/3416219839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6a810a3e0 0x7ff6a810a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.898+0000 7ff6acccd700 1 --2- 172.21.15.19:0/3416219839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6a81027e0 0x7ff6a8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.898+0000 7ff6acccd700 1 -- 172.21.15.19:0/3416219839 >> 172.21.15.19:0/3416219839 conn(0x7ff6a80fe6f0 msgr2=0x7ff6a8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:00.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.898+0000 7ff6acccd700 1 -- 172.21.15.19:0/3416219839 shutdown_connections 2024-05-09T02:26:00.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.898+0000 7ff6acccd700 1 -- 172.21.15.19:0/3416219839 wait complete. 2024-05-09T02:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.899+0000 7ff6acccd700 1 Processor -- start 2024-05-09T02:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.899+0000 7ff6acccd700 1 -- start start 2024-05-09T02:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6acccd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6a81027e0 0x7ff6a819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6acccd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6a8103130 0x7ff6a819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6acccd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a810a3e0 0x7ff6a819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6acccd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a8114480 con 0x7ff6a810a3e0 2024-05-09T02:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6acccd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a8114300 con 0x7ff6a8103130 2024-05-09T02:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6acccd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a8114600 con 0x7ff6a81027e0 2024-05-09T02:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6a659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6a81027e0 0x7ff6a819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6a659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6a81027e0 0x7ff6a819adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:33078/0 (socket says 172.21.15.19:33078) 2024-05-09T02:26:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6a659c700 1 -- 172.21.15.19:0/2296509731 learned_addr learned my addr 172.21.15.19:0/2296509731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6a5d9b700 1 --2- 172.21.15.19:0/2296509731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6a8103130 0x7ff6a819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6a6d9d700 1 --2- 172.21.15.19:0/2296509731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a810a3e0 0x7ff6a819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6a659c700 1 -- 172.21.15.19:0/2296509731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6a8103130 msgr2=0x7ff6a819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6a659c700 1 --2- 172.21.15.19:0/2296509731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6a8103130 0x7ff6a819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6a659c700 1 -- 172.21.15.19:0/2296509731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a810a3e0 msgr2=0x7ff6a819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6a659c700 1 --2- 172.21.15.19:0/2296509731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a810a3e0 0x7ff6a819f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.900+0000 7ff6a659c700 1 -- 172.21.15.19:0/2296509731 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a819fc80 con 0x7ff6a81027e0 2024-05-09T02:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.901+0000 7ff6a659c700 1 --2- 172.21.15.19:0/2296509731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6a81027e0 0x7ff6a819adb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff69800e9a0 tx=0x7ff69800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.901+0000 7ff6977fe700 1 -- 172.21.15.19:0/2296509731 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69800ca10 con 0x7ff6a81027e0 2024-05-09T02:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.901+0000 7ff6977fe700 1 -- 172.21.15.19:0/2296509731 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff69800cbb0 con 0x7ff6a81027e0 2024-05-09T02:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.901+0000 7ff6acccd700 1 -- 172.21.15.19:0/2296509731 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a819ff70 con 0x7ff6a81027e0 2024-05-09T02:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.901+0000 7ff6977fe700 1 -- 172.21.15.19:0/2296509731 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698018680 con 0x7ff6a81027e0 2024-05-09T02:26:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.901+0000 7ff6acccd700 1 -- 172.21.15.19:0/2296509731 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a81038f0 con 0x7ff6a81027e0 2024-05-09T02:26:00.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.904+0000 7ff6977fe700 1 -- 172.21.15.19:0/2296509731 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff698018820 con 0x7ff6a81027e0 2024-05-09T02:26:00.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.905+0000 7ff6977fe700 1 --2- 172.21.15.19:0/2296509731 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff6900b2a70 0x7ff6900b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.905+0000 7ff6977fe700 1 -- 172.21.15.19:0/2296509731 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7ff6980e28e0 con 0x7ff6a81027e0 2024-05-09T02:26:00.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.905+0000 7ff6a5d9b700 1 --2- 172.21.15.19:0/2296509731 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff6900b2a70 0x7ff6900b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.906+0000 7ff6bb6e3b80 1 Processor -- start 2024-05-09T02:26:00.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.906+0000 7ff6a5d9b700 1 --2- 172.21.15.19:0/2296509731 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff6900b2a70 0x7ff6900b4f80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8100120 tx=0x7ff69c009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:00.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.906+0000 7ff6bb6e3b80 1 -- start start 2024-05-09T02:26:00.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.906+0000 7ff6bb6e3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3bc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.906+0000 7ff6bb6e3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b2394a0 0x55917b3bc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.906+0000 7ff6bb6e3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b3b6400 0x55917b3b6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.906+0000 7ff6bb6e3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55917b2d90a0 con 0x55917b3b6400 2024-05-09T02:26:00.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.906+0000 7ff6bb6e3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55917b27e9a0 con 0x55917b2394a0 2024-05-09T02:26:00.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.906+0000 7ff6bb6e3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55917b2d5060 con 0x55917b2b3d00 2024-05-09T02:26:00.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.906+0000 7ff6a659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3bc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.906+0000 7ff6a659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3bc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:33088/0 (socket says 172.21.15.19:33088) 2024-05-09T02:26:00.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.906+0000 7ff6a659c700 1 -- 172.21.15.19:0/4040768962 learned_addr learned my addr 172.21.15.19:0/4040768962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:00.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.907+0000 7ff6a5d9b700 1 --2- 172.21.15.19:0/4040768962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b2394a0 0x55917b3bc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.907+0000 7ff6a6d9d700 1 --2- 172.21.15.19:0/4040768962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b3b6400 0x55917b3b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.907+0000 7ff6a659c700 1 -- 172.21.15.19:0/4040768962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b2394a0 msgr2=0x55917b3bc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.907+0000 7ff6a659c700 1 --2- 172.21.15.19:0/4040768962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b2394a0 0x55917b3bc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.907+0000 7ff6a659c700 1 -- 172.21.15.19:0/4040768962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b3b6400 msgr2=0x55917b3b6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.907+0000 7ff6a659c700 1 --2- 172.21.15.19:0/4040768962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b3b6400 0x55917b3b6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.907+0000 7ff6a659c700 1 -- 172.21.15.19:0/4040768962 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917b26f500 con 0x55917b2b3d00 2024-05-09T02:26:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.907+0000 7ff6a659c700 1 --2- 172.21.15.19:0/4040768962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3bc240 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff6980a5bb0 tx=0x7ff6980a95b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bc5689beb9a6c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.907+0000 7ff694ff9700 1 -- 172.21.15.19:0/4040768962 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6980aae60 con 0x55917b2b3d00 2024-05-09T02:26:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.907+0000 7ff694ff9700 1 -- 172.21.15.19:0/4040768962 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6980b22e0 con 0x55917b2b3d00 2024-05-09T02:26:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.907+0000 7ff694ff9700 1 -- 172.21.15.19:0/4040768962 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6980a9ff0 con 0x55917b2b3d00 2024-05-09T02:26:00.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.908+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/4040768962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 msgr2=0x55917b3bc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:00.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.908+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/4040768962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3bc240 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff6980a5bb0 tx=0x7ff6980a95b0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.908+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/4040768962 shutdown_connections 2024-05-09T02:26:00.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.908+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/4040768962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b3b6400 0x55917b3b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.908+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/4040768962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3bc240 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.908+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/4040768962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b2394a0 0x55917b3bc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.908+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/4040768962 >> 172.21.15.19:0/4040768962 conn(0x55917b23cf70 msgr2=0x55917b3b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:00.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.909+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/4040768962 shutdown_connections 2024-05-09T02:26:00.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.909+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/4040768962 wait complete. 2024-05-09T02:26:00.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.909+0000 7ff6bb6e3b80 1 -- start start 2024-05-09T02:26:00.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.909+0000 7ff6bb6e3b80 1 Processor -- start 2024-05-09T02:26:00.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6bb6e3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2394a0 0x55917b3c0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6bb6e3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2b3d00 0x55917b3c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6bb6e3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b3b6400 0x55917b3c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6bb6e3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55917b3b7100 con 0x55917b2b3d00 2024-05-09T02:26:00.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6bb6e3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55917b273b20 con 0x55917b3b6400 2024-05-09T02:26:00.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6a5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2b3d00 0x55917b3c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6a659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2394a0 0x55917b3c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6a5d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2b3d00 0x55917b3c05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43762/0 (socket says 172.21.15.19:43762) 2024-05-09T02:26:00.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6a5d9b700 1 -- 172.21.15.19:0/3012613324 learned_addr learned my addr 172.21.15.19:0/3012613324 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:00.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/3012613324 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55917b3cc1d0 con 0x55917b2394a0 2024-05-09T02:26:00.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6a6d9d700 1 --2- 172.21.15.19:0/3012613324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b3b6400 0x55917b3c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6bb6e3b80 10 client.0 _ll_register_callbacks cb 0x55917b14a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:26:00.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6a5d9b700 1 -- 172.21.15.19:0/3012613324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2394a0 msgr2=0x55917b3c0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:00.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6a5d9b700 1 --2- 172.21.15.19:0/3012613324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2394a0 0x55917b3c0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6a5d9b700 1 -- 172.21.15.19:0/3012613324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b3b6400 msgr2=0x55917b3c0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:00.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6a5d9b700 1 --2- 172.21.15.19:0/3012613324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b3b6400 0x55917b3c0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6a5d9b700 1 -- 172.21.15.19:0/3012613324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917b154490 con 0x55917b2b3d00 2024-05-09T02:26:00.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.910+0000 7ff6a659c700 1 --2- 172.21.15.19:0/3012613324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2394a0 0x55917b3c0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.911+0000 7ff6a5d9b700 1 --2- 172.21.15.19:0/3012613324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2b3d00 0x55917b3c05c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff69c004530 tx=0x7ff69c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5eaa06586032d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.911+0000 7ff68dffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.911+0000 7ff68dffb700 1 -- 172.21.15.19:0/3012613324 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c004560 con 0x55917b2b3d00 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.911+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/3012613324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6980b22e0 con 0x55917b2b3d00 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.911+0000 7ff68dffb700 1 -- 172.21.15.19:0/3012613324 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7ff69c01d410 con 0x55917b2b3d00 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.911+0000 7ff68dffb700 1 -- 172.21.15.19:0/3012613324 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff69c01d610 con 0x55917b2b3d00 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.911+0000 7ff6bb6e3b80 10 client.17106 fetch_fsmap learned FSMap version 58 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.911+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/3012613324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55917b3cdbc0 con 0x55917b2b3d00 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.911+0000 7ff68dffb700 1 -- 172.21.15.19:0/3012613324 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6980b22e0 con 0x55917b2b3d00 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.912+0000 7ff68dffb700 1 -- 172.21.15.19:0/3012613324 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7ff69c02f960 con 0x55917b2b3d00 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.912+0000 7ff6bb6e3b80 10 client.17106 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.912+0000 7ff6bb6e3b80 10 client.17106 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.912+0000 7ff6bb6e3b80 20 client.17106 populate_metadata read hostname 'smithi019' 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.912+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/3012613324 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55917b3d0200 0x55917b3d26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.912+0000 7ff6bb6e3b80 4 client.17106 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:26:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.912+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/3012613324 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55917b2e6a80 con 0x55917b3d0200 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.912+0000 7ff6a659c700 1 --2- 172.21.15.19:0/3012613324 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55917b3d0200 0x55917b3d26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.913+0000 7ff6a659c700 1 --2- 172.21.15.19:0/3012613324 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55917b3d0200 0x55917b3d26d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=351a93a8f33ee39f server_cookie=c3dd87d6fa8fa28c in_seq=0 out_seq=0 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.913+0000 7ff68dffb700 10 client.17106 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.913+0000 7ff68dffb700 1 -- 172.21.15.19:0/3012613324 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55917b2e6a80 con 0x55917b3d0200 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.913+0000 7ff68dffb700 10 client.17106 handle_command_reply: tid=0 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.913+0000 7ff6bb6e3b80 1 client.17106 shutdown 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.914+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/3012613324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2b3d00 msgr2=0x55917b3c05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.914+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/3012613324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2b3d00 0x55917b3c05c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff69c004530 tx=0x7ff69c0047c0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.914+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/3012613324 shutdown_connections 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.914+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/3012613324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b3b6400 0x55917b3c0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.914+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/3012613324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2394a0 0x55917b3c0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.914+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/3012613324 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55917b3d0200 0x55917b3d26d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.914+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/3012613324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2b3d00 0x55917b3c05c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.914+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/3012613324 >> 172.21.15.19:0/3012613324 conn(0x55917b23cf70 msgr2=0x55917b3aaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.914+0000 7ff68dffb700 0 client.17106 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:26:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.914+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/3012613324 shutdown_connections 2024-05-09T02:26:00.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.914+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/3012613324 wait complete. 2024-05-09T02:26:00.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.914+0000 7ff6bb6e3b80 20 client.17106 trim_cache size 0 max 16384 2024-05-09T02:26:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.932+0000 7ff6bb6e3b80 1 Processor -- start 2024-05-09T02:26:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.932+0000 7ff6bb6e3b80 1 -- start start 2024-05-09T02:26:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.932+0000 7ff6bb6e3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2394a0 0x55917b3bf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.932+0000 7ff6bb6e3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3c2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.932+0000 7ff6bb6e3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b3b6400 0x55917b3d7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.932+0000 7ff6bb6e3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a81b1190 con 0x55917b2394a0 2024-05-09T02:26:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.933+0000 7ff6bb6e3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a81b2cd0 con 0x55917b3b6400 2024-05-09T02:26:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.933+0000 7ff6bb6e3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a81b1860 con 0x55917b2b3d00 2024-05-09T02:26:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.933+0000 7ff6a659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2394a0 0x55917b3bf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.933+0000 7ff6a5d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.933+0000 7ff6a6d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b3b6400 0x55917b3d7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.933+0000 7ff6a659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2394a0 0x55917b3bf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43774/0 (socket says 172.21.15.19:43774) 2024-05-09T02:26:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.933+0000 7ff6a659c700 1 -- 172.21.15.19:0/660670636 learned_addr learned my addr 172.21.15.19:0/660670636 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.933+0000 7ff6a659c700 1 -- 172.21.15.19:0/660670636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 msgr2=0x55917b3c2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.933+0000 7ff6a659c700 1 --2- 172.21.15.19:0/660670636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3c2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.933+0000 7ff6a659c700 1 -- 172.21.15.19:0/660670636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b3b6400 msgr2=0x55917b3d7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.933+0000 7ff6a659c700 1 --2- 172.21.15.19:0/660670636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b3b6400 0x55917b3d7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.933+0000 7ff6a659c700 1 -- 172.21.15.19:0/660670636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917b3b0b80 con 0x55917b2394a0 2024-05-09T02:26:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.933+0000 7ff6a5d9b700 1 --2- 172.21.15.19:0/660670636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:26:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.933+0000 7ff6a659c700 1 --2- 172.21.15.19:0/660670636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2394a0 0x55917b3bf3a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff6980a6da0 tx=0x7ff6980cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ae5d8c8849bf48 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.934+0000 7ff68ffff700 1 -- 172.21.15.19:0/660670636 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6980aae60 con 0x55917b2394a0 2024-05-09T02:26:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.934+0000 7ff68ffff700 1 -- 172.21.15.19:0/660670636 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6980c41b0 con 0x55917b2394a0 2024-05-09T02:26:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.934+0000 7ff68ffff700 1 -- 172.21.15.19:0/660670636 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6980b2760 con 0x55917b2394a0 2024-05-09T02:26:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.935+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/660670636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2394a0 msgr2=0x55917b3bf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.935+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/660670636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2394a0 0x55917b3bf3a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff6980a6da0 tx=0x7ff6980cc040 comp rx=0 tx=0).stop 2024-05-09T02:26:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.935+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/660670636 shutdown_connections 2024-05-09T02:26:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.935+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/660670636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b3b6400 0x55917b3d7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.935+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/660670636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2394a0 0x55917b3bf3a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.935+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/660670636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.935+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/660670636 >> 172.21.15.19:0/660670636 conn(0x55917b23cf70 msgr2=0x55917b3bb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.935+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/660670636 shutdown_connections 2024-05-09T02:26:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.935+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/660670636 wait complete. 2024-05-09T02:26:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.936+0000 7ff6bb6e3b80 1 -- start start 2024-05-09T02:26:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.936+0000 7ff6bb6e3b80 1 Processor -- start 2024-05-09T02:26:00.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.936+0000 7ff6bb6e3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2394a0 0x55917b3c8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.936+0000 7ff6bb6e3b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.936+0000 7ff6bb6e3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b3b6400 0x55917b3c3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.936+0000 7ff6bb6e3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a81b2600 con 0x55917b2394a0 2024-05-09T02:26:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.936+0000 7ff6bb6e3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a81b1f30 con 0x55917b3b6400 2024-05-09T02:26:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.936+0000 7ff6bb6e3b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a81af170 con 0x55917b2b3d00 2024-05-09T02:26:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.937+0000 7ff6bb6e3b80 10 client.0 _ll_register_callbacks cb 0x55917b14a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:26:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.937+0000 7ff6a659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2394a0 0x55917b3c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.937+0000 7ff6a5d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.937+0000 7ff6a5d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3c30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:33118/0 (socket says 172.21.15.19:33118) 2024-05-09T02:26:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.937+0000 7ff6a5d9b700 1 -- 172.21.15.19:0/576313479 learned_addr learned my addr 172.21.15.19:0/576313479 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.937+0000 7ff6a659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2394a0 0x55917b3c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43786/0 (socket says 172.21.15.19:43786) 2024-05-09T02:26:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.937+0000 7ff6a6d9d700 1 --2- 172.21.15.19:0/576313479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b3b6400 0x55917b3c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.937+0000 7ff6a5d9b700 1 -- 172.21.15.19:0/576313479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b3b6400 msgr2=0x55917b3c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.937+0000 7ff6a5d9b700 1 --2- 172.21.15.19:0/576313479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b3b6400 0x55917b3c3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.937+0000 7ff6a5d9b700 1 -- 172.21.15.19:0/576313479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2394a0 msgr2=0x55917b3c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.937+0000 7ff6a5d9b700 1 --2- 172.21.15.19:0/576313479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2394a0 0x55917b3c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.937+0000 7ff6a5d9b700 1 -- 172.21.15.19:0/576313479 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917b3c3c70 con 0x55917b2b3d00 2024-05-09T02:26:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.937+0000 7ff6a659c700 1 --2- 172.21.15.19:0/576313479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2394a0 0x55917b3c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:26:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.937+0000 7ff6a5d9b700 1 --2- 172.21.15.19:0/576313479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3c30f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff69c013010 tx=0x7ff69c01d410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b82720f7413a752 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.937+0000 7ff68dffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:26:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.938+0000 7ff68dffb700 1 -- 172.21.15.19:0/576313479 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c03e060 con 0x55917b2b3d00 2024-05-09T02:26:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.938+0000 7ff68dffb700 1 -- 172.21.15.19:0/576313479 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7ff69c01d7b0 con 0x55917b2b3d00 2024-05-09T02:26:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.938+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/576313479 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6980c41b0 con 0x55917b2b3d00 2024-05-09T02:26:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.938+0000 7ff68dffb700 1 -- 172.21.15.19:0/576313479 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff69c01d9f0 con 0x55917b2b3d00 2024-05-09T02:26:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.938+0000 7ff6bb6e3b80 10 client.26902 fetch_fsmap learned FSMap version 58 2024-05-09T02:26:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.938+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/576313479 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55917b4397e0 con 0x55917b2b3d00 2024-05-09T02:26:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.938+0000 7ff68dffb700 1 -- 172.21.15.19:0/576313479 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6980c41b0 con 0x55917b2b3d00 2024-05-09T02:26:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.938+0000 7ff68dffb700 1 -- 172.21.15.19:0/576313479 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7ff69c027740 con 0x55917b2b3d00 2024-05-09T02:26:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.938+0000 7ff6bb6e3b80 10 client.26902 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:26:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.938+0000 7ff6bb6e3b80 10 client.26902 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:26:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.938+0000 7ff6bb6e3b80 20 client.26902 populate_metadata read hostname 'smithi019' 2024-05-09T02:26:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.938+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/576313479 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55917b439ec0 0x55917b43c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:00.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.938+0000 7ff6bb6e3b80 4 client.26902 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:26:00.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.938+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/576313479 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55917b2e0b20 con 0x55917b439ec0 2024-05-09T02:26:00.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.939+0000 7ff6a659c700 1 --2- 172.21.15.19:0/576313479 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55917b439ec0 0x55917b43c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:00.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.939+0000 7ff68dffb700 10 client.26902 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:26:00.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.940+0000 7ff6a659c700 1 --2- 172.21.15.19:0/576313479 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55917b439ec0 0x55917b43c2e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c95b86ea0a821cac server_cookie=10532e9748e350b1 in_seq=0 out_seq=0 2024-05-09T02:26:00.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.940+0000 7ff68dffb700 1 -- 172.21.15.19:0/576313479 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55917b2e0b20 con 0x55917b439ec0 2024-05-09T02:26:00.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.940+0000 7ff68dffb700 10 client.26902 handle_command_reply: tid=0 2024-05-09T02:26:00.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.940+0000 7ff6bb6e3b80 1 client.26902 shutdown 2024-05-09T02:26:00.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.940+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/576313479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 msgr2=0x55917b3c30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:00.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.940+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/576313479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3c30f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff69c013010 tx=0x7ff69c01d410 comp rx=0 tx=0).stop 2024-05-09T02:26:00.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.940+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/576313479 shutdown_connections 2024-05-09T02:26:00.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.940+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/576313479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55917b3b6400 0x55917b3c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.940+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/576313479 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55917b439ec0 0x55917b43c2e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.940+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/576313479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917b2394a0 0x55917b3c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.940+0000 7ff6bb6e3b80 1 --2- 172.21.15.19:0/576313479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55917b2b3d00 0x55917b3c30f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.940+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/576313479 >> 172.21.15.19:0/576313479 conn(0x55917b23cf70 msgr2=0x55917b3aaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:00.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.940+0000 7ff68dffb700 0 client.26902 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:26:00.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.940+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/576313479 shutdown_connections 2024-05-09T02:26:00.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.940+0000 7ff6bb6e3b80 1 -- 172.21.15.19:0/576313479 wait complete. 2024-05-09T02:26:00.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.941+0000 7ff6bb6e3b80 20 client.26902 trim_cache size 0 max 16384 2024-05-09T02:26:00.946 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:26:00.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.941+0000 7ff6acccd700 1 -- 172.21.15.19:0/2296509731 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff6900b2a70 msgr2=0x7ff6900b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:00.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.941+0000 7ff6acccd700 1 --2- 172.21.15.19:0/2296509731 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff6900b2a70 0x7ff6900b4f80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8100120 tx=0x7ff69c009260 comp rx=0 tx=0).stop 2024-05-09T02:26:00.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.941+0000 7ff6acccd700 1 -- 172.21.15.19:0/2296509731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6a81027e0 msgr2=0x7ff6a819adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:00.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.941+0000 7ff6acccd700 1 --2- 172.21.15.19:0/2296509731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6a81027e0 0x7ff6a819adb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff69800e9a0 tx=0x7ff69800ecb0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.941+0000 7ff6acccd700 1 -- 172.21.15.19:0/2296509731 shutdown_connections 2024-05-09T02:26:00.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.941+0000 7ff6acccd700 1 --2- 172.21.15.19:0/2296509731 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7ff6900b2a70 0x7ff6900b4f80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.941+0000 7ff6acccd700 1 --2- 172.21.15.19:0/2296509731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a810a3e0 0x7ff6a819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.941+0000 7ff6acccd700 1 --2- 172.21.15.19:0/2296509731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6a81027e0 0x7ff6a819adb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.941+0000 7ff6acccd700 1 --2- 172.21.15.19:0/2296509731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6a8103130 0x7ff6a819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:00.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.941+0000 7ff6acccd700 1 -- 172.21.15.19:0/2296509731 >> 172.21.15.19:0/2296509731 conn(0x7ff6a80fe6f0 msgr2=0x7ff6a8100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:00.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.941+0000 7ff6acccd700 1 -- 172.21.15.19:0/2296509731 shutdown_connections 2024-05-09T02:26:00.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:00.941+0000 7ff6acccd700 1 -- 172.21.15.19:0/2296509731 wait complete. 2024-05-09T02:26:00.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:00 smithi059 ceph-mon[34781]: pgmap v460: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.4 MiB/s wr, 477 op/s 2024-05-09T02:26:01.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:00 smithi196 ceph-mon[34718]: pgmap v460: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.4 MiB/s wr, 477 op/s 2024-05-09T02:26:01.762 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T02:26:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:26:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 860.0 s, 1442.6 tps, lat 69.801 ms stddev 102.113 2024-05-09T02:26:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:02 smithi019 ceph-mon[30151]: pgmap v461: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.7 MiB/s wr, 592 op/s 2024-05-09T02:26:02.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:02 smithi059 ceph-mon[34781]: pgmap v461: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.7 MiB/s wr, 592 op/s 2024-05-09T02:26:03.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:02 smithi196 ceph-mon[34718]: pgmap v461: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.7 MiB/s wr, 592 op/s 2024-05-09T02:26:04.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:04 smithi019 ceph-mon[30151]: pgmap v462: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 3.4 MiB/s wr, 355 op/s 2024-05-09T02:26:04.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:04 smithi059 ceph-mon[34781]: pgmap v462: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 3.4 MiB/s wr, 355 op/s 2024-05-09T02:26:05.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:04 smithi196 ceph-mon[34718]: pgmap v462: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 3.4 MiB/s wr, 355 op/s 2024-05-09T02:26:06.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.101+0000 7f791af7f700 1 -- 172.21.15.19:0/3793046039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7914108f10 msgr2=0x7f7914109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:06.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.101+0000 7f791af7f700 1 --2- 172.21.15.19:0/3793046039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7914108f10 0x7f7914109320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f79000099c0 tx=0x7f7900009cd0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.102+0000 7f791af7f700 1 -- 172.21.15.19:0/3793046039 shutdown_connections 2024-05-09T02:26:06.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.102+0000 7f791af7f700 1 --2- 172.21.15.19:0/3793046039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7914109860 0x7f79141111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.102+0000 7f791af7f700 1 --2- 172.21.15.19:0/3793046039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79141045e0 0x7f79141049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.102+0000 7f791af7f700 1 --2- 172.21.15.19:0/3793046039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7914108f10 0x7f7914109320 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.102+0000 7f791af7f700 1 -- 172.21.15.19:0/3793046039 >> 172.21.15.19:0/3793046039 conn(0x7f79140ffe70 msgr2=0x7f79141022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:06.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.103+0000 7f791af7f700 1 -- 172.21.15.19:0/3793046039 shutdown_connections 2024-05-09T02:26:06.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.103+0000 7f791af7f700 1 -- 172.21.15.19:0/3793046039 wait complete. 2024-05-09T02:26:06.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.104+0000 7f791af7f700 1 Processor -- start 2024-05-09T02:26:06.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.104+0000 7f791af7f700 1 -- start start 2024-05-09T02:26:06.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.104+0000 7f791af7f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79141045e0 0x7f791419d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.104+0000 7f791af7f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7914108f10 0x7f791419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.104+0000 7f791af7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7914109860 0x7f79141a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.104+0000 7f791af7f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7914076000 con 0x7f7914109860 2024-05-09T02:26:06.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.104+0000 7f791af7f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7914075e80 con 0x7f7914108f10 2024-05-09T02:26:06.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.104+0000 7f791af7f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7914076180 con 0x7f79141045e0 2024-05-09T02:26:06.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.104+0000 7f791951c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7914109860 0x7f79141a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.104+0000 7f7918d1b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79141045e0 0x7f791419d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.104+0000 7f7913fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7914108f10 0x7f791419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.104+0000 7f791951c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7914109860 0x7f79141a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43806/0 (socket says 172.21.15.19:43806) 2024-05-09T02:26:06.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.104+0000 7f791951c700 1 -- 172.21.15.19:0/2860288669 learned_addr learned my addr 172.21.15.19:0/2860288669 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:06.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.104+0000 7f7913fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7914108f10 0x7f791419d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:36780/0 (socket says 172.21.15.19:36780) 2024-05-09T02:26:06.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.105+0000 7f7913fff700 1 -- 172.21.15.19:0/2860288669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79141045e0 msgr2=0x7f791419d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:06.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.105+0000 7f7913fff700 1 --2- 172.21.15.19:0/2860288669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79141045e0 0x7f791419d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.105+0000 7f7913fff700 1 -- 172.21.15.19:0/2860288669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7914109860 msgr2=0x7f79141a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:06.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.105+0000 7f7913fff700 1 --2- 172.21.15.19:0/2860288669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7914109860 0x7f79141a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.105+0000 7f7913fff700 1 -- 172.21.15.19:0/2860288669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79141a1df0 con 0x7f7914108f10 2024-05-09T02:26:06.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.105+0000 7f7918d1b700 1 --2- 172.21.15.19:0/2860288669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79141045e0 0x7f791419d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:26:06.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.105+0000 7f7913fff700 1 --2- 172.21.15.19:0/2860288669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7914108f10 0x7f791419d740 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f79000098e0 tx=0x7f79000045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:06.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.105+0000 7f7911ffb700 1 -- 172.21.15.19:0/2860288669 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900009e60 con 0x7f7914108f10 2024-05-09T02:26:06.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.105+0000 7f7911ffb700 1 -- 172.21.15.19:0/2860288669 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7900004810 con 0x7f7914108f10 2024-05-09T02:26:06.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.105+0000 7f7911ffb700 1 -- 172.21.15.19:0/2860288669 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900005290 con 0x7f7914108f10 2024-05-09T02:26:06.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.105+0000 7f791af7f700 1 -- 172.21.15.19:0/2860288669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79141a2080 con 0x7f7914108f10 2024-05-09T02:26:06.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.105+0000 7f791af7f700 1 -- 172.21.15.19:0/2860288669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79141a24b0 con 0x7f7914108f10 2024-05-09T02:26:06.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.108+0000 7f7911ffb700 1 -- 172.21.15.19:0/2860288669 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7900028060 con 0x7f7914108f10 2024-05-09T02:26:06.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.110+0000 7f7911ffb700 1 --2- 172.21.15.19:0/2860288669 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79040b2a70 0x7f79040b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.110+0000 7f7918d1b700 1 --2- 172.21.15.19:0/2860288669 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79040b2a70 0x7f79040b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.110+0000 7f7911ffb700 1 -- 172.21.15.19:0/2860288669 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f79000e2e70 con 0x7f7914108f10 2024-05-09T02:26:06.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.110+0000 7f7929995b80 1 Processor -- start 2024-05-09T02:26:06.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.110+0000 7f7929995b80 1 -- start start 2024-05-09T02:26:06.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.110+0000 7f7918d1b700 1 --2- 172.21.15.19:0/2860288669 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79040b2a70 0x7f79040b4f80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7908011490 tx=0x7f7908009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:06.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7929995b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563056887160 0x5630569204f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7929995b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305679d4a0 0x563056920a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7929995b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305691a6a0 0x56305691aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7929995b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56305683cff0 con 0x56305691a6a0 2024-05-09T02:26:06.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7929995b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5630567db7f0 con 0x563056887160 2024-05-09T02:26:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7929995b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563056838fb0 con 0x56305679d4a0 2024-05-09T02:26:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7918d1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563056887160 0x5630569204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f791951c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305691a6a0 0x56305691aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7918d1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563056887160 0x5630569204f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:36792/0 (socket says 172.21.15.19:36792) 2024-05-09T02:26:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7918d1b700 1 -- 172.21.15.19:0/3837057944 learned_addr learned my addr 172.21.15.19:0/3837057944 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7913fff700 1 --2- 172.21.15.19:0/3837057944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305679d4a0 0x563056920a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7918d1b700 1 -- 172.21.15.19:0/3837057944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305679d4a0 msgr2=0x563056920a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7918d1b700 1 --2- 172.21.15.19:0/3837057944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305679d4a0 0x563056920a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7918d1b700 1 -- 172.21.15.19:0/3837057944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305691a6a0 msgr2=0x56305691aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7918d1b700 1 --2- 172.21.15.19:0/3837057944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305691a6a0 0x56305691aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7918d1b700 1 -- 172.21.15.19:0/3837057944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630567d3370 con 0x563056887160 2024-05-09T02:26:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.111+0000 7f7918d1b700 1 --2- 172.21.15.19:0/3837057944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563056887160 0x5630569204f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f79080043c0 tx=0x7f7908014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5acaa05feea7f35 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.112+0000 7f78feffd700 1 -- 172.21.15.19:0/3837057944 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908003d10 con 0x563056887160 2024-05-09T02:26:06.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.112+0000 7f78feffd700 1 -- 172.21.15.19:0/3837057944 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7908011e50 con 0x563056887160 2024-05-09T02:26:06.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.112+0000 7f78feffd700 1 -- 172.21.15.19:0/3837057944 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790800e6a0 con 0x563056887160 2024-05-09T02:26:06.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.113+0000 7f7929995b80 1 -- 172.21.15.19:0/3837057944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563056887160 msgr2=0x5630569204f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:06.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.113+0000 7f7929995b80 1 --2- 172.21.15.19:0/3837057944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563056887160 0x5630569204f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f79080043c0 tx=0x7f7908014000 comp rx=0 tx=0).stop 2024-05-09T02:26:06.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.113+0000 7f7929995b80 1 -- 172.21.15.19:0/3837057944 shutdown_connections 2024-05-09T02:26:06.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.113+0000 7f7929995b80 1 --2- 172.21.15.19:0/3837057944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56305691a6a0 0x56305691aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.113+0000 7f7929995b80 1 --2- 172.21.15.19:0/3837057944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563056887160 0x5630569204f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.113+0000 7f7929995b80 1 --2- 172.21.15.19:0/3837057944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305679d4a0 0x563056920a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.113+0000 7f7929995b80 1 -- 172.21.15.19:0/3837057944 >> 172.21.15.19:0/3837057944 conn(0x5630567fd6a0 msgr2=0x563056915400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:06.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.113+0000 7f7929995b80 1 -- 172.21.15.19:0/3837057944 shutdown_connections 2024-05-09T02:26:06.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.113+0000 7f7929995b80 1 -- 172.21.15.19:0/3837057944 wait complete. 2024-05-09T02:26:06.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.113+0000 7f7929995b80 1 -- start start 2024-05-09T02:26:06.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.113+0000 7f7929995b80 1 Processor -- start 2024-05-09T02:26:06.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.114+0000 7f7929995b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 0x5630569241d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.114+0000 7f7929995b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563056887160 0x563056924710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.114+0000 7f7929995b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305691a6a0 0x563056924c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.114+0000 7f7929995b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5630566b8ba0 con 0x563056887160 2024-05-09T02:26:06.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.114+0000 7f7929995b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5630567f7150 con 0x56305679d4a0 2024-05-09T02:26:06.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.114+0000 7f7929995b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5630569271d0 con 0x56305691a6a0 2024-05-09T02:26:06.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.114+0000 7f7929995b80 10 client.0 _ll_register_callbacks cb 0x5630566ae6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:26:06.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.114+0000 7f7913fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563056887160 0x563056924710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.114+0000 7f7918d1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 0x5630569241d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.114+0000 7f7918d1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 0x5630569241d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:36794/0 (socket says 172.21.15.19:36794) 2024-05-09T02:26:06.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.114+0000 7f7918d1b700 1 -- 172.21.15.19:0/3175392269 learned_addr learned my addr 172.21.15.19:0/3175392269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:06.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.114+0000 7f7913fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563056887160 0x563056924710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43814/0 (socket says 172.21.15.19:43814) 2024-05-09T02:26:06.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.114+0000 7f791951c700 1 --2- 172.21.15.19:0/3175392269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305691a6a0 0x563056924c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.114+0000 7f7918d1b700 1 -- 172.21.15.19:0/3175392269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305691a6a0 msgr2=0x563056924c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:06.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.114+0000 7f7918d1b700 1 --2- 172.21.15.19:0/3175392269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305691a6a0 0x563056924c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.115+0000 7f7918d1b700 1 -- 172.21.15.19:0/3175392269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563056887160 msgr2=0x563056924710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:06.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.115+0000 7f7918d1b700 1 --2- 172.21.15.19:0/3175392269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563056887160 0x563056924710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.115+0000 7f7918d1b700 1 -- 172.21.15.19:0/3175392269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630566b8490 con 0x56305679d4a0 2024-05-09T02:26:06.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.115+0000 7f7913fff700 1 --2- 172.21.15.19:0/3175392269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563056887160 0x563056924710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:26:06.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.116+0000 7f7918d1b700 1 --2- 172.21.15.19:0/3175392269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 0x5630569241d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7908005270 tx=0x7f79080052a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b13710ce67a0ad6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:06.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.116+0000 7f78f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:26:06.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.116+0000 7f78f3fff700 1 -- 172.21.15.19:0/3175392269 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908004710 con 0x56305679d4a0 2024-05-09T02:26:06.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.116+0000 7f78f3fff700 1 -- 172.21.15.19:0/3175392269 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f79080048b0 con 0x56305679d4a0 2024-05-09T02:26:06.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.117+0000 7f7929995b80 1 -- 172.21.15.19:0/3175392269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7908011e50 con 0x56305679d4a0 2024-05-09T02:26:06.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.117+0000 7f78f3fff700 1 -- 172.21.15.19:0/3175392269 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f7908037050 con 0x56305679d4a0 2024-05-09T02:26:06.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.117+0000 7f7929995b80 10 client.26846 fetch_fsmap learned FSMap version 58 2024-05-09T02:26:06.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.117+0000 7f7929995b80 1 -- 172.21.15.19:0/3175392269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563056931ba0 con 0x56305679d4a0 2024-05-09T02:26:06.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.117+0000 7f78f3fff700 1 -- 172.21.15.19:0/3175392269 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7908011e50 con 0x56305679d4a0 2024-05-09T02:26:06.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.118+0000 7f78f3fff700 1 -- 172.21.15.19:0/3175392269 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f790801ea90 con 0x56305679d4a0 2024-05-09T02:26:06.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.118+0000 7f7929995b80 10 client.26846 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:26:06.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.118+0000 7f7929995b80 10 client.26846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:26:06.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.118+0000 7f7929995b80 20 client.26846 populate_metadata read hostname 'smithi019' 2024-05-09T02:26:06.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.118+0000 7f7929995b80 1 --2- 172.21.15.19:0/3175392269 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x563056934120 0x563056936540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.118+0000 7f7929995b80 4 client.26846 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:26:06.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.118+0000 7f7929995b80 1 -- 172.21.15.19:0/3175392269 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56305684a9d0 con 0x563056934120 2024-05-09T02:26:06.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.118+0000 7f7913fff700 1 --2- 172.21.15.19:0/3175392269 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x563056934120 0x563056936540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.119+0000 7f7913fff700 1 --2- 172.21.15.19:0/3175392269 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x563056934120 0x563056936540 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=956a32e308bf436c server_cookie=d20abae834c22700 in_seq=0 out_seq=0 2024-05-09T02:26:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.119+0000 7f78f3fff700 10 client.26846 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.119+0000 7f78f3fff700 1 -- 172.21.15.19:0/3175392269 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56305684a9d0 con 0x563056934120 2024-05-09T02:26:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.119+0000 7f78f3fff700 10 client.26846 handle_command_reply: tid=0 2024-05-09T02:26:06.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.120+0000 7f7929995b80 1 client.26846 shutdown 2024-05-09T02:26:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.120+0000 7f7929995b80 1 -- 172.21.15.19:0/3175392269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 msgr2=0x5630569241d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.120+0000 7f7929995b80 1 --2- 172.21.15.19:0/3175392269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 0x5630569241d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7908005270 tx=0x7f79080052a0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.120+0000 7f7929995b80 1 -- 172.21.15.19:0/3175392269 shutdown_connections 2024-05-09T02:26:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.120+0000 7f7929995b80 1 --2- 172.21.15.19:0/3175392269 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x563056934120 0x563056936540 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.120+0000 7f78f3fff700 0 client.26846 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.120+0000 7f7929995b80 1 --2- 172.21.15.19:0/3175392269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56305691a6a0 0x563056924c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.120+0000 7f7929995b80 1 --2- 172.21.15.19:0/3175392269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 0x5630569241d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.120+0000 7f7929995b80 1 --2- 172.21.15.19:0/3175392269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563056887160 0x563056924710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.120+0000 7f7929995b80 1 -- 172.21.15.19:0/3175392269 >> 172.21.15.19:0/3175392269 conn(0x5630567fd6a0 msgr2=0x5630569151c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:06.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.120+0000 7f7929995b80 1 -- 172.21.15.19:0/3175392269 shutdown_connections 2024-05-09T02:26:06.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.120+0000 7f7929995b80 1 -- 172.21.15.19:0/3175392269 wait complete. 2024-05-09T02:26:06.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.120+0000 7f7929995b80 20 client.26846 trim_cache size 0 max 16384 2024-05-09T02:26:06.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.146+0000 7f7929995b80 1 Processor -- start 2024-05-09T02:26:06.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.147+0000 7f7929995b80 1 -- start start 2024-05-09T02:26:06.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.148+0000 7f7929995b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630567fd6a0 0x5630569302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.148+0000 7f7929995b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 0x5630569231d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.148+0000 7f7918d1b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630567fd6a0 0x5630569302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.148+0000 7f7918d1b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630567fd6a0 0x5630569302f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:33176/0 (socket says 172.21.15.19:33176) 2024-05-09T02:26:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.148+0000 7f7918d1b700 1 -- 172.21.15.19:0/529454907 learned_addr learned my addr 172.21.15.19:0/529454907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.148+0000 7f7929995b80 1 --2- 172.21.15.19:0/529454907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563056887160 0x563056919690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.148+0000 7f7929995b80 1 -- 172.21.15.19:0/529454907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79141b0e00 con 0x563056887160 2024-05-09T02:26:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.149+0000 7f7929995b80 1 -- 172.21.15.19:0/529454907 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f79141b2940 con 0x56305679d4a0 2024-05-09T02:26:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.149+0000 7f7929995b80 1 -- 172.21.15.19:0/529454907 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79141b14d0 con 0x5630567fd6a0 2024-05-09T02:26:06.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.149+0000 7f791951c700 1 --2- 172.21.15.19:0/529454907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563056887160 0x563056919690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.149+0000 7f791951c700 1 -- 172.21.15.19:0/529454907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630567fd6a0 msgr2=0x5630569302f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.149+0000 7f791951c700 1 --2- 172.21.15.19:0/529454907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630567fd6a0 0x5630569302f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.149+0000 7f791951c700 1 -- 172.21.15.19:0/529454907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 msgr2=0x5630569231d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.149+0000 7f791951c700 1 --2- 172.21.15.19:0/529454907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 0x5630569231d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.149+0000 7f791951c700 1 -- 172.21.15.19:0/529454907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630569162e0 con 0x563056887160 2024-05-09T02:26:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.150+0000 7f791951c700 1 --2- 172.21.15.19:0/529454907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563056887160 0x563056919690 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f790c009fd0 tx=0x7f790c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4c7f1d732c9b8c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:06.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.151+0000 7f78fe7fc700 1 -- 172.21.15.19:0/529454907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c0093a0 con 0x563056887160 2024-05-09T02:26:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.151+0000 7f78fe7fc700 1 -- 172.21.15.19:0/529454907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790c004db0 con 0x563056887160 2024-05-09T02:26:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.151+0000 7f78fe7fc700 1 -- 172.21.15.19:0/529454907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c0175b0 con 0x563056887160 2024-05-09T02:26:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.158+0000 7f7929995b80 1 -- 172.21.15.19:0/529454907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563056887160 msgr2=0x563056919690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.158+0000 7f7929995b80 1 --2- 172.21.15.19:0/529454907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563056887160 0x563056919690 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f790c009fd0 tx=0x7f790c009920 comp rx=0 tx=0).stop 2024-05-09T02:26:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.158+0000 7f7929995b80 1 -- 172.21.15.19:0/529454907 shutdown_connections 2024-05-09T02:26:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.158+0000 7f7929995b80 1 --2- 172.21.15.19:0/529454907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563056887160 0x563056919690 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.158+0000 7f7929995b80 1 --2- 172.21.15.19:0/529454907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630567fd6a0 0x5630569302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.158+0000 7f7929995b80 1 --2- 172.21.15.19:0/529454907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 0x5630569231d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.158+0000 7f7929995b80 1 -- 172.21.15.19:0/529454907 >> 172.21.15.19:0/529454907 conn(0x56305691ae10 msgr2=0x563056922ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.163+0000 7f7929995b80 1 -- 172.21.15.19:0/529454907 shutdown_connections 2024-05-09T02:26:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.163+0000 7f7929995b80 1 -- 172.21.15.19:0/529454907 wait complete. 2024-05-09T02:26:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.163+0000 7f7929995b80 1 -- start start 2024-05-09T02:26:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.164+0000 7f7929995b80 1 Processor -- start 2024-05-09T02:26:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.164+0000 7f7929995b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 0x563056996a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.165+0000 7f7929995b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630567fd6a0 0x563056996f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.316 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:26:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.165+0000 7f7918d1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 0x563056996a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.165+0000 7f7918d1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 0x563056996a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:36820/0 (socket says 172.21.15.19:36820) 2024-05-09T02:26:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.165+0000 7f7918d1b700 1 -- 172.21.15.19:0/1795019377 learned_addr learned my addr 172.21.15.19:0/1795019377 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.165+0000 7f7929995b80 1 --2- 172.21.15.19:0/1795019377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563056887160 0x563056936b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.165+0000 7f7929995b80 1 -- 172.21.15.19:0/1795019377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79141b2270 con 0x5630567fd6a0 2024-05-09T02:26:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.165+0000 7f7929995b80 1 -- 172.21.15.19:0/1795019377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f79141b1ba0 con 0x56305679d4a0 2024-05-09T02:26:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.165+0000 7f791951c700 1 --2- 172.21.15.19:0/1795019377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563056887160 0x563056936b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.166+0000 7f7929995b80 1 -- 172.21.15.19:0/1795019377 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79141aedc0 con 0x563056887160 2024-05-09T02:26:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.166+0000 7f7929995b80 10 client.0 _ll_register_callbacks cb 0x5630566ae6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:26:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.166+0000 7f7918d1b700 1 -- 172.21.15.19:0/1795019377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563056887160 msgr2=0x563056936b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.166+0000 7f7918d1b700 1 --2- 172.21.15.19:0/1795019377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563056887160 0x563056936b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.166+0000 7f7918d1b700 1 -- 172.21.15.19:0/1795019377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630567fd6a0 msgr2=0x563056996f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.166+0000 7f7918d1b700 1 --2- 172.21.15.19:0/1795019377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630567fd6a0 0x563056996f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.166+0000 7f7918d1b700 1 -- 172.21.15.19:0/1795019377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56305690ef60 con 0x56305679d4a0 2024-05-09T02:26:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.167+0000 7f7918d1b700 1 --2- 172.21.15.19:0/1795019377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 0x563056996a10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7908005950 tx=0x7f7908003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c920874248a13ddf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.167+0000 7f7929995b80 1 -- 172.21.15.19:0/1795019377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f790c004db0 con 0x56305679d4a0 2024-05-09T02:26:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.167+0000 7f78f3fff700 10 client.26849 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:26:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.167+0000 7f78f3fff700 1 -- 172.21.15.19:0/1795019377 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908040020 con 0x56305679d4a0 2024-05-09T02:26:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.167+0000 7f78f3fff700 1 -- 172.21.15.19:0/1795019377 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f7908026d00 con 0x56305679d4a0 2024-05-09T02:26:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.168+0000 7f78f3fff700 1 -- 172.21.15.19:0/1795019377 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f790804b050 con 0x56305679d4a0 2024-05-09T02:26:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.170+0000 7f7929995b80 10 client.26849 fetch_fsmap learned FSMap version 58 2024-05-09T02:26:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.170+0000 7f7929995b80 1 -- 172.21.15.19:0/1795019377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=58}) -- 0x56305699ddb0 con 0x56305679d4a0 2024-05-09T02:26:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.171+0000 7f78f3fff700 1 -- 172.21.15.19:0/1795019377 <== mon.2 v2:172.21.15.59:3300/0 4 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f790801e3f0 con 0x56305679d4a0 2024-05-09T02:26:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.171+0000 7f7929995b80 10 client.26849 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:26:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.171+0000 7f7929995b80 10 client.26849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:26:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.171+0000 7f7929995b80 20 client.26849 populate_metadata read hostname 'smithi019' 2024-05-09T02:26:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.171+0000 7f7929995b80 1 --2- 172.21.15.19:0/1795019377 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56305691a6a0 0x5630569a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.171+0000 7f7929995b80 4 client.26849 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:26:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.171+0000 7f7929995b80 1 -- 172.21.15.19:0/1795019377 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f790c004db0 con 0x56305691a6a0 2024-05-09T02:26:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.174+0000 7f7913fff700 1 --2- 172.21.15.19:0/1795019377 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56305691a6a0 0x5630569a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.182+0000 7f7913fff700 1 --2- 172.21.15.19:0/1795019377 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56305691a6a0 0x5630569a02d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e39128dd0ff5df server_cookie=5ed0220ee998ed30 in_seq=0 out_seq=0 2024-05-09T02:26:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.182+0000 7f78f3fff700 10 client.26849 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.186+0000 7f78f3fff700 1 -- 172.21.15.19:0/1795019377 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f790c004db0 con 0x56305691a6a0 2024-05-09T02:26:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.186+0000 7f78f3fff700 10 client.26849 handle_command_reply: tid=0 2024-05-09T02:26:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.186+0000 7f7929995b80 1 client.26849 shutdown 2024-05-09T02:26:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.193+0000 7f7929995b80 1 -- 172.21.15.19:0/1795019377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 msgr2=0x563056996a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.193+0000 7f7929995b80 1 --2- 172.21.15.19:0/1795019377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 0x563056996a10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7908005950 tx=0x7f7908003a10 comp rx=0 tx=0).stop 2024-05-09T02:26:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.197+0000 7f7929995b80 1 -- 172.21.15.19:0/1795019377 shutdown_connections 2024-05-09T02:26:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.197+0000 7f7929995b80 1 --2- 172.21.15.19:0/1795019377 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x56305691a6a0 0x5630569a02d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.197+0000 7f7929995b80 1 --2- 172.21.15.19:0/1795019377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563056887160 0x563056936b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.197+0000 7f7929995b80 1 --2- 172.21.15.19:0/1795019377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56305679d4a0 0x563056996a10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.197+0000 7f7929995b80 1 --2- 172.21.15.19:0/1795019377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630567fd6a0 0x563056996f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.197+0000 7f7929995b80 1 -- 172.21.15.19:0/1795019377 >> 172.21.15.19:0/1795019377 conn(0x56305691ae10 msgr2=0x56305691fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.197+0000 7f78f3fff700 0 client.26849 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.198+0000 7f7929995b80 1 -- 172.21.15.19:0/1795019377 shutdown_connections 2024-05-09T02:26:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.199+0000 7f7929995b80 1 -- 172.21.15.19:0/1795019377 wait complete. 2024-05-09T02:26:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.201+0000 7f7929995b80 20 client.26849 trim_cache size 0 max 16384 2024-05-09T02:26:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.205+0000 7f791af7f700 1 -- 172.21.15.19:0/2860288669 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79040b2a70 msgr2=0x7f79040b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.205+0000 7f791af7f700 1 --2- 172.21.15.19:0/2860288669 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79040b2a70 0x7f79040b4f80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7908011490 tx=0x7f7908009260 comp rx=0 tx=0).stop 2024-05-09T02:26:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.205+0000 7f791af7f700 1 -- 172.21.15.19:0/2860288669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7914108f10 msgr2=0x7f791419d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.205+0000 7f791af7f700 1 --2- 172.21.15.19:0/2860288669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7914108f10 0x7f791419d740 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f79000098e0 tx=0x7f79000045e0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.206+0000 7f791af7f700 1 -- 172.21.15.19:0/2860288669 shutdown_connections 2024-05-09T02:26:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.206+0000 7f791af7f700 1 --2- 172.21.15.19:0/2860288669 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f79040b2a70 0x7f79040b4f80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.206+0000 7f791af7f700 1 --2- 172.21.15.19:0/2860288669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7914109860 0x7f79141a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.206+0000 7f791af7f700 1 --2- 172.21.15.19:0/2860288669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79141045e0 0x7f791419d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.206+0000 7f791af7f700 1 --2- 172.21.15.19:0/2860288669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7914108f10 0x7f791419d740 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:06.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.206+0000 7f791af7f700 1 -- 172.21.15.19:0/2860288669 >> 172.21.15.19:0/2860288669 conn(0x7f79140ffe70 msgr2=0x7f7914107270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:06.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.206+0000 7f791af7f700 1 -- 172.21.15.19:0/2860288669 shutdown_connections 2024-05-09T02:26:06.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:06.207+0000 7f791af7f700 1 -- 172.21.15.19:0/2860288669 wait complete. 2024-05-09T02:26:06.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:06 smithi059 ceph-mon[34781]: pgmap v463: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2024-05-09T02:26:07.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:06 smithi196 ceph-mon[34718]: pgmap v463: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2024-05-09T02:26:07.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:06 smithi019 ceph-mon[30151]: pgmap v463: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2024-05-09T02:26:07.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 865.0 s, 984.8 tps, lat 99.770 ms stddev 133.743 2024-05-09T02:26:07.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:07 smithi059 ceph-mon[34781]: osdmap e99: 12 total, 12 up, 12 in 2024-05-09T02:26:07.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:07 smithi059 ceph-mon[34781]: pgmap v465: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 15 MiB/s wr, 1.37k op/s 2024-05-09T02:26:08.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:07 smithi196 ceph-mon[34718]: osdmap e99: 12 total, 12 up, 12 in 2024-05-09T02:26:08.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:07 smithi196 ceph-mon[34718]: pgmap v465: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 15 MiB/s wr, 1.37k op/s 2024-05-09T02:26:08.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:07 smithi019 ceph-mon[30151]: osdmap e99: 12 total, 12 up, 12 in 2024-05-09T02:26:08.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:07 smithi019 ceph-mon[30151]: pgmap v465: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 15 MiB/s wr, 1.37k op/s 2024-05-09T02:26:08.184 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T02:26:08.184 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T02:26:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:26:10.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:10 smithi019 ceph-mon[30151]: pgmap v466: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 15 MiB/s wr, 1.39k op/s 2024-05-09T02:26:10.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:10 smithi059 ceph-mon[34781]: pgmap v466: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 15 MiB/s wr, 1.39k op/s 2024-05-09T02:26:11.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:10 smithi196 ceph-mon[34718]: pgmap v466: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 15 MiB/s wr, 1.39k op/s 2024-05-09T02:26:11.877 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:11 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:26:11.878 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:11 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.001+0000 7f70002d2700 1 -- 172.21.15.19:0/2152638726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ff8110990 msgr2=0x7f6ff8110da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.001+0000 7f70002d2700 1 --2- 172.21.15.19:0/2152638726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ff8110990 0x7f6ff8110da0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009990 tx=0x7f6fe8009ca0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.002+0000 7f70002d2700 1 -- 172.21.15.19:0/2152638726 shutdown_connections 2024-05-09T02:26:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.002+0000 7f70002d2700 1 --2- 172.21.15.19:0/2152638726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ff810b2b0 0x7f6ff8117830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.002+0000 7f70002d2700 1 --2- 172.21.15.19:0/2152638726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ff8110990 0x7f6ff8110da0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.002+0000 7f70002d2700 1 --2- 172.21.15.19:0/2152638726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff810a960 0x7f6ff810ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.002+0000 7f70002d2700 1 -- 172.21.15.19:0/2152638726 >> 172.21.15.19:0/2152638726 conn(0x7f6ff8076d30 msgr2=0x7f6ff8077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.007+0000 7f70002d2700 1 -- 172.21.15.19:0/2152638726 shutdown_connections 2024-05-09T02:26:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.007+0000 7f70002d2700 1 -- 172.21.15.19:0/2152638726 wait complete. 2024-05-09T02:26:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.008+0000 7f70002d2700 1 Processor -- start 2024-05-09T02:26:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.009+0000 7f70002d2700 1 -- start start 2024-05-09T02:26:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.009+0000 7f70002d2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ff810a960 0x7f6ff81a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.009+0000 7f70002d2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff810b2b0 0x7f6ff81a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.009+0000 7f70002d2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ff8110990 0x7f6ff81a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.009+0000 7f70002d2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff811ac50 con 0x7f6ff810b2b0 2024-05-09T02:26:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.009+0000 7f70002d2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff811aad0 con 0x7f6ff8110990 2024-05-09T02:26:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.009+0000 7f70002d2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff811add0 con 0x7f6ff810a960 2024-05-09T02:26:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.010+0000 7f6ffe86f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ff8110990 0x7f6ff81a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.010+0000 7f6ffe86f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ff8110990 0x7f6ff81a5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:42414/0 (socket says 172.21.15.19:42414) 2024-05-09T02:26:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.010+0000 7f6ffe86f700 1 -- 172.21.15.19:0/1867879387 learned_addr learned my addr 172.21.15.19:0/1867879387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.010+0000 7f6ffd86d700 1 --2- 172.21.15.19:0/1867879387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff810b2b0 0x7f6ff81a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.010+0000 7f6ffe06e700 1 --2- 172.21.15.19:0/1867879387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ff810a960 0x7f6ff81a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.010+0000 7f6ffe86f700 1 -- 172.21.15.19:0/1867879387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ff810a960 msgr2=0x7f6ff81a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.010+0000 7f6ffe86f700 1 --2- 172.21.15.19:0/1867879387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ff810a960 0x7f6ff81a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.010+0000 7f6ffe86f700 1 -- 172.21.15.19:0/1867879387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff810b2b0 msgr2=0x7f6ff81a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.010+0000 7f6ffe86f700 1 --2- 172.21.15.19:0/1867879387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff810b2b0 0x7f6ff81a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.010+0000 7f6ffe86f700 1 -- 172.21.15.19:0/1867879387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff81a64e0 con 0x7f6ff8110990 2024-05-09T02:26:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.010+0000 7f6ffd86d700 1 --2- 172.21.15.19:0/1867879387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff810b2b0 0x7f6ff81a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:26:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.011+0000 7f6ffe86f700 1 --2- 172.21.15.19:0/1867879387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ff8110990 0x7f6ff81a5e40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000b780 tx=0x7f6ff000bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.011+0000 7f6fef7fe700 1 -- 172.21.15.19:0/1867879387 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff000f840 con 0x7f6ff8110990 2024-05-09T02:26:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.011+0000 7f6fef7fe700 1 -- 172.21.15.19:0/1867879387 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff0017400 con 0x7f6ff8110990 2024-05-09T02:26:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.011+0000 7f6fef7fe700 1 -- 172.21.15.19:0/1867879387 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff000d610 con 0x7f6ff8110990 2024-05-09T02:26:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.011+0000 7f70002d2700 1 -- 172.21.15.19:0/1867879387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff81a67d0 con 0x7f6ff8110990 2024-05-09T02:26:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.012+0000 7f70002d2700 1 -- 172.21.15.19:0/1867879387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff81a6c60 con 0x7f6ff8110990 2024-05-09T02:26:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.018+0000 7f700ece8b80 1 Processor -- start 2024-05-09T02:26:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.022+0000 7f700ece8b80 1 -- start start 2024-05-09T02:26:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.023+0000 7f700ece8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 0x556bb41792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.023+0000 7f700ece8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff6520 0x556bb4179800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.023+0000 7f700ece8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb4173480 0x556bb4173890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.023+0000 7f700ece8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556bb4096120 con 0x556bb4173480 2024-05-09T02:26:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.023+0000 7f700ece8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556bb403ba20 con 0x556bb3ff6520 2024-05-09T02:26:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.023+0000 7f700ece8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556bb40920e0 con 0x556bb4070d80 2024-05-09T02:26:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.023+0000 7f6fef7fe700 1 -- 172.21.15.19:0/1867879387 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6ff00175b0 con 0x7f6ff8110990 2024-05-09T02:26:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.024+0000 7f6ffe06e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff6520 0x556bb4179800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.024+0000 7f6ffd86d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 0x556bb41792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.024+0000 7f6ffe06e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff6520 0x556bb4179800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:42428/0 (socket says 172.21.15.19:42428) 2024-05-09T02:26:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.024+0000 7f6ffe06e700 1 -- 172.21.15.19:0/428244227 learned_addr learned my addr 172.21.15.19:0/428244227 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.027+0000 7f6ffd86d700 1 --2- 172.21.15.19:0/428244227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb4173480 0x556bb4173890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.028+0000 7f6ffe06e700 1 -- 172.21.15.19:0/428244227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 msgr2=0x556bb41792c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.028+0000 7f6ffe06e700 1 --2- 172.21.15.19:0/428244227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 0x556bb41792c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.028+0000 7f6ffe06e700 1 -- 172.21.15.19:0/428244227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb4173480 msgr2=0x556bb4173890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.028+0000 7f6ffe06e700 1 --2- 172.21.15.19:0/428244227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb4173480 0x556bb4173890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.028+0000 7f6ffe06e700 1 -- 172.21.15.19:0/428244227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bb402c580 con 0x556bb3ff6520 2024-05-09T02:26:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.028+0000 7f6ffe06e700 1 --2- 172.21.15.19:0/428244227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff6520 0x556bb4179800 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009f60 tx=0x7f6fe8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f48228bcc87c4a93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.029+0000 7f6fef7fe700 1 --2- 172.21.15.19:0/1867879387 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6fe40b2b20 0x7f6fe40b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.029+0000 7f6ffd86d700 1 --2- 172.21.15.19:0/1867879387 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6fe40b2b20 0x7f6fe40b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.029+0000 7f6fecd47700 1 -- 172.21.15.19:0/428244227 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe80038f0 con 0x556bb3ff6520 2024-05-09T02:26:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.030+0000 7f6ffd86d700 1 --2- 172.21.15.19:0/1867879387 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6fe40b2b20 0x7f6fe40b5030 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4006ea0 tx=0x7f6ff4006e30 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.030+0000 7f6fef7fe700 1 -- 172.21.15.19:0/1867879387 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8615+0+0 (secure 0 0 0) 0x7f6ff00e2480 con 0x7f6ff8110990 2024-05-09T02:26:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.030+0000 7f6fecd47700 1 -- 172.21.15.19:0/428244227 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe80050d0 con 0x556bb3ff6520 2024-05-09T02:26:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.030+0000 7f6fecd47700 1 -- 172.21.15.19:0/428244227 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe800bbf0 con 0x556bb3ff6520 2024-05-09T02:26:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.031+0000 7f700ece8b80 1 -- 172.21.15.19:0/428244227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff6520 msgr2=0x556bb4179800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.031+0000 7f700ece8b80 1 --2- 172.21.15.19:0/428244227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff6520 0x556bb4179800 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009f60 tx=0x7f6fe8009f90 comp rx=0 tx=0).stop 2024-05-09T02:26:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.032+0000 7f700ece8b80 1 -- 172.21.15.19:0/428244227 shutdown_connections 2024-05-09T02:26:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.032+0000 7f700ece8b80 1 --2- 172.21.15.19:0/428244227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb4173480 0x556bb4173890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.032+0000 7f700ece8b80 1 --2- 172.21.15.19:0/428244227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 0x556bb41792c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.032+0000 7f700ece8b80 1 --2- 172.21.15.19:0/428244227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff6520 0x556bb4179800 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.032+0000 7f700ece8b80 1 -- 172.21.15.19:0/428244227 >> 172.21.15.19:0/428244227 conn(0x556bb3ff9ff0 msgr2=0x556bb416e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:11.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.032+0000 7f700ece8b80 1 -- 172.21.15.19:0/428244227 shutdown_connections 2024-05-09T02:26:11.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.033+0000 7f700ece8b80 1 -- 172.21.15.19:0/428244227 wait complete. 2024-05-09T02:26:11.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.033+0000 7f700ece8b80 1 -- start start 2024-05-09T02:26:11.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.034+0000 7f700ece8b80 1 Processor -- start 2024-05-09T02:26:11.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.034+0000 7f700ece8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff6520 0x556bb417d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.034+0000 7f700ece8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 0x556bb417d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.034+0000 7f700ece8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb4173480 0x556bb417da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.034+0000 7f700ece8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556bb4174180 con 0x556bb4173480 2024-05-09T02:26:11.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.034+0000 7f700ece8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556bb4030ba0 con 0x556bb3ff6520 2024-05-09T02:26:11.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.034+0000 7f700ece8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556bb41ab300 con 0x556bb4070d80 2024-05-09T02:26:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.034+0000 7f700ece8b80 10 client.0 _ll_register_callbacks cb 0x556bb3f07740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:26:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.035+0000 7f6ffd86d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 0x556bb417d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.035+0000 7f6ffe06e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff6520 0x556bb417d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.035+0000 7f6ffd86d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 0x556bb417d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:48202/0 (socket says 172.21.15.19:48202) 2024-05-09T02:26:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.035+0000 7f6ffd86d700 1 -- 172.21.15.19:0/3359619480 learned_addr learned my addr 172.21.15.19:0/3359619480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.036+0000 7f6ffd86d700 1 -- 172.21.15.19:0/3359619480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff6520 msgr2=0x556bb417d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.036+0000 7f6ffd86d700 1 --2- 172.21.15.19:0/3359619480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff6520 0x556bb417d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.036+0000 7f6ffd86d700 1 -- 172.21.15.19:0/3359619480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb4173480 msgr2=0x556bb417da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.036+0000 7f6ffd86d700 1 --2- 172.21.15.19:0/3359619480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb4173480 0x556bb417da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.036+0000 7f6ffd86d700 1 -- 172.21.15.19:0/3359619480 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bb3f11510 con 0x556bb4070d80 2024-05-09T02:26:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.036+0000 7f6ffd86d700 1 --2- 172.21.15.19:0/3359619480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 0x556bb417d550 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4018de0 tx=0x7f6ff4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca2b695e277aef09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.036+0000 7f6fe1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:26:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.037+0000 7f6fe1ffb700 1 -- 172.21.15.19:0/3359619480 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4028060 con 0x556bb4070d80 2024-05-09T02:26:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.037+0000 7f6fe1ffb700 1 -- 172.21.15.19:0/3359619480 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8615+0+0 (secure 0 0 0) 0x7f6ff4016c10 con 0x556bb4070d80 2024-05-09T02:26:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.038+0000 7f700ece8b80 1 -- 172.21.15.19:0/3359619480 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fe80050d0 con 0x556bb4070d80 2024-05-09T02:26:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.042+0000 7f6fe1ffb700 1 -- 172.21.15.19:0/3359619480 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6ff400f040 con 0x556bb4070d80 2024-05-09T02:26:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.042+0000 7f700ece8b80 10 client.26929 fetch_fsmap learned FSMap version 58 2024-05-09T02:26:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.042+0000 7f700ece8b80 1 -- 172.21.15.19:0/3359619480 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556bb41acc00 con 0x556bb4070d80 2024-05-09T02:26:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.043+0000 7f6fe1ffb700 1 -- 172.21.15.19:0/3359619480 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe80050d0 con 0x556bb4070d80 2024-05-09T02:26:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.043+0000 7f6fe1ffb700 1 -- 172.21.15.19:0/3359619480 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f6ff4015040 con 0x556bb4070d80 2024-05-09T02:26:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.043+0000 7f700ece8b80 10 client.26929 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:26:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.043+0000 7f700ece8b80 10 client.26929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:26:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.043+0000 7f700ece8b80 20 client.26929 populate_metadata read hostname 'smithi019' 2024-05-09T02:26:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.043+0000 7f700ece8b80 1 --2- 172.21.15.19:0/3359619480 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556bb41af240 0x556bb41b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.043+0000 7f700ece8b80 4 client.26929 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:26:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.043+0000 7f700ece8b80 1 -- 172.21.15.19:0/3359619480 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bb40a3b00 con 0x556bb41af240 2024-05-09T02:26:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.043+0000 7f6ffe06e700 1 --2- 172.21.15.19:0/3359619480 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556bb41af240 0x556bb41b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.044+0000 7f6fe1ffb700 10 client.26929 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.045+0000 7f6ffe06e700 1 --2- 172.21.15.19:0/3359619480 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556bb41af240 0x556bb41b1680 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc7e09de1b7ed66b server_cookie=371b738dfa11ccdf in_seq=0 out_seq=0 2024-05-09T02:26:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.046+0000 7f6fe1ffb700 1 -- 172.21.15.19:0/3359619480 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556bb40a3b00 con 0x556bb41af240 2024-05-09T02:26:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.046+0000 7f6fe1ffb700 10 client.26929 handle_command_reply: tid=0 2024-05-09T02:26:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.046+0000 7f700ece8b80 1 client.26929 shutdown 2024-05-09T02:26:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.047+0000 7f700ece8b80 1 -- 172.21.15.19:0/3359619480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 msgr2=0x556bb417d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.047+0000 7f700ece8b80 1 --2- 172.21.15.19:0/3359619480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 0x556bb417d550 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4018de0 tx=0x7f6ff4004750 comp rx=0 tx=0).stop 2024-05-09T02:26:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.047+0000 7f700ece8b80 1 -- 172.21.15.19:0/3359619480 shutdown_connections 2024-05-09T02:26:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.047+0000 7f700ece8b80 1 --2- 172.21.15.19:0/3359619480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb4173480 0x556bb417da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.047+0000 7f700ece8b80 1 --2- 172.21.15.19:0/3359619480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff6520 0x556bb417d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.047+0000 7f700ece8b80 1 --2- 172.21.15.19:0/3359619480 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556bb41af240 0x556bb41b1680 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.047+0000 7f700ece8b80 1 --2- 172.21.15.19:0/3359619480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 0x556bb417d550 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.047+0000 7f700ece8b80 1 -- 172.21.15.19:0/3359619480 >> 172.21.15.19:0/3359619480 conn(0x556bb3ff9ff0 msgr2=0x556bb4168070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.047+0000 7f6fe1ffb700 0 client.26929 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.047+0000 7f700ece8b80 1 -- 172.21.15.19:0/3359619480 shutdown_connections 2024-05-09T02:26:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.047+0000 7f700ece8b80 1 -- 172.21.15.19:0/3359619480 wait complete. 2024-05-09T02:26:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.047+0000 7f700ece8b80 20 client.26929 trim_cache size 0 max 16384 2024-05-09T02:26:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.082+0000 7f700ece8b80 1 Processor -- start 2024-05-09T02:26:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.082+0000 7f700ece8b80 1 -- start start 2024-05-09T02:26:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.082+0000 7f700ece8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff9ff0 0x556bb4177780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.082+0000 7f700ece8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb3ff6520 0x556bb4179930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.083+0000 7f700ece8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 0x556bb4168780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.083+0000 7f700ece8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc00d800 con 0x556bb3ff6520 2024-05-09T02:26:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.083+0000 7f700ece8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc00f340 con 0x556bb3ff9ff0 2024-05-09T02:26:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.083+0000 7f700ece8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc00ded0 con 0x556bb4070d80 2024-05-09T02:26:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.083+0000 7f6ffe06e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff9ff0 0x556bb4177780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.083+0000 7f6ffe06e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff9ff0 0x556bb4177780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:42446/0 (socket says 172.21.15.19:42446) 2024-05-09T02:26:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.083+0000 7f6ffe06e700 1 -- 172.21.15.19:0/652899791 learned_addr learned my addr 172.21.15.19:0/652899791 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.083+0000 7f6ffd86d700 1 --2- 172.21.15.19:0/652899791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb3ff6520 0x556bb4179930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.083+0000 7f6ffe86f700 1 --2- 172.21.15.19:0/652899791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 0x556bb4168780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.083+0000 7f6ffe06e700 1 -- 172.21.15.19:0/652899791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 msgr2=0x556bb4168780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:11.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.083+0000 7f6ffe06e700 1 --2- 172.21.15.19:0/652899791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 0x556bb4168780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.083+0000 7f6ffe06e700 1 -- 172.21.15.19:0/652899791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb3ff6520 msgr2=0x556bb4179930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:11.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.083+0000 7f6ffe06e700 1 --2- 172.21.15.19:0/652899791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb3ff6520 0x556bb4179930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.083+0000 7f6ffe06e700 1 -- 172.21.15.19:0/652899791 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bb416fb20 con 0x556bb3ff9ff0 2024-05-09T02:26:11.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.084+0000 7f6ffe06e700 1 --2- 172.21.15.19:0/652899791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff9ff0 0x556bb4177780 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8003680 tx=0x7f6fe8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52fb8cedb5d5e284 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:11.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.084+0000 7f6ffd86d700 1 --2- 172.21.15.19:0/652899791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb3ff6520 0x556bb4179930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:26:11.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.084+0000 7f6fe3fff700 1 -- 172.21.15.19:0/652899791 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe801b040 con 0x556bb3ff9ff0 2024-05-09T02:26:11.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.084+0000 7f6fe3fff700 1 -- 172.21.15.19:0/652899791 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe802fe50 con 0x556bb3ff9ff0 2024-05-09T02:26:11.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.084+0000 7f6fe3fff700 1 -- 172.21.15.19:0/652899791 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe800baa0 con 0x556bb3ff9ff0 2024-05-09T02:26:11.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.085+0000 7f700ece8b80 1 -- 172.21.15.19:0/652899791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff9ff0 msgr2=0x556bb4177780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.085+0000 7f700ece8b80 1 --2- 172.21.15.19:0/652899791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff9ff0 0x556bb4177780 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8003680 tx=0x7f6fe8005fd0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.085+0000 7f700ece8b80 1 -- 172.21.15.19:0/652899791 shutdown_connections 2024-05-09T02:26:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.085+0000 7f700ece8b80 1 --2- 172.21.15.19:0/652899791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb4070d80 0x556bb4168780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.085+0000 7f700ece8b80 1 --2- 172.21.15.19:0/652899791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb3ff9ff0 0x556bb4177780 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.085+0000 7f700ece8b80 1 --2- 172.21.15.19:0/652899791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb3ff6520 0x556bb4179930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.085+0000 7f700ece8b80 1 -- 172.21.15.19:0/652899791 >> 172.21.15.19:0/652899791 conn(0x556bb4173b60 msgr2=0x556bb4168070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.086+0000 7f700ece8b80 1 -- 172.21.15.19:0/652899791 shutdown_connections 2024-05-09T02:26:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.086+0000 7f700ece8b80 1 -- 172.21.15.19:0/652899791 wait complete. 2024-05-09T02:26:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.086+0000 7f700ece8b80 1 -- start start 2024-05-09T02:26:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.086+0000 7f700ece8b80 1 Processor -- start 2024-05-09T02:26:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f700ece8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb3ff6520 0x556bb4196530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f700ece8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb3ff9ff0 0x556bb4193220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f700ece8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb4070d80 0x556bb4193760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f700ece8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc00ec70 con 0x556bb3ff6520 2024-05-09T02:26:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f700ece8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc00e5a0 con 0x556bb4070d80 2024-05-09T02:26:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f700ece8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc00b7c0 con 0x556bb3ff9ff0 2024-05-09T02:26:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f700ece8b80 10 client.0 _ll_register_callbacks cb 0x556bb3f07740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:26:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f6ffe86f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb4070d80 0x556bb4193760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f6ffd86d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb3ff9ff0 0x556bb4193220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f6ffe86f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb4070d80 0x556bb4193760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:42458/0 (socket says 172.21.15.19:42458) 2024-05-09T02:26:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f6ffd86d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb3ff9ff0 0x556bb4193220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:48224/0 (socket says 172.21.15.19:48224) 2024-05-09T02:26:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f6ffe86f700 1 -- 172.21.15.19:0/3067496166 learned_addr learned my addr 172.21.15.19:0/3067496166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f6ffe06e700 1 --2- 172.21.15.19:0/3067496166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb3ff6520 0x556bb4196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f6ffd86d700 1 -- 172.21.15.19:0/3067496166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb4070d80 msgr2=0x556bb4193760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f6ffd86d700 1 --2- 172.21.15.19:0/3067496166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb4070d80 0x556bb4193760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f6ffd86d700 1 -- 172.21.15.19:0/3067496166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb3ff6520 msgr2=0x556bb4196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f6ffd86d700 1 --2- 172.21.15.19:0/3067496166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb3ff6520 0x556bb4196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f6ffd86d700 1 -- 172.21.15.19:0/3067496166 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bb4193ca0 con 0x556bb3ff9ff0 2024-05-09T02:26:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f6ffe86f700 1 --2- 172.21.15.19:0/3067496166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb4070d80 0x556bb4193760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:26:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f6ffe06e700 1 --2- 172.21.15.19:0/3067496166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb3ff6520 0x556bb4196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:26:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f6ffd86d700 1 --2- 172.21.15.19:0/3067496166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb3ff9ff0 0x556bb4193220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x556bb417e060 tx=0x7f6ff402cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f60405c0ac1f3b47 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.087+0000 7f6fe1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:26:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.088+0000 7f6fe1ffb700 1 -- 172.21.15.19:0/3067496166 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff402ce80 con 0x556bb3ff9ff0 2024-05-09T02:26:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.088+0000 7f6fe1ffb700 1 -- 172.21.15.19:0/3067496166 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8615+0+0 (secure 0 0 0) 0x7f6ff400b950 con 0x556bb3ff9ff0 2024-05-09T02:26:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.088+0000 7f700ece8b80 1 -- 172.21.15.19:0/3067496166 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fe802fe50 con 0x556bb3ff9ff0 2024-05-09T02:26:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.088+0000 7f6fe1ffb700 1 -- 172.21.15.19:0/3067496166 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6ff4016a10 con 0x556bb3ff9ff0 2024-05-09T02:26:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.088+0000 7f700ece8b80 10 client.26935 fetch_fsmap learned FSMap version 58 2024-05-09T02:26:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.088+0000 7f700ece8b80 1 -- 172.21.15.19:0/3067496166 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556bb421a750 con 0x556bb3ff9ff0 2024-05-09T02:26:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.088+0000 7f6fe1ffb700 1 -- 172.21.15.19:0/3067496166 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe802fe50 con 0x556bb3ff9ff0 2024-05-09T02:26:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.088+0000 7f6fe1ffb700 1 -- 172.21.15.19:0/3067496166 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f6ff4015040 con 0x556bb3ff9ff0 2024-05-09T02:26:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.088+0000 7f700ece8b80 10 client.26935 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:26:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.088+0000 7f700ece8b80 10 client.26935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:26:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.088+0000 7f700ece8b80 20 client.26935 populate_metadata read hostname 'smithi019' 2024-05-09T02:26:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.088+0000 7f700ece8b80 1 --2- 172.21.15.19:0/3067496166 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556bb4173480 0x556bb421ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.088+0000 7f700ece8b80 4 client.26935 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T02:26:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.088+0000 7f700ece8b80 1 -- 172.21.15.19:0/3067496166 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556bb409dba0 con 0x556bb4173480 2024-05-09T02:26:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.089+0000 7f6ffe06e700 1 --2- 172.21.15.19:0/3067496166 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556bb4173480 0x556bb421ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.089+0000 7f6ffe06e700 1 --2- 172.21.15.19:0/3067496166 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556bb4173480 0x556bb421ce60 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ba30000e61fe7e2 server_cookie=ccc5b2855b834da0 in_seq=0 out_seq=0 2024-05-09T02:26:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.089+0000 7f6fe1ffb700 10 client.26935 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:11.896 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:26:11.896 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-09T02:26:11.896 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "af77a585-5518-4755-b241-b5103f8d38aa", 2024-05-09T02:26:11.897 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-09T02:26:11.897 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:26:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.090+0000 7f6fe1ffb700 1 -- 172.21.15.19:0/3067496166 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556bb409dba0 con 0x556bb4173480 2024-05-09T02:26:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.090+0000 7f6fe1ffb700 10 client.26935 handle_command_reply: tid=0 2024-05-09T02:26:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.090+0000 7f700ece8b80 1 client.26935 shutdown 2024-05-09T02:26:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.091+0000 7f700ece8b80 1 -- 172.21.15.19:0/3067496166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb3ff9ff0 msgr2=0x556bb4193220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.091+0000 7f700ece8b80 1 --2- 172.21.15.19:0/3067496166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb3ff9ff0 0x556bb4193220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x556bb417e060 tx=0x7f6ff402cd60 comp rx=0 tx=0).stop 2024-05-09T02:26:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.091+0000 7f700ece8b80 1 -- 172.21.15.19:0/3067496166 shutdown_connections 2024-05-09T02:26:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.091+0000 7f700ece8b80 1 --2- 172.21.15.19:0/3067496166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bb4070d80 0x556bb4193760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.091+0000 7f700ece8b80 1 --2- 172.21.15.19:0/3067496166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556bb3ff6520 0x556bb4196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.091+0000 7f700ece8b80 1 --2- 172.21.15.19:0/3067496166 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556bb4173480 0x556bb421ce60 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.092+0000 7f700ece8b80 1 --2- 172.21.15.19:0/3067496166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bb3ff9ff0 0x556bb4193220 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.092+0000 7f6fe1ffb700 0 client.26935 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.092+0000 7f700ece8b80 1 -- 172.21.15.19:0/3067496166 >> 172.21.15.19:0/3067496166 conn(0x556bb4173b60 msgr2=0x556bb41774e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.092+0000 7f700ece8b80 1 -- 172.21.15.19:0/3067496166 shutdown_connections 2024-05-09T02:26:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.092+0000 7f700ece8b80 1 -- 172.21.15.19:0/3067496166 wait complete. 2024-05-09T02:26:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.092+0000 7f700ece8b80 20 client.26935 trim_cache size 0 max 16384 2024-05-09T02:26:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.093+0000 7f70002d2700 1 -- 172.21.15.19:0/1867879387 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6fe40b2b20 msgr2=0x7f6fe40b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.093+0000 7f70002d2700 1 --2- 172.21.15.19:0/1867879387 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6fe40b2b20 0x7f6fe40b5030 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4006ea0 tx=0x7f6ff4006e30 comp rx=0 tx=0).stop 2024-05-09T02:26:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.093+0000 7f70002d2700 1 -- 172.21.15.19:0/1867879387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ff8110990 msgr2=0x7f6ff81a5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.093+0000 7f70002d2700 1 --2- 172.21.15.19:0/1867879387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ff8110990 0x7f6ff81a5e40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000b780 tx=0x7f6ff000bb40 comp rx=0 tx=0).stop 2024-05-09T02:26:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.093+0000 7f70002d2700 1 -- 172.21.15.19:0/1867879387 shutdown_connections 2024-05-09T02:26:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.093+0000 7f70002d2700 1 --2- 172.21.15.19:0/1867879387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ff8110990 0x7f6ff81a5e40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.093+0000 7f70002d2700 1 --2- 172.21.15.19:0/1867879387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ff810a960 0x7f6ff81a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.093+0000 7f70002d2700 1 --2- 172.21.15.19:0/1867879387 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f6fe40b2b20 0x7f6fe40b5030 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.093+0000 7f70002d2700 1 --2- 172.21.15.19:0/1867879387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff810b2b0 0x7f6ff81a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.093+0000 7f70002d2700 1 -- 172.21.15.19:0/1867879387 >> 172.21.15.19:0/1867879387 conn(0x7f6ff8076d30 msgr2=0x7f6ff81092f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.093+0000 7f70002d2700 1 -- 172.21.15.19:0/1867879387 shutdown_connections 2024-05-09T02:26:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:11.093+0000 7f70002d2700 1 -- 172.21.15.19:0/1867879387 wait complete. 2024-05-09T02:26:11.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:11 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:26:11.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:11 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:11 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:26:12.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:11 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 870.0 s, 1498.4 tps, lat 66.678 ms stddev 75.629 2024-05-09T02:26:12.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub queued for path: / 2024-05-09T02:26:12.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:26:12.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:26:12.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:26:12.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:26:12.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:26:12.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:26:12.886 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T02:26:12.886 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.886 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.886 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.886 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.886 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:12 smithi019 ceph-mon[30151]: pgmap v467: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 16 MiB/s wr, 1.46k op/s 2024-05-09T02:26:12.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub queued for path: / 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub summary: active paths [/] 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:12.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:12 smithi059 ceph-mon[34781]: pgmap v467: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 16 MiB/s wr, 1.46k op/s 2024-05-09T02:26:13.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub queued for path: / 2024-05-09T02:26:13.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:13.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:26:13.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:26:13.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:13.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:26:13.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:26:13.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:26:13.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:13.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:13.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:26:13.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub summary: active paths [/] 2024-05-09T02:26:13.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:13.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:13.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:13.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: scrub summary: idle+waiting paths [/] 2024-05-09T02:26:13.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:12 smithi196 ceph-mon[34718]: pgmap v467: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 16 MiB/s wr, 1.46k op/s 2024-05-09T02:26:13.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:26:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:26:13.902 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:13 smithi019 ceph-mon[30151]: pgmap v468: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 16 MiB/s wr, 1.46k op/s 2024-05-09T02:26:13.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:13 smithi059 ceph-mon[34781]: pgmap v468: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 16 MiB/s wr, 1.46k op/s 2024-05-09T02:26:14.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:13 smithi196 ceph-mon[34718]: pgmap v468: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 16 MiB/s wr, 1.46k op/s 2024-05-09T02:26:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.702+0000 7fa8dc793700 1 -- 172.21.15.19:0/1945088376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d4108740 msgr2=0x7fa8d41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.702+0000 7fa8dc793700 1 --2- 172.21.15.19:0/1945088376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d4108740 0x7fa8d41100a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00b0d0 tx=0x7fa8cc00b3e0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.703+0000 7fa8dc793700 1 -- 172.21.15.19:0/1945088376 shutdown_connections 2024-05-09T02:26:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.703+0000 7fa8dc793700 1 --2- 172.21.15.19:0/1945088376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d4108740 0x7fa8d41100a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.703+0000 7fa8dc793700 1 --2- 172.21.15.19:0/1945088376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8d41034c0 0x7fa8d41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.703+0000 7fa8dc793700 1 --2- 172.21.15.19:0/1945088376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8d4107df0 0x7fa8d4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.703+0000 7fa8dc793700 1 -- 172.21.15.19:0/1945088376 >> 172.21.15.19:0/1945088376 conn(0x7fa8d40fedc0 msgr2=0x7fa8d41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.703+0000 7fa8dc793700 1 -- 172.21.15.19:0/1945088376 shutdown_connections 2024-05-09T02:26:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.703+0000 7fa8dc793700 1 -- 172.21.15.19:0/1945088376 wait complete. 2024-05-09T02:26:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.703+0000 7fa8dc793700 1 Processor -- start 2024-05-09T02:26:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.704+0000 7fa8dc793700 1 -- start start 2024-05-09T02:26:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.704+0000 7fa8dc793700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41034c0 0x7fa8d4110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.704+0000 7fa8da52f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41034c0 0x7fa8d4110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.704+0000 7fa8dc793700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8d4107df0 0x7fa8d4110fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.704+0000 7fa8da52f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41034c0 0x7fa8d4110a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42868/0 (socket says 172.21.15.19:42868) 2024-05-09T02:26:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.704+0000 7fa8da52f700 1 -- 172.21.15.19:0/3246269431 learned_addr learned my addr 172.21.15.19:0/3246269431 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.704+0000 7fa8dc793700 1 --2- 172.21.15.19:0/3246269431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8d4108740 0x7fa8d4111500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.704+0000 7fa8dc793700 1 -- 172.21.15.19:0/3246269431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41144c0 con 0x7fa8d41034c0 2024-05-09T02:26:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.704+0000 7fa8dc793700 1 -- 172.21.15.19:0/3246269431 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d4114340 con 0x7fa8d4107df0 2024-05-09T02:26:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.704+0000 7fa8dc793700 1 -- 172.21.15.19:0/3246269431 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d4114640 con 0x7fa8d4108740 2024-05-09T02:26:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.704+0000 7fa8d9d2e700 1 --2- 172.21.15.19:0/3246269431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8d4107df0 0x7fa8d4110fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.705+0000 7fa8dad30700 1 --2- 172.21.15.19:0/3246269431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8d4108740 0x7fa8d4111500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.705+0000 7fa8da52f700 1 -- 172.21.15.19:0/3246269431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8d4108740 msgr2=0x7fa8d4111500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.705+0000 7fa8da52f700 1 --2- 172.21.15.19:0/3246269431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8d4108740 0x7fa8d4111500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.705+0000 7fa8da52f700 1 -- 172.21.15.19:0/3246269431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8d4107df0 msgr2=0x7fa8d4110fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.705+0000 7fa8da52f700 1 --2- 172.21.15.19:0/3246269431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8d4107df0 0x7fa8d4110fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.705+0000 7fa8da52f700 1 -- 172.21.15.19:0/3246269431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d41a9130 con 0x7fa8d41034c0 2024-05-09T02:26:15.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.705+0000 7fa8da52f700 1 --2- 172.21.15.19:0/3246269431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41034c0 0x7fa8d4110a80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000e9a0 tx=0x7fa8d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:15.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.706+0000 7fa8cb7fe700 1 -- 172.21.15.19:0/3246269431 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d000cbd0 con 0x7fa8d41034c0 2024-05-09T02:26:15.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.706+0000 7fa8dc793700 1 -- 172.21.15.19:0/3246269431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d41a9420 con 0x7fa8d41034c0 2024-05-09T02:26:15.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.706+0000 7fa8cb7fe700 1 -- 172.21.15.19:0/3246269431 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d000cd70 con 0x7fa8d41034c0 2024-05-09T02:26:15.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.706+0000 7fa8cb7fe700 1 -- 172.21.15.19:0/3246269431 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0018680 con 0x7fa8d41034c0 2024-05-09T02:26:15.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.706+0000 7fa8dc793700 1 -- 172.21.15.19:0/3246269431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d41a9960 con 0x7fa8d41034c0 2024-05-09T02:26:15.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.710+0000 7fa8cb7fe700 1 -- 172.21.15.19:0/3246269431 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa8d0018820 con 0x7fa8d41034c0 2024-05-09T02:26:15.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.712+0000 7fa8cb7fe700 1 --2- 172.21.15.19:0/3246269431 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8c00b29f0 0x7fa8c00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.712+0000 7fa8cb7fe700 1 -- 172.21.15.19:0/3246269431 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fa8d00e2970 con 0x7fa8d41034c0 2024-05-09T02:26:15.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.712+0000 7fa8d9d2e700 1 --2- 172.21.15.19:0/3246269431 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8c00b29f0 0x7fa8c00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.712+0000 7fa8eb1a9b80 1 Processor -- start 2024-05-09T02:26:15.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.713+0000 7fa8eb1a9b80 1 -- start start 2024-05-09T02:26:15.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.713+0000 7fa8d9d2e700 1 --2- 172.21.15.19:0/3246269431 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8c00b29f0 0x7fa8c00b4f00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4106710 tx=0x7fa8c4009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:15.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.713+0000 7fa8eb1a9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 0x555ae265b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.713+0000 7fa8eb1a9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae24d84a0 0x555ae265b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.713+0000 7fa8eb1a9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae2655400 0x555ae2655810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.713+0000 7fa8eb1a9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555ae25780a0 con 0x555ae2552d00 2024-05-09T02:26:15.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.713+0000 7fa8eb1a9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555ae251d9a0 con 0x555ae24d84a0 2024-05-09T02:26:15.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.713+0000 7fa8eb1a9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ae2574060 con 0x555ae2655400 2024-05-09T02:26:15.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.713+0000 7fa8dad30700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae2655400 0x555ae2655810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.714+0000 7fa8dad30700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae2655400 0x555ae2655810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:48250/0 (socket says 172.21.15.19:48250) 2024-05-09T02:26:15.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.714+0000 7fa8d9d2e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae24d84a0 0x555ae265b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.714+0000 7fa8dad30700 1 -- 172.21.15.19:0/807498504 learned_addr learned my addr 172.21.15.19:0/807498504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:15.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.714+0000 7fa8da52f700 1 --2- 172.21.15.19:0/807498504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 0x555ae265b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.714+0000 7fa8d9d2e700 1 -- 172.21.15.19:0/807498504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae2655400 msgr2=0x555ae2655810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:15.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.714+0000 7fa8d9d2e700 1 --2- 172.21.15.19:0/807498504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae2655400 0x555ae2655810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.714+0000 7fa8d9d2e700 1 -- 172.21.15.19:0/807498504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 msgr2=0x555ae265b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:15.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.714+0000 7fa8d9d2e700 1 --2- 172.21.15.19:0/807498504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 0x555ae265b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.714+0000 7fa8d9d2e700 1 -- 172.21.15.19:0/807498504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ae250e500 con 0x555ae24d84a0 2024-05-09T02:26:15.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.714+0000 7fa8dad30700 1 --2- 172.21.15.19:0/807498504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae2655400 0x555ae2655810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:26:15.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.714+0000 7fa8d9d2e700 1 --2- 172.21.15.19:0/807498504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae24d84a0 0x555ae265b780 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009fd0 tx=0x7fa8c4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bafbb1fb58d7f230 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:15.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.714+0000 7fa8c8ff9700 1 -- 172.21.15.19:0/807498504 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4004020 con 0x555ae24d84a0 2024-05-09T02:26:15.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.715+0000 7fa8c8ff9700 1 -- 172.21.15.19:0/807498504 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8c400e790 con 0x555ae24d84a0 2024-05-09T02:26:15.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.715+0000 7fa8c8ff9700 1 -- 172.21.15.19:0/807498504 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4003af0 con 0x555ae24d84a0 2024-05-09T02:26:15.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.715+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/807498504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae24d84a0 msgr2=0x555ae265b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.715+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/807498504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae24d84a0 0x555ae265b780 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009fd0 tx=0x7fa8c4014000 comp rx=0 tx=0).stop 2024-05-09T02:26:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.716+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/807498504 shutdown_connections 2024-05-09T02:26:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.716+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/807498504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae2655400 0x555ae2655810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.716+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/807498504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 0x555ae265b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.716+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/807498504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae24d84a0 0x555ae265b780 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.716+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/807498504 >> 172.21.15.19:0/807498504 conn(0x555ae24dbf70 msgr2=0x555ae2650220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.716+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/807498504 shutdown_connections 2024-05-09T02:26:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.716+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/807498504 wait complete. 2024-05-09T02:26:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.716+0000 7fa8eb1a9b80 1 -- start start 2024-05-09T02:26:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.716+0000 7fa8eb1a9b80 1 Processor -- start 2024-05-09T02:26:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8eb1a9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae24d84a0 0x555ae265f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8eb1a9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 0x555ae265f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8eb1a9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2655400 0x555ae265fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8eb1a9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555ae2656160 con 0x555ae2552d00 2024-05-09T02:26:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8eb1a9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555ae2512b20 con 0x555ae2655400 2024-05-09T02:26:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8eb1a9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ae2662020 con 0x555ae24d84a0 2024-05-09T02:26:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8eb1a9b80 10 client.0 _ll_register_callbacks cb 0x555ae23e96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:26:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8d9d2e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 0x555ae265f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8da52f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae24d84a0 0x555ae265f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8da52f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae24d84a0 0x555ae265f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:48260/0 (socket says 172.21.15.19:48260) 2024-05-09T02:26:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8d9d2e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 0x555ae265f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42884/0 (socket says 172.21.15.19:42884) 2024-05-09T02:26:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8d9d2e700 1 -- 172.21.15.19:0/2142523117 learned_addr learned my addr 172.21.15.19:0/2142523117 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8dad30700 1 --2- 172.21.15.19:0/2142523117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2655400 0x555ae265fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8da52f700 1 -- 172.21.15.19:0/2142523117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2655400 msgr2=0x555ae265fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8da52f700 1 --2- 172.21.15.19:0/2142523117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2655400 0x555ae265fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8da52f700 1 -- 172.21.15.19:0/2142523117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 msgr2=0x555ae265f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8da52f700 1 --2- 172.21.15.19:0/2142523117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 0x555ae265f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.717+0000 7fa8da52f700 1 -- 172.21.15.19:0/2142523117 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ae23f3490 con 0x555ae24d84a0 2024-05-09T02:26:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.718+0000 7fa8da52f700 1 --2- 172.21.15.19:0/2142523117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae24d84a0 0x555ae265f040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00e0090 tx=0x7fa8d00b28d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2828ab17eb4f696e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.718+0000 7fa8dad30700 1 --2- 172.21.15.19:0/2142523117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2655400 0x555ae265fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:26:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.718+0000 7fa8bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:26:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.718+0000 7fa8bdffb700 1 -- 172.21.15.19:0/2142523117 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d00a53b0 con 0x555ae24d84a0 2024-05-09T02:26:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.718+0000 7fa8bdffb700 1 -- 172.21.15.19:0/2142523117 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fa8d0018d20 con 0x555ae24d84a0 2024-05-09T02:26:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.718+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/2142523117 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c400e790 con 0x555ae24d84a0 2024-05-09T02:26:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.718+0000 7fa8bdffb700 1 -- 172.21.15.19:0/2142523117 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa8d00b23c0 con 0x555ae24d84a0 2024-05-09T02:26:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.718+0000 7fa8eb1a9b80 10 client.26947 fetch_fsmap learned FSMap version 58 2024-05-09T02:26:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.718+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/2142523117 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x555ae266ca70 con 0x555ae24d84a0 2024-05-09T02:26:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.718+0000 7fa8bdffb700 1 -- 172.21.15.19:0/2142523117 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8c400e790 con 0x555ae24d84a0 2024-05-09T02:26:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.719+0000 7fa8bdffb700 1 -- 172.21.15.19:0/2142523117 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fa8d00a88f0 con 0x555ae24d84a0 2024-05-09T02:26:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.719+0000 7fa8eb1a9b80 10 client.26947 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:26:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.719+0000 7fa8eb1a9b80 10 client.26947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:26:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.719+0000 7fa8eb1a9b80 20 client.26947 populate_metadata read hostname 'smithi019' 2024-05-09T02:26:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.719+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/2142523117 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x555ae266f000 0x555ae2671460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.719+0000 7fa8eb1a9b80 4 client.26947 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:26:15.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.719+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/2142523117 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ae2585a80 con 0x555ae266f000 2024-05-09T02:26:15.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.719+0000 7fa8d9d2e700 1 --2- 172.21.15.19:0/2142523117 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x555ae266f000 0x555ae2671460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.720+0000 7fa8bdffb700 10 client.26947 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:15.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.720+0000 7fa8d9d2e700 1 --2- 172.21.15.19:0/2142523117 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x555ae266f000 0x555ae2671460 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac79c779ac4993ba server_cookie=53450276e236f769 in_seq=0 out_seq=0 2024-05-09T02:26:15.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.720+0000 7fa8bdffb700 1 -- 172.21.15.19:0/2142523117 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ae2585a80 con 0x555ae266f000 2024-05-09T02:26:15.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.720+0000 7fa8bdffb700 10 client.26947 handle_command_reply: tid=0 2024-05-09T02:26:15.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.720+0000 7fa8eb1a9b80 1 client.26947 shutdown 2024-05-09T02:26:15.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.720+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/2142523117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae24d84a0 msgr2=0x555ae265f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:15.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.720+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/2142523117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae24d84a0 0x555ae265f040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00e0090 tx=0x7fa8d00b28d0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.720+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/2142523117 shutdown_connections 2024-05-09T02:26:15.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.720+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/2142523117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2655400 0x555ae265fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.720+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/2142523117 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x555ae266f000 0x555ae2671460 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.720+0000 7fa8bdffb700 0 client.26947 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:15.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.721+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/2142523117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae24d84a0 0x555ae265f040 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.721+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/2142523117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 0x555ae265f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.721+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/2142523117 >> 172.21.15.19:0/2142523117 conn(0x555ae24dbf70 msgr2=0x555ae2649ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:15.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.721+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/2142523117 shutdown_connections 2024-05-09T02:26:15.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.721+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/2142523117 wait complete. 2024-05-09T02:26:15.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.721+0000 7fa8eb1a9b80 20 client.26947 trim_cache size 0 max 16384 2024-05-09T02:26:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.747+0000 7fa8eb1a9b80 1 Processor -- start 2024-05-09T02:26:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.748+0000 7fa8eb1a9b80 1 -- start start 2024-05-09T02:26:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.748+0000 7fa8eb1a9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae24d84a0 0x555ae265a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.748+0000 7fa8eb1a9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2552d00 0x555ae264aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.748+0000 7fa8eb1a9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae2655400 0x555ae265f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.748+0000 7fa8eb1a9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41b1170 con 0x555ae24d84a0 2024-05-09T02:26:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.748+0000 7fa8eb1a9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41b2cb0 con 0x555ae2552d00 2024-05-09T02:26:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.748+0000 7fa8eb1a9b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41b1840 con 0x555ae2655400 2024-05-09T02:26:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.748+0000 7fa8dad30700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae2655400 0x555ae265f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.748+0000 7fa8d9d2e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2552d00 0x555ae264aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.749+0000 7fa8d9d2e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2552d00 0x555ae264aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:42492/0 (socket says 172.21.15.19:42492) 2024-05-09T02:26:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.749+0000 7fa8dad30700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae2655400 0x555ae265f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:48268/0 (socket says 172.21.15.19:48268) 2024-05-09T02:26:15.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.749+0000 7fa8d9d2e700 1 -- 172.21.15.19:0/1856018383 learned_addr learned my addr 172.21.15.19:0/1856018383 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:15.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.749+0000 7fa8d9d2e700 1 -- 172.21.15.19:0/1856018383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae2655400 msgr2=0x555ae265f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.749+0000 7fa8d9d2e700 1 --2- 172.21.15.19:0/1856018383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae2655400 0x555ae265f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.749+0000 7fa8da52f700 1 --2- 172.21.15.19:0/1856018383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae24d84a0 0x555ae265a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.749+0000 7fa8d9d2e700 1 -- 172.21.15.19:0/1856018383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae24d84a0 msgr2=0x555ae265a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.749+0000 7fa8d9d2e700 1 --2- 172.21.15.19:0/1856018383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae24d84a0 0x555ae265a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.749+0000 7fa8d9d2e700 1 -- 172.21.15.19:0/1856018383 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ae26618e0 con 0x555ae2552d00 2024-05-09T02:26:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.749+0000 7fa8da52f700 1 --2- 172.21.15.19:0/1856018383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae24d84a0 0x555ae265a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:26:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.749+0000 7fa8d9d2e700 1 --2- 172.21.15.19:0/1856018383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2552d00 0x555ae264aa20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x555ae2671bc0 tx=0x7fa8c400ef30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb99930650ade2f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.750+0000 7fa8bffff700 1 -- 172.21.15.19:0/1856018383 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4038070 con 0x555ae2552d00 2024-05-09T02:26:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.750+0000 7fa8bffff700 1 -- 172.21.15.19:0/1856018383 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8c40054f0 con 0x555ae2552d00 2024-05-09T02:26:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.750+0000 7fa8bffff700 1 -- 172.21.15.19:0/1856018383 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4003bc0 con 0x555ae2552d00 2024-05-09T02:26:15.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.751+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/1856018383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2552d00 msgr2=0x555ae264aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:15.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.751+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/1856018383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2552d00 0x555ae264aa20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x555ae2671bc0 tx=0x7fa8c400ef30 comp rx=0 tx=0).stop 2024-05-09T02:26:15.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.751+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/1856018383 shutdown_connections 2024-05-09T02:26:15.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.751+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/1856018383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae2655400 0x555ae265f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.751+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/1856018383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae24d84a0 0x555ae265a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.751+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/1856018383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2552d00 0x555ae264aa20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.751+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/1856018383 >> 172.21.15.19:0/1856018383 conn(0x555ae24dbf70 msgr2=0x555ae265f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:15.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.751+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/1856018383 shutdown_connections 2024-05-09T02:26:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.751+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/1856018383 wait complete. 2024-05-09T02:26:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.752+0000 7fa8eb1a9b80 1 -- start start 2024-05-09T02:26:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.752+0000 7fa8eb1a9b80 1 Processor -- start 2024-05-09T02:26:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.752+0000 7fa8eb1a9b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae24d84a0 0x555ae26d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.752+0000 7fa8eb1a9b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 0x555ae2667100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.752+0000 7fa8eb1a9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2655400 0x555ae2667640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.752+0000 7fa8eb1a9b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41b25e0 con 0x555ae2552d00 2024-05-09T02:26:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.752+0000 7fa8da52f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae24d84a0 0x555ae26d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.752+0000 7fa8d9d2e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 0x555ae2667100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.752+0000 7fa8d9d2e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 0x555ae2667100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42906/0 (socket says 172.21.15.19:42906) 2024-05-09T02:26:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.752+0000 7fa8d9d2e700 1 -- 172.21.15.19:0/3716362960 learned_addr learned my addr 172.21.15.19:0/3716362960 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.752+0000 7fa8da52f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae24d84a0 0x555ae26d4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:48276/0 (socket says 172.21.15.19:48276) 2024-05-09T02:26:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.752+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/3716362960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41b1f10 con 0x555ae2655400 2024-05-09T02:26:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.752+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/3716362960 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41af130 con 0x555ae24d84a0 2024-05-09T02:26:15.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.753+0000 7fa8dad30700 1 --2- 172.21.15.19:0/3716362960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2655400 0x555ae2667640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.753+0000 7fa8eb1a9b80 10 client.0 _ll_register_callbacks cb 0x555ae23e96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:26:15.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.753+0000 7fa8dad30700 1 -- 172.21.15.19:0/3716362960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae24d84a0 msgr2=0x555ae26d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:15.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.753+0000 7fa8dad30700 1 --2- 172.21.15.19:0/3716362960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae24d84a0 0x555ae26d4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.753+0000 7fa8dad30700 1 -- 172.21.15.19:0/3716362960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 msgr2=0x555ae2667100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:15.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.753+0000 7fa8dad30700 1 --2- 172.21.15.19:0/3716362960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 0x555ae2667100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.753+0000 7fa8dad30700 1 -- 172.21.15.19:0/3716362960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ae2667b80 con 0x555ae2655400 2024-05-09T02:26:15.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.753+0000 7fa8d9d2e700 1 --2- 172.21.15.19:0/3716362960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 0x555ae2667100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:26:15.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.753+0000 7fa8dad30700 1 --2- 172.21.15.19:0/3716362960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2655400 0x555ae2667640 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00b770 tx=0x7fa8cc00bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=131fc9034b0d3ac8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:15.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.753+0000 7fa8bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:26:15.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.754+0000 7fa8bdffb700 1 -- 172.21.15.19:0/3716362960 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc02a060 con 0x555ae2655400 2024-05-09T02:26:15.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.754+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/3716362960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c40054f0 con 0x555ae2655400 2024-05-09T02:26:15.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.754+0000 7fa8bdffb700 1 -- 172.21.15.19:0/3716362960 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fa8cc004c60 con 0x555ae2655400 2024-05-09T02:26:15.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.754+0000 7fa8bdffb700 1 -- 172.21.15.19:0/3716362960 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa8cc0075c0 con 0x555ae2655400 2024-05-09T02:26:15.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.754+0000 7fa8eb1a9b80 10 client.26879 fetch_fsmap learned FSMap version 58 2024-05-09T02:26:15.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.754+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/3716362960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x555ae26e1fe0 con 0x555ae2655400 2024-05-09T02:26:15.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.754+0000 7fa8bdffb700 1 -- 172.21.15.19:0/3716362960 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8c40054f0 con 0x555ae2655400 2024-05-09T02:26:15.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.754+0000 7fa8bdffb700 1 -- 172.21.15.19:0/3716362960 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7fa8cc024070 con 0x555ae2655400 2024-05-09T02:26:15.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.754+0000 7fa8eb1a9b80 10 client.26879 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:26:15.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.754+0000 7fa8eb1a9b80 10 client.26879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:26:15.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.754+0000 7fa8eb1a9b80 20 client.26879 populate_metadata read hostname 'smithi019' 2024-05-09T02:26:15.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.755+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/3716362960 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x555ae26e2640 0x555ae26e4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:15.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.755+0000 7fa8eb1a9b80 4 client.26879 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:26:15.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.755+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/3716362960 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ae257fb20 con 0x555ae26e2640 2024-05-09T02:26:15.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.755+0000 7fa8da52f700 1 --2- 172.21.15.19:0/3716362960 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x555ae26e2640 0x555ae26e4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:15.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.755+0000 7fa8da52f700 1 --2- 172.21.15.19:0/3716362960 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x555ae26e2640 0x555ae26e4a60 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaa4f4de3cff1e66 server_cookie=d692a3811011b2e0 in_seq=0 out_seq=0 2024-05-09T02:26:15.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.755+0000 7fa8bdffb700 10 client.26879 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.755+0000 7fa8bdffb700 1 -- 172.21.15.19:0/3716362960 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555ae257fb20 con 0x555ae26e2640 2024-05-09T02:26:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.755+0000 7fa8bdffb700 10 client.26879 handle_command_reply: tid=0 2024-05-09T02:26:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.755+0000 7fa8eb1a9b80 1 client.26879 shutdown 2024-05-09T02:26:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.756+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/3716362960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2655400 msgr2=0x555ae2667640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.756+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/3716362960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2655400 0x555ae2667640 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00b770 tx=0x7fa8cc00bd80 comp rx=0 tx=0).stop 2024-05-09T02:26:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.756+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/3716362960 shutdown_connections 2024-05-09T02:26:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.756+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/3716362960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ae2655400 0x555ae2667640 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.756+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/3716362960 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x555ae26e2640 0x555ae26e4a60 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.756+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/3716362960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ae24d84a0 0x555ae26d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.756+0000 7fa8eb1a9b80 1 --2- 172.21.15.19:0/3716362960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ae2552d00 0x555ae2667100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.756+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/3716362960 >> 172.21.15.19:0/3716362960 conn(0x555ae24dbf70 msgr2=0x555ae264a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.756+0000 7fa8bdffb700 0 client.26879 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.756+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/3716362960 shutdown_connections 2024-05-09T02:26:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.756+0000 7fa8eb1a9b80 1 -- 172.21.15.19:0/3716362960 wait complete. 2024-05-09T02:26:15.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.756+0000 7fa8eb1a9b80 20 client.26879 trim_cache size 0 max 16384 2024-05-09T02:26:15.763 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:26:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T02:26:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-09T02:26:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: "af77a585-5518-4755-b241-b5103f8d38aa": { 2024-05-09T02:26:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-09T02:26:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "af77a585-5518-4755-b241-b5103f8d38aa", 2024-05-09T02:26:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-09T02:26:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:26:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:26:15.763 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:26:15.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.757+0000 7fa8dc793700 1 -- 172.21.15.19:0/3246269431 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8c00b29f0 msgr2=0x7fa8c00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:15.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.757+0000 7fa8dc793700 1 --2- 172.21.15.19:0/3246269431 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8c00b29f0 0x7fa8c00b4f00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4106710 tx=0x7fa8c4009260 comp rx=0 tx=0).stop 2024-05-09T02:26:15.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.757+0000 7fa8dc793700 1 -- 172.21.15.19:0/3246269431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41034c0 msgr2=0x7fa8d4110a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:15.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.757+0000 7fa8dc793700 1 --2- 172.21.15.19:0/3246269431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41034c0 0x7fa8d4110a80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000e9a0 tx=0x7fa8d000ecb0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.757+0000 7fa8dc793700 1 -- 172.21.15.19:0/3246269431 shutdown_connections 2024-05-09T02:26:15.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.757+0000 7fa8dc793700 1 --2- 172.21.15.19:0/3246269431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8d4108740 0x7fa8d4111500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.757+0000 7fa8dc793700 1 --2- 172.21.15.19:0/3246269431 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa8c00b29f0 0x7fa8c00b4f00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.757+0000 7fa8dc793700 1 --2- 172.21.15.19:0/3246269431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8d41034c0 0x7fa8d4110a80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.757+0000 7fa8dc793700 1 --2- 172.21.15.19:0/3246269431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8d4107df0 0x7fa8d4110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:15.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.757+0000 7fa8dc793700 1 -- 172.21.15.19:0/3246269431 >> 172.21.15.19:0/3246269431 conn(0x7fa8d40fedc0 msgr2=0x7fa8d41070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:15.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.757+0000 7fa8dc793700 1 -- 172.21.15.19:0/3246269431 shutdown_connections 2024-05-09T02:26:15.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:15.757+0000 7fa8dc793700 1 -- 172.21.15.19:0/3246269431 wait complete. 2024-05-09T02:26:15.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:15 smithi019 ceph-mon[30151]: osdmap e100: 12 total, 12 up, 12 in 2024-05-09T02:26:15.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:15 smithi059 ceph-mon[34781]: osdmap e100: 12 total, 12 up, 12 in 2024-05-09T02:26:16.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:15 smithi196 ceph-mon[34718]: osdmap e100: 12 total, 12 up, 12 in 2024-05-09T02:26:16.807 INFO:tasks.cephfs.filesystem:scrub status for tag:af77a585-5518-4755-b241-b5103f8d38aa - {'path': '/', 'tag': 'af77a585-5518-4755-b241-b5103f8d38aa', 'options': 'recursive,force'} 2024-05-09T02:26:16.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:16 smithi059 ceph-mon[34781]: pgmap v470: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.3 MiB/s wr, 572 op/s 2024-05-09T02:26:17.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:16 smithi196 ceph-mon[34718]: pgmap v470: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.3 MiB/s wr, 572 op/s 2024-05-09T02:26:17.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:16 smithi019 ceph-mon[30151]: pgmap v470: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.3 MiB/s wr, 572 op/s 2024-05-09T02:26:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 875.0 s, 1370.2 tps, lat 74.413 ms stddev 91.749 2024-05-09T02:26:18.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:18 smithi059 ceph-mon[34781]: pgmap v471: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.8 MiB/s wr, 606 op/s 2024-05-09T02:26:19.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:18 smithi196 ceph-mon[34718]: pgmap v471: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.8 MiB/s wr, 606 op/s 2024-05-09T02:26:19.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:18 smithi019 ceph-mon[30151]: pgmap v471: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.8 MiB/s wr, 606 op/s 2024-05-09T02:26:20.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:20 smithi059 ceph-mon[34781]: pgmap v472: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.7 MiB/s wr, 680 op/s 2024-05-09T02:26:21.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:20 smithi196 ceph-mon[34718]: pgmap v472: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.7 MiB/s wr, 680 op/s 2024-05-09T02:26:21.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:20 smithi019 ceph-mon[30151]: pgmap v472: 97 pgs: 97 active+clean; 4.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.7 MiB/s wr, 680 op/s 2024-05-09T02:26:21.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:21 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:26:21.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:21 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:26:21.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:21 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:26:21.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:21 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:26:21.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:21 smithi059 ceph-mon[34781]: scrub summary: idle 2024-05-09T02:26:22.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:21 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:26:22.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:21 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:26:22.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:21 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:26:22.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:21 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:26:22.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:21 smithi196 ceph-mon[34718]: scrub summary: idle 2024-05-09T02:26:22.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:21 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:26:22.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:21 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:26:22.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:21 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:26:22.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:21 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:26:22.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:21 smithi019 ceph-mon[30151]: scrub summary: idle 2024-05-09T02:26:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 880.0 s, 1676.2 tps, lat 59.039 ms stddev 67.321 2024-05-09T02:26:22.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:22 smithi059 ceph-mon[34781]: pgmap v473: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.9 MiB/s wr, 625 op/s 2024-05-09T02:26:23.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:22 smithi196 ceph-mon[34718]: pgmap v473: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.9 MiB/s wr, 625 op/s 2024-05-09T02:26:23.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:22 smithi019 ceph-mon[30151]: pgmap v473: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.9 MiB/s wr, 625 op/s 2024-05-09T02:26:23.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:23 smithi059 ceph-mon[34781]: pgmap v474: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.1 MiB/s wr, 721 op/s 2024-05-09T02:26:24.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:23 smithi196 ceph-mon[34718]: pgmap v474: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.1 MiB/s wr, 721 op/s 2024-05-09T02:26:24.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:23 smithi019 ceph-mon[30151]: pgmap v474: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.1 MiB/s wr, 721 op/s 2024-05-09T02:26:26.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:26 smithi196 ceph-mon[34718]: pgmap v475: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.1 MiB/s wr, 620 op/s 2024-05-09T02:26:26.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:26 smithi019 ceph-mon[30151]: pgmap v475: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.1 MiB/s wr, 620 op/s 2024-05-09T02:26:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:26 smithi059 ceph-mon[34781]: pgmap v475: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.1 MiB/s wr, 620 op/s 2024-05-09T02:26:27.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 885.0 s, 1359.4 tps, lat 73.447 ms stddev 77.564 2024-05-09T02:26:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:28 smithi196 ceph-mon[34718]: pgmap v476: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.6 MiB/s wr, 588 op/s 2024-05-09T02:26:28.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:28 smithi019 ceph-mon[30151]: pgmap v476: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.6 MiB/s wr, 588 op/s 2024-05-09T02:26:28.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:28 smithi059 ceph-mon[34781]: pgmap v476: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.6 MiB/s wr, 588 op/s 2024-05-09T02:26:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:30 smithi196 ceph-mon[34718]: pgmap v477: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.9 MiB/s wr, 615 op/s 2024-05-09T02:26:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:30 smithi019 ceph-mon[30151]: pgmap v477: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.9 MiB/s wr, 615 op/s 2024-05-09T02:26:30.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:30 smithi059 ceph-mon[34781]: pgmap v477: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.9 MiB/s wr, 615 op/s 2024-05-09T02:26:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 890.0 s, 793.6 tps, lat 125.497 ms stddev 310.112 2024-05-09T02:26:32.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:32 smithi196 ceph-mon[34718]: pgmap v478: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.9 MiB/s wr, 648 op/s 2024-05-09T02:26:32.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:32 smithi019 ceph-mon[30151]: pgmap v478: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.9 MiB/s wr, 648 op/s 2024-05-09T02:26:32.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:32 smithi059 ceph-mon[34781]: pgmap v478: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.9 MiB/s wr, 648 op/s 2024-05-09T02:26:34.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:34 smithi019 ceph-mon[30151]: pgmap v479: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.5 MiB/s wr, 566 op/s 2024-05-09T02:26:34.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:34 smithi196 ceph-mon[34718]: pgmap v479: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.5 MiB/s wr, 566 op/s 2024-05-09T02:26:34.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:34 smithi059 ceph-mon[34781]: pgmap v479: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.5 MiB/s wr, 566 op/s 2024-05-09T02:26:36.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:36 smithi196 ceph-mon[34718]: pgmap v480: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.3 MiB/s wr, 579 op/s 2024-05-09T02:26:36.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:36 smithi019 ceph-mon[30151]: pgmap v480: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.3 MiB/s wr, 579 op/s 2024-05-09T02:26:36.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:36 smithi059 ceph-mon[34781]: pgmap v480: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.3 MiB/s wr, 579 op/s 2024-05-09T02:26:37.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 895.0 s, 1165.8 tps, lat 85.288 ms stddev 92.837 2024-05-09T02:26:38.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:38 smithi196 ceph-mon[34718]: pgmap v481: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.6 MiB/s wr, 511 op/s 2024-05-09T02:26:38.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:38 smithi019 ceph-mon[30151]: pgmap v481: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.6 MiB/s wr, 511 op/s 2024-05-09T02:26:38.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:38 smithi059 ceph-mon[34781]: pgmap v481: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.6 MiB/s wr, 511 op/s 2024-05-09T02:26:40.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:40 smithi196 ceph-mon[34718]: pgmap v482: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.9 MiB/s wr, 536 op/s 2024-05-09T02:26:40.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:40 smithi019 ceph-mon[30151]: pgmap v482: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.9 MiB/s wr, 536 op/s 2024-05-09T02:26:40.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:40 smithi059 ceph-mon[34781]: pgmap v482: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.9 MiB/s wr, 536 op/s 2024-05-09T02:26:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:progress: 900.0 s, 1248.6 tps, lat 80.916 ms stddev 91.819 2024-05-09T02:26:42.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:42 smithi196 ceph-mon[34718]: pgmap v483: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 5.5 MiB/s wr, 610 op/s 2024-05-09T02:26:42.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:42 smithi019 ceph-mon[30151]: pgmap v483: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 5.5 MiB/s wr, 610 op/s 2024-05-09T02:26:42.872 INFO:teuthology.orchestra.run.smithi019.stdout:transaction type: 2024-05-09T02:26:42.872 INFO:teuthology.orchestra.run.smithi019.stdout:scaling factor: 32 2024-05-09T02:26:42.872 INFO:teuthology.orchestra.run.smithi019.stdout:query mode: simple 2024-05-09T02:26:42.872 INFO:teuthology.orchestra.run.smithi019.stdout:number of clients: 100 2024-05-09T02:26:42.872 INFO:teuthology.orchestra.run.smithi019.stdout:number of threads: 4 2024-05-09T02:26:42.872 INFO:teuthology.orchestra.run.smithi019.stdout:duration: 900 s 2024-05-09T02:26:42.872 INFO:teuthology.orchestra.run.smithi019.stdout:number of transactions actually processed: 1785726 2024-05-09T02:26:42.872 INFO:teuthology.orchestra.run.smithi019.stdout:latency average = 50.397 ms 2024-05-09T02:26:42.873 INFO:teuthology.orchestra.run.smithi019.stdout:latency stddev = 106.259 ms 2024-05-09T02:26:42.873 INFO:teuthology.orchestra.run.smithi019.stdout:tps = 1983.661282 (including connections establishing) 2024-05-09T02:26:42.873 INFO:teuthology.orchestra.run.smithi019.stdout:tps = 1983.666915 (excluding connections establishing) 2024-05-09T02:26:42.876 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-05-09T02:26:42.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:42 smithi059 ceph-mon[34781]: pgmap v483: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 5.5 MiB/s wr, 610 op/s 2024-05-09T02:26:44.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:44 smithi196 ceph-mon[34718]: pgmap v484: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 4.6 MiB/s wr, 491 op/s 2024-05-09T02:26:44.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:44 smithi019 ceph-mon[30151]: pgmap v484: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 4.6 MiB/s wr, 491 op/s 2024-05-09T02:26:44.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:44 smithi059 ceph-mon[34781]: pgmap v484: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 4.6 MiB/s wr, 491 op/s 2024-05-09T02:26:46.808 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T02:26:46.825 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:46 smithi019 ceph-mon[30151]: pgmap v485: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.5 MiB/s wr, 620 op/s 2024-05-09T02:26:46.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:46 smithi059 ceph-mon[34781]: pgmap v485: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.5 MiB/s wr, 620 op/s 2024-05-09T02:26:47.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:46 smithi196 ceph-mon[34718]: pgmap v485: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.5 MiB/s wr, 620 op/s 2024-05-09T02:26:47.429 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:26:48.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:48 smithi019 ceph-mon[30151]: pgmap v486: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 5.8 MiB/s wr, 640 op/s 2024-05-09T02:26:48.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:48 smithi059 ceph-mon[34781]: pgmap v486: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 5.8 MiB/s wr, 640 op/s 2024-05-09T02:26:49.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:48 smithi196 ceph-mon[34718]: pgmap v486: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 5.8 MiB/s wr, 640 op/s 2024-05-09T02:26:49.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.935+0000 7f1fe9280700 1 -- 172.21.15.19:0/1464385632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe4110fc0 msgr2=0x7f1fe410ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:49.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.935+0000 7f1fe9280700 1 --2- 172.21.15.19:0/1464385632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe4110fc0 0x7f1fe410ad80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009990 tx=0x7f1fd4009ca0 comp rx=0 tx=0).stop 2024-05-09T02:26:49.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.936+0000 7f1fe9280700 1 -- 172.21.15.19:0/1464385632 shutdown_connections 2024-05-09T02:26:49.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.936+0000 7f1fe9280700 1 --2- 172.21.15.19:0/1464385632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe4110fc0 0x7f1fe410ad80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:49.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.936+0000 7f1fe9280700 1 --2- 172.21.15.19:0/1464385632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4112740 0x7f1fe4110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:49.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.936+0000 7f1fe9280700 1 --2- 172.21.15.19:0/1464385632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe4110ad0 0x7f1fe410a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:49.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.936+0000 7f1fe9280700 1 -- 172.21.15.19:0/1464385632 >> 172.21.15.19:0/1464385632 conn(0x7f1fe406f260 msgr2=0x7f1fe406f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:49.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.943+0000 7f1fe9280700 1 -- 172.21.15.19:0/1464385632 shutdown_connections 2024-05-09T02:26:49.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.945+0000 7f1fe9280700 1 -- 172.21.15.19:0/1464385632 wait complete. 2024-05-09T02:26:49.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.946+0000 7f1fe9280700 1 Processor -- start 2024-05-09T02:26:49.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.946+0000 7f1fe9280700 1 -- start start 2024-05-09T02:26:49.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.946+0000 7f1fe9280700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe4110ad0 0x7f1fe41af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:49.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.946+0000 7f1fe9280700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe4110fc0 0x7f1fe41afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:49.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.946+0000 7f1fe9280700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4112740 0x7f1fe41a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:49.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.946+0000 7f1fe9280700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4123160 con 0x7f1fe4112740 2024-05-09T02:26:49.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.946+0000 7f1fe9280700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4122fe0 con 0x7f1fe4110fc0 2024-05-09T02:26:49.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.946+0000 7f1fe9280700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe41232e0 con 0x7f1fe4110ad0 2024-05-09T02:26:49.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.947+0000 7f1fe3fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe4110ad0 0x7f1fe41af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:49.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.947+0000 7f1fe8a7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4112740 0x7f1fe41a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:49.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.947+0000 7f1fe8a7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4112740 0x7f1fe41a98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47282/0 (socket says 172.21.15.19:47282) 2024-05-09T02:26:49.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.947+0000 7f1fe8a7f700 1 -- 172.21.15.19:0/1565696273 learned_addr learned my addr 172.21.15.19:0/1565696273 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:49.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.949+0000 7f1fe37fe700 1 --2- 172.21.15.19:0/1565696273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe4110fc0 0x7f1fe41afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:49.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.949+0000 7f1fe3fff700 1 -- 172.21.15.19:0/1565696273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe4110fc0 msgr2=0x7f1fe41afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:49.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.949+0000 7f1fe3fff700 1 --2- 172.21.15.19:0/1565696273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe4110fc0 0x7f1fe41afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:49.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.949+0000 7f1fe3fff700 1 -- 172.21.15.19:0/1565696273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4112740 msgr2=0x7f1fe41a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:49.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.949+0000 7f1fe3fff700 1 --2- 172.21.15.19:0/1565696273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4112740 0x7f1fe41a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:49.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.949+0000 7f1fe3fff700 1 -- 172.21.15.19:0/1565696273 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe41a9fd0 con 0x7f1fe4110ad0 2024-05-09T02:26:49.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.954+0000 7f1fe3fff700 1 --2- 172.21.15.19:0/1565696273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe4110ad0 0x7f1fe41af610 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00bd70 tx=0x7f1fdc00bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:49.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.956+0000 7f1fe17fa700 1 -- 172.21.15.19:0/1565696273 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc019070 con 0x7f1fe4110ad0 2024-05-09T02:26:49.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.956+0000 7f1fe9280700 1 -- 172.21.15.19:0/1565696273 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe41aa2c0 con 0x7f1fe4110ad0 2024-05-09T02:26:49.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.956+0000 7f1fe9280700 1 -- 172.21.15.19:0/1565696273 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe41b6470 con 0x7f1fe4110ad0 2024-05-09T02:26:49.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.956+0000 7f1fe17fa700 1 -- 172.21.15.19:0/1565696273 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fdc00cc90 con 0x7f1fe4110ad0 2024-05-09T02:26:49.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.956+0000 7f1fe17fa700 1 -- 172.21.15.19:0/1565696273 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc0125a0 con 0x7f1fe4110ad0 2024-05-09T02:26:49.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.963+0000 7f1ff8ef8b80 1 Processor -- start 2024-05-09T02:26:49.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.963+0000 7f1ff8ef8b80 1 -- start start 2024-05-09T02:26:49.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.964+0000 7f1ff8ef8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a699d00 0x556a0a79ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:49.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.964+0000 7f1ff8ef8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a61f4a0 0x556a0a79cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:49.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.964+0000 7f1ff8ef8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a7a6540 0x556a0a7a8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:49.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.964+0000 7f1ff8ef8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556a0a6bf0a0 con 0x556a0a61f4a0 2024-05-09T02:26:49.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.964+0000 7f1ff8ef8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556a0a6649a0 con 0x556a0a7a6540 2024-05-09T02:26:49.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.964+0000 7f1ff8ef8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556a0a6bb060 con 0x556a0a699d00 2024-05-09T02:26:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.964+0000 7f1fe37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a7a6540 0x556a0a7a8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.964+0000 7f1fe37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a7a6540 0x556a0a7a8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:41624/0 (socket says 172.21.15.19:41624) 2024-05-09T02:26:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.964+0000 7f1fe37fe700 1 -- 172.21.15.19:0/3595965804 learned_addr learned my addr 172.21.15.19:0/3595965804 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.965+0000 7f1fe37fe700 1 -- 172.21.15.19:0/3595965804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a699d00 msgr2=0x556a0a79ca30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T02:26:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.965+0000 7f1fe37fe700 1 --2- 172.21.15.19:0/3595965804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a699d00 0x556a0a79ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.965+0000 7f1fe37fe700 1 -- 172.21.15.19:0/3595965804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a61f4a0 msgr2=0x556a0a79cf70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T02:26:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.965+0000 7f1fe37fe700 1 --2- 172.21.15.19:0/3595965804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a61f4a0 0x556a0a79cf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.965+0000 7f1fe37fe700 1 -- 172.21.15.19:0/3595965804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a0a655500 con 0x556a0a7a6540 2024-05-09T02:26:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.965+0000 7f1fe37fe700 1 --2- 172.21.15.19:0/3595965804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a7a6540 0x556a0a7a8960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800bce0 tx=0x7f1fd800f360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14047c9b150fabfd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.966+0000 7f1fca7fc700 1 -- 172.21.15.19:0/3595965804 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8015070 con 0x556a0a7a6540 2024-05-09T02:26:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.966+0000 7f1fca7fc700 1 -- 172.21.15.19:0/3595965804 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fd800fc90 con 0x556a0a7a6540 2024-05-09T02:26:49.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.966+0000 7f1fca7fc700 1 -- 172.21.15.19:0/3595965804 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80116a0 con 0x556a0a7a6540 2024-05-09T02:26:49.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.968+0000 7f1fe17fa700 1 -- 172.21.15.19:0/1565696273 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1fdc01c070 con 0x7f1fe4110ad0 2024-05-09T02:26:49.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.971+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/3595965804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a7a6540 msgr2=0x556a0a7a8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.971+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/3595965804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a7a6540 0x556a0a7a8960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800bce0 tx=0x7f1fd800f360 comp rx=0 tx=0).stop 2024-05-09T02:26:49.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.975+0000 7f1fe17fa700 1 --2- 172.21.15.19:0/1565696273 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1fcc0b2c00 0x7f1fcc0b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.975+0000 7f1fe17fa700 1 -- 172.21.15.19:0/1565696273 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f1fdc0e19f0 con 0x7f1fe4110ad0 2024-05-09T02:26:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.975+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/3595965804 shutdown_connections 2024-05-09T02:26:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.975+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/3595965804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a7a6540 0x556a0a7a8960 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.975+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/3595965804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a699d00 0x556a0a79ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.975+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/3595965804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a61f4a0 0x556a0a79cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.975+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/3595965804 >> 172.21.15.19:0/3595965804 conn(0x556a0a622f70 msgr2=0x556a0a797760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:49.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.976+0000 7f1fe8a7f700 1 --2- 172.21.15.19:0/1565696273 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1fcc0b2c00 0x7f1fcc0b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:49.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.979+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/3595965804 shutdown_connections 2024-05-09T02:26:49.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.979+0000 7f1fe8a7f700 1 --2- 172.21.15.19:0/1565696273 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1fcc0b2c00 0x7f1fcc0b5110 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x556a0a798110 tx=0x7f1fd401a040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.982+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/3595965804 wait complete. 2024-05-09T02:26:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.983+0000 7f1ff8ef8b80 1 -- start start 2024-05-09T02:26:49.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.983+0000 7f1ff8ef8b80 1 Processor -- start 2024-05-09T02:26:49.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.984+0000 7f1ff8ef8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a61f4a0 0x556a0a7a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:49.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.984+0000 7f1ff8ef8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a699d00 0x556a0a7a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:49.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.984+0000 7f1ff8ef8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a7a6540 0x556a0a7a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:49.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.984+0000 7f1ff8ef8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556a0a7b9f70 con 0x556a0a7a6540 2024-05-09T02:26:49.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.984+0000 7f1ff8ef8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556a0a659b20 con 0x556a0a699d00 2024-05-09T02:26:49.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.984+0000 7f1ff8ef8b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556a0a7ba0f0 con 0x556a0a61f4a0 2024-05-09T02:26:49.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.984+0000 7f1ff8ef8b80 10 client.0 _ll_register_callbacks cb 0x556a0a5306c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:26:49.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.984+0000 7f1fe8a7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a7a6540 0x556a0a7a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:49.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.984+0000 7f1fe8a7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a7a6540 0x556a0a7a3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47296/0 (socket says 172.21.15.19:47296) 2024-05-09T02:26:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.984+0000 7f1fe8a7f700 1 -- 172.21.15.19:0/2208605740 learned_addr learned my addr 172.21.15.19:0/2208605740 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.984+0000 7f1fe3fff700 1 --2- 172.21.15.19:0/2208605740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a61f4a0 0x556a0a7a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.985+0000 7f1fe8a7f700 1 -- 172.21.15.19:0/2208605740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a61f4a0 msgr2=0x556a0a7a34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.985+0000 7f1fe8a7f700 1 --2- 172.21.15.19:0/2208605740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a61f4a0 0x556a0a7a34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.985+0000 7f1fe8a7f700 1 -- 172.21.15.19:0/2208605740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a699d00 msgr2=0x556a0a7a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.985+0000 7f1fe8a7f700 1 --2- 172.21.15.19:0/2208605740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a699d00 0x556a0a7a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.985+0000 7f1fe8a7f700 1 -- 172.21.15.19:0/2208605740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a0a53a490 con 0x556a0a7a6540 2024-05-09T02:26:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.985+0000 7f1fe8a7f700 1 --2- 172.21.15.19:0/2208605740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a7a6540 0x556a0a7a3f60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400bf10 tx=0x7f1fd4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5c4f6679543d26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.986+0000 7f1fc37fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:26:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.986+0000 7f1fc37fe700 1 -- 172.21.15.19:0/2208605740 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4005400 con 0x556a0a7a6540 2024-05-09T02:26:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.986+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/2208605740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fd800fc90 con 0x556a0a7a6540 2024-05-09T02:26:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.987+0000 7f1fc37fe700 1 -- 172.21.15.19:0/2208605740 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f1fd40055a0 con 0x556a0a7a6540 2024-05-09T02:26:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.988+0000 7f1fc37fe700 1 -- 172.21.15.19:0/2208605740 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1fd400f460 con 0x556a0a7a6540 2024-05-09T02:26:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.990+0000 7f1ff8ef8b80 10 client.17157 fetch_fsmap learned FSMap version 58 2024-05-09T02:26:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.990+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/2208605740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556a0a7c4ca0 con 0x556a0a7a6540 2024-05-09T02:26:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.994+0000 7f1fc37fe700 1 -- 172.21.15.19:0/2208605740 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fd800fc90 con 0x556a0a7a6540 2024-05-09T02:26:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.994+0000 7f1fc37fe700 1 -- 172.21.15.19:0/2208605740 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f1fd402e020 con 0x556a0a7a6540 2024-05-09T02:26:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.995+0000 7f1ff8ef8b80 10 client.17157 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:26:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.995+0000 7f1ff8ef8b80 10 client.17157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:26:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.995+0000 7f1ff8ef8b80 20 client.17157 populate_metadata read hostname 'smithi019' 2024-05-09T02:26:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.995+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/2208605740 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556a0a7c76a0 0x556a0a7c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.995+0000 7f1ff8ef8b80 4 client.17157 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:26:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:49.995+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/2208605740 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a0a6cca80 con 0x556a0a7c76a0 2024-05-09T02:26:50.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.000+0000 7f1fe3fff700 1 --2- 172.21.15.19:0/2208605740 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556a0a7c76a0 0x556a0a7c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:50.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.004+0000 7f1fe3fff700 1 --2- 172.21.15.19:0/2208605740 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556a0a7c76a0 0x556a0a7c9b70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67365c3ba6eacdd0 server_cookie=ef16869b2368012f in_seq=0 out_seq=0 2024-05-09T02:26:50.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.004+0000 7f1fc37fe700 10 client.17157 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:50.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.009+0000 7f1fc37fe700 1 -- 172.21.15.19:0/2208605740 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556a0a6cca80 con 0x556a0a7c76a0 2024-05-09T02:26:50.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.009+0000 7f1fc37fe700 10 client.17157 handle_command_reply: tid=0 2024-05-09T02:26:50.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.009+0000 7f1ff8ef8b80 1 client.17157 shutdown 2024-05-09T02:26:50.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.012+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/2208605740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a7a6540 msgr2=0x556a0a7a3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:50.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.012+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/2208605740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a7a6540 0x556a0a7a3f60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400bf10 tx=0x7f1fd4009310 comp rx=0 tx=0).stop 2024-05-09T02:26:50.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.012+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/2208605740 shutdown_connections 2024-05-09T02:26:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.012+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/2208605740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a7a6540 0x556a0a7a3f60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.012+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/2208605740 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556a0a7c76a0 0x556a0a7c9b70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.012+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/2208605740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a61f4a0 0x556a0a7a34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.012+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/2208605740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a699d00 0x556a0a7a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.012+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/2208605740 >> 172.21.15.19:0/2208605740 conn(0x556a0a622f70 msgr2=0x556a0a797250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:50.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.016+0000 7f1fc37fe700 0 client.17157 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:50.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.017+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/2208605740 shutdown_connections 2024-05-09T02:26:50.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.020+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/2208605740 wait complete. 2024-05-09T02:26:50.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.022+0000 7f1ff8ef8b80 20 client.17157 trim_cache size 0 max 16384 2024-05-09T02:26:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.073+0000 7f1ff8ef8b80 1 Processor -- start 2024-05-09T02:26:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.073+0000 7f1ff8ef8b80 1 -- start start 2024-05-09T02:26:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.074+0000 7f1ff8ef8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a61f4a0 0x556a0a7c3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.074+0000 7f1ff8ef8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a699d00 0x556a0a7a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.074+0000 7f1ff8ef8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a7a6540 0x556a0a7cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.074+0000 7f1fe3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a61f4a0 0x556a0a7c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.074+0000 7f1fe3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a61f4a0 0x556a0a7c3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:41628/0 (socket says 172.21.15.19:41628) 2024-05-09T02:26:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.074+0000 7f1fe3fff700 1 -- 172.21.15.19:0/517259787 learned_addr learned my addr 172.21.15.19:0/517259787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.074+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/517259787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd000d770 con 0x556a0a699d00 2024-05-09T02:26:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.075+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/517259787 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd000f2b0 con 0x556a0a61f4a0 2024-05-09T02:26:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.075+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/517259787 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd000de40 con 0x556a0a7a6540 2024-05-09T02:26:50.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.076+0000 7f1fe37fe700 1 --2- 172.21.15.19:0/517259787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a699d00 0x556a0a7a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:50.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.076+0000 7f1fe8a7f700 1 --2- 172.21.15.19:0/517259787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a7a6540 0x556a0a7cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:50.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.078+0000 7f1fe37fe700 1 -- 172.21.15.19:0/517259787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a7a6540 msgr2=0x556a0a7cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:50.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.078+0000 7f1fe37fe700 1 --2- 172.21.15.19:0/517259787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a7a6540 0x556a0a7cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.078+0000 7f1fe37fe700 1 -- 172.21.15.19:0/517259787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a61f4a0 msgr2=0x556a0a7c3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:50.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.078+0000 7f1fe37fe700 1 --2- 172.21.15.19:0/517259787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a61f4a0 0x556a0a7c3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.078+0000 7f1fe37fe700 1 -- 172.21.15.19:0/517259787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a0a761ca0 con 0x556a0a699d00 2024-05-09T02:26:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.081+0000 7f1fe3fff700 1 --2- 172.21.15.19:0/517259787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a61f4a0 0x556a0a7c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:26:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.081+0000 7f1fe37fe700 1 --2- 172.21.15.19:0/517259787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a699d00 0x556a0a7a3a50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x556a0a79c760 tx=0x7f1fd800fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a4d227cd30a879 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.087+0000 7f1fc9ffb700 1 -- 172.21.15.19:0/517259787 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8015070 con 0x556a0a699d00 2024-05-09T02:26:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.087+0000 7f1fc9ffb700 1 -- 172.21.15.19:0/517259787 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fd80044e0 con 0x556a0a699d00 2024-05-09T02:26:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.087+0000 7f1fc9ffb700 1 -- 172.21.15.19:0/517259787 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8004970 con 0x556a0a699d00 2024-05-09T02:26:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.088+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/517259787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a699d00 msgr2=0x556a0a7a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.088+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/517259787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a699d00 0x556a0a7a3a50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x556a0a79c760 tx=0x7f1fd800fc20 comp rx=0 tx=0).stop 2024-05-09T02:26:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.088+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/517259787 shutdown_connections 2024-05-09T02:26:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.088+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/517259787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a7a6540 0x556a0a7cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.088+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/517259787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a61f4a0 0x556a0a7c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.088+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/517259787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a699d00 0x556a0a7a3a50 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.088+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/517259787 >> 172.21.15.19:0/517259787 conn(0x556a0a622f70 msgr2=0x556a0a7a1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:50.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.093+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/517259787 shutdown_connections 2024-05-09T02:26:50.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.093+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/517259787 wait complete. 2024-05-09T02:26:50.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.093+0000 7f1ff8ef8b80 1 -- start start 2024-05-09T02:26:50.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.094+0000 7f1ff8ef8b80 1 Processor -- start 2024-05-09T02:26:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.094+0000 7f1ff8ef8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a61f4a0 0x556a0a7ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.095+0000 7f1ff8ef8b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a699d00 0x556a0a7ba900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.095+0000 7f1fe3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a61f4a0 0x556a0a7ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.095+0000 7f1fe3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a61f4a0 0x556a0a7ba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:41638/0 (socket says 172.21.15.19:41638) 2024-05-09T02:26:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.095+0000 7f1fe37fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a699d00 0x556a0a7ba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.095+0000 7f1fe37fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a699d00 0x556a0a7ba900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:46288/0 (socket says 172.21.15.19:46288) 2024-05-09T02:26:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.095+0000 7f1ff8ef8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a7a6540 0x556a0a82ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:50.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.095+0000 7f1fe3fff700 1 -- 172.21.15.19:0/936721100 learned_addr learned my addr 172.21.15.19:0/936721100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:50.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.096+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/936721100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd000ebe0 con 0x556a0a7a6540 2024-05-09T02:26:50.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.096+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/936721100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd000e510 con 0x556a0a61f4a0 2024-05-09T02:26:50.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.096+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/936721100 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd000b730 con 0x556a0a699d00 2024-05-09T02:26:50.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.096+0000 7f1ff8ef8b80 10 client.0 _ll_register_callbacks cb 0x556a0a5306c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:26:50.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.098+0000 7f1fe8a7f700 1 --2- 172.21.15.19:0/936721100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a7a6540 0x556a0a82ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:50.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.099+0000 7f1fe8a7f700 1 -- 172.21.15.19:0/936721100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a699d00 msgr2=0x556a0a7ba900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:50.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.099+0000 7f1fe8a7f700 1 --2- 172.21.15.19:0/936721100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a699d00 0x556a0a7ba900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.099+0000 7f1fe8a7f700 1 -- 172.21.15.19:0/936721100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a61f4a0 msgr2=0x556a0a7ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:50.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.099+0000 7f1fe8a7f700 1 --2- 172.21.15.19:0/936721100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a61f4a0 0x556a0a7ba3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.099+0000 7f1fe8a7f700 1 -- 172.21.15.19:0/936721100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a0a830450 con 0x556a0a7a6540 2024-05-09T02:26:50.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.101+0000 7f1fe8a7f700 1 --2- 172.21.15.19:0/936721100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a7a6540 0x556a0a82ff10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4030830 tx=0x7f1fd40304f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d5b29b782cef6f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:50.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.101+0000 7f1fc37fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:26:50.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.102+0000 7f1fc37fe700 1 -- 172.21.15.19:0/936721100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd403e020 con 0x556a0a7a6540 2024-05-09T02:26:50.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.102+0000 7f1fc37fe700 1 -- 172.21.15.19:0/936721100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f1fd4005400 con 0x556a0a7a6540 2024-05-09T02:26:50.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.103+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/936721100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fd80044e0 con 0x556a0a7a6540 2024-05-09T02:26:50.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.109+0000 7f1fc37fe700 1 -- 172.21.15.19:0/936721100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1fd4030c30 con 0x556a0a7a6540 2024-05-09T02:26:50.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.110+0000 7f1ff8ef8b80 10 client.17163 fetch_fsmap learned FSMap version 58 2024-05-09T02:26:50.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.110+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/936721100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556a0a831530 con 0x556a0a7a6540 2024-05-09T02:26:50.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.110+0000 7f1fc37fe700 1 -- 172.21.15.19:0/936721100 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fd80044e0 con 0x556a0a7a6540 2024-05-09T02:26:50.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.112+0000 7f1fc37fe700 1 -- 172.21.15.19:0/936721100 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7f1fd402e040 con 0x556a0a7a6540 2024-05-09T02:26:50.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.112+0000 7f1ff8ef8b80 10 client.17163 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:26:50.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.112+0000 7f1ff8ef8b80 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:26:50.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.112+0000 7f1ff8ef8b80 20 client.17163 populate_metadata read hostname 'smithi019' 2024-05-09T02:26:50.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.112+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/936721100 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556a0a831b20 0x556a0a833f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:50.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.112+0000 7f1ff8ef8b80 4 client.17163 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T02:26:50.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.112+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/936721100 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a0a6c6b20 con 0x556a0a831b20 2024-05-09T02:26:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.116+0000 7f1fe3fff700 1 --2- 172.21.15.19:0/936721100 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556a0a831b20 0x556a0a833f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.116+0000 7f1fc37fe700 10 client.17163 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.116+0000 7f1fe3fff700 1 --2- 172.21.15.19:0/936721100 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556a0a831b20 0x556a0a833f40 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b512478e31e9619e server_cookie=adef6e10cc8b4385 in_seq=0 out_seq=0 2024-05-09T02:26:50.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.117+0000 7f1fc37fe700 1 -- 172.21.15.19:0/936721100 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556a0a6c6b20 con 0x556a0a831b20 2024-05-09T02:26:50.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.117+0000 7f1fc37fe700 10 client.17163 handle_command_reply: tid=0 2024-05-09T02:26:50.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.117+0000 7f1ff8ef8b80 1 client.17163 shutdown 2024-05-09T02:26:50.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.123+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/936721100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a7a6540 msgr2=0x556a0a82ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:50.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.123+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/936721100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a7a6540 0x556a0a82ff10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4030830 tx=0x7f1fd40304f0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.127+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/936721100 shutdown_connections 2024-05-09T02:26:50.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.127+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/936721100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a0a7a6540 0x556a0a82ff10 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.127+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/936721100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a0a61f4a0 0x556a0a7ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.127+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/936721100 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x556a0a831b20 0x556a0a833f40 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.127+0000 7f1ff8ef8b80 1 --2- 172.21.15.19:0/936721100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a0a699d00 0x556a0a7ba900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.127+0000 7f1fc37fe700 0 client.17163 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:26:50.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.127+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/936721100 >> 172.21.15.19:0/936721100 conn(0x556a0a622f70 msgr2=0x556a0a79e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:50.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.127+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/936721100 shutdown_connections 2024-05-09T02:26:50.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.127+0000 7f1ff8ef8b80 1 -- 172.21.15.19:0/936721100 wait complete. 2024-05-09T02:26:50.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.127+0000 7f1ff8ef8b80 20 client.17163 trim_cache size 0 max 16384 2024-05-09T02:26:50.132 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:26:50.132 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-09T02:26:50.132 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-09T02:26:50.132 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:26:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.130+0000 7f1fe9280700 1 -- 172.21.15.19:0/1565696273 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1fcc0b2c00 msgr2=0x7f1fcc0b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:50.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.130+0000 7f1fe9280700 1 --2- 172.21.15.19:0/1565696273 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1fcc0b2c00 0x7f1fcc0b5110 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x556a0a798110 tx=0x7f1fd401a040 comp rx=0 tx=0).stop 2024-05-09T02:26:50.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.130+0000 7f1fe9280700 1 -- 172.21.15.19:0/1565696273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe4110ad0 msgr2=0x7f1fe41af610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:50.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.130+0000 7f1fe9280700 1 --2- 172.21.15.19:0/1565696273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe4110ad0 0x7f1fe41af610 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00bd70 tx=0x7f1fdc00bda0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.133+0000 7f1fe9280700 1 -- 172.21.15.19:0/1565696273 shutdown_connections 2024-05-09T02:26:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.133+0000 7f1fe9280700 1 --2- 172.21.15.19:0/1565696273 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f1fcc0b2c00 0x7f1fcc0b5110 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.133+0000 7f1fe9280700 1 --2- 172.21.15.19:0/1565696273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4112740 0x7f1fe41a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.133+0000 7f1fe9280700 1 --2- 172.21.15.19:0/1565696273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe4110ad0 0x7f1fe41af610 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.133+0000 7f1fe9280700 1 --2- 172.21.15.19:0/1565696273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe4110fc0 0x7f1fe41afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.133+0000 7f1fe9280700 1 -- 172.21.15.19:0/1565696273 >> 172.21.15.19:0/1565696273 conn(0x7f1fe406f260 msgr2=0x7f1fe410c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.134+0000 7f1fe9280700 1 -- 172.21.15.19:0/1565696273 shutdown_connections 2024-05-09T02:26:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:50.134+0000 7f1fe9280700 1 -- 172.21.15.19:0/1565696273 wait complete. 2024-05-09T02:26:50.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:50 smithi019 ceph-mon[30151]: pgmap v487: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 10 MiB/s wr, 1.06k op/s 2024-05-09T02:26:50.919 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T02:26:50.919 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:26:50.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:50 smithi059 ceph-mon[34781]: pgmap v487: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 10 MiB/s wr, 1.06k op/s 2024-05-09T02:26:51.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:50 smithi196 ceph-mon[34718]: pgmap v487: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 10 MiB/s wr, 1.06k op/s 2024-05-09T02:26:51.333 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:26:52.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:52 smithi019 ceph-mon[30151]: pgmap v488: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 15 MiB/s wr, 1.46k op/s 2024-05-09T02:26:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:52 smithi059 ceph-mon[34781]: pgmap v488: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 15 MiB/s wr, 1.46k op/s 2024-05-09T02:26:53.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:52 smithi196 ceph-mon[34718]: pgmap v488: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 15 MiB/s wr, 1.46k op/s 2024-05-09T02:26:53.220 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-09T02:26:53.297 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/: 2024-05-09T02:26:53.297 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.297 INFO:teuthology.orchestra.run.smithi019.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 . 2024-05-09T02:26:53.297 INFO:teuthology.orchestra.run.smithi019.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 .. 2024-05-09T02:26:53.297 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 data 2024-05-09T02:26:53.297 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.297 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data: 2024-05-09T02:26:53.297 INFO:teuthology.orchestra.run.smithi019.stdout:total 36 2024-05-09T02:26:53.297 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 . 2024-05-09T02:26:53.297 INFO:teuthology.orchestra.run.smithi019.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 .. 2024-05-09T02:26:53.297 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 9 02:11 base 2024-05-09T02:26:53.297 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 May 9 02:11 current_logfiles 2024-05-09T02:26:53.297 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 May 9 02:11 global 2024-05-09T02:26:53.298 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 log 2024-05-09T02:26:53.298 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_commit_ts 2024-05-09T02:26:53.298 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_dynshmem 2024-05-09T02:26:53.298 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 9 02:11 pg_hba.conf 2024-05-09T02:26:53.298 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 9 02:11 pg_ident.conf 2024-05-09T02:26:53.298 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:26 pg_logical 2024-05-09T02:26:53.298 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 9 02:11 pg_multixact 2024-05-09T02:26:53.298 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_notify 2024-05-09T02:26:53.298 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_replslot 2024-05-09T02:26:53.298 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_serial 2024-05-09T02:26:53.298 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_snapshots 2024-05-09T02:26:53.298 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:26 pg_stat 2024-05-09T02:26:53.298 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:26 pg_stat_tmp 2024-05-09T02:26:53.298 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:26 pg_subtrans 2024-05-09T02:26:53.299 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_tblspc 2024-05-09T02:26:53.299 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pg_twophase 2024-05-09T02:26:53.299 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 PG_VERSION 2024-05-09T02:26:53.299 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 May 9 02:26 pg_wal 2024-05-09T02:26:53.299 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 9 02:18 pg_xact 2024-05-09T02:26:53.299 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 9 02:11 postgresql.auto.conf 2024-05-09T02:26:53.299 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 9 02:11 postgresql.conf 2024-05-09T02:26:53.299 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 May 9 02:11 postmaster.opts 2024-05-09T02:26:53.299 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.299 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/base: 2024-05-09T02:26:53.299 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.299 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 9 02:11 . 2024-05-09T02:26:53.299 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.300 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 9 02:11 1 2024-05-09T02:26:53.300 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 9 02:11 13493 2024-05-09T02:26:53.300 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 May 9 02:15 13494 2024-05-09T02:26:53.300 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 pgsql_tmp 2024-05-09T02:26:53.300 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.300 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-09T02:26:53.300 INFO:teuthology.orchestra.run.smithi019.stdout:total 7777 2024-05-09T02:26:53.300 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 9 02:11 . 2024-05-09T02:26:53.300 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 9 02:11 .. 2024-05-09T02:26:53.300 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 112 2024-05-09T02:26:53.300 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 113 2024-05-09T02:26:53.300 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 02:11 1247 2024-05-09T02:26:53.300 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1247_fsm 2024-05-09T02:26:53.300 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1247_vm 2024-05-09T02:26:53.300 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 9 02:11 1249 2024-05-09T02:26:53.301 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1249_fsm 2024-05-09T02:26:53.301 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1249_vm 2024-05-09T02:26:53.301 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 9 02:11 1255 2024-05-09T02:26:53.301 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1255_fsm 2024-05-09T02:26:53.301 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1255_vm 2024-05-09T02:26:53.301 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 9 02:11 1259 2024-05-09T02:26:53.301 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1259_fsm 2024-05-09T02:26:53.301 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1259_vm 2024-05-09T02:26:53.301 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 9 02:11 13344 2024-05-09T02:26:53.301 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13344_fsm 2024-05-09T02:26:53.301 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13344_vm 2024-05-09T02:26:53.301 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13346 2024-05-09T02:26:53.301 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13348 2024-05-09T02:26:53.301 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13349 2024-05-09T02:26:53.301 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13349_fsm 2024-05-09T02:26:53.302 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13349_vm 2024-05-09T02:26:53.302 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13351 2024-05-09T02:26:53.302 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13353 2024-05-09T02:26:53.302 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13354 2024-05-09T02:26:53.302 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13354_fsm 2024-05-09T02:26:53.302 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13354_vm 2024-05-09T02:26:53.302 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13356 2024-05-09T02:26:53.302 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13358 2024-05-09T02:26:53.302 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13359 2024-05-09T02:26:53.302 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13359_fsm 2024-05-09T02:26:53.302 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13359_vm 2024-05-09T02:26:53.302 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13361 2024-05-09T02:26:53.302 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13363 2024-05-09T02:26:53.302 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 1417 2024-05-09T02:26:53.302 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 1418 2024-05-09T02:26:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 174 2024-05-09T02:26:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 175 2024-05-09T02:26:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2187 2024-05-09T02:26:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2224 2024-05-09T02:26:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2328 2024-05-09T02:26:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2336 2024-05-09T02:26:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2337 2024-05-09T02:26:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2579 2024-05-09T02:26:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2600 2024-05-09T02:26:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2600_fsm 2024-05-09T02:26:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2600_vm 2024-05-09T02:26:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2601 2024-05-09T02:26:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2601_fsm 2024-05-09T02:26:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2601_vm 2024-05-09T02:26:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2602 2024-05-09T02:26:53.304 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2602_fsm 2024-05-09T02:26:53.304 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2602_vm 2024-05-09T02:26:53.304 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2603 2024-05-09T02:26:53.304 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2603_fsm 2024-05-09T02:26:53.304 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2603_vm 2024-05-09T02:26:53.304 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2604 2024-05-09T02:26:53.304 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2605 2024-05-09T02:26:53.304 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2605_fsm 2024-05-09T02:26:53.304 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2605_vm 2024-05-09T02:26:53.304 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2606 2024-05-09T02:26:53.304 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2606_fsm 2024-05-09T02:26:53.304 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2606_vm 2024-05-09T02:26:53.304 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2607 2024-05-09T02:26:53.304 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2607_fsm 2024-05-09T02:26:53.304 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2607_vm 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 9 02:11 2608 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2608_fsm 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2608_vm 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 9 02:11 2609 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2609_fsm 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2609_vm 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2610 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2610_fsm 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2610_vm 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2611 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2612 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2612_fsm 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2612_vm 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2613 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2615 2024-05-09T02:26:53.305 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2615_fsm 2024-05-09T02:26:53.306 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2615_vm 2024-05-09T02:26:53.306 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2616 2024-05-09T02:26:53.306 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2616_fsm 2024-05-09T02:26:53.306 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2616_vm 2024-05-09T02:26:53.306 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 9 02:11 2617 2024-05-09T02:26:53.306 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2617_fsm 2024-05-09T02:26:53.306 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2617_vm 2024-05-09T02:26:53.306 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 9 02:11 2618 2024-05-09T02:26:53.306 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2618_fsm 2024-05-09T02:26:53.306 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2618_vm 2024-05-09T02:26:53.306 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 9 02:11 2619 2024-05-09T02:26:53.306 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2619_fsm 2024-05-09T02:26:53.306 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2619_vm 2024-05-09T02:26:53.306 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2620 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2650 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2651 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2652 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2653 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2654 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2655 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2656 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2657 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 9 02:11 2658 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 02:11 2659 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2660 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2661 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2662 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2663 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2664 2024-05-09T02:26:53.307 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2665 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2666 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2667 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2668 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2669 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2670 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 9 02:11 2673 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 9 02:11 2674 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 9 02:11 2675 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2678 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2679 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2680 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2681 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2682 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2683 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2684 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2685 2024-05-09T02:26:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2686 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2687 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2688 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2689 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 9 02:11 2690 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 9 02:11 2691 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2692 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2693 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2696 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2699 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2701 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2702 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2703 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2704 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2753 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2753_fsm 2024-05-09T02:26:53.309 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2753_vm 2024-05-09T02:26:53.310 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2754 2024-05-09T02:26:53.310 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2755 2024-05-09T02:26:53.310 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2756 2024-05-09T02:26:53.310 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2757 2024-05-09T02:26:53.310 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2830 2024-05-09T02:26:53.310 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2831 2024-05-09T02:26:53.310 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2832 2024-05-09T02:26:53.310 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2833 2024-05-09T02:26:53.310 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2834 2024-05-09T02:26:53.310 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2835 2024-05-09T02:26:53.310 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2836 2024-05-09T02:26:53.310 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2837 2024-05-09T02:26:53.310 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 9 02:11 2838 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2838_fsm 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2838_vm 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2839 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2840 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2840_fsm 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2840_vm 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2841 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2995 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2996 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3079 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3079_fsm 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3079_vm 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3080 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3081 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 3085 2024-05-09T02:26:53.311 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3118 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3119 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 9 02:11 3164 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3256 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3257 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3258 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3350 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3351 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3379 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3380 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3381 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3394 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3394_fsm 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3394_vm 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3395 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3429 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3430 2024-05-09T02:26:53.312 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3431 2024-05-09T02:26:53.313 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3433 2024-05-09T02:26:53.313 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3439 2024-05-09T02:26:53.313 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3440 2024-05-09T02:26:53.313 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 3455 2024-05-09T02:26:53.313 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 9 02:11 3456 2024-05-09T02:26:53.313 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3456_fsm 2024-05-09T02:26:53.313 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3456_vm 2024-05-09T02:26:53.313 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3466 2024-05-09T02:26:53.313 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3467 2024-05-09T02:26:53.313 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3468 2024-05-09T02:26:53.313 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3501 2024-05-09T02:26:53.313 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3502 2024-05-09T02:26:53.313 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3503 2024-05-09T02:26:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3534 2024-05-09T02:26:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3541 2024-05-09T02:26:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3541_fsm 2024-05-09T02:26:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3541_vm 2024-05-09T02:26:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3542 2024-05-09T02:26:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3574 2024-05-09T02:26:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3575 2024-05-09T02:26:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3576 2024-05-09T02:26:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3596 2024-05-09T02:26:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3597 2024-05-09T02:26:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3598 2024-05-09T02:26:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3599 2024-05-09T02:26:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3600 2024-05-09T02:26:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3600_fsm 2024-05-09T02:26:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3600_vm 2024-05-09T02:26:53.315 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3601 2024-05-09T02:26:53.315 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3601_fsm 2024-05-09T02:26:53.315 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3601_vm 2024-05-09T02:26:53.315 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3602 2024-05-09T02:26:53.315 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3602_fsm 2024-05-09T02:26:53.315 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3602_vm 2024-05-09T02:26:53.315 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3603 2024-05-09T02:26:53.315 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3603_fsm 2024-05-09T02:26:53.315 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3603_vm 2024-05-09T02:26:53.315 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3604 2024-05-09T02:26:53.315 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3605 2024-05-09T02:26:53.315 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3606 2024-05-09T02:26:53.315 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3607 2024-05-09T02:26:53.315 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3608 2024-05-09T02:26:53.315 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 3609 2024-05-09T02:26:53.316 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3712 2024-05-09T02:26:53.316 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3764 2024-05-09T02:26:53.316 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3764_fsm 2024-05-09T02:26:53.316 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3764_vm 2024-05-09T02:26:53.316 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3766 2024-05-09T02:26:53.316 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3767 2024-05-09T02:26:53.316 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3997 2024-05-09T02:26:53.316 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4143 2024-05-09T02:26:53.316 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4144 2024-05-09T02:26:53.316 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4145 2024-05-09T02:26:53.316 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4146 2024-05-09T02:26:53.316 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4147 2024-05-09T02:26:53.316 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4148 2024-05-09T02:26:53.316 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4149 2024-05-09T02:26:53.316 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4150 2024-05-09T02:26:53.317 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4151 2024-05-09T02:26:53.317 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4152 2024-05-09T02:26:53.317 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4153 2024-05-09T02:26:53.317 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4154 2024-05-09T02:26:53.317 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4155 2024-05-09T02:26:53.317 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4156 2024-05-09T02:26:53.317 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4157 2024-05-09T02:26:53.317 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4158 2024-05-09T02:26:53.317 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4159 2024-05-09T02:26:53.317 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4160 2024-05-09T02:26:53.317 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4161 2024-05-09T02:26:53.317 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4162 2024-05-09T02:26:53.318 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4163 2024-05-09T02:26:53.318 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4164 2024-05-09T02:26:53.318 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4165 2024-05-09T02:26:53.319 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4166 2024-05-09T02:26:53.319 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4167 2024-05-09T02:26:53.319 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4168 2024-05-09T02:26:53.319 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4169 2024-05-09T02:26:53.319 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4170 2024-05-09T02:26:53.319 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4171 2024-05-09T02:26:53.319 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4172 2024-05-09T02:26:53.319 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4173 2024-05-09T02:26:53.319 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4174 2024-05-09T02:26:53.319 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 5002 2024-05-09T02:26:53.319 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 548 2024-05-09T02:26:53.319 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 549 2024-05-09T02:26:53.319 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6102 2024-05-09T02:26:53.319 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6104 2024-05-09T02:26:53.319 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6106 2024-05-09T02:26:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6110 2024-05-09T02:26:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6111 2024-05-09T02:26:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6112 2024-05-09T02:26:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6113 2024-05-09T02:26:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6117 2024-05-09T02:26:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 826 2024-05-09T02:26:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 827 2024-05-09T02:26:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 828 2024-05-09T02:26:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 02:11 pg_filenode.map 2024-05-09T02:26:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 PG_VERSION 2024-05-09T02:26:53.320 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-09T02:26:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:total 7777 2024-05-09T02:26:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 9 02:11 . 2024-05-09T02:26:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 9 02:11 .. 2024-05-09T02:26:53.321 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 112 2024-05-09T02:26:53.321 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 113 2024-05-09T02:26:53.321 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 02:11 1247 2024-05-09T02:26:53.321 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1247_fsm 2024-05-09T02:26:53.321 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1247_vm 2024-05-09T02:26:53.321 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 9 02:11 1249 2024-05-09T02:26:53.321 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1249_fsm 2024-05-09T02:26:53.321 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1249_vm 2024-05-09T02:26:53.321 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 9 02:11 1255 2024-05-09T02:26:53.321 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1255_fsm 2024-05-09T02:26:53.321 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1255_vm 2024-05-09T02:26:53.321 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 9 02:11 1259 2024-05-09T02:26:53.321 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1259_fsm 2024-05-09T02:26:53.321 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1259_vm 2024-05-09T02:26:53.322 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 9 02:11 13344 2024-05-09T02:26:53.322 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13344_fsm 2024-05-09T02:26:53.322 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13344_vm 2024-05-09T02:26:53.322 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13346 2024-05-09T02:26:53.322 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13348 2024-05-09T02:26:53.322 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13349 2024-05-09T02:26:53.322 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13349_fsm 2024-05-09T02:26:53.322 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13349_vm 2024-05-09T02:26:53.322 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13351 2024-05-09T02:26:53.322 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13353 2024-05-09T02:26:53.322 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13354 2024-05-09T02:26:53.322 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13354_fsm 2024-05-09T02:26:53.322 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13354_vm 2024-05-09T02:26:53.322 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13356 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13358 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13359 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13359_fsm 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13359_vm 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13361 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13363 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 1417 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 1418 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 174 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 175 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2187 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2224 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2328 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2336 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2337 2024-05-09T02:26:53.323 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2579 2024-05-09T02:26:53.324 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2600 2024-05-09T02:26:53.324 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2600_fsm 2024-05-09T02:26:53.324 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2600_vm 2024-05-09T02:26:53.324 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2601 2024-05-09T02:26:53.324 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2601_fsm 2024-05-09T02:26:53.324 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2601_vm 2024-05-09T02:26:53.324 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2602 2024-05-09T02:26:53.324 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2602_fsm 2024-05-09T02:26:53.324 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2602_vm 2024-05-09T02:26:53.324 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2603 2024-05-09T02:26:53.324 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2603_fsm 2024-05-09T02:26:53.324 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2603_vm 2024-05-09T02:26:53.324 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2604 2024-05-09T02:26:53.324 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2605 2024-05-09T02:26:53.324 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2605_fsm 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2605_vm 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2606 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2606_fsm 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2606_vm 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2607 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2607_fsm 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2607_vm 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 9 02:11 2608 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2608_fsm 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2608_vm 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 9 02:11 2609 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2609_fsm 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2609_vm 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2610 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2610_fsm 2024-05-09T02:26:53.325 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2610_vm 2024-05-09T02:26:53.326 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2611 2024-05-09T02:26:53.326 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2612 2024-05-09T02:26:53.326 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2612_fsm 2024-05-09T02:26:53.326 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2612_vm 2024-05-09T02:26:53.326 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2613 2024-05-09T02:26:53.326 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2615 2024-05-09T02:26:53.326 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2615_fsm 2024-05-09T02:26:53.326 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2615_vm 2024-05-09T02:26:53.326 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2616 2024-05-09T02:26:53.326 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2616_fsm 2024-05-09T02:26:53.326 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2616_vm 2024-05-09T02:26:53.326 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 9 02:11 2617 2024-05-09T02:26:53.326 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2617_fsm 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2617_vm 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 9 02:11 2618 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2618_fsm 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2618_vm 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 9 02:11 2619 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2619_fsm 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2619_vm 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2620 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2650 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2651 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2652 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2653 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2654 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2655 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2656 2024-05-09T02:26:53.327 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2657 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 9 02:11 2658 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 02:11 2659 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2660 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2661 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2662 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2663 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2664 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2665 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2666 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2667 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2668 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2669 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2670 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 9 02:11 2673 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 9 02:11 2674 2024-05-09T02:26:53.328 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 9 02:11 2675 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2678 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2679 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2680 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2681 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2682 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2683 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2684 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2685 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2686 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2687 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2688 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2689 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 9 02:11 2690 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 9 02:11 2691 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2692 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2693 2024-05-09T02:26:53.329 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2696 2024-05-09T02:26:53.330 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2699 2024-05-09T02:26:53.330 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2701 2024-05-09T02:26:53.330 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2702 2024-05-09T02:26:53.330 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2703 2024-05-09T02:26:53.330 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2704 2024-05-09T02:26:53.330 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2753 2024-05-09T02:26:53.330 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2753_fsm 2024-05-09T02:26:53.330 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2753_vm 2024-05-09T02:26:53.330 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2754 2024-05-09T02:26:53.330 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2755 2024-05-09T02:26:53.330 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2756 2024-05-09T02:26:53.330 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2757 2024-05-09T02:26:53.330 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2830 2024-05-09T02:26:53.330 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2831 2024-05-09T02:26:53.331 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2832 2024-05-09T02:26:53.331 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2833 2024-05-09T02:26:53.331 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2834 2024-05-09T02:26:53.331 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2835 2024-05-09T02:26:53.331 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2836 2024-05-09T02:26:53.331 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2837 2024-05-09T02:26:53.331 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 9 02:11 2838 2024-05-09T02:26:53.331 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2838_fsm 2024-05-09T02:26:53.331 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2838_vm 2024-05-09T02:26:53.331 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2839 2024-05-09T02:26:53.331 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2840 2024-05-09T02:26:53.331 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2840_fsm 2024-05-09T02:26:53.331 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2840_vm 2024-05-09T02:26:53.331 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2841 2024-05-09T02:26:53.331 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2995 2024-05-09T02:26:53.332 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2996 2024-05-09T02:26:53.332 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3079 2024-05-09T02:26:53.332 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3079_fsm 2024-05-09T02:26:53.332 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3079_vm 2024-05-09T02:26:53.332 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3080 2024-05-09T02:26:53.332 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3081 2024-05-09T02:26:53.332 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 3085 2024-05-09T02:26:53.332 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3118 2024-05-09T02:26:53.332 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3119 2024-05-09T02:26:53.332 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 9 02:11 3164 2024-05-09T02:26:53.332 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3256 2024-05-09T02:26:53.332 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3257 2024-05-09T02:26:53.332 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3258 2024-05-09T02:26:53.332 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3350 2024-05-09T02:26:53.332 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3351 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3379 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3380 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3381 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3394 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3394_fsm 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3394_vm 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3395 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3429 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3430 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3431 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3433 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3439 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3440 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 3455 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 9 02:11 3456 2024-05-09T02:26:53.333 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3456_fsm 2024-05-09T02:26:53.334 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3456_vm 2024-05-09T02:26:53.334 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3466 2024-05-09T02:26:53.334 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3467 2024-05-09T02:26:53.334 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3468 2024-05-09T02:26:53.334 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3501 2024-05-09T02:26:53.334 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3502 2024-05-09T02:26:53.334 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3503 2024-05-09T02:26:53.334 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3534 2024-05-09T02:26:53.334 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3541 2024-05-09T02:26:53.334 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3541_fsm 2024-05-09T02:26:53.334 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3541_vm 2024-05-09T02:26:53.334 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3542 2024-05-09T02:26:53.334 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3574 2024-05-09T02:26:53.334 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3575 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3576 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3596 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3597 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3598 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3599 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3600 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3600_fsm 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3600_vm 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3601 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3601_fsm 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3601_vm 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3602 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3602_fsm 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3602_vm 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3603 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3603_fsm 2024-05-09T02:26:53.335 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3603_vm 2024-05-09T02:26:53.336 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3604 2024-05-09T02:26:53.336 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3605 2024-05-09T02:26:53.336 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3606 2024-05-09T02:26:53.336 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3607 2024-05-09T02:26:53.336 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3608 2024-05-09T02:26:53.336 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 3609 2024-05-09T02:26:53.336 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3712 2024-05-09T02:26:53.336 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3764 2024-05-09T02:26:53.336 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3764_fsm 2024-05-09T02:26:53.336 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3764_vm 2024-05-09T02:26:53.336 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3766 2024-05-09T02:26:53.336 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3767 2024-05-09T02:26:53.336 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3997 2024-05-09T02:26:53.336 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4143 2024-05-09T02:26:53.337 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4144 2024-05-09T02:26:53.337 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4145 2024-05-09T02:26:53.337 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4146 2024-05-09T02:26:53.337 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4147 2024-05-09T02:26:53.337 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4148 2024-05-09T02:26:53.337 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4149 2024-05-09T02:26:53.337 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4150 2024-05-09T02:26:53.337 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4151 2024-05-09T02:26:53.337 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4152 2024-05-09T02:26:53.337 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4153 2024-05-09T02:26:53.337 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4154 2024-05-09T02:26:53.337 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4155 2024-05-09T02:26:53.337 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4156 2024-05-09T02:26:53.337 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4157 2024-05-09T02:26:53.337 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4158 2024-05-09T02:26:53.338 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4159 2024-05-09T02:26:53.338 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4160 2024-05-09T02:26:53.338 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4161 2024-05-09T02:26:53.338 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4162 2024-05-09T02:26:53.338 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4163 2024-05-09T02:26:53.338 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4164 2024-05-09T02:26:53.338 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4165 2024-05-09T02:26:53.338 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4166 2024-05-09T02:26:53.338 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4167 2024-05-09T02:26:53.338 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4168 2024-05-09T02:26:53.338 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4169 2024-05-09T02:26:53.338 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4170 2024-05-09T02:26:53.338 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4171 2024-05-09T02:26:53.338 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4172 2024-05-09T02:26:53.369 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4173 2024-05-09T02:26:53.369 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4174 2024-05-09T02:26:53.369 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 5002 2024-05-09T02:26:53.369 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 548 2024-05-09T02:26:53.369 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 549 2024-05-09T02:26:53.369 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6102 2024-05-09T02:26:53.369 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6104 2024-05-09T02:26:53.369 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6106 2024-05-09T02:26:53.369 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6110 2024-05-09T02:26:53.369 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6111 2024-05-09T02:26:53.369 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6112 2024-05-09T02:26:53.369 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6113 2024-05-09T02:26:53.370 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6117 2024-05-09T02:26:53.370 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 826 2024-05-09T02:26:53.370 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 827 2024-05-09T02:26:53.370 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 828 2024-05-09T02:26:53.370 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 02:11 pg_filenode.map 2024-05-09T02:26:53.370 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 PG_VERSION 2024-05-09T02:26:53.370 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.370 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-09T02:26:53.370 INFO:teuthology.orchestra.run.smithi019.stdout:total 606038 2024-05-09T02:26:53.370 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 May 9 02:15 . 2024-05-09T02:26:53.370 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 9 02:11 .. 2024-05-09T02:26:53.370 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 112 2024-05-09T02:26:53.370 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 113 2024-05-09T02:26:53.370 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 02:11 1247 2024-05-09T02:26:53.371 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1247_fsm 2024-05-09T02:26:53.371 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1247_vm 2024-05-09T02:26:53.371 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 9 02:11 1249 2024-05-09T02:26:53.371 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1249_fsm 2024-05-09T02:26:53.371 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1249_vm 2024-05-09T02:26:53.371 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 9 02:11 1255 2024-05-09T02:26:53.371 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1255_fsm 2024-05-09T02:26:53.371 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1255_vm 2024-05-09T02:26:53.371 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 9 02:26 1259 2024-05-09T02:26:53.371 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1259_fsm 2024-05-09T02:26:53.371 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1259_vm 2024-05-09T02:26:53.371 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 9 02:11 13344 2024-05-09T02:26:53.371 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13344_fsm 2024-05-09T02:26:53.371 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13344_vm 2024-05-09T02:26:53.371 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13346 2024-05-09T02:26:53.372 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13348 2024-05-09T02:26:53.372 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13349 2024-05-09T02:26:53.372 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13349_fsm 2024-05-09T02:26:53.372 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13349_vm 2024-05-09T02:26:53.372 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13351 2024-05-09T02:26:53.372 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13353 2024-05-09T02:26:53.372 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13354 2024-05-09T02:26:53.372 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13354_fsm 2024-05-09T02:26:53.372 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13354_vm 2024-05-09T02:26:53.372 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13356 2024-05-09T02:26:53.372 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13358 2024-05-09T02:26:53.372 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13359 2024-05-09T02:26:53.372 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 13359_fsm 2024-05-09T02:26:53.372 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13359_vm 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 13361 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 13363 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 1417 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 1418 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 443670528 May 9 02:26 16396 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 9 02:26 16396_fsm 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:15 16396_vm 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 778240 May 9 02:26 16397 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:26 16397_fsm 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:26 16397_vm 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1351680 May 9 02:26 16399 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:26 16399_fsm 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:26 16399_vm 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:26 16400 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 9 02:26 16402 2024-05-09T02:26:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 May 9 02:26 16404 2024-05-09T02:26:53.374 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 94396416 May 9 02:26 16406 2024-05-09T02:26:53.374 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:26 16406_fsm 2024-05-09T02:26:53.374 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:25 16406_vm 2024-05-09T02:26:53.374 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 174 2024-05-09T02:26:53.374 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 175 2024-05-09T02:26:53.374 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2187 2024-05-09T02:26:53.374 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2224 2024-05-09T02:26:53.374 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2328 2024-05-09T02:26:53.374 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2336 2024-05-09T02:26:53.374 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2337 2024-05-09T02:26:53.374 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2579 2024-05-09T02:26:53.374 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2600 2024-05-09T02:26:53.374 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2600_fsm 2024-05-09T02:26:53.374 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2600_vm 2024-05-09T02:26:53.374 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2601 2024-05-09T02:26:53.375 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2601_fsm 2024-05-09T02:26:53.375 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2601_vm 2024-05-09T02:26:53.375 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2602 2024-05-09T02:26:53.375 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2602_fsm 2024-05-09T02:26:53.375 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2602_vm 2024-05-09T02:26:53.375 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2603 2024-05-09T02:26:53.375 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2603_fsm 2024-05-09T02:26:53.375 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2603_vm 2024-05-09T02:26:53.375 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2604 2024-05-09T02:26:53.375 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2605 2024-05-09T02:26:53.375 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2605_fsm 2024-05-09T02:26:53.375 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2605_vm 2024-05-09T02:26:53.375 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2606 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2606_fsm 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2606_vm 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2607 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2607_fsm 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2607_vm 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 9 02:11 2608 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2608_fsm 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2608_vm 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 9 02:11 2609 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2609_fsm 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2609_vm 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2610 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2610_fsm 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2610_vm 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2611 2024-05-09T02:26:53.376 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2612 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2612_fsm 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2612_vm 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2613 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2615 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2615_fsm 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2615_vm 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2616 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2616_fsm 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2616_vm 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 9 02:11 2617 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2617_fsm 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2617_vm 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 9 02:11 2618 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2618_fsm 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2618_vm 2024-05-09T02:26:53.377 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 May 9 02:26 2619 2024-05-09T02:26:53.378 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:13 2619_fsm 2024-05-09T02:26:53.378 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2619_vm 2024-05-09T02:26:53.378 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2620 2024-05-09T02:26:53.378 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2650 2024-05-09T02:26:53.378 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2651 2024-05-09T02:26:53.378 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2652 2024-05-09T02:26:53.378 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2653 2024-05-09T02:26:53.378 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2654 2024-05-09T02:26:53.378 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2655 2024-05-09T02:26:53.378 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2656 2024-05-09T02:26:53.378 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2657 2024-05-09T02:26:53.378 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 9 02:11 2658 2024-05-09T02:26:53.378 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 9 02:11 2659 2024-05-09T02:26:53.378 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2660 2024-05-09T02:26:53.379 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2661 2024-05-09T02:26:53.379 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2662 2024-05-09T02:26:53.379 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2663 2024-05-09T02:26:53.379 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2664 2024-05-09T02:26:53.379 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2665 2024-05-09T02:26:53.379 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2666 2024-05-09T02:26:53.379 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2667 2024-05-09T02:26:53.379 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2668 2024-05-09T02:26:53.379 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2669 2024-05-09T02:26:53.379 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2670 2024-05-09T02:26:53.379 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 9 02:11 2673 2024-05-09T02:26:53.379 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 9 02:11 2674 2024-05-09T02:26:53.379 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 9 02:11 2675 2024-05-09T02:26:53.379 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2678 2024-05-09T02:26:53.379 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2679 2024-05-09T02:26:53.380 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2680 2024-05-09T02:26:53.380 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2681 2024-05-09T02:26:53.380 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2682 2024-05-09T02:26:53.380 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2683 2024-05-09T02:26:53.380 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2684 2024-05-09T02:26:53.380 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2685 2024-05-09T02:26:53.380 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2686 2024-05-09T02:26:53.380 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2687 2024-05-09T02:26:53.380 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2688 2024-05-09T02:26:53.380 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 9 02:11 2689 2024-05-09T02:26:53.380 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 9 02:11 2690 2024-05-09T02:26:53.380 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 9 02:11 2691 2024-05-09T02:26:53.380 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2692 2024-05-09T02:26:53.380 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2693 2024-05-09T02:26:53.380 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:16 2696 2024-05-09T02:26:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2699 2024-05-09T02:26:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2701 2024-05-09T02:26:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2702 2024-05-09T02:26:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2703 2024-05-09T02:26:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 2704 2024-05-09T02:26:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2753 2024-05-09T02:26:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2753_fsm 2024-05-09T02:26:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2753_vm 2024-05-09T02:26:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2754 2024-05-09T02:26:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2755 2024-05-09T02:26:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2756 2024-05-09T02:26:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 2757 2024-05-09T02:26:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2830 2024-05-09T02:26:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2831 2024-05-09T02:26:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2832 2024-05-09T02:26:53.382 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2833 2024-05-09T02:26:53.382 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2834 2024-05-09T02:26:53.382 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2835 2024-05-09T02:26:53.382 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2836 2024-05-09T02:26:53.382 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2837 2024-05-09T02:26:53.382 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 9 02:11 2838 2024-05-09T02:26:53.382 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2838_fsm 2024-05-09T02:26:53.382 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2838_vm 2024-05-09T02:26:53.382 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2839 2024-05-09T02:26:53.382 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2840 2024-05-09T02:26:53.382 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2840_fsm 2024-05-09T02:26:53.382 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2840_vm 2024-05-09T02:26:53.382 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2841 2024-05-09T02:26:53.382 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2995 2024-05-09T02:26:53.383 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2996 2024-05-09T02:26:53.383 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3079 2024-05-09T02:26:53.383 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3079_fsm 2024-05-09T02:26:53.383 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3079_vm 2024-05-09T02:26:53.383 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3080 2024-05-09T02:26:53.383 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3081 2024-05-09T02:26:53.383 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 3085 2024-05-09T02:26:53.383 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3118 2024-05-09T02:26:53.383 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3119 2024-05-09T02:26:53.383 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 9 02:11 3164 2024-05-09T02:26:53.383 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3256 2024-05-09T02:26:53.383 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3257 2024-05-09T02:26:53.383 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3258 2024-05-09T02:26:53.383 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3350 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3351 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3379 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3380 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3381 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3394 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3394_fsm 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3394_vm 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3395 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3429 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3430 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3431 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3433 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3439 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3440 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 9 02:11 3455 2024-05-09T02:26:53.384 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 9 02:11 3456 2024-05-09T02:26:53.385 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3456_fsm 2024-05-09T02:26:53.385 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3456_vm 2024-05-09T02:26:53.385 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3466 2024-05-09T02:26:53.385 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3467 2024-05-09T02:26:53.385 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3468 2024-05-09T02:26:53.385 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3501 2024-05-09T02:26:53.385 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3502 2024-05-09T02:26:53.385 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3503 2024-05-09T02:26:53.385 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3534 2024-05-09T02:26:53.385 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3541 2024-05-09T02:26:53.385 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3541_fsm 2024-05-09T02:26:53.385 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3541_vm 2024-05-09T02:26:53.385 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3542 2024-05-09T02:26:53.385 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3574 2024-05-09T02:26:53.385 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3575 2024-05-09T02:26:53.386 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3576 2024-05-09T02:26:53.386 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3596 2024-05-09T02:26:53.386 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3597 2024-05-09T02:26:53.386 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3598 2024-05-09T02:26:53.386 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3599 2024-05-09T02:26:53.386 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3600 2024-05-09T02:26:53.386 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3600_fsm 2024-05-09T02:26:53.386 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3600_vm 2024-05-09T02:26:53.386 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3601 2024-05-09T02:26:53.386 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3601_fsm 2024-05-09T02:26:53.386 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3601_vm 2024-05-09T02:26:53.386 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3602 2024-05-09T02:26:53.386 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3602_fsm 2024-05-09T02:26:53.386 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3602_vm 2024-05-09T02:26:53.386 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3603 2024-05-09T02:26:53.387 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3603_fsm 2024-05-09T02:26:53.387 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3603_vm 2024-05-09T02:26:53.387 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3604 2024-05-09T02:26:53.387 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3605 2024-05-09T02:26:53.387 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3606 2024-05-09T02:26:53.387 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3607 2024-05-09T02:26:53.387 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3608 2024-05-09T02:26:53.387 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 9 02:11 3609 2024-05-09T02:26:53.387 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3712 2024-05-09T02:26:53.387 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3764 2024-05-09T02:26:53.387 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 3764_fsm 2024-05-09T02:26:53.388 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3764_vm 2024-05-09T02:26:53.388 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3766 2024-05-09T02:26:53.388 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 3767 2024-05-09T02:26:53.388 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3997 2024-05-09T02:26:53.388 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4143 2024-05-09T02:26:53.388 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4144 2024-05-09T02:26:53.388 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4145 2024-05-09T02:26:53.388 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4146 2024-05-09T02:26:53.388 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4147 2024-05-09T02:26:53.388 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4148 2024-05-09T02:26:53.388 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4149 2024-05-09T02:26:53.388 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4150 2024-05-09T02:26:53.388 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4151 2024-05-09T02:26:53.388 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4152 2024-05-09T02:26:53.388 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4153 2024-05-09T02:26:53.389 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4154 2024-05-09T02:26:53.389 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4155 2024-05-09T02:26:53.389 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4156 2024-05-09T02:26:53.389 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4157 2024-05-09T02:26:53.389 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4158 2024-05-09T02:26:53.389 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4159 2024-05-09T02:26:53.390 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4160 2024-05-09T02:26:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4161 2024-05-09T02:26:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4162 2024-05-09T02:26:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4163 2024-05-09T02:26:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4164 2024-05-09T02:26:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4165 2024-05-09T02:26:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4166 2024-05-09T02:26:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4167 2024-05-09T02:26:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4168 2024-05-09T02:26:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4169 2024-05-09T02:26:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4170 2024-05-09T02:26:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4171 2024-05-09T02:26:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4172 2024-05-09T02:26:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4173 2024-05-09T02:26:53.391 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4174 2024-05-09T02:26:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 5002 2024-05-09T02:26:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 548 2024-05-09T02:26:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 549 2024-05-09T02:26:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6102 2024-05-09T02:26:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6104 2024-05-09T02:26:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6106 2024-05-09T02:26:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6110 2024-05-09T02:26:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6111 2024-05-09T02:26:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6112 2024-05-09T02:26:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6113 2024-05-09T02:26:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6117 2024-05-09T02:26:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 826 2024-05-09T02:26:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 827 2024-05-09T02:26:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 828 2024-05-09T02:26:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 02:11 pg_filenode.map 2024-05-09T02:26:53.393 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 May 9 02:11 pg_internal.init 2024-05-09T02:26:53.393 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:11 PG_VERSION 2024-05-09T02:26:53.393 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.393 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-05-09T02:26:53.393 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.393 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:26:53.393 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 9 02:11 .. 2024-05-09T02:26:53.393 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.393 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/global: 2024-05-09T02:26:53.393 INFO:teuthology.orchestra.run.smithi019.stdout:total 560 2024-05-09T02:26:53.393 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 May 9 02:11 . 2024-05-09T02:26:53.393 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.393 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1213 2024-05-09T02:26:53.393 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1213_fsm 2024-05-09T02:26:53.394 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1213_vm 2024-05-09T02:26:53.394 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1214 2024-05-09T02:26:53.394 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1214_fsm 2024-05-09T02:26:53.394 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1214_vm 2024-05-09T02:26:53.394 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 1232 2024-05-09T02:26:53.394 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 1233 2024-05-09T02:26:53.394 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1260 2024-05-09T02:26:53.394 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1260_fsm 2024-05-09T02:26:53.394 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1260_vm 2024-05-09T02:26:53.394 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1261 2024-05-09T02:26:53.394 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1261_fsm 2024-05-09T02:26:53.394 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1261_vm 2024-05-09T02:26:53.394 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1262 2024-05-09T02:26:53.394 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 1262_fsm 2024-05-09T02:26:53.395 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 1262_vm 2024-05-09T02:26:53.395 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2396 2024-05-09T02:26:53.395 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 9 02:11 2396_fsm 2024-05-09T02:26:53.395 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2396_vm 2024-05-09T02:26:53.395 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2397 2024-05-09T02:26:53.395 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2671 2024-05-09T02:26:53.395 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2672 2024-05-09T02:26:53.395 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2676 2024-05-09T02:26:53.395 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2677 2024-05-09T02:26:53.395 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2694 2024-05-09T02:26:53.395 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2695 2024-05-09T02:26:53.395 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2697 2024-05-09T02:26:53.395 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 9 02:11 2698 2024-05-09T02:26:53.395 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2846 2024-05-09T02:26:53.395 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2847 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2964 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2965 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 2966 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 2967 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 3592 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 3593 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4060 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4061 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4175 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4176 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4177 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4178 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4181 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4182 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4183 2024-05-09T02:26:53.396 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4184 2024-05-09T02:26:53.397 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 4185 2024-05-09T02:26:53.397 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 4186 2024-05-09T02:26:53.397 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6000 2024-05-09T02:26:53.397 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6001 2024-05-09T02:26:53.397 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6002 2024-05-09T02:26:53.397 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 6100 2024-05-09T02:26:53.397 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6114 2024-05-09T02:26:53.397 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 6115 2024-05-09T02:26:53.397 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:26 pg_control 2024-05-09T02:26:53.397 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 9 02:11 pg_filenode.map 2024-05-09T02:26:53.397 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 May 9 02:11 pg_internal.init 2024-05-09T02:26:53.397 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.397 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/log: 2024-05-09T02:26:53.397 INFO:teuthology.orchestra.run.smithi019.stdout:total 2 2024-05-09T02:26:53.397 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 . 2024-05-09T02:26:53.398 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.398 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1118 May 9 02:26 postgresql-Thu.log 2024-05-09T02:26:53.398 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.398 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-09T02:26:53.398 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.398 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:26:53.398 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.398 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.398 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-09T02:26:53.398 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.398 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:26:53.398 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.398 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.398 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-09T02:26:53.399 INFO:teuthology.orchestra.run.smithi019.stdout:total 1 2024-05-09T02:26:53.399 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:26 . 2024-05-09T02:26:53.399 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.399 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 mappings 2024-05-09T02:26:53.399 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 9 02:26 replorigin_checkpoint 2024-05-09T02:26:53.399 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 snapshots 2024-05-09T02:26:53.399 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.399 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-09T02:26:53.399 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.399 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:26:53.399 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:26 .. 2024-05-09T02:26:53.399 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.399 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-09T02:26:53.399 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.399 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:26 .. 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 9 02:11 . 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 members 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 offsets 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout:total 8 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 . 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 9 02:11 .. 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:11 0000 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.400 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-09T02:26:53.401 INFO:teuthology.orchestra.run.smithi019.stdout:total 8 2024-05-09T02:26:53.401 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:11 . 2024-05-09T02:26:53.401 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 9 02:11 .. 2024-05-09T02:26:53.401 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 9 02:15 0000 2024-05-09T02:26:53.401 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.401 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-09T02:26:53.401 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.401 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:26:53.401 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.401 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.401 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-09T02:26:53.401 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.401 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:26:53.401 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.401 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.402 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-09T02:26:53.402 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.402 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:26:53.402 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.402 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.402 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-09T02:26:53.402 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.402 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:26:53.402 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.402 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.402 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-09T02:26:53.402 INFO:teuthology.orchestra.run.smithi019.stdout:total 15 2024-05-09T02:26:53.402 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 9 02:26 . 2024-05-09T02:26:53.402 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.403 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 May 9 02:26 db_0.stat 2024-05-09T02:26:53.403 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 May 9 02:26 db_13494.stat 2024-05-09T02:26:53.403 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 May 9 02:26 global.stat 2024-05-09T02:26:53.403 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.403 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-09T02:26:53.403 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.403 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:26 . 2024-05-09T02:26:53.403 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.403 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.403 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-09T02:26:53.403 INFO:teuthology.orchestra.run.smithi019.stdout:total 72 2024-05-09T02:26:53.403 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 9 02:26 . 2024-05-09T02:26:53.403 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.403 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 May 9 02:26 001B 2024-05-09T02:26:53.403 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout:total 1048576 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 May 9 02:26 . 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:26 00000001000000000000009F 2024-05-09T02:26:53.404 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:21 0000000100000000000000A0 2024-05-09T02:26:53.405 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:19 0000000100000000000000A1 2024-05-09T02:26:53.405 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:19 0000000100000000000000A2 2024-05-09T02:26:53.405 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:23 0000000100000000000000A3 2024-05-09T02:26:53.405 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:19 0000000100000000000000A4 2024-05-09T02:26:53.405 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:19 0000000100000000000000A5 2024-05-09T02:26:53.405 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:18 0000000100000000000000A6 2024-05-09T02:26:53.405 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:19 0000000100000000000000A7 2024-05-09T02:26:53.405 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:18 0000000100000000000000A8 2024-05-09T02:26:53.405 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:18 0000000100000000000000A9 2024-05-09T02:26:53.405 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:19 0000000100000000000000AA 2024-05-09T02:26:53.405 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:19 0000000100000000000000AB 2024-05-09T02:26:53.405 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:19 0000000100000000000000AC 2024-05-09T02:26:53.405 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:18 0000000100000000000000AD 2024-05-09T02:26:53.405 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:19 0000000100000000000000AE 2024-05-09T02:26:53.405 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:18 0000000100000000000000AF 2024-05-09T02:26:53.406 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:19 0000000100000000000000B0 2024-05-09T02:26:53.406 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:19 0000000100000000000000B1 2024-05-09T02:26:53.406 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:18 0000000100000000000000B2 2024-05-09T02:26:53.406 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:19 0000000100000000000000B3 2024-05-09T02:26:53.406 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:19 0000000100000000000000B4 2024-05-09T02:26:53.406 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:18 0000000100000000000000B5 2024-05-09T02:26:53.406 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:19 0000000100000000000000B6 2024-05-09T02:26:53.406 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:18 0000000100000000000000B7 2024-05-09T02:26:53.406 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:21 0000000100000000000000B8 2024-05-09T02:26:53.406 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:19 0000000100000000000000B9 2024-05-09T02:26:53.406 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:22 0000000100000000000000BA 2024-05-09T02:26:53.406 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:22 0000000100000000000000BB 2024-05-09T02:26:53.406 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:20 0000000100000000000000BC 2024-05-09T02:26:53.406 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:18 0000000100000000000000BD 2024-05-09T02:26:53.406 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:24 0000000100000000000000BE 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:23 0000000100000000000000BF 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:24 0000000100000000000000C0 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:23 0000000100000000000000C1 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:25 0000000100000000000000C2 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:25 0000000100000000000000C3 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:23 0000000100000000000000C4 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:24 0000000100000000000000C5 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:24 0000000100000000000000C6 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:23 0000000100000000000000C7 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:24 0000000100000000000000C8 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:23 0000000100000000000000C9 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:26 0000000100000000000000CA 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:23 0000000100000000000000CB 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:24 0000000100000000000000CC 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:24 0000000100000000000000CD 2024-05-09T02:26:53.407 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:25 0000000100000000000000CE 2024-05-09T02:26:53.408 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:24 0000000100000000000000CF 2024-05-09T02:26:53.408 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:26 0000000100000000000000D0 2024-05-09T02:26:53.408 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:23 0000000100000000000000D1 2024-05-09T02:26:53.408 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:24 0000000100000000000000D2 2024-05-09T02:26:53.408 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:23 0000000100000000000000D3 2024-05-09T02:26:53.408 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:23 0000000100000000000000D4 2024-05-09T02:26:53.408 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:23 0000000100000000000000D5 2024-05-09T02:26:53.408 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:24 0000000100000000000000D6 2024-05-09T02:26:53.408 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:23 0000000100000000000000D7 2024-05-09T02:26:53.408 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:25 0000000100000000000000D8 2024-05-09T02:26:53.408 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:23 0000000100000000000000D9 2024-05-09T02:26:53.408 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:25 0000000100000000000000DA 2024-05-09T02:26:53.408 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:24 0000000100000000000000DB 2024-05-09T02:26:53.408 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:24 0000000100000000000000DC 2024-05-09T02:26:53.408 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:24 0000000100000000000000DD 2024-05-09T02:26:53.409 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 9 02:24 0000000100000000000000DE 2024-05-09T02:26:53.409 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 archive_status 2024-05-09T02:26:53.409 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.409 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-09T02:26:53.409 INFO:teuthology.orchestra.run.smithi019.stdout:total 0 2024-05-09T02:26:53.409 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 9 02:11 . 2024-05-09T02:26:53.409 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 May 9 02:26 .. 2024-05-09T02:26:53.409 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:53.409 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-09T02:26:53.409 INFO:teuthology.orchestra.run.smithi019.stdout:total 440 2024-05-09T02:26:53.409 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 9 02:18 . 2024-05-09T02:26:53.409 INFO:teuthology.orchestra.run.smithi019.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 9 02:26 .. 2024-05-09T02:26:53.409 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 May 9 02:18 0000 2024-05-09T02:26:53.409 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 188416 May 9 02:26 0001 2024-05-09T02:26:53.410 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-09T02:26:53.421 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-09T02:26:53.421 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T02:26:53.421 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T02:26:53.422 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-09T02:26:53.462 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-09T02:26:53.533 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-09T02:26:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.103+0000 7f7d98896640 1 -- 172.21.15.19:0/363034941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d94108b90 msgr2=0x7f7d94102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.104+0000 7f7d98896640 1 --2- 172.21.15.19:0/363034941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d94108b90 0x7f7d94102730 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009900 tx=0x7f7d8402a410 comp rx=0 tx=0).stop 2024-05-09T02:26:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.104+0000 7f7d98896640 1 -- 172.21.15.19:0/363034941 shutdown_connections 2024-05-09T02:26:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.104+0000 7f7d98896640 1 --2- 172.21.15.19:0/363034941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d941032e0 0x7f7d9410f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.104+0000 7f7d98896640 1 --2- 172.21.15.19:0/363034941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d94109030 0x7f7d94102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.104+0000 7f7d98896640 1 --2- 172.21.15.19:0/363034941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d94108b90 0x7f7d94102730 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.104+0000 7f7d98896640 1 -- 172.21.15.19:0/363034941 >> 172.21.15.19:0/363034941 conn(0x7f7d940fe390 msgr2=0x7f7d941007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.104+0000 7f7d98896640 1 -- 172.21.15.19:0/363034941 shutdown_connections 2024-05-09T02:26:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.104+0000 7f7d98896640 1 -- 172.21.15.19:0/363034941 wait complete. 2024-05-09T02:26:54.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d98896640 1 Processor -- start 2024-05-09T02:26:54.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d98896640 1 -- start start 2024-05-09T02:26:54.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d98896640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d941032e0 0x7f7d941a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:54.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d98896640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d94108b90 0x7f7d941a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:54.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d98896640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94109030 0x7f7d9419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:54.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d98896640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7d94113ef0 con 0x7f7d941032e0 2024-05-09T02:26:54.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d98896640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d94113d70 con 0x7f7d94109030 2024-05-09T02:26:54.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d98896640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d94114070 con 0x7f7d94108b90 2024-05-09T02:26:54.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d9259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d941032e0 0x7f7d941a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d9259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d941032e0 0x7f7d941a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47330/0 (socket says 172.21.15.19:47330) 2024-05-09T02:26:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d9259c640 1 -- 172.21.15.19:0/1636193520 learned_addr learned my addr 172.21.15.19:0/1636193520 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d91d9b640 1 --2- 172.21.15.19:0/1636193520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d94108b90 0x7f7d941a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d92d9d640 1 --2- 172.21.15.19:0/1636193520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94109030 0x7f7d9419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d9259c640 1 -- 172.21.15.19:0/1636193520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d94108b90 msgr2=0x7f7d941a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d9259c640 1 --2- 172.21.15.19:0/1636193520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d94108b90 0x7f7d941a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d9259c640 1 -- 172.21.15.19:0/1636193520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94109030 msgr2=0x7f7d9419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d9259c640 1 --2- 172.21.15.19:0/1636193520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94109030 0x7f7d9419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d9259c640 1 -- 172.21.15.19:0/1636193520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9419ae20 con 0x7f7d941032e0 2024-05-09T02:26:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.105+0000 7f7d91d9b640 1 --2- 172.21.15.19:0/1636193520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d94108b90 0x7f7d941a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:26:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.106+0000 7f7d9259c640 1 --2- 172.21.15.19:0/1636193520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d941032e0 0x7f7d941a0570 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7d84005ab0 tx=0x7f7d84003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.106+0000 7f7d837fe640 1 -- 172.21.15.19:0/1636193520 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8403f070 con 0x7f7d941032e0 2024-05-09T02:26:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.106+0000 7f7d837fe640 1 -- 172.21.15.19:0/1636193520 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d84031d60 con 0x7f7d941032e0 2024-05-09T02:26:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.106+0000 7f7d98896640 1 -- 172.21.15.19:0/1636193520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9419b050 con 0x7f7d941032e0 2024-05-09T02:26:54.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.106+0000 7f7d837fe640 1 -- 172.21.15.19:0/1636193520 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84035030 con 0x7f7d941032e0 2024-05-09T02:26:54.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.106+0000 7f7d98896640 1 -- 172.21.15.19:0/1636193520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9419b540 con 0x7f7d941032e0 2024-05-09T02:26:54.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.107+0000 7f7d98896640 1 -- 172.21.15.19:0/1636193520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d94103e70 con 0x7f7d941032e0 2024-05-09T02:26:54.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.109+0000 7f7d837fe640 1 -- 172.21.15.19:0/1636193520 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7d84042070 con 0x7f7d941032e0 2024-05-09T02:26:54.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.109+0000 7f7d837fe640 1 --2- 172.21.15.19:0/1636193520 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7d7c0b2950 0x7f7d7c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.109+0000 7f7d91d9b640 1 --2- 172.21.15.19:0/1636193520 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7d7c0b2950 0x7f7d7c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.109+0000 7f7d837fe640 1 -- 172.21.15.19:0/1636193520 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f7d84103660 con 0x7f7d941032e0 2024-05-09T02:26:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.110+0000 7f7d91d9b640 1 --2- 172.21.15.19:0/1636193520 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7d7c0b2950 0x7f7d7c0b4e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7d9419be90 tx=0x7f7d8800b5d0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:54.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.113+0000 7f7d837fe640 1 -- 172.21.15.19:0/1636193520 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d840cafb0 con 0x7f7d941032e0 2024-05-09T02:26:54.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.332+0000 7f7d98896640 1 -- 172.21.15.19:0/1636193520 --> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f7d94100700 con 0x7f7d7c0b2950 2024-05-09T02:26:54.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.342+0000 7f7d837fe640 1 -- 172.21.15.19:0/1636193520 <== mgr.15222 v2:172.21.15.19:6800/2638922161 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f7d94100700 con 0x7f7d7c0b2950 2024-05-09T02:26:54.345 INFO:teuthology.orchestra.run.smithi019.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-09T00:00:00", "created": "2024-05-09T02:11:27", "first": "2024-05-09T02:12:00", "last": "2024-05-09T02:26:00", "last_pruned": "2024-05-09T02:26:00", "created_count": 15, "pruned_count": 9, "active": true} 2024-05-09T02:26:54.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.344+0000 7f7d98896640 1 -- 172.21.15.19:0/1636193520 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7d7c0b2950 msgr2=0x7f7d7c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:54.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.344+0000 7f7d98896640 1 --2- 172.21.15.19:0/1636193520 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7d7c0b2950 0x7f7d7c0b4e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7d9419be90 tx=0x7f7d8800b5d0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.344+0000 7f7d98896640 1 -- 172.21.15.19:0/1636193520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d941032e0 msgr2=0x7f7d941a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:54.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.344+0000 7f7d98896640 1 --2- 172.21.15.19:0/1636193520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d941032e0 0x7f7d941a0570 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7d84005ab0 tx=0x7f7d84003cd0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.344+0000 7f7d98896640 1 -- 172.21.15.19:0/1636193520 shutdown_connections 2024-05-09T02:26:54.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.344+0000 7f7d98896640 1 --2- 172.21.15.19:0/1636193520 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f7d7c0b2950 0x7f7d7c0b4e60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.344+0000 7f7d98896640 1 --2- 172.21.15.19:0/1636193520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94109030 0x7f7d9419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.344+0000 7f7d98896640 1 --2- 172.21.15.19:0/1636193520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d94108b90 0x7f7d941a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.344+0000 7f7d98896640 1 --2- 172.21.15.19:0/1636193520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d941032e0 0x7f7d941a0570 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.344+0000 7f7d98896640 1 -- 172.21.15.19:0/1636193520 >> 172.21.15.19:0/1636193520 conn(0x7f7d940fe390 msgr2=0x7f7d940ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:54.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.344+0000 7f7d98896640 1 -- 172.21.15.19:0/1636193520 shutdown_connections 2024-05-09T02:26:54.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.345+0000 7f7d98896640 1 -- 172.21.15.19:0/1636193520 wait complete. 2024-05-09T02:26:54.360 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-09T02:26:54.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.478+0000 7f0236470640 1 -- 172.21.15.19:0/3294825884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023010b7f0 msgr2=0x7f023010dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:54.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.478+0000 7f0236470640 1 --2- 172.21.15.19:0/3294825884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023010b7f0 0x7f023010dc10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0224009900 tx=0x7f022402a410 comp rx=0 tx=0).stop 2024-05-09T02:26:54.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.478+0000 7f0236470640 1 -- 172.21.15.19:0/3294825884 shutdown_connections 2024-05-09T02:26:54.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.478+0000 7f0236470640 1 --2- 172.21.15.19:0/3294825884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f023010e1e0 0x7f02301106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.478+0000 7f0236470640 1 --2- 172.21.15.19:0/3294825884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023010b7f0 0x7f023010dc10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.478+0000 7f0236470640 1 --2- 172.21.15.19:0/3294825884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230104d90 0x7f02301051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.478+0000 7f0236470640 1 -- 172.21.15.19:0/3294825884 >> 172.21.15.19:0/3294825884 conn(0x7f02300fe390 msgr2=0x7f02301007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:54.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.478+0000 7f0236470640 1 -- 172.21.15.19:0/3294825884 shutdown_connections 2024-05-09T02:26:54.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.478+0000 7f0236470640 1 -- 172.21.15.19:0/3294825884 wait complete. 2024-05-09T02:26:54.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.478+0000 7f0236470640 1 Processor -- start 2024-05-09T02:26:54.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.479+0000 7f0236470640 1 -- start start 2024-05-09T02:26:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.479+0000 7f0236470640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0230104d90 0x7f02301a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.479+0000 7f0236470640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023010b7f0 0x7f02301a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.479+0000 7f0236470640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f023010e1e0 0x7f023019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.479+0000 7f0236470640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0230113e90 con 0x7f023010b7f0 2024-05-09T02:26:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.479+0000 7f0236470640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0230113d10 con 0x7f0230104d90 2024-05-09T02:26:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.479+0000 7f0236470640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0230114010 con 0x7f023010e1e0 2024-05-09T02:26:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.479+0000 7f022f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023010b7f0 0x7f02301a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.479+0000 7f022f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023010b7f0 0x7f02301a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47356/0 (socket says 172.21.15.19:47356) 2024-05-09T02:26:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.479+0000 7f022f7fe640 1 -- 172.21.15.19:0/311695503 learned_addr learned my addr 172.21.15.19:0/311695503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.479+0000 7f022ffff640 1 --2- 172.21.15.19:0/311695503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0230104d90 0x7f02301a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.479+0000 7f0234a0d640 1 --2- 172.21.15.19:0/311695503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f023010e1e0 0x7f023019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.480+0000 7f022f7fe640 1 -- 172.21.15.19:0/311695503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f023010e1e0 msgr2=0x7f023019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.480+0000 7f022f7fe640 1 --2- 172.21.15.19:0/311695503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f023010e1e0 0x7f023019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.480+0000 7f022f7fe640 1 -- 172.21.15.19:0/311695503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0230104d90 msgr2=0x7f02301a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:54.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.480+0000 7f022f7fe640 1 --2- 172.21.15.19:0/311695503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0230104d90 0x7f02301a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.480+0000 7f022f7fe640 1 -- 172.21.15.19:0/311695503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023019ae20 con 0x7f023010b7f0 2024-05-09T02:26:54.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.480+0000 7f022f7fe640 1 --2- 172.21.15.19:0/311695503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023010b7f0 0x7f02301a0ab0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0224005ab0 tx=0x7f0224003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:54.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.480+0000 7f022d7fa640 1 -- 172.21.15.19:0/311695503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022403f070 con 0x7f023010b7f0 2024-05-09T02:26:54.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.480+0000 7f022d7fa640 1 -- 172.21.15.19:0/311695503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0224031d60 con 0x7f023010b7f0 2024-05-09T02:26:54.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.480+0000 7f0236470640 1 -- 172.21.15.19:0/311695503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023019b050 con 0x7f023010b7f0 2024-05-09T02:26:54.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.480+0000 7f022d7fa640 1 -- 172.21.15.19:0/311695503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0224035030 con 0x7f023010b7f0 2024-05-09T02:26:54.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.480+0000 7f0236470640 1 -- 172.21.15.19:0/311695503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f023019b540 con 0x7f023010b7f0 2024-05-09T02:26:54.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.481+0000 7f0236470640 1 -- 172.21.15.19:0/311695503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02300757e0 con 0x7f023010b7f0 2024-05-09T02:26:54.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.484+0000 7f022d7fa640 1 -- 172.21.15.19:0/311695503 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0224042070 con 0x7f023010b7f0 2024-05-09T02:26:54.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.485+0000 7f022d7fa640 1 --2- 172.21.15.19:0/311695503 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f02180b2950 0x7f02180b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:54.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.485+0000 7f022ffff640 1 --2- 172.21.15.19:0/311695503 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f02180b2950 0x7f02180b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:54.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.485+0000 7f022d7fa640 1 -- 172.21.15.19:0/311695503 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f0224103470 con 0x7f023010b7f0 2024-05-09T02:26:54.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.485+0000 7f022ffff640 1 --2- 172.21.15.19:0/311695503 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f02180b2950 0x7f02180b4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0220005fd0 tx=0x7f022000b390 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:54.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.489+0000 7f022d7fa640 1 -- 172.21.15.19:0/311695503 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02240ca770 con 0x7f023010b7f0 2024-05-09T02:26:54.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:54 smithi019 ceph-mon[30151]: pgmap v489: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 16 MiB/s wr, 1.56k op/s 2024-05-09T02:26:54.610 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:54 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:26:54.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.723+0000 7fc9fefd9700 1 -- 172.21.15.19:0/3837180988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f810e900 msgr2=0x7fc9f8110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:54.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.723+0000 7fc9fefd9700 1 --2- 172.21.15.19:0/3837180988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f810e900 0x7fc9f8110dc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000b0a0 tx=0x7fc9f000b3b0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.723+0000 7fc9fefd9700 1 -- 172.21.15.19:0/3837180988 shutdown_connections 2024-05-09T02:26:54.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.723+0000 7fc9fefd9700 1 --2- 172.21.15.19:0/3837180988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f810e900 0x7fc9f8110dc0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.723+0000 7fc9fefd9700 1 --2- 172.21.15.19:0/3837180988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9f8105520 0x7fc9f8105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.723+0000 7fc9fefd9700 1 --2- 172.21.15.19:0/3837180988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f8105e70 0x7fc9f810e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.723+0000 7fc9fefd9700 1 -- 172.21.15.19:0/3837180988 >> 172.21.15.19:0/3837180988 conn(0x7fc9f80fedc0 msgr2=0x7fc9f81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:54.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.723+0000 7fc9fefd9700 1 -- 172.21.15.19:0/3837180988 shutdown_connections 2024-05-09T02:26:54.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.723+0000 7fc9fefd9700 1 -- 172.21.15.19:0/3837180988 wait complete. 2024-05-09T02:26:54.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.724+0000 7fc9fefd9700 1 Processor -- start 2024-05-09T02:26:54.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fefd9700 1 -- start start 2024-05-09T02:26:54.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fefd9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f8105520 0x7fc9f819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:54.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fefd9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f8105e70 0x7fc9f819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:54.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fefd9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9f810e900 0x7fc9f819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:54.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fefd9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f8114440 con 0x7fc9f8105520 2024-05-09T02:26:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fefd9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f81142c0 con 0x7fc9f810e900 2024-05-09T02:26:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fefd9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f81145c0 con 0x7fc9f8105e70 2024-05-09T02:26:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fcd75700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f8105520 0x7fc9f819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fcd75700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f8105520 0x7fc9f819ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47378/0 (socket says 172.21.15.19:47378) 2024-05-09T02:26:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fcd75700 1 -- 172.21.15.19:0/1498701770 learned_addr learned my addr 172.21.15.19:0/1498701770 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9f7fff700 1 --2- 172.21.15.19:0/1498701770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f8105e70 0x7fc9f819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fd576700 1 --2- 172.21.15.19:0/1498701770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9f810e900 0x7fc9f819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fcd75700 1 -- 172.21.15.19:0/1498701770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f8105e70 msgr2=0x7fc9f819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fcd75700 1 --2- 172.21.15.19:0/1498701770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f8105e70 0x7fc9f819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fcd75700 1 -- 172.21.15.19:0/1498701770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9f810e900 msgr2=0x7fc9f819f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fcd75700 1 --2- 172.21.15.19:0/1498701770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9f810e900 0x7fc9f819f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.725+0000 7fc9fcd75700 1 -- 172.21.15.19:0/1498701770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f819fc00 con 0x7fc9f8105520 2024-05-09T02:26:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.726+0000 7fc9fcd75700 1 --2- 172.21.15.19:0/1498701770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f8105520 0x7fc9f819ad30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec00e9a0 tx=0x7fc9ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.726+0000 7fc9f5ffb700 1 -- 172.21.15.19:0/1498701770 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec00ca10 con 0x7fc9f8105520 2024-05-09T02:26:54.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.726+0000 7fc9f5ffb700 1 -- 172.21.15.19:0/1498701770 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9ec00cbb0 con 0x7fc9f8105520 2024-05-09T02:26:54.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.726+0000 7fc9f5ffb700 1 -- 172.21.15.19:0/1498701770 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec018680 con 0x7fc9f8105520 2024-05-09T02:26:54.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.726+0000 7fc9fefd9700 1 -- 172.21.15.19:0/1498701770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f819fef0 con 0x7fc9f8105520 2024-05-09T02:26:54.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.726+0000 7fc9fefd9700 1 -- 172.21.15.19:0/1498701770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f81a7810 con 0x7fc9f8105520 2024-05-09T02:26:54.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.727+0000 7fc9fefd9700 1 -- 172.21.15.19:0/1498701770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f80512c0 con 0x7fc9f8105520 2024-05-09T02:26:54.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.729+0000 7fc9f5ffb700 1 -- 172.21.15.19:0/1498701770 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc9ec010430 con 0x7fc9f8105520 2024-05-09T02:26:54.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.730+0000 7fc9f5ffb700 1 --2- 172.21.15.19:0/1498701770 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc9e80b29f0 0x7fc9e80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:54.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.730+0000 7fc9f7fff700 1 --2- 172.21.15.19:0/1498701770 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc9e80b29f0 0x7fc9e80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:54.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.730+0000 7fc9f5ffb700 1 -- 172.21.15.19:0/1498701770 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fc9ec0e2800 con 0x7fc9f8105520 2024-05-09T02:26:54.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.730+0000 7fc9f7fff700 1 --2- 172.21.15.19:0/1498701770 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc9e80b29f0 0x7fc9e80b4f00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8102200 tx=0x7fc9e4009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:54.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.735+0000 7fc9f5ffb700 1 -- 172.21.15.19:0/1498701770 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc9ec0aa150 con 0x7fc9f8105520 2024-05-09T02:26:54.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.736+0000 7f0236470640 1 -- 172.21.15.19:0/311695503 --> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f023010edd0 con 0x7f02180b2950 2024-05-09T02:26:54.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.744+0000 7f022d7fa640 1 -- 172.21.15.19:0/311695503 <== mgr.15222 v2:172.21.15.19:6800/2638922161 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f023010edd0 con 0x7f02180b2950 2024-05-09T02:26:54.746 INFO:teuthology.orchestra.run.smithi019.stdout:/ 1m 6m3h 2024-05-09T02:26:54.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.746+0000 7f0236470640 1 -- 172.21.15.19:0/311695503 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f02180b2950 msgr2=0x7f02180b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:54.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.746+0000 7f0236470640 1 --2- 172.21.15.19:0/311695503 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f02180b2950 0x7f02180b4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0220005fd0 tx=0x7f022000b390 comp rx=0 tx=0).stop 2024-05-09T02:26:54.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.746+0000 7f0236470640 1 -- 172.21.15.19:0/311695503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023010b7f0 msgr2=0x7f02301a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:54.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.746+0000 7f0236470640 1 --2- 172.21.15.19:0/311695503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023010b7f0 0x7f02301a0ab0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0224005ab0 tx=0x7f0224003c80 comp rx=0 tx=0).stop 2024-05-09T02:26:54.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.746+0000 7f0236470640 1 -- 172.21.15.19:0/311695503 shutdown_connections 2024-05-09T02:26:54.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.746+0000 7f0236470640 1 --2- 172.21.15.19:0/311695503 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f02180b2950 0x7f02180b4e60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.746+0000 7f0236470640 1 --2- 172.21.15.19:0/311695503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f023010e1e0 0x7f023019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.746+0000 7f0236470640 1 --2- 172.21.15.19:0/311695503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f023010b7f0 0x7f02301a0ab0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.746+0000 7f0236470640 1 --2- 172.21.15.19:0/311695503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0230104d90 0x7f02301a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:54.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.746+0000 7f0236470640 1 -- 172.21.15.19:0/311695503 >> 172.21.15.19:0/311695503 conn(0x7f02300fe390 msgr2=0x7f023010e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:54.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.746+0000 7f0236470640 1 -- 172.21.15.19:0/311695503 shutdown_connections 2024-05-09T02:26:54.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:54.746+0000 7f0236470640 1 -- 172.21.15.19:0/311695503 wait complete. 2024-05-09T02:26:54.761 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-09T02:26:54.771 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-09T02:26:54.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:54 smithi059 ceph-mon[34781]: pgmap v489: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 16 MiB/s wr, 1.56k op/s 2024-05-09T02:26:54.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:54 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:26:55.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:55.038+0000 7fc9fefd9700 1 -- 172.21.15.19:0/1498701770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9f802e5f0 con 0x7fc9f8105520 2024-05-09T02:26:55.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:55.039+0000 7fc9f5ffb700 1 -- 172.21.15.19:0/1498701770 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11917 (secure 0 0 0) 0x7fc9ec0b21d0 con 0x7fc9f8105520 2024-05-09T02:26:55.042 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:26:55.042 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:22:01.074232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24457,24455,24473,24463,24484]},"id":1}]} 2024-05-09T02:26:55.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:54 smithi196 ceph-mon[34718]: pgmap v489: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 16 MiB/s wr, 1.56k op/s 2024-05-09T02:26:55.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:54 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:26:55.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:55.042+0000 7fc9fefd9700 1 -- 172.21.15.19:0/1498701770 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc9e80b29f0 msgr2=0x7fc9e80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:55.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:55.042+0000 7fc9fefd9700 1 --2- 172.21.15.19:0/1498701770 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc9e80b29f0 0x7fc9e80b4f00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8102200 tx=0x7fc9e4009260 comp rx=0 tx=0).stop 2024-05-09T02:26:55.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:55.042+0000 7fc9fefd9700 1 -- 172.21.15.19:0/1498701770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f8105520 msgr2=0x7fc9f819ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:55.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:55.042+0000 7fc9fefd9700 1 --2- 172.21.15.19:0/1498701770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f8105520 0x7fc9f819ad30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec00e9a0 tx=0x7fc9ec00ecb0 comp rx=0 tx=0).stop 2024-05-09T02:26:55.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:55.042+0000 7fc9fefd9700 1 -- 172.21.15.19:0/1498701770 shutdown_connections 2024-05-09T02:26:55.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:55.042+0000 7fc9fefd9700 1 --2- 172.21.15.19:0/1498701770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9f810e900 0x7fc9f819f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:55.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:55.042+0000 7fc9fefd9700 1 --2- 172.21.15.19:0/1498701770 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc9e80b29f0 0x7fc9e80b4f00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:55.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:55.042+0000 7fc9fefd9700 1 --2- 172.21.15.19:0/1498701770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9f8105520 0x7fc9f819ad30 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:55.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:55.042+0000 7fc9fefd9700 1 --2- 172.21.15.19:0/1498701770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f8105e70 0x7fc9f819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:55.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:55.042+0000 7fc9fefd9700 1 -- 172.21.15.19:0/1498701770 >> 172.21.15.19:0/1498701770 conn(0x7fc9f80fedc0 msgr2=0x7fc9f80ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:55.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:55.042+0000 7fc9fefd9700 1 -- 172.21.15.19:0/1498701770 shutdown_connections 2024-05-09T02:26:55.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:55.042+0000 7fc9fefd9700 1 -- 172.21.15.19:0/1498701770 wait complete. 2024-05-09T02:26:55.046 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-05-09T02:26:55.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:55 smithi019 ceph-mon[30151]: from='client.17169 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:26:55.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:55 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1498701770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:26:55.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:55 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:26:55.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:55 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:26:55.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:55 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:26:55.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:55 smithi059 ceph-mon[34781]: from='client.17169 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:26:55.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:55 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1498701770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:26:55.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:55 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:26:55.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:55 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:26:55.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:55 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:26:55.992 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T02:26:56.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:55 smithi196 ceph-mon[34718]: from='client.17169 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:26:56.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:55 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1498701770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:26:56.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:55 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:26:56.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:55 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:26:56.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:55 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:26:56.244 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:26:56.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:56 smithi019 ceph-mon[30151]: from='client.17175 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:26:56.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:56 smithi019 ceph-mon[30151]: pgmap v490: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 18 MiB/s wr, 1.72k op/s 2024-05-09T02:26:56.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:56 smithi059 ceph-mon[34781]: from='client.17175 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:26:56.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:56 smithi059 ceph-mon[34781]: pgmap v490: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 18 MiB/s wr, 1.72k op/s 2024-05-09T02:26:57.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:56 smithi196 ceph-mon[34718]: from='client.17175 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T02:26:57.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:56 smithi196 ceph-mon[34718]: pgmap v490: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 18 MiB/s wr, 1.72k op/s 2024-05-09T02:26:57.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.950+0000 7efbfa7a8700 1 -- 172.21.15.19:0/334103822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf4107df0 msgr2=0x7efbf4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:57.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.950+0000 7efbfa7a8700 1 --2- 172.21.15.19:0/334103822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf4107df0 0x7efbf4108200 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7efbdc0099c0 tx=0x7efbdc009cd0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.950+0000 7efbfa7a8700 1 -- 172.21.15.19:0/334103822 shutdown_connections 2024-05-09T02:26:57.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.950+0000 7efbfa7a8700 1 --2- 172.21.15.19:0/334103822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf4108740 0x7efbf41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.950+0000 7efbfa7a8700 1 --2- 172.21.15.19:0/334103822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efbf41034c0 0x7efbf41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.950+0000 7efbfa7a8700 1 --2- 172.21.15.19:0/334103822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf4107df0 0x7efbf4108200 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.950+0000 7efbfa7a8700 1 -- 172.21.15.19:0/334103822 >> 172.21.15.19:0/334103822 conn(0x7efbf40fedc0 msgr2=0x7efbf41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:57.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.950+0000 7efbfa7a8700 1 -- 172.21.15.19:0/334103822 shutdown_connections 2024-05-09T02:26:57.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.950+0000 7efbfa7a8700 1 -- 172.21.15.19:0/334103822 wait complete. 2024-05-09T02:26:57.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.951+0000 7efbfa7a8700 1 Processor -- start 2024-05-09T02:26:57.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.951+0000 7efbfa7a8700 1 -- start start 2024-05-09T02:26:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.951+0000 7efbfa7a8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf41034c0 0x7efbf4110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.951+0000 7efbfa7a8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf4107df0 0x7efbf4110fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.951+0000 7efbfa7a8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efbf4108740 0x7efbf4111500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.951+0000 7efbfa7a8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efbf41144c0 con 0x7efbf41034c0 2024-05-09T02:26:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.951+0000 7efbfa7a8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efbf4114340 con 0x7efbf4107df0 2024-05-09T02:26:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.951+0000 7efbfa7a8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efbf4114640 con 0x7efbf4108740 2024-05-09T02:26:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbf3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf41034c0 0x7efbf4110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbf3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf41034c0 0x7efbf4110a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52498/0 (socket says 172.21.15.19:52498) 2024-05-09T02:26:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbf3fff700 1 -- 172.21.15.19:0/1149483795 learned_addr learned my addr 172.21.15.19:0/1149483795 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbebfff700 1 --2- 172.21.15.19:0/1149483795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf4107df0 0x7efbf4110fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbf8d45700 1 --2- 172.21.15.19:0/1149483795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efbf4108740 0x7efbf4111500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbf3fff700 1 -- 172.21.15.19:0/1149483795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efbf4108740 msgr2=0x7efbf4111500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbf3fff700 1 --2- 172.21.15.19:0/1149483795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efbf4108740 0x7efbf4111500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbf3fff700 1 -- 172.21.15.19:0/1149483795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf4107df0 msgr2=0x7efbf4110fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbf3fff700 1 --2- 172.21.15.19:0/1149483795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf4107df0 0x7efbf4110fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbf3fff700 1 -- 172.21.15.19:0/1149483795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf41a91d0 con 0x7efbf41034c0 2024-05-09T02:26:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbf3fff700 1 --2- 172.21.15.19:0/1149483795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf41034c0 0x7efbf4110a80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efbe400d950 tx=0x7efbe400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbf1ffb700 1 -- 172.21.15.19:0/1149483795 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe4016070 con 0x7efbf41034c0 2024-05-09T02:26:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbfa7a8700 1 -- 172.21.15.19:0/1149483795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf41a94c0 con 0x7efbf41034c0 2024-05-09T02:26:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbf1ffb700 1 -- 172.21.15.19:0/1149483795 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbe4009df0 con 0x7efbf41034c0 2024-05-09T02:26:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbf1ffb700 1 -- 172.21.15.19:0/1149483795 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe400f3c0 con 0x7efbf41034c0 2024-05-09T02:26:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.952+0000 7efbfa7a8700 1 -- 172.21.15.19:0/1149483795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf41a99d0 con 0x7efbf41034c0 2024-05-09T02:26:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.955+0000 7efbf1ffb700 1 -- 172.21.15.19:0/1149483795 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efbe4019070 con 0x7efbf41034c0 2024-05-09T02:26:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.956+0000 7efbf1ffb700 1 --2- 172.21.15.19:0/1149483795 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efbe00b29f0 0x7efbe00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.956+0000 7efc091beb80 1 Processor -- start 2024-05-09T02:26:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.956+0000 7efc091beb80 1 -- start start 2024-05-09T02:26:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.956+0000 7efbebfff700 1 --2- 172.21.15.19:0/1149483795 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efbe00b29f0 0x7efbe00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.956+0000 7efbf1ffb700 1 -- 172.21.15.19:0/1149483795 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7efbe40e2270 con 0x7efbf41034c0 2024-05-09T02:26:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efc091beb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d85d6160 0x55f2d866f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efc091beb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d84ec4a0 0x55f2d866fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efc091beb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d86696a0 0x55f2d8669ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efc091beb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f2d858bff0 con 0x55f2d85d6160 2024-05-09T02:26:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efc091beb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f2d852a7f0 con 0x55f2d84ec4a0 2024-05-09T02:26:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efc091beb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f2d8587fb0 con 0x55f2d86696a0 2024-05-09T02:26:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efbf3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d85d6160 0x55f2d866f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efbf3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d85d6160 0x55f2d866f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52510/0 (socket says 172.21.15.19:52510) 2024-05-09T02:26:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efbf3fff700 1 -- 172.21.15.19:0/2630569296 learned_addr learned my addr 172.21.15.19:0/2630569296 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efbebfff700 1 --2- 172.21.15.19:0/1149483795 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efbe00b29f0 0x7efbe00b4f00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efbdc006010 tx=0x7efbdc000bc0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efbebfff700 1 --2- 172.21.15.19:0/2630569296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d84ec4a0 0x55f2d866fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efbf8d45700 1 --2- 172.21.15.19:0/2630569296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d86696a0 0x55f2d8669ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efbf3fff700 1 -- 172.21.15.19:0/2630569296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d86696a0 msgr2=0x55f2d8669ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efbf3fff700 1 --2- 172.21.15.19:0/2630569296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d86696a0 0x55f2d8669ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efbf3fff700 1 -- 172.21.15.19:0/2630569296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d84ec4a0 msgr2=0x55f2d866fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efbf3fff700 1 --2- 172.21.15.19:0/2630569296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d84ec4a0 0x55f2d866fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efbf3fff700 1 -- 172.21.15.19:0/2630569296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2d8522370 con 0x55f2d85d6160 2024-05-09T02:26:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efbebfff700 1 --2- 172.21.15.19:0/2630569296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d84ec4a0 0x55f2d866fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:26:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efbf8d45700 1 --2- 172.21.15.19:0/2630569296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d86696a0 0x55f2d8669ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:26:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.957+0000 7efbf3fff700 1 --2- 172.21.15.19:0/2630569296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d85d6160 0x55f2d866f4f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efbe40a9220 tx=0x7efbe40a9cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5792c107695f59b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.958+0000 7efbea7fc700 1 -- 172.21.15.19:0/2630569296 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe4020070 con 0x55f2d85d6160 2024-05-09T02:26:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.958+0000 7efbea7fc700 1 -- 172.21.15.19:0/2630569296 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbe40ad8d0 con 0x55f2d85d6160 2024-05-09T02:26:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.958+0000 7efbea7fc700 1 -- 172.21.15.19:0/2630569296 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe40b1ef0 con 0x55f2d85d6160 2024-05-09T02:26:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.958+0000 7efc091beb80 1 -- 172.21.15.19:0/2630569296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d85d6160 msgr2=0x55f2d866f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.958+0000 7efc091beb80 1 --2- 172.21.15.19:0/2630569296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d85d6160 0x55f2d866f4f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efbe40a9220 tx=0x7efbe40a9cf0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.958+0000 7efc091beb80 1 -- 172.21.15.19:0/2630569296 shutdown_connections 2024-05-09T02:26:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.958+0000 7efc091beb80 1 --2- 172.21.15.19:0/2630569296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d86696a0 0x55f2d8669ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.958+0000 7efc091beb80 1 --2- 172.21.15.19:0/2630569296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d85d6160 0x55f2d866f4f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.959+0000 7efc091beb80 1 --2- 172.21.15.19:0/2630569296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d84ec4a0 0x55f2d866fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.959+0000 7efc091beb80 1 -- 172.21.15.19:0/2630569296 >> 172.21.15.19:0/2630569296 conn(0x55f2d854c6a0 msgr2=0x55f2d8664400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.959+0000 7efc091beb80 1 -- 172.21.15.19:0/2630569296 shutdown_connections 2024-05-09T02:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.959+0000 7efc091beb80 1 -- 172.21.15.19:0/2630569296 wait complete. 2024-05-09T02:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.959+0000 7efc091beb80 1 -- start start 2024-05-09T02:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.959+0000 7efc091beb80 1 Processor -- start 2024-05-09T02:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efc091beb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d84ec4a0 0x55f2d8673290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efc091beb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d85d6160 0x55f2d86737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efc091beb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d86696a0 0x55f2d8673d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efc091beb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f2d8407ba0 con 0x55f2d86696a0 2024-05-09T02:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efc091beb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f2d8546150 con 0x55f2d85d6160 2024-05-09T02:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efc091beb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f2d86761d0 con 0x55f2d84ec4a0 2024-05-09T02:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efc091beb80 10 client.0 _ll_register_callbacks cb 0x55f2d83fd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efbf8d45700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d86696a0 0x55f2d8673d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efbf3fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d84ec4a0 0x55f2d8673290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efbf8d45700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d86696a0 0x55f2d8673d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52512/0 (socket says 172.21.15.19:52512) 2024-05-09T02:26:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efbf8d45700 1 -- 172.21.15.19:0/4137792544 learned_addr learned my addr 172.21.15.19:0/4137792544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efbebfff700 1 --2- 172.21.15.19:0/4137792544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d85d6160 0x55f2d86737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efbf3fff700 1 -- 172.21.15.19:0/4137792544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d85d6160 msgr2=0x55f2d86737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efbf3fff700 1 --2- 172.21.15.19:0/4137792544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d85d6160 0x55f2d86737d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efbf3fff700 1 -- 172.21.15.19:0/4137792544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d86696a0 msgr2=0x55f2d8673d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efbf3fff700 1 --2- 172.21.15.19:0/4137792544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d86696a0 0x55f2d8673d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efbf3fff700 1 -- 172.21.15.19:0/4137792544 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2d8407490 con 0x55f2d84ec4a0 2024-05-09T02:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efbf8d45700 1 --2- 172.21.15.19:0/4137792544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d86696a0 0x55f2d8673d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efbf3fff700 1 --2- 172.21.15.19:0/4137792544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d84ec4a0 0x55f2d8673290 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efbe40a9220 tx=0x7efbe4010ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=246b1e83d794c52a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.960+0000 7efbd37fe700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-09T02:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.961+0000 7efbd37fe700 1 -- 172.21.15.19:0/4137792544 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe40c2070 con 0x55f2d84ec4a0 2024-05-09T02:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.961+0000 7efbd37fe700 1 -- 172.21.15.19:0/4137792544 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7efbe40b1ef0 con 0x55f2d84ec4a0 2024-05-09T02:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.961+0000 7efc091beb80 1 -- 172.21.15.19:0/4137792544 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbe40ad8d0 con 0x55f2d84ec4a0 2024-05-09T02:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.961+0000 7efbd37fe700 1 -- 172.21.15.19:0/4137792544 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efbe4020040 con 0x55f2d84ec4a0 2024-05-09T02:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.961+0000 7efc091beb80 10 client.26995 fetch_fsmap learned FSMap version 58 2024-05-09T02:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.961+0000 7efc091beb80 1 -- 172.21.15.19:0/4137792544 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55f2d8680b40 con 0x55f2d84ec4a0 2024-05-09T02:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.961+0000 7efbd37fe700 1 -- 172.21.15.19:0/4137792544 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbe40a9990 con 0x55f2d84ec4a0 2024-05-09T02:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.961+0000 7efbd37fe700 1 -- 172.21.15.19:0/4137792544 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7efbe40ca350 con 0x55f2d84ec4a0 2024-05-09T02:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.962+0000 7efc091beb80 10 client.26995 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:26:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.962+0000 7efc091beb80 10 client.26995 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:26:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.962+0000 7efc091beb80 20 client.26995 populate_metadata read hostname 'smithi019' 2024-05-09T02:26:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.962+0000 7efc091beb80 1 --2- 172.21.15.19:0/4137792544 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55f2d8683010 0x55f2d8685430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.962+0000 7efc091beb80 4 client.26995 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:26:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.962+0000 7efc091beb80 1 -- 172.21.15.19:0/4137792544 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efbe40ad8d0 con 0x55f2d8683010 2024-05-09T02:26:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.962+0000 7efbebfff700 1 --2- 172.21.15.19:0/4137792544 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55f2d8683010 0x55f2d8685430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.963+0000 7efbebfff700 1 --2- 172.21.15.19:0/4137792544 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55f2d8683010 0x55f2d8685430 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6dd608544a113 server_cookie=f1471e6da3249dc7 in_seq=0 out_seq=0 2024-05-09T02:26:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.963+0000 7efbd37fe700 10 client.26995 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:26:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.964+0000 7efbd37fe700 1 -- 172.21.15.19:0/4137792544 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7efbe40ad8d0 con 0x55f2d8683010 2024-05-09T02:26:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.964+0000 7efbd37fe700 10 client.26995 handle_command_reply: tid=0 2024-05-09T02:26:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.964+0000 7efc091beb80 1 client.26995 shutdown 2024-05-09T02:26:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.964+0000 7efc091beb80 1 -- 172.21.15.19:0/4137792544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d84ec4a0 msgr2=0x55f2d8673290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.964+0000 7efc091beb80 1 --2- 172.21.15.19:0/4137792544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d84ec4a0 0x55f2d8673290 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efbe40a9220 tx=0x7efbe4010ae0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.964+0000 7efc091beb80 1 -- 172.21.15.19:0/4137792544 shutdown_connections 2024-05-09T02:26:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.964+0000 7efc091beb80 1 --2- 172.21.15.19:0/4137792544 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55f2d8683010 0x55f2d8685430 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.964+0000 7efc091beb80 1 --2- 172.21.15.19:0/4137792544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d86696a0 0x55f2d8673d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.964+0000 7efc091beb80 1 --2- 172.21.15.19:0/4137792544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d84ec4a0 0x55f2d8673290 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.964+0000 7efc091beb80 1 --2- 172.21.15.19:0/4137792544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d85d6160 0x55f2d86737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.964+0000 7efbd37fe700 0 client.26995 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:26:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.964+0000 7efc091beb80 1 -- 172.21.15.19:0/4137792544 >> 172.21.15.19:0/4137792544 conn(0x55f2d854c6a0 msgr2=0x55f2d86641c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.964+0000 7efc091beb80 1 -- 172.21.15.19:0/4137792544 shutdown_connections 2024-05-09T02:26:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.964+0000 7efc091beb80 1 -- 172.21.15.19:0/4137792544 wait complete. 2024-05-09T02:26:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.964+0000 7efc091beb80 20 client.26995 trim_cache size 0 max 16384 2024-05-09T02:26:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efc091beb80 1 Processor -- start 2024-05-09T02:26:57.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efc091beb80 1 -- start start 2024-05-09T02:26:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efc091beb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d84ec4a0 0x55f2d866e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efc091beb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d85d6160 0x55f2d8673250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efc091beb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d86696a0 0x55f2d8673790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efc091beb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efbf41b1240 con 0x55f2d86696a0 2024-05-09T02:26:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efc091beb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efbf41b2d80 con 0x55f2d85d6160 2024-05-09T02:26:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efc091beb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efbf41b1910 con 0x55f2d84ec4a0 2024-05-09T02:26:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efbf8d45700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d86696a0 0x55f2d8673790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efbf8d45700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d86696a0 0x55f2d8673790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52520/0 (socket says 172.21.15.19:52520) 2024-05-09T02:26:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efbf8d45700 1 -- 172.21.15.19:0/2830517326 learned_addr learned my addr 172.21.15.19:0/2830517326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efbebfff700 1 --2- 172.21.15.19:0/2830517326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d85d6160 0x55f2d8673250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efbf3fff700 1 --2- 172.21.15.19:0/2830517326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d84ec4a0 0x55f2d866e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efbf8d45700 1 -- 172.21.15.19:0/2830517326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d84ec4a0 msgr2=0x55f2d866e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efbf8d45700 1 --2- 172.21.15.19:0/2830517326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d84ec4a0 0x55f2d866e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efbf8d45700 1 -- 172.21.15.19:0/2830517326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d85d6160 msgr2=0x55f2d8673250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efbf8d45700 1 --2- 172.21.15.19:0/2830517326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d85d6160 0x55f2d8673250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efbf8d45700 1 -- 172.21.15.19:0/2830517326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2d8665ab0 con 0x55f2d86696a0 2024-05-09T02:26:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.980+0000 7efbebfff700 1 --2- 172.21.15.19:0/2830517326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d85d6160 0x55f2d8673250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:26:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.981+0000 7efbf8d45700 1 --2- 172.21.15.19:0/2830517326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d86696a0 0x55f2d8673790 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7efbec009790 tx=0x7efbec009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ab0747d62d5958 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.981+0000 7efbe9ffb700 1 -- 172.21.15.19:0/2830517326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec01b020 con 0x55f2d86696a0 2024-05-09T02:26:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.981+0000 7efbe9ffb700 1 -- 172.21.15.19:0/2830517326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbec007e20 con 0x55f2d86696a0 2024-05-09T02:26:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.981+0000 7efbe9ffb700 1 -- 172.21.15.19:0/2830517326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec016a80 con 0x55f2d86696a0 2024-05-09T02:26:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.982+0000 7efc091beb80 1 -- 172.21.15.19:0/2830517326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d86696a0 msgr2=0x55f2d8673790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.982+0000 7efc091beb80 1 --2- 172.21.15.19:0/2830517326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d86696a0 0x55f2d8673790 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7efbec009790 tx=0x7efbec009040 comp rx=0 tx=0).stop 2024-05-09T02:26:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.982+0000 7efc091beb80 1 -- 172.21.15.19:0/2830517326 shutdown_connections 2024-05-09T02:26:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.982+0000 7efc091beb80 1 --2- 172.21.15.19:0/2830517326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d86696a0 0x55f2d8673790 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.982+0000 7efc091beb80 1 --2- 172.21.15.19:0/2830517326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d84ec4a0 0x55f2d866e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.982+0000 7efc091beb80 1 --2- 172.21.15.19:0/2830517326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d85d6160 0x55f2d8673250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.982+0000 7efc091beb80 1 -- 172.21.15.19:0/2830517326 >> 172.21.15.19:0/2830517326 conn(0x55f2d854c6a0 msgr2=0x55f2d866e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.982+0000 7efc091beb80 1 -- 172.21.15.19:0/2830517326 shutdown_connections 2024-05-09T02:26:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.982+0000 7efc091beb80 1 -- 172.21.15.19:0/2830517326 wait complete. 2024-05-09T02:26:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.982+0000 7efc091beb80 1 -- start start 2024-05-09T02:26:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.982+0000 7efc091beb80 1 Processor -- start 2024-05-09T02:26:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efc091beb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d84ec4a0 0x55f2d8670000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efc091beb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d85d6160 0x55f2d8670540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efc091beb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d86696a0 0x55f2d8670a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efc091beb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efbf41b26b0 con 0x55f2d85d6160 2024-05-09T02:26:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efc091beb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efbf41b1fe0 con 0x55f2d84ec4a0 2024-05-09T02:26:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efc091beb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efbf41af200 con 0x55f2d86696a0 2024-05-09T02:26:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efc091beb80 10 client.0 _ll_register_callbacks cb 0x55f2d83fd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:26:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efbebfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d85d6160 0x55f2d8670540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efbebfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d85d6160 0x55f2d8670540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52528/0 (socket says 172.21.15.19:52528) 2024-05-09T02:26:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efbebfff700 1 -- 172.21.15.19:0/1494701421 learned_addr learned my addr 172.21.15.19:0/1494701421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:26:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efbf3fff700 1 --2- 172.21.15.19:0/1494701421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d84ec4a0 0x55f2d8670000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efbf8d45700 1 --2- 172.21.15.19:0/1494701421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d86696a0 0x55f2d8670a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efbebfff700 1 -- 172.21.15.19:0/1494701421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d86696a0 msgr2=0x55f2d8670a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efbebfff700 1 --2- 172.21.15.19:0/1494701421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d86696a0 0x55f2d8670a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efbebfff700 1 -- 172.21.15.19:0/1494701421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d84ec4a0 msgr2=0x55f2d8670000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:26:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efbebfff700 1 --2- 172.21.15.19:0/1494701421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d84ec4a0 0x55f2d8670000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efbebfff700 1 -- 172.21.15.19:0/1494701421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2d86eb760 con 0x55f2d85d6160 2024-05-09T02:26:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efbebfff700 1 --2- 172.21.15.19:0/1494701421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d85d6160 0x55f2d8670540 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efbf4076ab0 tx=0x7efbdc009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e261b83cf189f155 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:26:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.983+0000 7efbd37fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:26:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.984+0000 7efbd37fe700 1 -- 172.21.15.19:0/1494701421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbdc036060 con 0x55f2d85d6160 2024-05-09T02:26:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.984+0000 7efc091beb80 1 -- 172.21.15.19:0/1494701421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbec007e20 con 0x55f2d85d6160 2024-05-09T02:26:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.984+0000 7efbd37fe700 1 -- 172.21.15.19:0/1494701421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7efbdc00b990 con 0x55f2d85d6160 2024-05-09T02:26:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.984+0000 7efbd37fe700 1 -- 172.21.15.19:0/1494701421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efbdc00bb90 con 0x55f2d85d6160 2024-05-09T02:26:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.984+0000 7efc091beb80 10 client.17199 fetch_fsmap learned FSMap version 58 2024-05-09T02:26:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.984+0000 7efc091beb80 1 -- 172.21.15.19:0/1494701421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55f2d86eca40 con 0x55f2d85d6160 2024-05-09T02:26:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.984+0000 7efbd37fe700 1 -- 172.21.15.19:0/1494701421 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbec007e20 con 0x55f2d85d6160 2024-05-09T02:26:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.984+0000 7efbd37fe700 1 -- 172.21.15.19:0/1494701421 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8354+0+0 (secure 0 0 0) 0x7efbdc022070 con 0x55f2d85d6160 2024-05-09T02:26:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.984+0000 7efc091beb80 10 client.17199 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T02:26:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.984+0000 7efc091beb80 10 client.17199 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T02:26:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.984+0000 7efc091beb80 20 client.17199 populate_metadata read hostname 'smithi019' 2024-05-09T02:26:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.984+0000 7efc091beb80 1 --2- 172.21.15.19:0/1494701421 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55f2d86ed130 0x55f2d86ef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:26:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.984+0000 7efc091beb80 4 client.17199 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:26:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.984+0000 7efc091beb80 1 -- 172.21.15.19:0/1494701421 --> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2d85999d0 con 0x55f2d86ed130 2024-05-09T02:26:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.984+0000 7efbf3fff700 1 --2- 172.21.15.19:0/1494701421 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55f2d86ed130 0x55f2d86ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:26:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.985+0000 7efbd37fe700 10 client.17199 ms_handle_connect on v2:172.21.15.59:6836/2268419175 2024-05-09T02:26:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.985+0000 7efbf3fff700 1 --2- 172.21.15.19:0/1494701421 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55f2d86ed130 0x55f2d86ef550 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b389fd82029f92ee server_cookie=2842a3cfbd173513 in_seq=0 out_seq=0 2024-05-09T02:26:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.985+0000 7efbd37fe700 1 -- 172.21.15.19:0/1494701421 <== mds.4 v2:172.21.15.59:6836/2268419175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2d85999d0 con 0x55f2d86ed130 2024-05-09T02:26:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.985+0000 7efbd37fe700 10 client.17199 handle_command_reply: tid=0 2024-05-09T02:26:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.985+0000 7efc091beb80 1 client.17199 shutdown 2024-05-09T02:26:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.985+0000 7efc091beb80 1 -- 172.21.15.19:0/1494701421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d85d6160 msgr2=0x55f2d8670540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.985+0000 7efc091beb80 1 --2- 172.21.15.19:0/1494701421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d85d6160 0x55f2d8670540 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efbf4076ab0 tx=0x7efbdc009cd0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.985+0000 7efc091beb80 1 -- 172.21.15.19:0/1494701421 shutdown_connections 2024-05-09T02:26:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.985+0000 7efc091beb80 1 --2- 172.21.15.19:0/1494701421 >> [v2:172.21.15.59:6836/2268419175,v1:172.21.15.59:6837/2268419175] conn(0x55f2d86ed130 0x55f2d86ef550 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.985+0000 7efc091beb80 1 --2- 172.21.15.19:0/1494701421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d86696a0 0x55f2d8670a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.985+0000 7efc091beb80 1 --2- 172.21.15.19:0/1494701421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2d84ec4a0 0x55f2d8670000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.985+0000 7efc091beb80 1 --2- 172.21.15.19:0/1494701421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2d85d6160 0x55f2d8670540 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.985+0000 7efbd37fe700 0 client.17199 ms_handle_reset on v2:172.21.15.59:6836/2268419175 2024-05-09T02:26:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.985+0000 7efc091beb80 1 -- 172.21.15.19:0/1494701421 >> 172.21.15.19:0/1494701421 conn(0x55f2d854c6a0 msgr2=0x55f2d8671a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.986+0000 7efc091beb80 1 -- 172.21.15.19:0/1494701421 shutdown_connections 2024-05-09T02:26:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.986+0000 7efc091beb80 1 -- 172.21.15.19:0/1494701421 wait complete. 2024-05-09T02:26:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.986+0000 7efc091beb80 20 client.17199 trim_cache size 0 max 16384 2024-05-09T02:26:57.994 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:26:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.986+0000 7efbfa7a8700 1 -- 172.21.15.19:0/1149483795 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efbe00b29f0 msgr2=0x7efbe00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.986+0000 7efbfa7a8700 1 --2- 172.21.15.19:0/1149483795 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efbe00b29f0 0x7efbe00b4f00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efbdc006010 tx=0x7efbdc000bc0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.986+0000 7efbfa7a8700 1 -- 172.21.15.19:0/1149483795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf41034c0 msgr2=0x7efbf4110a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:26:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.986+0000 7efbfa7a8700 1 --2- 172.21.15.19:0/1149483795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf41034c0 0x7efbf4110a80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efbe400d950 tx=0x7efbe400dc60 comp rx=0 tx=0).stop 2024-05-09T02:26:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.986+0000 7efbfa7a8700 1 -- 172.21.15.19:0/1149483795 shutdown_connections 2024-05-09T02:26:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.986+0000 7efbfa7a8700 1 --2- 172.21.15.19:0/1149483795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efbf4108740 0x7efbf4111500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.986+0000 7efbfa7a8700 1 --2- 172.21.15.19:0/1149483795 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7efbe00b29f0 0x7efbe00b4f00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.986+0000 7efbfa7a8700 1 --2- 172.21.15.19:0/1149483795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf41034c0 0x7efbf4110a80 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.986+0000 7efbfa7a8700 1 --2- 172.21.15.19:0/1149483795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf4107df0 0x7efbf4110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:26:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.986+0000 7efbfa7a8700 1 -- 172.21.15.19:0/1149483795 >> 172.21.15.19:0/1149483795 conn(0x7efbf40fedc0 msgr2=0x7efbf4101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:26:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.986+0000 7efbfa7a8700 1 -- 172.21.15.19:0/1149483795 shutdown_connections 2024-05-09T02:26:57.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:26:57.986+0000 7efbfa7a8700 1 -- 172.21.15.19:0/1149483795 wait complete. 2024-05-09T02:26:58.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:58 smithi019 ceph-mon[30151]: pgmap v491: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 18 MiB/s wr, 1.68k op/s 2024-05-09T02:26:58.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T02:26:58.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:58 smithi059 ceph-mon[34781]: pgmap v491: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 18 MiB/s wr, 1.68k op/s 2024-05-09T02:26:58.967 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:26:59.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:58 smithi196 ceph-mon[34718]: pgmap v491: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 18 MiB/s wr, 1.68k op/s 2024-05-09T02:26:59.789 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:26:59 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:26:59.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:26:59 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:27:00.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:26:59 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:27:00.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.663+0000 7fe93a3f3700 1 -- 172.21.15.19:0/482076855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe934107df0 msgr2=0x7fe934108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:00.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.663+0000 7fe93a3f3700 1 --2- 172.21.15.19:0/482076855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe934107df0 0x7fe934108200 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe9280099c0 tx=0x7fe928009cd0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:00 smithi019 ceph-mon[30151]: pgmap v492: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 17 MiB/s wr, 1.57k op/s 2024-05-09T02:27:00.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.663+0000 7fe93a3f3700 1 -- 172.21.15.19:0/482076855 shutdown_connections 2024-05-09T02:27:00.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.663+0000 7fe93a3f3700 1 --2- 172.21.15.19:0/482076855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe934108740 0x7fe9341100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.663+0000 7fe93a3f3700 1 --2- 172.21.15.19:0/482076855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9341034c0 0x7fe9341038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.663+0000 7fe93a3f3700 1 --2- 172.21.15.19:0/482076855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe934107df0 0x7fe934108200 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.663+0000 7fe93a3f3700 1 -- 172.21.15.19:0/482076855 >> 172.21.15.19:0/482076855 conn(0x7fe9340fedf0 msgr2=0x7fe934101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.664+0000 7fe93a3f3700 1 -- 172.21.15.19:0/482076855 shutdown_connections 2024-05-09T02:27:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.664+0000 7fe93a3f3700 1 -- 172.21.15.19:0/482076855 wait complete. 2024-05-09T02:27:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.664+0000 7fe93a3f3700 1 Processor -- start 2024-05-09T02:27:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.664+0000 7fe93a3f3700 1 -- start start 2024-05-09T02:27:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.665+0000 7fe93a3f3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9341034c0 0x7fe93419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.665+0000 7fe93a3f3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe934107df0 0x7fe93419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.665+0000 7fe93a3f3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe934108740 0x7fe93419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.665+0000 7fe93a3f3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341144c0 con 0x7fe934107df0 2024-05-09T02:27:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.665+0000 7fe93a3f3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe934114340 con 0x7fe9341034c0 2024-05-09T02:27:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.665+0000 7fe93a3f3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe934114640 con 0x7fe934108740 2024-05-09T02:27:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.665+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe934107df0 0x7fe93419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.665+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe934107df0 0x7fe93419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52548/0 (socket says 172.21.15.19:52548) 2024-05-09T02:27:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.665+0000 7fe933fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9341034c0 0x7fe93419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.665+0000 7fe9337fe700 1 -- 172.21.15.19:0/3793677626 learned_addr learned my addr 172.21.15.19:0/3793677626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.665+0000 7fe938990700 1 --2- 172.21.15.19:0/3793677626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe934108740 0x7fe93419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.666+0000 7fe9337fe700 1 -- 172.21.15.19:0/3793677626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe934108740 msgr2=0x7fe93419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.666+0000 7fe9337fe700 1 --2- 172.21.15.19:0/3793677626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe934108740 0x7fe93419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.666+0000 7fe9337fe700 1 -- 172.21.15.19:0/3793677626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9341034c0 msgr2=0x7fe93419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.666+0000 7fe9337fe700 1 --2- 172.21.15.19:0/3793677626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9341034c0 0x7fe93419ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.666+0000 7fe9337fe700 1 -- 172.21.15.19:0/3793677626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93419fc60 con 0x7fe934107df0 2024-05-09T02:27:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.666+0000 7fe938990700 1 --2- 172.21.15.19:0/3793677626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe934108740 0x7fe93419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:27:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.666+0000 7fe9337fe700 1 --2- 172.21.15.19:0/3793677626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe934107df0 0x7fe93419b430 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe9280098e0 tx=0x7fe9280057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.666+0000 7fe9317fa700 1 -- 172.21.15.19:0/3793677626 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928027070 con 0x7fe934107df0 2024-05-09T02:27:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.666+0000 7fe9317fa700 1 -- 172.21.15.19:0/3793677626 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe928004020 con 0x7fe934107df0 2024-05-09T02:27:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.666+0000 7fe93a3f3700 1 -- 172.21.15.19:0/3793677626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93419fef0 con 0x7fe934107df0 2024-05-09T02:27:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.666+0000 7fe9317fa700 1 -- 172.21.15.19:0/3793677626 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928022e30 con 0x7fe934107df0 2024-05-09T02:27:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.666+0000 7fe93a3f3700 1 -- 172.21.15.19:0/3793677626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9341a03d0 con 0x7fe934107df0 2024-05-09T02:27:00.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.669+0000 7fe9317fa700 1 -- 172.21.15.19:0/3793677626 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe92802a070 con 0x7fe934107df0 2024-05-09T02:27:00.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.670+0000 7fe9317fa700 1 --2- 172.21.15.19:0/3793677626 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe91c0b29f0 0x7fe91c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.670+0000 7fe933fff700 1 --2- 172.21.15.19:0/3793677626 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe91c0b29f0 0x7fe91c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.670+0000 7fe9317fa700 1 -- 172.21.15.19:0/3793677626 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fe9280e33b0 con 0x7fe934107df0 2024-05-09T02:27:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.670+0000 7fe933fff700 1 --2- 172.21.15.19:0/3793677626 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe91c0b29f0 0x7fe91c0b4f00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe924005950 tx=0x7fe92400a300 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.670+0000 7fe948e09b80 1 Processor -- start 2024-05-09T02:27:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.670+0000 7fe948e09b80 1 -- start start 2024-05-09T02:27:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe948e09b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52efdd00 0x558a53006240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe948e09b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52e834a0 0x558a53006780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe948e09b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a53000400 0x558a53000810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe948e09b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a52f230a0 con 0x558a53000400 2024-05-09T02:27:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe948e09b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558a52ec89a0 con 0x558a52efdd00 2024-05-09T02:27:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe948e09b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558a52f1f060 con 0x558a52e834a0 2024-05-09T02:27:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe938990700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a53000400 0x558a53000810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe938990700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a53000400 0x558a53000810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52550/0 (socket says 172.21.15.19:52550) 2024-05-09T02:27:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe938990700 1 -- 172.21.15.19:0/3120747558 learned_addr learned my addr 172.21.15.19:0/3120747558 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe933fff700 1 --2- 172.21.15.19:0/3120747558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52efdd00 0x558a53006240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe9337fe700 1 --2- 172.21.15.19:0/3120747558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52e834a0 0x558a53006780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe938990700 1 -- 172.21.15.19:0/3120747558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52e834a0 msgr2=0x558a53006780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe938990700 1 --2- 172.21.15.19:0/3120747558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52e834a0 0x558a53006780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe938990700 1 -- 172.21.15.19:0/3120747558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52efdd00 msgr2=0x558a53006240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe938990700 1 --2- 172.21.15.19:0/3120747558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52efdd00 0x558a53006240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe938990700 1 -- 172.21.15.19:0/3120747558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a52eb9500 con 0x558a53000400 2024-05-09T02:27:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe9337fe700 1 --2- 172.21.15.19:0/3120747558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52e834a0 0x558a53006780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:27:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.671+0000 7fe938990700 1 --2- 172.21.15.19:0/3120747558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a53000400 0x558a53000810 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00e160 tx=0x7fe92c00e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8689196b0f8516 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.672+0000 7fe91a7fc700 1 -- 172.21.15.19:0/3120747558 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00ed00 con 0x558a53000400 2024-05-09T02:27:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.672+0000 7fe91a7fc700 1 -- 172.21.15.19:0/3120747558 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92c004bb0 con 0x558a53000400 2024-05-09T02:27:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.672+0000 7fe91a7fc700 1 -- 172.21.15.19:0/3120747558 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c014df0 con 0x558a53000400 2024-05-09T02:27:00.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.673+0000 7fe948e09b80 1 -- 172.21.15.19:0/3120747558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a53000400 msgr2=0x558a53000810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:00.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.673+0000 7fe948e09b80 1 --2- 172.21.15.19:0/3120747558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a53000400 0x558a53000810 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00e160 tx=0x7fe92c00e470 comp rx=0 tx=0).stop 2024-05-09T02:27:00.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.673+0000 7fe948e09b80 1 -- 172.21.15.19:0/3120747558 shutdown_connections 2024-05-09T02:27:00.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.673+0000 7fe948e09b80 1 --2- 172.21.15.19:0/3120747558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a53000400 0x558a53000810 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.673+0000 7fe948e09b80 1 --2- 172.21.15.19:0/3120747558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52efdd00 0x558a53006240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.673+0000 7fe948e09b80 1 --2- 172.21.15.19:0/3120747558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52e834a0 0x558a53006780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.673+0000 7fe948e09b80 1 -- 172.21.15.19:0/3120747558 >> 172.21.15.19:0/3120747558 conn(0x558a52e86f70 msgr2=0x558a52ffb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.673+0000 7fe948e09b80 1 -- 172.21.15.19:0/3120747558 shutdown_connections 2024-05-09T02:27:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.673+0000 7fe948e09b80 1 -- 172.21.15.19:0/3120747558 wait complete. 2024-05-09T02:27:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.673+0000 7fe948e09b80 1 -- start start 2024-05-09T02:27:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.673+0000 7fe948e09b80 1 Processor -- start 2024-05-09T02:27:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe948e09b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52e834a0 0x558a5300a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe948e09b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52efdd00 0x558a5300a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe948e09b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a53000400 0x558a5300ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe948e09b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a53001160 con 0x558a53000400 2024-05-09T02:27:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe948e09b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558a52ebdb20 con 0x558a52e834a0 2024-05-09T02:27:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe948e09b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558a5300d020 con 0x558a52efdd00 2024-05-09T02:27:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe948e09b80 10 client.0 _ll_register_callbacks cb 0x558a52d946c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:27:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe938990700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a53000400 0x558a5300ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe938990700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a53000400 0x558a5300ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52564/0 (socket says 172.21.15.19:52564) 2024-05-09T02:27:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe938990700 1 -- 172.21.15.19:0/2162753486 learned_addr learned my addr 172.21.15.19:0/2162753486 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe933fff700 1 --2- 172.21.15.19:0/2162753486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52e834a0 0x558a5300a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe9337fe700 1 --2- 172.21.15.19:0/2162753486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52efdd00 0x558a5300a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe933fff700 1 -- 172.21.15.19:0/2162753486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52efdd00 msgr2=0x558a5300a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe933fff700 1 --2- 172.21.15.19:0/2162753486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52efdd00 0x558a5300a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe933fff700 1 -- 172.21.15.19:0/2162753486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a53000400 msgr2=0x558a5300ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe933fff700 1 --2- 172.21.15.19:0/2162753486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a53000400 0x558a5300ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe933fff700 1 -- 172.21.15.19:0/2162753486 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a52d9e490 con 0x558a52e834a0 2024-05-09T02:27:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.674+0000 7fe938990700 1 --2- 172.21.15.19:0/2162753486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a53000400 0x558a5300ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:27:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.675+0000 7fe933fff700 1 --2- 172.21.15.19:0/2162753486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52e834a0 0x558a5300a0f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe92400f750 tx=0x7fe9240047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee395bddb69aad05 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.675+0000 7fe9137fe700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-09T02:27:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.675+0000 7fe9137fe700 1 -- 172.21.15.19:0/2162753486 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924016070 con 0x558a52e834a0 2024-05-09T02:27:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.675+0000 7fe948e09b80 1 -- 172.21.15.19:0/2162753486 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe92c004bb0 con 0x558a52e834a0 2024-05-09T02:27:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.675+0000 7fe9137fe700 1 -- 172.21.15.19:0/2162753486 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fe924004a30 con 0x558a52e834a0 2024-05-09T02:27:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.675+0000 7fe9137fe700 1 -- 172.21.15.19:0/2162753486 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe924004c30 con 0x558a52e834a0 2024-05-09T02:27:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.675+0000 7fe948e09b80 10 client.26936 fetch_fsmap learned FSMap version 59 2024-05-09T02:27:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.675+0000 7fe948e09b80 1 -- 172.21.15.19:0/2162753486 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x558a53017b00 con 0x558a52e834a0 2024-05-09T02:27:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.675+0000 7fe9137fe700 1 -- 172.21.15.19:0/2162753486 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92c004bb0 con 0x558a52e834a0 2024-05-09T02:27:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.675+0000 7fe9137fe700 1 -- 172.21.15.19:0/2162753486 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 59) ==== 8358+0+0 (secure 0 0 0) 0x7fe924010a10 con 0x558a52e834a0 2024-05-09T02:27:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.676+0000 7fe948e09b80 10 client.26936 fetch_fsmap finished waiting for FSMap version 59 2024-05-09T02:27:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.676+0000 7fe948e09b80 10 client.26936 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:27:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.676+0000 7fe948e09b80 20 client.26936 populate_metadata read hostname 'smithi019' 2024-05-09T02:27:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.676+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2162753486 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x558a5301a090 0x558a5301c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.676+0000 7fe948e09b80 4 client.26936 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:27:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.676+0000 7fe948e09b80 1 -- 172.21.15.19:0/2162753486 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a52f30a80 con 0x558a5301a090 2024-05-09T02:27:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.676+0000 7fe9337fe700 1 --2- 172.21.15.19:0/2162753486 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x558a5301a090 0x558a5301c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.677+0000 7fe9337fe700 1 --2- 172.21.15.19:0/2162753486 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x558a5301a090 0x558a5301c530 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=da36d88da4aeb35f server_cookie=65befadca39509d9 in_seq=0 out_seq=0 2024-05-09T02:27:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.677+0000 7fe9137fe700 10 client.26936 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:27:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.677+0000 7fe9137fe700 1 -- 172.21.15.19:0/2162753486 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a52f30a80 con 0x558a5301a090 2024-05-09T02:27:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.677+0000 7fe9137fe700 10 client.26936 handle_command_reply: tid=0 2024-05-09T02:27:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.677+0000 7fe948e09b80 1 client.26936 shutdown 2024-05-09T02:27:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.677+0000 7fe948e09b80 1 -- 172.21.15.19:0/2162753486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52e834a0 msgr2=0x558a5300a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.678+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2162753486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52e834a0 0x558a5300a0f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe92400f750 tx=0x7fe9240047c0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.678+0000 7fe948e09b80 1 -- 172.21.15.19:0/2162753486 shutdown_connections 2024-05-09T02:27:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.678+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2162753486 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x558a5301a090 0x558a5301c530 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.678+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2162753486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a53000400 0x558a5300ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.678+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2162753486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52e834a0 0x558a5300a0f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.678+0000 7fe9137fe700 0 client.26936 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:27:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.678+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2162753486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52efdd00 0x558a5300a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.678+0000 7fe948e09b80 1 -- 172.21.15.19:0/2162753486 >> 172.21.15.19:0/2162753486 conn(0x558a52e86f70 msgr2=0x558a52ff4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.678+0000 7fe948e09b80 1 -- 172.21.15.19:0/2162753486 shutdown_connections 2024-05-09T02:27:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.678+0000 7fe948e09b80 1 -- 172.21.15.19:0/2162753486 wait complete. 2024-05-09T02:27:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.678+0000 7fe948e09b80 20 client.26936 trim_cache size 0 max 16384 2024-05-09T02:27:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe948e09b80 1 Processor -- start 2024-05-09T02:27:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe948e09b80 1 -- start start 2024-05-09T02:27:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe948e09b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a52e834a0 0x558a530058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe948e09b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52efdd00 0x558a5300c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe948e09b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a53000400 0x558a5300a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe948e09b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341aee50 con 0x558a52e834a0 2024-05-09T02:27:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe948e09b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341b09c0 con 0x558a53000400 2024-05-09T02:27:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe948e09b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341af550 con 0x558a52efdd00 2024-05-09T02:27:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe933fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a52e834a0 0x558a530058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe933fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a52e834a0 0x558a530058f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52580/0 (socket says 172.21.15.19:52580) 2024-05-09T02:27:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe933fff700 1 -- 172.21.15.19:0/2756040693 learned_addr learned my addr 172.21.15.19:0/2756040693 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe9337fe700 1 --2- 172.21.15.19:0/2756040693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52efdd00 0x558a5300c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe938990700 1 --2- 172.21.15.19:0/2756040693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a53000400 0x558a5300a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe933fff700 1 -- 172.21.15.19:0/2756040693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52efdd00 msgr2=0x558a5300c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:00.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe933fff700 1 --2- 172.21.15.19:0/2756040693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52efdd00 0x558a5300c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe933fff700 1 -- 172.21.15.19:0/2756040693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a53000400 msgr2=0x558a5300a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:00.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.693+0000 7fe933fff700 1 --2- 172.21.15.19:0/2756040693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a53000400 0x558a5300a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.694+0000 7fe933fff700 1 -- 172.21.15.19:0/2756040693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a53006c70 con 0x558a52e834a0 2024-05-09T02:27:00.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.694+0000 7fe9337fe700 1 --2- 172.21.15.19:0/2756040693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52efdd00 0x558a5300c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:27:00.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.694+0000 7fe933fff700 1 --2- 172.21.15.19:0/2756040693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a52e834a0 0x558a530058f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe92400faa0 tx=0x7fe92400f8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48a654b61d95ea5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:00.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.694+0000 7fe919ffb700 1 -- 172.21.15.19:0/2756040693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924033070 con 0x558a52e834a0 2024-05-09T02:27:00.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.694+0000 7fe919ffb700 1 -- 172.21.15.19:0/2756040693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9240277c0 con 0x558a52e834a0 2024-05-09T02:27:00.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.694+0000 7fe919ffb700 1 -- 172.21.15.19:0/2756040693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924005030 con 0x558a52e834a0 2024-05-09T02:27:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.695+0000 7fe948e09b80 1 -- 172.21.15.19:0/2756040693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a52e834a0 msgr2=0x558a530058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.695+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2756040693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a52e834a0 0x558a530058f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe92400faa0 tx=0x7fe92400f8e0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.695+0000 7fe948e09b80 1 -- 172.21.15.19:0/2756040693 shutdown_connections 2024-05-09T02:27:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.695+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2756040693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a53000400 0x558a5300a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.695+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2756040693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a52e834a0 0x558a530058f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.695+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2756040693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a52efdd00 0x558a5300c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.695+0000 7fe948e09b80 1 -- 172.21.15.19:0/2756040693 >> 172.21.15.19:0/2756040693 conn(0x558a52e86f70 msgr2=0x558a530051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.695+0000 7fe948e09b80 1 -- 172.21.15.19:0/2756040693 shutdown_connections 2024-05-09T02:27:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.695+0000 7fe948e09b80 1 -- 172.21.15.19:0/2756040693 wait complete. 2024-05-09T02:27:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.696+0000 7fe948e09b80 1 -- start start 2024-05-09T02:27:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.696+0000 7fe948e09b80 1 Processor -- start 2024-05-09T02:27:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.696+0000 7fe948e09b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a52e834a0 0x558a530121e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.696+0000 7fe948e09b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52efdd00 0x558a53012720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.696+0000 7fe948e09b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a53000400 0x558a53020ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.696+0000 7fe948e09b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341b02f0 con 0x558a52e834a0 2024-05-09T02:27:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.696+0000 7fe948e09b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341afc20 con 0x558a52efdd00 2024-05-09T02:27:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.696+0000 7fe948e09b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341ace10 con 0x558a53000400 2024-05-09T02:27:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.696+0000 7fe948e09b80 10 client.0 _ll_register_callbacks cb 0x558a53000dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:27:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.696+0000 7fe933fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a52e834a0 0x558a530121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.696+0000 7fe933fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a52e834a0 0x558a530121e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52594/0 (socket says 172.21.15.19:52594) 2024-05-09T02:27:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.696+0000 7fe933fff700 1 -- 172.21.15.19:0/2659471981 learned_addr learned my addr 172.21.15.19:0/2659471981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.697+0000 7fe9337fe700 1 --2- 172.21.15.19:0/2659471981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52efdd00 0x558a53012720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.697+0000 7fe938990700 1 --2- 172.21.15.19:0/2659471981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a53000400 0x558a53020ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.697+0000 7fe933fff700 1 -- 172.21.15.19:0/2659471981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a53000400 msgr2=0x558a53020ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.697+0000 7fe933fff700 1 --2- 172.21.15.19:0/2659471981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a53000400 0x558a53020ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.697+0000 7fe933fff700 1 -- 172.21.15.19:0/2659471981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52efdd00 msgr2=0x558a53012720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.697+0000 7fe933fff700 1 --2- 172.21.15.19:0/2659471981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52efdd00 0x558a53012720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.697+0000 7fe933fff700 1 -- 172.21.15.19:0/2659471981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a53021220 con 0x558a52e834a0 2024-05-09T02:27:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.697+0000 7fe933fff700 1 --2- 172.21.15.19:0/2659471981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a52e834a0 0x558a530121e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe92400a3e0 tx=0x7fe924027a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09d35c7dff9541b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.697+0000 7fe9137fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:27:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.697+0000 7fe9137fe700 1 -- 172.21.15.19:0/2659471981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924033040 con 0x558a52e834a0 2024-05-09T02:27:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.697+0000 7fe9137fe700 1 -- 172.21.15.19:0/2659471981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fe924010900 con 0x558a52e834a0 2024-05-09T02:27:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.697+0000 7fe948e09b80 1 -- 172.21.15.19:0/2659471981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9240277c0 con 0x558a52e834a0 2024-05-09T02:27:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.697+0000 7fe9137fe700 1 -- 172.21.15.19:0/2659471981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe924049050 con 0x558a52e834a0 2024-05-09T02:27:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.697+0000 7fe948e09b80 10 client.17217 fetch_fsmap learned FSMap version 59 2024-05-09T02:27:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.697+0000 7fe948e09b80 1 -- 172.21.15.19:0/2659471981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x558a53083f30 con 0x558a52e834a0 2024-05-09T02:27:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.698+0000 7fe9137fe700 1 -- 172.21.15.19:0/2659471981 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9240277c0 con 0x558a52e834a0 2024-05-09T02:27:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.698+0000 7fe9137fe700 1 -- 172.21.15.19:0/2659471981 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 59) ==== 8358+0+0 (secure 0 0 0) 0x7fe9240394a0 con 0x558a52e834a0 2024-05-09T02:27:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.698+0000 7fe948e09b80 10 client.17217 fetch_fsmap finished waiting for FSMap version 59 2024-05-09T02:27:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.698+0000 7fe948e09b80 10 client.17217 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T02:27:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.698+0000 7fe948e09b80 20 client.17217 populate_metadata read hostname 'smithi019' 2024-05-09T02:27:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.698+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2659471981 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x558a530844c0 0x558a530868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.698+0000 7fe948e09b80 4 client.17217 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:27:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.698+0000 7fe948e09b80 1 -- 172.21.15.19:0/2659471981 --> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a52f2ab20 con 0x558a530844c0 2024-05-09T02:27:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.698+0000 7fe9337fe700 1 --2- 172.21.15.19:0/2659471981 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x558a530844c0 0x558a530868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe9337fe700 1 --2- 172.21.15.19:0/2659471981 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x558a530844c0 0x558a530868e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=21aa5d85691abd46 server_cookie=11bab79e2bef4182 in_seq=0 out_seq=0 2024-05-09T02:27:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe9137fe700 10 client.17217 ms_handle_connect on v2:172.21.15.196:6834/442694937 2024-05-09T02:27:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe9137fe700 1 -- 172.21.15.19:0/2659471981 <== mds.3 v2:172.21.15.196:6834/442694937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a52f2ab20 con 0x558a530844c0 2024-05-09T02:27:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe9137fe700 10 client.17217 handle_command_reply: tid=0 2024-05-09T02:27:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe948e09b80 1 client.17217 shutdown 2024-05-09T02:27:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe948e09b80 1 -- 172.21.15.19:0/2659471981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a52e834a0 msgr2=0x558a530121e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2659471981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a52e834a0 0x558a530121e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe92400a3e0 tx=0x7fe924027a80 comp rx=0 tx=0).stop 2024-05-09T02:27:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe948e09b80 1 -- 172.21.15.19:0/2659471981 shutdown_connections 2024-05-09T02:27:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2659471981 >> [v2:172.21.15.196:6834/442694937,v1:172.21.15.196:6835/442694937] conn(0x558a530844c0 0x558a530868e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2659471981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a53000400 0x558a53020ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2659471981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a52e834a0 0x558a530121e0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe948e09b80 1 --2- 172.21.15.19:0/2659471981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558a52efdd00 0x558a53012720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe948e09b80 1 -- 172.21.15.19:0/2659471981 >> 172.21.15.19:0/2659471981 conn(0x558a52e86f70 msgr2=0x558a52ff4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe9137fe700 0 client.17217 ms_handle_reset on v2:172.21.15.196:6834/442694937 2024-05-09T02:27:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe948e09b80 1 -- 172.21.15.19:0/2659471981 shutdown_connections 2024-05-09T02:27:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe948e09b80 1 -- 172.21.15.19:0/2659471981 wait complete. 2024-05-09T02:27:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.699+0000 7fe948e09b80 20 client.17217 trim_cache size 0 max 16384 2024-05-09T02:27:00.707 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:27:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.700+0000 7fe93a3f3700 1 -- 172.21.15.19:0/3793677626 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe91c0b29f0 msgr2=0x7fe91c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.700+0000 7fe93a3f3700 1 --2- 172.21.15.19:0/3793677626 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe91c0b29f0 0x7fe91c0b4f00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe924005950 tx=0x7fe92400a300 comp rx=0 tx=0).stop 2024-05-09T02:27:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.700+0000 7fe93a3f3700 1 -- 172.21.15.19:0/3793677626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe934107df0 msgr2=0x7fe93419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.700+0000 7fe93a3f3700 1 --2- 172.21.15.19:0/3793677626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe934107df0 0x7fe93419b430 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe9280098e0 tx=0x7fe9280057c0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.700+0000 7fe93a3f3700 1 -- 172.21.15.19:0/3793677626 shutdown_connections 2024-05-09T02:27:00.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.700+0000 7fe93a3f3700 1 --2- 172.21.15.19:0/3793677626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe934108740 0x7fe93419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.700+0000 7fe93a3f3700 1 --2- 172.21.15.19:0/3793677626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9341034c0 0x7fe93419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.700+0000 7fe93a3f3700 1 --2- 172.21.15.19:0/3793677626 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fe91c0b29f0 0x7fe91c0b4f00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.700+0000 7fe93a3f3700 1 --2- 172.21.15.19:0/3793677626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe934107df0 0x7fe93419b430 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:00.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.700+0000 7fe93a3f3700 1 -- 172.21.15.19:0/3793677626 >> 172.21.15.19:0/3793677626 conn(0x7fe9340fedf0 msgr2=0x7fe934101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:00.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.700+0000 7fe93a3f3700 1 -- 172.21.15.19:0/3793677626 shutdown_connections 2024-05-09T02:27:00.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:00.700+0000 7fe93a3f3700 1 -- 172.21.15.19:0/3793677626 wait complete. 2024-05-09T02:27:00.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:00 smithi059 ceph-mon[34781]: pgmap v492: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 17 MiB/s wr, 1.57k op/s 2024-05-09T02:27:01.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:00 smithi196 ceph-mon[34718]: pgmap v492: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 17 MiB/s wr, 1.57k op/s 2024-05-09T02:27:01.726 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T02:27:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:27:02.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:02 smithi019 ceph-mon[30151]: pgmap v493: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 12 MiB/s wr, 1.11k op/s 2024-05-09T02:27:02.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:02 smithi059 ceph-mon[34781]: pgmap v493: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 12 MiB/s wr, 1.11k op/s 2024-05-09T02:27:03.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:02 smithi196 ceph-mon[34718]: pgmap v493: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 12 MiB/s wr, 1.11k op/s 2024-05-09T02:27:03.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.681+0000 7f290b3a1700 1 -- 172.21.15.19:0/4008026686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f290410e740 msgr2=0x7f2904110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:03.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.681+0000 7f290b3a1700 1 --2- 172.21.15.19:0/4008026686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f290410e740 0x7f2904110be0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009990 tx=0x7f28fc009ca0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.681+0000 7f290b3a1700 1 -- 172.21.15.19:0/4008026686 shutdown_connections 2024-05-09T02:27:03.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.681+0000 7f290b3a1700 1 --2- 172.21.15.19:0/4008026686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f290410e740 0x7f2904110be0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.681+0000 7f290b3a1700 1 --2- 172.21.15.19:0/4008026686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2904105310 0x7f2904105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.681+0000 7f290b3a1700 1 --2- 172.21.15.19:0/4008026686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2904105c60 0x7f290410e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.681+0000 7f290b3a1700 1 -- 172.21.15.19:0/4008026686 >> 172.21.15.19:0/4008026686 conn(0x7f2904078ac0 msgr2=0x7f2904078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:03.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.681+0000 7f290b3a1700 1 -- 172.21.15.19:0/4008026686 shutdown_connections 2024-05-09T02:27:03.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.681+0000 7f290b3a1700 1 -- 172.21.15.19:0/4008026686 wait complete. 2024-05-09T02:27:03.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.681+0000 7f290b3a1700 1 Processor -- start 2024-05-09T02:27:03.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.681+0000 7f290b3a1700 1 -- start start 2024-05-09T02:27:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290b3a1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2904105310 0x7f29041a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290b3a1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2904105c60 0x7f29041a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290b3a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f290410e740 0x7f290419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290b3a1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2904114260 con 0x7f290410e740 2024-05-09T02:27:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290b3a1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29041140e0 con 0x7f2904105310 2024-05-09T02:27:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290b3a1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29041143e0 con 0x7f2904105c60 2024-05-09T02:27:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290aba0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f290410e740 0x7f290419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290aba0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f290410e740 0x7f290419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52602/0 (socket says 172.21.15.19:52602) 2024-05-09T02:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290aba0700 1 -- 172.21.15.19:0/374747301 learned_addr learned my addr 172.21.15.19:0/374747301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f2909b9e700 1 --2- 172.21.15.19:0/374747301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2904105c60 0x7f29041a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290a39f700 1 --2- 172.21.15.19:0/374747301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2904105310 0x7f29041a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290aba0700 1 -- 172.21.15.19:0/374747301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2904105c60 msgr2=0x7f29041a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290aba0700 1 --2- 172.21.15.19:0/374747301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2904105c60 0x7f29041a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290aba0700 1 -- 172.21.15.19:0/374747301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2904105310 msgr2=0x7f29041a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290aba0700 1 --2- 172.21.15.19:0/374747301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2904105310 0x7f29041a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290aba0700 1 -- 172.21.15.19:0/374747301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290419b090 con 0x7f290410e740 2024-05-09T02:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.682+0000 7f290aba0700 1 --2- 172.21.15.19:0/374747301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f290410e740 0x7f290419a9e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0047b0 tx=0x7f28fc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.683+0000 7f28fb7fe700 1 -- 172.21.15.19:0/374747301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc026070 con 0x7f290410e740 2024-05-09T02:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.683+0000 7f290b3a1700 1 -- 172.21.15.19:0/374747301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f290419b320 con 0x7f290410e740 2024-05-09T02:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.683+0000 7f28fb7fe700 1 -- 172.21.15.19:0/374747301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28fc004c90 con 0x7f290410e740 2024-05-09T02:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.683+0000 7f28fb7fe700 1 -- 172.21.15.19:0/374747301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc003cd0 con 0x7f290410e740 2024-05-09T02:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.683+0000 7f290b3a1700 1 -- 172.21.15.19:0/374747301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29041a7650 con 0x7f290410e740 2024-05-09T02:27:03.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.685+0000 7f28fb7fe700 1 -- 172.21.15.19:0/374747301 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f28fc029070 con 0x7f290410e740 2024-05-09T02:27:03.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.686+0000 7f28fb7fe700 1 --2- 172.21.15.19:0/374747301 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f28f00b29f0 0x7f28f00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.686+0000 7f290a39f700 1 --2- 172.21.15.19:0/374747301 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f28f00b29f0 0x7f28f00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.686+0000 7f28fb7fe700 1 -- 172.21.15.19:0/374747301 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f28fc0e2ee0 con 0x7f290410e740 2024-05-09T02:27:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.687+0000 7f290a39f700 1 --2- 172.21.15.19:0/374747301 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f28f00b29f0 0x7f28f00b4f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2900009550 tx=0x7f29000091b0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.687+0000 7f291b019b80 1 Processor -- start 2024-05-09T02:27:03.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.687+0000 7f291b019b80 1 -- start start 2024-05-09T02:27:03.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.687+0000 7f291b019b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a2a6160 0x56354a33f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.687+0000 7f291b019b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a1bc4a0 0x56354a33fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.687+0000 7f291b019b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a339800 0x56354a339c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.687+0000 7f291b019b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56354a25bff0 con 0x56354a1bc4a0 2024-05-09T02:27:03.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.687+0000 7f291b019b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56354a1fa7f0 con 0x56354a2a6160 2024-05-09T02:27:03.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.687+0000 7f291b019b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56354a257fb0 con 0x56354a339800 2024-05-09T02:27:03.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.688+0000 7f2909b9e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a1bc4a0 0x56354a33fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.688+0000 7f2909b9e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a1bc4a0 0x56354a33fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52616/0 (socket says 172.21.15.19:52616) 2024-05-09T02:27:03.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.688+0000 7f2909b9e700 1 -- 172.21.15.19:0/3779980465 learned_addr learned my addr 172.21.15.19:0/3779980465 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:03.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.688+0000 7f290a39f700 1 --2- 172.21.15.19:0/3779980465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a2a6160 0x56354a33f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.688+0000 7f290aba0700 1 --2- 172.21.15.19:0/3779980465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a339800 0x56354a339c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.688+0000 7f2909b9e700 1 -- 172.21.15.19:0/3779980465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a339800 msgr2=0x56354a339c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:03.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.688+0000 7f2909b9e700 1 --2- 172.21.15.19:0/3779980465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a339800 0x56354a339c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.688+0000 7f2909b9e700 1 -- 172.21.15.19:0/3779980465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a2a6160 msgr2=0x56354a33f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:03.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.688+0000 7f2909b9e700 1 --2- 172.21.15.19:0/3779980465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a2a6160 0x56354a33f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.688+0000 7f2909b9e700 1 -- 172.21.15.19:0/3779980465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56354a1f2370 con 0x56354a1bc4a0 2024-05-09T02:27:03.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.688+0000 7f2909b9e700 1 --2- 172.21.15.19:0/3779980465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a1bc4a0 0x56354a33fb80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f28f400bc70 tx=0x7f28f400bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6f12de64c3ffab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:03.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.688+0000 7f28f8ff9700 1 -- 172.21.15.19:0/3779980465 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4010be0 con 0x56354a1bc4a0 2024-05-09T02:27:03.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.688+0000 7f28f8ff9700 1 -- 172.21.15.19:0/3779980465 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28f40044e0 con 0x56354a1bc4a0 2024-05-09T02:27:03.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.688+0000 7f28f8ff9700 1 -- 172.21.15.19:0/3779980465 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4005280 con 0x56354a1bc4a0 2024-05-09T02:27:03.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.689+0000 7f291b019b80 1 -- 172.21.15.19:0/3779980465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a1bc4a0 msgr2=0x56354a33fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:03.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.689+0000 7f291b019b80 1 --2- 172.21.15.19:0/3779980465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a1bc4a0 0x56354a33fb80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f28f400bc70 tx=0x7f28f400bf80 comp rx=0 tx=0).stop 2024-05-09T02:27:03.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.689+0000 7f291b019b80 1 -- 172.21.15.19:0/3779980465 shutdown_connections 2024-05-09T02:27:03.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.689+0000 7f291b019b80 1 --2- 172.21.15.19:0/3779980465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a339800 0x56354a339c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.689+0000 7f291b019b80 1 --2- 172.21.15.19:0/3779980465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a2a6160 0x56354a33f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.689+0000 7f291b019b80 1 --2- 172.21.15.19:0/3779980465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a1bc4a0 0x56354a33fb80 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.689+0000 7f291b019b80 1 -- 172.21.15.19:0/3779980465 >> 172.21.15.19:0/3779980465 conn(0x56354a21c6a0 msgr2=0x56354a334620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:03.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.689+0000 7f291b019b80 1 -- 172.21.15.19:0/3779980465 shutdown_connections 2024-05-09T02:27:03.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.689+0000 7f291b019b80 1 -- 172.21.15.19:0/3779980465 wait complete. 2024-05-09T02:27:03.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.690+0000 7f291b019b80 1 -- start start 2024-05-09T02:27:03.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.690+0000 7f291b019b80 1 Processor -- start 2024-05-09T02:27:03.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.690+0000 7f291b019b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a1bc4a0 0x56354a3433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.690+0000 7f291b019b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a2a6160 0x56354a343900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.690+0000 7f291b019b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a339800 0x56354a343e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.690+0000 7f291b019b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56354a0d7ba0 con 0x56354a1bc4a0 2024-05-09T02:27:03.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.690+0000 7f291b019b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56354a216150 con 0x56354a339800 2024-05-09T02:27:03.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.690+0000 7f291b019b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56354a33a5f0 con 0x56354a2a6160 2024-05-09T02:27:03.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.690+0000 7f291b019b80 10 client.0 _ll_register_callbacks cb 0x56354a0cd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:27:03.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.690+0000 7f290a39f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a1bc4a0 0x56354a3433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.690+0000 7f290a39f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a1bc4a0 0x56354a3433c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52630/0 (socket says 172.21.15.19:52630) 2024-05-09T02:27:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.690+0000 7f290a39f700 1 -- 172.21.15.19:0/268081621 learned_addr learned my addr 172.21.15.19:0/268081621 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.690+0000 7f2909b9e700 1 --2- 172.21.15.19:0/268081621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a2a6160 0x56354a343900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.691+0000 7f290aba0700 1 --2- 172.21.15.19:0/268081621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a339800 0x56354a343e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.691+0000 7f290a39f700 1 -- 172.21.15.19:0/268081621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a2a6160 msgr2=0x56354a343900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.691+0000 7f290a39f700 1 --2- 172.21.15.19:0/268081621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a2a6160 0x56354a343900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.691+0000 7f290a39f700 1 -- 172.21.15.19:0/268081621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a339800 msgr2=0x56354a343e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.691+0000 7f290a39f700 1 --2- 172.21.15.19:0/268081621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a339800 0x56354a343e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.691+0000 7f290a39f700 1 -- 172.21.15.19:0/268081621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56354a0d7490 con 0x56354a1bc4a0 2024-05-09T02:27:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.691+0000 7f290a39f700 1 --2- 172.21.15.19:0/268081621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a1bc4a0 0x56354a3433c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2900007ab0 tx=0x7f2900004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d512f042486ee5c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.691+0000 7f28edffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:27:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.691+0000 7f28edffb700 1 -- 172.21.15.19:0/268081621 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900026070 con 0x56354a1bc4a0 2024-05-09T02:27:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.691+0000 7f28edffb700 1 -- 172.21.15.19:0/268081621 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f2900015ce0 con 0x56354a1bc4a0 2024-05-09T02:27:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.691+0000 7f291b019b80 1 -- 172.21.15.19:0/268081621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28f40044e0 con 0x56354a1bc4a0 2024-05-09T02:27:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.691+0000 7f28edffb700 1 -- 172.21.15.19:0/268081621 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2900032050 con 0x56354a1bc4a0 2024-05-09T02:27:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.691+0000 7f291b019b80 10 client.17229 fetch_fsmap learned FSMap version 59 2024-05-09T02:27:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.691+0000 7f291b019b80 1 -- 172.21.15.19:0/268081621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x56354a350d00 con 0x56354a1bc4a0 2024-05-09T02:27:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.692+0000 7f28edffb700 1 -- 172.21.15.19:0/268081621 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28f40044e0 con 0x56354a1bc4a0 2024-05-09T02:27:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.692+0000 7f28edffb700 1 -- 172.21.15.19:0/268081621 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 59) ==== 8358+0+0 (secure 0 0 0) 0x7f29000163f0 con 0x56354a1bc4a0 2024-05-09T02:27:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.692+0000 7f291b019b80 10 client.17229 fetch_fsmap finished waiting for FSMap version 59 2024-05-09T02:27:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.692+0000 7f291b019b80 10 client.17229 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:27:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.692+0000 7f291b019b80 20 client.17229 populate_metadata read hostname 'smithi019' 2024-05-09T02:27:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.692+0000 7f291b019b80 1 --2- 172.21.15.19:0/268081621 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x56354a3532e0 0x56354a355700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.692+0000 7f291b019b80 4 client.17229 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:27:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.692+0000 7f291b019b80 1 -- 172.21.15.19:0/268081621 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56354a2699d0 con 0x56354a3532e0 2024-05-09T02:27:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.692+0000 7f2909b9e700 1 --2- 172.21.15.19:0/268081621 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x56354a3532e0 0x56354a355700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.693+0000 7f2909b9e700 1 --2- 172.21.15.19:0/268081621 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x56354a3532e0 0x56354a355700 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3198b12d78ce35d3 server_cookie=57242a38b210bf76 in_seq=0 out_seq=0 2024-05-09T02:27:03.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.693+0000 7f28edffb700 10 client.17229 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:27:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.693+0000 7f28edffb700 1 -- 172.21.15.19:0/268081621 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56354a2699d0 con 0x56354a3532e0 2024-05-09T02:27:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.693+0000 7f28edffb700 10 client.17229 handle_command_reply: tid=0 2024-05-09T02:27:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.694+0000 7f291b019b80 1 client.17229 shutdown 2024-05-09T02:27:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.694+0000 7f291b019b80 1 -- 172.21.15.19:0/268081621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a1bc4a0 msgr2=0x56354a3433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.694+0000 7f291b019b80 1 --2- 172.21.15.19:0/268081621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a1bc4a0 0x56354a3433c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2900007ab0 tx=0x7f2900004b40 comp rx=0 tx=0).stop 2024-05-09T02:27:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.694+0000 7f291b019b80 1 -- 172.21.15.19:0/268081621 shutdown_connections 2024-05-09T02:27:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.694+0000 7f291b019b80 1 --2- 172.21.15.19:0/268081621 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x56354a3532e0 0x56354a355700 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.694+0000 7f291b019b80 1 --2- 172.21.15.19:0/268081621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a339800 0x56354a343e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.694+0000 7f291b019b80 1 --2- 172.21.15.19:0/268081621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a1bc4a0 0x56354a3433c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.694+0000 7f291b019b80 1 --2- 172.21.15.19:0/268081621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a2a6160 0x56354a343900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.694+0000 7f28edffb700 0 client.17229 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:27:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.694+0000 7f291b019b80 1 -- 172.21.15.19:0/268081621 >> 172.21.15.19:0/268081621 conn(0x56354a21c6a0 msgr2=0x56354a334380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.694+0000 7f291b019b80 1 -- 172.21.15.19:0/268081621 shutdown_connections 2024-05-09T02:27:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.694+0000 7f291b019b80 1 -- 172.21.15.19:0/268081621 wait complete. 2024-05-09T02:27:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.694+0000 7f291b019b80 20 client.17229 trim_cache size 0 max 16384 2024-05-09T02:27:03.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.709+0000 7f291b019b80 1 Processor -- start 2024-05-09T02:27:03.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.709+0000 7f291b019b80 1 -- start start 2024-05-09T02:27:03.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.709+0000 7f291b019b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a1bc4a0 0x56354a340e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.709+0000 7f291b019b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a2a6160 0x56354a34f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.709+0000 7f291b019b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a339800 0x56354a35af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.709+0000 7f291b019b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29041aec90 con 0x56354a2a6160 2024-05-09T02:27:03.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.709+0000 7f291b019b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29041b0800 con 0x56354a339800 2024-05-09T02:27:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.709+0000 7f291b019b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29041af390 con 0x56354a1bc4a0 2024-05-09T02:27:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.709+0000 7f2909b9e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a2a6160 0x56354a34f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.709+0000 7f2909b9e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a2a6160 0x56354a34f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52638/0 (socket says 172.21.15.19:52638) 2024-05-09T02:27:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.709+0000 7f2909b9e700 1 -- 172.21.15.19:0/2312066483 learned_addr learned my addr 172.21.15.19:0/2312066483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.710+0000 7f290aba0700 1 --2- 172.21.15.19:0/2312066483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a339800 0x56354a35af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.710+0000 7f290a39f700 1 --2- 172.21.15.19:0/2312066483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a1bc4a0 0x56354a340e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.710+0000 7f2909b9e700 1 -- 172.21.15.19:0/2312066483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a1bc4a0 msgr2=0x56354a340e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.710+0000 7f2909b9e700 1 --2- 172.21.15.19:0/2312066483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a1bc4a0 0x56354a340e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.710+0000 7f2909b9e700 1 -- 172.21.15.19:0/2312066483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a339800 msgr2=0x56354a35af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.710+0000 7f2909b9e700 1 --2- 172.21.15.19:0/2312066483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a339800 0x56354a35af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.710+0000 7f2909b9e700 1 -- 172.21.15.19:0/2312066483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56354a335ea0 con 0x56354a2a6160 2024-05-09T02:27:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.710+0000 7f290aba0700 1 --2- 172.21.15.19:0/2312066483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a339800 0x56354a35af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:27:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.710+0000 7f2909b9e700 1 --2- 172.21.15.19:0/2312066483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a2a6160 0x56354a34f170 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f28f40106b0 tx=0x7f28f4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fe44b61aab60ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.710+0000 7f28effff700 1 -- 172.21.15.19:0/2312066483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4010be0 con 0x56354a2a6160 2024-05-09T02:27:03.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.710+0000 7f28effff700 1 -- 172.21.15.19:0/2312066483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28f40044d0 con 0x56354a2a6160 2024-05-09T02:27:03.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.710+0000 7f28effff700 1 -- 172.21.15.19:0/2312066483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4003c20 con 0x56354a2a6160 2024-05-09T02:27:03.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.711+0000 7f291b019b80 1 -- 172.21.15.19:0/2312066483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a2a6160 msgr2=0x56354a34f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:03.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.711+0000 7f291b019b80 1 --2- 172.21.15.19:0/2312066483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a2a6160 0x56354a34f170 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f28f40106b0 tx=0x7f28f4004d10 comp rx=0 tx=0).stop 2024-05-09T02:27:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.711+0000 7f291b019b80 1 -- 172.21.15.19:0/2312066483 shutdown_connections 2024-05-09T02:27:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.711+0000 7f291b019b80 1 --2- 172.21.15.19:0/2312066483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a339800 0x56354a35af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.711+0000 7f291b019b80 1 --2- 172.21.15.19:0/2312066483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a1bc4a0 0x56354a340e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.711+0000 7f291b019b80 1 --2- 172.21.15.19:0/2312066483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a2a6160 0x56354a34f170 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.711+0000 7f291b019b80 1 -- 172.21.15.19:0/2312066483 >> 172.21.15.19:0/2312066483 conn(0x56354a21c6a0 msgr2=0x56354a341cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.711+0000 7f291b019b80 1 -- 172.21.15.19:0/2312066483 shutdown_connections 2024-05-09T02:27:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.711+0000 7f291b019b80 1 -- 172.21.15.19:0/2312066483 wait complete. 2024-05-09T02:27:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.711+0000 7f291b019b80 1 -- start start 2024-05-09T02:27:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f291b019b80 1 Processor -- start 2024-05-09T02:27:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f291b019b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a1bc4a0 0x56354a34b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f291b019b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a2a6160 0x56354a34ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f291b019b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a339800 0x56354a359ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f291b019b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29041b0130 con 0x56354a339800 2024-05-09T02:27:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f291b019b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29041afa60 con 0x56354a1bc4a0 2024-05-09T02:27:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f291b019b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29041acc50 con 0x56354a2a6160 2024-05-09T02:27:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f291b019b80 10 client.0 _ll_register_callbacks cb 0x56354a0cd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:27:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f290aba0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a339800 0x56354a359ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f290aba0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a339800 0x56354a359ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52652/0 (socket says 172.21.15.19:52652) 2024-05-09T02:27:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f290aba0700 1 -- 172.21.15.19:0/2770082926 learned_addr learned my addr 172.21.15.19:0/2770082926 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f290a39f700 1 --2- 172.21.15.19:0/2770082926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a1bc4a0 0x56354a34b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f2909b9e700 1 --2- 172.21.15.19:0/2770082926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a2a6160 0x56354a34ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f290aba0700 1 -- 172.21.15.19:0/2770082926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a2a6160 msgr2=0x56354a34ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f290aba0700 1 --2- 172.21.15.19:0/2770082926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a2a6160 0x56354a34ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f290aba0700 1 -- 172.21.15.19:0/2770082926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a1bc4a0 msgr2=0x56354a34b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f290aba0700 1 --2- 172.21.15.19:0/2770082926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a1bc4a0 0x56354a34b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f290aba0700 1 -- 172.21.15.19:0/2770082926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56354a35a560 con 0x56354a339800 2024-05-09T02:27:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.712+0000 7f290a39f700 1 --2- 172.21.15.19:0/2770082926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a1bc4a0 0x56354a34b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:27:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f290aba0700 1 --2- 172.21.15.19:0/2770082926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a339800 0x56354a359ef0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0e0c00 tx=0x7f28fc003790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d534946b0929131d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f28edffb700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:27:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f28edffb700 1 -- 172.21.15.19:0/2770082926 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0aa920 con 0x56354a339800 2024-05-09T02:27:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f28edffb700 1 -- 172.21.15.19:0/2770082926 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f28fc0ae370 con 0x56354a339800 2024-05-09T02:27:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f291b019b80 1 -- 172.21.15.19:0/2770082926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28f40044d0 con 0x56354a339800 2024-05-09T02:27:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f28edffb700 1 -- 172.21.15.19:0/2770082926 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f28fc0aa220 con 0x56354a339800 2024-05-09T02:27:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f291b019b80 10 client.17235 fetch_fsmap learned FSMap version 59 2024-05-09T02:27:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f291b019b80 1 -- 172.21.15.19:0/2770082926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x56354a3bd3d0 con 0x56354a339800 2024-05-09T02:27:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f28edffb700 1 -- 172.21.15.19:0/2770082926 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28f40044d0 con 0x56354a339800 2024-05-09T02:27:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f28edffb700 1 -- 172.21.15.19:0/2770082926 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 59) ==== 8358+0+0 (secure 0 0 0) 0x7f28fc004e40 con 0x56354a339800 2024-05-09T02:27:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f291b019b80 10 client.17235 fetch_fsmap finished waiting for FSMap version 59 2024-05-09T02:27:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f291b019b80 10 client.17235 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T02:27:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f291b019b80 20 client.17235 populate_metadata read hostname 'smithi019' 2024-05-09T02:27:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f291b019b80 1 --2- 172.21.15.19:0/2770082926 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x56354a3bdad0 0x56354a3bfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f291b019b80 4 client.17235 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:27:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.713+0000 7f291b019b80 1 -- 172.21.15.19:0/2770082926 --> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56354a263a70 con 0x56354a3bdad0 2024-05-09T02:27:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.714+0000 7f290a39f700 1 --2- 172.21.15.19:0/2770082926 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x56354a3bdad0 0x56354a3bfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.714+0000 7f290a39f700 1 --2- 172.21.15.19:0/2770082926 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x56354a3bdad0 0x56354a3bfef0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a9a2b9df4e3c8cb server_cookie=3c0c9b0ecefe403b in_seq=0 out_seq=0 2024-05-09T02:27:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.714+0000 7f28edffb700 10 client.17235 ms_handle_connect on v2:172.21.15.196:6836/1033177921 2024-05-09T02:27:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.714+0000 7f28edffb700 1 -- 172.21.15.19:0/2770082926 <== mds.2 v2:172.21.15.196:6836/1033177921 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56354a263a70 con 0x56354a3bdad0 2024-05-09T02:27:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.714+0000 7f28edffb700 10 client.17235 handle_command_reply: tid=0 2024-05-09T02:27:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.714+0000 7f291b019b80 1 client.17235 shutdown 2024-05-09T02:27:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.714+0000 7f291b019b80 1 -- 172.21.15.19:0/2770082926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a339800 msgr2=0x56354a359ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.714+0000 7f291b019b80 1 --2- 172.21.15.19:0/2770082926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a339800 0x56354a359ef0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0e0c00 tx=0x7f28fc003790 comp rx=0 tx=0).stop 2024-05-09T02:27:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f291b019b80 1 -- 172.21.15.19:0/2770082926 shutdown_connections 2024-05-09T02:27:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f291b019b80 1 --2- 172.21.15.19:0/2770082926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354a339800 0x56354a359ef0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f291b019b80 1 --2- 172.21.15.19:0/2770082926 >> [v2:172.21.15.196:6836/1033177921,v1:172.21.15.196:6837/1033177921] conn(0x56354a3bdad0 0x56354a3bfef0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f291b019b80 1 --2- 172.21.15.19:0/2770082926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56354a1bc4a0 0x56354a34b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f291b019b80 1 --2- 172.21.15.19:0/2770082926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56354a2a6160 0x56354a34ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f291b019b80 1 -- 172.21.15.19:0/2770082926 >> 172.21.15.19:0/2770082926 conn(0x56354a21c6a0 msgr2=0x56354a33e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:03.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f28edffb700 0 client.17235 ms_handle_reset on v2:172.21.15.196:6836/1033177921 2024-05-09T02:27:03.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f291b019b80 1 -- 172.21.15.19:0/2770082926 shutdown_connections 2024-05-09T02:27:03.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f291b019b80 1 -- 172.21.15.19:0/2770082926 wait complete. 2024-05-09T02:27:03.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f291b019b80 20 client.17235 trim_cache size 0 max 16384 2024-05-09T02:27:03.723 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:27:03.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f290b3a1700 1 -- 172.21.15.19:0/374747301 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f28f00b29f0 msgr2=0x7f28f00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:03.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f290b3a1700 1 --2- 172.21.15.19:0/374747301 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f28f00b29f0 0x7f28f00b4f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2900009550 tx=0x7f29000091b0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f290b3a1700 1 -- 172.21.15.19:0/374747301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f290410e740 msgr2=0x7f290419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:03.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f290b3a1700 1 --2- 172.21.15.19:0/374747301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f290410e740 0x7f290419a9e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0047b0 tx=0x7f28fc005e80 comp rx=0 tx=0).stop 2024-05-09T02:27:03.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f290b3a1700 1 -- 172.21.15.19:0/374747301 shutdown_connections 2024-05-09T02:27:03.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f290b3a1700 1 --2- 172.21.15.19:0/374747301 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f28f00b29f0 0x7f28f00b4f00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f290b3a1700 1 --2- 172.21.15.19:0/374747301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f290410e740 0x7f290419a9e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f290b3a1700 1 --2- 172.21.15.19:0/374747301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2904105310 0x7f29041a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f290b3a1700 1 --2- 172.21.15.19:0/374747301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2904105c60 0x7f29041a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:03.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f290b3a1700 1 -- 172.21.15.19:0/374747301 >> 172.21.15.19:0/374747301 conn(0x7f2904078ac0 msgr2=0x7f29040ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:03.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.715+0000 7f290b3a1700 1 -- 172.21.15.19:0/374747301 shutdown_connections 2024-05-09T02:27:03.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:03.716+0000 7f290b3a1700 1 -- 172.21.15.19:0/374747301 wait complete. 2024-05-09T02:27:04.493 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T02:27:04.752 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:27:04.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:04 smithi019 ceph-mon[30151]: pgmap v494: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 6.9 MiB/s wr, 633 op/s 2024-05-09T02:27:04.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:04 smithi059 ceph-mon[34781]: pgmap v494: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 6.9 MiB/s wr, 633 op/s 2024-05-09T02:27:05.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:04 smithi196 ceph-mon[34718]: pgmap v494: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 6.9 MiB/s wr, 633 op/s 2024-05-09T02:27:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.552+0000 7f9fb675e700 1 -- 172.21.15.19:0/1727860411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb01039e0 msgr2=0x7f9fb0103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.552+0000 7f9fb675e700 1 --2- 172.21.15.19:0/1727860411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb01039e0 0x7f9fb0103df0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9f98009990 tx=0x7f9f98009ca0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.552+0000 7f9fb675e700 1 -- 172.21.15.19:0/1727860411 shutdown_connections 2024-05-09T02:27:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.552+0000 7f9fb675e700 1 --2- 172.21.15.19:0/1727860411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fb0104330 0x7f9fb0110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.552+0000 7f9fb675e700 1 --2- 172.21.15.19:0/1727860411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb01092d0 0x7f9fb01096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.552+0000 7f9fb675e700 1 --2- 172.21.15.19:0/1727860411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb01039e0 0x7f9fb0103df0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.552+0000 7f9fb675e700 1 -- 172.21.15.19:0/1727860411 >> 172.21.15.19:0/1727860411 conn(0x7f9fb00fe520 msgr2=0x7f9fb0100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.552+0000 7f9fb675e700 1 -- 172.21.15.19:0/1727860411 shutdown_connections 2024-05-09T02:27:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.552+0000 7f9fb675e700 1 -- 172.21.15.19:0/1727860411 wait complete. 2024-05-09T02:27:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.553+0000 7f9fb675e700 1 Processor -- start 2024-05-09T02:27:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.553+0000 7f9fb675e700 1 -- start start 2024-05-09T02:27:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.553+0000 7f9fb675e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb01039e0 0x7f9fb0104e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.553+0000 7f9fb675e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fb0104330 0x7f9fb0105380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.553+0000 7f9fb675e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb01092d0 0x7f9fb01058c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.553+0000 7f9fb675e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb0113350 con 0x7f9fb01039e0 2024-05-09T02:27:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.553+0000 7f9fb675e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb01131d0 con 0x7f9fb01092d0 2024-05-09T02:27:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.553+0000 7f9fb675e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb01134d0 con 0x7f9fb0104330 2024-05-09T02:27:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.553+0000 7f9faffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb01039e0 0x7f9fb0104e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.553+0000 7f9faffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb01039e0 0x7f9fb0104e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52674/0 (socket says 172.21.15.19:52674) 2024-05-09T02:27:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.553+0000 7f9faffff700 1 -- 172.21.15.19:0/60270420 learned_addr learned my addr 172.21.15.19:0/60270420 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.553+0000 7f9fa7fff700 1 --2- 172.21.15.19:0/60270420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fb0104330 0x7f9fb0105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.553+0000 7f9fb4cfb700 1 --2- 172.21.15.19:0/60270420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb01092d0 0x7f9fb01058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.554+0000 7f9faffff700 1 -- 172.21.15.19:0/60270420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fb0104330 msgr2=0x7f9fb0105380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.554+0000 7f9faffff700 1 --2- 172.21.15.19:0/60270420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fb0104330 0x7f9fb0105380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.554+0000 7f9faffff700 1 -- 172.21.15.19:0/60270420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb01092d0 msgr2=0x7f9fb01058c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.554+0000 7f9faffff700 1 --2- 172.21.15.19:0/60270420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb01092d0 0x7f9fb01058c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.554+0000 7f9faffff700 1 -- 172.21.15.19:0/60270420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb01a9210 con 0x7f9fb01039e0 2024-05-09T02:27:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.554+0000 7f9faffff700 1 --2- 172.21.15.19:0/60270420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb01039e0 0x7f9fb0104e40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000d950 tx=0x7f9fa000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:06.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.554+0000 7f9fadffb700 1 -- 172.21.15.19:0/60270420 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0016070 con 0x7f9fb01039e0 2024-05-09T02:27:06.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.554+0000 7f9fadffb700 1 -- 172.21.15.19:0/60270420 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fa000fd40 con 0x7f9fb01039e0 2024-05-09T02:27:06.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.554+0000 7f9fadffb700 1 -- 172.21.15.19:0/60270420 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa000e3b0 con 0x7f9fb01039e0 2024-05-09T02:27:06.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.554+0000 7f9fb675e700 1 -- 172.21.15.19:0/60270420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb01a9500 con 0x7f9fb01039e0 2024-05-09T02:27:06.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.554+0000 7f9fb675e700 1 -- 172.21.15.19:0/60270420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb01a99c0 con 0x7f9fb01039e0 2024-05-09T02:27:06.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.557+0000 7f9fadffb700 1 -- 172.21.15.19:0/60270420 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9fa0019070 con 0x7f9fb01039e0 2024-05-09T02:27:06.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.558+0000 7f9fadffb700 1 --2- 172.21.15.19:0/60270420 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f9f9c0b2a70 0x7f9f9c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.558+0000 7f9fa7fff700 1 --2- 172.21.15.19:0/60270420 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f9f9c0b2a70 0x7f9f9c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.558+0000 7f9fadffb700 1 -- 172.21.15.19:0/60270420 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f9fa00e2070 con 0x7f9fb01039e0 2024-05-09T02:27:06.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.558+0000 7f9fc5174b80 1 Processor -- start 2024-05-09T02:27:06.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.558+0000 7f9fc5174b80 1 -- start start 2024-05-09T02:27:06.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fc5174b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa72b160 0x55e9fa7c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fc5174b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa6414a0 0x55e9fa7c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fc5174b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa7bead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fc5174b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e9fa6e0ff0 con 0x55e9fa7be6a0 2024-05-09T02:27:06.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fc5174b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e9fa67f7f0 con 0x55e9fa6414a0 2024-05-09T02:27:06.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fc5174b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e9fa6dcfb0 con 0x55e9fa72b160 2024-05-09T02:27:06.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fa7fff700 1 --2- 172.21.15.19:0/60270420 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f9f9c0b2a70 0x7f9f9c0b4f80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f98006010 tx=0x7f9f98005960 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:06.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fb4cfb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa7bead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fb4cfb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa7bead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52680/0 (socket says 172.21.15.19:52680) 2024-05-09T02:27:06.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fb4cfb700 1 -- 172.21.15.19:0/1534236147 learned_addr learned my addr 172.21.15.19:0/1534236147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:06.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fa7fff700 1 --2- 172.21.15.19:0/1534236147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa6414a0 0x55e9fa7c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9faffff700 1 --2- 172.21.15.19:0/1534236147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa72b160 0x55e9fa7c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fb4cfb700 1 -- 172.21.15.19:0/1534236147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa72b160 msgr2=0x55e9fa7c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:06.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fb4cfb700 1 --2- 172.21.15.19:0/1534236147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa72b160 0x55e9fa7c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fb4cfb700 1 -- 172.21.15.19:0/1534236147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa6414a0 msgr2=0x55e9fa7c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:06.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fb4cfb700 1 --2- 172.21.15.19:0/1534236147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa6414a0 0x55e9fa7c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fb4cfb700 1 -- 172.21.15.19:0/1534236147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9fa677370 con 0x55e9fa7be6a0 2024-05-09T02:27:06.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fb4cfb700 1 --2- 172.21.15.19:0/1534236147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa7bead0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80098f0 tx=0x7f9fa8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc2c96e1d6548ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:06.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.559+0000 7f9fa67fc700 1 -- 172.21.15.19:0/1534236147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8004bb0 con 0x55e9fa7be6a0 2024-05-09T02:27:06.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.560+0000 7f9fa67fc700 1 -- 172.21.15.19:0/1534236147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fa8004d50 con 0x55e9fa7be6a0 2024-05-09T02:27:06.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.560+0000 7f9fa67fc700 1 -- 172.21.15.19:0/1534236147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8007e40 con 0x55e9fa7be6a0 2024-05-09T02:27:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.560+0000 7f9fc5174b80 1 -- 172.21.15.19:0/1534236147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 msgr2=0x55e9fa7bead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.560+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/1534236147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa7bead0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80098f0 tx=0x7f9fa8009c00 comp rx=0 tx=0).stop 2024-05-09T02:27:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.560+0000 7f9fc5174b80 1 -- 172.21.15.19:0/1534236147 shutdown_connections 2024-05-09T02:27:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.560+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/1534236147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa7bead0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.560+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/1534236147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa72b160 0x55e9fa7c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.560+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/1534236147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa6414a0 0x55e9fa7c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.560+0000 7f9fc5174b80 1 -- 172.21.15.19:0/1534236147 >> 172.21.15.19:0/1534236147 conn(0x55e9fa6a16a0 msgr2=0x55e9fa7b9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.561+0000 7f9fc5174b80 1 -- 172.21.15.19:0/1534236147 shutdown_connections 2024-05-09T02:27:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.561+0000 7f9fc5174b80 1 -- 172.21.15.19:0/1534236147 wait complete. 2024-05-09T02:27:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.561+0000 7f9fc5174b80 1 -- start start 2024-05-09T02:27:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.561+0000 7f9fc5174b80 1 Processor -- start 2024-05-09T02:27:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.561+0000 7f9fc5174b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa6414a0 0x55e9fa7c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.561+0000 7f9fc5174b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa72b160 0x55e9fa7c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.561+0000 7f9fc5174b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa7c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9fc5174b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e9fa55cba0 con 0x55e9fa7be6a0 2024-05-09T02:27:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9fc5174b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e9fa69b150 con 0x55e9fa72b160 2024-05-09T02:27:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9fc5174b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e9fa7cb1d0 con 0x55e9fa6414a0 2024-05-09T02:27:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9fc5174b80 10 client.0 _ll_register_callbacks cb 0x55e9fa5526c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:27:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9faffff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa6414a0 0x55e9fa7c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9fb4cfb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa7c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9faffff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa6414a0 0x55e9fa7c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.19:39856/0 (socket says 172.21.15.19:39856) 2024-05-09T02:27:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9faffff700 1 -- 172.21.15.19:0/1285756122 learned_addr learned my addr 172.21.15.19:0/1285756122 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9fa7fff700 1 --2- 172.21.15.19:0/1285756122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa72b160 0x55e9fa7c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9fb4cfb700 1 -- 172.21.15.19:0/1285756122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa6414a0 msgr2=0x55e9fa7c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9fb4cfb700 1 --2- 172.21.15.19:0/1285756122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa6414a0 0x55e9fa7c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9fb4cfb700 1 -- 172.21.15.19:0/1285756122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa72b160 msgr2=0x55e9fa7c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9fb4cfb700 1 --2- 172.21.15.19:0/1285756122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa72b160 0x55e9fa7c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9fb4cfb700 1 -- 172.21.15.19:0/1285756122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9fa55c490 con 0x55e9fa7be6a0 2024-05-09T02:27:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9fa7fff700 1 --2- 172.21.15.19:0/1285756122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa72b160 0x55e9fa7c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:27:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9fb4cfb700 1 --2- 172.21.15.19:0/1285756122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa7c8c50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8004040 tx=0x7f9fa8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d86de8a0cdfa0aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9f8f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:27:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9f8f7fe700 1 -- 172.21.15.19:0/1285756122 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8022070 con 0x55e9fa7be6a0 2024-05-09T02:27:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.562+0000 7f9f8f7fe700 1 -- 172.21.15.19:0/1285756122 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f9fa8007cd0 con 0x55e9fa7be6a0 2024-05-09T02:27:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.563+0000 7f9fc5174b80 1 -- 172.21.15.19:0/1285756122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fa8004d50 con 0x55e9fa7be6a0 2024-05-09T02:27:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.563+0000 7f9f8f7fe700 1 -- 172.21.15.19:0/1285756122 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f9fa802e050 con 0x55e9fa7be6a0 2024-05-09T02:27:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.563+0000 7f9fc5174b80 10 client.17247 fetch_fsmap learned FSMap version 59 2024-05-09T02:27:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.563+0000 7f9fc5174b80 1 -- 172.21.15.19:0/1285756122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55e9fa7d5ba0 con 0x55e9fa7be6a0 2024-05-09T02:27:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.563+0000 7f9f8f7fe700 1 -- 172.21.15.19:0/1285756122 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fa8004d50 con 0x55e9fa7be6a0 2024-05-09T02:27:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.563+0000 7f9f8f7fe700 1 -- 172.21.15.19:0/1285756122 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 59) ==== 8358+0+0 (secure 0 0 0) 0x7f9fa801e410 con 0x55e9fa7be6a0 2024-05-09T02:27:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.563+0000 7f9fc5174b80 10 client.17247 fetch_fsmap finished waiting for FSMap version 59 2024-05-09T02:27:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.563+0000 7f9fc5174b80 10 client.17247 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:27:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.563+0000 7f9fc5174b80 20 client.17247 populate_metadata read hostname 'smithi019' 2024-05-09T02:27:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.563+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/1285756122 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e9fa7d8070 0x55e9fa7da490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.563+0000 7f9fc5174b80 4 client.17247 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:27:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.563+0000 7f9fc5174b80 1 -- 172.21.15.19:0/1285756122 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9fa6ee9d0 con 0x55e9fa7d8070 2024-05-09T02:27:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.564+0000 7f9faffff700 1 --2- 172.21.15.19:0/1285756122 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e9fa7d8070 0x55e9fa7da490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.564+0000 7f9f8f7fe700 10 client.17247 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:27:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.564+0000 7f9faffff700 1 --2- 172.21.15.19:0/1285756122 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e9fa7d8070 0x55e9fa7da490 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce4d44c8ac8bdf8f server_cookie=d4ce6ebba6595f79 in_seq=0 out_seq=0 2024-05-09T02:27:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.565+0000 7f9f8f7fe700 1 -- 172.21.15.19:0/1285756122 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e9fa6ee9d0 con 0x55e9fa7d8070 2024-05-09T02:27:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.565+0000 7f9f8f7fe700 10 client.17247 handle_command_reply: tid=0 2024-05-09T02:27:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.565+0000 7f9fc5174b80 1 client.17247 shutdown 2024-05-09T02:27:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.565+0000 7f9fc5174b80 1 -- 172.21.15.19:0/1285756122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 msgr2=0x55e9fa7c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.565+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/1285756122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa7c8c50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8004040 tx=0x7f9fa8004600 comp rx=0 tx=0).stop 2024-05-09T02:27:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.565+0000 7f9fc5174b80 1 -- 172.21.15.19:0/1285756122 shutdown_connections 2024-05-09T02:27:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.565+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/1285756122 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e9fa7d8070 0x55e9fa7da490 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.565+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/1285756122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa7c8c50 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.565+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/1285756122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa6414a0 0x55e9fa7c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.565+0000 7f9f8f7fe700 0 client.17247 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:27:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.565+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/1285756122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa72b160 0x55e9fa7c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.565+0000 7f9fc5174b80 1 -- 172.21.15.19:0/1285756122 >> 172.21.15.19:0/1285756122 conn(0x55e9fa6a16a0 msgr2=0x55e9fa7b91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.565+0000 7f9fc5174b80 1 -- 172.21.15.19:0/1285756122 shutdown_connections 2024-05-09T02:27:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.565+0000 7f9fc5174b80 1 -- 172.21.15.19:0/1285756122 wait complete. 2024-05-09T02:27:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.565+0000 7f9fc5174b80 20 client.17247 trim_cache size 0 max 16384 2024-05-09T02:27:06.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9fc5174b80 1 Processor -- start 2024-05-09T02:27:06.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9fc5174b80 1 -- start start 2024-05-09T02:27:06.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9fc5174b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa6414a0 0x55e9fa7d43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9fc5174b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa72b160 0x55e9fa7c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9fc5174b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa7be6a0 0x55e9fa7c0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9fc5174b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb01b1250 con 0x55e9fa6414a0 2024-05-09T02:27:06.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9fc5174b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb01b2dc0 con 0x55e9fa72b160 2024-05-09T02:27:06.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9fc5174b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb01b1950 con 0x55e9fa7be6a0 2024-05-09T02:27:06.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9faffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa6414a0 0x55e9fa7d43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9faffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa6414a0 0x55e9fa7d43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52696/0 (socket says 172.21.15.19:52696) 2024-05-09T02:27:06.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9faffff700 1 -- 172.21.15.19:0/680399717 learned_addr learned my addr 172.21.15.19:0/680399717 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:06.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9fb4cfb700 1 --2- 172.21.15.19:0/680399717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa7be6a0 0x55e9fa7c0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9fa7fff700 1 --2- 172.21.15.19:0/680399717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa72b160 0x55e9fa7c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9faffff700 1 -- 172.21.15.19:0/680399717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa7be6a0 msgr2=0x55e9fa7c0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:06.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9faffff700 1 --2- 172.21.15.19:0/680399717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa7be6a0 0x55e9fa7c0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9faffff700 1 -- 172.21.15.19:0/680399717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa72b160 msgr2=0x55e9fa7c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:06.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9faffff700 1 --2- 172.21.15.19:0/680399717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa72b160 0x55e9fa7c38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.581+0000 7f9faffff700 1 -- 172.21.15.19:0/680399717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9fa7cb0c0 con 0x55e9fa6414a0 2024-05-09T02:27:06.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.582+0000 7f9faffff700 1 --2- 172.21.15.19:0/680399717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa6414a0 0x55e9fa7d43a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00a8fc0 tx=0x7f9fa0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf1b2cecc61726e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:06.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.582+0000 7f9fa5ffb700 1 -- 172.21.15.19:0/680399717 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0020040 con 0x55e9fa6414a0 2024-05-09T02:27:06.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.582+0000 7f9fa5ffb700 1 -- 172.21.15.19:0/680399717 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fa00c3070 con 0x55e9fa6414a0 2024-05-09T02:27:06.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.582+0000 7f9fa5ffb700 1 -- 172.21.15.19:0/680399717 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa00a9780 con 0x55e9fa6414a0 2024-05-09T02:27:06.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.582+0000 7f9fc5174b80 1 -- 172.21.15.19:0/680399717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa6414a0 msgr2=0x55e9fa7d43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:06.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.582+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/680399717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa6414a0 0x55e9fa7d43a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00a8fc0 tx=0x7f9fa0010000 comp rx=0 tx=0).stop 2024-05-09T02:27:06.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.583+0000 7f9fc5174b80 1 -- 172.21.15.19:0/680399717 shutdown_connections 2024-05-09T02:27:06.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.583+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/680399717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa7be6a0 0x55e9fa7c0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.583+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/680399717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa6414a0 0x55e9fa7d43a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.583+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/680399717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa72b160 0x55e9fa7c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.583+0000 7f9fc5174b80 1 -- 172.21.15.19:0/680399717 >> 172.21.15.19:0/680399717 conn(0x55e9fa6a16a0 msgr2=0x55e9fa7c31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:06.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.583+0000 7f9fc5174b80 1 -- 172.21.15.19:0/680399717 shutdown_connections 2024-05-09T02:27:06.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.583+0000 7f9fc5174b80 1 -- 172.21.15.19:0/680399717 wait complete. 2024-05-09T02:27:06.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.583+0000 7f9fc5174b80 1 -- start start 2024-05-09T02:27:06.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.583+0000 7f9fc5174b80 1 Processor -- start 2024-05-09T02:27:06.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fc5174b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa6414a0 0x55e9fa7da7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fc5174b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa72b160 0x55e9fa7dad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fc5174b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa8405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fc5174b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb01b26f0 con 0x55e9fa7be6a0 2024-05-09T02:27:06.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fc5174b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb01b2020 con 0x55e9fa6414a0 2024-05-09T02:27:06.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fc5174b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb01af210 con 0x55e9fa72b160 2024-05-09T02:27:06.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fc5174b80 10 client.0 _ll_register_callbacks cb 0x55e9fa5526c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:27:06.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fb4cfb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa8405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fa7fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa72b160 0x55e9fa7dad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fb4cfb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa8405b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52700/0 (socket says 172.21.15.19:52700) 2024-05-09T02:27:06.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fb4cfb700 1 -- 172.21.15.19:0/3976200184 learned_addr learned my addr 172.21.15.19:0/3976200184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:06.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9faffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa6414a0 0x55e9fa7da7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fb4cfb700 1 -- 172.21.15.19:0/3976200184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa72b160 msgr2=0x55e9fa7dad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:06.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fb4cfb700 1 --2- 172.21.15.19:0/3976200184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa72b160 0x55e9fa7dad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fb4cfb700 1 -- 172.21.15.19:0/3976200184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa6414a0 msgr2=0x55e9fa7da7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:06.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fb4cfb700 1 --2- 172.21.15.19:0/3976200184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa6414a0 0x55e9fa7da7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fb4cfb700 1 -- 172.21.15.19:0/3976200184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9fa840af0 con 0x55e9fa7be6a0 2024-05-09T02:27:06.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9fb4cfb700 1 --2- 172.21.15.19:0/3976200184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa8405b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80092b0 tx=0x7f9fa801e620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d5e4772dbfa7b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:06.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9f8f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:27:06.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.584+0000 7f9f8f7fe700 1 -- 172.21.15.19:0/3976200184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8022040 con 0x55e9fa7be6a0 2024-05-09T02:27:06.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.585+0000 7f9fc5174b80 1 -- 172.21.15.19:0/3976200184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fa00c3070 con 0x55e9fa7be6a0 2024-05-09T02:27:06.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.585+0000 7f9f8f7fe700 1 -- 172.21.15.19:0/3976200184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f9fa801a040 con 0x55e9fa7be6a0 2024-05-09T02:27:06.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.585+0000 7f9f8f7fe700 1 -- 172.21.15.19:0/3976200184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f9fa8040050 con 0x55e9fa7be6a0 2024-05-09T02:27:06.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.585+0000 7f9fc5174b80 10 client.17253 fetch_fsmap learned FSMap version 59 2024-05-09T02:27:06.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.585+0000 7f9fc5174b80 1 -- 172.21.15.19:0/3976200184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55e9fa841cd0 con 0x55e9fa7be6a0 2024-05-09T02:27:06.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.585+0000 7f9f8f7fe700 1 -- 172.21.15.19:0/3976200184 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fa00c3070 con 0x55e9fa7be6a0 2024-05-09T02:27:06.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.585+0000 7f9f8f7fe700 1 -- 172.21.15.19:0/3976200184 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 59) ==== 8358+0+0 (secure 0 0 0) 0x7f9fa801eab0 con 0x55e9fa7be6a0 2024-05-09T02:27:06.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.585+0000 7f9fc5174b80 10 client.17253 fetch_fsmap finished waiting for FSMap version 59 2024-05-09T02:27:06.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.585+0000 7f9fc5174b80 10 client.17253 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T02:27:06.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.585+0000 7f9fc5174b80 20 client.17253 populate_metadata read hostname 'smithi019' 2024-05-09T02:27:06.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.585+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/3976200184 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e9fa8422b0 0x55e9fa8446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:06.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.585+0000 7f9fc5174b80 4 client.17253 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:27:06.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.585+0000 7f9fc5174b80 1 -- 172.21.15.19:0/3976200184 --> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9fa6e8a70 con 0x55e9fa8422b0 2024-05-09T02:27:06.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.585+0000 7f9faffff700 1 --2- 172.21.15.19:0/3976200184 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e9fa8422b0 0x55e9fa8446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:06.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.586+0000 7f9f8f7fe700 10 client.17253 ms_handle_connect on v2:172.21.15.59:6838/1206568409 2024-05-09T02:27:06.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.586+0000 7f9faffff700 1 --2- 172.21.15.19:0/3976200184 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e9fa8422b0 0x55e9fa8446d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d57a15c72e72cda2 server_cookie=51cce8bc0054d4d9 in_seq=0 out_seq=0 2024-05-09T02:27:06.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.586+0000 7f9f8f7fe700 1 -- 172.21.15.19:0/3976200184 <== mds.1 v2:172.21.15.59:6838/1206568409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9fa6e8a70 con 0x55e9fa8422b0 2024-05-09T02:27:06.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.586+0000 7f9f8f7fe700 10 client.17253 handle_command_reply: tid=0 2024-05-09T02:27:06.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.586+0000 7f9fc5174b80 1 client.17253 shutdown 2024-05-09T02:27:06.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.586+0000 7f9fc5174b80 1 -- 172.21.15.19:0/3976200184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 msgr2=0x55e9fa8405b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:06.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.586+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/3976200184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa8405b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80092b0 tx=0x7f9fa801e620 comp rx=0 tx=0).stop 2024-05-09T02:27:06.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.586+0000 7f9fc5174b80 1 -- 172.21.15.19:0/3976200184 shutdown_connections 2024-05-09T02:27:06.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.586+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/3976200184 >> [v2:172.21.15.59:6838/1206568409,v1:172.21.15.59:6839/1206568409] conn(0x55e9fa8422b0 0x55e9fa8446d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.586+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/3976200184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9fa7be6a0 0x55e9fa8405b0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.586+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/3976200184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e9fa6414a0 0x55e9fa7da7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.586+0000 7f9fc5174b80 1 --2- 172.21.15.19:0/3976200184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9fa72b160 0x55e9fa7dad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.586+0000 7f9fc5174b80 1 -- 172.21.15.19:0/3976200184 >> 172.21.15.19:0/3976200184 conn(0x55e9fa6a16a0 msgr2=0x55e9fa7c1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:06.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.586+0000 7f9f8f7fe700 0 client.17253 ms_handle_reset on v2:172.21.15.59:6838/1206568409 2024-05-09T02:27:06.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.586+0000 7f9fc5174b80 1 -- 172.21.15.19:0/3976200184 shutdown_connections 2024-05-09T02:27:06.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.587+0000 7f9fc5174b80 1 -- 172.21.15.19:0/3976200184 wait complete. 2024-05-09T02:27:06.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.587+0000 7f9fc5174b80 20 client.17253 trim_cache size 0 max 16384 2024-05-09T02:27:06.594 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:27:06.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.587+0000 7f9fb675e700 1 -- 172.21.15.19:0/60270420 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f9f9c0b2a70 msgr2=0x7f9f9c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:06.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.587+0000 7f9fb675e700 1 --2- 172.21.15.19:0/60270420 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f9f9c0b2a70 0x7f9f9c0b4f80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f98006010 tx=0x7f9f98005960 comp rx=0 tx=0).stop 2024-05-09T02:27:06.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.587+0000 7f9fb675e700 1 -- 172.21.15.19:0/60270420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb01039e0 msgr2=0x7f9fb0104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:06.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.587+0000 7f9fb675e700 1 --2- 172.21.15.19:0/60270420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb01039e0 0x7f9fb0104e40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000d950 tx=0x7f9fa000dc60 comp rx=0 tx=0).stop 2024-05-09T02:27:06.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.587+0000 7f9fb675e700 1 -- 172.21.15.19:0/60270420 shutdown_connections 2024-05-09T02:27:06.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.587+0000 7f9fb675e700 1 --2- 172.21.15.19:0/60270420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb01092d0 0x7f9fb01058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.587+0000 7f9fb675e700 1 --2- 172.21.15.19:0/60270420 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f9f9c0b2a70 0x7f9f9c0b4f80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.587+0000 7f9fb675e700 1 --2- 172.21.15.19:0/60270420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fb01039e0 0x7f9fb0104e40 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.587+0000 7f9fb675e700 1 --2- 172.21.15.19:0/60270420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fb0104330 0x7f9fb0105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:06.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.587+0000 7f9fb675e700 1 -- 172.21.15.19:0/60270420 >> 172.21.15.19:0/60270420 conn(0x7f9fb00fe520 msgr2=0x7f9fb01070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:06.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.587+0000 7f9fb675e700 1 -- 172.21.15.19:0/60270420 shutdown_connections 2024-05-09T02:27:06.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:06.587+0000 7f9fb675e700 1 -- 172.21.15.19:0/60270420 wait complete. 2024-05-09T02:27:06.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:06 smithi019 ceph-mon[30151]: pgmap v495: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 3.9 MiB/s wr, 372 op/s 2024-05-09T02:27:06.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:06 smithi196 ceph-mon[34718]: pgmap v495: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 3.9 MiB/s wr, 372 op/s 2024-05-09T02:27:06.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:06 smithi059 ceph-mon[34781]: pgmap v495: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 3.9 MiB/s wr, 372 op/s 2024-05-09T02:27:07.292 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T02:27:07.542 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:27:07.900 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:07 smithi019 ceph-mon[30151]: osdmap e101: 12 total, 12 up, 12 in 2024-05-09T02:27:07.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:07 smithi059 ceph-mon[34781]: osdmap e101: 12 total, 12 up, 12 in 2024-05-09T02:27:08.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:07 smithi196 ceph-mon[34718]: osdmap e101: 12 total, 12 up, 12 in 2024-05-09T02:27:08.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:08 smithi019 ceph-mon[30151]: pgmap v497: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 125 KiB/s wr, 40 op/s 2024-05-09T02:27:08.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:08 smithi059 ceph-mon[34781]: pgmap v497: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 125 KiB/s wr, 40 op/s 2024-05-09T02:27:09.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:08 smithi196 ceph-mon[34718]: pgmap v497: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 125 KiB/s wr, 40 op/s 2024-05-09T02:27:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.329+0000 7f0a615f8700 1 -- 172.21.15.19:0/3644673900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c107df0 msgr2=0x7f0a5c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.329+0000 7f0a615f8700 1 --2- 172.21.15.19:0/3644673900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c107df0 0x7f0a5c108200 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0a500099c0 tx=0x7f0a50009cd0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.329+0000 7f0a615f8700 1 -- 172.21.15.19:0/3644673900 shutdown_connections 2024-05-09T02:27:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.329+0000 7f0a615f8700 1 --2- 172.21.15.19:0/3644673900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a5c108740 0x7f0a5c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.329+0000 7f0a615f8700 1 --2- 172.21.15.19:0/3644673900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a5c1034c0 0x7f0a5c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.329+0000 7f0a615f8700 1 --2- 172.21.15.19:0/3644673900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c107df0 0x7f0a5c108200 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.329+0000 7f0a615f8700 1 -- 172.21.15.19:0/3644673900 >> 172.21.15.19:0/3644673900 conn(0x7f0a5c0fedc0 msgr2=0x7f0a5c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.329+0000 7f0a615f8700 1 -- 172.21.15.19:0/3644673900 shutdown_connections 2024-05-09T02:27:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.329+0000 7f0a615f8700 1 -- 172.21.15.19:0/3644673900 wait complete. 2024-05-09T02:27:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.330+0000 7f0a615f8700 1 Processor -- start 2024-05-09T02:27:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.330+0000 7f0a615f8700 1 -- start start 2024-05-09T02:27:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a615f8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a5c1034c0 0x7f0a5c0ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a615f8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a5c107df0 0x7f0a5c0ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a615f8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c108740 0x7f0a5c0ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a615f8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c113350 con 0x7f0a5c108740 2024-05-09T02:27:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a615f8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1131d0 con 0x7f0a5c107df0 2024-05-09T02:27:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a615f8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1134d0 con 0x7f0a5c1034c0 2024-05-09T02:27:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a5b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c108740 0x7f0a5c0ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a5b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c108740 0x7f0a5c0ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38586/0 (socket says 172.21.15.19:38586) 2024-05-09T02:27:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a5b7fe700 1 -- 172.21.15.19:0/512201373 learned_addr learned my addr 172.21.15.19:0/512201373 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a5affd700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a5c1034c0 0x7f0a5c0ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a5a7fc700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a5c107df0 0x7f0a5c0ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a5b7fe700 1 -- 172.21.15.19:0/512201373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a5c1034c0 msgr2=0x7f0a5c0ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a5b7fe700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a5c1034c0 0x7f0a5c0ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a5b7fe700 1 -- 172.21.15.19:0/512201373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a5c107df0 msgr2=0x7f0a5c0ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a5b7fe700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a5c107df0 0x7f0a5c0ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a5b7fe700 1 -- 172.21.15.19:0/512201373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5c1a9150 con 0x7f0a5c108740 2024-05-09T02:27:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a5a7fc700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a5c107df0 0x7f0a5c0ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:27:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a5affd700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a5c1034c0 0x7f0a5c0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:27:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.331+0000 7f0a5b7fe700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c108740 0x7f0a5c0ffd20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400e940 tx=0x7f0a5400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.332+0000 7f0a43fff700 1 -- 172.21.15.19:0/512201373 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5400cbd0 con 0x7f0a5c108740 2024-05-09T02:27:09.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.332+0000 7f0a43fff700 1 -- 172.21.15.19:0/512201373 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a5400cd70 con 0x7f0a5c108740 2024-05-09T02:27:09.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.332+0000 7f0a615f8700 1 -- 172.21.15.19:0/512201373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5c1a9440 con 0x7f0a5c108740 2024-05-09T02:27:09.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.332+0000 7f0a43fff700 1 -- 172.21.15.19:0/512201373 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54005680 con 0x7f0a5c108740 2024-05-09T02:27:09.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.332+0000 7f0a615f8700 1 -- 172.21.15.19:0/512201373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5c1a9980 con 0x7f0a5c108740 2024-05-09T02:27:09.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.334+0000 7f0a43fff700 1 -- 172.21.15.19:0/512201373 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0a54021440 con 0x7f0a5c108740 2024-05-09T02:27:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.335+0000 7f0a43fff700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0a440b2a70 0x7f0a440b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.336+0000 7f0a5affd700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0a440b2a70 0x7f0a440b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.336+0000 7f0a43fff700 1 -- 172.21.15.19:0/512201373 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8615+0+0 (secure 0 0 0) 0x7f0a540e2880 con 0x7f0a5c108740 2024-05-09T02:27:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.336+0000 7f0a5affd700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0a440b2a70 0x7f0a440b4f80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c009dd0 tx=0x7f0a4c009040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.336+0000 7f0a7000eb80 1 Processor -- start 2024-05-09T02:27:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.336+0000 7f0a7000eb80 1 -- start start 2024-05-09T02:27:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.336+0000 7f0a7000eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556cc3d00 0x557556dcc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.336+0000 7f0a7000eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556c494a0 0x557556dcc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.336+0000 7f0a7000eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556dc6400 0x557556dc6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a7000eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557556ce90a0 con 0x557556c494a0 2024-05-09T02:27:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a7000eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557556c8e9a0 con 0x557556cc3d00 2024-05-09T02:27:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a7000eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557556ce5060 con 0x557556dc6400 2024-05-09T02:27:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a5a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556c494a0 0x557556dcc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a5affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556cc3d00 0x557556dcc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a5a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556c494a0 0x557556dcc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38594/0 (socket says 172.21.15.19:38594) 2024-05-09T02:27:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a5a7fc700 1 -- 172.21.15.19:0/3600028672 learned_addr learned my addr 172.21.15.19:0/3600028672 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a5b7fe700 1 --2- 172.21.15.19:0/3600028672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556dc6400 0x557556dc6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a5a7fc700 1 -- 172.21.15.19:0/3600028672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556dc6400 msgr2=0x557556dc6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a5a7fc700 1 --2- 172.21.15.19:0/3600028672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556dc6400 0x557556dc6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a5a7fc700 1 -- 172.21.15.19:0/3600028672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556cc3d00 msgr2=0x557556dcc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a5a7fc700 1 --2- 172.21.15.19:0/3600028672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556cc3d00 0x557556dcc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a5a7fc700 1 -- 172.21.15.19:0/3600028672 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557556c7f500 con 0x557556c494a0 2024-05-09T02:27:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a5a7fc700 1 --2- 172.21.15.19:0/3600028672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556c494a0 0x557556dcc780 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0a50006010 tx=0x7f0a500036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db32d45d5f60acdc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a417fa700 1 -- 172.21.15.19:0/3600028672 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a500037c0 con 0x557556c494a0 2024-05-09T02:27:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a417fa700 1 -- 172.21.15.19:0/3600028672 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a50003e10 con 0x557556c494a0 2024-05-09T02:27:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.337+0000 7f0a417fa700 1 -- 172.21.15.19:0/3600028672 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5000b990 con 0x557556c494a0 2024-05-09T02:27:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.338+0000 7f0a7000eb80 1 -- 172.21.15.19:0/3600028672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556c494a0 msgr2=0x557556dcc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.338+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/3600028672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556c494a0 0x557556dcc780 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0a50006010 tx=0x7f0a500036a0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.338+0000 7f0a7000eb80 1 -- 172.21.15.19:0/3600028672 shutdown_connections 2024-05-09T02:27:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.338+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/3600028672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556dc6400 0x557556dc6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.338+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/3600028672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556cc3d00 0x557556dcc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.338+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/3600028672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556c494a0 0x557556dcc780 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.338+0000 7f0a7000eb80 1 -- 172.21.15.19:0/3600028672 >> 172.21.15.19:0/3600028672 conn(0x557556c4cf70 msgr2=0x557556dc1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.338+0000 7f0a7000eb80 1 -- 172.21.15.19:0/3600028672 shutdown_connections 2024-05-09T02:27:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.338+0000 7f0a7000eb80 1 -- 172.21.15.19:0/3600028672 wait complete. 2024-05-09T02:27:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.339+0000 7f0a7000eb80 1 -- start start 2024-05-09T02:27:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.339+0000 7f0a7000eb80 1 Processor -- start 2024-05-09T02:27:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.339+0000 7f0a7000eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556c494a0 0x557556dcffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.339+0000 7f0a7000eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556cc3d00 0x557556dd0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.339+0000 7f0a7000eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556dc6400 0x557556dd0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.339+0000 7f0a7000eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557556dc70a0 con 0x557556cc3d00 2024-05-09T02:27:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.339+0000 7f0a7000eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557556c83b20 con 0x557556c494a0 2024-05-09T02:27:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.339+0000 7f0a7000eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557556dc7220 con 0x557556dc6400 2024-05-09T02:27:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.339+0000 7f0a7000eb80 10 client.0 _ll_register_callbacks cb 0x557556b5a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:27:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.339+0000 7f0a5a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556cc3d00 0x557556dd0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.339+0000 7f0a5a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556cc3d00 0x557556dd0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38608/0 (socket says 172.21.15.19:38608) 2024-05-09T02:27:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.339+0000 7f0a5a7fc700 1 -- 172.21.15.19:0/3281854178 learned_addr learned my addr 172.21.15.19:0/3281854178 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a5b7fe700 1 --2- 172.21.15.19:0/3281854178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556dc6400 0x557556dd0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a5affd700 1 --2- 172.21.15.19:0/3281854178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556c494a0 0x557556dcffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a5a7fc700 1 -- 172.21.15.19:0/3281854178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556dc6400 msgr2=0x557556dd0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a5a7fc700 1 --2- 172.21.15.19:0/3281854178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556dc6400 0x557556dd0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a5a7fc700 1 -- 172.21.15.19:0/3281854178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556c494a0 msgr2=0x557556dcffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a5a7fc700 1 --2- 172.21.15.19:0/3281854178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556c494a0 0x557556dcffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a5a7fc700 1 -- 172.21.15.19:0/3281854178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557556b64490 con 0x557556cc3d00 2024-05-09T02:27:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a5b7fe700 1 --2- 172.21.15.19:0/3281854178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556dc6400 0x557556dd0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:27:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a5a7fc700 1 --2- 172.21.15.19:0/3281854178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556cc3d00 0x557556dd0500 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0a500059d0 tx=0x7f0a50003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dfa96d645dd392 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a3eff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:27:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a3eff5700 1 -- 172.21.15.19:0/3281854178 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a50005490 con 0x557556cc3d00 2024-05-09T02:27:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a3eff5700 1 -- 172.21.15.19:0/3281854178 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8615+0+0 (secure 0 0 0) 0x7f0a5000b9a0 con 0x557556cc3d00 2024-05-09T02:27:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a7000eb80 1 -- 172.21.15.19:0/3281854178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a50003e10 con 0x557556cc3d00 2024-05-09T02:27:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a3eff5700 1 -- 172.21.15.19:0/3281854178 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f0a50038050 con 0x557556cc3d00 2024-05-09T02:27:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a7000eb80 10 client.17265 fetch_fsmap learned FSMap version 59 2024-05-09T02:27:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.340+0000 7f0a7000eb80 1 -- 172.21.15.19:0/3281854178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x557556ddd8e0 con 0x557556cc3d00 2024-05-09T02:27:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.341+0000 7f0a3eff5700 1 -- 172.21.15.19:0/3281854178 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a50003e10 con 0x557556cc3d00 2024-05-09T02:27:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.341+0000 7f0a3eff5700 1 -- 172.21.15.19:0/3281854178 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 59) ==== 8358+0+0 (secure 0 0 0) 0x7f0a5002a9c0 con 0x557556cc3d00 2024-05-09T02:27:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.341+0000 7f0a7000eb80 10 client.17265 fetch_fsmap finished waiting for FSMap version 59 2024-05-09T02:27:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.341+0000 7f0a7000eb80 10 client.17265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:27:09.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.341+0000 7f0a7000eb80 20 client.17265 populate_metadata read hostname 'smithi019' 2024-05-09T02:27:09.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.341+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/3281854178 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557556ddfdb0 0x557556de21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.341+0000 7f0a7000eb80 4 client.17265 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T02:27:09.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.341+0000 7f0a7000eb80 1 -- 172.21.15.19:0/3281854178 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557556cf6a80 con 0x557556ddfdb0 2024-05-09T02:27:09.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.341+0000 7f0a5affd700 1 --2- 172.21.15.19:0/3281854178 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557556ddfdb0 0x557556de21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.342+0000 7f0a3eff5700 10 client.17265 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:27:09.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.342+0000 7f0a5affd700 1 --2- 172.21.15.19:0/3281854178 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557556ddfdb0 0x557556de21d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a84071221c87d39a server_cookie=e25a9586c8b6c6b2 in_seq=0 out_seq=0 2024-05-09T02:27:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.343+0000 7f0a3eff5700 1 -- 172.21.15.19:0/3281854178 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557556cf6a80 con 0x557556ddfdb0 2024-05-09T02:27:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.343+0000 7f0a3eff5700 10 client.17265 handle_command_reply: tid=0 2024-05-09T02:27:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.343+0000 7f0a7000eb80 1 client.17265 shutdown 2024-05-09T02:27:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.343+0000 7f0a7000eb80 1 -- 172.21.15.19:0/3281854178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556cc3d00 msgr2=0x557556dd0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.343+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/3281854178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556cc3d00 0x557556dd0500 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0a500059d0 tx=0x7f0a50003b20 comp rx=0 tx=0).stop 2024-05-09T02:27:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.343+0000 7f0a7000eb80 1 -- 172.21.15.19:0/3281854178 shutdown_connections 2024-05-09T02:27:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.343+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/3281854178 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557556ddfdb0 0x557556de21d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.343+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/3281854178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556dc6400 0x557556dd0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.343+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/3281854178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556c494a0 0x557556dcffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.343+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/3281854178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556cc3d00 0x557556dd0500 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.343+0000 7f0a7000eb80 1 -- 172.21.15.19:0/3281854178 >> 172.21.15.19:0/3281854178 conn(0x557556c4cf70 msgr2=0x557556dbaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.343+0000 7f0a3eff5700 0 client.17265 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:27:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.343+0000 7f0a7000eb80 1 -- 172.21.15.19:0/3281854178 shutdown_connections 2024-05-09T02:27:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.343+0000 7f0a7000eb80 1 -- 172.21.15.19:0/3281854178 wait complete. 2024-05-09T02:27:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.343+0000 7f0a7000eb80 20 client.17265 trim_cache size 0 max 16384 2024-05-09T02:27:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.358+0000 7f0a7000eb80 1 Processor -- start 2024-05-09T02:27:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.358+0000 7f0a7000eb80 1 -- start start 2024-05-09T02:27:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.358+0000 7f0a7000eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556c494a0 0x557556ddc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.358+0000 7f0a7000eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556cc3d00 0x557556dd0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.358+0000 7f0a7000eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556dc6400 0x557556dcefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.358+0000 7f0a7000eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1b1060 con 0x557556c494a0 2024-05-09T02:27:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.358+0000 7f0a7000eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1b2bd0 con 0x557556dc6400 2024-05-09T02:27:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.358+0000 7f0a7000eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1b1760 con 0x557556cc3d00 2024-05-09T02:27:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a5affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556c494a0 0x557556ddc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a5affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556c494a0 0x557556ddc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38612/0 (socket says 172.21.15.19:38612) 2024-05-09T02:27:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a5affd700 1 -- 172.21.15.19:0/591779220 learned_addr learned my addr 172.21.15.19:0/591779220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a5b7fe700 1 --2- 172.21.15.19:0/591779220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556dc6400 0x557556dcefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a5a7fc700 1 --2- 172.21.15.19:0/591779220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556cc3d00 0x557556dd0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a5affd700 1 -- 172.21.15.19:0/591779220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556cc3d00 msgr2=0x557556dd0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a5affd700 1 --2- 172.21.15.19:0/591779220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556cc3d00 0x557556dd0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a5affd700 1 -- 172.21.15.19:0/591779220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556dc6400 msgr2=0x557556dcefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a5affd700 1 --2- 172.21.15.19:0/591779220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556dc6400 0x557556dcefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a5affd700 1 -- 172.21.15.19:0/591779220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557556dbd9d0 con 0x557556c494a0 2024-05-09T02:27:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a5a7fc700 1 --2- 172.21.15.19:0/591779220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556cc3d00 0x557556dd0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:27:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a5b7fe700 1 --2- 172.21.15.19:0/591779220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556dc6400 0x557556dcefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:27:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a5affd700 1 --2- 172.21.15.19:0/591779220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556c494a0 0x557556ddc100 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c009370 tx=0x7f0a4c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866ddd0613de4b2b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a40ff9700 1 -- 172.21.15.19:0/591779220 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c02a290 con 0x557556c494a0 2024-05-09T02:27:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a40ff9700 1 -- 172.21.15.19:0/591779220 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a4c02a8e0 con 0x557556c494a0 2024-05-09T02:27:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.359+0000 7f0a40ff9700 1 -- 172.21.15.19:0/591779220 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c024cc0 con 0x557556c494a0 2024-05-09T02:27:09.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.360+0000 7f0a7000eb80 1 -- 172.21.15.19:0/591779220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556c494a0 msgr2=0x557556ddc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:09.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.360+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/591779220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556c494a0 0x557556ddc100 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c009370 tx=0x7f0a4c015000 comp rx=0 tx=0).stop 2024-05-09T02:27:09.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.360+0000 7f0a7000eb80 1 -- 172.21.15.19:0/591779220 shutdown_connections 2024-05-09T02:27:09.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.360+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/591779220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556dc6400 0x557556dcefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.360+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/591779220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556c494a0 0x557556ddc100 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.360+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/591779220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556cc3d00 0x557556dd0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.360+0000 7f0a7000eb80 1 -- 172.21.15.19:0/591779220 >> 172.21.15.19:0/591779220 conn(0x557556c4cf70 msgr2=0x557556dce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:09.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.360+0000 7f0a7000eb80 1 -- 172.21.15.19:0/591779220 shutdown_connections 2024-05-09T02:27:09.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.360+0000 7f0a7000eb80 1 -- 172.21.15.19:0/591779220 wait complete. 2024-05-09T02:27:09.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.361+0000 7f0a7000eb80 1 -- start start 2024-05-09T02:27:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.361+0000 7f0a7000eb80 1 Processor -- start 2024-05-09T02:27:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.361+0000 7f0a7000eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556c494a0 0x557556de2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.361+0000 7f0a7000eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556cc3d00 0x557556de2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.361+0000 7f0a7000eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556dc6400 0x557556e482f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.361+0000 7f0a7000eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1b2500 con 0x557556cc3d00 2024-05-09T02:27:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.361+0000 7f0a7000eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1b1e30 con 0x557556dc6400 2024-05-09T02:27:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.361+0000 7f0a7000eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1af020 con 0x557556c494a0 2024-05-09T02:27:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.361+0000 7f0a7000eb80 10 client.0 _ll_register_callbacks cb 0x557556b5a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T02:27:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.361+0000 7f0a5a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556cc3d00 0x557556de2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.361+0000 7f0a5a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556cc3d00 0x557556de2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38622/0 (socket says 172.21.15.19:38622) 2024-05-09T02:27:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.361+0000 7f0a5a7fc700 1 -- 172.21.15.19:0/1286399850 learned_addr learned my addr 172.21.15.19:0/1286399850 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.361+0000 7f0a5affd700 1 --2- 172.21.15.19:0/1286399850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556c494a0 0x557556de2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.361+0000 7f0a5b7fe700 1 --2- 172.21.15.19:0/1286399850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556dc6400 0x557556e482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a5a7fc700 1 -- 172.21.15.19:0/1286399850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556c494a0 msgr2=0x557556de2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a5a7fc700 1 --2- 172.21.15.19:0/1286399850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556c494a0 0x557556de2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a5a7fc700 1 -- 172.21.15.19:0/1286399850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556dc6400 msgr2=0x557556e482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T02:27:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a5a7fc700 1 --2- 172.21.15.19:0/1286399850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556dc6400 0x557556e482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a5a7fc700 1 -- 172.21.15.19:0/1286399850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557556dbacc0 con 0x557556cc3d00 2024-05-09T02:27:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a5b7fe700 1 --2- 172.21.15.19:0/1286399850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556dc6400 0x557556e482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:27:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a5a7fc700 1 --2- 172.21.15.19:0/1286399850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556cc3d00 0x557556de2540 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009d10 tx=0x7f0a50005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283890c6126fd7ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a3eff5700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-09T02:27:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a3eff5700 1 -- 172.21.15.19:0/1286399850 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a50005690 con 0x557556cc3d00 2024-05-09T02:27:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a3eff5700 1 -- 172.21.15.19:0/1286399850 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8615+0+0 (secure 0 0 0) 0x7f0a5000b9a0 con 0x557556cc3d00 2024-05-09T02:27:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a7000eb80 1 -- 172.21.15.19:0/1286399850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a4c02a8e0 con 0x557556cc3d00 2024-05-09T02:27:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a3eff5700 1 -- 172.21.15.19:0/1286399850 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f0a50049050 con 0x557556cc3d00 2024-05-09T02:27:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a7000eb80 10 client.17271 fetch_fsmap learned FSMap version 59 2024-05-09T02:27:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a7000eb80 1 -- 172.21.15.19:0/1286399850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x557556e496c0 con 0x557556cc3d00 2024-05-09T02:27:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a3eff5700 1 -- 172.21.15.19:0/1286399850 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a4c02a8e0 con 0x557556cc3d00 2024-05-09T02:27:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a3eff5700 1 -- 172.21.15.19:0/1286399850 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 59) ==== 8358+0+0 (secure 0 0 0) 0x7f0a50044030 con 0x557556cc3d00 2024-05-09T02:27:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.362+0000 7f0a7000eb80 10 client.17271 fetch_fsmap finished waiting for FSMap version 59 2024-05-09T02:27:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.363+0000 7f0a7000eb80 10 client.17271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T02:27:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.363+0000 7f0a7000eb80 20 client.17271 populate_metadata read hostname 'smithi019' 2024-05-09T02:27:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.363+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/1286399850 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557556e49dc0 0x557556e4c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.363+0000 7f0a7000eb80 4 client.17271 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T02:27:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.363+0000 7f0a7000eb80 1 -- 172.21.15.19:0/1286399850 --> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557556cf0b20 con 0x557556e49dc0 2024-05-09T02:27:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.363+0000 7f0a5affd700 1 --2- 172.21.15.19:0/1286399850 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557556e49dc0 0x557556e4c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.363+0000 7f0a5affd700 1 --2- 172.21.15.19:0/1286399850 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557556e49dc0 0x557556e4c1e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9b701aab4c47ca7 server_cookie=f4d0bfa3966bdd04 in_seq=0 out_seq=0 2024-05-09T02:27:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.363+0000 7f0a3eff5700 10 client.17271 ms_handle_connect on v2:172.21.15.196:6838/89436664 2024-05-09T02:27:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.364+0000 7f0a3eff5700 1 -- 172.21.15.19:0/1286399850 <== mds.0 v2:172.21.15.196:6838/89436664 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557556cf0b20 con 0x557556e49dc0 2024-05-09T02:27:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.364+0000 7f0a3eff5700 10 client.17271 handle_command_reply: tid=0 2024-05-09T02:27:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.364+0000 7f0a7000eb80 1 client.17271 shutdown 2024-05-09T02:27:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.364+0000 7f0a7000eb80 1 -- 172.21.15.19:0/1286399850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556cc3d00 msgr2=0x557556de2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.364+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/1286399850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556cc3d00 0x557556de2540 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009d10 tx=0x7f0a50005d50 comp rx=0 tx=0).stop 2024-05-09T02:27:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.364+0000 7f0a7000eb80 1 -- 172.21.15.19:0/1286399850 shutdown_connections 2024-05-09T02:27:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.364+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/1286399850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557556dc6400 0x557556e482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.364+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/1286399850 >> [v2:172.21.15.196:6838/89436664,v1:172.21.15.196:6839/89436664] conn(0x557556e49dc0 0x557556e4c1e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.364+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/1286399850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557556c494a0 0x557556de2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.364+0000 7f0a7000eb80 1 --2- 172.21.15.19:0/1286399850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557556cc3d00 0x557556de2540 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.364+0000 7f0a7000eb80 1 -- 172.21.15.19:0/1286399850 >> 172.21.15.19:0/1286399850 conn(0x557556c4cf70 msgr2=0x557556dc9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.364+0000 7f0a3eff5700 0 client.17271 ms_handle_reset on v2:172.21.15.196:6838/89436664 2024-05-09T02:27:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.364+0000 7f0a7000eb80 1 -- 172.21.15.19:0/1286399850 shutdown_connections 2024-05-09T02:27:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.364+0000 7f0a7000eb80 1 -- 172.21.15.19:0/1286399850 wait complete. 2024-05-09T02:27:09.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.364+0000 7f0a7000eb80 20 client.17271 trim_cache size 0 max 16384 2024-05-09T02:27:09.374 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-09T02:27:09.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.365+0000 7f0a615f8700 1 -- 172.21.15.19:0/512201373 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0a440b2a70 msgr2=0x7f0a440b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:09.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.365+0000 7f0a615f8700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0a440b2a70 0x7f0a440b4f80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c009dd0 tx=0x7f0a4c009040 comp rx=0 tx=0).stop 2024-05-09T02:27:09.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.365+0000 7f0a615f8700 1 -- 172.21.15.19:0/512201373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c108740 msgr2=0x7f0a5c0ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:09.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.365+0000 7f0a615f8700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c108740 0x7f0a5c0ffd20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400e940 tx=0x7f0a5400ed00 comp rx=0 tx=0).stop 2024-05-09T02:27:09.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.365+0000 7f0a615f8700 1 -- 172.21.15.19:0/512201373 shutdown_connections 2024-05-09T02:27:09.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.365+0000 7f0a615f8700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f0a440b2a70 0x7f0a440b4f80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.365+0000 7f0a615f8700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c108740 0x7f0a5c0ffd20 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.365+0000 7f0a615f8700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a5c1034c0 0x7f0a5c0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.365+0000 7f0a615f8700 1 --2- 172.21.15.19:0/512201373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a5c107df0 0x7f0a5c0ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:09.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.365+0000 7f0a615f8700 1 -- 172.21.15.19:0/512201373 >> 172.21.15.19:0/512201373 conn(0x7f0a5c0fedc0 msgr2=0x7f0a5c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:09.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.365+0000 7f0a615f8700 1 -- 172.21.15.19:0/512201373 shutdown_connections 2024-05-09T02:27:09.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:09.365+0000 7f0a615f8700 1 -- 172.21.15.19:0/512201373 wait complete. 2024-05-09T02:27:10.137 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-09T02:27:10.137 INFO:tasks.fwd_scrub:done joining 2024-05-09T02:27:10.138 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-09T02:27:10.149 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T02:27:10.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.266+0000 7f5abfbdc640 1 -- 172.21.15.19:0/378836614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab810b7d0 msgr2=0x7f5ab810dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:10.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.266+0000 7f5abfbdc640 1 --2- 172.21.15.19:0/378836614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab810b7d0 0x7f5ab810dbf0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009900 tx=0x7f5aa802a410 comp rx=0 tx=0).stop 2024-05-09T02:27:10.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.266+0000 7f5abfbdc640 1 -- 172.21.15.19:0/378836614 shutdown_connections 2024-05-09T02:27:10.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.266+0000 7f5abfbdc640 1 --2- 172.21.15.19:0/378836614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ab810e1c0 0x7f5ab81106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.266+0000 7f5abfbdc640 1 --2- 172.21.15.19:0/378836614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab810b7d0 0x7f5ab810dbf0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.266+0000 7f5abfbdc640 1 --2- 172.21.15.19:0/378836614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab8104d70 0x7f5ab8105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.266+0000 7f5abfbdc640 1 -- 172.21.15.19:0/378836614 >> 172.21.15.19:0/378836614 conn(0x7f5ab80782b0 msgr2=0x7f5ab80786f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:10.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.266+0000 7f5abfbdc640 1 -- 172.21.15.19:0/378836614 shutdown_connections 2024-05-09T02:27:10.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.266+0000 7f5abfbdc640 1 -- 172.21.15.19:0/378836614 wait complete. 2024-05-09T02:27:10.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.267+0000 7f5abfbdc640 1 Processor -- start 2024-05-09T02:27:10.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.267+0000 7f5abfbdc640 1 -- start start 2024-05-09T02:27:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.267+0000 7f5abfbdc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8104d70 0x7f5ab81a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.267+0000 7f5abfbdc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab810b7d0 0x7f5ab81a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.267+0000 7f5abfbdc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ab810e1c0 0x7f5ab819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.267+0000 7f5abfbdc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8113e20 con 0x7f5ab8104d70 2024-05-09T02:27:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.267+0000 7f5abfbdc640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8113ca0 con 0x7f5ab810e1c0 2024-05-09T02:27:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.267+0000 7f5abfbdc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8113fa0 con 0x7f5ab810b7d0 2024-05-09T02:27:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.267+0000 7f5abd978640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8104d70 0x7f5ab81a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.267+0000 7f5abd978640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8104d70 0x7f5ab81a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38634/0 (socket says 172.21.15.19:38634) 2024-05-09T02:27:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.267+0000 7f5abd978640 1 -- 172.21.15.19:0/2678155858 learned_addr learned my addr 172.21.15.19:0/2678155858 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.267+0000 7f5abd177640 1 --2- 172.21.15.19:0/2678155858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab810b7d0 0x7f5ab81a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.267+0000 7f5abe179640 1 --2- 172.21.15.19:0/2678155858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ab810e1c0 0x7f5ab819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.268+0000 7f5abd978640 1 -- 172.21.15.19:0/2678155858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab810b7d0 msgr2=0x7f5ab81a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.268+0000 7f5abd978640 1 --2- 172.21.15.19:0/2678155858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab810b7d0 0x7f5ab81a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.268+0000 7f5abd978640 1 -- 172.21.15.19:0/2678155858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ab810e1c0 msgr2=0x7f5ab819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.268+0000 7f5abd978640 1 --2- 172.21.15.19:0/2678155858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ab810e1c0 0x7f5ab819a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.268+0000 7f5abd978640 1 -- 172.21.15.19:0/2678155858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab819adb0 con 0x7f5ab8104d70 2024-05-09T02:27:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.268+0000 7f5abd177640 1 --2- 172.21.15.19:0/2678155858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab810b7d0 0x7f5ab81a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:27:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.268+0000 7f5abd978640 1 --2- 172.21.15.19:0/2678155858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8104d70 0x7f5ab81a0500 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400e9f0 tx=0x7f5ab400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.268+0000 7f5aaeffd640 1 -- 172.21.15.19:0/2678155858 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab400cd80 con 0x7f5ab8104d70 2024-05-09T02:27:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.268+0000 7f5abfbdc640 1 -- 172.21.15.19:0/2678155858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab819b040 con 0x7f5ab8104d70 2024-05-09T02:27:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.268+0000 7f5aaeffd640 1 -- 172.21.15.19:0/2678155858 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab40054d0 con 0x7f5ab8104d70 2024-05-09T02:27:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.268+0000 7f5aaeffd640 1 -- 172.21.15.19:0/2678155858 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4010690 con 0x7f5ab8104d70 2024-05-09T02:27:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.268+0000 7f5abfbdc640 1 -- 172.21.15.19:0/2678155858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab81a72a0 con 0x7f5ab8104d70 2024-05-09T02:27:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.269+0000 7f5abfbdc640 1 -- 172.21.15.19:0/2678155858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab80757e0 con 0x7f5ab8104d70 2024-05-09T02:27:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.271+0000 7f5aaeffd640 1 -- 172.21.15.19:0/2678155858 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5ab4004e30 con 0x7f5ab8104d70 2024-05-09T02:27:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.271+0000 7f5aaeffd640 1 --2- 172.21.15.19:0/2678155858 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5aa40b2960 0x7f5aa40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.272+0000 7f5abd177640 1 --2- 172.21.15.19:0/2678155858 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5aa40b2960 0x7f5aa40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.272+0000 7f5aaeffd640 1 -- 172.21.15.19:0/2678155858 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8615+0+0 (secure 0 0 0) 0x7f5ab40e2db0 con 0x7f5ab8104d70 2024-05-09T02:27:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.272+0000 7f5abd177640 1 --2- 172.21.15.19:0/2678155858 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5aa40b2960 0x7f5aa40b4e70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8031440 tx=0x7f5aa8009680 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:10.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.275+0000 7f5aaeffd640 1 -- 172.21.15.19:0/2678155858 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ab4054df0 con 0x7f5ab8104d70 2024-05-09T02:27:10.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.470+0000 7f5abfbdc640 1 -- 172.21.15.19:0/2678155858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ab81a7470 con 0x7f5ab8104d70 2024-05-09T02:27:10.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.472+0000 7f5aaeffd640 1 -- 172.21.15.19:0/2678155858 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+23234 (secure 0 0 0) 0x7f5ab40aa730 con 0x7f5ab8104d70 2024-05-09T02:27:10.475 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:27:10.475 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":101,"fsid":"a0611f3e-0da7-11ef-bc97-c7b262605968","created":"2024-05-09T01:59:27.591343+0000","modified":"2024-05-09T02:27:06.591175+0000","last_up_change":"2024-05-09T02:06:45.565065+0000","last_in_change":"2024-05-09T02:06:24.967186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T02:02:50.018959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T02:07:43.283868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T02:07:47.110614+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"996c6a3d-3e3a-451c-8789-36006e0b3dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6803","nonce":3002373576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6805","nonce":3002373576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6809","nonce":3002373576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6807","nonce":3002373576}]},"public_addr":"172.21.15.19:6803/3002373576","cluster_addr":"172.21.15.19:6805/3002373576","heartbeat_back_addr":"172.21.15.19:6809/3002373576","heartbeat_front_addr":"172.21.15.19:6807/3002373576","state":["exists","up"]},{"osd":1,"uuid":"1ac68d09-1198-4e81-86c7-e37f7c41aa5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6801","nonce":827458353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6803","nonce":827458353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6807","nonce":827458353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6805","nonce":827458353}]},"public_addr":"172.21.15.59:6801/827458353","cluster_addr":"172.21.15.59:6803/827458353","heartbeat_back_addr":"172.21.15.59:6807/827458353","heartbeat_front_addr":"172.21.15.59:6805/827458353","state":["exists","up"]},{"osd":2,"uuid":"818a0de1-04e6-471d-83df-a68833011b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6801","nonce":1659504639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6803","nonce":1659504639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6807","nonce":1659504639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6805","nonce":1659504639}]},"public_addr":"172.21.15.196:6801/1659504639","cluster_addr":"172.21.15.196:6803/1659504639","heartbeat_back_addr":"172.21.15.196:6807/1659504639","heartbeat_front_addr":"172.21.15.196:6805/1659504639","state":["exists","up"]},{"osd":3,"uuid":"7b204854-86b8-454d-aee0-8bb5775ce828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6811","nonce":2707671599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6813","nonce":2707671599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6817","nonce":2707671599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6815","nonce":2707671599}]},"public_addr":"172.21.15.19:6811/2707671599","cluster_addr":"172.21.15.19:6813/2707671599","heartbeat_back_addr":"172.21.15.19:6817/2707671599","heartbeat_front_addr":"172.21.15.19:6815/2707671599","state":["exists","up"]},{"osd":4,"uuid":"091667f1-e887-4a94-8723-6f85b676fef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6809","nonce":223037330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6811","nonce":223037330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6815","nonce":223037330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6813","nonce":223037330}]},"public_addr":"172.21.15.59:6809/223037330","cluster_addr":"172.21.15.59:6811/223037330","heartbeat_back_addr":"172.21.15.59:6815/223037330","heartbeat_front_addr":"172.21.15.59:6813/223037330","state":["exists","up"]},{"osd":5,"uuid":"2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6809","nonce":2516161898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6811","nonce":2516161898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6815","nonce":2516161898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6813","nonce":2516161898}]},"public_addr":"172.21.15.196:6809/2516161898","cluster_addr":"172.21.15.196:6811/2516161898","heartbeat_back_addr":"172.21.15.196:6815/2516161898","heartbeat_front_addr":"172.21.15.196:6813/2516161898","state":["exists","up"]},{"osd":6,"uuid":"4e352851-1279-475f-ba8b-eaf9ca73df74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6819","nonce":2150451637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6821","nonce":2150451637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6825","nonce":2150451637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6823","nonce":2150451637}]},"public_addr":"172.21.15.19:6819/2150451637","cluster_addr":"172.21.15.19:6821/2150451637","heartbeat_back_addr":"172.21.15.19:6825/2150451637","heartbeat_front_addr":"172.21.15.19:6823/2150451637","state":["exists","up"]},{"osd":7,"uuid":"83f53f59-9a60-4d87-9a8a-67e5c6f57004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6817","nonce":967902053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6819","nonce":967902053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6823","nonce":967902053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6821","nonce":967902053}]},"public_addr":"172.21.15.59:6817/967902053","cluster_addr":"172.21.15.59:6819/967902053","heartbeat_back_addr":"172.21.15.59:6823/967902053","heartbeat_front_addr":"172.21.15.59:6821/967902053","state":["exists","up"]},{"osd":8,"uuid":"a8f1bfb5-1cc8-4a65-8396-239bd781e1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6817","nonce":2946064259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6819","nonce":2946064259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6823","nonce":2946064259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6821","nonce":2946064259}]},"public_addr":"172.21.15.196:6817/2946064259","cluster_addr":"172.21.15.196:6819/2946064259","heartbeat_back_addr":"172.21.15.196:6823/2946064259","heartbeat_front_addr":"172.21.15.196:6821/2946064259","state":["exists","up"]},{"osd":9,"uuid":"73c58dc3-fcb0-4440-92b3-bf41a2e58fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6827","nonce":1735017886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6829","nonce":1735017886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6833","nonce":1735017886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6831","nonce":1735017886}]},"public_addr":"172.21.15.19:6827/1735017886","cluster_addr":"172.21.15.19:6829/1735017886","heartbeat_back_addr":"172.21.15.19:6833/1735017886","heartbeat_front_addr":"172.21.15.19:6831/1735017886","state":["exists","up"]},{"osd":10,"uuid":"8325282f-7325-4880-ab7d-b972717be9dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6825","nonce":2932899995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6827","nonce":2932899995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6831","nonce":2932899995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6829","nonce":2932899995}]},"public_addr":"172.21.15.59:6825/2932899995","cluster_addr":"172.21.15.59:6827/2932899995","heartbeat_back_addr":"172.21.15.59:6831/2932899995","heartbeat_front_addr":"172.21.15.59:6829/2932899995","state":["exists","up"]},{"osd":11,"uuid":"8b19228c-b286-43c8-a408-169782f13075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6825","nonce":2608023001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6827","nonce":2608023001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6831","nonce":2608023001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6829","nonce":2608023001}]},"public_addr":"172.21.15.196:6825/2608023001","cluster_addr":"172.21.15.196:6827/2608023001","heartbeat_back_addr":"172.21.15.196:6831/2608023001","heartbeat_front_addr":"172.21.15.196:6829/2608023001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:01:59.457748+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:23.640666+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:47.768243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:13.016746+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:39.060898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:04.508313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:31.366217+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:57.579758+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:23.177396+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:50.966399+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:16.838666+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:43.959387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3660764754":"2024-05-10T02:11:17.084815+0000","172.21.15.19:0/3573608809":"2024-05-10T02:11:17.084815+0000","172.21.15.19:0/3291838961":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1024133854":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1176455101":"2024-05-10T02:00:23.920230+0000","172.21.15.19:6801/2101052000":"2024-05-10T02:00:23.920230+0000","172.21.15.19:6801/972543870":"2024-05-10T02:11:17.084815+0000","172.21.15.19:0/4018987404":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/3079777324":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/972543870":"2024-05-10T02:11:17.084815+0000","172.21.15.19:6800/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/557070897":"2024-05-10T02:11:17.084815+0000","172.21.15.19:0/758334732":"2024-05-10T02:11:17.084815+0000","172.21.15.19:6801/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/179703528":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/2101052000":"2024-05-10T02:00:23.920230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T02:27:10.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.474+0000 7f5abfbdc640 1 -- 172.21.15.19:0/2678155858 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5aa40b2960 msgr2=0x7f5aa40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:10.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.474+0000 7f5abfbdc640 1 --2- 172.21.15.19:0/2678155858 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5aa40b2960 0x7f5aa40b4e70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8031440 tx=0x7f5aa8009680 comp rx=0 tx=0).stop 2024-05-09T02:27:10.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.474+0000 7f5abfbdc640 1 -- 172.21.15.19:0/2678155858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8104d70 msgr2=0x7f5ab81a0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:10.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.474+0000 7f5abfbdc640 1 --2- 172.21.15.19:0/2678155858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8104d70 0x7f5ab81a0500 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400e9f0 tx=0x7f5ab400edf0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.474+0000 7f5abfbdc640 1 -- 172.21.15.19:0/2678155858 shutdown_connections 2024-05-09T02:27:10.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.474+0000 7f5abfbdc640 1 --2- 172.21.15.19:0/2678155858 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f5aa40b2960 0x7f5aa40b4e70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.474+0000 7f5abfbdc640 1 --2- 172.21.15.19:0/2678155858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ab810e1c0 0x7f5ab819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.474+0000 7f5abfbdc640 1 --2- 172.21.15.19:0/2678155858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab810b7d0 0x7f5ab81a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.474+0000 7f5abfbdc640 1 --2- 172.21.15.19:0/2678155858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8104d70 0x7f5ab81a0500 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.474+0000 7f5abfbdc640 1 -- 172.21.15.19:0/2678155858 >> 172.21.15.19:0/2678155858 conn(0x7f5ab80782b0 msgr2=0x7f5ab81012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:10.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.474+0000 7f5abfbdc640 1 -- 172.21.15.19:0/2678155858 shutdown_connections 2024-05-09T02:27:10.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.474+0000 7f5abfbdc640 1 -- 172.21.15.19:0/2678155858 wait complete. 2024-05-09T02:27:10.489 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T02:02:50.018959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T02:07:43.283868+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T02:07:47.110614+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T02:27:10.490 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T02:27:10.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.606+0000 7f60dcb7d640 1 -- 172.21.15.19:0/3045990461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60d81032f0 msgr2=0x7f60d810f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:10.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.606+0000 7f60dcb7d640 1 --2- 172.21.15.19:0/3045990461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60d81032f0 0x7f60d810f670 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009ca0 tx=0x7f60d002a8c0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.606+0000 7f60dcb7d640 1 -- 172.21.15.19:0/3045990461 shutdown_connections 2024-05-09T02:27:10.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.606+0000 7f60dcb7d640 1 --2- 172.21.15.19:0/3045990461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60d81032f0 0x7f60d810f670 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.606+0000 7f60dcb7d640 1 --2- 172.21.15.19:0/3045990461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d8108ff0 0x7f60d8102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.606+0000 7f60dcb7d640 1 --2- 172.21.15.19:0/3045990461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60d8108b50 0x7f60d8102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.606+0000 7f60dcb7d640 1 -- 172.21.15.19:0/3045990461 >> 172.21.15.19:0/3045990461 conn(0x7f60d8078240 msgr2=0x7f60d8078680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:10.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.606+0000 7f60dcb7d640 1 -- 172.21.15.19:0/3045990461 shutdown_connections 2024-05-09T02:27:10.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.606+0000 7f60dcb7d640 1 -- 172.21.15.19:0/3045990461 wait complete. 2024-05-09T02:27:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.607+0000 7f60dcb7d640 1 Processor -- start 2024-05-09T02:27:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.607+0000 7f60dcb7d640 1 -- start start 2024-05-09T02:27:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.607+0000 7f60dcb7d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d81032f0 0x7f60d819b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.607+0000 7f60dcb7d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60d8108b50 0x7f60d819bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.607+0000 7f60dcb7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60d8108ff0 0x7f60d819c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.607+0000 7f60dcb7d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8112d40 con 0x7f60d8108ff0 2024-05-09T02:27:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.607+0000 7f60dcb7d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8112bc0 con 0x7f60d8108b50 2024-05-09T02:27:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.607+0000 7f60dcb7d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8112ec0 con 0x7f60d81032f0 2024-05-09T02:27:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.607+0000 7f60d6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60d8108ff0 0x7f60d819c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.607+0000 7f60d6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60d8108ff0 0x7f60d819c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38656/0 (socket says 172.21.15.19:38656) 2024-05-09T02:27:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.607+0000 7f60d6d9d640 1 -- 172.21.15.19:0/3001463893 learned_addr learned my addr 172.21.15.19:0/3001463893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.607+0000 7f60d5d9b640 1 --2- 172.21.15.19:0/3001463893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60d8108b50 0x7f60d819bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.607+0000 7f60d659c640 1 --2- 172.21.15.19:0/3001463893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d81032f0 0x7f60d819b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.608+0000 7f60d6d9d640 1 -- 172.21.15.19:0/3001463893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d81032f0 msgr2=0x7f60d819b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.608+0000 7f60d6d9d640 1 --2- 172.21.15.19:0/3001463893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d81032f0 0x7f60d819b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.608+0000 7f60d6d9d640 1 -- 172.21.15.19:0/3001463893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60d8108b50 msgr2=0x7f60d819bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.608+0000 7f60d6d9d640 1 --2- 172.21.15.19:0/3001463893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60d8108b50 0x7f60d819bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.608+0000 7f60d6d9d640 1 -- 172.21.15.19:0/3001463893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d81a6a30 con 0x7f60d8108ff0 2024-05-09T02:27:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.608+0000 7f60d6d9d640 1 --2- 172.21.15.19:0/3001463893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60d8108ff0 0x7f60d819c1a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f60d0005cc0 tx=0x7f60d002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.609+0000 7f60cf7fe640 1 -- 172.21.15.19:0/3001463893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0003bf0 con 0x7f60d8108ff0 2024-05-09T02:27:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.609+0000 7f60cf7fe640 1 -- 172.21.15.19:0/3001463893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60d0003d90 con 0x7f60d8108ff0 2024-05-09T02:27:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.609+0000 7f60cf7fe640 1 -- 172.21.15.19:0/3001463893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d002fbc0 con 0x7f60d8108ff0 2024-05-09T02:27:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.609+0000 7f60dcb7d640 1 -- 172.21.15.19:0/3001463893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d81a6cc0 con 0x7f60d8108ff0 2024-05-09T02:27:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.609+0000 7f60dcb7d640 1 -- 172.21.15.19:0/3001463893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d81a7200 con 0x7f60d8108ff0 2024-05-09T02:27:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.611+0000 7f60dcb7d640 1 -- 172.21.15.19:0/3001463893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d806a290 con 0x7f60d8108ff0 2024-05-09T02:27:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.612+0000 7f60cf7fe640 1 -- 172.21.15.19:0/3001463893 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f60d002fd60 con 0x7f60d8108ff0 2024-05-09T02:27:10.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.613+0000 7f60cf7fe640 1 --2- 172.21.15.19:0/3001463893 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f60c40b29e0 0x7f60c40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:10.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.613+0000 7f60d659c640 1 --2- 172.21.15.19:0/3001463893 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f60c40b29e0 0x7f60c40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:10.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.613+0000 7f60cf7fe640 1 -- 172.21.15.19:0/3001463893 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8615+0+0 (secure 0 0 0) 0x7f60d0103d20 con 0x7f60d8108ff0 2024-05-09T02:27:10.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.614+0000 7f60d659c640 1 --2- 172.21.15.19:0/3001463893 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f60c40b29e0 0x7f60c40b4ef0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f60c80058c0 tx=0x7f60c800a2d0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:10.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.617+0000 7f60cf7fe640 1 -- 172.21.15.19:0/3001463893 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f60d00cb7b0 con 0x7f60d8108ff0 2024-05-09T02:27:10.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.809+0000 7f60dcb7d640 1 -- 172.21.15.19:0/3001463893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f60d810ac60 con 0x7f60d8108ff0 2024-05-09T02:27:10.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.810+0000 7f60cf7fe640 1 -- 172.21.15.19:0/3001463893 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f60d00cb0b0 con 0x7f60d8108ff0 2024-05-09T02:27:10.813 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-05-09T02:27:10.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.812+0000 7f60dcb7d640 1 -- 172.21.15.19:0/3001463893 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f60c40b29e0 msgr2=0x7f60c40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:10.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.812+0000 7f60dcb7d640 1 --2- 172.21.15.19:0/3001463893 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f60c40b29e0 0x7f60c40b4ef0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f60c80058c0 tx=0x7f60c800a2d0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.812+0000 7f60dcb7d640 1 -- 172.21.15.19:0/3001463893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60d8108ff0 msgr2=0x7f60d819c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.812+0000 7f60dcb7d640 1 --2- 172.21.15.19:0/3001463893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60d8108ff0 0x7f60d819c1a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f60d0005cc0 tx=0x7f60d002f380 comp rx=0 tx=0).stop 2024-05-09T02:27:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.812+0000 7f60dcb7d640 1 -- 172.21.15.19:0/3001463893 shutdown_connections 2024-05-09T02:27:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.812+0000 7f60dcb7d640 1 --2- 172.21.15.19:0/3001463893 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f60c40b29e0 0x7f60c40b4ef0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.812+0000 7f60dcb7d640 1 --2- 172.21.15.19:0/3001463893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60d8108ff0 0x7f60d819c1a0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.812+0000 7f60dcb7d640 1 --2- 172.21.15.19:0/3001463893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60d8108b50 0x7f60d819bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.812+0000 7f60dcb7d640 1 --2- 172.21.15.19:0/3001463893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d81032f0 0x7f60d819b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.812+0000 7f60dcb7d640 1 -- 172.21.15.19:0/3001463893 >> 172.21.15.19:0/3001463893 conn(0x7f60d8078240 msgr2=0x7f60d80fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.812+0000 7f60dcb7d640 1 -- 172.21.15.19:0/3001463893 shutdown_connections 2024-05-09T02:27:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.812+0000 7f60dcb7d640 1 -- 172.21.15.19:0/3001463893 wait complete. 2024-05-09T02:27:10.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:10 smithi019 ceph-mon[30151]: pgmap v498: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 122 KiB/s wr, 35 op/s 2024-05-09T02:27:10.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:10 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2678155858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:27:10.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T02:27:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.942+0000 7f060abb7640 1 -- 172.21.15.19:0/3344907308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0604104400 msgr2=0x7f0604110780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.942+0000 7f060abb7640 1 --2- 172.21.15.19:0/3344907308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0604104400 0x7f0604110780 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009900 tx=0x7f05ec02a3f0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.942+0000 7f060abb7640 1 -- 172.21.15.19:0/3344907308 shutdown_connections 2024-05-09T02:27:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.942+0000 7f060abb7640 1 --2- 172.21.15.19:0/3344907308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0604104400 0x7f0604110780 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.942+0000 7f060abb7640 1 --2- 172.21.15.19:0/3344907308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f060410a100 0x7f0604103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.942+0000 7f060abb7640 1 --2- 172.21.15.19:0/3344907308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0604109c60 0x7f0604103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.942+0000 7f060abb7640 1 -- 172.21.15.19:0/3344907308 >> 172.21.15.19:0/3344907308 conn(0x7f06040793b0 msgr2=0x7f06040797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.942+0000 7f060abb7640 1 -- 172.21.15.19:0/3344907308 shutdown_connections 2024-05-09T02:27:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.942+0000 7f060abb7640 1 -- 172.21.15.19:0/3344907308 wait complete. 2024-05-09T02:27:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f060abb7640 1 Processor -- start 2024-05-09T02:27:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f060abb7640 1 -- start start 2024-05-09T02:27:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f060abb7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0604104400 0x7f06041a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f060abb7640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0604109c60 0x7f06041a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f060abb7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f060410a100 0x7f060419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f060abb7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0604113e50 con 0x7f0604104400 2024-05-09T02:27:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f060abb7640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0604113cd0 con 0x7f0604109c60 2024-05-09T02:27:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f060abb7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0604113fd0 con 0x7f060410a100 2024-05-09T02:27:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f0608953640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0604104400 0x7f06041a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f0608953640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0604104400 0x7f06041a2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38664/0 (socket says 172.21.15.19:38664) 2024-05-09T02:27:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f0608953640 1 -- 172.21.15.19:0/1423031339 learned_addr learned my addr 172.21.15.19:0/1423031339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f0609154640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f060410a100 0x7f060419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f0608953640 1 -- 172.21.15.19:0/1423031339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f060410a100 msgr2=0x7f060419c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f0608953640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f060410a100 0x7f060419c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f0603fff640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0604109c60 0x7f06041a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f0608953640 1 -- 172.21.15.19:0/1423031339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0604109c60 msgr2=0x7f06041a2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f0608953640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0604109c60 0x7f06041a2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f0608953640 1 -- 172.21.15.19:0/1423031339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060419cee0 con 0x7f0604104400 2024-05-09T02:27:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f0609154640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f060410a100 0x7f060419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T02:27:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.943+0000 7f0603fff640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0604109c60 0x7f06041a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T02:27:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.944+0000 7f0608953640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0604104400 0x7f06041a2640 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00c970 tx=0x7f05fc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.944+0000 7f0601ffb640 1 -- 172.21.15.19:0/1423031339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc014070 con 0x7f0604104400 2024-05-09T02:27:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.944+0000 7f0601ffb640 1 -- 172.21.15.19:0/1423031339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05fc004440 con 0x7f0604104400 2024-05-09T02:27:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.944+0000 7f0601ffb640 1 -- 172.21.15.19:0/1423031339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc003e50 con 0x7f0604104400 2024-05-09T02:27:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.944+0000 7f060abb7640 1 -- 172.21.15.19:0/1423031339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060419d1d0 con 0x7f0604104400 2024-05-09T02:27:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.944+0000 7f060abb7640 1 -- 172.21.15.19:0/1423031339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06041a9490 con 0x7f0604104400 2024-05-09T02:27:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.945+0000 7f060abb7640 1 -- 172.21.15.19:0/1423031339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0604104f40 con 0x7f0604104400 2024-05-09T02:27:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.947+0000 7f0601ffb640 1 -- 172.21.15.19:0/1423031339 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f05fc017070 con 0x7f0604104400 2024-05-09T02:27:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.947+0000 7f0601ffb640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f05f00b2b80 0x7f05f00b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.947+0000 7f0603fff640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f05f00b2b80 0x7f05f00b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.947+0000 7f0601ffb640 1 -- 172.21.15.19:0/1423031339 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8615+0+0 (secure 0 0 0) 0x7f05fc0e25b0 con 0x7f0604104400 2024-05-09T02:27:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.948+0000 7f0603fff640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f05f00b2b80 0x7f05f00b5090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f05f4005fd0 tx=0x7f05f40082f0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:10.952+0000 7f0601ffb640 1 -- 172.21.15.19:0/1423031339 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f05fc01e020 con 0x7f0604104400 2024-05-09T02:27:10.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:10 smithi059 ceph-mon[34781]: pgmap v498: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 122 KiB/s wr, 35 op/s 2024-05-09T02:27:10.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:10 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2678155858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:27:11.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:10 smithi196 ceph-mon[34718]: pgmap v498: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 122 KiB/s wr, 35 op/s 2024-05-09T02:27:11.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:10 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2678155858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:27:11.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.144+0000 7f060abb7640 1 -- 172.21.15.19:0/1423031339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f060406b0d0 con 0x7f0604104400 2024-05-09T02:27:11.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.145+0000 7f0601ffb640 1 -- 172.21.15.19:0/1423031339 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7f060406b0d0 con 0x7f0604104400 2024-05-09T02:27:11.148 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-05-09T02:27:11.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.147+0000 7f060abb7640 1 -- 172.21.15.19:0/1423031339 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f05f00b2b80 msgr2=0x7f05f00b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:11.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.147+0000 7f060abb7640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f05f00b2b80 0x7f05f00b5090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f05f4005fd0 tx=0x7f05f40082f0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.147+0000 7f060abb7640 1 -- 172.21.15.19:0/1423031339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0604104400 msgr2=0x7f06041a2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:11.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.147+0000 7f060abb7640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0604104400 0x7f06041a2640 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00c970 tx=0x7f05fc00ce30 comp rx=0 tx=0).stop 2024-05-09T02:27:11.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.147+0000 7f060abb7640 1 -- 172.21.15.19:0/1423031339 shutdown_connections 2024-05-09T02:27:11.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.147+0000 7f060abb7640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f05f00b2b80 0x7f05f00b5090 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.147+0000 7f060abb7640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f060410a100 0x7f060419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.147+0000 7f060abb7640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0604109c60 0x7f06041a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.147+0000 7f060abb7640 1 --2- 172.21.15.19:0/1423031339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0604104400 0x7f06041a2640 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.147+0000 7f060abb7640 1 -- 172.21.15.19:0/1423031339 >> 172.21.15.19:0/1423031339 conn(0x7f06040793b0 msgr2=0x7f0604076900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:11.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.147+0000 7f060abb7640 1 -- 172.21.15.19:0/1423031339 shutdown_connections 2024-05-09T02:27:11.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.147+0000 7f060abb7640 1 -- 172.21.15.19:0/1423031339 wait complete. 2024-05-09T02:27:11.162 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T02:27:11.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.278+0000 7f65f33a0640 1 -- 172.21.15.19:0/532966180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65ec109f20 msgr2=0x7f65ec103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:11.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.278+0000 7f65f33a0640 1 --2- 172.21.15.19:0/532966180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65ec109f20 0x7f65ec103bb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009900 tx=0x7f65dc02a410 comp rx=0 tx=0).stop 2024-05-09T02:27:11.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.278+0000 7f65f33a0640 1 -- 172.21.15.19:0/532966180 shutdown_connections 2024-05-09T02:27:11.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.278+0000 7f65f33a0640 1 --2- 172.21.15.19:0/532966180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65ec104220 0x7f65ec1105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.278+0000 7f65f33a0640 1 --2- 172.21.15.19:0/532966180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65ec109f20 0x7f65ec103bb0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.278+0000 7f65f33a0640 1 --2- 172.21.15.19:0/532966180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ec109a80 0x7f65ec103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.278+0000 7f65f33a0640 1 -- 172.21.15.19:0/532966180 >> 172.21.15.19:0/532966180 conn(0x7f65ec0ff2d0 msgr2=0x7f65ec1016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:11.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.278+0000 7f65f33a0640 1 -- 172.21.15.19:0/532966180 shutdown_connections 2024-05-09T02:27:11.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.278+0000 7f65f33a0640 1 -- 172.21.15.19:0/532966180 wait complete. 2024-05-09T02:27:11.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.278+0000 7f65f33a0640 1 Processor -- start 2024-05-09T02:27:11.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f33a0640 1 -- start start 2024-05-09T02:27:11.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f33a0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65ec104220 0x7f65ec19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:11.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f33a0640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ec109a80 0x7f65ec19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:11.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f33a0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65ec109f20 0x7f65ec19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:11.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f33a0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec113c70 con 0x7f65ec109f20 2024-05-09T02:27:11.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f33a0640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec113af0 con 0x7f65ec109a80 2024-05-09T02:27:11.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f33a0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec113df0 con 0x7f65ec104220 2024-05-09T02:27:11.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f2b9f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65ec109f20 0x7f65ec19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:11.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f2b9f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65ec109f20 0x7f65ec19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38672/0 (socket says 172.21.15.19:38672) 2024-05-09T02:27:11.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f2b9f640 1 -- 172.21.15.19:0/1388875243 learned_addr learned my addr 172.21.15.19:0/1388875243 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:11.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f239e640 1 --2- 172.21.15.19:0/1388875243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65ec104220 0x7f65ec19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:11.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f1b9d640 1 --2- 172.21.15.19:0/1388875243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ec109a80 0x7f65ec19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:11.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f2b9f640 1 -- 172.21.15.19:0/1388875243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65ec104220 msgr2=0x7f65ec19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:11.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f2b9f640 1 --2- 172.21.15.19:0/1388875243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65ec104220 0x7f65ec19a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f2b9f640 1 -- 172.21.15.19:0/1388875243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ec109a80 msgr2=0x7f65ec19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:11.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f2b9f640 1 --2- 172.21.15.19:0/1388875243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ec109a80 0x7f65ec19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.279+0000 7f65f2b9f640 1 -- 172.21.15.19:0/1388875243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec19f650 con 0x7f65ec109f20 2024-05-09T02:27:11.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.280+0000 7f65f2b9f640 1 --2- 172.21.15.19:0/1388875243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65ec109f20 0x7f65ec19f010 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f65e400eb10 tx=0x7f65e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:11.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.280+0000 7f65e37fe640 1 -- 172.21.15.19:0/1388875243 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4004b20 con 0x7f65ec109f20 2024-05-09T02:27:11.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.280+0000 7f65f33a0640 1 -- 172.21.15.19:0/1388875243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ec19f940 con 0x7f65ec109f20 2024-05-09T02:27:11.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.280+0000 7f65e37fe640 1 -- 172.21.15.19:0/1388875243 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e4004cc0 con 0x7f65ec109f20 2024-05-09T02:27:11.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.280+0000 7f65e37fe640 1 -- 172.21.15.19:0/1388875243 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4010690 con 0x7f65ec109f20 2024-05-09T02:27:11.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.280+0000 7f65f33a0640 1 -- 172.21.15.19:0/1388875243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ec1a72c0 con 0x7f65ec109f20 2024-05-09T02:27:11.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.281+0000 7f65f33a0640 1 -- 172.21.15.19:0/1388875243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65ec06a160 con 0x7f65ec109f20 2024-05-09T02:27:11.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.282+0000 7f65e37fe640 1 -- 172.21.15.19:0/1388875243 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f65e4005020 con 0x7f65ec109f20 2024-05-09T02:27:11.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.284+0000 7f65e37fe640 1 --2- 172.21.15.19:0/1388875243 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65d80b2950 0x7f65d80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:11.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.284+0000 7f65f239e640 1 --2- 172.21.15.19:0/1388875243 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65d80b2950 0x7f65d80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:11.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.284+0000 7f65e37fe640 1 -- 172.21.15.19:0/1388875243 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8615+0+0 (secure 0 0 0) 0x7f65e40e2f10 con 0x7f65ec109f20 2024-05-09T02:27:11.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.284+0000 7f65f239e640 1 --2- 172.21.15.19:0/1388875243 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65d80b2950 0x7f65d80b4e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f65ec07a280 tx=0x7f65e8005f90 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:11.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.288+0000 7f65e37fe640 1 -- 172.21.15.19:0/1388875243 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f65e4054df0 con 0x7f65ec109f20 2024-05-09T02:27:11.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.480+0000 7f65f33a0640 1 -- 172.21.15.19:0/1388875243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f65ec101e00 con 0x7f65ec109f20 2024-05-09T02:27:11.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.480+0000 7f65e37fe640 1 -- 172.21.15.19:0/1388875243 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7f65e40aa890 con 0x7f65ec109f20 2024-05-09T02:27:11.483 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-05-09T02:27:11.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.482+0000 7f65f33a0640 1 -- 172.21.15.19:0/1388875243 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65d80b2950 msgr2=0x7f65d80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:11.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.482+0000 7f65f33a0640 1 --2- 172.21.15.19:0/1388875243 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65d80b2950 0x7f65d80b4e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f65ec07a280 tx=0x7f65e8005f90 comp rx=0 tx=0).stop 2024-05-09T02:27:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.482+0000 7f65f33a0640 1 -- 172.21.15.19:0/1388875243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65ec109f20 msgr2=0x7f65ec19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.482+0000 7f65f33a0640 1 --2- 172.21.15.19:0/1388875243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65ec109f20 0x7f65ec19f010 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f65e400eb10 tx=0x7f65e400c2d0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.482+0000 7f65f33a0640 1 -- 172.21.15.19:0/1388875243 shutdown_connections 2024-05-09T02:27:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.482+0000 7f65f33a0640 1 --2- 172.21.15.19:0/1388875243 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f65d80b2950 0x7f65d80b4e60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.482+0000 7f65f33a0640 1 --2- 172.21.15.19:0/1388875243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65ec109f20 0x7f65ec19f010 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.482+0000 7f65f33a0640 1 --2- 172.21.15.19:0/1388875243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ec109a80 0x7f65ec19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.482+0000 7f65f33a0640 1 --2- 172.21.15.19:0/1388875243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65ec104220 0x7f65ec19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.482+0000 7f65f33a0640 1 -- 172.21.15.19:0/1388875243 >> 172.21.15.19:0/1388875243 conn(0x7f65ec0ff2d0 msgr2=0x7f65ec10fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.483+0000 7f65f33a0640 1 -- 172.21.15.19:0/1388875243 shutdown_connections 2024-05-09T02:27:11.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.483+0000 7f65f33a0640 1 -- 172.21.15.19:0/1388875243 wait complete. 2024-05-09T02:27:11.497 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-09T02:27:11.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.611+0000 7fc6a6eb8640 1 -- 172.21.15.19:0/1970680750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a0108fc0 msgr2=0x7fc6a0102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:11.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.611+0000 7fc6a6eb8640 1 --2- 172.21.15.19:0/1970680750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a0108fc0 0x7fc6a0102c00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc690009900 tx=0x7fc69002a410 comp rx=0 tx=0).stop 2024-05-09T02:27:11.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.611+0000 7fc6a6eb8640 1 -- 172.21.15.19:0/1970680750 shutdown_connections 2024-05-09T02:27:11.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.611+0000 7fc6a6eb8640 1 --2- 172.21.15.19:0/1970680750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6a0103270 0x7fc6a010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.611+0000 7fc6a6eb8640 1 --2- 172.21.15.19:0/1970680750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a0108fc0 0x7fc6a0102c00 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.611+0000 7fc6a6eb8640 1 --2- 172.21.15.19:0/1970680750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a0108b20 0x7fc6a01026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.611+0000 7fc6a6eb8640 1 -- 172.21.15.19:0/1970680750 >> 172.21.15.19:0/1970680750 conn(0x7fc6a0078240 msgr2=0x7fc6a0078680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:11.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.611+0000 7fc6a6eb8640 1 -- 172.21.15.19:0/1970680750 shutdown_connections 2024-05-09T02:27:11.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.611+0000 7fc6a6eb8640 1 -- 172.21.15.19:0/1970680750 wait complete. 2024-05-09T02:27:11.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.612+0000 7fc6a6eb8640 1 Processor -- start 2024-05-09T02:27:11.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.612+0000 7fc6a6eb8640 1 -- start start 2024-05-09T02:27:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.612+0000 7fc6a6eb8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6a0103270 0x7fc6a01a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.612+0000 7fc6a6eb8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a0108b20 0x7fc6a01a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.612+0000 7fc6a6eb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a0108fc0 0x7fc6a019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.612+0000 7fc6a6eb8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a0112d10 con 0x7fc6a0108fc0 2024-05-09T02:27:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.612+0000 7fc6a6eb8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a0112b90 con 0x7fc6a0108b20 2024-05-09T02:27:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.612+0000 7fc6a6eb8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a0112e90 con 0x7fc6a0103270 2024-05-09T02:27:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.613+0000 7fc6a5455640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a0108fc0 0x7fc6a019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.613+0000 7fc6a5455640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a0108fc0 0x7fc6a019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38678/0 (socket says 172.21.15.19:38678) 2024-05-09T02:27:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.613+0000 7fc6a5455640 1 -- 172.21.15.19:0/3202427488 learned_addr learned my addr 172.21.15.19:0/3202427488 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.613+0000 7fc697fff640 1 --2- 172.21.15.19:0/3202427488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a0108b20 0x7fc6a01a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.613+0000 7fc6a4c54640 1 --2- 172.21.15.19:0/3202427488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6a0103270 0x7fc6a01a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.613+0000 7fc6a5455640 1 -- 172.21.15.19:0/3202427488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6a0103270 msgr2=0x7fc6a01a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.613+0000 7fc6a5455640 1 --2- 172.21.15.19:0/3202427488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6a0103270 0x7fc6a01a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.613+0000 7fc6a5455640 1 -- 172.21.15.19:0/3202427488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a0108b20 msgr2=0x7fc6a01a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.613+0000 7fc6a5455640 1 --2- 172.21.15.19:0/3202427488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a0108b20 0x7fc6a01a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.613+0000 7fc6a5455640 1 -- 172.21.15.19:0/3202427488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a019adf0 con 0x7fc6a0108fc0 2024-05-09T02:27:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.613+0000 7fc6a5455640 1 --2- 172.21.15.19:0/3202427488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a0108fc0 0x7fc6a019a6c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc69800eb10 tx=0x7fc69800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.616+0000 7fc695ffb640 1 -- 172.21.15.19:0/3202427488 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698004b20 con 0x7fc6a0108fc0 2024-05-09T02:27:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.616+0000 7fc695ffb640 1 -- 172.21.15.19:0/3202427488 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc698004cc0 con 0x7fc6a0108fc0 2024-05-09T02:27:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.616+0000 7fc6a6eb8640 1 -- 172.21.15.19:0/3202427488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a019b080 con 0x7fc6a0108fc0 2024-05-09T02:27:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.616+0000 7fc695ffb640 1 -- 172.21.15.19:0/3202427488 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698010670 con 0x7fc6a0108fc0 2024-05-09T02:27:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.616+0000 7fc6a6eb8640 1 -- 172.21.15.19:0/3202427488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a01a72e0 con 0x7fc6a0108fc0 2024-05-09T02:27:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.617+0000 7fc6a6eb8640 1 -- 172.21.15.19:0/3202427488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a0103e00 con 0x7fc6a0108fc0 2024-05-09T02:27:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.620+0000 7fc695ffb640 1 -- 172.21.15.19:0/3202427488 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc698005000 con 0x7fc6a0108fc0 2024-05-09T02:27:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.620+0000 7fc695ffb640 1 --2- 172.21.15.19:0/3202427488 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc6880b29e0 0x7fc6880b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.620+0000 7fc6a4c54640 1 --2- 172.21.15.19:0/3202427488 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc6880b29e0 0x7fc6880b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.620+0000 7fc695ffb640 1 -- 172.21.15.19:0/3202427488 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8615+0+0 (secure 0 0 0) 0x7fc6980e2d30 con 0x7fc6a0108fc0 2024-05-09T02:27:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.621+0000 7fc6a4c54640 1 --2- 172.21.15.19:0/3202427488 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc6880b29e0 0x7fc6880b4ef0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0103800 tx=0x7fc69c005f90 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:11.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.624+0000 7fc695ffb640 1 -- 172.21.15.19:0/3202427488 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc698054df0 con 0x7fc6a0108fc0 2024-05-09T02:27:11.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:11 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3001463893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T02:27:11.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:11 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1423031339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T02:27:11.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:11 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1388875243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T02:27:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.876+0000 7fc6a6eb8640 1 -- 172.21.15.19:0/3202427488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fc6a01a74b0 con 0x7fc6a0108fc0 2024-05-09T02:27:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.885+0000 7fc695ffb640 1 -- 172.21.15.19:0/3202427488 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+816175 (secure 0 0 0) 0x7fc6980aa6b0 con 0x7fc6a0108fc0 2024-05-09T02:27:11.899 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:27:11.899 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:27:11.899 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 26, 2024-05-09T02:27:11.899 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.899 INFO:teuthology.orchestra.run.smithi019.stdout: "active_gid": 15222, 2024-05-09T02:27:11.899 INFO:teuthology.orchestra.run.smithi019.stdout: "active_name": "x", 2024-05-09T02:27:11.899 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addrs": { 2024-05-09T02:27:11.899 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-09T02:27:11.899 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:11.899 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-09T02:27:11.899 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2638922161 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2638922161 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addr": "172.21.15.19:6801/2638922161", 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: "active_change": "2024-05-09T02:11:17.085183+0000", 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: "active_mgr_features": 4540701547738038271, 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: "available": true, 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: "standbys": [ 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 15216, 2024-05-09T02:27:11.901 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "y", 2024-05-09T02:27:11.901 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-05-09T02:27:11.901 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-05-09T02:27:11.901 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:11.901 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-05-09T02:27:11.901 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:11.901 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:11.901 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:11.901 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T02:27:11.901 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T02:27:11.901 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:11.901 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.901 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T02:27:11.902 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.902 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.902 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.902 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T02:27:11.902 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.902 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.902 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.902 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.902 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:11.902 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:11.902 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.902 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.902 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.902 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.902 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.903 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:11.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:11.904 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.904 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.904 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.904 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:11.904 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.904 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.904 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.904 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:11.904 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.904 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.904 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.904 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:11.905 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:11.905 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.905 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.905 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.905 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:11.905 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.905 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:11.905 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:11.905 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:11.905 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:11.905 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:11.905 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:11.905 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:11.905 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.906 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:11.907 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.907 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.907 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.907 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.907 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-05-09T02:27:11.907 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-05-09T02:27:11.907 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.907 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.907 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.907 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.907 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.907 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.907 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.907 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-05-09T02:27:11.907 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.908 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.908 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.908 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-05-09T02:27:11.908 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-05-09T02:27:11.908 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.908 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.908 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.908 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-05-09T02:27:11.908 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.908 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.908 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.908 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-05-09T02:27:11.908 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.908 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.910 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-05-09T02:27:11.910 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-05-09T02:27:11.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.910 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.910 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.910 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.910 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.910 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.910 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.910 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-05-09T02:27:11.910 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.910 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.910 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.910 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.910 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-05-09T02:27:11.911 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.912 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.912 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.912 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.912 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.912 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.912 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.912 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-05-09T02:27:11.912 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.912 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.912 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.912 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-05-09T02:27:11.912 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-05-09T02:27:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T02:27:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-05-09T02:27:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-05-09T02:27:11.913 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-05-09T02:27:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:11.915 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.915 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:11.915 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.915 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.915 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:11.915 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.915 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.915 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.915 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:11.915 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:11.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.915 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:11.915 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-05-09T02:27:11.915 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:11.916 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:11.916 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:11.916 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-05-09T02:27:11.916 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-05-09T02:27:11.916 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.916 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.916 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.916 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:11.916 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.916 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.916 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.916 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T02:27:11.916 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.916 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.916 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.917 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-05-09T02:27:11.917 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-05-09T02:27:11.917 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.917 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.917 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.917 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-05-09T02:27:11.917 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.917 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.917 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.917 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T02:27:11.917 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T02:27:11.917 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.917 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-05-09T02:27:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-05-09T02:27:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T02:27:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-09T02:27:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-09T02:27:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T02:27:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T02:27:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.918 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.919 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-05-09T02:27:11.919 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-05-09T02:27:11.919 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T02:27:11.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.919 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.919 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-05-09T02:27:11.919 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T02:27:11.919 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-05-09T02:27:11.919 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.919 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T02:27:11.919 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.919 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.919 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.919 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-05-09T02:27:11.920 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-05-09T02:27:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-05-09T02:27:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-05-09T02:27:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-05-09T02:27:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T02:27:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-05-09T02:27:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-05-09T02:27:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.921 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.922 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.922 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-05-09T02:27:11.922 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.922 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.922 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.922 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T02:27:11.922 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T02:27:11.922 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.922 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.922 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-05-09T02:27:11.922 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-05-09T02:27:11.922 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T02:27:11.922 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.922 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.923 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:11.923 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-09T02:27:11.923 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-09T02:27:11.923 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.923 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T02:27:11.923 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T02:27:11.923 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.923 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.923 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:11.923 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:11.923 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.923 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.923 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.923 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.924 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.925 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:11.925 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:11.925 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.925 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.925 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.925 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:11.925 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.925 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.925 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.925 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:11.925 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.925 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.925 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.926 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:11.926 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:11.926 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.926 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.926 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.926 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:11.926 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.926 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.926 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:11.926 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:11.926 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:11.926 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:11.926 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:11.926 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:11.926 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:11.927 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:11.927 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:11.927 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.927 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.927 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.927 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:11.927 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:11.927 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.927 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.927 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.927 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:11.927 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.927 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.927 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.928 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:11.928 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.928 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.928 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.928 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-05-09T02:27:11.928 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-05-09T02:27:11.928 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:11.928 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.928 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.928 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:11.928 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.928 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.928 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.928 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T02:27:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-09T02:27:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-09T02:27:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-05-09T02:27:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-05-09T02:27:11.929 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-05-09T02:27:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-05-09T02:27:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-05-09T02:27:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-05-09T02:27:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-05-09T02:27:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-05-09T02:27:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-05-09T02:27:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.930 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T02:27:11.931 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.932 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.932 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.932 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T02:27:11.932 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.932 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.932 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.932 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:11.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:11.932 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:11.932 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:11.932 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.932 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:11.932 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.932 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.933 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.933 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:11.933 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.933 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.933 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.933 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-05-09T02:27:11.933 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-05-09T02:27:11.933 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:11.933 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.933 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.933 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T02:27:11.933 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T02:27:11.933 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.933 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.934 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T02:27:11.934 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T02:27:11.934 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.934 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.934 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.934 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-05-09T02:27:11.934 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-05-09T02:27:11.934 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T02:27:11.934 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.934 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.934 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:11.934 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.934 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.934 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.934 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.935 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.936 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-09T02:27:11.936 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.936 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.936 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.936 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T02:27:11.936 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.936 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.936 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.936 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-05-09T02:27:11.936 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-05-09T02:27:11.936 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:11.936 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-05-09T02:27:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T02:27:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-05-09T02:27:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-05-09T02:27:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.937 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.938 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-05-09T02:27:11.938 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.938 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.938 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.938 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T02:27:11.938 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.938 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.938 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.938 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-05-09T02:27:11.938 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-05-09T02:27:11.938 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.938 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.938 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:11.939 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.939 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.939 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.939 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T02:27:11.939 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T02:27:11.939 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.939 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.939 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-05-09T02:27:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-05-09T02:27:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T02:27:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T02:27:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-05-09T02:27:11.940 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T02:27:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-05-09T02:27:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T02:27:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-05-09T02:27:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-05-09T02:27:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-05-09T02:27:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-05-09T02:27:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-05-09T02:27:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T02:27:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-05-09T02:27:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-05-09T02:27:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T02:27:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.943 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-05-09T02:27:11.944 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-05-09T02:27:11.944 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.944 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.944 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.944 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:11.944 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.944 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.944 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.944 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T02:27:11.944 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.944 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.944 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.944 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.944 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-05-09T02:27:11.944 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.945 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.946 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.946 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T02:27:11.946 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.946 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.946 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.946 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T02:27:11.946 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.946 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.946 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.946 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-05-09T02:27:11.946 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-05-09T02:27:11.946 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.946 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T02:27:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-05-09T02:27:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-05-09T02:27:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-05-09T02:27:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.948 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T02:27:11.948 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.948 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.948 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.948 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-05-09T02:27:11.948 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.948 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.948 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.948 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.948 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-05-09T02:27:11.948 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-05-09T02:27:11.948 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.948 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.948 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.949 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T02:27:11.949 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.949 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.949 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.949 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T02:27:11.949 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.949 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.949 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.949 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.949 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-05-09T02:27:11.949 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-05-09T02:27:11.949 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.949 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.949 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.950 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T02:27:11.950 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.950 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.950 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.950 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-05-09T02:27:11.950 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.950 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.950 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.950 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.950 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-05-09T02:27:11.950 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-05-09T02:27:11.950 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.950 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.950 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.950 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T02:27:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-05-09T02:27:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-05-09T02:27:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-05-09T02:27:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T02:27:11.951 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-05-09T02:27:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-05-09T02:27:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-05-09T02:27:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T02:27:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.952 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.953 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-05-09T02:27:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-05-09T02:27:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-05-09T02:27:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T02:27:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-05-09T02:27:11.954 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.955 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.955 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.955 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-05-09T02:27:11.955 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-05-09T02:27:11.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.955 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.955 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.955 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T02:27:11.955 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.955 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.955 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.955 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T02:27:11.955 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.955 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-05-09T02:27:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-05-09T02:27:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T02:27:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-05-09T02:27:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-05-09T02:27:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-05-09T02:27:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T02:27:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-05-09T02:27:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-05-09T02:27:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-05-09T02:27:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T02:27:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-05-09T02:27:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-05-09T02:27:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-05-09T02:27:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.962 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T02:27:11.962 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.962 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.962 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.962 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T02:27:11.962 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.962 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.962 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.962 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-05-09T02:27:11.962 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T02:27:11.962 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:11.962 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.962 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-05-09T02:27:11.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.964 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.964 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.964 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T02:27:11.964 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.964 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.964 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.964 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-05-09T02:27:11.964 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-05-09T02:27:11.964 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:11.964 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.964 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.964 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-05-09T02:27:11.964 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.964 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.965 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-05-09T02:27:11.965 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.965 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.965 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.965 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-05-09T02:27:11.965 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-05-09T02:27:11.965 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.965 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.965 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.965 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:11.965 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.965 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.966 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T02:27:11.966 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.966 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.966 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.966 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-05-09T02:27:11.966 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-05-09T02:27:11.966 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:11.966 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.966 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.966 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T02:27:11.966 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.966 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.966 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.966 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-05-09T02:27:11.967 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.967 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.967 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.967 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-05-09T02:27:11.967 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-05-09T02:27:11.967 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.967 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.967 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.967 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T02:27:11.967 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.967 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.967 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T02:27:11.967 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.968 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.969 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-05-09T02:27:11.970 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-05-09T02:27:11.970 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.970 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.970 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.970 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:11.970 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.970 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.970 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.970 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T02:27:11.970 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.970 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.970 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-05-09T02:27:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-05-09T02:27:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T02:27:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T02:27:11.974 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.974 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.974 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.974 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:11.974 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:11.974 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.974 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.974 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.974 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:11.974 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.974 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.974 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:11.974 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:11.974 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.975 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:11.976 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.976 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.976 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.976 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:11.976 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.976 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.976 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.976 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T02:27:11.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T02:27:11.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.976 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.976 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.976 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:11.976 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.977 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.978 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T02:27:11.978 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.978 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.978 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-05-09T02:27:11.978 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-05-09T02:27:11.978 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:11.978 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.978 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:11.978 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.978 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.978 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.979 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-05-09T02:27:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-05-09T02:27:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-05-09T02:27:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.981 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-05-09T02:27:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-05-09T02:27:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-05-09T02:27:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T02:27:11.982 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.983 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.984 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-05-09T02:27:11.984 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-05-09T02:27:11.984 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.984 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.984 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.984 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:11.984 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.984 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.984 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.984 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T02:27:11.984 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.984 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.984 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-05-09T02:27:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-05-09T02:27:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T02:27:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-05-09T02:27:11.986 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-05-09T02:27:11.986 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.986 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.986 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.986 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.986 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.986 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.986 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.986 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T02:27:11.986 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.986 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.986 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.989 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-05-09T02:27:11.989 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-05-09T02:27:11.989 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.989 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.989 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.989 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.989 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.989 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.989 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.989 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T02:27:11.989 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.990 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.990 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.990 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-05-09T02:27:11.990 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-05-09T02:27:11.990 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.990 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.990 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.990 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:11.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.990 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.990 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.990 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T02:27:11.990 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.990 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.991 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.991 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-05-09T02:27:11.991 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-05-09T02:27:11.991 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:11.991 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.991 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.991 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-05-09T02:27:11.991 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.991 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.991 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.991 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-05-09T02:27:11.991 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.991 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.991 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-05-09T02:27:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-05-09T02:27:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T02:27:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.993 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-05-09T02:27:11.994 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-05-09T02:27:11.994 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.994 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.994 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.994 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:11.994 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.994 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.994 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.994 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T02:27:11.994 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.994 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.994 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.994 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-05-09T02:27:11.994 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-05-09T02:27:11.995 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.995 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.995 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:11.995 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.995 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.995 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.995 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T02:27:11.995 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.995 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.995 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.995 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-05-09T02:27:11.995 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-05-09T02:27:11.995 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.996 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.996 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.996 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:11.996 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.996 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.996 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.996 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T02:27:11.996 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.996 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.996 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-05-09T02:27:11.996 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-05-09T02:27:11.996 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.997 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.997 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:11.997 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.997 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.997 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.997 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T02:27:11.997 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.997 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.997 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.997 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-05-09T02:27:11.997 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-05-09T02:27:11.997 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:11.998 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.998 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:11.998 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:11.998 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.998 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.998 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:11.998 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T02:27:11.998 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:11.998 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:11.998 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:11.998 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:11.998 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:11.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:11.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:11.998 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-05-09T02:27:11.999 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:11.999 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:11.999 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:11.999 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:11.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:11.999 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:11.999 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:11.999 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:11.999 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:11.999 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:11.999 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:11.999 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:11.999 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:11.999 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.000 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.001 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.002 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.002 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.002 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.002 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.002 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.002 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.002 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.002 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.002 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.002 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.002 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.002 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.002 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.003 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.003 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.003 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.003 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.003 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.003 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.003 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.003 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.003 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.003 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.003 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.003 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.003 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-05-09T02:27:12.003 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-05-09T02:27:12.004 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.004 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.004 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.004 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-05-09T02:27:12.004 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.004 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.004 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.004 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T02:27:12.004 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.004 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.004 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.004 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.004 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.004 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.005 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.006 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-05-09T02:27:12.006 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.006 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.006 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.006 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T02:27:12.006 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.006 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.006 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.006 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.006 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.006 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-05-09T02:27:12.006 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.006 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.007 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T02:27:12.008 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T02:27:12.008 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.008 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.008 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.008 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.008 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.008 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.008 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.008 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.008 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.008 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.008 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.009 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T02:27:12.009 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T02:27:12.009 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.009 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.009 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.009 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.009 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.009 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.009 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.009 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.009 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.009 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.009 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.009 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-05-09T02:27:12.010 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T02:27:12.010 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.010 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.010 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.010 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.010 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.010 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.010 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.010 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.010 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.010 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.010 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.010 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T02:27:12.010 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T02:27:12.011 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.011 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.011 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.011 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.011 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.011 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.011 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.011 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.011 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.011 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.011 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.011 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T02:27:12.011 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T02:27:12.011 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.012 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.012 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.012 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.012 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.012 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.012 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.012 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.012 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.012 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.012 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T02:27:12.012 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T02:27:12.012 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.012 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T02:27:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T02:27:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.014 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.014 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.014 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.014 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.014 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.014 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.014 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.014 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.014 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T02:27:12.014 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T02:27:12.014 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.014 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.014 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.015 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.016 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.016 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.016 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.016 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.016 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.016 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.016 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T02:27:12.016 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T02:27:12.016 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.016 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.016 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.016 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.016 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.017 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.018 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.018 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.018 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.018 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T02:27:12.018 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T02:27:12.018 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.018 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.018 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.018 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.018 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.018 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.018 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.018 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.019 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.020 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.020 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T02:27:12.020 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T02:27:12.020 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.020 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.020 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.020 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T02:27:12.020 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.020 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.020 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.020 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.020 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.020 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.020 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.021 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T02:27:12.021 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T02:27:12.021 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.021 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.021 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.021 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.021 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.021 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.021 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.021 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.021 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.021 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-05-09T02:27:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-05-09T02:27:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.022 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.023 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T02:27:12.023 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T02:27:12.023 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.023 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.023 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.023 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T02:27:12.023 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.023 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.023 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.023 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.023 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.023 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.023 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.024 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T02:27:12.024 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T02:27:12.024 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.024 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.024 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.024 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.024 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.024 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.024 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.024 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.024 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.024 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.024 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.024 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T02:27:12.025 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T02:27:12.025 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.025 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.025 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.025 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.025 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.025 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.025 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.025 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.025 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.025 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.025 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.025 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.025 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T02:27:12.025 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T02:27:12.026 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.026 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.026 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.026 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.026 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.026 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.026 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.026 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.026 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.026 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.026 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.026 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T02:27:12.026 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T02:27:12.026 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T02:27:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T02:27:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.027 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-05-09T02:27:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T02:27:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T02:27:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.028 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.029 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.030 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.030 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.030 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.030 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.030 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.030 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.030 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T02:27:12.030 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T02:27:12.030 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.030 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.030 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.030 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.030 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.030 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T02:27:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T02:27:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.032 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.033 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.033 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.033 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T02:27:12.033 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T02:27:12.033 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.033 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.033 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.033 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.033 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.033 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.033 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.033 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.034 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.034 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.034 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.034 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T02:27:12.034 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T02:27:12.034 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.034 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.034 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.034 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.034 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.034 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.034 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.034 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.035 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.035 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T02:27:12.035 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T02:27:12.035 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.035 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.035 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.035 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.035 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.035 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.035 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.035 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.036 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.037 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T02:27:12.037 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T02:27:12.037 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.037 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.037 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.037 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.037 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.037 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.037 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.037 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.037 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.037 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.038 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T02:27:12.038 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T02:27:12.038 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.038 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.038 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.038 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.038 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.038 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.038 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.038 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.038 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.038 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.038 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T02:27:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T02:27:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T02:27:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.039 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T02:27:12.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T02:27:12.040 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.040 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.040 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.040 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.040 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.040 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.040 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.040 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.040 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.040 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T02:27:12.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T02:27:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-05-09T02:27:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T02:27:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T02:27:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-05-09T02:27:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T02:27:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T02:27:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.043 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.043 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.043 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.043 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.043 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.043 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.043 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.043 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.043 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.043 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.043 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T02:27:12.043 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T02:27:12.043 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.044 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.044 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.044 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T02:27:12.044 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.044 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.044 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.044 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.044 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.044 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.044 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.044 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T02:27:12.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T02:27:12.044 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.045 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.045 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.045 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.045 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.045 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.045 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.045 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.045 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.045 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.045 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.045 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T02:27:12.045 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T02:27:12.045 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.046 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.046 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.046 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.046 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.046 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.046 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.046 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.046 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.046 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.046 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.046 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T02:27:12.046 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T02:27:12.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.047 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.047 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.047 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.047 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.047 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.047 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.047 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.047 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.047 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.047 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.047 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T02:27:12.047 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T02:27:12.047 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.048 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.048 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.048 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.048 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.048 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.048 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.048 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.048 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.048 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T02:27:12.048 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T02:27:12.048 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T02:27:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T02:27:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.049 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.050 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T02:27:12.050 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.050 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.050 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.050 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.050 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.050 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.050 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.050 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-05-09T02:27:12.050 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-05-09T02:27:12.050 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.050 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.050 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.051 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.051 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.051 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.051 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.051 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.051 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.051 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.051 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.051 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-05-09T02:27:12.051 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-05-09T02:27:12.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.051 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.051 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.051 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.052 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.052 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.052 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.052 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.052 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.052 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.052 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.052 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-05-09T02:27:12.052 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-05-09T02:27:12.052 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.052 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.052 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.052 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.052 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-05-09T02:27:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-05-09T02:27:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-05-09T02:27:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-05-09T02:27:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.053 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.054 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.054 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.054 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.054 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.054 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-09T02:27:12.054 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-09T02:27:12.054 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.054 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.054 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.054 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.054 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.058 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.058 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.058 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.058 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.058 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.058 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.058 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.058 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.058 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.058 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.058 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.058 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.058 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.058 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.060 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-05-09T02:27:12.060 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-05-09T02:27:12.060 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.060 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.060 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.060 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.060 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.060 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.060 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.060 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-05-09T02:27:12.060 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.060 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.060 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.060 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-05-09T02:27:12.061 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T02:27:12.061 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.061 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.061 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.061 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.061 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.061 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.061 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.061 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.061 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.061 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.061 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T02:27:12.061 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T02:27:12.062 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.062 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.062 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.062 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-09T02:27:12.062 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.062 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.062 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.062 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.062 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.062 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.062 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.062 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T02:27:12.062 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T02:27:12.063 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.063 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.063 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.063 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-05-09T02:27:12.063 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.063 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.063 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.063 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.063 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.063 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.063 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.063 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.063 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.064 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.065 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.065 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.065 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.065 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.065 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.065 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.065 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.065 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-05-09T02:27:12.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-05-09T02:27:12.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.065 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.065 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.065 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-05-09T02:27:12.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.066 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.066 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.066 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.066 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.066 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.066 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-09T02:27:12.066 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-09T02:27:12.066 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.066 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.066 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.066 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-05-09T02:27:12.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.067 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.067 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.067 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.067 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-05-09T02:27:12.067 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.067 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.067 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.067 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.067 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.067 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-09T02:27:12.067 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-09T02:27:12.067 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.067 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.068 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.068 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-09T02:27:12.068 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-09T02:27:12.068 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-09T02:27:12.068 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.068 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.068 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.068 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.068 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.068 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.068 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-05-09T02:27:12.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-05-09T02:27:12.068 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.068 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.068 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.069 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-05-09T02:27:12.070 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-05-09T02:27:12.070 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.070 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.070 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.070 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.070 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.070 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.070 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-05-09T02:27:12.070 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.070 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.070 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.070 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.071 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.071 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.071 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.071 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.071 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.071 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.071 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.071 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.071 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.071 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.071 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.071 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.071 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.072 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.072 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.072 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.072 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.072 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.072 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.072 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.072 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.072 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.072 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.072 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.072 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.072 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.072 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.073 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.073 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.073 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.073 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.073 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.073 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.073 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.073 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.073 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.073 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.073 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.074 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.074 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.074 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.074 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.074 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.074 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.074 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.074 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.074 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.074 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.074 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.074 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.074 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-05-09T02:27:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-05-09T02:27:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.076 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.076 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.076 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-05-09T02:27:12.076 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.076 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.076 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.076 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T02:27:12.076 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.076 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.076 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.076 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-05-09T02:27:12.076 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-05-09T02:27:12.076 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.077 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.078 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-05-09T02:27:12.078 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.078 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.078 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.078 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-05-09T02:27:12.078 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.078 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.078 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.078 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-05-09T02:27:12.078 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-05-09T02:27:12.078 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.078 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.078 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-09T02:27:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T02:27:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-05-09T02:27:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-05-09T02:27:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.079 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.081 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.081 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T02:27:12.081 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.081 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.081 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.081 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.081 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.081 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.081 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.081 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.081 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.082 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.082 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.082 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.082 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-05-09T02:27:12.082 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-05-09T02:27:12.082 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.082 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.082 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.082 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-05-09T02:27:12.082 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.082 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.082 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T02:27:12.083 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.083 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.083 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.083 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.083 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.083 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.083 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-05-09T02:27:12.083 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.083 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.083 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.083 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.083 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.083 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.084 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.084 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.084 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.084 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.084 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.084 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.084 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.084 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.084 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.084 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.084 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.084 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.084 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.085 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.085 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.085 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.085 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.085 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.085 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.085 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.085 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.085 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.085 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.085 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.086 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.086 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.086 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.086 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.086 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.086 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.086 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.086 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.086 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.086 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.086 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.086 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.087 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.088 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.088 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.088 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.088 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.088 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.088 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.088 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.088 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.088 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.088 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-05-09T02:27:12.088 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-05-09T02:27:12.088 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.088 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-09T02:27:12.089 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.089 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.089 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.089 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.089 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.089 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.089 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.089 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-05-09T02:27:12.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-05-09T02:27:12.089 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.089 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.090 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-05-09T02:27:12.090 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.090 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.090 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.090 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.090 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.090 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.090 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T02:27:12.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T02:27:12.090 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.090 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.090 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.091 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T02:27:12.091 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.091 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.091 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.091 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.091 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.091 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.091 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.091 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.091 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.091 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.091 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.091 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.091 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.092 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.092 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.092 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.092 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.092 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.092 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.092 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.092 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.092 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.092 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.092 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-05-09T02:27:12.092 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-05-09T02:27:12.092 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-05-09T02:27:12.092 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-05-09T02:27:12.093 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-05-09T02:27:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-05-09T02:27:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T02:27:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-05-09T02:27:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-05-09T02:27:12.094 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.095 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.095 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.095 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.095 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.095 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.095 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.095 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-05-09T02:27:12.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.095 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.095 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.095 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.095 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T02:27:12.095 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T02:27:12.095 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.095 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-05-09T02:27:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-05-09T02:27:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T02:27:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.096 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.097 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.097 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.097 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.097 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.097 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.097 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.097 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.097 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.097 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.097 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.097 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.097 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.098 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.098 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.098 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.098 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.098 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.098 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.098 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.098 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.098 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.098 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.098 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.098 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.099 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.099 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.099 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.099 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.099 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.099 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.099 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.099 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.099 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.099 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.099 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.099 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-05-09T02:27:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-05-09T02:27:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.101 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.102 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.102 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.102 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-05-09T02:27:12.102 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.102 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.102 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.102 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-05-09T02:27:12.102 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-05-09T02:27:12.102 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.102 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.102 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.102 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-05-09T02:27:12.102 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.102 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.103 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.103 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-05-09T02:27:12.103 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.103 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.103 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.103 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.103 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.103 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.103 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.103 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.103 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T02:27:12.104 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-05-09T02:27:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-05-09T02:27:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T02:27:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T02:27:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-05-09T02:27:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T02:27:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.106 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.106 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-05-09T02:27:12.106 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-05-09T02:27:12.106 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.106 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.106 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.106 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.106 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.106 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.106 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.106 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-05-09T02:27:12.106 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.106 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.106 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.107 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.108 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-05-09T02:27:12.108 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.108 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.108 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.108 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.108 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.108 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.108 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.108 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.108 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.108 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.109 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.109 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.109 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.109 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.109 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.109 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.109 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.109 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.109 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.109 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.109 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.110 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.110 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.110 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.110 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.110 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.110 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.110 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.110 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.110 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.110 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.110 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.110 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.110 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.111 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.111 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.111 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.111 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.111 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.111 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.111 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.111 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.111 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.111 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.111 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.111 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.111 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.112 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.112 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.112 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.112 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.112 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.112 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.112 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.112 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.112 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.112 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.112 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.112 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.113 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.113 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.113 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.113 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.113 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.113 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.113 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.113 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.113 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.113 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.113 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.113 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-05-09T02:27:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.114 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.116 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.117 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.118 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.118 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.118 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.118 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.118 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.118 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.118 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.118 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.118 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.118 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.118 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.118 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.118 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.119 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.119 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.119 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.119 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.119 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.119 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.119 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.119 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.119 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.119 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-05-09T02:27:12.121 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.121 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.121 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.121 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T02:27:12.121 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.121 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.121 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-05-09T02:27:12.121 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-05-09T02:27:12.121 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.121 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.121 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.121 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.122 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-05-09T02:27:12.122 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.122 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.122 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T02:27:12.122 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.122 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.122 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.122 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.122 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.122 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.122 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.125 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.125 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.125 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.125 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.125 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.125 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.125 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.125 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.125 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.125 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.125 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.125 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.125 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.125 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.125 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.126 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.126 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.126 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.126 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.126 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.126 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.126 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.126 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.126 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.126 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.126 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.126 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.126 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.126 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.127 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.127 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.127 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.127 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.127 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.127 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.127 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.127 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.127 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.127 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.127 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.127 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.127 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.127 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-05-09T02:27:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-05-09T02:27:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-05-09T02:27:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.128 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.129 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.130 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.131 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.134 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.134 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-05-09T02:27:12.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-05-09T02:27:12.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.134 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.134 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.134 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.134 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.134 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-05-09T02:27:12.134 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T02:27:12.134 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.134 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.135 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-05-09T02:27:12.135 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-05-09T02:27:12.135 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.135 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.135 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.135 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-05-09T02:27:12.135 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.135 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.135 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.135 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-05-09T02:27:12.135 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.135 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-05-09T02:27:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-05-09T02:27:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-05-09T02:27:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T02:27:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.136 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.137 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-05-09T02:27:12.137 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-05-09T02:27:12.137 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.137 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.137 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.137 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.137 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.137 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.137 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-05-09T02:27:12.137 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.137 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.137 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.138 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.138 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.138 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.138 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.138 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.138 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.138 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.138 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.138 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.138 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.138 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.138 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.139 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-05-09T02:27:12.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-05-09T02:27:12.139 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.139 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.139 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.139 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-05-09T02:27:12.139 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.139 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.139 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.139 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T02:27:12.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T02:27:12.139 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.139 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.139 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.140 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.141 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.142 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.142 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.142 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.142 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.142 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.142 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.142 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.142 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.142 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.142 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.142 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.142 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.142 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.142 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.143 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.143 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.143 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.143 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.143 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.143 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.143 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.143 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.143 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.143 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.143 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.143 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.143 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.143 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.143 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-05-09T02:27:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.146 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.147 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.147 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.147 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.147 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.147 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.147 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.147 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.147 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.147 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.147 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.147 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.147 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.147 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.148 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.148 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.148 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.148 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.148 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.148 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.148 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.148 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.148 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.148 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.148 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.148 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.148 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.148 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.149 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.149 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.149 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.149 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.149 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.149 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.149 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.149 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.149 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.149 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.149 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.149 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.149 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.149 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.149 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-05-09T02:27:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.153 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.153 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.153 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.153 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.153 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.153 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.153 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.153 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.153 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.153 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.153 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.154 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.157 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.158 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-09T02:27:12.158 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.158 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.158 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.158 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-05-09T02:27:12.158 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-05-09T02:27:12.158 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.158 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.158 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.158 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.158 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.158 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.158 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T02:27:12.158 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.159 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.160 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.160 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.160 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.160 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.160 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.160 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.160 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.160 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.160 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.160 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.160 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.161 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.162 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.162 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.162 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.162 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.162 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.162 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.162 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.162 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.162 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.162 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.162 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.162 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.162 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.162 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-05-09T02:27:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-09T02:27:12.163 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.164 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.164 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.164 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.164 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.164 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.164 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.164 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-09T02:27:12.164 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-05-09T02:27:12.164 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-05-09T02:27:12.164 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.164 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-05-09T02:27:12.164 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.164 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.164 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.166 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.166 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-05-09T02:27:12.166 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.166 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.166 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.166 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.166 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.166 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.166 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.166 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.166 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.166 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.166 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.166 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.167 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.167 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.167 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.167 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.167 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.167 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.167 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.167 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.167 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.167 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.167 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.167 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.169 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.169 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.169 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.169 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.169 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.169 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.169 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.169 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.169 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.169 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.169 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.169 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.169 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.169 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.169 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.170 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.170 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.170 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.170 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.170 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.170 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.170 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.170 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.170 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.170 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.170 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.170 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.170 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.171 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.171 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.171 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.171 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.171 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.171 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.171 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.171 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.171 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.171 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.171 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.171 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.171 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.172 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.173 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.173 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.173 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.173 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.173 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.173 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.173 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.173 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.173 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.173 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.173 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.173 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.173 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.176 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.177 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.177 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.177 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.177 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.177 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.177 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.177 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.177 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.177 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.177 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.177 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.177 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.177 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.178 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.178 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.178 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-05-09T02:27:12.178 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.178 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.178 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.178 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.178 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.178 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.178 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.178 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.181 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.181 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.181 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.181 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.181 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.181 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.181 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.181 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.181 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.181 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.181 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.182 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.183 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T02:27:12.183 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T02:27:12.183 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.183 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.183 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.183 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T02:27:12.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.183 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.183 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.183 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-05-09T02:27:12.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-05-09T02:27:12.185 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:12.185 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.185 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-09T02:27:12.185 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-05-09T02:27:12.185 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.185 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.185 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-05-09T02:27:12.185 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T02:27:12.185 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.185 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.185 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.185 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.185 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.186 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-05-09T02:27:12.186 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.186 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.186 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.186 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-05-09T02:27:12.186 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-05-09T02:27:12.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.186 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.186 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.186 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.189 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.193 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.193 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-05-09T02:27:12.193 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.193 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.193 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.193 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-05-09T02:27:12.193 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.193 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.193 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.193 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T02:27:12.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T02:27:12.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.193 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.194 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.194 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T02:27:12.194 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.194 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.194 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.194 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-05-09T02:27:12.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.194 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.194 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.194 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.194 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.194 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.195 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.196 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-05-09T02:27:12.196 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-05-09T02:27:12.196 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.196 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.196 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.196 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.196 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.196 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.196 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.196 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.196 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.198 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.198 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.198 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.198 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.198 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.198 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.198 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.198 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.198 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.199 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.199 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.199 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.199 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.199 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.199 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.199 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.199 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.199 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.200 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.200 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.200 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.200 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.200 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.200 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.200 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.200 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.200 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.201 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.201 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.201 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.201 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.201 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.201 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.201 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.201 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.201 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.201 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.201 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.201 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.201 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.202 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.202 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.202 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.202 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.202 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.202 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.202 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.202 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.202 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-05-09T02:27:12.202 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-05-09T02:27:12.202 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.202 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T02:27:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T02:27:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.203 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.204 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-05-09T02:27:12.204 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.204 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.204 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.204 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.204 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.204 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.204 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-05-09T02:27:12.204 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-05-09T02:27:12.204 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:12.204 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.204 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.204 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-05-09T02:27:12.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.205 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.205 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.205 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.205 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.205 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.205 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T02:27:12.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T02:27:12.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.205 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-09T02:27:12.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.206 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.206 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.206 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T02:27:12.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.206 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.206 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.206 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T02:27:12.206 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T02:27:12.206 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.206 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.206 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-05-09T02:27:12.206 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.206 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.207 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-05-09T02:27:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-05-09T02:27:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-05-09T02:27:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.210 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.210 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.210 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.210 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-09T02:27:12.210 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-09T02:27:12.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.210 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.210 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.210 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-09T02:27:12.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-09T02:27:12.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-09T02:27:12.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.210 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.211 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.211 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.211 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.211 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.211 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-09T02:27:12.211 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.211 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.211 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.211 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.212 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.213 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.213 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.213 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.213 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.213 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.213 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.213 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.213 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.213 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.213 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.213 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.213 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.214 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.214 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.214 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.214 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.214 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.214 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.214 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.214 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.214 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.214 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.215 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.215 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.215 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.215 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.215 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.215 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.215 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.215 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.215 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.215 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.215 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.215 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.217 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.217 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.217 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-05-09T02:27:12.217 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T02:27:12.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.217 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.217 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.217 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.217 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.217 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.218 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.219 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.219 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-05-09T02:27:12.219 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-05-09T02:27:12.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.219 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.219 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.219 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.219 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.219 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.219 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.219 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.220 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.220 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.220 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-05-09T02:27:12.220 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.220 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.220 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.220 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-05-09T02:27:12.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-05-09T02:27:12.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.220 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.220 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-09T02:27:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-09T02:27:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.221 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.222 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.223 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.223 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.223 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.223 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.223 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.223 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.223 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.223 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.223 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.223 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.223 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.224 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.224 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.224 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.224 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.224 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.224 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.224 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.225 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.225 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.225 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.225 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.225 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.225 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.225 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.225 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.225 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.225 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.225 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.225 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.225 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.226 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.226 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.226 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.226 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.226 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.226 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.226 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.226 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.226 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.226 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.226 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.226 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.226 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.227 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T02:27:12.227 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T02:27:12.227 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.227 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.227 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.227 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.227 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.227 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.227 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.227 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.227 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T02:27:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T02:27:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.228 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.229 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.229 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.229 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.229 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.229 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.229 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.229 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.229 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.229 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.229 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.229 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-05-09T02:27:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.230 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.233 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.234 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.234 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.234 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.234 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.234 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.234 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.234 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.234 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.234 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.234 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.234 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.235 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.236 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.237 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.237 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.237 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.237 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.237 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.237 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.237 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.237 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.237 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.237 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.237 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.238 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.238 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.238 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.238 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.238 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.238 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.238 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.238 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.238 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.238 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.238 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.238 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.240 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.240 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.240 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.240 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.240 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.240 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.240 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.240 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.240 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.240 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.240 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-05-09T02:27:12.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-05-09T02:27:12.242 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.242 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.242 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.242 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-05-09T02:27:12.242 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.242 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.242 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T02:27:12.242 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.242 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.242 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.242 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.242 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.242 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.243 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.244 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.244 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.244 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.244 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.244 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.244 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.244 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.244 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.244 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.244 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.244 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.244 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.244 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.244 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.245 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.246 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.247 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.247 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.247 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.247 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.247 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.247 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.247 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.247 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.247 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.247 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.247 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-05-09T02:27:12.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-05-09T02:27:12.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.248 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.248 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.248 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.248 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.248 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.248 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.248 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.248 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-05-09T02:27:12.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-05-09T02:27:12.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-05-09T02:27:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-05-09T02:27:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-05-09T02:27:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.249 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.250 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.250 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.250 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.250 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.250 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.250 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.250 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.250 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.250 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-05-09T02:27:12.250 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-05-09T02:27:12.250 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.250 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-05-09T02:27:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-05-09T02:27:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-05-09T02:27:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-05-09T02:27:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.252 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.253 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.253 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.253 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.253 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-05-09T02:27:12.253 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-05-09T02:27:12.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.254 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.254 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.254 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.254 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.254 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.254 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.254 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-05-09T02:27:12.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-05-09T02:27:12.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.254 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.254 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.254 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.254 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-05-09T02:27:12.255 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.256 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.256 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.256 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.256 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.256 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.256 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.256 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.256 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.256 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.256 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-05-09T02:27:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-05-09T02:27:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.257 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.258 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.258 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-05-09T02:27:12.258 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-05-09T02:27:12.258 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.258 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.258 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.258 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.258 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.258 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.258 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.258 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.258 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.259 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smb", 2024-05-09T02:27:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.260 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.261 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.263 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.264 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.265 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "update_orchestration": { 2024-05-09T02:27:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "update_orchestration", 2024-05-09T02:27:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T02:27:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-09T02:27:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-05-09T02:27:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-05-09T02:27:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.267 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T02:27:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-05-09T02:27:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-05-09T02:27:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.268 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.269 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.270 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.271 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.273 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.274 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.275 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.276 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.276 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.276 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.276 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.276 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.276 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.276 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.276 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.276 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.276 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.276 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.276 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.276 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.277 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.277 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.277 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.277 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.277 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.277 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.277 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.277 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.277 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.277 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.278 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.278 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.278 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.278 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.278 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.278 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.278 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.278 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.278 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.278 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.278 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.278 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.278 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.278 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.280 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.280 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.280 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.280 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.280 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.280 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.280 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.280 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.280 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.280 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.280 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-05-09T02:27:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.281 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.282 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.282 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.282 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.282 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.282 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.282 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.282 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.282 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.282 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.282 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.282 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.282 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.282 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.283 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.283 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.283 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.283 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.283 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.283 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.283 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.283 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.283 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.283 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.283 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.284 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.286 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.286 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.286 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.286 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.286 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.286 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.286 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.286 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.286 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.286 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.286 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.286 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.286 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.286 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T02:27:12.287 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.288 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.288 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.288 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.288 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.288 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.288 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.288 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T02:27:12.288 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T02:27:12.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.288 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.288 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.288 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-05-09T02:27:12.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.288 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.289 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.289 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.289 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.289 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.289 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.289 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.289 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.289 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.289 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.289 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.289 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.290 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.290 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.290 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.290 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.290 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.290 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.290 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.290 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.290 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.290 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.290 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.291 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.291 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.291 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.291 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.291 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.291 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.291 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.291 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.291 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.292 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.293 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.293 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.293 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.293 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.293 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.293 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.293 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.293 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.293 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.293 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.294 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.294 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.294 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.294 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.294 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.294 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.294 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.294 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-05-09T02:27:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-05-09T02:27:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-05-09T02:27:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.295 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.296 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.296 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.296 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.296 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.296 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T02:27:12.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.296 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.296 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.296 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-05-09T02:27:12.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-05-09T02:27:12.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.296 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.297 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.297 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.297 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T02:27:12.297 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.297 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.297 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.297 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-05-09T02:27:12.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-05-09T02:27:12.297 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.297 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.297 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.297 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.298 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T02:27:12.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.298 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-05-09T02:27:12.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-05-09T02:27:12.298 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.298 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.298 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.298 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.298 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T02:27:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-05-09T02:27:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-05-09T02:27:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.300 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.301 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.301 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-05-09T02:27:12.301 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-05-09T02:27:12.301 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.301 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.301 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.301 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.301 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.301 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.301 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.301 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-05-09T02:27:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-05-09T02:27:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T02:27:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-09T02:27:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-09T02:27:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T02:27:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T02:27:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-05-09T02:27:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-05-09T02:27:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-05-09T02:27:12.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-05-09T02:27:12.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.305 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.305 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.305 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-05-09T02:27:12.305 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.305 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.305 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.305 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.305 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.305 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.305 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.305 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-05-09T02:27:12.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-05-09T02:27:12.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.306 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.306 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.306 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.306 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.306 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.306 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-05-09T02:27:12.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-05-09T02:27:12.307 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.307 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.307 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.307 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.307 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.307 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.307 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.307 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.307 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.307 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.307 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.308 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.309 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.310 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.310 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.310 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.310 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.310 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.310 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.310 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.310 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.310 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.310 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.311 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.311 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.311 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.311 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.311 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.311 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.311 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.311 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.311 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.311 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.311 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.311 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.312 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.313 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.313 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.313 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.313 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-05-09T02:27:12.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-05-09T02:27:12.313 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.313 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.313 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.313 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.313 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.313 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.313 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.313 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.313 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.314 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.314 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.314 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.314 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.314 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.314 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.314 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.314 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.314 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.314 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.314 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.314 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.315 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.315 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.315 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-05-09T02:27:12.315 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-05-09T02:27:12.315 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.315 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.315 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.315 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T02:27:12.315 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.315 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.315 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.315 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.315 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.316 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.316 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.316 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.316 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.316 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-05-09T02:27:12.316 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.316 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.316 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.316 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.316 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.316 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.318 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.318 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.318 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.318 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.318 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.318 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.318 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.318 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.318 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.318 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.318 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.318 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.318 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.318 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.319 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.319 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.319 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.319 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.319 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.319 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.319 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.319 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.319 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.319 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.319 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.319 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.319 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-09T02:27:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.322 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.323 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.323 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.323 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.323 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.323 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.323 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.323 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.323 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.323 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.323 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.323 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.323 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.323 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.324 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.324 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.324 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.324 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.324 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.324 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.324 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.324 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.324 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.324 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.324 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.324 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.325 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.326 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.326 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.326 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.326 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.326 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.326 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.326 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.326 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.326 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.326 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.326 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.326 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.327 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.327 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.327 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.327 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-05-09T02:27:12.327 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-05-09T02:27:12.327 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.327 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-05-09T02:27:12.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.327 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.327 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.327 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T02:27:12.328 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.328 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.328 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.328 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-05-09T02:27:12.328 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-05-09T02:27:12.328 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.328 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.328 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.328 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.328 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.328 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.328 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.328 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-05-09T02:27:12.328 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.329 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.329 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.329 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-05-09T02:27:12.329 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-05-09T02:27:12.329 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.329 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.329 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.329 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.329 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.329 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.329 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.329 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T02:27:12.329 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.329 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.330 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.330 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-05-09T02:27:12.330 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T02:27:12.330 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.330 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.330 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.330 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.330 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.330 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.330 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.330 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T02:27:12.330 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.330 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.331 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.331 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.331 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.331 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.331 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.331 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.331 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.331 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.331 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.331 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.331 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.331 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.331 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.331 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.332 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.332 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-05-09T02:27:12.332 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.332 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.332 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.332 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-05-09T02:27:12.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-05-09T02:27:12.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T02:27:12.332 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.332 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-05-09T02:27:12.332 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-05-09T02:27:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-05-09T02:27:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.333 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.334 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.334 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.334 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.334 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.334 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T02:27:12.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T02:27:12.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.334 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.334 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T02:27:12.334 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.334 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.334 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.335 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.335 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.335 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.335 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.335 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.335 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.335 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.335 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.335 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.335 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.335 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.336 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.336 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.336 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.336 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.336 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.336 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.336 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.336 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.336 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.336 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.336 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.336 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.339 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.340 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.340 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.340 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.340 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.340 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.340 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.340 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.340 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.340 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.340 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-05-09T02:27:12.340 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.341 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-05-09T02:27:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-05-09T02:27:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-05-09T02:27:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T02:27:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T02:27:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.343 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 24952, 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "z", 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.345 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.345 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T02:27:12.345 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.345 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.345 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.345 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.345 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.345 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.345 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.345 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.345 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.346 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.346 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.346 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.346 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.346 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.346 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.346 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.346 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.346 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.346 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.346 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.346 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.346 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.346 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.347 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.348 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.350 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-05-09T02:27:12.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-05-09T02:27:12.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.350 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.350 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.350 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.350 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.350 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.350 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-05-09T02:27:12.350 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.350 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.350 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-05-09T02:27:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-05-09T02:27:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-05-09T02:27:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-05-09T02:27:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-05-09T02:27:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-05-09T02:27:12.352 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.352 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.352 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.352 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-05-09T02:27:12.352 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.352 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.352 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.352 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-05-09T02:27:12.352 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-05-09T02:27:12.353 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.353 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.353 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.353 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.353 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.353 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.353 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.353 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-05-09T02:27:12.353 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.353 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-05-09T02:27:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-05-09T02:27:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-05-09T02:27:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-05-09T02:27:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-05-09T02:27:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-05-09T02:27:12.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.355 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.355 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.355 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.355 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.355 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.355 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.355 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-05-09T02:27:12.355 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.355 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.355 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.355 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-05-09T02:27:12.355 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-05-09T02:27:12.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.356 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.356 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.356 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.356 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.356 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T02:27:12.356 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.356 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.356 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.356 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-05-09T02:27:12.356 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-05-09T02:27:12.356 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.356 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-05-09T02:27:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.358 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.358 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.358 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.358 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.358 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.358 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.358 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.358 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.358 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.358 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-05-09T02:27:12.358 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.358 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.359 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.359 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-05-09T02:27:12.359 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-05-09T02:27:12.359 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.359 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.359 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.359 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T02:27:12.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.359 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.360 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-05-09T02:27:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-05-09T02:27:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T02:27:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-09T02:27:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-09T02:27:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T02:27:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T02:27:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-05-09T02:27:12.361 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-05-09T02:27:12.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T02:27:12.362 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.362 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-05-09T02:27:12.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T02:27:12.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-05-09T02:27:12.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.362 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T02:27:12.362 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.362 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.362 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.362 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-05-09T02:27:12.362 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-05-09T02:27:12.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.363 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T02:27:12.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.363 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.363 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.363 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T02:27:12.363 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T02:27:12.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.363 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.363 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-05-09T02:27:12.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-05-09T02:27:12.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:12.364 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.364 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-05-09T02:27:12.364 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-05-09T02:27:12.364 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-05-09T02:27:12.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.364 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-05-09T02:27:12.364 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T02:27:12.364 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.364 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.364 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-05-09T02:27:12.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-05-09T02:27:12.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.364 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.365 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-05-09T02:27:12.365 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.365 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.365 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.365 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T02:27:12.365 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T02:27:12.365 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.365 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-05-09T02:27:12.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-05-09T02:27:12.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T02:27:12.365 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.366 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-09T02:27:12.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-09T02:27:12.366 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T02:27:12.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T02:27:12.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.366 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.366 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.366 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.366 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.367 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.367 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.367 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.367 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.367 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.367 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.367 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.367 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.367 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.367 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.367 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.367 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.369 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.369 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.369 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.369 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.369 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.369 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.369 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.369 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.369 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.369 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.371 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.371 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.371 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.371 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.371 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-05-09T02:27:12.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-05-09T02:27:12.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:12.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.371 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-05-09T02:27:12.372 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-05-09T02:27:12.373 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-05-09T02:27:12.373 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-05-09T02:27:12.373 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-05-09T02:27:12.373 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.373 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-05-09T02:27:12.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.373 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.373 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-05-09T02:27:12.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-05-09T02:27:12.373 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.373 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T02:27:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T02:27:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T02:27:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.375 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.376 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.376 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.376 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.376 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.376 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-05-09T02:27:12.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-05-09T02:27:12.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.376 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T02:27:12.376 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T02:27:12.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.377 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.377 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T02:27:12.377 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T02:27:12.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.377 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-05-09T02:27:12.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-05-09T02:27:12.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T02:27:12.377 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.377 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.377 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.378 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.378 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T02:27:12.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.378 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.378 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.378 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.378 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-05-09T02:27:12.378 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.378 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.378 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.378 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-05-09T02:27:12.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-05-09T02:27:12.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:12.379 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-09T02:27:12.379 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.379 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.379 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.379 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T02:27:12.379 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.379 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.379 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.379 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-05-09T02:27:12.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-05-09T02:27:12.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-05-09T02:27:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T02:27:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-05-09T02:27:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-05-09T02:27:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-05-09T02:27:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T02:27:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-05-09T02:27:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-05-09T02:27:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T02:27:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T02:27:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-05-09T02:27:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-05-09T02:27:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T02:27:12.383 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.383 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.383 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T02:27:12.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.383 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-05-09T02:27:12.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T02:27:12.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:12.383 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.383 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.383 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-05-09T02:27:12.383 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T02:27:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-05-09T02:27:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-05-09T02:27:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.384 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.385 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-05-09T02:27:12.385 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-05-09T02:27:12.385 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-05-09T02:27:12.385 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.385 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T02:27:12.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.385 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-05-09T02:27:12.385 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-05-09T02:27:12.385 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.385 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.385 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.386 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.386 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.386 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T02:27:12.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.386 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.386 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.386 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-05-09T02:27:12.386 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-05-09T02:27:12.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.386 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.386 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.387 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.387 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.387 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.387 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T02:27:12.387 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.387 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-05-09T02:27:12.387 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-05-09T02:27:12.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.387 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.388 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.388 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T02:27:12.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.388 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.388 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-05-09T02:27:12.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-05-09T02:27:12.388 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.388 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.388 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.388 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T02:27:12.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.389 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.389 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.389 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T02:27:12.389 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.389 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.389 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.389 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-05-09T02:27:12.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-05-09T02:27:12.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.389 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T02:27:12.389 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.389 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.390 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-05-09T02:27:12.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.390 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.390 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-05-09T02:27:12.390 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-05-09T02:27:12.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.390 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.390 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.390 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T02:27:12.390 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.390 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.391 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T02:27:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-05-09T02:27:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-05-09T02:27:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T02:27:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-05-09T02:27:12.392 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.393 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.394 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-05-09T02:27:12.394 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-05-09T02:27:12.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.394 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.394 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T02:27:12.394 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.394 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.394 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-05-09T02:27:12.394 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.394 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.394 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-05-09T02:27:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-05-09T02:27:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T02:27:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-05-09T02:27:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-05-09T02:27:12.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-05-09T02:27:12.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.396 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T02:27:12.396 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.396 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.396 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.396 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-05-09T02:27:12.397 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.397 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-05-09T02:27:12.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-05-09T02:27:12.397 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.397 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.397 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.397 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T02:27:12.397 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-05-09T02:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-05-09T02:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-05-09T02:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T02:27:12.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.400 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-05-09T02:27:12.400 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.400 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.400 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.400 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-05-09T02:27:12.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-05-09T02:27:12.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.400 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.400 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T02:27:12.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.400 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T02:27:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-05-09T02:27:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-05-09T02:27:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T02:27:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-05-09T02:27:12.402 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.402 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.402 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-05-09T02:27:12.402 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-05-09T02:27:12.402 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.402 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.402 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.402 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T02:27:12.402 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.402 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.402 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-05-09T02:27:12.403 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.403 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.403 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.403 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-05-09T02:27:12.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-05-09T02:27:12.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.403 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.403 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T02:27:12.403 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.403 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.403 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-05-09T02:27:12.403 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.404 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.404 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.404 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-05-09T02:27:12.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-05-09T02:27:12.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.404 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.404 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.404 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.404 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.404 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.404 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T02:27:12.404 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.404 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-05-09T02:27:12.406 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T02:27:12.406 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.406 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-05-09T02:27:12.406 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.406 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.406 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.406 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T02:27:12.406 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.406 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.406 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-05-09T02:27:12.407 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-05-09T02:27:12.407 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.407 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.407 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.407 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-05-09T02:27:12.407 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.407 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.407 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.407 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T02:27:12.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.407 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.407 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.407 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-05-09T02:27:12.407 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-05-09T02:27:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-05-09T02:27:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-05-09T02:27:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-05-09T02:27:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-05-09T02:27:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T02:27:12.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.410 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.410 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-05-09T02:27:12.410 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.410 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.410 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.410 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-05-09T02:27:12.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-05-09T02:27:12.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.410 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T02:27:12.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.411 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T02:27:12.411 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.411 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.411 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.411 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-05-09T02:27:12.411 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-05-09T02:27:12.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.411 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.411 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T02:27:12.412 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.412 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.412 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.412 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-05-09T02:27:12.412 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.412 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.412 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.412 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-05-09T02:27:12.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-05-09T02:27:12.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.412 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.412 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.412 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T02:27:12.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.413 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.413 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-05-09T02:27:12.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-05-09T02:27:12.413 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.414 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.414 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T02:27:12.414 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.414 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.414 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.414 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.414 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.414 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.414 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.414 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.414 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.415 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.415 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.415 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.415 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.415 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.415 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.415 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.415 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-05-09T02:27:12.415 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-05-09T02:27:12.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.416 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.416 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.416 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.416 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.416 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.416 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T02:27:12.416 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.416 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.416 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.416 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T02:27:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.418 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.418 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.418 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.418 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.418 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.418 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.418 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.418 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.418 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.418 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.418 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.418 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.418 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.420 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T02:27:12.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T02:27:12.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.420 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.420 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.420 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.420 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T02:27:12.420 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.420 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.420 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-05-09T02:27:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-05-09T02:27:12.422 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.422 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.422 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.422 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.422 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.422 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.422 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-05-09T02:27:12.422 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.422 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.422 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.422 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-05-09T02:27:12.422 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-05-09T02:27:12.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.423 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.423 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.423 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.423 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.423 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.423 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T02:27:12.423 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.423 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.423 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.423 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-05-09T02:27:12.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-05-09T02:27:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-05-09T02:27:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-09T02:27:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-09T02:27:12.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-05-09T02:27:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-05-09T02:27:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-05-09T02:27:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T02:27:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.425 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-05-09T02:27:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-05-09T02:27:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-05-09T02:27:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T02:27:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-05-09T02:27:12.426 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-05-09T02:27:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T02:27:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T02:27:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-05-09T02:27:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-05-09T02:27:12.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.428 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.428 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.428 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.428 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T02:27:12.428 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.428 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.428 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.428 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-05-09T02:27:12.428 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-05-09T02:27:12.428 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.429 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T02:27:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-05-09T02:27:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-05-09T02:27:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.430 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T02:27:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-05-09T02:27:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-05-09T02:27:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.431 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.432 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-05-09T02:27:12.433 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.433 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.433 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.433 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.433 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.433 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.433 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.433 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.433 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.433 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-05-09T02:27:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-05-09T02:27:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T02:27:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.434 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.435 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-05-09T02:27:12.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-05-09T02:27:12.435 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.435 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.435 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.435 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.435 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.435 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.435 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.435 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T02:27:12.435 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.435 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.436 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-05-09T02:27:12.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-05-09T02:27:12.436 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.436 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.436 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.436 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.436 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T02:27:12.436 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.436 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.436 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-05-09T02:27:12.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-05-09T02:27:12.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.438 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.438 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.438 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.438 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.438 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T02:27:12.438 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.438 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.438 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.438 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-05-09T02:27:12.438 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-05-09T02:27:12.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T02:27:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-05-09T02:27:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.440 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.441 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.441 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.441 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.441 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.441 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.441 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.441 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.441 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.441 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.441 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.441 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.441 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.441 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.441 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.442 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.442 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.442 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.442 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.442 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.442 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.442 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.442 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.442 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.442 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.443 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.443 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.443 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.443 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.443 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.443 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.443 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.443 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.443 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.443 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.443 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.443 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.443 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-05-09T02:27:12.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.445 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.445 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.445 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-05-09T02:27:12.445 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.445 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.445 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T02:27:12.445 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.445 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.445 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.445 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.445 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.445 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.446 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.446 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.446 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.446 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.446 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.446 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.446 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.446 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.446 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.446 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-05-09T02:27:12.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-05-09T02:27:12.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-05-09T02:27:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T02:27:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.447 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-05-09T02:27:12.448 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.448 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.448 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.448 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T02:27:12.448 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T02:27:12.448 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.448 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.448 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.448 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.448 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.448 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.448 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.448 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.448 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.449 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.449 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T02:27:12.449 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T02:27:12.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.449 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.449 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.449 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.449 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.449 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.449 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.449 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.450 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.450 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T02:27:12.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T02:27:12.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.450 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.450 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.450 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.450 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.450 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.450 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.450 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.450 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.451 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.451 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-05-09T02:27:12.451 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T02:27:12.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.451 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.451 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.451 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.451 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.451 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.451 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.451 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.452 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T02:27:12.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T02:27:12.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.452 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.452 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.452 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.452 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.452 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.452 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.452 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.452 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T02:27:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T02:27:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.453 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T02:27:12.454 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T02:27:12.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.454 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.454 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.454 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.454 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.454 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.454 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.454 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.454 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.454 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.454 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.454 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T02:27:12.455 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T02:27:12.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.455 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.455 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.455 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.455 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.455 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.455 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.455 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.455 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.455 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.455 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T02:27:12.455 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T02:27:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T02:27:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T02:27:12.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.457 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.458 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.458 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.458 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.458 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.458 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.458 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.458 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.458 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.458 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T02:27:12.458 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T02:27:12.458 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.458 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.458 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.459 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.460 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.460 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.460 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.460 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.460 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.460 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T02:27:12.460 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T02:27:12.460 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.460 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.460 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.461 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.461 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.461 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.461 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.461 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.461 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.461 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T02:27:12.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T02:27:12.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.461 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.461 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.461 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T02:27:12.461 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.461 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.462 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.462 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.462 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.462 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T02:27:12.462 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T02:27:12.462 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.462 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.462 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.462 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.462 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.462 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.463 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.464 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.464 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.464 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.464 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T02:27:12.464 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T02:27:12.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.464 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.464 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T02:27:12.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.464 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.464 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.465 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.465 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.465 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T02:27:12.465 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T02:27:12.465 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.465 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.465 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.465 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.465 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.465 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.465 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T02:27:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T02:27:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.466 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.467 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.467 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T02:27:12.467 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T02:27:12.467 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.467 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.467 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.467 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.467 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.467 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.467 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.468 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T02:27:12.469 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T02:27:12.469 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.469 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.469 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-05-09T02:27:12.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.469 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.469 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.469 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.469 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.469 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.469 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T02:27:12.469 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T02:27:12.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.470 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.470 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.470 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-05-09T02:27:12.470 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.470 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.470 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.470 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.470 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T02:27:12.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T02:27:12.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.471 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.471 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.471 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.471 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.471 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.471 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.471 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.471 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.471 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.471 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T02:27:12.471 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T02:27:12.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.472 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.472 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.472 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.472 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.472 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.472 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.472 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.472 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T02:27:12.472 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T02:27:12.472 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.473 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.473 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.473 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.473 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.473 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.473 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.473 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.473 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.473 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.473 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T02:27:12.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T02:27:12.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.473 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.474 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.474 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.474 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.474 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.474 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T02:27:12.474 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T02:27:12.474 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.474 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.474 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.475 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.475 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.475 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.475 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.475 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.475 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.475 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T02:27:12.475 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T02:27:12.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.475 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.476 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.476 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.476 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.476 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.476 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.476 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T02:27:12.476 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T02:27:12.476 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.476 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.476 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.476 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T02:27:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T02:27:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.478 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T02:27:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T02:27:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.480 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.481 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T02:27:12.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T02:27:12.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.481 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.481 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.481 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.481 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.481 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.481 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.482 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T02:27:12.482 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T02:27:12.482 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.482 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.482 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.482 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-05-09T02:27:12.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.482 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.482 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.482 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.482 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.482 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.483 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T02:27:12.484 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T02:27:12.484 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.484 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.484 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.484 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.484 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.484 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.484 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.484 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.484 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.484 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.484 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.484 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T02:27:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T02:27:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T02:27:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T02:27:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T02:27:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T02:27:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T02:27:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.487 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T02:27:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T02:27:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.488 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.489 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.489 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.489 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.489 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.489 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.489 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.489 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.489 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T02:27:12.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T02:27:12.489 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.489 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.489 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.489 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.489 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.490 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.490 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.490 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.490 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.490 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.490 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T02:27:12.490 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T02:27:12.490 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.490 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.490 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.490 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T02:27:12.490 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.491 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-05-09T02:27:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-05-09T02:27:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.492 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.493 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.493 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.493 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.493 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-05-09T02:27:12.493 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-05-09T02:27:12.493 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.493 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.493 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.493 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.493 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.493 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-05-09T02:27:12.493 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.494 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.494 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.494 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-05-09T02:27:12.494 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-05-09T02:27:12.494 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.494 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.494 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.494 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-05-09T02:27:12.494 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.494 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.494 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.494 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.494 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.494 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.496 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.496 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.496 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.496 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.496 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.496 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.496 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.496 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.496 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.496 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.496 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.496 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.496 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.497 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.498 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.498 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.498 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.498 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.498 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.498 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.498 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.498 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.498 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.498 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.498 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.498 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.498 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.499 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.499 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.499 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.499 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.499 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.499 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.499 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.499 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.499 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.499 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.499 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.499 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.499 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-05-09T02:27:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-05-09T02:27:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.501 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.501 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.501 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.501 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.501 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-05-09T02:27:12.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.501 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.501 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-05-09T02:27:12.501 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T02:27:12.501 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.501 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.501 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.501 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.502 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.502 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.502 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.502 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.502 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.502 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T02:27:12.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T02:27:12.502 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.502 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.502 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.502 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-09T02:27:12.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.503 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.503 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.503 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.503 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.503 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.503 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.503 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T02:27:12.503 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T02:27:12.503 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.503 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.503 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.503 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-05-09T02:27:12.503 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.503 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.504 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.504 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.504 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.504 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.504 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.504 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.504 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.504 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.504 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.504 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.504 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.504 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.504 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-09T02:27:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-09T02:27:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.506 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.506 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.506 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.506 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-05-09T02:27:12.506 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-05-09T02:27:12.506 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.506 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.506 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.506 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-05-09T02:27:12.506 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.506 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.506 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.506 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.507 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.507 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-09T02:27:12.507 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-09T02:27:12.507 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.507 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-05-09T02:27:12.507 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.507 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.507 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.507 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.507 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-05-09T02:27:12.507 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-09T02:27:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-09T02:27:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-09T02:27:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-09T02:27:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-09T02:27:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.508 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.509 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.509 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.509 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.509 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-05-09T02:27:12.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-05-09T02:27:12.509 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.509 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.509 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.509 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.509 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.509 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.509 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.509 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.510 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.510 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.510 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.510 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.510 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.510 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.510 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-09T02:27:12.510 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.510 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.510 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.510 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-05-09T02:27:12.510 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-05-09T02:27:12.510 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.511 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.511 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.511 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.511 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.511 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.511 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.511 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-05-09T02:27:12.511 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.511 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.511 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.511 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.511 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.511 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.512 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.512 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.512 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.512 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.512 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.512 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.512 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.512 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.512 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.512 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.512 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.512 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.512 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.512 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.512 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.513 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.513 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.513 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.513 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.513 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.513 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.513 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.513 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.513 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.513 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.513 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.513 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.513 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.514 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.514 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.514 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.514 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.514 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.514 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.514 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.514 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.514 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.514 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.514 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.514 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.514 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.514 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.515 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-05-09T02:27:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-05-09T02:27:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-05-09T02:27:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T02:27:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-05-09T02:27:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-05-09T02:27:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-05-09T02:27:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T02:27:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-05-09T02:27:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-05-09T02:27:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-05-09T02:27:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-05-09T02:27:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.520 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-05-09T02:27:12.520 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-05-09T02:27:12.520 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.520 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.520 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.520 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.520 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.520 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.520 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.520 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T02:27:12.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.520 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.520 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T02:27:12.521 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T02:27:12.521 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.521 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.521 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.521 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T02:27:12.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.521 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.521 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.521 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T02:27:12.521 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.521 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.521 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.521 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.522 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.522 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.522 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.522 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.522 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.522 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.522 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.522 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.522 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.522 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.522 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.522 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.523 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-05-09T02:27:12.523 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-05-09T02:27:12.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.523 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.523 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.523 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-05-09T02:27:12.523 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.523 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.523 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.523 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T02:27:12.523 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.523 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.523 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.523 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.523 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.524 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-05-09T02:27:12.524 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.524 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.524 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.524 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.524 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.524 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.524 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.524 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.524 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.524 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.524 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.526 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.526 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.526 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.526 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.526 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.526 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.526 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.526 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.526 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.526 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.526 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.526 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.526 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.527 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.527 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.527 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.527 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.527 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.527 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.527 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.527 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.527 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.527 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.527 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.527 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.527 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.528 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.528 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.528 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.528 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.528 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.528 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.528 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.528 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.528 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.528 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.528 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.528 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.528 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.528 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.529 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-05-09T02:27:12.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-05-09T02:27:12.529 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.529 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.529 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.529 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-09T02:27:12.529 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.529 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.529 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.529 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.529 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.529 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.529 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.530 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-05-09T02:27:12.530 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-05-09T02:27:12.530 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.530 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.530 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.530 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-05-09T02:27:12.530 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.530 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.530 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.530 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.530 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.530 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.532 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.532 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.532 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.532 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.532 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.532 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.532 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.532 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.532 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.532 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.532 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.532 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.532 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.532 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-05-09T02:27:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-05-09T02:27:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-05-09T02:27:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-05-09T02:27:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-05-09T02:27:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-05-09T02:27:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T02:27:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.534 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.534 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.534 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-05-09T02:27:12.534 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-05-09T02:27:12.534 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.534 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.534 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.534 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-05-09T02:27:12.534 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.534 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.534 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T02:27:12.534 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.536 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T02:27:12.536 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T02:27:12.536 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.536 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.536 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.536 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-05-09T02:27:12.536 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-05-09T02:27:12.536 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.536 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T02:27:12.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.536 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.536 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.537 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.538 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.538 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.538 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.538 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.538 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.538 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.538 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.538 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.538 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.538 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.538 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.538 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.538 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.539 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-05-09T02:27:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-05-09T02:27:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.542 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.542 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.542 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-05-09T02:27:12.542 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.542 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.542 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.542 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-05-09T02:27:12.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-05-09T02:27:12.542 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.542 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.542 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.542 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-05-09T02:27:12.542 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.543 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T02:27:12.544 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.545 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.545 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.545 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-05-09T02:27:12.545 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-05-09T02:27:12.545 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.545 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.545 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.545 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T02:27:12.545 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T02:27:12.545 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-05-09T02:27:12.545 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.545 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T02:27:12.545 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.545 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-05-09T02:27:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-05-09T02:27:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-05-09T02:27:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.546 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.547 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-05-09T02:27:12.547 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-05-09T02:27:12.547 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.547 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.547 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.547 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-05-09T02:27:12.547 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.547 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.547 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T02:27:12.547 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.547 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.547 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.547 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-05-09T02:27:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.548 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.549 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.549 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.549 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.549 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.549 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.549 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.549 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.549 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.549 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.549 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.549 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.549 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.549 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.550 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.550 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.550 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.550 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.550 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.550 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.550 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.550 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.550 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.550 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.550 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.550 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.550 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.550 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.551 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.551 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.551 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.551 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.551 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.551 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.551 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.551 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.551 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.551 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.551 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.551 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.551 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.551 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.551 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.552 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.553 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.554 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.554 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.554 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-05-09T02:27:12.554 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.554 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.554 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.554 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.554 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.554 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.554 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.554 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.554 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.554 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.555 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.556 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.558 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.559 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.559 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.559 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.559 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.559 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.559 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.559 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.559 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.559 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.559 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.559 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.559 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.559 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.559 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T02:27:12.560 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-05-09T02:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-05-09T02:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-05-09T02:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T02:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.561 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.562 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.563 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.564 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.564 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.564 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.564 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.564 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.564 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.564 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.564 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.564 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.564 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.564 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.564 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.564 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.564 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.565 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.566 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-05-09T02:27:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.567 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.568 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.569 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.570 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.570 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.570 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.570 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.570 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.570 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.570 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.570 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.570 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.570 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.570 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.570 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.570 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.570 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.571 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.571 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.571 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.571 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.571 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.571 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.571 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.571 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.571 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.571 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.571 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.571 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.571 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.571 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.572 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.572 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.572 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.572 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.572 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.572 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.572 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.572 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.572 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.572 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.572 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.572 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.574 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.574 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.574 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-05-09T02:27:12.574 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T02:27:12.574 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.574 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.574 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-05-09T02:27:12.574 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-05-09T02:27:12.574 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.574 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.574 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.574 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-05-09T02:27:12.574 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-05-09T02:27:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-05-09T02:27:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-05-09T02:27:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-05-09T02:27:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.575 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-05-09T02:27:12.577 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.577 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.577 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.577 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.577 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.577 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.577 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.577 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.577 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.577 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.577 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.579 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.579 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.579 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-05-09T02:27:12.579 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.579 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.579 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.579 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.579 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.579 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.579 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.579 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.579 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.580 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.582 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.582 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.582 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.582 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.582 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.582 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.582 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.582 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.582 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.582 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.582 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.582 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.582 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.582 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.582 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.583 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.583 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.583 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.583 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.583 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.583 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.583 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.583 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.583 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.583 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.583 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.583 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.583 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.583 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.584 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.585 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.586 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.586 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.586 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.586 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.586 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.586 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.586 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.586 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.586 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.586 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.586 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.586 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.586 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.586 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.588 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.588 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.588 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.588 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.588 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.588 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.588 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.588 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.588 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.588 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.588 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.588 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.588 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.588 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.588 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.589 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.589 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.589 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.589 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.589 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.589 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.589 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.589 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.589 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.589 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.589 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.589 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.589 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.589 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.590 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.590 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.590 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.590 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.590 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.590 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.590 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.590 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.590 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.590 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.590 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.590 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.590 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.590 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.590 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.591 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.591 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.591 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.591 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.591 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-05-09T02:27:12.591 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.591 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.591 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.591 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.591 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.591 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.591 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.591 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.591 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.591 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.592 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.593 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.597 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.597 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.597 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-09T02:27:12.597 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.597 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.597 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.597 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-05-09T02:27:12.597 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-05-09T02:27:12.597 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.597 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.597 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.597 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.597 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.597 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.597 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.598 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T02:27:12.598 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.598 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.598 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.598 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.598 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.598 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.598 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.598 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.598 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.598 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.598 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.598 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.598 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.599 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.600 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.600 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.600 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.600 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.600 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.600 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.600 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.600 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.601 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.601 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.601 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.601 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.601 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.601 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.601 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.601 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.601 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.601 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.601 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.602 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.603 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.603 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.603 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.603 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.603 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.603 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.603 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.603 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.603 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-05-09T02:27:12.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-09T02:27:12.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.603 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.604 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.605 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.605 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.605 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.605 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.605 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.605 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.605 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.605 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.605 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.605 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.605 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.605 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.605 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.605 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-05-09T02:27:12.606 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.606 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.606 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.606 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.606 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.606 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.606 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.606 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.606 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.606 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.606 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.606 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.606 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.606 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.606 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.607 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.608 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.608 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.608 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.608 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.608 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.608 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.608 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.608 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.608 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.608 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.608 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.608 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.608 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.608 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.608 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.609 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.610 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.610 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.610 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.610 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.610 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.610 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.610 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.610 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.610 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.610 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.610 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.610 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.610 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.610 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.611 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.611 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.611 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.611 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.611 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.611 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.611 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.611 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.611 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.611 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.611 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.611 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.611 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-05-09T02:27:12.612 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.612 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.612 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.612 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.612 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.612 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.612 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.612 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.612 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.612 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.612 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.612 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.612 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.612 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.613 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.614 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.614 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.614 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.614 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.614 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.614 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.614 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.614 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.614 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.614 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.614 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.614 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.614 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.614 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.615 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.615 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.615 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.615 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.615 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.615 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.615 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.615 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.615 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.615 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.615 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.615 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.615 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.615 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.617 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.617 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.617 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.617 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.617 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.617 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.617 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.617 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.617 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.617 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.617 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.617 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.617 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.617 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-05-09T02:27:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.619 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.620 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.621 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T02:27:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T02:27:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T02:27:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.623 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-05-09T02:27:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-05-09T02:27:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-09T02:27:12.625 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-05-09T02:27:12.625 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.625 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.625 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-05-09T02:27:12.625 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T02:27:12.625 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.625 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.625 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.625 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.625 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.625 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.625 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-05-09T02:27:12.625 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.625 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.625 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.626 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-05-09T02:27:12.626 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-05-09T02:27:12.626 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.626 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.626 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.626 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.626 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.626 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.626 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T02:27:12.626 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.626 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.626 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.626 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.626 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-09T02:27:12.627 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-09T02:27:12.627 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.627 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.627 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.627 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.627 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.627 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.627 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.627 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.627 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.627 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.627 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.627 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.627 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.628 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.628 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.628 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.628 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.628 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.628 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.628 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.628 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.628 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.628 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.628 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.628 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.628 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.628 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.629 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.630 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.630 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.630 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.630 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.630 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.630 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.630 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.630 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.630 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.630 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.630 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.630 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.630 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.630 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.631 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.632 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.633 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-05-09T02:27:12.633 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.633 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.633 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.633 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.633 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T02:27:12.633 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T02:27:12.633 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.633 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.633 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T02:27:12.633 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.633 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.633 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.634 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-05-09T02:27:12.634 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.634 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.634 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.634 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.634 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.634 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.634 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.634 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.634 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.634 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.634 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.634 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.634 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.635 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.636 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.636 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.636 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.636 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.636 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.636 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.636 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.636 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.636 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-05-09T02:27:12.636 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-05-09T02:27:12.636 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.636 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.636 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.637 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.637 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.637 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.637 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.637 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T02:27:12.637 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T02:27:12.637 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.637 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.637 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.637 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.637 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.637 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.637 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.637 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.638 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.638 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.638 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.638 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.638 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.638 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.638 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.638 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.638 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.638 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.638 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.638 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.638 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.639 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.639 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.639 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.639 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.639 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.639 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.639 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.639 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.639 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.639 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.639 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.639 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.639 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.639 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.640 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.640 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.640 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.640 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.640 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.640 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.640 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.640 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.640 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.640 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.640 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.640 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.640 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.641 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.642 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.643 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.643 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.643 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T02:27:12.643 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T02:27:12.643 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.643 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.643 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.643 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-05-09T02:27:12.643 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.643 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.643 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.643 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.643 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.643 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.645 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T02:27:12.645 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T02:27:12.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.645 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.645 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.645 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-09T02:27:12.645 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.645 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.645 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.645 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T02:27:12.645 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.645 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.645 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.645 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T02:27:12.646 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T02:27:12.646 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.646 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.646 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.646 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-05-09T02:27:12.646 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.646 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.646 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T02:27:12.646 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.646 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.646 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.646 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.646 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.646 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.647 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.648 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.648 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-05-09T02:27:12.648 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.648 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.648 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.648 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.648 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.648 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.648 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.648 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-09T02:27:12.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-09T02:27:12.648 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.648 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.648 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.649 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-05-09T02:27:12.649 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.649 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.649 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.649 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-05-09T02:27:12.649 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-05-09T02:27:12.649 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.649 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.649 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.649 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.649 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.649 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-09T02:27:12.649 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-09T02:27:12.649 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-09T02:27:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-09T02:27:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-09T02:27:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.650 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-09T02:27:12.651 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.651 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.651 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.651 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.651 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.651 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.651 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.651 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.651 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.651 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.651 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.651 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.651 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.651 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.651 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.652 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.652 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.652 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.652 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.652 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.652 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.652 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.652 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.652 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.652 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.652 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.652 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.652 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.652 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.653 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.654 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.654 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.654 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.654 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.654 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.654 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.654 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.654 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.654 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.654 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.654 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.654 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.654 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-05-09T02:27:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.656 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.657 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.657 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.657 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.657 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.657 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.657 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.657 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.657 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.657 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.657 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.657 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.657 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.658 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.658 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.658 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.658 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.658 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.658 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.658 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.658 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.658 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-05-09T02:27:12.658 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-05-09T02:27:12.658 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.658 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.658 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.658 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-05-09T02:27:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.659 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.660 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-05-09T02:27:12.660 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-05-09T02:27:12.660 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.660 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.660 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.660 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.660 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.660 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.660 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.660 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.660 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.660 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.660 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.660 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-09T02:27:12.661 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-09T02:27:12.661 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.661 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.661 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.661 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.661 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.661 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.661 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.661 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.661 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.661 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.661 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.661 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.662 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.662 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.662 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.662 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.662 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.662 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.662 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.662 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.662 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.662 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.662 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.662 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.662 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.662 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.662 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.664 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.664 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.664 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.664 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.664 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.664 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.664 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.664 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.664 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.664 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.664 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.664 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.664 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.664 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.664 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.665 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T02:27:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T02:27:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.667 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.667 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.667 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.667 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.667 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.667 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.667 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T02:27:12.667 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T02:27:12.667 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.667 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.667 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.667 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.667 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.667 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.669 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.671 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.671 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.671 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.671 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.671 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.671 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.671 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.671 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.671 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.671 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.671 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.671 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.671 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.672 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-05-09T02:27:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-05-09T02:27:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-05-09T02:27:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-05-09T02:27:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T02:27:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-05-09T02:27:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.685 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.685 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.685 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.685 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.685 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.685 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.685 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.685 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.685 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.685 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.685 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.685 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.685 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.685 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.686 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.687 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.687 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.687 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.687 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.687 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.687 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-05-09T02:27:12.687 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-05-09T02:27:12.687 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.687 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.687 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.687 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.687 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.687 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.688 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.689 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.689 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.689 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.689 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-05-09T02:27:12.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-05-09T02:27:12.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.689 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.689 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.689 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.689 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.689 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.689 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.689 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.689 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-05-09T02:27:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-05-09T02:27:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.691 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.691 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.691 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-05-09T02:27:12.691 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-05-09T02:27:12.691 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:12.691 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.691 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.691 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.691 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.691 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.691 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.691 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.691 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.691 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.692 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.692 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-05-09T02:27:12.692 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-05-09T02:27:12.692 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.692 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.692 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.692 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.692 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.692 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.692 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.692 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.692 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.693 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.694 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-05-09T02:27:12.694 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-05-09T02:27:12.694 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.694 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.694 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.694 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.694 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.694 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.694 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.694 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.694 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.694 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-05-09T02:27:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-05-09T02:27:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T02:27:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-05-09T02:27:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.695 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-05-09T02:27:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.697 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-05-09T02:27:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-05-09T02:27:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.698 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.699 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.699 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.699 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.699 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.699 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.699 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.699 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.699 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.699 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.699 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.699 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.699 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smb", 2024-05-09T02:27:12.700 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.700 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.700 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.700 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.700 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.700 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.700 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.700 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.700 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.700 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.700 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.700 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.700 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.700 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.701 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.701 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.701 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.701 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.701 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.701 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.701 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.701 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.701 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.701 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.701 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.701 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.701 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.701 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.702 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.702 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.702 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.702 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.702 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.702 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.702 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.702 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.702 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.702 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.702 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.702 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.702 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.702 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.703 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.703 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.703 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.703 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.703 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.703 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.703 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.703 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.703 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.703 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.703 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.703 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.703 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.703 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.704 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.704 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.704 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.704 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.704 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.704 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.704 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.704 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.704 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.704 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.704 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.704 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.704 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.704 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: "update_orchestration": { 2024-05-09T02:27:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "update_orchestration", 2024-05-09T02:27:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.706 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.706 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.706 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.706 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.706 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.706 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.706 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T02:27:12.706 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.706 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.706 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.706 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.706 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.706 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.706 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-09T02:27:12.707 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.707 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.707 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.707 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-05-09T02:27:12.707 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-05-09T02:27:12.707 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.707 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.707 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.707 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.707 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.707 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.707 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T02:27:12.707 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.707 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.708 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.708 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-05-09T02:27:12.708 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-05-09T02:27:12.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.708 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.708 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.708 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.708 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.708 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.708 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.708 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-05-09T02:27:12.708 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.708 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.708 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.709 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.710 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.710 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.710 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.710 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.710 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.710 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.710 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.710 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.710 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.710 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.710 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.710 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.710 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.710 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.711 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.713 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-05-09T02:27:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.715 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.716 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.716 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.716 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.716 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.716 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.716 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.716 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.716 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.716 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.716 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.716 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.716 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.716 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.717 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.717 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.717 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.717 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.717 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.717 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.717 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.717 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.717 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.717 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.717 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.717 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.717 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.720 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.720 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.720 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.720 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.720 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.720 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.720 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.720 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.720 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-05-09T02:27:12.720 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.720 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.720 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.720 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.721 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.722 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.722 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.722 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.722 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.722 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.722 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.722 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.722 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.722 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.722 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.722 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.722 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.722 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.722 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.723 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.724 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.725 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.725 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.725 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.725 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.725 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.725 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.725 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.725 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.725 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.725 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.725 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.725 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.725 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-05-09T02:27:12.726 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-05-09T02:27:12.727 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.727 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.727 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.727 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T02:27:12.727 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.727 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.727 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.727 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.727 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.727 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.727 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.727 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T02:27:12.727 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T02:27:12.727 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.728 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.728 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.728 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-05-09T02:27:12.728 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.728 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.728 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.728 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.728 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.728 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.728 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.728 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.728 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.728 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.728 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.729 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.729 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.729 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.729 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.729 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.729 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.729 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.729 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.729 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.729 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.729 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.729 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.729 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.729 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.729 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.730 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.730 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.730 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.730 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.730 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.730 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.730 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.730 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.730 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.730 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.730 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.730 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.730 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.730 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.731 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.731 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.731 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.731 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.731 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.731 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.731 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.731 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.731 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.731 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.731 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.731 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.731 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.731 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.732 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.734 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.734 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.734 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.734 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.734 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.734 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.734 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-05-09T02:27:12.734 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.734 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.734 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.734 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-05-09T02:27:12.734 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-05-09T02:27:12.734 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.734 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T02:27:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-05-09T02:27:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-05-09T02:27:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.735 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.736 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.736 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.736 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.736 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.736 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T02:27:12.736 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.736 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.736 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.736 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-05-09T02:27:12.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-05-09T02:27:12.736 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.736 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.736 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.736 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T02:27:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-05-09T02:27:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-05-09T02:27:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.738 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.738 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.738 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T02:27:12.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.738 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.738 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.738 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-05-09T02:27:12.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-05-09T02:27:12.738 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.738 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.738 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.738 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.739 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.739 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.739 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T02:27:12.739 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.739 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.739 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.739 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-05-09T02:27:12.739 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-05-09T02:27:12.739 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.739 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.739 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.739 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-05-09T02:27:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-05-09T02:27:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.740 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-05-09T02:27:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-05-09T02:27:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T02:27:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.741 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.742 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.742 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.742 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.742 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.742 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.742 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-09T02:27:12.742 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-09T02:27:12.742 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.742 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.742 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.742 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.742 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.742 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.742 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.743 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.743 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.743 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.743 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.743 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T02:27:12.743 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T02:27:12.743 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.743 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.743 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.743 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-05-09T02:27:12.743 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-05-09T02:27:12.743 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.743 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.743 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.744 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-05-09T02:27:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-05-09T02:27:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.745 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.746 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-05-09T02:27:12.746 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-05-09T02:27:12.746 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.746 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.746 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.746 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.746 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.746 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.746 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.746 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.746 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.746 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.746 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.747 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.748 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.749 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.749 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.749 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.749 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.749 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.749 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.749 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.749 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.749 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.749 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.749 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.749 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.749 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.751 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.751 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.751 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.751 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.751 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.751 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.751 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.751 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.751 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-05-09T02:27:12.751 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-05-09T02:27:12.751 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.751 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.751 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-05-09T02:27:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-05-09T02:27:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.752 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.753 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-05-09T02:27:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-05-09T02:27:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T02:27:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.754 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.755 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.755 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.755 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.755 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.755 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.755 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.755 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.755 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.755 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-05-09T02:27:12.755 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.755 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.756 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.756 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.756 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.756 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.756 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.756 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.756 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.756 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.756 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.756 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.756 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.756 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.756 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.756 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.756 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.757 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.757 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.757 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.757 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.757 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.757 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.757 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.757 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.757 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.757 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.757 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.757 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.757 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.757 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.758 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.758 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.758 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.758 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.758 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.758 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.758 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.758 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.758 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.758 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.758 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.758 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.759 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.760 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.761 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-09T02:27:12.762 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.762 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.762 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.762 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.762 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.762 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.762 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.762 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.762 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.762 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.762 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.762 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.762 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.762 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.763 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.764 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.764 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.764 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.764 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.764 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.764 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.764 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.764 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.764 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.764 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.764 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.764 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.764 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.765 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.765 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.765 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.765 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.765 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.765 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.765 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.765 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.765 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.765 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.765 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.765 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.765 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.765 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.766 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.766 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.766 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.766 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.766 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.766 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.766 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.766 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.766 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.766 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.766 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.766 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.766 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.766 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-05-09T02:27:12.767 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-05-09T02:27:12.767 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.767 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.767 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.767 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-05-09T02:27:12.767 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.767 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.767 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.767 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T02:27:12.767 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.767 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.767 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.767 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-05-09T02:27:12.767 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.768 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T02:27:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-05-09T02:27:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T02:27:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.769 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.770 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.770 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.770 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.770 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.770 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T02:27:12.770 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.770 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.770 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.770 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.770 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.770 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.770 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.770 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.771 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.771 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.771 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.771 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.771 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.771 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.771 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.771 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.771 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.771 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.771 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.771 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-05-09T02:27:12.771 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.771 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.772 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.772 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-05-09T02:27:12.772 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-05-09T02:27:12.772 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T02:27:12.772 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.772 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.772 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-05-09T02:27:12.772 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.772 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.772 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.772 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.772 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.772 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.772 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-05-09T02:27:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-05-09T02:27:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.774 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T02:27:12.774 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T02:27:12.774 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.774 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.774 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.774 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T02:27:12.774 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.774 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.774 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.774 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.774 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.774 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.774 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.774 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.775 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.775 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.775 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.775 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.775 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.775 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.775 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.775 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.775 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.775 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.775 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.775 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.775 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.775 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.775 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.776 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.776 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.776 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.776 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.776 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.776 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.776 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.776 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.776 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.776 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.776 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.776 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.776 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.776 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.777 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.777 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.777 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.777 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.777 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.777 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.777 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.777 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.777 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.777 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.777 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.777 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.777 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.777 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.778 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.778 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.778 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.778 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.778 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.778 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.778 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.778 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.778 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.778 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.778 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.778 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.778 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.779 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.780 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.780 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.780 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.780 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.780 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.780 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.780 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.780 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-05-09T02:27:12.780 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-05-09T02:27:12.780 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.780 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.780 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.780 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.780 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.781 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.781 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.781 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.781 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.781 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.781 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.781 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-05-09T02:27:12.781 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-05-09T02:27:12.781 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.781 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.781 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.781 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-05-09T02:27:12.781 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.781 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.782 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.782 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.782 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.782 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.782 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.782 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-05-09T02:27:12.782 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-05-09T02:27:12.782 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.782 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.782 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.782 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T02:27:12.782 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.782 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.782 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T02:27:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: "modules": [ 2024-05-09T02:27:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-09T02:27:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard", 2024-05-09T02:27:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: "iostat", 2024-05-09T02:27:12.783 INFO:teuthology.orchestra.run.smithi019.stdout: "nfs", 2024-05-09T02:27:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: "restful", 2024-05-09T02:27:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_schedule" 2024-05-09T02:27:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-05-09T02:27:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-05-09T02:27:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T02:27:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T02:27:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.785 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.786 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.786 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.786 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.786 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.786 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.786 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.786 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.786 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.786 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.786 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.786 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.786 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.786 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.786 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.787 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.788 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.790 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-05-09T02:27:12.790 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-05-09T02:27:12.790 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.790 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.790 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.790 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.790 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.790 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.790 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.790 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-05-09T02:27:12.790 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.790 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.790 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.790 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-05-09T02:27:12.791 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-05-09T02:27:12.791 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.791 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.791 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.791 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-05-09T02:27:12.791 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.791 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.791 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.791 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-05-09T02:27:12.791 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.791 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.791 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.791 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-05-09T02:27:12.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.793 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.793 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.793 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.793 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.793 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.793 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-05-09T02:27:12.793 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.793 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.793 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.793 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-05-09T02:27:12.793 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-05-09T02:27:12.793 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.794 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.794 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.794 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-05-09T02:27:12.794 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.794 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.794 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.794 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-05-09T02:27:12.794 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.794 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.794 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.794 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-05-09T02:27:12.794 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-05-09T02:27:12.794 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.794 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-05-09T02:27:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-05-09T02:27:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-05-09T02:27:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.795 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.796 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.796 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.796 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.796 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.796 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T02:27:12.796 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.796 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.796 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.796 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-05-09T02:27:12.796 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-05-09T02:27:12.796 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.796 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.796 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.796 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.797 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.797 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.797 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.797 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-05-09T02:27:12.797 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.797 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.797 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.797 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.797 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.797 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.797 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.797 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.798 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.798 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.798 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.798 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.798 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.798 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.798 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.798 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.798 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.798 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.798 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-05-09T02:27:12.798 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.798 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.798 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.799 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-05-09T02:27:12.799 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-05-09T02:27:12.799 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.799 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.799 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.799 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.799 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.799 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.799 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T02:27:12.799 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.799 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.799 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.799 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-05-09T02:27:12.800 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-05-09T02:27:12.800 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.800 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.800 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.800 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-05-09T02:27:12.800 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.800 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.872 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.872 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T02:27:12.872 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T02:27:12.872 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-05-09T02:27:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-05-09T02:27:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T02:27:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-09T02:27:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-09T02:27:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T02:27:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T02:27:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.874 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-05-09T02:27:12.874 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-05-09T02:27:12.874 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T02:27:12.874 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.874 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.874 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-05-09T02:27:12.874 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T02:27:12.874 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-05-09T02:27:12.874 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T02:27:12.874 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.874 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.874 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.875 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.875 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-05-09T02:27:12.875 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-05-09T02:27:12.875 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.875 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.875 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.875 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T02:27:12.875 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.875 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.875 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.875 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T02:27:12.875 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T02:27:12.875 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.875 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.875 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.876 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-05-09T02:27:12.876 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-05-09T02:27:12.876 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:12.876 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.876 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.876 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-05-09T02:27:12.876 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-05-09T02:27:12.876 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-05-09T02:27:12.876 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.876 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-05-09T02:27:12.876 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T02:27:12.876 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.876 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-05-09T02:27:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-05-09T02:27:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-05-09T02:27:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T02:27:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T02:27:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.877 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-05-09T02:27:12.878 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-05-09T02:27:12.878 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T02:27:12.878 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.878 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.878 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.878 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-09T02:27:12.878 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-09T02:27:12.878 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.878 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T02:27:12.878 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T02:27:12.878 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.878 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.878 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.879 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.879 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.879 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.879 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.879 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.879 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.879 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.879 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.879 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.879 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.879 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.879 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.879 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.879 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.879 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.880 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.880 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.880 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.880 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.880 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.880 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.880 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.880 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.880 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.880 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.880 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.880 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.880 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.880 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.881 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.881 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.881 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.881 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.881 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.881 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.881 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.881 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.881 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.881 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.881 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.881 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.881 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.881 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.881 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-05-09T02:27:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-05-09T02:27:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.884 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.884 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.884 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.884 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.884 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T02:27:12.884 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.884 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.884 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.884 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-09T02:27:12.884 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-09T02:27:12.884 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.884 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.884 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.885 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-05-09T02:27:12.885 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.885 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.885 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.885 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-05-09T02:27:12.885 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-05-09T02:27:12.885 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-05-09T02:27:12.885 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-05-09T02:27:12.885 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-05-09T02:27:12.885 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.885 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-05-09T02:27:12.885 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.885 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.885 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.886 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-05-09T02:27:12.886 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-05-09T02:27:12.886 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.886 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.886 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.886 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.886 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.886 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.886 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.886 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T02:27:12.886 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.886 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.886 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.887 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T02:27:12.887 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T02:27:12.887 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.887 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.887 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.887 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T02:27:12.887 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.887 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.887 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.887 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T02:27:12.887 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.887 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.887 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-05-09T02:27:12.888 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-05-09T02:27:12.889 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.889 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.889 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.889 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T02:27:12.889 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T02:27:12.889 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.889 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.889 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T02:27:12.889 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T02:27:12.889 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.889 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.889 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.889 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-05-09T02:27:12.889 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-05-09T02:27:12.889 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T02:27:12.890 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.890 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.890 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.890 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.890 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.890 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.890 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T02:27:12.890 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.890 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.890 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.890 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.890 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.890 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.891 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-05-09T02:27:12.891 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.891 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.891 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.891 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-05-09T02:27:12.891 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-05-09T02:27:12.891 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:12.891 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.891 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.891 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-09T02:27:12.891 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.891 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.891 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.891 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T02:27:12.891 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.892 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.893 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-05-09T02:27:12.893 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-05-09T02:27:12.893 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.893 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.893 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.893 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-05-09T02:27:12.893 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.893 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.893 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.893 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T02:27:12.893 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.893 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.893 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.894 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-05-09T02:27:12.894 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-05-09T02:27:12.894 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.894 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.894 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.894 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.894 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.894 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.894 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.894 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T02:27:12.894 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T02:27:12.894 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.894 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.895 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-05-09T02:27:12.895 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-05-09T02:27:12.895 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.895 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.895 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.895 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T02:27:12.895 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.895 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.895 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.895 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T02:27:12.895 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.895 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.895 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.896 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-05-09T02:27:12.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T02:27:12.896 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:12.896 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.896 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.896 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-05-09T02:27:12.896 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.896 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.896 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.896 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T02:27:12.896 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.896 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.896 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.896 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.896 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.898 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.898 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-05-09T02:27:12.898 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-05-09T02:27:12.898 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.898 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.898 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.898 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.898 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.898 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.898 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.898 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T02:27:12.898 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.898 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.898 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.899 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.899 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-05-09T02:27:12.899 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-05-09T02:27:12.899 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.899 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.899 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.899 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.899 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.899 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.899 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.899 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T02:27:12.899 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.899 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.899 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.900 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-05-09T02:27:12.900 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-05-09T02:27:12.900 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.900 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.900 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.900 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.900 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.900 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.900 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.900 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T02:27:12.900 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.900 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.900 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.901 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-05-09T02:27:12.902 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-05-09T02:27:12.902 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.902 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.902 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.902 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T02:27:12.902 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.902 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.902 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.902 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-05-09T02:27:12.902 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.902 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.902 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.902 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.902 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-05-09T02:27:12.903 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-05-09T02:27:12.903 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.903 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.903 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.903 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T02:27:12.903 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.903 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.903 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.903 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-05-09T02:27:12.903 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.903 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.903 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.903 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.903 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-05-09T02:27:12.903 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T02:27:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-05-09T02:27:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-05-09T02:27:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-05-09T02:27:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T02:27:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-05-09T02:27:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-05-09T02:27:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-05-09T02:27:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T02:27:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-05-09T02:27:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-05-09T02:27:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-05-09T02:27:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T02:27:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-05-09T02:27:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-05-09T02:27:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-05-09T02:27:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T02:27:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T02:27:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-05-09T02:27:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-05-09T02:27:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T02:27:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-05-09T02:27:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-05-09T02:27:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-05-09T02:27:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T02:27:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-09T02:27:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-05-09T02:27:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-05-09T02:27:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T02:27:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-05-09T02:27:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-05-09T02:27:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-05-09T02:27:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T02:27:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-05-09T02:27:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-05-09T02:27:12.916 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-05-09T02:27:12.916 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.916 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.916 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.916 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.916 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.916 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.916 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.916 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T02:27:12.916 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.916 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.916 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.916 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.916 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-05-09T02:27:12.916 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-05-09T02:27:12.917 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.917 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.917 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.917 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T02:27:12.917 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.917 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.917 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.917 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T02:27:12.917 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.917 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.917 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.917 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-05-09T02:27:12.917 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T02:27:12.917 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.918 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.918 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-05-09T02:27:12.918 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.918 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.918 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.918 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T02:27:12.918 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.918 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.918 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.918 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.918 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-05-09T02:27:12.918 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-05-09T02:27:12.918 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.919 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.920 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-05-09T02:27:12.920 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.920 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.920 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.920 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-05-09T02:27:12.920 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.920 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.920 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-05-09T02:27:12.920 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-05-09T02:27:12.920 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.920 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.920 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.921 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.921 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.921 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.921 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T02:27:12.921 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.921 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.921 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.921 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-05-09T02:27:12.921 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-05-09T02:27:12.921 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.921 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.921 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.921 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T02:27:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-05-09T02:27:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-05-09T02:27:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-05-09T02:27:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T02:27:12.922 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.923 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.923 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.923 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T02:27:12.923 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.923 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.923 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.923 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-05-09T02:27:12.923 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-05-09T02:27:12.923 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.923 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.923 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.923 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T02:27:12.923 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.924 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T02:27:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-05-09T02:27:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-05-09T02:27:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.925 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T02:27:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.926 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.927 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.927 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.927 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.927 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.927 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.927 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.927 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.927 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.927 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.927 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-05-09T02:27:12.927 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-05-09T02:27:12.927 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.927 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.927 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.928 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.928 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.928 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.928 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.928 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T02:27:12.928 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.928 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.928 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.928 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.928 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.928 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.928 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.928 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.929 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.930 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.930 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.930 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.930 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.930 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.930 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.930 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.930 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.930 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.930 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.930 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.930 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.930 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T02:27:12.931 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T02:27:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T02:27:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T02:27:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T02:27:12.932 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-05-09T02:27:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T02:27:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-05-09T02:27:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-05-09T02:27:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.933 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.934 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.934 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.934 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.934 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.934 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.934 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-05-09T02:27:12.934 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.934 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.934 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.934 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.934 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-05-09T02:27:12.934 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-05-09T02:27:12.935 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.935 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.935 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.935 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.935 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.935 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.935 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.935 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T02:27:12.935 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.935 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.935 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.935 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-05-09T02:27:12.935 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-05-09T02:27:12.936 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.936 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.936 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.936 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.936 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.936 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.936 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.936 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-05-09T02:27:12.936 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.936 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.936 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.936 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-09T02:27:12.936 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-09T02:27:12.936 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.937 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.937 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.937 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-05-09T02:27:12.937 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.937 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.937 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.937 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-05-09T02:27:12.937 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-05-09T02:27:12.937 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.937 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T02:27:12.937 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.937 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.937 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.937 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-05-09T02:27:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-05-09T02:27:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-05-09T02:27:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T02:27:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.938 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-05-09T02:27:12.939 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-05-09T02:27:12.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.939 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T02:27:12.939 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.939 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.939 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.939 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T02:27:12.939 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.939 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.939 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.939 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-05-09T02:27:12.940 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-05-09T02:27:12.940 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.940 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.940 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.940 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.940 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.940 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.940 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.940 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T02:27:12.940 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.940 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.940 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.940 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-05-09T02:27:12.940 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-05-09T02:27:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T02:27:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-05-09T02:27:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-05-09T02:27:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.942 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.942 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.942 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.942 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.942 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.942 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.942 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T02:27:12.942 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.942 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.942 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.942 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-05-09T02:27:12.942 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-05-09T02:27:12.942 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.943 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.943 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.943 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.943 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.943 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.943 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.943 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T02:27:12.943 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.943 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.943 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.943 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-05-09T02:27:12.943 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-05-09T02:27:12.943 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.943 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.944 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.944 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.944 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.944 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.944 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.944 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T02:27:12.944 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.944 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.944 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.944 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.944 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-05-09T02:27:12.944 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-05-09T02:27:12.944 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.944 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.944 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.945 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-05-09T02:27:12.945 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.945 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.945 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.945 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-05-09T02:27:12.945 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.945 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.945 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.945 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.945 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.945 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.945 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-05-09T02:27:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-05-09T02:27:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.946 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T02:27:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-05-09T02:27:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-05-09T02:27:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.948 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.949 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.949 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T02:27:12.949 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.949 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.949 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.949 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.949 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-05-09T02:27:12.949 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-05-09T02:27:12.949 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.949 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.949 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.949 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.949 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.949 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.950 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.950 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T02:27:12.950 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.950 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.950 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.950 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.950 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-05-09T02:27:12.950 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-05-09T02:27:12.950 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.950 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.950 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.950 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.950 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.950 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.950 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T02:27:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-05-09T02:27:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-05-09T02:27:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T02:27:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-05-09T02:27:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.952 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.953 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.954 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:12.954 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:12.954 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.954 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.954 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.954 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.954 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.954 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.954 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.954 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.954 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.954 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:12.955 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.956 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.957 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.958 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T02:27:12.958 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.958 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.958 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.958 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:12.958 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:12.958 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.958 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:12.958 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.958 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:12.958 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.958 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.958 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.959 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.959 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.959 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.959 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.959 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-05-09T02:27:12.959 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-05-09T02:27:12.959 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:12.959 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.959 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:12.959 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-05-09T02:27:12.959 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.959 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.959 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.959 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T02:27:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-05-09T02:27:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T02:27:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T02:27:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.960 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.961 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.961 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.961 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.961 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.961 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.961 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.961 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.961 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.961 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T02:27:12.961 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T02:27:12.961 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.961 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.962 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.962 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.962 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.962 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.962 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.962 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.962 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.962 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.962 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T02:27:12.962 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T02:27:12.962 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.962 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.962 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.962 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-05-09T02:27:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T02:27:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.964 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.964 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.964 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.964 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.964 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.964 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.964 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T02:27:12.964 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T02:27:12.964 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.964 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.964 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.964 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.964 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.964 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.965 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.965 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.965 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.965 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.965 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T02:27:12.965 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T02:27:12.965 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.965 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.965 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.965 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.965 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.965 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T02:27:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T02:27:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.966 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.967 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.967 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.967 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.967 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T02:27:12.967 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T02:27:12.967 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.967 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.967 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.967 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.967 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.967 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.967 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.967 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T02:27:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T02:27:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.968 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.969 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.969 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T02:27:12.969 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T02:27:12.969 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.969 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.969 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.969 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.969 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.969 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.969 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.969 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.969 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.969 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.969 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.970 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T02:27:12.970 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T02:27:12.970 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.970 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.970 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.970 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.970 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.970 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.970 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.970 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.970 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.970 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.970 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.971 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T02:27:12.971 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T02:27:12.971 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.971 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.971 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.971 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.971 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.971 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.971 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.971 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.971 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.971 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.971 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T02:27:12.972 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T02:27:12.973 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.973 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.973 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.973 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.973 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.973 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.973 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.973 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.973 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.973 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.973 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.973 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T02:27:12.973 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T02:27:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T02:27:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T02:27:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T02:27:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-05-09T02:27:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-05-09T02:27:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.976 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.977 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.978 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.978 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.978 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.978 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.978 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.978 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T02:27:12.978 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T02:27:12.978 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.978 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.978 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.978 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.979 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.979 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.979 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.979 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.979 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.979 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T02:27:12.979 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T02:27:12.979 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.979 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.979 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.979 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.979 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.979 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T02:27:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T02:27:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.980 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.981 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.981 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.981 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.981 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.981 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T02:27:12.981 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T02:27:12.981 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.981 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.981 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-05-09T02:27:12.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.981 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.981 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.981 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.982 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.982 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.982 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.982 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T02:27:12.982 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T02:27:12.982 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.982 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.982 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.982 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-05-09T02:27:12.982 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.982 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.982 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.983 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.983 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.983 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.983 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.983 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T02:27:12.983 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T02:27:12.983 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.983 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.983 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.983 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.983 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.983 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.983 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.983 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.984 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.985 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T02:27:12.985 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T02:27:12.985 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.985 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.985 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.985 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.985 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.985 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.985 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.985 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.985 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.985 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.985 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T02:27:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T02:27:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T02:27:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T02:27:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T02:27:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T02:27:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T02:27:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T02:27:12.989 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.990 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.990 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.990 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.990 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.990 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.990 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.990 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.990 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.990 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.990 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T02:27:12.990 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T02:27:12.990 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.991 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.992 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.992 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.992 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.992 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.992 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.992 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.992 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.992 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.992 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T02:27:12.992 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T02:27:12.992 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.992 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.992 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.993 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T02:27:12.993 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.993 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.993 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.993 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.993 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.993 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.993 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.993 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T02:27:12.993 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T02:27:12.993 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.993 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.993 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.993 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:12.994 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.994 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.994 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.994 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.994 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.994 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.994 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.994 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T02:27:12.994 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T02:27:12.994 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.994 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.994 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-05-09T02:27:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T02:27:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T02:27:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.995 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-05-09T02:27:12.996 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.996 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.996 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.996 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.996 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.996 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.996 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T02:27:12.996 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T02:27:12.996 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.996 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.996 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.996 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.997 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.998 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.998 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.998 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.998 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.998 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.998 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T02:27:12.998 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T02:27:12.998 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:12.998 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.998 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.998 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:12.998 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.999 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:12.999 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:12.999 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:12.999 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:12.999 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:12.999 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:12.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:12.999 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T02:27:12.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T02:27:12.999 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:12.999 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:12.999 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:12.999 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:12.999 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:12.999 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T02:27:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T02:27:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.000 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T02:27:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T02:27:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.001 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T02:27:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T02:27:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T02:27:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T02:27:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T02:27:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.003 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-05-09T02:27:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-05-09T02:27:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.004 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-05-09T02:27:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-05-09T02:27:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.005 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-05-09T02:27:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-05-09T02:27:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-05-09T02:27:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-05-09T02:27:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-05-09T02:27:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-05-09T02:27:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.007 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-09T02:27:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-09T02:27:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.009 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.010 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.011 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.012 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.012 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.012 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.012 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.012 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.012 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.012 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.012 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.012 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.012 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.012 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.012 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.013 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.014 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.014 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.014 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.014 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.014 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.014 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.014 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.014 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.014 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.014 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.014 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.014 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.014 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-05-09T02:27:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-05-09T02:27:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-05-09T02:27:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.015 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-05-09T02:27:13.016 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T02:27:13.016 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.016 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.016 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.016 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.016 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.016 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.016 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.016 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.016 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.016 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.017 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T02:27:13.017 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T02:27:13.017 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.017 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.017 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.017 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-09T02:27:13.017 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.017 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.017 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.017 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.017 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.017 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.017 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.017 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T02:27:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T02:27:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-05-09T02:27:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.019 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.019 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.019 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.019 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.019 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.019 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.019 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.019 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.019 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.019 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.019 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.019 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-09T02:27:13.019 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-09T02:27:13.020 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.020 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.020 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.020 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:13.020 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.020 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.020 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.020 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.020 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.020 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.020 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.020 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-05-09T02:27:13.020 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-05-09T02:27:13.020 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.021 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.021 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.021 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-05-09T02:27:13.021 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.021 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.021 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.021 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.021 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.021 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.021 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.021 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-09T02:27:13.021 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-09T02:27:13.021 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.021 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.022 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.022 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-05-09T02:27:13.022 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.022 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.022 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.022 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.022 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-05-09T02:27:13.022 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.022 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.022 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.022 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.022 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.022 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-09T02:27:13.022 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-09T02:27:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-09T02:27:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-09T02:27:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-09T02:27:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-05-09T02:27:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-05-09T02:27:13.023 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.024 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.024 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.024 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.024 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.024 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.024 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.024 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.024 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.024 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.024 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.024 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.024 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.024 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.024 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-09T02:27:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-05-09T02:27:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-05-09T02:27:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-05-09T02:27:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.025 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.026 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.026 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.026 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.026 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.026 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.026 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.026 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.026 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.026 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.026 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.026 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.026 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.026 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.027 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.027 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.027 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.027 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.027 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.027 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.027 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.027 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.027 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.027 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.027 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.027 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.027 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.029 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.029 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.029 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.029 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.029 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.029 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.029 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.029 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.029 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.029 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.029 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.029 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.029 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.029 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.030 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.030 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.030 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.030 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.030 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.030 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.030 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.030 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.030 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.030 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.030 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.030 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.030 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.031 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-05-09T02:27:13.032 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-05-09T02:27:13.032 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.032 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.032 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.032 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-05-09T02:27:13.032 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.032 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.032 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.032 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T02:27:13.032 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.032 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.032 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.032 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-05-09T02:27:13.033 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-05-09T02:27:13.033 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:13.033 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.033 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.033 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-05-09T02:27:13.033 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.033 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.033 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.033 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-05-09T02:27:13.033 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.033 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.033 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.033 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-05-09T02:27:13.033 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-05-09T02:27:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-09T02:27:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T02:27:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-05-09T02:27:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-05-09T02:27:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T02:27:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T02:27:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T02:27:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:13.035 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T02:27:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T02:27:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-05-09T02:27:13.037 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.038 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.038 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.038 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T02:27:13.038 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.038 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.038 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.038 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.038 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.038 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.038 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-05-09T02:27:13.038 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.038 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.038 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.038 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.039 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.040 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.040 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.040 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.040 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.040 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.040 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.040 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.040 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.040 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.040 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.040 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.041 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.041 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.041 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.041 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.041 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.041 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.041 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.041 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.041 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.041 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.041 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.041 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.041 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.041 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.042 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.042 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.042 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.042 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.042 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.042 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.042 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.042 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.042 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.042 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.042 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.042 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.042 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-05-09T02:27:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-05-09T02:27:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-09T02:27:13.043 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.044 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.044 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.044 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.044 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.044 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.044 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.044 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-05-09T02:27:13.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-05-09T02:27:13.044 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.044 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.044 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.044 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-05-09T02:27:13.044 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.044 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.046 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.046 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.046 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.046 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.046 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.046 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.046 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.046 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.046 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.046 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.046 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.046 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.047 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.047 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.047 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.047 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.047 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.047 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.047 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-05-09T02:27:13.047 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-05-09T02:27:13.047 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-05-09T02:27:13.047 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.047 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-05-09T02:27:13.047 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-05-09T02:27:13.047 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.047 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.048 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-05-09T02:27:13.048 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.048 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.048 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.048 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T02:27:13.048 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.048 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.048 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.048 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-05-09T02:27:13.048 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-05-09T02:27:13.048 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.049 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-05-09T02:27:13.049 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.049 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.049 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.049 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T02:27:13.049 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.049 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.049 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.049 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-05-09T02:27:13.049 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-05-09T02:27:13.049 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.049 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.049 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.050 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.050 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.050 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.050 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.050 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-05-09T02:27:13.050 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.050 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.050 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.050 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T02:27:13.050 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T02:27:13.050 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:13.050 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.050 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.051 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-05-09T02:27:13.051 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-05-09T02:27:13.051 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.051 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.051 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T02:27:13.051 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.051 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.051 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.051 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.051 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.051 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.051 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.053 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.053 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.053 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.053 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.053 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.053 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.053 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.053 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.053 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.053 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.053 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.053 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.053 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.053 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.054 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.054 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.054 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.054 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.054 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.054 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.054 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.054 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.054 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.054 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.054 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.054 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.055 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.055 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.055 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.055 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.055 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.055 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.055 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.055 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.055 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.055 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.055 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.055 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.055 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-05-09T02:27:13.056 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-05-09T02:27:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-05-09T02:27:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-05-09T02:27:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-05-09T02:27:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.057 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.058 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.058 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.058 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.058 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.058 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.058 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.058 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.058 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.058 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.058 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.058 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.058 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.058 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.058 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-05-09T02:27:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-05-09T02:27:13.061 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.061 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.061 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.061 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.061 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.061 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.061 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-05-09T02:27:13.061 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.061 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.061 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.061 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-05-09T02:27:13.061 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-05-09T02:27:13.061 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.062 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.062 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.062 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-05-09T02:27:13.062 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.062 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.062 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.062 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T02:27:13.062 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.062 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.062 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.062 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.062 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.062 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-05-09T02:27:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.063 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.065 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.065 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.065 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.065 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.065 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.065 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.065 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.065 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.065 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.065 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.067 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.067 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.067 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.067 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.067 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.067 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.067 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.067 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.067 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.067 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.067 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.067 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.067 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.068 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.069 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.069 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-05-09T02:27:13.069 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.069 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.069 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.069 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.069 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.069 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.069 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.069 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.069 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.069 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.069 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.069 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.069 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.070 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.070 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.070 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.070 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.070 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.070 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.070 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.070 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.070 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.070 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.070 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.070 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.070 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.070 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.071 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.071 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.071 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.071 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.071 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.071 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.071 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.071 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.071 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.071 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.071 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.071 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.071 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.072 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.073 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.073 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.073 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.073 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.073 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.073 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.073 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.073 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.073 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.073 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.073 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.074 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.074 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.074 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.074 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.074 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.074 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.074 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.074 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.074 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.074 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.074 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.074 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.074 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.075 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.075 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.075 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.075 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-05-09T02:27:13.075 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.075 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.075 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.075 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-05-09T02:27:13.075 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-05-09T02:27:13.075 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.075 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.075 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.075 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-05-09T02:27:13.075 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.076 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.076 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.076 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T02:27:13.076 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.076 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.076 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.076 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-05-09T02:27:13.076 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-05-09T02:27:13.076 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.076 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.076 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.076 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:13.076 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-05-09T02:27:13.076 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.077 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.077 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T02:27:13.077 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.077 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.077 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.077 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.077 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.077 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.077 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.077 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.077 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.078 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.078 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.078 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.078 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.078 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.078 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.078 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.078 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.078 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.078 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.078 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.078 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.078 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.078 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.079 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.079 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.079 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.079 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.079 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.079 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.079 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.079 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.079 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.079 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.079 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.079 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.079 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.080 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.081 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.081 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.081 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.081 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.081 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.081 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.081 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.081 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.081 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.081 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.082 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.082 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.082 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.082 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.082 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.082 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.082 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.082 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.082 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.082 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.082 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.083 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.083 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.083 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.083 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.083 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.083 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-05-09T02:27:13.083 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.083 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.083 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.083 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-05-09T02:27:13.083 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-05-09T02:27:13.083 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.083 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.084 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.084 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-05-09T02:27:13.084 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.084 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.084 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.084 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-05-09T02:27:13.084 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T02:27:13.084 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.084 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.084 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.084 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.084 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.084 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.084 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.084 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.085 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.085 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.085 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.085 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.085 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.085 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.085 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.085 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.085 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.085 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.085 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.086 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.086 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.086 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.086 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.086 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.086 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.086 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.086 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.086 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.086 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.086 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.086 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.086 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.088 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.089 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.089 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.089 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.089 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.089 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.089 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-05-09T02:27:13.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-05-09T02:27:13.089 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.089 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.089 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.089 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.089 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.090 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-05-09T02:27:13.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T02:27:13.090 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.090 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.090 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-05-09T02:27:13.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-05-09T02:27:13.090 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.090 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.090 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.090 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-05-09T02:27:13.090 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.090 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.090 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.091 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-05-09T02:27:13.091 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.091 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.091 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.091 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-05-09T02:27:13.091 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-05-09T02:27:13.091 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.091 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.091 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.091 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-05-09T02:27:13.091 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.091 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.091 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.092 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T02:27:13.092 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.092 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.092 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.092 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-05-09T02:27:13.092 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-05-09T02:27:13.092 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.092 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.092 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.092 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.092 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.092 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.092 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-05-09T02:27:13.093 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.093 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.093 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.093 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.093 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.093 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.093 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.093 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.093 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.093 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.093 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.093 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.093 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.094 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.094 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.094 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.094 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.094 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.094 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-05-09T02:27:13.094 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-05-09T02:27:13.094 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.094 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.094 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.094 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-05-09T02:27:13.094 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.094 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.094 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.094 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T02:27:13.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T02:27:13.095 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.095 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.095 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.095 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.095 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.095 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.095 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-05-09T02:27:13.095 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.095 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.095 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.095 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.095 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.095 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.095 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.096 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.096 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.096 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.096 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.096 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.096 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.096 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.096 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.096 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.096 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.096 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.096 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.096 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.096 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.097 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.097 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.097 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.097 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.097 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.097 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.097 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.097 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.097 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.097 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.097 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.098 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.099 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.100 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.100 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.100 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.100 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.100 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.100 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.100 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.100 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.100 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.100 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.100 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.100 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.100 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.100 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.101 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.101 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.101 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.101 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.101 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.101 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.101 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.101 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.101 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-05-09T02:27:13.101 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.101 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.101 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.101 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.101 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.102 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.102 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.102 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.102 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.102 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.102 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.102 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.102 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.102 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.102 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.102 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.102 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.102 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.102 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.102 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.103 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.103 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.103 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.103 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.103 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.103 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.103 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.103 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.103 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.103 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.103 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.104 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.104 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.104 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.104 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.104 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.104 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.104 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.104 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.104 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.104 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.104 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.104 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.104 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.104 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.105 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.105 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.105 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.105 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.105 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.105 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.105 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.105 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.105 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.105 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.105 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.105 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.106 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.106 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.106 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.106 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.106 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.106 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.106 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.106 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.106 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.106 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.106 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.106 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.106 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.107 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.108 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.108 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.108 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.108 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.108 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.108 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.108 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.108 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.108 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.108 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.108 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.108 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.108 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.109 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.110 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.110 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.110 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.110 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.110 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.110 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.110 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.110 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.110 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.110 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.110 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.110 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.110 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.111 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.111 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.111 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.111 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.111 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.111 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.111 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.111 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.111 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.111 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.111 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.112 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.112 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.112 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.112 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.112 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.112 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.112 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.112 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.112 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.112 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.112 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.112 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-09T02:27:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.113 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.114 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.114 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-05-09T02:27:13.114 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-05-09T02:27:13.114 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.114 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.114 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.114 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.114 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.114 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.114 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.114 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T02:27:13.114 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.114 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.114 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.115 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.115 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.115 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.115 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.115 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.115 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.115 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.115 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.115 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.115 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.115 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.115 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.115 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.116 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.116 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.116 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.116 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.116 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.116 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.116 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.116 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.116 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.116 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.117 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.118 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.118 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.118 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.118 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.118 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.118 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.118 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.118 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.118 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.118 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.118 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.118 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.118 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.118 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-05-09T02:27:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-09T02:27:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.119 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.120 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.120 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.120 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.120 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.120 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-09T02:27:13.120 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-05-09T02:27:13.120 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-05-09T02:27:13.120 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.120 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-05-09T02:27:13.120 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.120 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.120 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.120 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.121 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.121 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.121 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.121 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.121 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.121 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.121 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.121 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.121 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.121 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.121 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.121 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.121 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-05-09T02:27:13.122 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.122 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.122 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.122 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.122 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.122 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.122 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.122 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.122 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.122 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.122 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.123 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.123 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.123 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.123 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.123 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.123 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.123 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.123 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.123 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.123 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.123 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.123 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.124 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.124 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.124 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.124 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.124 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.124 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.124 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.124 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.124 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.124 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.125 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.125 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.125 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.125 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.125 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.125 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.125 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.125 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.125 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.125 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.125 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.125 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.125 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.125 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.126 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.127 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.127 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.127 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.127 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.127 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.127 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.127 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.127 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.127 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.127 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.127 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.127 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.127 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.127 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.128 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.129 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.129 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.129 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.129 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.129 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.129 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.129 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.129 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.129 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.129 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.129 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.129 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.130 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.130 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.130 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.130 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.130 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.130 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.130 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.130 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.130 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.130 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.130 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.130 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.131 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.131 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.131 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.131 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.131 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.131 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.131 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.131 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.131 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.131 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.131 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.131 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.131 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.131 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.131 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.132 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.132 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.132 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.132 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.132 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.132 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.132 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.132 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.132 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.132 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.132 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.134 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.134 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.134 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.134 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.134 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.134 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-05-09T02:27:13.134 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.134 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.134 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.134 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.135 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.135 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.135 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.135 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.135 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.135 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.135 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.135 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.135 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.135 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.135 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.135 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.135 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.135 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.135 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.136 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.136 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.136 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.136 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.136 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.136 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.136 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.136 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.136 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.136 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.136 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.136 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.137 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.137 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.137 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.137 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.137 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.137 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.137 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.137 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.137 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.137 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.137 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.137 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.137 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.138 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.138 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.138 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.138 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.138 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.138 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.138 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.138 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.138 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.138 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.138 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.138 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T02:27:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T02:27:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T02:27:13.139 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.140 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.141 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.142 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-05-09T02:27:13.142 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T02:27:13.142 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.142 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.142 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.142 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.142 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.142 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-05-09T02:27:13.142 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.142 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.142 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.142 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-05-09T02:27:13.142 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-05-09T02:27:13.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.143 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.143 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.143 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.143 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.143 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.143 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.143 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T02:27:13.143 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.143 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.143 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.143 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-09T02:27:13.143 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-09T02:27:13.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.145 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.145 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.145 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.145 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.145 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.145 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.145 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.145 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.145 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.145 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.145 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.145 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.145 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.145 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.146 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.146 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.146 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.146 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.146 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.146 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.146 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.146 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.146 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.146 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.146 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.146 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.146 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.146 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.147 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.148 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-05-09T02:27:13.150 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.150 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.150 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.150 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-09T02:27:13.150 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-09T02:27:13.150 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:13.150 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.150 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.150 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-09T02:27:13.150 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.150 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.150 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-05-09T02:27:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-05-09T02:27:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-05-09T02:27:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-05-09T02:27:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.152 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.153 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:13.153 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.153 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.153 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.153 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.153 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.153 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.153 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.153 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-05-09T02:27:13.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-05-09T02:27:13.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.154 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:13.154 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.154 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.154 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.154 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T02:27:13.154 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T02:27:13.154 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.154 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.154 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.154 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.154 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.154 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.154 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.155 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.155 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.155 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.155 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.155 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.155 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.155 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.155 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.155 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.155 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.155 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.155 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.155 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.155 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.155 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.157 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.159 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.161 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-05-09T02:27:13.161 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-05-09T02:27:13.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:13.161 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.161 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-05-09T02:27:13.161 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.161 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.161 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.161 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.161 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.161 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.161 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.162 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T02:27:13.162 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T02:27:13.162 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.162 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.162 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.162 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-09T02:27:13.162 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.162 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.162 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.162 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T02:27:13.162 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.162 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.162 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.163 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T02:27:13.163 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T02:27:13.163 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.163 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.163 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.163 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-05-09T02:27:13.163 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.163 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.163 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.163 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T02:27:13.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.163 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.163 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.163 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-05-09T02:27:13.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-05-09T02:27:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-05-09T02:27:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-09T02:27:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-09T02:27:13.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.166 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.166 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.166 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-05-09T02:27:13.166 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.166 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.166 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.166 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-05-09T02:27:13.166 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-05-09T02:27:13.166 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.166 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.166 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.166 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.166 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.167 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-09T02:27:13.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-09T02:27:13.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.167 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.167 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.167 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-09T02:27:13.167 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-09T02:27:13.167 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-09T02:27:13.167 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.167 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.167 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.167 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.167 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.167 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.167 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-09T02:27:13.168 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.168 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.168 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.168 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.168 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.168 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.168 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.168 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.168 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.168 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.169 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.169 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.169 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.169 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.169 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.169 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.169 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.169 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.169 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.169 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.169 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.169 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.169 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.169 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.170 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.170 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.170 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.170 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.170 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.170 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.170 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.170 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.170 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.170 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.170 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.170 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.171 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.171 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.171 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.171 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.171 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.171 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.171 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.171 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.171 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.171 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.171 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.171 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.171 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.171 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-05-09T02:27:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-05-09T02:27:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-09T02:27:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.173 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.174 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-05-09T02:27:13.174 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T02:27:13.174 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.174 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.174 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.174 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.174 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.174 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.174 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.174 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.174 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.174 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.174 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.176 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-05-09T02:27:13.176 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-05-09T02:27:13.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.176 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.176 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.176 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.176 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.176 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.176 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.176 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.176 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.176 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.176 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-05-09T02:27:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-05-09T02:27:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-05-09T02:27:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.178 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.178 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.178 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.178 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.178 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.178 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-09T02:27:13.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-09T02:27:13.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.178 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.178 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.178 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.178 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.178 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.178 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.179 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.180 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.180 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.180 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.180 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.180 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.180 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.180 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.180 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.180 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.180 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.180 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.180 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.181 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.181 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.181 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.181 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.181 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.181 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.181 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.181 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.181 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.183 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.183 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.183 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.183 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.183 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.183 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.183 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.183 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.183 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-09T02:27:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-09T02:27:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-09T02:27:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-09T02:27:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.186 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.186 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.186 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.186 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.186 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.186 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.186 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.186 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.186 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.186 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.186 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.187 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-05-09T02:27:13.187 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.187 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.187 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.187 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.187 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.187 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.187 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.187 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.187 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.187 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.188 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.188 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.188 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.188 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.188 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.188 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.188 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.188 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.188 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.188 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.188 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.188 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.188 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.188 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.189 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.190 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.191 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.191 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.191 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.191 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.191 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.191 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.191 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.191 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.191 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.191 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.191 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.192 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.192 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.192 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.192 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.192 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.192 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.192 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.192 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.192 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.192 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.192 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.192 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.192 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.193 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.193 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.193 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.193 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-05-09T02:27:13.193 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.193 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.193 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.193 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.193 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.193 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.193 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.194 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.194 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.194 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.194 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.194 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.194 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.194 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.194 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.194 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.194 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.194 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.196 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.196 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.196 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.196 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.196 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.196 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.196 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.196 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.196 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.196 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.196 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.196 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.197 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.198 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.198 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.198 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.198 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.198 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.198 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.198 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.198 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.198 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.198 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.198 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-05-09T02:27:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-05-09T02:27:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-05-09T02:27:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T02:27:13.199 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-05-09T02:27:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.200 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.201 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.201 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.201 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.201 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.201 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.201 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.201 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.201 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.201 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.201 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.201 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.201 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.201 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.201 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.201 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.202 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.203 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.203 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.203 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.203 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.203 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.203 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.203 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.203 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.203 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.203 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.204 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.204 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.204 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.204 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.204 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.204 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.204 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.204 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.204 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.204 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.204 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-05-09T02:27:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-05-09T02:27:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.206 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.206 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.206 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.206 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.206 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.206 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.206 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-05-09T02:27:13.206 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-05-09T02:27:13.206 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.206 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.207 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-05-09T02:27:13.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.207 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.207 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.207 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.207 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.207 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.207 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-05-09T02:27:13.207 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-05-09T02:27:13.207 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.208 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.208 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.208 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.208 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.208 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.208 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.208 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.208 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-05-09T02:27:13.208 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-05-09T02:27:13.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.208 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.208 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.208 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.209 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.209 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.209 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.209 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.209 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-05-09T02:27:13.209 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-05-09T02:27:13.209 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-09T02:27:13.209 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.209 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.209 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.210 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.210 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.210 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.210 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.210 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-05-09T02:27:13.210 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-05-09T02:27:13.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.210 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.210 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.210 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.211 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.211 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.211 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.211 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.211 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.211 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-05-09T02:27:13.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-05-09T02:27:13.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.211 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.211 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.211 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.211 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.212 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.212 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.212 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.212 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.212 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-05-09T02:27:13.212 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-05-09T02:27:13.213 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.213 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.213 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:13.213 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.213 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.213 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.213 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.213 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.213 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.214 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.214 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-05-09T02:27:13.214 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-05-09T02:27:13.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.214 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.214 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.214 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-09T02:27:13.214 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-05-09T02:27:13.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.214 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.214 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.214 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.214 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-05-09T02:27:13.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.217 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.217 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.217 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.217 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.217 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.217 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.217 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-05-09T02:27:13.217 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-05-09T02:27:13.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.217 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.217 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.218 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.218 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.218 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.218 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.218 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.218 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.218 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.218 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.218 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.218 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.218 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smb", 2024-05-09T02:27:13.218 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.218 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.219 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.219 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.219 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.219 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.219 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.219 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.219 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.219 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.219 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.219 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.219 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.219 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.219 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.220 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.220 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.220 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.220 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.220 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.220 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.220 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.220 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.222 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.222 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.222 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.222 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.222 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.222 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.222 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.222 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.222 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.222 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.222 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.222 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.223 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.223 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.223 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.223 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.223 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.223 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.223 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.223 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.223 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.223 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: "update_orchestration": { 2024-05-09T02:27:13.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "update_orchestration", 2024-05-09T02:27:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T02:27:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.225 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.226 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-09T02:27:13.226 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.226 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.226 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.226 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-05-09T02:27:13.226 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-05-09T02:27:13.226 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.226 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.226 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.226 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.226 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.226 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.226 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.226 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T02:27:13.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.227 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.227 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.227 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-05-09T02:27:13.227 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-05-09T02:27:13.227 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.227 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.227 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.227 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.227 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.227 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.227 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-05-09T02:27:13.228 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.228 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.228 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.228 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.228 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.228 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.228 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.228 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.228 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.228 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.229 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.229 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.229 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.229 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.229 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.229 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.229 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.229 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.229 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.229 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.229 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.229 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.230 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.230 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.230 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.230 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.230 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.230 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.230 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.230 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.230 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.231 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-05-09T02:27:13.233 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.234 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.234 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.234 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.234 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.234 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.234 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.234 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.234 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.234 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.234 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.234 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.234 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.234 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.235 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.235 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.235 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.235 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.235 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.235 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.235 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.235 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.235 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.235 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.235 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.235 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.235 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.237 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.237 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.237 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.237 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.237 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.237 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.237 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.237 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.237 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.237 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.237 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.237 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.238 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.238 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.238 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.238 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.238 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.238 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.238 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.238 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.238 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.238 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.238 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.238 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.239 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.239 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.239 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.239 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.239 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.239 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.239 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.239 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.239 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.239 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-05-09T02:27:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.244 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.244 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.244 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.244 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.244 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.244 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.244 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.244 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.244 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.244 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.244 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.245 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.245 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.245 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.245 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.245 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.245 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.245 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.245 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.245 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.245 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.245 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.246 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-05-09T02:27:13.246 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.246 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.246 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.246 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-05-09T02:27:13.246 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-05-09T02:27:13.246 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.246 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.246 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.246 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T02:27:13.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.246 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.246 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.246 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T02:27:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T02:27:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-05-09T02:27:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.248 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.248 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.248 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.248 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.248 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.248 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.248 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.248 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.248 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.249 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.250 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.250 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.250 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.250 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.250 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.250 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.250 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.250 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.250 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.250 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.250 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.250 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.251 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.251 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.251 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.251 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.251 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.251 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.251 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.251 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.251 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.251 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.251 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.251 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.252 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.252 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.252 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.252 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.252 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.252 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.252 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.252 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.252 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.253 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.253 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.253 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.253 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.253 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.253 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-05-09T02:27:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-05-09T02:27:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-05-09T02:27:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T02:27:13.255 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.255 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.255 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.255 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-05-09T02:27:13.255 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-05-09T02:27:13.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.255 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.255 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.255 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:13.255 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.255 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.255 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T02:27:13.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.256 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.256 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.256 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-05-09T02:27:13.256 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-05-09T02:27:13.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.256 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.256 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.256 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:13.256 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.256 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.256 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T02:27:13.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.257 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.257 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.257 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-05-09T02:27:13.257 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-05-09T02:27:13.257 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.257 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.257 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.257 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.257 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.257 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.257 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T02:27:13.257 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.257 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.258 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-05-09T02:27:13.258 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-05-09T02:27:13.258 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.258 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.258 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.258 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.258 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.258 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.258 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T02:27:13.258 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.258 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.259 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.259 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-05-09T02:27:13.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-05-09T02:27:13.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.259 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.259 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.259 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.259 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.259 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.259 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.261 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-05-09T02:27:13.261 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-05-09T02:27:13.261 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.261 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T02:27:13.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.261 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.261 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.261 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.261 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.261 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.261 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-09T02:27:13.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-09T02:27:13.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.262 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.262 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.262 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.262 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.262 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.262 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.262 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.262 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.262 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.262 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T02:27:13.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.263 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.264 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.264 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-05-09T02:27:13.264 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.264 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.264 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.264 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.264 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.264 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.264 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.264 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-05-09T02:27:13.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-05-09T02:27:13.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.265 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.266 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.266 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.266 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.266 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.266 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.266 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.266 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.266 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.266 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.266 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.268 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.268 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.268 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.268 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.268 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.268 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.268 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.268 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.268 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.268 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.268 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.269 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.269 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.269 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.269 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.269 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.269 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.269 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.269 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.269 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.269 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.269 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.269 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.269 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.269 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.270 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.270 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.270 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.270 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.270 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.270 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.270 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.270 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.271 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.271 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.271 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.271 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-05-09T02:27:13.271 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-05-09T02:27:13.271 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.271 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.271 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.271 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.271 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.271 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-05-09T02:27:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-05-09T02:27:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.273 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.273 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.273 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.273 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.273 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.273 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.273 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.273 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.273 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.273 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.273 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.273 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-05-09T02:27:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.275 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.276 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.277 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.277 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.277 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.277 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.277 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.277 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.277 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.277 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.277 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.277 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.278 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.278 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.278 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.278 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.278 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.278 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.278 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.278 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.278 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.278 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.278 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.278 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.278 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.278 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.278 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.279 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.280 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.280 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.280 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.280 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.280 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.280 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.280 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.280 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.280 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.280 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.280 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.282 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.282 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.282 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.282 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.282 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.282 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.282 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.282 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.282 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.282 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.282 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.282 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.284 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.284 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.284 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.284 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.284 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.284 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.284 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.284 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.284 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.284 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.284 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.284 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.284 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.284 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.285 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.285 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.285 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.285 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.285 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.285 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.285 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.285 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.285 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.285 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-05-09T02:27:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-05-09T02:27:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-05-09T02:27:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T02:27:13.286 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-05-09T02:27:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-05-09T02:27:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-05-09T02:27:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.287 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-05-09T02:27:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-05-09T02:27:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T02:27:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.288 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.289 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-05-09T02:27:13.289 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T02:27:13.289 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.289 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.289 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-09T02:27:13.289 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.289 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.289 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.289 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T02:27:13.289 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.289 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.290 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.290 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.290 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.290 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.290 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.290 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.290 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.290 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.290 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.290 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.290 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-05-09T02:27:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-05-09T02:27:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-09T02:27:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-09T02:27:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-09T02:27:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-09T02:27:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-09T02:27:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-09T02:27:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.294 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-09T02:27:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-09T02:27:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.295 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-09T02:27:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-09T02:27:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-09T02:27:13.296 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.297 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-09T02:27:13.297 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-09T02:27:13.297 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-09T02:27:13.297 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-09T02:27:13.297 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-09T02:27:13.297 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-09T02:27:13.297 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-09T02:27:13.297 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.297 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.297 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.297 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.297 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.297 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-09T02:27:13.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-09T02:27:13.298 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-09T02:27:13.298 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.298 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.298 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-09T02:27:13.298 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.298 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.298 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-09T02:27:13.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-09T02:27:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-09T02:27:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-09T02:27:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-09T02:27:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-05-09T02:27:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-05-09T02:27:13.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-09T02:27:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-05-09T02:27:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-05-09T02:27:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-09T02:27:13.300 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-09T02:27:13.301 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T02:27:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-09T02:27:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-09T02:27:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-09T02:27:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-09T02:27:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-09T02:27:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-09T02:27:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-09T02:27:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: "services": { 2024-05-09T02:27:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard": "https://172.21.15.19:8443/" 2024-05-09T02:27:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.302 INFO:teuthology.orchestra.run.smithi019.stdout: "always_on_modules": { 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus": [ 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific": [ 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-09T02:27:13.303 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-09T02:27:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-09T02:27:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-09T02:27:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-09T02:27:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-09T02:27:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-09T02:27:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-09T02:27:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-09T02:27:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy": [ 2024-05-09T02:27:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-09T02:27:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-09T02:27:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-09T02:27:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-09T02:27:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-09T02:27:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: "reef": [ 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-09T02:27:13.305 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-09T02:27:13.306 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-09T02:27:13.306 INFO:teuthology.orchestra.run.smithi019.stdout: "squid": [ 2024-05-09T02:27:13.306 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-09T02:27:13.306 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-09T02:27:13.306 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-09T02:27:13.306 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-09T02:27:13.306 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-09T02:27:13.306 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-09T02:27:13.306 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-09T02:27:13.306 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-09T02:27:13.306 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-09T02:27:13.306 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-09T02:27:13.306 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T02:27:13.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.306 INFO:teuthology.orchestra.run.smithi019.stdout: "last_failure_osd_epoch": 82, 2024-05-09T02:27:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: "active_clients": [ 2024-05-09T02:27:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-09T02:27:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-09T02:27:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-09T02:27:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-09T02:27:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1652701186 2024-05-09T02:27:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T02:27:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "libcephsqlite", 2024-05-09T02:27:13.307 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1034714846 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2002833662 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.308 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T02:27:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-09T02:27:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-09T02:27:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-09T02:27:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-09T02:27:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 4115869140 2024-05-09T02:27:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T02:27:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-09T02:27:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-09T02:27:13.309 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3714554182 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2791270785 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T02:27:13.310 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:13.311 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-09T02:27:13.311 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:27:13.312 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:11 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3001463893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T02:27:13.312 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:11 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1423031339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T02:27:13.312 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:11 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1388875243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T02:27:13.312 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:12 smithi196 ceph-mon[34718]: pgmap v499: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 4.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 123 KiB/s wr, 35 op/s 2024-05-09T02:27:13.312 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:12 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:27:13.312 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:12 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3202427488' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T02:27:13.312 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:11 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3001463893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T02:27:13.312 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:11 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1423031339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T02:27:13.313 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:11 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1388875243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T02:27:13.313 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:12 smithi059 ceph-mon[34781]: pgmap v499: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 4.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 123 KiB/s wr, 35 op/s 2024-05-09T02:27:13.313 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:12 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:27:13.313 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:12 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3202427488' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T02:27:13.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.891+0000 7fc6a6eb8640 1 -- 172.21.15.19:0/3202427488 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc6880b29e0 msgr2=0x7fc6880b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:13.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.892+0000 7fc6a6eb8640 1 --2- 172.21.15.19:0/3202427488 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc6880b29e0 0x7fc6880b4ef0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0103800 tx=0x7fc69c005f90 comp rx=0 tx=0).stop 2024-05-09T02:27:13.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.892+0000 7fc6a6eb8640 1 -- 172.21.15.19:0/3202427488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a0108fc0 msgr2=0x7fc6a019a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:13.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.892+0000 7fc6a6eb8640 1 --2- 172.21.15.19:0/3202427488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a0108fc0 0x7fc6a019a6c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc69800eb10 tx=0x7fc69800c2d0 comp rx=0 tx=0).stop 2024-05-09T02:27:13.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.892+0000 7fc6a6eb8640 1 -- 172.21.15.19:0/3202427488 shutdown_connections 2024-05-09T02:27:13.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.892+0000 7fc6a6eb8640 1 --2- 172.21.15.19:0/3202427488 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fc6880b29e0 0x7fc6880b4ef0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:13.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.892+0000 7fc6a6eb8640 1 --2- 172.21.15.19:0/3202427488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a0108fc0 0x7fc6a019a6c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:13.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.892+0000 7fc6a6eb8640 1 --2- 172.21.15.19:0/3202427488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a0108b20 0x7fc6a01a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:13.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.892+0000 7fc6a6eb8640 1 --2- 172.21.15.19:0/3202427488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6a0103270 0x7fc6a01a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:13.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.892+0000 7fc6a6eb8640 1 -- 172.21.15.19:0/3202427488 >> 172.21.15.19:0/3202427488 conn(0x7fc6a0078240 msgr2=0x7fc6a01061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:13.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.892+0000 7fc6a6eb8640 1 -- 172.21.15.19:0/3202427488 shutdown_connections 2024-05-09T02:27:13.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:11.892+0000 7fc6a6eb8640 1 -- 172.21.15.19:0/3202427488 wait complete. 2024-05-09T02:27:13.320 DEBUG:tasks.check_counter:Getting stats from a 2024-05-09T02:27:13.321 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-09T02:27:13.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:12 smithi019 ceph-mon[30151]: pgmap v499: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 4.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 123 KiB/s wr, 35 op/s 2024-05-09T02:27:13.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:12 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:27:13.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:12 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3202427488' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T02:27:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:27:14.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:14 smithi019 ceph-mon[30151]: pgmap v500: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 139 KiB/s wr, 34 op/s 2024-05-09T02:27:14.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:14 smithi059 ceph-mon[34781]: pgmap v500: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 139 KiB/s wr, 34 op/s 2024-05-09T02:27:15.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:14 smithi196 ceph-mon[34718]: pgmap v500: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 139 KiB/s wr, 34 op/s 2024-05-09T02:27:15.353 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:27:15.353 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T02:27:15.353 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2519, 2024-05-09T02:27:15.353 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2519, 2024-05-09T02:27:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 4130551, 2024-05-09T02:27:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 800219, 2024-05-09T02:27:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-05-09T02:27:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-05-09T02:27:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.269651054, 2024-05-09T02:27:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.067340092, 2024-05-09T02:27:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.070579262, 2024-05-09T02:27:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.112247446, 2024-05-09T02:27:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2518, 2024-05-09T02:27:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.151181708, 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000060040 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 907, 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1146, 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 223602, 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 874514, 2024-05-09T02:27:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 8, 2024-05-09T02:27:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-05-09T02:27:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.109307389, 2024-05-09T02:27:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.035219295, 2024-05-09T02:27:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.024054402, 2024-05-09T02:27:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.038564752, 2024-05-09T02:27:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1143, 2024-05-09T02:27:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.071733585, 2024-05-09T02:27:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000062759 2024-05-09T02:27:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T02:27:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 586656 2024-05-09T02:27:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T02:27:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 387, 2024-05-09T02:27:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 343, 2024-05-09T02:27:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1993532, 2024-05-09T02:27:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 199518, 2024-05-09T02:27:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-05-09T02:27:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-05-09T02:27:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.066161090, 2024-05-09T02:27:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.021401218, 2024-05-09T02:27:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.050562664, 2024-05-09T02:27:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.007034765, 2024-05-09T02:27:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 342, 2024-05-09T02:27:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.020021604, 2024-05-09T02:27:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000058542 2024-05-09T02:27:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1993264, 2024-05-09T02:27:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 199200 2024-05-09T02:27:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-09T02:27:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 5805, 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.202778259, 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000034931 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-05-09T02:27:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-05-09T02:27:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-05-09T02:27:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-05-09T02:27:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-05-09T02:27:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-05-09T02:27:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-05-09T02:27:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 753, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 20, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 733, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 725, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 30, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-05-09T02:27:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-05-09T02:27:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-05-09T02:27:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-05-09T02:27:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-05-09T02:27:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-05-09T02:27:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-05-09T02:27:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-05-09T02:27:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-05-09T02:27:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-05-09T02:27:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-05-09T02:27:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-05-09T02:27:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-05-09T02:27:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-05-09T02:27:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-05-09T02:27:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-05-09T02:27:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-05-09T02:27:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T02:27:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-05-09T02:27:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T02:27:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-05-09T02:27:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-05-09T02:27:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T02:27:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T02:27:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-05-09T02:27:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-05-09T02:27:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T02:27:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-05-09T02:27:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T02:27:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T02:27:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T02:27:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-05-09T02:27:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-05-09T02:27:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-05-09T02:27:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-05-09T02:27:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-05-09T02:27:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-05-09T02:27:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-05-09T02:27:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-05-09T02:27:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-05-09T02:27:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-05-09T02:27:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T02:27:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-05-09T02:27:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-05-09T02:27:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-05-09T02:27:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-05-09T02:27:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-05-09T02:27:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-05-09T02:27:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-05-09T02:27:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-05-09T02:27:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-05-09T02:27:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-05-09T02:27:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 2623, 2024-05-09T02:27:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-05-09T02:27:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-05-09T02:27:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-05-09T02:27:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-05-09T02:27:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-05-09T02:27:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 4, 2024-05-09T02:27:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-05-09T02:27:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-05-09T02:27:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-05-09T02:27:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-05-09T02:27:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 7582346, 2024-05-09T02:27:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 7581772, 2024-05-09T02:27:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-05-09T02:27:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 2623 2024-05-09T02:27:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-05-09T02:27:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 612, 2024-05-09T02:27:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 638, 2024-05-09T02:27:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 26, 2024-05-09T02:27:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 40, 2024-05-09T02:27:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 43, 2024-05-09T02:27:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 3, 2024-05-09T02:27:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 753, 2024-05-09T02:27:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 753, 2024-05-09T02:27:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 0, 2024-05-09T02:27:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-05-09T02:27:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-05-09T02:27:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-05-09T02:27:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 123492, 2024-05-09T02:27:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215440 2024-05-09T02:27:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-05-09T02:27:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-05-09T02:27:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-05-09T02:27:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-05-09T02:27:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-05-09T02:27:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-05-09T02:27:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-05-09T02:27:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-05-09T02:27:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-05-09T02:27:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-05-09T02:27:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-05-09T02:27:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-05-09T02:27:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-05-09T02:27:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-05-09T02:27:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-05-09T02:27:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-05-09T02:27:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-05-09T02:27:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-05-09T02:27:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-05-09T02:27:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-05-09T02:27:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-05-09T02:27:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-05-09T02:27:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-05-09T02:27:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-05-09T02:27:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-05-09T02:27:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-05-09T02:27:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-05-09T02:27:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-05-09T02:27:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-05-09T02:27:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-05-09T02:27:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-05-09T02:27:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-05-09T02:27:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-05-09T02:27:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-05-09T02:27:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-05-09T02:27:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-05-09T02:27:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 1, 2024-05-09T02:27:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 1, 2024-05-09T02:27:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 0, 2024-05-09T02:27:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 1, 2024-05-09T02:27:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-05-09T02:27:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-05-09T02:27:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-05-09T02:27:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 2169, 2024-05-09T02:27:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-05-09T02:27:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-09T02:27:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-09T02:27:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-09T02:27:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T02:27:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-09T02:27:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T02:27:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-09T02:27:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T02:27:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T02:27:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T02:27:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T02:27:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T02:27:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-09T02:27:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T02:27:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T02:27:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-09T02:27:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-09T02:27:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-09T02:27:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-09T02:27:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T02:27:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T02:27:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T02:27:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T02:27:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T02:27:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-09T02:27:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-09T02:27:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-09T02:27:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T02:27:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T02:27:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-09T02:27:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-09T02:27:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-09T02:27:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 215735, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 724, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 8344, 2024-05-09T02:27:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 174, 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 2097093, 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 36517, 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 3421, 2024-05-09T02:27:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 21, 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 3, 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 3421, 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3421, 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 7.900864763, 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.002309519 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3421, 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 3422 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 3421, 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 3421, 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-05-09T02:27:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-05-09T02:27:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-05-09T02:27:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 2042, 2024-05-09T02:27:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-05-09T02:27:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 1377, 2024-05-09T02:27:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-05-09T02:27:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-05-09T02:27:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-05-09T02:27:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-05-09T02:27:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-05-09T02:27:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-05-09T02:27:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-05-09T02:27:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-05-09T02:27:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-05-09T02:27:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-05-09T02:27:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-05-09T02:27:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-05-09T02:27:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-05-09T02:27:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-05-09T02:27:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-05-09T02:27:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-05-09T02:27:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-05-09T02:27:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-05-09T02:27:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-05-09T02:27:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-05-09T02:27:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-05-09T02:27:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-05-09T02:27:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-05-09T02:27:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 102, 2024-05-09T02:27:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-05-09T02:27:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 23, 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 5, 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 7, 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 2, 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-05-09T02:27:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-05-09T02:27:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-05-09T02:27:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-05-09T02:27:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T02:27:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-05-09T02:27:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-05-09T02:27:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-05-09T02:27:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-05-09T02:27:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-05-09T02:27:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T02:27:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T02:27:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-09T02:27:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T02:27:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 3809, 2024-05-09T02:27:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 5982271, 2024-05-09T02:27:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 3809, 2024-05-09T02:27:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T02:27:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T02:27:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3809, 2024-05-09T02:27:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 5982271, 2024-05-09T02:27:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T02:27:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-05-09T02:27:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T02:27:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-09T02:27:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T02:27:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T02:27:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T02:27:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 3421, 2024-05-09T02:27:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 3388047, 2024-05-09T02:27:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3421, 2024-05-09T02:27:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 3388047, 2024-05-09T02:27:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T02:27:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.392 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 3421, 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 3421, 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3421, 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 3421, 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T02:27:15.393 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-05-09T02:27:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T02:27:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-05-09T02:27:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T02:27:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T02:27:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T02:27:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T02:27:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55be3a6128f0": { 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T02:27:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T02:27:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-05-09T02:27:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-05-09T02:27:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T02:27:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:15.396 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:27:15.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:15 smithi019 ceph-mon[30151]: osdmap e102: 12 total, 12 up, 12 in 2024-05-09T02:27:15.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:15 smithi019 ceph-mon[30151]: pgmap v502: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 90 KiB/s wr, 18 op/s 2024-05-09T02:27:15.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:15 smithi059 ceph-mon[34781]: osdmap e102: 12 total, 12 up, 12 in 2024-05-09T02:27:15.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:15 smithi059 ceph-mon[34781]: pgmap v502: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 90 KiB/s wr, 18 op/s 2024-05-09T02:27:16.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:15 smithi196 ceph-mon[34718]: osdmap e102: 12 total, 12 up, 12 in 2024-05-09T02:27:16.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:15 smithi196 ceph-mon[34718]: pgmap v502: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 90 KiB/s wr, 18 op/s 2024-05-09T02:27:16.157 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-09T02:27:16.157 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-09T02:27:16.157 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-05-09T02:27:16.157 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-05-09T02:27:16.157 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T02:27:16.157 DEBUG:tasks.check_counter:Getting stats from d 2024-05-09T02:27:16.158 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-09T02:27:16.412 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:27:17.933 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:17 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:27:17.933 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:17 smithi019 ceph-mon[30151]: pgmap v503: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 100 KiB/s wr, 18 op/s 2024-05-09T02:27:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:17 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:27:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:17 smithi059 ceph-mon[34781]: pgmap v503: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 100 KiB/s wr, 18 op/s 2024-05-09T02:27:18.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:17 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:27:18.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:17 smithi196 ceph-mon[34718]: pgmap v503: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 100 KiB/s wr, 18 op/s 2024-05-09T02:27:18.365 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:27:18.365 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T02:27:18.365 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 4, 2024-05-09T02:27:18.365 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 4, 2024-05-09T02:27:18.365 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 2944, 2024-05-09T02:27:18.365 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 448, 2024-05-09T02:27:18.365 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-05-09T02:27:18.366 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-05-09T02:27:18.366 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.001719229, 2024-05-09T02:27:18.366 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000038728, 2024-05-09T02:27:18.366 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000595231, 2024-05-09T02:27:18.366 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T02:27:18.366 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:18.366 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3, 2024-05-09T02:27:18.366 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000815717, 2024-05-09T02:27:18.366 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000271905 2024-05-09T02:27:18.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:18.366 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:18.366 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:18.366 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:18.366 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:18.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:18.367 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T02:27:18.367 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T02:27:18.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:18.367 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T02:27:18.367 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 332, 2024-05-09T02:27:18.367 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 321, 2024-05-09T02:27:18.367 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1742128, 2024-05-09T02:27:18.367 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 198320, 2024-05-09T02:27:18.367 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-05-09T02:27:18.367 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-05-09T02:27:18.367 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.063097716, 2024-05-09T02:27:18.367 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.019846934, 2024-05-09T02:27:18.367 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.044456905, 2024-05-09T02:27:18.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.009655687, 2024-05-09T02:27:18.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:18.368 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 320, 2024-05-09T02:27:18.368 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.024330889, 2024-05-09T02:27:18.368 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000076034 2024-05-09T02:27:18.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:18.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:18.368 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:18.368 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:18.368 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:18.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:18.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1742128, 2024-05-09T02:27:18.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 198320 2024-05-09T02:27:18.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:18.368 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T02:27:18.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 3, 2024-05-09T02:27:18.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 247, 2024-05-09T02:27:18.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 480, 2024-05-09T02:27:18.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 279584, 2024-05-09T02:27:18.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-05-09T02:27:18.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-05-09T02:27:18.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.019715339, 2024-05-09T02:27:18.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.013982506, 2024-05-09T02:27:18.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002556132, 2024-05-09T02:27:18.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T02:27:18.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:18.369 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 244, 2024-05-09T02:27:18.369 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.023940425, 2024-05-09T02:27:18.369 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000098116 2024-05-09T02:27:18.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:18.370 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:18.370 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:18.370 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:18.370 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:18.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:18.370 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-09T02:27:18.370 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 279584 2024-05-09T02:27:18.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:18.370 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-09T02:27:18.370 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-05-09T02:27:18.370 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-09T02:27:18.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:18.370 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-09T02:27:18.370 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-09T02:27:18.371 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-09T02:27:18.371 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T02:27:18.371 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-09T02:27:18.371 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T02:27:18.371 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-09T02:27:18.371 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T02:27:18.371 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T02:27:18.371 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T02:27:18.371 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T02:27:18.371 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T02:27:18.371 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-09T02:27:18.371 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T02:27:18.371 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T02:27:18.371 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-09T02:27:18.371 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-09T02:27:18.372 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-09T02:27:18.372 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-09T02:27:18.372 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T02:27:18.372 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T02:27:18.372 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T02:27:18.372 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T02:27:18.372 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T02:27:18.372 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-09T02:27:18.372 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-09T02:27:18.372 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-09T02:27:18.372 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T02:27:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T02:27:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-09T02:27:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-09T02:27:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-09T02:27:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-09T02:27:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T02:27:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-09T02:27:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T02:27:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-09T02:27:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 147898, 2024-05-09T02:27:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 27, 2024-05-09T02:27:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-05-09T02:27:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-05-09T02:27:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-09T02:27:18.373 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-09T02:27:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-09T02:27:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-09T02:27:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-09T02:27:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-09T02:27:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-05-09T02:27:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-05-09T02:27:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T02:27:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-09T02:27:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-09T02:27:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-09T02:27:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-05-09T02:27:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-05-09T02:27:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-09T02:27:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-09T02:27:18.374 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-09T02:27:18.375 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-09T02:27:18.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:18.375 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T02:27:18.375 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T02:27:18.375 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-09T02:27:18.375 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T02:27:18.375 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 335, 2024-05-09T02:27:18.375 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1695258, 2024-05-09T02:27:18.375 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:18.375 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 335, 2024-05-09T02:27:18.375 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T02:27:18.375 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T02:27:18.375 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 335, 2024-05-09T02:27:18.375 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1695258, 2024-05-09T02:27:18.376 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T02:27:18.376 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:18.376 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:18.376 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:18.376 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:18.376 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:18.376 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:27:19.111 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-09T02:27:19.111 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-09T02:27:19.111 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-09T02:27:19.111 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-09T02:27:19.111 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-09T02:27:19.111 DEBUG:tasks.check_counter:Getting stats from g 2024-05-09T02:27:19.112 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-09T02:27:19.367 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:27:20.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:20 smithi019 ceph-mon[30151]: pgmap v504: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 116 KiB/s wr, 18 op/s 2024-05-09T02:27:20.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:20 smithi196 ceph-mon[34718]: pgmap v504: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 116 KiB/s wr, 18 op/s 2024-05-09T02:27:20.969 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:20 smithi059 ceph-mon[34781]: pgmap v504: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 116 KiB/s wr, 18 op/s 2024-05-09T02:27:21.386 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:27:21.386 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T02:27:21.386 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1140, 2024-05-09T02:27:21.386 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1142, 2024-05-09T02:27:21.386 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 382985, 2024-05-09T02:27:21.386 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 362521, 2024-05-09T02:27:21.386 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-05-09T02:27:21.386 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-05-09T02:27:21.386 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.142283846, 2024-05-09T02:27:21.386 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.052432630, 2024-05-09T02:27:21.387 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.034806123, 2024-05-09T02:27:21.387 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.042292841, 2024-05-09T02:27:21.387 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:21.387 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1140, 2024-05-09T02:27:21.387 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.074995114, 2024-05-09T02:27:21.387 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000065785 2024-05-09T02:27:21.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.387 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:21.387 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.387 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.387 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T02:27:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T02:27:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T02:27:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2662, 2024-05-09T02:27:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2617, 2024-05-09T02:27:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 6515411, 2024-05-09T02:27:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 923515, 2024-05-09T02:27:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-05-09T02:27:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-05-09T02:27:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.265741195, 2024-05-09T02:27:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.065906421, 2024-05-09T02:27:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.105194221, 2024-05-09T02:27:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.094875922, 2024-05-09T02:27:21.389 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:21.389 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2616, 2024-05-09T02:27:21.389 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.138906162, 2024-05-09T02:27:21.389 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000053098 2024-05-09T02:27:21.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.389 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:21.389 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.389 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.389 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.389 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 2001168, 2024-05-09T02:27:21.389 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 201280 2024-05-09T02:27:21.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.389 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T02:27:21.390 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 9, 2024-05-09T02:27:21.390 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 248, 2024-05-09T02:27:21.390 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 4612, 2024-05-09T02:27:21.390 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 591653, 2024-05-09T02:27:21.390 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 8, 2024-05-09T02:27:21.390 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-05-09T02:27:21.390 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.020609152, 2024-05-09T02:27:21.390 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.013934270, 2024-05-09T02:27:21.390 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.003541935, 2024-05-09T02:27:21.390 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000061307, 2024-05-09T02:27:21.390 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:21.391 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 246, 2024-05-09T02:27:21.391 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.020765942, 2024-05-09T02:27:21.391 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000084414 2024-05-09T02:27:21.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.391 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:21.391 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.391 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.391 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.391 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-09T02:27:21.391 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 590688 2024-05-09T02:27:21.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.392 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-09T02:27:21.392 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-05-09T02:27:21.392 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-09T02:27:21.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.392 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-05-09T02:27:21.392 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-05-09T02:27:21.392 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-05-09T02:27:21.392 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 5796, 2024-05-09T02:27:21.393 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.181546673, 2024-05-09T02:27:21.393 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000031322 2024-05-09T02:27:21.393 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:21.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.393 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-05-09T02:27:21.393 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-05-09T02:27:21.393 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-05-09T02:27:21.393 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.393 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.393 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.393 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:21.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.393 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-05-09T02:27:21.393 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-05-09T02:27:21.394 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-05-09T02:27:21.394 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-05-09T02:27:21.394 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.394 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.394 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.394 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-05-09T02:27:21.394 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-05-09T02:27:21.394 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-05-09T02:27:21.394 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-05-09T02:27:21.394 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-05-09T02:27:21.394 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-05-09T02:27:21.394 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-05-09T02:27:21.394 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 534, 2024-05-09T02:27:21.395 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 7, 2024-05-09T02:27:21.395 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 527, 2024-05-09T02:27:21.395 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-05-09T02:27:21.395 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 525, 2024-05-09T02:27:21.395 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-05-09T02:27:21.395 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-05-09T02:27:21.395 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-05-09T02:27:21.395 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 7, 2024-05-09T02:27:21.395 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-05-09T02:27:21.395 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-05-09T02:27:21.395 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-05-09T02:27:21.395 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-05-09T02:27:21.395 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-05-09T02:27:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-05-09T02:27:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-05-09T02:27:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-05-09T02:27:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-05-09T02:27:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-05-09T02:27:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-05-09T02:27:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-05-09T02:27:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-05-09T02:27:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-05-09T02:27:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-05-09T02:27:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-05-09T02:27:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-05-09T02:27:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-05-09T02:27:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-05-09T02:27:21.397 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T02:27:21.397 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-05-09T02:27:21.397 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T02:27:21.397 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-05-09T02:27:21.397 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-05-09T02:27:21.397 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T02:27:21.397 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T02:27:21.397 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-05-09T02:27:21.397 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-05-09T02:27:21.397 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T02:27:21.397 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-05-09T02:27:21.397 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T02:27:21.397 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T02:27:21.398 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T02:27:21.398 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-05-09T02:27:21.398 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-05-09T02:27:21.398 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-05-09T02:27:21.398 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-05-09T02:27:21.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.398 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-05-09T02:27:21.398 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-05-09T02:27:21.398 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-05-09T02:27:21.398 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-05-09T02:27:21.398 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-05-09T02:27:21.398 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-05-09T02:27:21.398 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-05-09T02:27:21.398 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-05-09T02:27:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-05-09T02:27:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-05-09T02:27:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-05-09T02:27:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-05-09T02:27:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-05-09T02:27:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-05-09T02:27:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-05-09T02:27:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-05-09T02:27:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-05-09T02:27:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-05-09T02:27:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-05-09T02:27:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-05-09T02:27:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T02:27:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-05-09T02:27:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-05-09T02:27:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-05-09T02:27:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-05-09T02:27:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-05-09T02:27:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-05-09T02:27:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-05-09T02:27:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-05-09T02:27:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-05-09T02:27:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-05-09T02:27:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 1520, 2024-05-09T02:27:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-05-09T02:27:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-05-09T02:27:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-05-09T02:27:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-05-09T02:27:21.400 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-05-09T02:27:21.401 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 3, 2024-05-09T02:27:21.401 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-05-09T02:27:21.401 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-05-09T02:27:21.401 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-05-09T02:27:21.401 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-05-09T02:27:21.401 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 8131593, 2024-05-09T02:27:21.401 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 8123356, 2024-05-09T02:27:21.401 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-05-09T02:27:21.401 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.401 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.401 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.401 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 1520 2024-05-09T02:27:21.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.401 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-05-09T02:27:21.402 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 280, 2024-05-09T02:27:21.402 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 631, 2024-05-09T02:27:21.402 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 351, 2024-05-09T02:27:21.402 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 16, 2024-05-09T02:27:21.402 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 20, 2024-05-09T02:27:21.402 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 4, 2024-05-09T02:27:21.402 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 534, 2024-05-09T02:27:21.402 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 690, 2024-05-09T02:27:21.402 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 156, 2024-05-09T02:27:21.402 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-05-09T02:27:21.402 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-05-09T02:27:21.402 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-05-09T02:27:21.402 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 127720, 2024-05-09T02:27:21.402 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215440 2024-05-09T02:27:21.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.403 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-05-09T02:27:21.403 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-05-09T02:27:21.403 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-05-09T02:27:21.403 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-05-09T02:27:21.403 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-05-09T02:27:21.403 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-05-09T02:27:21.403 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-05-09T02:27:21.403 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.403 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.403 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.403 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-05-09T02:27:21.403 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.403 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-05-09T02:27:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-05-09T02:27:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-05-09T02:27:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.404 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-05-09T02:27:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-05-09T02:27:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-05-09T02:27:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.405 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.406 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.406 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-05-09T02:27:21.406 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.406 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.406 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.406 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-05-09T02:27:21.406 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.406 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.406 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.406 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-05-09T02:27:21.406 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.406 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.407 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.407 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-05-09T02:27:21.408 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.408 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.408 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.408 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-05-09T02:27:21.408 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.408 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.408 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.408 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-05-09T02:27:21.408 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.408 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.408 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.409 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-05-09T02:27:21.409 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.409 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.409 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.409 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-05-09T02:27:21.409 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.409 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.409 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.409 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-05-09T02:27:21.409 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.409 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.409 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.410 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-05-09T02:27:21.410 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.410 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.410 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.410 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-05-09T02:27:21.410 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.410 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.410 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.410 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-05-09T02:27:21.411 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.411 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.411 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.411 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-05-09T02:27:21.411 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.411 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.411 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.411 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-05-09T02:27:21.411 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.411 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.411 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.412 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-05-09T02:27:21.412 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.412 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.412 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.412 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-05-09T02:27:21.412 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.412 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.412 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.412 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-05-09T02:27:21.412 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.412 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.413 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.413 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-05-09T02:27:21.413 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.413 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.413 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.413 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-05-09T02:27:21.413 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.413 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.413 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.413 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-05-09T02:27:21.414 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.414 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.414 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.414 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-05-09T02:27:21.414 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.414 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.414 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.414 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-05-09T02:27:21.414 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-05-09T02:27:21.414 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-05-09T02:27:21.414 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.414 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.415 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.415 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:21.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.415 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-05-09T02:27:21.415 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 2, 2024-05-09T02:27:21.415 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 4, 2024-05-09T02:27:21.415 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 2, 2024-05-09T02:27:21.415 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 2, 2024-05-09T02:27:21.415 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-05-09T02:27:21.415 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-05-09T02:27:21.415 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-05-09T02:27:21.415 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 4602, 2024-05-09T02:27:21.415 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-05-09T02:27:21.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.416 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-09T02:27:21.416 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-09T02:27:21.416 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-09T02:27:21.416 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T02:27:21.416 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-09T02:27:21.416 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T02:27:21.416 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-09T02:27:21.416 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T02:27:21.416 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T02:27:21.416 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T02:27:21.416 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T02:27:21.416 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T02:27:21.416 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-09T02:27:21.416 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T02:27:21.417 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T02:27:21.417 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-09T02:27:21.417 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-09T02:27:21.417 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-09T02:27:21.417 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-09T02:27:21.417 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T02:27:21.417 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T02:27:21.417 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T02:27:21.417 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T02:27:21.417 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T02:27:21.417 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-09T02:27:21.417 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-09T02:27:21.417 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-09T02:27:21.417 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T02:27:21.418 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T02:27:21.418 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-09T02:27:21.418 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-09T02:27:21.418 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-09T02:27:21.418 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-09T02:27:21.418 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T02:27:21.418 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-09T02:27:21.418 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T02:27:21.418 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-09T02:27:21.418 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 209906, 2024-05-09T02:27:21.418 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 352, 2024-05-09T02:27:21.418 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-05-09T02:27:21.418 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-05-09T02:27:21.418 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-09T02:27:21.418 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-09T02:27:21.419 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-09T02:27:21.419 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-09T02:27:21.419 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-09T02:27:21.419 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-09T02:27:21.419 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 8224, 2024-05-09T02:27:21.419 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 171, 2024-05-09T02:27:21.419 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T02:27:21.419 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-09T02:27:21.419 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-09T02:27:21.419 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-09T02:27:21.419 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 1090006, 2024-05-09T02:27:21.419 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 13688, 2024-05-09T02:27:21.419 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-09T02:27:21.420 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-09T02:27:21.420 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-09T02:27:21.420 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-09T02:27:21.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.420 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-05-09T02:27:21.420 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-05-09T02:27:21.420 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-05-09T02:27:21.420 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 3414, 2024-05-09T02:27:21.420 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 21, 2024-05-09T02:27:21.420 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 3, 2024-05-09T02:27:21.420 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 3414, 2024-05-09T02:27:21.420 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-05-09T02:27:21.420 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3414, 2024-05-09T02:27:21.420 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 15.783717397, 2024-05-09T02:27:21.420 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.004623232 2024-05-09T02:27:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-05-09T02:27:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-05-09T02:27:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3414, 2024-05-09T02:27:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 3415 2024-05-09T02:27:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 3414, 2024-05-09T02:27:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 3414, 2024-05-09T02:27:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-05-09T02:27:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-05-09T02:27:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-05-09T02:27:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 2089, 2024-05-09T02:27:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-05-09T02:27:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 1323, 2024-05-09T02:27:21.421 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-05-09T02:27:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-05-09T02:27:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-05-09T02:27:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-05-09T02:27:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-05-09T02:27:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-05-09T02:27:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-05-09T02:27:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-05-09T02:27:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-05-09T02:27:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-05-09T02:27:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1, 2024-05-09T02:27:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-05-09T02:27:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-05-09T02:27:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-05-09T02:27:21.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-05-09T02:27:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-05-09T02:27:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-05-09T02:27:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-05-09T02:27:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T02:27:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-05-09T02:27:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-05-09T02:27:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-05-09T02:27:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-05-09T02:27:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-05-09T02:27:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-05-09T02:27:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-05-09T02:27:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-05-09T02:27:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-05-09T02:27:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-05-09T02:27:21.423 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-05-09T02:27:21.424 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-05-09T02:27:21.424 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-05-09T02:27:21.424 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-05-09T02:27:21.424 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-05-09T02:27:21.424 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-05-09T02:27:21.424 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-05-09T02:27:21.424 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-05-09T02:27:21.424 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-05-09T02:27:21.424 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 102, 2024-05-09T02:27:21.424 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-05-09T02:27:21.424 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 23, 2024-05-09T02:27:21.424 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 6, 2024-05-09T02:27:21.424 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 8, 2024-05-09T02:27:21.424 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 2, 2024-05-09T02:27:21.424 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-05-09T02:27:21.425 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-05-09T02:27:21.425 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-05-09T02:27:21.425 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-05-09T02:27:21.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.425 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-05-09T02:27:21.425 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-05-09T02:27:21.425 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-05-09T02:27:21.425 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-05-09T02:27:21.425 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-05-09T02:27:21.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.425 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-05-09T02:27:21.425 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-05-09T02:27:21.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.425 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-05-09T02:27:21.425 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-05-09T02:27:21.426 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T02:27:21.426 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-05-09T02:27:21.426 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-05-09T02:27:21.426 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-05-09T02:27:21.426 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-05-09T02:27:21.426 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-05-09T02:27:21.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.426 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T02:27:21.426 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T02:27:21.426 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-09T02:27:21.426 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T02:27:21.426 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 3807, 2024-05-09T02:27:21.426 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 6537525, 2024-05-09T02:27:21.426 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:21.427 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 3807, 2024-05-09T02:27:21.427 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T02:27:21.427 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T02:27:21.427 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3807, 2024-05-09T02:27:21.427 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 6537525, 2024-05-09T02:27:21.427 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T02:27:21.427 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.427 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.427 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:21.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.427 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-05-09T02:27:21.427 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T02:27:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-09T02:27:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T02:27:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T02:27:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T02:27:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 3414, 2024-05-09T02:27:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 3937294, 2024-05-09T02:27:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3414, 2024-05-09T02:27:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 3937294, 2024-05-09T02:27:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T02:27:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.428 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:21.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.429 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-05-09T02:27:21.429 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T02:27:21.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-05-09T02:27:21.429 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T02:27:21.429 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T02:27:21.429 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T02:27:21.429 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:21.429 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:21.429 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 3414, 2024-05-09T02:27:21.429 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 3414, 2024-05-09T02:27:21.429 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3414, 2024-05-09T02:27:21.429 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 3414, 2024-05-09T02:27:21.429 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T02:27:21.429 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-05-09T02:27:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T02:27:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-05-09T02:27:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T02:27:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T02:27:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T02:27:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T02:27:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T02:27:21.430 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-05-09T02:27:21.431 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-05-09T02:27:21.431 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T02:27:21.431 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.431 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.431 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.431 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:21.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:21.431 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55cb80bb68f0": { 2024-05-09T02:27:21.431 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T02:27:21.431 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-05-09T02:27:21.432 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T02:27:21.432 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-09T02:27:21.432 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-09T02:27:21.432 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:21.432 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:21.432 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T02:27:21.432 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T02:27:21.432 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-05-09T02:27:21.432 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-05-09T02:27:21.432 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T02:27:21.432 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:21.432 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:21.432 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:21.432 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:21.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:21.433 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:27:22.228 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-09T02:27:22.228 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-09T02:27:22.228 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-05-09T02:27:22.229 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-05-09T02:27:22.229 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T02:27:22.229 DEBUG:tasks.check_counter:Getting stats from j 2024-05-09T02:27:22.229 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-09T02:27:22.487 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:27:22.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:22 smithi019 ceph-mon[30151]: pgmap v505: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 117 KiB/s wr, 19 op/s 2024-05-09T02:27:22.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:22 smithi019 ceph-mon[30151]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:27:22.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:22 smithi196 ceph-mon[34718]: pgmap v505: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 117 KiB/s wr, 19 op/s 2024-05-09T02:27:22.795 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:22 smithi196 ceph-mon[34718]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:27:22.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:22 smithi059 ceph-mon[34781]: pgmap v505: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 117 KiB/s wr, 19 op/s 2024-05-09T02:27:22.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:22 smithi059 ceph-mon[34781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T02:27:24.565 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:24 smithi019 ceph-mon[30151]: pgmap v506: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 102 KiB/s wr, 18 op/s 2024-05-09T02:27:24.600 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-09T02:27:24.600 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T02:27:24.600 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-05-09T02:27:24.601 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-05-09T02:27:24.601 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-05-09T02:27:24.601 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-05-09T02:27:24.601 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-05-09T02:27:24.601 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-05-09T02:27:24.601 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.001244865, 2024-05-09T02:27:24.601 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T02:27:24.601 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000238273, 2024-05-09T02:27:24.601 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T02:27:24.601 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:24.601 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:24.601 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:24.601 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:24.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:24.602 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:24.602 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:24.602 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:24.602 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:24.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:24.602 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T02:27:24.602 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T02:27:24.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:24.602 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T02:27:24.602 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 3, 2024-05-09T02:27:24.602 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 249, 2024-05-09T02:27:24.602 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 480, 2024-05-09T02:27:24.602 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 279936, 2024-05-09T02:27:24.602 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-05-09T02:27:24.603 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-05-09T02:27:24.603 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.020345037, 2024-05-09T02:27:24.603 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.014110308, 2024-05-09T02:27:24.603 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002619261, 2024-05-09T02:27:24.603 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T02:27:24.603 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:24.603 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 246, 2024-05-09T02:27:24.603 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.038070348, 2024-05-09T02:27:24.603 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000154757 2024-05-09T02:27:24.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:24.603 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:24.603 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:24.603 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:24.603 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:24.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:24.604 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-09T02:27:24.604 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 279936 2024-05-09T02:27:24.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:24.604 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T02:27:24.604 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 339, 2024-05-09T02:27:24.604 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 326, 2024-05-09T02:27:24.604 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1745888, 2024-05-09T02:27:24.604 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 200688, 2024-05-09T02:27:24.604 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-05-09T02:27:24.604 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-05-09T02:27:24.604 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.065804042, 2024-05-09T02:27:24.604 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.019165144, 2024-05-09T02:27:24.604 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.052196081, 2024-05-09T02:27:24.604 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.007381681, 2024-05-09T02:27:24.605 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:24.605 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 326, 2024-05-09T02:27:24.605 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.016095314, 2024-05-09T02:27:24.605 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000049372 2024-05-09T02:27:24.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:24.605 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:24.605 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:24.605 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:24.605 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:24.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:24.605 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1745888, 2024-05-09T02:27:24.605 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 200688 2024-05-09T02:27:24.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:24.605 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-09T02:27:24.606 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-05-09T02:27:24.606 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-09T02:27:24.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:24.606 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-09T02:27:24.606 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-09T02:27:24.606 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-09T02:27:24.606 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T02:27:24.606 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-09T02:27:24.606 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T02:27:24.606 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-09T02:27:24.606 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T02:27:24.606 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T02:27:24.606 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T02:27:24.606 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T02:27:24.607 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T02:27:24.607 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-09T02:27:24.607 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T02:27:24.607 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T02:27:24.607 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-09T02:27:24.607 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-09T02:27:24.607 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-09T02:27:24.607 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-09T02:27:24.607 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T02:27:24.607 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T02:27:24.607 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T02:27:24.607 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T02:27:24.607 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T02:27:24.607 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-09T02:27:24.608 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-09T02:27:24.608 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-09T02:27:24.608 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T02:27:24.608 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T02:27:24.608 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-09T02:27:24.608 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-09T02:27:24.608 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-09T02:27:24.608 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-09T02:27:24.608 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T02:27:24.608 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-09T02:27:24.608 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T02:27:24.608 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-09T02:27:24.608 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 147930, 2024-05-09T02:27:24.608 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 28, 2024-05-09T02:27:24.609 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-05-09T02:27:24.609 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-05-09T02:27:24.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-09T02:27:24.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-09T02:27:24.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-09T02:27:24.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-09T02:27:24.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-09T02:27:24.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-09T02:27:24.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-05-09T02:27:24.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-05-09T02:27:24.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T02:27:24.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-09T02:27:24.609 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-09T02:27:24.609 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-09T02:27:24.610 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-05-09T02:27:24.610 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-05-09T02:27:24.610 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-09T02:27:24.610 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-09T02:27:24.610 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-09T02:27:24.610 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-09T02:27:24.610 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-09T02:27:24.610 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T02:27:24.610 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-09T02:27:24.610 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-09T02:27:24.610 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-09T02:27:24.611 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 338, 2024-05-09T02:27:24.611 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1695648, 2024-05-09T02:27:24.611 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:24.611 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 338, 2024-05-09T02:27:24.611 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-09T02:27:24.611 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-09T02:27:24.611 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 338, 2024-05-09T02:27:24.611 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1695648, 2024-05-09T02:27:24.611 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-09T02:27:24.611 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-09T02:27:24.611 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-09T02:27:24.611 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-09T02:27:24.611 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:24.611 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-09T02:27:24.612 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-09T02:27:24.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:24 smithi196 ceph-mon[34718]: pgmap v506: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 102 KiB/s wr, 18 op/s 2024-05-09T02:27:24.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:24 smithi059 ceph-mon[34781]: pgmap v506: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 102 KiB/s wr, 18 op/s 2024-05-09T02:27:25.409 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-09T02:27:25.409 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-09T02:27:25.409 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-09T02:27:25.409 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-09T02:27:25.410 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-09T02:27:25.410 DEBUG:tasks.check_counter:Getting stats from b 2024-05-09T02:27:25.411 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-09T02:27:25.707 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T02:27:26.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:26 smithi196 ceph-mon[34718]: pgmap v507: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 94 KiB/s wr, 18 op/s 2024-05-09T02:27:26.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:26 smithi019 ceph-mon[30151]: pgmap v507: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 94 KiB/s wr, 18 op/s 2024-05-09T02:27:26.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:26 smithi059 ceph-mon[34781]: pgmap v507: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 94 KiB/s wr, 18 op/s 2024-05-09T02:27:27.753 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-09T02:27:27.753 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T02:27:27.753 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 8728, 2024-05-09T02:27:27.753 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 8247, 2024-05-09T02:27:27.753 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 4045026, 2024-05-09T02:27:27.753 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 6749825, 2024-05-09T02:27:27.753 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 13, 2024-05-09T02:27:27.753 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 4, 2024-05-09T02:27:27.753 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.519684169, 2024-05-09T02:27:27.753 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.187731440, 2024-05-09T02:27:27.753 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.360415616, 2024-05-09T02:27:27.754 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.036955876, 2024-05-09T02:27:27.754 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:27.754 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 8249, 2024-05-09T02:27:27.754 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.351176408, 2024-05-09T02:27:27.754 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000042571 2024-05-09T02:27:27.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.754 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:27.754 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 14221, 2024-05-09T02:27:27.754 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.003045642, 2024-05-09T02:27:27.754 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000214 2024-05-09T02:27:27.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.754 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T02:27:27.754 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T02:27:27.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.755 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T02:27:27.755 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 2097, 2024-05-09T02:27:27.755 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 3205, 2024-05-09T02:27:27.755 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 640379, 2024-05-09T02:27:27.755 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 2385918, 2024-05-09T02:27:27.755 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 47, 2024-05-09T02:27:27.755 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 10, 2024-05-09T02:27:27.755 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.175010772, 2024-05-09T02:27:27.755 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.066639534, 2024-05-09T02:27:27.755 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.124046965, 2024-05-09T02:27:27.755 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.003141612, 2024-05-09T02:27:27.755 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:27.755 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3199, 2024-05-09T02:27:27.755 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.138855368, 2024-05-09T02:27:27.756 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000043405 2024-05-09T02:27:27.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.756 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:27.756 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 4123, 2024-05-09T02:27:27.756 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000899479, 2024-05-09T02:27:27.756 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000218 2024-05-09T02:27:27.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.756 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 6048, 2024-05-09T02:27:27.756 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 596528 2024-05-09T02:27:27.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.756 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T02:27:27.756 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 1855, 2024-05-09T02:27:27.756 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 1804, 2024-05-09T02:27:27.756 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 2649646, 2024-05-09T02:27:27.756 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 906979, 2024-05-09T02:27:27.757 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 9, 2024-05-09T02:27:27.757 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 7, 2024-05-09T02:27:27.757 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.178116583, 2024-05-09T02:27:27.757 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.064023847, 2024-05-09T02:27:27.757 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.119766086, 2024-05-09T02:27:27.757 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.015728014, 2024-05-09T02:27:27.757 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:27.757 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1803, 2024-05-09T02:27:27.757 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.089285685, 2024-05-09T02:27:27.757 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000049520 2024-05-09T02:27:27.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.757 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:27.757 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2147, 2024-05-09T02:27:27.757 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000728485, 2024-05-09T02:27:27.758 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000339 2024-05-09T02:27:27.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.758 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 2042480, 2024-05-09T02:27:27.758 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 229456 2024-05-09T02:27:27.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.758 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-05-09T02:27:27.758 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 1, 2024-05-09T02:27:27.758 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-05-09T02:27:27.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.758 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-MDSRank": { 2024-05-09T02:27:27.758 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T02:27:27.758 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T02:27:27.758 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2167, 2024-05-09T02:27:27.758 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.561564841, 2024-05-09T02:27:27.758 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000259143 2024-05-09T02:27:27.759 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:27.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.759 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-PurgeQueue": { 2024-05-09T02:27:27.759 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T02:27:27.759 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T02:27:27.759 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 20, 2024-05-09T02:27:27.759 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000409268, 2024-05-09T02:27:27.759 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000020463 2024-05-09T02:27:27.759 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:27.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.759 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": { 2024-05-09T02:27:27.759 INFO:teuthology.orchestra.run.smithi059.stdout: "request": 777, 2024-05-09T02:27:27.759 INFO:teuthology.orchestra.run.smithi059.stdout: "reply": 695, 2024-05-09T02:27:27.759 INFO:teuthology.orchestra.run.smithi059.stdout: "reply_latency": { 2024-05-09T02:27:27.760 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 695, 2024-05-09T02:27:27.760 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 15.275206236, 2024-05-09T02:27:27.760 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.021978714 2024-05-09T02:27:27.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.760 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_reply": 0, 2024-05-09T02:27:27.760 INFO:teuthology.orchestra.run.smithi059.stdout: "forward": 82, 2024-05-09T02:27:27.760 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_complete": 6, 2024-05-09T02:27:27.760 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_keys": 0, 2024-05-09T02:27:27.760 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_commit": 20, 2024-05-09T02:27:27.760 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_split": 1, 2024-05-09T02:27:27.760 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_merge": 0, 2024-05-09T02:27:27.760 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes": 477, 2024-05-09T02:27:27.760 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_top": 20, 2024-05-09T02:27:27.760 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_bottom": 373, 2024-05-09T02:27:27.761 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pin_tail": 84, 2024-05-09T02:27:27.761 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pinned": 447, 2024-05-09T02:27:27.761 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_expired": 468, 2024-05-09T02:27:27.761 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_with_caps": 71, 2024-05-09T02:27:27.761 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": 71, 2024-05-09T02:27:27.761 INFO:teuthology.orchestra.run.smithi059.stdout: "subtrees": 15, 2024-05-09T02:27:27.761 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse": 2970, 2024-05-09T02:27:27.761 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_hit": 2185, 2024-05-09T02:27:27.761 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_forward": 75, 2024-05-09T02:27:27.761 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_discover": 76, 2024-05-09T02:27:27.761 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_dir_fetch": 3, 2024-05-09T02:27:27.761 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_remote_ino": 0, 2024-05-09T02:27:27.761 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_lock": 136, 2024-05-09T02:27:27.761 INFO:teuthology.orchestra.run.smithi059.stdout: "load_cent": 1295, 2024-05-09T02:27:27.761 INFO:teuthology.orchestra.run.smithi059.stdout: "q": 0, 2024-05-09T02:27:27.762 INFO:teuthology.orchestra.run.smithi059.stdout: "exported": 8, 2024-05-09T02:27:27.762 INFO:teuthology.orchestra.run.smithi059.stdout: "exported_inodes": 304, 2024-05-09T02:27:27.762 INFO:teuthology.orchestra.run.smithi059.stdout: "imported": 6, 2024-05-09T02:27:27.762 INFO:teuthology.orchestra.run.smithi059.stdout: "imported_inodes": 356, 2024-05-09T02:27:27.762 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_dir_fetch": 0, 2024-05-09T02:27:27.762 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_backtrace_fetch": 398, 2024-05-09T02:27:27.762 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_peer_discover": 0, 2024-05-09T02:27:27.762 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rfiles": 998, 2024-05-09T02:27:27.762 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rbytes": 1205583148, 2024-05-09T02:27:27.762 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rsnaps": 6, 2024-05-09T02:27:27.762 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_fetch": 398, 2024-05-09T02:27:27.762 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_set_tag": 0, 2024-05-09T02:27:27.762 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T02:27:27.762 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_inotable_repaired": 0, 2024-05-09T02:27:27.763 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_inodes": 14, 2024-05-09T02:27:27.763 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T02:27:27.763 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dirfrag_rstats": 14, 2024-05-09T02:27:27.763 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_file_inodes": 384, 2024-05-09T02:27:27.763 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_inode_file_caps": 38, 2024-05-09T02:27:27.763 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_revoke": 376, 2024-05-09T02:27:27.763 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_grant": 703, 2024-05-09T02:27:27.763 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T02:27:27.763 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T02:27:27.763 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flush_ack": 1, 2024-05-09T02:27:27.763 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps": 948, 2024-05-09T02:27:27.763 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps_dirty": 436, 2024-05-09T02:27:27.763 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_cap_release": 47, 2024-05-09T02:27:27.763 INFO:teuthology.orchestra.run.smithi059.stdout: "process_request_cap_release": 318 2024-05-09T02:27:27.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.764 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache": { 2024-05-09T02:27:27.764 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update": 0, 2024-05-09T02:27:27.764 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update_receipt": 0, 2024-05-09T02:27:27.764 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_try_discover": 0, 2024-05-09T02:27:27.764 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_send_discover": 76, 2024-05-09T02:27:27.764 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_handle_discover": 77, 2024-05-09T02:27:27.764 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays": 5, 2024-05-09T02:27:27.764 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_delayed": 0, 2024-05-09T02:27:27.764 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_enqueuing": 0, 2024-05-09T02:27:27.764 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_created": 82, 2024-05-09T02:27:27.764 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_enqueued": 1, 2024-05-09T02:27:27.764 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_reintegrated": 73, 2024-05-09T02:27:27.764 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_migrated": 30, 2024-05-09T02:27:27.765 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_processing": 0, 2024-05-09T02:27:27.765 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_enqueued": 0, 2024-05-09T02:27:27.765 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_prioritized": 0, 2024-05-09T02:27:27.765 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_started": 0, 2024-05-09T02:27:27.765 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_completed": 0, 2024-05-09T02:27:27.765 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_path": 0, 2024-05-09T02:27:27.765 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_inode": 0, 2024-05-09T02:27:27.765 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T02:27:27.765 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_exportdir": 10, 2024-05-09T02:27:27.765 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_flush": 0, 2024-05-09T02:27:27.765 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragmentdir": 1, 2024-05-09T02:27:27.765 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragstats": 0, 2024-05-09T02:27:27.765 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_inodestats": 0 2024-05-09T02:27:27.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.766 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log": { 2024-05-09T02:27:27.766 INFO:teuthology.orchestra.run.smithi059.stdout: "evlrg": 0, 2024-05-09T02:27:27.766 INFO:teuthology.orchestra.run.smithi059.stdout: "evadd": 1558, 2024-05-09T02:27:27.766 INFO:teuthology.orchestra.run.smithi059.stdout: "evex": 0, 2024-05-09T02:27:27.766 INFO:teuthology.orchestra.run.smithi059.stdout: "evtrm": 0, 2024-05-09T02:27:27.766 INFO:teuthology.orchestra.run.smithi059.stdout: "ev": 1558, 2024-05-09T02:27:27.766 INFO:teuthology.orchestra.run.smithi059.stdout: "evexg": 0, 2024-05-09T02:27:27.766 INFO:teuthology.orchestra.run.smithi059.stdout: "evexd": 0, 2024-05-09T02:27:27.766 INFO:teuthology.orchestra.run.smithi059.stdout: "segadd": 3, 2024-05-09T02:27:27.766 INFO:teuthology.orchestra.run.smithi059.stdout: "segex": 0, 2024-05-09T02:27:27.766 INFO:teuthology.orchestra.run.smithi059.stdout: "segtrm": 0, 2024-05-09T02:27:27.766 INFO:teuthology.orchestra.run.smithi059.stdout: "seg": 3, 2024-05-09T02:27:27.766 INFO:teuthology.orchestra.run.smithi059.stdout: "segmjr": 2, 2024-05-09T02:27:27.766 INFO:teuthology.orchestra.run.smithi059.stdout: "segexg": 0, 2024-05-09T02:27:27.767 INFO:teuthology.orchestra.run.smithi059.stdout: "segexd": 0, 2024-05-09T02:27:27.767 INFO:teuthology.orchestra.run.smithi059.stdout: "expos": 4194304, 2024-05-09T02:27:27.767 INFO:teuthology.orchestra.run.smithi059.stdout: "wrpos": 7286442, 2024-05-09T02:27:27.767 INFO:teuthology.orchestra.run.smithi059.stdout: "rdpos": 0, 2024-05-09T02:27:27.767 INFO:teuthology.orchestra.run.smithi059.stdout: "jlat": { 2024-05-09T02:27:27.767 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 855, 2024-05-09T02:27:27.767 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 3.522894462, 2024-05-09T02:27:27.767 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.004120344 2024-05-09T02:27:27.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.767 INFO:teuthology.orchestra.run.smithi059.stdout: "replayed": 0 2024-05-09T02:27:27.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.767 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mem": { 2024-05-09T02:27:27.767 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 225, 2024-05-09T02:27:27.767 INFO:teuthology.orchestra.run.smithi059.stdout: "ino+": 522, 2024-05-09T02:27:27.767 INFO:teuthology.orchestra.run.smithi059.stdout: "ino-": 297, 2024-05-09T02:27:27.768 INFO:teuthology.orchestra.run.smithi059.stdout: "dir": 50, 2024-05-09T02:27:27.768 INFO:teuthology.orchestra.run.smithi059.stdout: "dir+": 57, 2024-05-09T02:27:27.768 INFO:teuthology.orchestra.run.smithi059.stdout: "dir-": 7, 2024-05-09T02:27:27.768 INFO:teuthology.orchestra.run.smithi059.stdout: "dn": 488, 2024-05-09T02:27:27.768 INFO:teuthology.orchestra.run.smithi059.stdout: "dn+": 960, 2024-05-09T02:27:27.768 INFO:teuthology.orchestra.run.smithi059.stdout: "dn-": 472, 2024-05-09T02:27:27.768 INFO:teuthology.orchestra.run.smithi059.stdout: "cap": 71, 2024-05-09T02:27:27.768 INFO:teuthology.orchestra.run.smithi059.stdout: "cap+": 276, 2024-05-09T02:27:27.768 INFO:teuthology.orchestra.run.smithi059.stdout: "cap-": 205, 2024-05-09T02:27:27.768 INFO:teuthology.orchestra.run.smithi059.stdout: "rss": 127916, 2024-05-09T02:27:27.768 INFO:teuthology.orchestra.run.smithi059.stdout: "heap": 223632 2024-05-09T02:27:27.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.768 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_server": { 2024-05-09T02:27:27.768 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_client_request": 2220, 2024-05-09T02:27:27.768 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_server_request": 837, 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_request": 777, 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_session": 191, 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_peer_request": 1510, 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: "req_create_latency": { 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 36, 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.075615871, 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.002100440 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getattr_latency": { 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 38, 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.018837193, 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000495715 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getfilelock_latency": { 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.769 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.770 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.770 INFO:teuthology.orchestra.run.smithi059.stdout: "req_link_latency": { 2024-05-09T02:27:27.770 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 90, 2024-05-09T02:27:27.770 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.572558468, 2024-05-09T02:27:27.770 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.006361760 2024-05-09T02:27:27.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.770 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookup_latency": { 2024-05-09T02:27:27.770 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 292, 2024-05-09T02:27:27.770 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 2.467422424, 2024-05-09T02:27:27.770 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.008450076 2024-05-09T02:27:27.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.770 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookuphash_latency": { 2024-05-09T02:27:27.770 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.771 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.771 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.771 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupino_latency": { 2024-05-09T02:27:27.771 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.771 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.771 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.771 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupname_latency": { 2024-05-09T02:27:27.771 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.771 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.771 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.771 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupparent_latency": { 2024-05-09T02:27:27.771 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.772 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.772 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.772 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupsnap_latency": { 2024-05-09T02:27:27.772 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.772 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.772 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.772 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lssnap_latency": { 2024-05-09T02:27:27.772 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.773 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.773 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.773 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mkdir_latency": { 2024-05-09T02:27:27.773 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.773 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.773 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.773 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mknod_latency": { 2024-05-09T02:27:27.773 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.773 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.773 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.774 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mksnap_latency": { 2024-05-09T02:27:27.774 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.774 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.774 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.774 INFO:teuthology.orchestra.run.smithi059.stdout: "req_open_latency": { 2024-05-09T02:27:27.774 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 45, 2024-05-09T02:27:27.774 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 4.403559715, 2024-05-09T02:27:27.774 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.097856882 2024-05-09T02:27:27.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.774 INFO:teuthology.orchestra.run.smithi059.stdout: "req_readdir_latency": { 2024-05-09T02:27:27.774 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 12, 2024-05-09T02:27:27.774 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 2.040482022, 2024-05-09T02:27:27.774 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.170040168 2024-05-09T02:27:27.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.775 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rename_latency": { 2024-05-09T02:27:27.775 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 100, 2024-05-09T02:27:27.775 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 5.431552135, 2024-05-09T02:27:27.775 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.054315521 2024-05-09T02:27:27.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.775 INFO:teuthology.orchestra.run.smithi059.stdout: "req_renamesnap_latency": { 2024-05-09T02:27:27.775 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.775 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.775 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.775 INFO:teuthology.orchestra.run.smithi059.stdout: "req_snapdiff_latency": { 2024-05-09T02:27:27.775 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.775 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.776 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.776 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmdir_latency": { 2024-05-09T02:27:27.776 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.776 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.776 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.776 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmsnap_latency": { 2024-05-09T02:27:27.776 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.776 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.776 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.776 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmxattr_latency": { 2024-05-09T02:27:27.776 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.776 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.777 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.777 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setattr_latency": { 2024-05-09T02:27:27.777 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.777 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.777 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.777 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setdirlayout_latency": { 2024-05-09T02:27:27.777 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.777 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.777 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.777 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setfilelock_latency": { 2024-05-09T02:27:27.777 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.778 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.778 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.778 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setlayout_latency": { 2024-05-09T02:27:27.778 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.778 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.778 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.778 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setxattr_latency": { 2024-05-09T02:27:27.778 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.778 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.778 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.778 INFO:teuthology.orchestra.run.smithi059.stdout: "req_symlink_latency": { 2024-05-09T02:27:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: "req_unlink_latency": { 2024-05-09T02:27:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 82, 2024-05-09T02:27:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.265178408, 2024-05-09T02:27:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.003233883 2024-05-09T02:27:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_revoke_eviction": 0, 2024-05-09T02:27:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_acquisition_throttle": 0, 2024-05-09T02:27:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getvxattr_latency": { 2024-05-09T02:27:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessions": { 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: "session_count": 2, 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: "session_add": 4, 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: "session_remove": 2, 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_open": 2, 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_stale": 0, 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: "total_load": 85, 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: "average_load": 42, 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: "avg_session_uptime": 947, 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: "mdthresh_evicted": 0 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-05-09T02:27:27.780 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-05-09T02:27:27.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T02:27:27.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-05-09T02:27:27.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T02:27:27.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-05-09T02:27:27.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T02:27:27.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T02:27:27.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T02:27:27.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T02:27:27.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T02:27:27.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-05-09T02:27:27.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T02:27:27.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T02:27:27.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-05-09T02:27:27.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-05-09T02:27:27.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-05-09T02:27:27.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-05-09T02:27:27.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T02:27:27.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T02:27:27.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T02:27:27.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T02:27:27.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T02:27:27.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-05-09T02:27:27.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-05-09T02:27:27.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-05-09T02:27:27.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T02:27:27.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T02:27:27.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-05-09T02:27:27.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-05-09T02:27:27.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-05-09T02:27:27.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-05-09T02:27:27.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T02:27:27.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-05-09T02:27:27.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T02:27:27.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-05-09T02:27:27.783 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 720753, 2024-05-09T02:27:27.783 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 497, 2024-05-09T02:27:27.783 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-05-09T02:27:27.783 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-05-09T02:27:27.783 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-05-09T02:27:27.783 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-05-09T02:27:27.783 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-05-09T02:27:27.783 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-05-09T02:27:27.784 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-05-09T02:27:27.784 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-05-09T02:27:27.784 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 8224, 2024-05-09T02:27:27.784 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 171, 2024-05-09T02:27:27.784 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T02:27:27.784 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-05-09T02:27:27.784 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-05-09T02:27:27.784 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-05-09T02:27:27.784 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 1165765, 2024-05-09T02:27:27.784 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 37032, 2024-05-09T02:27:27.784 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-05-09T02:27:27.784 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-05-09T02:27:27.784 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-05-09T02:27:27.784 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-05-09T02:27:27.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.785 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter": { 2024-05-09T02:27:27.785 INFO:teuthology.orchestra.run.smithi059.stdout: "op_active": 0, 2024-05-09T02:27:27.785 INFO:teuthology.orchestra.run.smithi059.stdout: "op_laggy": 0, 2024-05-09T02:27:27.785 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send": 1513, 2024-05-09T02:27:27.785 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send_bytes": 3279909, 2024-05-09T02:27:27.785 INFO:teuthology.orchestra.run.smithi059.stdout: "op_resend": 33, 2024-05-09T02:27:27.785 INFO:teuthology.orchestra.run.smithi059.stdout: "op_reply": 1513, 2024-05-09T02:27:27.785 INFO:teuthology.orchestra.run.smithi059.stdout: "op_latency": { 2024-05-09T02:27:27.785 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1513, 2024-05-09T02:27:27.785 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 5.597931401, 2024-05-09T02:27:27.785 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.003699888 2024-05-09T02:27:27.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.785 INFO:teuthology.orchestra.run.smithi059.stdout: "op_inflight": 0, 2024-05-09T02:27:27.785 INFO:teuthology.orchestra.run.smithi059.stdout: "oplen_avg": { 2024-05-09T02:27:27.785 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1513, 2024-05-09T02:27:27.786 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 1709 2024-05-09T02:27:27.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.786 INFO:teuthology.orchestra.run.smithi059.stdout: "op": 1513, 2024-05-09T02:27:27.786 INFO:teuthology.orchestra.run.smithi059.stdout: "op_r": 407, 2024-05-09T02:27:27.786 INFO:teuthology.orchestra.run.smithi059.stdout: "op_w": 1106, 2024-05-09T02:27:27.786 INFO:teuthology.orchestra.run.smithi059.stdout: "op_rmw": 0, 2024-05-09T02:27:27.786 INFO:teuthology.orchestra.run.smithi059.stdout: "op_pg": 0, 2024-05-09T02:27:27.786 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_stat": 0, 2024-05-09T02:27:27.786 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_create": 10, 2024-05-09T02:27:27.786 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_read": 1, 2024-05-09T02:27:27.786 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_write": 856, 2024-05-09T02:27:27.786 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writefull": 62, 2024-05-09T02:27:27.786 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writesame": 0, 2024-05-09T02:27:27.786 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_append": 0, 2024-05-09T02:27:27.786 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_zero": 0, 2024-05-09T02:27:27.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_truncate": 1, 2024-05-09T02:27:27.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_delete": 16, 2024-05-09T02:27:27.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_mapext": 0, 2024-05-09T02:27:27.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_sparse_read": 0, 2024-05-09T02:27:27.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_clonerange": 0, 2024-05-09T02:27:27.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_getxattr": 398, 2024-05-09T02:27:27.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_setxattr": 20, 2024-05-09T02:27:27.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_cmpxattr": 0, 2024-05-09T02:27:27.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_rmxattr": 0, 2024-05-09T02:27:27.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_resetxattrs": 0, 2024-05-09T02:27:27.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_call": 0, 2024-05-09T02:27:27.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_watch": 0, 2024-05-09T02:27:27.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_notify": 0, 2024-05-09T02:27:27.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T02:27:27.788 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls": 0, 2024-05-09T02:27:27.788 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls_filter": 0, 2024-05-09T02:27:27.788 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_other": 0, 2024-05-09T02:27:27.788 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_active": 0, 2024-05-09T02:27:27.788 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_send": 0, 2024-05-09T02:27:27.788 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_resend": 0, 2024-05-09T02:27:27.788 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_ping": 0, 2024-05-09T02:27:27.788 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_active": 0, 2024-05-09T02:27:27.788 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_send": 0, 2024-05-09T02:27:27.788 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_resend": 0, 2024-05-09T02:27:27.788 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_active": 0, 2024-05-09T02:27:27.788 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_send": 0, 2024-05-09T02:27:27.788 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_resend": 0, 2024-05-09T02:27:27.788 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_active": 0, 2024-05-09T02:27:27.788 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_send": 0, 2024-05-09T02:27:27.789 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_resend": 0, 2024-05-09T02:27:27.789 INFO:teuthology.orchestra.run.smithi059.stdout: "command_active": 0, 2024-05-09T02:27:27.789 INFO:teuthology.orchestra.run.smithi059.stdout: "command_send": 0, 2024-05-09T02:27:27.789 INFO:teuthology.orchestra.run.smithi059.stdout: "command_resend": 0, 2024-05-09T02:27:27.789 INFO:teuthology.orchestra.run.smithi059.stdout: "map_epoch": 102, 2024-05-09T02:27:27.789 INFO:teuthology.orchestra.run.smithi059.stdout: "map_full": 0, 2024-05-09T02:27:27.789 INFO:teuthology.orchestra.run.smithi059.stdout: "map_inc": 23, 2024-05-09T02:27:27.789 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_sessions": 12, 2024-05-09T02:27:27.789 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_open": 12, 2024-05-09T02:27:27.789 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_close": 0, 2024-05-09T02:27:27.789 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_laggy": 0, 2024-05-09T02:27:27.790 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_wr": 273, 2024-05-09T02:27:27.790 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_rd": 14, 2024-05-09T02:27:27.790 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_del": 58 2024-05-09T02:27:27.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.790 INFO:teuthology.orchestra.run.smithi059.stdout: "oft": { 2024-05-09T02:27:27.790 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_objs": 1, 2024-05-09T02:27:27.790 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_kv_pairs": 0, 2024-05-09T02:27:27.790 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_updates": 0, 2024-05-09T02:27:27.791 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_removes": 0 2024-05-09T02:27:27.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.791 INFO:teuthology.orchestra.run.smithi059.stdout: "osd-slow-ops": { 2024-05-09T02:27:27.791 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_ops_count": 0 2024-05-09T02:27:27.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.791 INFO:teuthology.orchestra.run.smithi059.stdout: "purge_queue": { 2024-05-09T02:27:27.791 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops": 0, 2024-05-09T02:27:27.791 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops_high_water": 4, 2024-05-09T02:27:27.791 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing": 0, 2024-05-09T02:27:27.791 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_high_water": 1, 2024-05-09T02:27:27.791 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed_ops": 4, 2024-05-09T02:27:27.791 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed": 1, 2024-05-09T02:27:27.791 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_item_in_journal": 0 2024-05-09T02:27:27.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.791 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T02:27:27.792 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:27.792 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T02:27:27.792 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:27.792 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 12676, 2024-05-09T02:27:27.792 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 6167405, 2024-05-09T02:27:27.792 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:27.792 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 12676, 2024-05-09T02:27:27.792 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T02:27:27.792 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T02:27:27.792 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 12676, 2024-05-09T02:27:27.792 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 6167405, 2024-05-09T02:27:27.792 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:27.792 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.792 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.792 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.793 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:27.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.793 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_bytes": { 2024-05-09T02:27:27.793 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:27.793 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T02:27:27.793 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:27.793 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T02:27:27.793 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T02:27:27.793 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:27.793 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:27.793 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 1513, 2024-05-09T02:27:27.793 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 3279862, 2024-05-09T02:27:27.793 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 1513, 2024-05-09T02:27:27.793 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 3279862, 2024-05-09T02:27:27.793 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:27.794 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.794 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.794 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.794 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:27.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.794 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_ops": { 2024-05-09T02:27:27.794 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:27.794 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 1024, 2024-05-09T02:27:27.794 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:27.794 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T02:27:27.794 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T02:27:27.794 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:27.794 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:27.794 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 1513, 2024-05-09T02:27:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 1513, 2024-05-09T02:27:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 1513, 2024-05-09T02:27:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 1513, 2024-05-09T02:27:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle": { 2024-05-09T02:27:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T02:27:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 1, 2024-05-09T02:27:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 133, 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 1, 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 1, 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 133, 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle-0x5654bfd1a5b0": { 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T02:27:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:27.797 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 1558, 2024-05-09T02:27:27.797 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 3092138, 2024-05-09T02:27:27.797 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:27.797 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 1558, 2024-05-09T02:27:27.797 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T02:27:27.797 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T02:27:27.797 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 855, 2024-05-09T02:27:27.797 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 3092138, 2024-05-09T02:27:27.797 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:27.797 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:27.797 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:27.797 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:27.797 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:27.797 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:27.797 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-09T02:27:28.556 INFO:tasks.check_counter:Daemon mds.b mds.exported=8 2024-05-09T02:27:28.556 INFO:tasks.check_counter:Daemon mds.b mds.imported=6 2024-05-09T02:27:28.556 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-09T02:27:28.556 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-05-09T02:27:28.557 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T02:27:28.557 DEBUG:tasks.check_counter:Getting stats from e 2024-05-09T02:27:28.557 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-09T02:27:28.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:28 smithi196 ceph-mon[34718]: pgmap v508: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 68 KiB/s wr, 13 op/s 2024-05-09T02:27:28.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:28 smithi019 ceph-mon[30151]: pgmap v508: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 68 KiB/s wr, 13 op/s 2024-05-09T02:27:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T02:27:28.952 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:28 smithi059 ceph-mon[34781]: pgmap v508: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 68 KiB/s wr, 13 op/s 2024-05-09T02:27:30.674 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:30 smithi059 ceph-mon[34781]: pgmap v509: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-05-09T02:27:30.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:30 smithi196 ceph-mon[34718]: pgmap v509: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-05-09T02:27:30.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:30 smithi019 ceph-mon[30151]: pgmap v509: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-05-09T02:27:30.854 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-09T02:27:30.854 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T02:27:30.854 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 2176, 2024-05-09T02:27:30.854 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 2128, 2024-05-09T02:27:30.854 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 10948170, 2024-05-09T02:27:30.854 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 769546, 2024-05-09T02:27:30.854 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 5, 2024-05-09T02:27:30.855 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 1, 2024-05-09T02:27:30.855 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.239519940, 2024-05-09T02:27:30.855 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.077378122, 2024-05-09T02:27:30.855 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.098577893, 2024-05-09T02:27:30.855 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.061771990, 2024-05-09T02:27:30.855 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:30.855 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2128, 2024-05-09T02:27:30.855 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.104208672, 2024-05-09T02:27:30.855 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000048970 2024-05-09T02:27:30.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.855 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:30.855 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.855 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.856 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.856 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 2012672, 2024-05-09T02:27:30.856 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 205728 2024-05-09T02:27:30.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.856 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T02:27:30.856 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 1246, 2024-05-09T02:27:30.856 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 1491, 2024-05-09T02:27:30.856 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 12710961, 2024-05-09T02:27:30.856 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 988218, 2024-05-09T02:27:30.856 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 7, 2024-05-09T02:27:30.856 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 5, 2024-05-09T02:27:30.856 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.113675913, 2024-05-09T02:27:30.856 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.030612600, 2024-05-09T02:27:30.857 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.037888474, 2024-05-09T02:27:30.857 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.035069977, 2024-05-09T02:27:30.857 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:30.857 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1488, 2024-05-09T02:27:30.857 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.072762898, 2024-05-09T02:27:30.857 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000048899 2024-05-09T02:27:30.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.857 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:30.857 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.857 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.857 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.857 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T02:27:30.858 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 592560 2024-05-09T02:27:30.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.858 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T02:27:30.858 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 865, 2024-05-09T02:27:30.858 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 867, 2024-05-09T02:27:30.858 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 8607584, 2024-05-09T02:27:30.858 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 275077, 2024-05-09T02:27:30.858 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 5, 2024-05-09T02:27:30.858 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 4, 2024-05-09T02:27:30.858 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.076808420, 2024-05-09T02:27:30.858 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.018946920, 2024-05-09T02:27:30.858 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.026403806, 2024-05-09T02:27:30.858 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.024819166, 2024-05-09T02:27:30.858 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:30.859 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 865, 2024-05-09T02:27:30.859 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.037143894, 2024-05-09T02:27:30.859 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000042940 2024-05-09T02:27:30.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.859 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:30.859 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.859 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.859 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.859 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T02:27:30.859 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T02:27:30.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.859 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-05-09T02:27:30.859 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 1, 2024-05-09T02:27:30.860 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-05-09T02:27:30.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.860 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-MDSRank": { 2024-05-09T02:27:30.860 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T02:27:30.860 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T02:27:30.860 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 6537, 2024-05-09T02:27:30.860 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.168191932, 2024-05-09T02:27:30.860 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000025729 2024-05-09T02:27:30.860 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:30.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.860 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-PurgeQueue": { 2024-05-09T02:27:30.860 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T02:27:30.860 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T02:27:30.860 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.861 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.861 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.861 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:30.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.861 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": { 2024-05-09T02:27:30.861 INFO:teuthology.orchestra.run.smithi059.stdout: "request": 0, 2024-05-09T02:27:30.861 INFO:teuthology.orchestra.run.smithi059.stdout: "reply": 0, 2024-05-09T02:27:30.861 INFO:teuthology.orchestra.run.smithi059.stdout: "reply_latency": { 2024-05-09T02:27:30.861 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.861 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.861 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.861 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_reply": 0, 2024-05-09T02:27:30.861 INFO:teuthology.orchestra.run.smithi059.stdout: "forward": 0, 2024-05-09T02:27:30.861 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_complete": 0, 2024-05-09T02:27:30.862 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_keys": 0, 2024-05-09T02:27:30.862 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_commit": 0, 2024-05-09T02:27:30.862 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_split": 0, 2024-05-09T02:27:30.862 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_merge": 0, 2024-05-09T02:27:30.862 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes": 889, 2024-05-09T02:27:30.862 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_top": 0, 2024-05-09T02:27:30.862 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_bottom": 889, 2024-05-09T02:27:30.862 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pin_tail": 0, 2024-05-09T02:27:30.862 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pinned": 889, 2024-05-09T02:27:30.862 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_expired": 0, 2024-05-09T02:27:30.862 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_with_caps": 0, 2024-05-09T02:27:30.862 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": 0, 2024-05-09T02:27:30.863 INFO:teuthology.orchestra.run.smithi059.stdout: "subtrees": 59, 2024-05-09T02:27:30.863 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse": 0, 2024-05-09T02:27:30.863 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_hit": 0, 2024-05-09T02:27:30.863 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_forward": 0, 2024-05-09T02:27:30.863 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_discover": 0, 2024-05-09T02:27:30.863 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_dir_fetch": 0, 2024-05-09T02:27:30.863 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_remote_ino": 0, 2024-05-09T02:27:30.863 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_lock": 0, 2024-05-09T02:27:30.863 INFO:teuthology.orchestra.run.smithi059.stdout: "load_cent": 0, 2024-05-09T02:27:30.863 INFO:teuthology.orchestra.run.smithi059.stdout: "q": 0, 2024-05-09T02:27:30.864 INFO:teuthology.orchestra.run.smithi059.stdout: "exported": 0, 2024-05-09T02:27:30.864 INFO:teuthology.orchestra.run.smithi059.stdout: "exported_inodes": 0, 2024-05-09T02:27:30.864 INFO:teuthology.orchestra.run.smithi059.stdout: "imported": 0, 2024-05-09T02:27:30.864 INFO:teuthology.orchestra.run.smithi059.stdout: "imported_inodes": 0, 2024-05-09T02:27:30.864 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_dir_fetch": 0, 2024-05-09T02:27:30.864 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_backtrace_fetch": 0, 2024-05-09T02:27:30.864 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_peer_discover": 0, 2024-05-09T02:27:30.864 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rfiles": 1001, 2024-05-09T02:27:30.864 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rbytes": 1291982165, 2024-05-09T02:27:30.864 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rsnaps": 6, 2024-05-09T02:27:30.864 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T02:27:30.864 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_set_tag": 0, 2024-05-09T02:27:30.864 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T02:27:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_inotable_repaired": 0, 2024-05-09T02:27:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_inodes": 0, 2024-05-09T02:27:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T02:27:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T02:27:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_file_inodes": 0, 2024-05-09T02:27:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_inode_file_caps": 0, 2024-05-09T02:27:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T02:27:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_grant": 0, 2024-05-09T02:27:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T02:27:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T02:27:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T02:27:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps": 0, 2024-05-09T02:27:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps_dirty": 0, 2024-05-09T02:27:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_cap_release": 0, 2024-05-09T02:27:30.865 INFO:teuthology.orchestra.run.smithi059.stdout: "process_request_cap_release": 0 2024-05-09T02:27:30.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.866 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache": { 2024-05-09T02:27:30.866 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update": 0, 2024-05-09T02:27:30.866 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update_receipt": 0, 2024-05-09T02:27:30.866 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_try_discover": 0, 2024-05-09T02:27:30.866 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_send_discover": 0, 2024-05-09T02:27:30.866 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_handle_discover": 0, 2024-05-09T02:27:30.866 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays": 0, 2024-05-09T02:27:30.866 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_delayed": 0, 2024-05-09T02:27:30.866 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_enqueuing": 0, 2024-05-09T02:27:30.866 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_created": 0, 2024-05-09T02:27:30.866 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_enqueued": 0, 2024-05-09T02:27:30.866 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_reintegrated": 0, 2024-05-09T02:27:30.866 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_migrated": 0, 2024-05-09T02:27:30.866 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_processing": 0, 2024-05-09T02:27:30.867 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_enqueued": 0, 2024-05-09T02:27:30.867 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_prioritized": 0, 2024-05-09T02:27:30.867 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_started": 0, 2024-05-09T02:27:30.867 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_completed": 0, 2024-05-09T02:27:30.867 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_path": 0, 2024-05-09T02:27:30.867 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_inode": 0, 2024-05-09T02:27:30.867 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T02:27:30.867 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_exportdir": 0, 2024-05-09T02:27:30.867 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_flush": 0, 2024-05-09T02:27:30.867 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragmentdir": 0, 2024-05-09T02:27:30.867 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragstats": 0, 2024-05-09T02:27:30.867 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_inodestats": 0 2024-05-09T02:27:30.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.868 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log": { 2024-05-09T02:27:30.868 INFO:teuthology.orchestra.run.smithi059.stdout: "evlrg": 0, 2024-05-09T02:27:30.868 INFO:teuthology.orchestra.run.smithi059.stdout: "evadd": 0, 2024-05-09T02:27:30.868 INFO:teuthology.orchestra.run.smithi059.stdout: "evex": 0, 2024-05-09T02:27:30.868 INFO:teuthology.orchestra.run.smithi059.stdout: "evtrm": 0, 2024-05-09T02:27:30.868 INFO:teuthology.orchestra.run.smithi059.stdout: "ev": 4483, 2024-05-09T02:27:30.868 INFO:teuthology.orchestra.run.smithi059.stdout: "evexg": 0, 2024-05-09T02:27:30.868 INFO:teuthology.orchestra.run.smithi059.stdout: "evexd": 0, 2024-05-09T02:27:30.868 INFO:teuthology.orchestra.run.smithi059.stdout: "segadd": 0, 2024-05-09T02:27:30.868 INFO:teuthology.orchestra.run.smithi059.stdout: "segex": 0, 2024-05-09T02:27:30.868 INFO:teuthology.orchestra.run.smithi059.stdout: "segtrm": 0, 2024-05-09T02:27:30.868 INFO:teuthology.orchestra.run.smithi059.stdout: "seg": 9, 2024-05-09T02:27:30.868 INFO:teuthology.orchestra.run.smithi059.stdout: "segmjr": 2, 2024-05-09T02:27:30.868 INFO:teuthology.orchestra.run.smithi059.stdout: "segexg": 0, 2024-05-09T02:27:30.869 INFO:teuthology.orchestra.run.smithi059.stdout: "segexd": 0, 2024-05-09T02:27:30.869 INFO:teuthology.orchestra.run.smithi059.stdout: "expos": 4194304, 2024-05-09T02:27:30.869 INFO:teuthology.orchestra.run.smithi059.stdout: "wrpos": 33189737, 2024-05-09T02:27:30.869 INFO:teuthology.orchestra.run.smithi059.stdout: "rdpos": 33125568, 2024-05-09T02:27:30.869 INFO:teuthology.orchestra.run.smithi059.stdout: "jlat": { 2024-05-09T02:27:30.869 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.869 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.869 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.869 INFO:teuthology.orchestra.run.smithi059.stdout: "replayed": 4483 2024-05-09T02:27:30.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.869 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mem": { 2024-05-09T02:27:30.869 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 726, 2024-05-09T02:27:30.870 INFO:teuthology.orchestra.run.smithi059.stdout: "ino+": 1744, 2024-05-09T02:27:30.870 INFO:teuthology.orchestra.run.smithi059.stdout: "ino-": 1018, 2024-05-09T02:27:30.870 INFO:teuthology.orchestra.run.smithi059.stdout: "dir": 113, 2024-05-09T02:27:30.870 INFO:teuthology.orchestra.run.smithi059.stdout: "dir+": 126, 2024-05-09T02:27:30.870 INFO:teuthology.orchestra.run.smithi059.stdout: "dir-": 13, 2024-05-09T02:27:30.870 INFO:teuthology.orchestra.run.smithi059.stdout: "dn": 889, 2024-05-09T02:27:30.870 INFO:teuthology.orchestra.run.smithi059.stdout: "dn+": 1944, 2024-05-09T02:27:30.870 INFO:teuthology.orchestra.run.smithi059.stdout: "dn-": 1055, 2024-05-09T02:27:30.870 INFO:teuthology.orchestra.run.smithi059.stdout: "cap": 0, 2024-05-09T02:27:30.870 INFO:teuthology.orchestra.run.smithi059.stdout: "cap+": 0, 2024-05-09T02:27:30.870 INFO:teuthology.orchestra.run.smithi059.stdout: "cap-": 0, 2024-05-09T02:27:30.870 INFO:teuthology.orchestra.run.smithi059.stdout: "rss": 128304, 2024-05-09T02:27:30.871 INFO:teuthology.orchestra.run.smithi059.stdout: "heap": 215440 2024-05-09T02:27:30.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.871 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_server": { 2024-05-09T02:27:30.871 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_client_request": 0, 2024-05-09T02:27:30.871 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_server_request": 0, 2024-05-09T02:27:30.871 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_request": 0, 2024-05-09T02:27:30.871 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_session": 0, 2024-05-09T02:27:30.871 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_peer_request": 0, 2024-05-09T02:27:30.871 INFO:teuthology.orchestra.run.smithi059.stdout: "req_create_latency": { 2024-05-09T02:27:30.871 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.871 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.871 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.871 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getattr_latency": { 2024-05-09T02:27:30.872 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.872 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.872 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.872 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getfilelock_latency": { 2024-05-09T02:27:30.872 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.872 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.872 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.872 INFO:teuthology.orchestra.run.smithi059.stdout: "req_link_latency": { 2024-05-09T02:27:30.872 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.872 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.872 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.873 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookup_latency": { 2024-05-09T02:27:30.873 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.873 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.873 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.873 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookuphash_latency": { 2024-05-09T02:27:30.873 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.873 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.873 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.873 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupino_latency": { 2024-05-09T02:27:30.873 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.874 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.874 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.874 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupname_latency": { 2024-05-09T02:27:30.874 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.874 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.874 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.874 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupparent_latency": { 2024-05-09T02:27:30.874 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.874 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.874 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.874 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupsnap_latency": { 2024-05-09T02:27:30.874 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.875 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.875 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.875 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lssnap_latency": { 2024-05-09T02:27:30.875 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.875 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.875 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.876 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mkdir_latency": { 2024-05-09T02:27:30.876 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.876 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.876 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.876 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mknod_latency": { 2024-05-09T02:27:30.876 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.876 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.876 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.876 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mksnap_latency": { 2024-05-09T02:27:30.877 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.877 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.877 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.877 INFO:teuthology.orchestra.run.smithi059.stdout: "req_open_latency": { 2024-05-09T02:27:30.877 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.877 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.877 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.877 INFO:teuthology.orchestra.run.smithi059.stdout: "req_readdir_latency": { 2024-05-09T02:27:30.877 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.877 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.877 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.877 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rename_latency": { 2024-05-09T02:27:30.878 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.878 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.878 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.878 INFO:teuthology.orchestra.run.smithi059.stdout: "req_renamesnap_latency": { 2024-05-09T02:27:30.878 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.878 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.878 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.878 INFO:teuthology.orchestra.run.smithi059.stdout: "req_snapdiff_latency": { 2024-05-09T02:27:30.878 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.878 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.878 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.879 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmdir_latency": { 2024-05-09T02:27:30.879 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.879 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.879 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.879 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmsnap_latency": { 2024-05-09T02:27:30.879 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.879 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.879 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.879 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmxattr_latency": { 2024-05-09T02:27:30.879 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.879 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.879 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.880 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setattr_latency": { 2024-05-09T02:27:30.880 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.880 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.880 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.880 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setdirlayout_latency": { 2024-05-09T02:27:30.880 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.880 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.880 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.880 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setfilelock_latency": { 2024-05-09T02:27:30.880 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.880 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setlayout_latency": { 2024-05-09T02:27:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setxattr_latency": { 2024-05-09T02:27:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: "req_symlink_latency": { 2024-05-09T02:27:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.882 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.882 INFO:teuthology.orchestra.run.smithi059.stdout: "req_unlink_latency": { 2024-05-09T02:27:30.882 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.882 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.882 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.882 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_revoke_eviction": 0, 2024-05-09T02:27:30.882 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_acquisition_throttle": 0, 2024-05-09T02:27:30.882 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getvxattr_latency": { 2024-05-09T02:27:30.882 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.882 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.882 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.882 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:30.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.883 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessions": { 2024-05-09T02:27:30.883 INFO:teuthology.orchestra.run.smithi059.stdout: "session_count": 2, 2024-05-09T02:27:30.883 INFO:teuthology.orchestra.run.smithi059.stdout: "session_add": 4, 2024-05-09T02:27:30.883 INFO:teuthology.orchestra.run.smithi059.stdout: "session_remove": 2, 2024-05-09T02:27:30.883 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_open": 3, 2024-05-09T02:27:30.883 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_stale": 0, 2024-05-09T02:27:30.883 INFO:teuthology.orchestra.run.smithi059.stdout: "total_load": 0, 2024-05-09T02:27:30.883 INFO:teuthology.orchestra.run.smithi059.stdout: "average_load": 0, 2024-05-09T02:27:30.883 INFO:teuthology.orchestra.run.smithi059.stdout: "avg_session_uptime": 3117, 2024-05-09T02:27:30.883 INFO:teuthology.orchestra.run.smithi059.stdout: "mdthresh_evicted": 0 2024-05-09T02:27:30.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.883 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-05-09T02:27:30.883 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-05-09T02:27:30.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-05-09T02:27:30.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T02:27:30.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-05-09T02:27:30.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T02:27:30.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-05-09T02:27:30.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T02:27:30.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T02:27:30.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T02:27:30.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T02:27:30.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T02:27:30.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-05-09T02:27:30.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T02:27:30.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T02:27:30.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-05-09T02:27:30.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-05-09T02:27:30.885 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-05-09T02:27:30.885 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-05-09T02:27:30.885 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T02:27:30.885 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T02:27:30.885 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T02:27:30.885 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T02:27:30.885 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T02:27:30.885 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-05-09T02:27:30.885 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-05-09T02:27:30.885 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-05-09T02:27:30.885 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T02:27:30.885 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T02:27:30.885 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-05-09T02:27:30.885 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-05-09T02:27:30.885 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-05-09T02:27:30.886 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-05-09T02:27:30.886 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T02:27:30.886 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-05-09T02:27:30.886 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T02:27:30.886 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-05-09T02:27:30.886 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 231205, 2024-05-09T02:27:30.886 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 927, 2024-05-09T02:27:30.886 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-05-09T02:27:30.886 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-05-09T02:27:30.886 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-05-09T02:27:30.886 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-05-09T02:27:30.886 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-05-09T02:27:30.886 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-05-09T02:27:30.886 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-05-09T02:27:30.887 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-05-09T02:27:30.887 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 8224, 2024-05-09T02:27:30.887 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 171, 2024-05-09T02:27:30.887 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T02:27:30.887 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-05-09T02:27:30.887 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-05-09T02:27:30.887 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-05-09T02:27:30.887 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 2678354, 2024-05-09T02:27:30.887 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 25660, 2024-05-09T02:27:30.888 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-05-09T02:27:30.888 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-05-09T02:27:30.888 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-05-09T02:27:30.888 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-05-09T02:27:30.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.888 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter": { 2024-05-09T02:27:30.888 INFO:teuthology.orchestra.run.smithi059.stdout: "op_active": 0, 2024-05-09T02:27:30.888 INFO:teuthology.orchestra.run.smithi059.stdout: "op_laggy": 0, 2024-05-09T02:27:30.888 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send": 3882, 2024-05-09T02:27:30.888 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send_bytes": 26, 2024-05-09T02:27:30.888 INFO:teuthology.orchestra.run.smithi059.stdout: "op_resend": 4, 2024-05-09T02:27:30.888 INFO:teuthology.orchestra.run.smithi059.stdout: "op_reply": 3882, 2024-05-09T02:27:30.888 INFO:teuthology.orchestra.run.smithi059.stdout: "op_latency": { 2024-05-09T02:27:30.888 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3882, 2024-05-09T02:27:30.889 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 6.025927138, 2024-05-09T02:27:30.889 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.001552273 2024-05-09T02:27:30.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.889 INFO:teuthology.orchestra.run.smithi059.stdout: "op_inflight": 0, 2024-05-09T02:27:30.889 INFO:teuthology.orchestra.run.smithi059.stdout: "oplen_avg": { 2024-05-09T02:27:30.889 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3882, 2024-05-09T02:27:30.889 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 3883 2024-05-09T02:27:30.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.889 INFO:teuthology.orchestra.run.smithi059.stdout: "op": 3882, 2024-05-09T02:27:30.889 INFO:teuthology.orchestra.run.smithi059.stdout: "op_r": 3882, 2024-05-09T02:27:30.889 INFO:teuthology.orchestra.run.smithi059.stdout: "op_w": 0, 2024-05-09T02:27:30.889 INFO:teuthology.orchestra.run.smithi059.stdout: "op_rmw": 0, 2024-05-09T02:27:30.889 INFO:teuthology.orchestra.run.smithi059.stdout: "op_pg": 0, 2024-05-09T02:27:30.889 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_stat": 2306, 2024-05-09T02:27:30.889 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_create": 0, 2024-05-09T02:27:30.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_read": 1573, 2024-05-09T02:27:30.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_write": 0, 2024-05-09T02:27:30.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writefull": 0, 2024-05-09T02:27:30.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writesame": 0, 2024-05-09T02:27:30.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_append": 0, 2024-05-09T02:27:30.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_zero": 0, 2024-05-09T02:27:30.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_truncate": 0, 2024-05-09T02:27:30.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_delete": 0, 2024-05-09T02:27:30.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_mapext": 0, 2024-05-09T02:27:30.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_sparse_read": 0, 2024-05-09T02:27:30.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_clonerange": 0, 2024-05-09T02:27:30.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_getxattr": 2, 2024-05-09T02:27:30.891 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_setxattr": 0, 2024-05-09T02:27:30.891 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_cmpxattr": 0, 2024-05-09T02:27:30.891 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_rmxattr": 0, 2024-05-09T02:27:30.891 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_resetxattrs": 0, 2024-05-09T02:27:30.891 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_call": 0, 2024-05-09T02:27:30.891 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_watch": 0, 2024-05-09T02:27:30.891 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_notify": 0, 2024-05-09T02:27:30.891 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T02:27:30.891 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls": 0, 2024-05-09T02:27:30.891 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls_filter": 0, 2024-05-09T02:27:30.891 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_other": 0, 2024-05-09T02:27:30.891 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_active": 0, 2024-05-09T02:27:30.891 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_send": 0, 2024-05-09T02:27:30.891 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_resend": 0, 2024-05-09T02:27:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_ping": 0, 2024-05-09T02:27:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_active": 0, 2024-05-09T02:27:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_send": 0, 2024-05-09T02:27:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_resend": 0, 2024-05-09T02:27:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_active": 0, 2024-05-09T02:27:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_send": 0, 2024-05-09T02:27:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_resend": 0, 2024-05-09T02:27:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_active": 0, 2024-05-09T02:27:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_send": 0, 2024-05-09T02:27:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_resend": 0, 2024-05-09T02:27:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "command_active": 0, 2024-05-09T02:27:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "command_send": 0, 2024-05-09T02:27:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "command_resend": 0, 2024-05-09T02:27:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "map_epoch": 102, 2024-05-09T02:27:30.892 INFO:teuthology.orchestra.run.smithi059.stdout: "map_full": 0, 2024-05-09T02:27:30.893 INFO:teuthology.orchestra.run.smithi059.stdout: "map_inc": 23, 2024-05-09T02:27:30.893 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_sessions": 6, 2024-05-09T02:27:30.893 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_open": 6, 2024-05-09T02:27:30.893 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_close": 0, 2024-05-09T02:27:30.893 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_laggy": 0, 2024-05-09T02:27:30.893 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_wr": 0, 2024-05-09T02:27:30.893 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_rd": 2, 2024-05-09T02:27:30.893 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_del": 0 2024-05-09T02:27:30.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.893 INFO:teuthology.orchestra.run.smithi059.stdout: "oft": { 2024-05-09T02:27:30.893 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_objs": 0, 2024-05-09T02:27:30.893 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_kv_pairs": 0, 2024-05-09T02:27:30.893 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_updates": 0, 2024-05-09T02:27:30.893 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_removes": 0 2024-05-09T02:27:30.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.894 INFO:teuthology.orchestra.run.smithi059.stdout: "osd-slow-ops": { 2024-05-09T02:27:30.894 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_ops_count": 0 2024-05-09T02:27:30.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.894 INFO:teuthology.orchestra.run.smithi059.stdout: "purge_queue": { 2024-05-09T02:27:30.894 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops": 0, 2024-05-09T02:27:30.894 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T02:27:30.894 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing": 0, 2024-05-09T02:27:30.894 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_high_water": 0, 2024-05-09T02:27:30.894 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed_ops": 0, 2024-05-09T02:27:30.894 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed": 0, 2024-05-09T02:27:30.894 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_item_in_journal": 0 2024-05-09T02:27:30.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.894 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T02:27:30.895 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:30.895 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T02:27:30.895 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:30.895 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 4283, 2024-05-09T02:27:30.895 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 31857981, 2024-05-09T02:27:30.895 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:30.895 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 4283, 2024-05-09T02:27:30.895 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T02:27:30.895 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T02:27:30.895 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 4283, 2024-05-09T02:27:30.895 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 31857981, 2024-05-09T02:27:30.895 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:30.895 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.895 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.895 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.896 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:30.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.896 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_bytes": { 2024-05-09T02:27:30.896 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:30.896 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T02:27:30.896 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:30.896 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T02:27:30.896 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T02:27:30.896 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:30.896 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:30.896 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 3882, 2024-05-09T02:27:30.896 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 29173827, 2024-05-09T02:27:30.896 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 3882, 2024-05-09T02:27:30.896 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 29173827, 2024-05-09T02:27:30.897 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:30.897 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.897 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.897 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.897 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:30.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.897 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_ops": { 2024-05-09T02:27:30.897 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:30.897 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 1024, 2024-05-09T02:27:30.897 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:30.897 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T02:27:30.897 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T02:27:30.897 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:30.897 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:30.897 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 3882, 2024-05-09T02:27:30.898 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 3882, 2024-05-09T02:27:30.898 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 3882, 2024-05-09T02:27:30.898 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 3882, 2024-05-09T02:27:30.898 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:30.898 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.898 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.898 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.898 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:30.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.898 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle": { 2024-05-09T02:27:30.898 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:30.898 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T02:27:30.898 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:30.898 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T02:27:30.898 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T02:27:30.899 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:30.899 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:30.899 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T02:27:30.899 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T02:27:30.899 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-05-09T02:27:30.899 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-05-09T02:27:30.899 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:30.899 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.899 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.899 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.899 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:30.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:30.899 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle-0x5592f736c410": { 2024-05-09T02:27:30.899 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T02:27:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T02:27:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T02:27:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T02:27:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T02:27:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-05-09T02:27:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-05-09T02:27:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:30.901 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:30.901 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-09T02:27:31.622 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-09T02:27:31.622 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-09T02:27:31.622 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-05-09T02:27:31.622 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-09T02:27:31.622 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T02:27:31.623 DEBUG:tasks.check_counter:Getting stats from h 2024-05-09T02:27:31.623 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-09T02:27:31.875 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T02:27:32.714 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:32 smithi059 ceph-mon[34781]: pgmap v510: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-05-09T02:27:32.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:32 smithi196 ceph-mon[34718]: pgmap v510: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-05-09T02:27:32.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:32 smithi019 ceph-mon[30151]: pgmap v510: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-05-09T02:27:33.622 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-09T02:27:33.622 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T02:27:33.622 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 5339, 2024-05-09T02:27:33.622 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 5205, 2024-05-09T02:27:33.622 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 2764244, 2024-05-09T02:27:33.622 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 6537810, 2024-05-09T02:27:33.622 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 37, 2024-05-09T02:27:33.622 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 7, 2024-05-09T02:27:33.622 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.384271685, 2024-05-09T02:27:33.622 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.138005440, 2024-05-09T02:27:33.622 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.239465464, 2024-05-09T02:27:33.622 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.035861018, 2024-05-09T02:27:33.623 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:33.623 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 5203, 2024-05-09T02:27:33.623 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.252003384, 2024-05-09T02:27:33.623 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000048434 2024-05-09T02:27:33.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.623 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:33.623 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 8363, 2024-05-09T02:27:33.623 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.001870997, 2024-05-09T02:27:33.623 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000223 2024-05-09T02:27:33.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.623 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 2208, 2024-05-09T02:27:33.623 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 9152 2024-05-09T02:27:33.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.623 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T02:27:33.624 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 3654, 2024-05-09T02:27:33.624 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 4731, 2024-05-09T02:27:33.624 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 3198214, 2024-05-09T02:27:33.624 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 2488210, 2024-05-09T02:27:33.624 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 16, 2024-05-09T02:27:33.624 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 8, 2024-05-09T02:27:33.624 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.317364629, 2024-05-09T02:27:33.624 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.116605020, 2024-05-09T02:27:33.624 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.207759362, 2024-05-09T02:27:33.624 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.019396029, 2024-05-09T02:27:33.624 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:33.624 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 4728, 2024-05-09T02:27:33.624 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.189108286, 2024-05-09T02:27:33.624 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000039997 2024-05-09T02:27:33.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.625 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:33.625 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 6183, 2024-05-09T02:27:33.625 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.001375050, 2024-05-09T02:27:33.625 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000222 2024-05-09T02:27:33.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.625 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 2026832, 2024-05-09T02:27:33.625 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 229152 2024-05-09T02:27:33.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.625 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T02:27:33.625 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 1087, 2024-05-09T02:27:33.625 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 1342, 2024-05-09T02:27:33.625 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 477823, 2024-05-09T02:27:33.625 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 1186417, 2024-05-09T02:27:33.625 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 15, 2024-05-09T02:27:33.626 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 5, 2024-05-09T02:27:33.626 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.091502515, 2024-05-09T02:27:33.626 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.032963162, 2024-05-09T02:27:33.626 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.037222545, 2024-05-09T02:27:33.626 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.017508818, 2024-05-09T02:27:33.626 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:33.626 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1342, 2024-05-09T02:27:33.626 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.088098880, 2024-05-09T02:27:33.626 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000065647 2024-05-09T02:27:33.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.626 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:33.626 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 769, 2024-05-09T02:27:33.626 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000168048, 2024-05-09T02:27:33.626 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000218 2024-05-09T02:27:33.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.627 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 4192, 2024-05-09T02:27:33.627 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 597728 2024-05-09T02:27:33.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.627 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-05-09T02:27:33.627 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 1, 2024-05-09T02:27:33.627 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-05-09T02:27:33.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.627 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-MDSRank": { 2024-05-09T02:27:33.627 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T02:27:33.627 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T02:27:33.627 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1727, 2024-05-09T02:27:33.627 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.680641172, 2024-05-09T02:27:33.627 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000394117 2024-05-09T02:27:33.628 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:33.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.628 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-PurgeQueue": { 2024-05-09T02:27:33.628 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T02:27:33.628 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T02:27:33.628 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 699, 2024-05-09T02:27:33.628 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.032001612, 2024-05-09T02:27:33.628 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000045781 2024-05-09T02:27:33.628 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:33.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.628 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": { 2024-05-09T02:27:33.628 INFO:teuthology.orchestra.run.smithi059.stdout: "request": 863, 2024-05-09T02:27:33.628 INFO:teuthology.orchestra.run.smithi059.stdout: "reply": 847, 2024-05-09T02:27:33.628 INFO:teuthology.orchestra.run.smithi059.stdout: "reply_latency": { 2024-05-09T02:27:33.628 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 847, 2024-05-09T02:27:33.629 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 6.572578097, 2024-05-09T02:27:33.629 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.007759832 2024-05-09T02:27:33.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.629 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_reply": 0, 2024-05-09T02:27:33.629 INFO:teuthology.orchestra.run.smithi059.stdout: "forward": 16, 2024-05-09T02:27:33.629 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_complete": 1, 2024-05-09T02:27:33.629 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_keys": 0, 2024-05-09T02:27:33.629 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_commit": 11, 2024-05-09T02:27:33.629 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_split": 0, 2024-05-09T02:27:33.629 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_merge": 0, 2024-05-09T02:27:33.629 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes": 357, 2024-05-09T02:27:33.629 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_top": 14, 2024-05-09T02:27:33.629 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_bottom": 308, 2024-05-09T02:27:33.629 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pin_tail": 35, 2024-05-09T02:27:33.629 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pinned": 336, 2024-05-09T02:27:33.630 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_expired": 199, 2024-05-09T02:27:33.630 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_with_caps": 195, 2024-05-09T02:27:33.630 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": 195, 2024-05-09T02:27:33.630 INFO:teuthology.orchestra.run.smithi059.stdout: "subtrees": 18, 2024-05-09T02:27:33.630 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse": 2501, 2024-05-09T02:27:33.630 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_hit": 2055, 2024-05-09T02:27:33.630 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_forward": 16, 2024-05-09T02:27:33.630 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_discover": 39, 2024-05-09T02:27:33.630 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_dir_fetch": 0, 2024-05-09T02:27:33.630 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_remote_ino": 0, 2024-05-09T02:27:33.630 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_lock": 84, 2024-05-09T02:27:33.630 INFO:teuthology.orchestra.run.smithi059.stdout: "load_cent": 135, 2024-05-09T02:27:33.630 INFO:teuthology.orchestra.run.smithi059.stdout: "q": 0, 2024-05-09T02:27:33.630 INFO:teuthology.orchestra.run.smithi059.stdout: "exported": 3, 2024-05-09T02:27:33.630 INFO:teuthology.orchestra.run.smithi059.stdout: "exported_inodes": 151, 2024-05-09T02:27:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: "imported": 8, 2024-05-09T02:27:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: "imported_inodes": 366, 2024-05-09T02:27:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_dir_fetch": 0, 2024-05-09T02:27:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_backtrace_fetch": 930, 2024-05-09T02:27:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_peer_discover": 6, 2024-05-09T02:27:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rfiles": 998, 2024-05-09T02:27:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rbytes": 1205583148, 2024-05-09T02:27:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rsnaps": 6, 2024-05-09T02:27:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_fetch": 930, 2024-05-09T02:27:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_set_tag": 0, 2024-05-09T02:27:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T02:27:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_inotable_repaired": 0, 2024-05-09T02:27:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_inodes": 0, 2024-05-09T02:27:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T02:27:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_file_inodes": 930, 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_inode_file_caps": 15, 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_revoke": 182, 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_grant": 737, 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps": 633, 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps_dirty": 267, 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_cap_release": 159, 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: "process_request_cap_release": 858 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache": { 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update": 0, 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update_receipt": 0, 2024-05-09T02:27:33.632 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_try_discover": 0, 2024-05-09T02:27:33.633 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_send_discover": 50, 2024-05-09T02:27:33.633 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_handle_discover": 24, 2024-05-09T02:27:33.633 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays": 2, 2024-05-09T02:27:33.633 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_delayed": 0, 2024-05-09T02:27:33.633 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_enqueuing": 0, 2024-05-09T02:27:33.633 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_created": 217, 2024-05-09T02:27:33.633 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_enqueued": 202, 2024-05-09T02:27:33.633 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_reintegrated": 11, 2024-05-09T02:27:33.633 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_migrated": 12, 2024-05-09T02:27:33.633 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_processing": 0, 2024-05-09T02:27:33.633 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_enqueued": 0, 2024-05-09T02:27:33.633 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_prioritized": 0, 2024-05-09T02:27:33.633 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_started": 0, 2024-05-09T02:27:33.633 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_completed": 0, 2024-05-09T02:27:33.633 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_path": 0, 2024-05-09T02:27:33.634 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_inode": 0, 2024-05-09T02:27:33.634 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T02:27:33.634 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_exportdir": 3, 2024-05-09T02:27:33.634 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_flush": 0, 2024-05-09T02:27:33.634 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragmentdir": 0, 2024-05-09T02:27:33.634 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragstats": 0, 2024-05-09T02:27:33.634 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_inodestats": 0 2024-05-09T02:27:33.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.634 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log": { 2024-05-09T02:27:33.634 INFO:teuthology.orchestra.run.smithi059.stdout: "evlrg": 0, 2024-05-09T02:27:33.634 INFO:teuthology.orchestra.run.smithi059.stdout: "evadd": 1523, 2024-05-09T02:27:33.634 INFO:teuthology.orchestra.run.smithi059.stdout: "evex": 0, 2024-05-09T02:27:33.634 INFO:teuthology.orchestra.run.smithi059.stdout: "evtrm": 0, 2024-05-09T02:27:33.634 INFO:teuthology.orchestra.run.smithi059.stdout: "ev": 1523, 2024-05-09T02:27:33.634 INFO:teuthology.orchestra.run.smithi059.stdout: "evexg": 0, 2024-05-09T02:27:33.635 INFO:teuthology.orchestra.run.smithi059.stdout: "evexd": 0, 2024-05-09T02:27:33.635 INFO:teuthology.orchestra.run.smithi059.stdout: "segadd": 3, 2024-05-09T02:27:33.635 INFO:teuthology.orchestra.run.smithi059.stdout: "segex": 0, 2024-05-09T02:27:33.635 INFO:teuthology.orchestra.run.smithi059.stdout: "segtrm": 0, 2024-05-09T02:27:33.635 INFO:teuthology.orchestra.run.smithi059.stdout: "seg": 3, 2024-05-09T02:27:33.635 INFO:teuthology.orchestra.run.smithi059.stdout: "segmjr": 2, 2024-05-09T02:27:33.635 INFO:teuthology.orchestra.run.smithi059.stdout: "segexg": 0, 2024-05-09T02:27:33.635 INFO:teuthology.orchestra.run.smithi059.stdout: "segexd": 0, 2024-05-09T02:27:33.635 INFO:teuthology.orchestra.run.smithi059.stdout: "expos": 4194304, 2024-05-09T02:27:33.635 INFO:teuthology.orchestra.run.smithi059.stdout: "wrpos": 8157209, 2024-05-09T02:27:33.635 INFO:teuthology.orchestra.run.smithi059.stdout: "rdpos": 0, 2024-05-09T02:27:33.635 INFO:teuthology.orchestra.run.smithi059.stdout: "jlat": { 2024-05-09T02:27:33.635 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 419, 2024-05-09T02:27:33.636 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 3.225809791, 2024-05-09T02:27:33.636 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.007698830 2024-05-09T02:27:33.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.636 INFO:teuthology.orchestra.run.smithi059.stdout: "replayed": 0 2024-05-09T02:27:33.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.636 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mem": { 2024-05-09T02:27:33.636 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 308, 2024-05-09T02:27:33.636 INFO:teuthology.orchestra.run.smithi059.stdout: "ino+": 659, 2024-05-09T02:27:33.636 INFO:teuthology.orchestra.run.smithi059.stdout: "ino-": 351, 2024-05-09T02:27:33.636 INFO:teuthology.orchestra.run.smithi059.stdout: "dir": 44, 2024-05-09T02:27:33.636 INFO:teuthology.orchestra.run.smithi059.stdout: "dir+": 50, 2024-05-09T02:27:33.637 INFO:teuthology.orchestra.run.smithi059.stdout: "dir-": 6, 2024-05-09T02:27:33.637 INFO:teuthology.orchestra.run.smithi059.stdout: "dn": 361, 2024-05-09T02:27:33.637 INFO:teuthology.orchestra.run.smithi059.stdout: "dn+": 764, 2024-05-09T02:27:33.637 INFO:teuthology.orchestra.run.smithi059.stdout: "dn-": 403, 2024-05-09T02:27:33.637 INFO:teuthology.orchestra.run.smithi059.stdout: "cap": 195, 2024-05-09T02:27:33.637 INFO:teuthology.orchestra.run.smithi059.stdout: "cap+": 539, 2024-05-09T02:27:33.637 INFO:teuthology.orchestra.run.smithi059.stdout: "cap-": 344, 2024-05-09T02:27:33.637 INFO:teuthology.orchestra.run.smithi059.stdout: "rss": 131028, 2024-05-09T02:27:33.637 INFO:teuthology.orchestra.run.smithi059.stdout: "heap": 223632 2024-05-09T02:27:33.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.637 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_server": { 2024-05-09T02:27:33.637 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_client_request": 1819, 2024-05-09T02:27:33.637 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_server_request": 323, 2024-05-09T02:27:33.637 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_request": 863, 2024-05-09T02:27:33.637 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_session": 57, 2024-05-09T02:27:33.638 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_peer_request": 894, 2024-05-09T02:27:33.638 INFO:teuthology.orchestra.run.smithi059.stdout: "req_create_latency": { 2024-05-09T02:27:33.638 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 199, 2024-05-09T02:27:33.638 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.282953511, 2024-05-09T02:27:33.638 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.001421876 2024-05-09T02:27:33.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.638 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getattr_latency": { 2024-05-09T02:27:33.638 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 15, 2024-05-09T02:27:33.638 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.006548011, 2024-05-09T02:27:33.638 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000436534 2024-05-09T02:27:33.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.638 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getfilelock_latency": { 2024-05-09T02:27:33.639 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.639 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.639 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.639 INFO:teuthology.orchestra.run.smithi059.stdout: "req_link_latency": { 2024-05-09T02:27:33.639 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 16, 2024-05-09T02:27:33.639 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.154685867, 2024-05-09T02:27:33.641 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.009667866 2024-05-09T02:27:33.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.642 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookup_latency": { 2024-05-09T02:27:33.642 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 245, 2024-05-09T02:27:33.642 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.957265488, 2024-05-09T02:27:33.642 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.003907206 2024-05-09T02:27:33.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.642 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookuphash_latency": { 2024-05-09T02:27:33.642 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.642 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.642 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.642 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupino_latency": { 2024-05-09T02:27:33.642 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.643 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.643 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.643 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupname_latency": { 2024-05-09T02:27:33.643 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.643 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.643 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.643 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupparent_latency": { 2024-05-09T02:27:33.643 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.643 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.644 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.644 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupsnap_latency": { 2024-05-09T02:27:33.644 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.644 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.644 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.644 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lssnap_latency": { 2024-05-09T02:27:33.645 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.645 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.645 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.646 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mkdir_latency": { 2024-05-09T02:27:33.646 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.646 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.646 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.646 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mknod_latency": { 2024-05-09T02:27:33.646 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.646 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.646 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.646 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mksnap_latency": { 2024-05-09T02:27:33.646 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.647 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.647 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.647 INFO:teuthology.orchestra.run.smithi059.stdout: "req_open_latency": { 2024-05-09T02:27:33.647 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 125, 2024-05-09T02:27:33.647 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.085339103, 2024-05-09T02:27:33.647 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000682712 2024-05-09T02:27:33.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.647 INFO:teuthology.orchestra.run.smithi059.stdout: "req_readdir_latency": { 2024-05-09T02:27:33.647 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 9, 2024-05-09T02:27:33.647 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.011407337, 2024-05-09T02:27:33.647 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.001267481 2024-05-09T02:27:33.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.647 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rename_latency": { 2024-05-09T02:27:33.647 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 220, 2024-05-09T02:27:33.648 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 4.989709479, 2024-05-09T02:27:33.648 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.022680497 2024-05-09T02:27:33.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.648 INFO:teuthology.orchestra.run.smithi059.stdout: "req_renamesnap_latency": { 2024-05-09T02:27:33.648 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.648 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.648 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.648 INFO:teuthology.orchestra.run.smithi059.stdout: "req_snapdiff_latency": { 2024-05-09T02:27:33.648 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.648 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.648 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.648 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmdir_latency": { 2024-05-09T02:27:33.649 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.649 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.649 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.649 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmsnap_latency": { 2024-05-09T02:27:33.649 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.649 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.649 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.649 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmxattr_latency": { 2024-05-09T02:27:33.649 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.649 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.649 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.649 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setattr_latency": { 2024-05-09T02:27:33.650 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.650 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.650 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.650 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setdirlayout_latency": { 2024-05-09T02:27:33.650 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.650 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.650 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.650 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setfilelock_latency": { 2024-05-09T02:27:33.650 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.650 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.650 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.650 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setlayout_latency": { 2024-05-09T02:27:33.651 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.651 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.651 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.651 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setxattr_latency": { 2024-05-09T02:27:33.651 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.651 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.651 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.651 INFO:teuthology.orchestra.run.smithi059.stdout: "req_symlink_latency": { 2024-05-09T02:27:33.651 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.651 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.652 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.652 INFO:teuthology.orchestra.run.smithi059.stdout: "req_unlink_latency": { 2024-05-09T02:27:33.652 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 18, 2024-05-09T02:27:33.652 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.084669301, 2024-05-09T02:27:33.652 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.004703850 2024-05-09T02:27:33.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.652 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_revoke_eviction": 0, 2024-05-09T02:27:33.652 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_acquisition_throttle": 0, 2024-05-09T02:27:33.652 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getvxattr_latency": { 2024-05-09T02:27:33.652 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.652 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.652 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.653 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:33.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.653 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessions": { 2024-05-09T02:27:33.653 INFO:teuthology.orchestra.run.smithi059.stdout: "session_count": 2, 2024-05-09T02:27:33.653 INFO:teuthology.orchestra.run.smithi059.stdout: "session_add": 4, 2024-05-09T02:27:33.653 INFO:teuthology.orchestra.run.smithi059.stdout: "session_remove": 2, 2024-05-09T02:27:33.653 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_open": 2, 2024-05-09T02:27:33.653 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_stale": 0, 2024-05-09T02:27:33.653 INFO:teuthology.orchestra.run.smithi059.stdout: "total_load": 114, 2024-05-09T02:27:33.653 INFO:teuthology.orchestra.run.smithi059.stdout: "average_load": 57, 2024-05-09T02:27:33.653 INFO:teuthology.orchestra.run.smithi059.stdout: "avg_session_uptime": 932, 2024-05-09T02:27:33.653 INFO:teuthology.orchestra.run.smithi059.stdout: "mdthresh_evicted": 0 2024-05-09T02:27:33.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.653 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-05-09T02:27:33.653 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-05-09T02:27:33.654 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-05-09T02:27:33.654 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T02:27:33.654 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-05-09T02:27:33.654 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T02:27:33.654 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-05-09T02:27:33.654 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T02:27:33.654 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T02:27:33.654 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T02:27:33.654 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T02:27:33.654 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T02:27:33.654 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-05-09T02:27:33.654 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T02:27:33.654 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T02:27:33.654 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-05-09T02:27:33.655 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-05-09T02:27:33.655 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-05-09T02:27:33.655 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-05-09T02:27:33.655 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T02:27:33.655 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T02:27:33.655 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T02:27:33.655 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T02:27:33.655 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T02:27:33.655 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-05-09T02:27:33.655 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-05-09T02:27:33.655 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-05-09T02:27:33.655 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T02:27:33.655 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T02:27:33.655 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-05-09T02:27:33.655 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-05-09T02:27:33.656 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-05-09T02:27:33.656 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-05-09T02:27:33.656 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T02:27:33.656 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-05-09T02:27:33.656 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T02:27:33.656 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-05-09T02:27:33.656 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 528886, 2024-05-09T02:27:33.656 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 489, 2024-05-09T02:27:33.656 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 616, 2024-05-09T02:27:33.656 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 7, 2024-05-09T02:27:33.656 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-05-09T02:27:33.656 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-05-09T02:27:33.656 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-05-09T02:27:33.656 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-05-09T02:27:33.657 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-05-09T02:27:33.657 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-05-09T02:27:33.657 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 8224, 2024-05-09T02:27:33.657 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 171, 2024-05-09T02:27:33.657 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T02:27:33.657 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-05-09T02:27:33.657 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-05-09T02:27:33.657 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-05-09T02:27:33.657 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 1219948, 2024-05-09T02:27:33.657 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 18656, 2024-05-09T02:27:33.657 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-05-09T02:27:33.657 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-05-09T02:27:33.657 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-05-09T02:27:33.657 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-05-09T02:27:33.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.658 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter": { 2024-05-09T02:27:33.658 INFO:teuthology.orchestra.run.smithi059.stdout: "op_active": 0, 2024-05-09T02:27:33.658 INFO:teuthology.orchestra.run.smithi059.stdout: "op_laggy": 0, 2024-05-09T02:27:33.658 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send": 1990, 2024-05-09T02:27:33.658 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send_bytes": 4043738, 2024-05-09T02:27:33.658 INFO:teuthology.orchestra.run.smithi059.stdout: "op_resend": 33, 2024-05-09T02:27:33.658 INFO:teuthology.orchestra.run.smithi059.stdout: "op_reply": 1990, 2024-05-09T02:27:33.658 INFO:teuthology.orchestra.run.smithi059.stdout: "op_latency": { 2024-05-09T02:27:33.658 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1990, 2024-05-09T02:27:33.659 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 12.311850503, 2024-05-09T02:27:33.659 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.006186859 2024-05-09T02:27:33.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.659 INFO:teuthology.orchestra.run.smithi059.stdout: "op_inflight": 0, 2024-05-09T02:27:33.659 INFO:teuthology.orchestra.run.smithi059.stdout: "oplen_avg": { 2024-05-09T02:27:33.659 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1990, 2024-05-09T02:27:33.659 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 2128 2024-05-09T02:27:33.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.659 INFO:teuthology.orchestra.run.smithi059.stdout: "op": 1990, 2024-05-09T02:27:33.659 INFO:teuthology.orchestra.run.smithi059.stdout: "op_r": 1012, 2024-05-09T02:27:33.659 INFO:teuthology.orchestra.run.smithi059.stdout: "op_w": 978, 2024-05-09T02:27:33.659 INFO:teuthology.orchestra.run.smithi059.stdout: "op_rmw": 0, 2024-05-09T02:27:33.659 INFO:teuthology.orchestra.run.smithi059.stdout: "op_pg": 0, 2024-05-09T02:27:33.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_stat": 0, 2024-05-09T02:27:33.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_create": 10, 2024-05-09T02:27:33.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_read": 80, 2024-05-09T02:27:33.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_write": 499, 2024-05-09T02:27:33.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writefull": 130, 2024-05-09T02:27:33.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writesame": 0, 2024-05-09T02:27:33.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_append": 0, 2024-05-09T02:27:33.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_zero": 0, 2024-05-09T02:27:33.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_truncate": 0, 2024-05-09T02:27:33.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_delete": 214, 2024-05-09T02:27:33.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_mapext": 0, 2024-05-09T02:27:33.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_sparse_read": 0, 2024-05-09T02:27:33.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_clonerange": 0, 2024-05-09T02:27:33.660 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_getxattr": 930, 2024-05-09T02:27:33.661 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_setxattr": 20, 2024-05-09T02:27:33.661 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_cmpxattr": 0, 2024-05-09T02:27:33.661 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_rmxattr": 0, 2024-05-09T02:27:33.661 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_resetxattrs": 0, 2024-05-09T02:27:33.661 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_call": 0, 2024-05-09T02:27:33.661 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_watch": 0, 2024-05-09T02:27:33.661 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_notify": 0, 2024-05-09T02:27:33.661 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T02:27:33.661 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls": 0, 2024-05-09T02:27:33.661 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls_filter": 0, 2024-05-09T02:27:33.661 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_other": 0, 2024-05-09T02:27:33.661 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_active": 0, 2024-05-09T02:27:33.661 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_send": 0, 2024-05-09T02:27:33.661 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_resend": 0, 2024-05-09T02:27:33.662 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_ping": 0, 2024-05-09T02:27:33.662 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_active": 0, 2024-05-09T02:27:33.662 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_send": 0, 2024-05-09T02:27:33.662 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_resend": 0, 2024-05-09T02:27:33.662 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_active": 0, 2024-05-09T02:27:33.662 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_send": 0, 2024-05-09T02:27:33.662 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_resend": 0, 2024-05-09T02:27:33.662 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_active": 0, 2024-05-09T02:27:33.662 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_send": 0, 2024-05-09T02:27:33.662 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_resend": 0, 2024-05-09T02:27:33.662 INFO:teuthology.orchestra.run.smithi059.stdout: "command_active": 0, 2024-05-09T02:27:33.662 INFO:teuthology.orchestra.run.smithi059.stdout: "command_send": 0, 2024-05-09T02:27:33.663 INFO:teuthology.orchestra.run.smithi059.stdout: "command_resend": 0, 2024-05-09T02:27:33.663 INFO:teuthology.orchestra.run.smithi059.stdout: "map_epoch": 102, 2024-05-09T02:27:33.663 INFO:teuthology.orchestra.run.smithi059.stdout: "map_full": 0, 2024-05-09T02:27:33.663 INFO:teuthology.orchestra.run.smithi059.stdout: "map_inc": 23, 2024-05-09T02:27:33.663 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_sessions": 12, 2024-05-09T02:27:33.663 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_open": 12, 2024-05-09T02:27:33.663 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_close": 0, 2024-05-09T02:27:33.663 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_laggy": 0, 2024-05-09T02:27:33.663 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_wr": 179, 2024-05-09T02:27:33.663 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_rd": 3, 2024-05-09T02:27:33.663 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_del": 63 2024-05-09T02:27:33.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.663 INFO:teuthology.orchestra.run.smithi059.stdout: "oft": { 2024-05-09T02:27:33.663 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_objs": 1, 2024-05-09T02:27:33.663 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_kv_pairs": 0, 2024-05-09T02:27:33.664 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_updates": 0, 2024-05-09T02:27:33.664 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_removes": 0 2024-05-09T02:27:33.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.664 INFO:teuthology.orchestra.run.smithi059.stdout: "osd-slow-ops": { 2024-05-09T02:27:33.664 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_ops_count": 0 2024-05-09T02:27:33.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.664 INFO:teuthology.orchestra.run.smithi059.stdout: "purge_queue": { 2024-05-09T02:27:33.664 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops": 0, 2024-05-09T02:27:33.664 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops_high_water": 6, 2024-05-09T02:27:33.664 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing": 0, 2024-05-09T02:27:33.664 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_high_water": 6, 2024-05-09T02:27:33.664 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed_ops": 202, 2024-05-09T02:27:33.664 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed": 202, 2024-05-09T02:27:33.664 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_item_in_journal": 0 2024-05-09T02:27:33.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.665 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T02:27:33.665 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:33.665 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T02:27:33.665 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:33.665 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 10076, 2024-05-09T02:27:33.665 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 5506859, 2024-05-09T02:27:33.665 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:33.665 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 10076, 2024-05-09T02:27:33.665 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T02:27:33.665 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T02:27:33.665 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 10076, 2024-05-09T02:27:33.665 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 5506859, 2024-05-09T02:27:33.665 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:33.665 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.666 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.666 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.666 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:33.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.666 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_bytes": { 2024-05-09T02:27:33.666 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:33.666 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T02:27:33.666 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:33.666 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T02:27:33.666 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T02:27:33.666 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:33.666 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:33.666 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 1990, 2024-05-09T02:27:33.666 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 4064418, 2024-05-09T02:27:33.666 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 1990, 2024-05-09T02:27:33.667 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 4064418, 2024-05-09T02:27:33.667 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:33.667 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.667 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.667 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.667 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:33.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.667 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_ops": { 2024-05-09T02:27:33.667 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:33.667 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 1024, 2024-05-09T02:27:33.667 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:33.667 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T02:27:33.667 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T02:27:33.667 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:33.667 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:33.668 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 1990, 2024-05-09T02:27:33.668 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 1990, 2024-05-09T02:27:33.668 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 1990, 2024-05-09T02:27:33.668 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 1990, 2024-05-09T02:27:33.668 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:33.668 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.668 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.668 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.668 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:33.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.668 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle": { 2024-05-09T02:27:33.668 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:33.668 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T02:27:33.668 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 202, 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 20738, 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 202, 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 80, 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 20738, 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle-0x559e4ed8c5b0": { 2024-05-09T02:27:33.669 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:33.670 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T02:27:33.670 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:33.670 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 1523, 2024-05-09T02:27:33.670 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 3962905, 2024-05-09T02:27:33.670 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:33.670 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 1523, 2024-05-09T02:27:33.670 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T02:27:33.670 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T02:27:33.670 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 419, 2024-05-09T02:27:33.670 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 3962905, 2024-05-09T02:27:33.670 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:33.670 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:33.670 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:33.670 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:33.670 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:33.671 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:33.671 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-09T02:27:34.473 INFO:tasks.check_counter:Daemon mds.h mds.exported=3 2024-05-09T02:27:34.473 INFO:tasks.check_counter:Daemon mds.h mds.imported=8 2024-05-09T02:27:34.473 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-05-09T02:27:34.473 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-09T02:27:34.473 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T02:27:34.474 DEBUG:tasks.check_counter:Getting stats from k 2024-05-09T02:27:34.474 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-09T02:27:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T02:27:34.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:34 smithi019 ceph-mon[30151]: pgmap v511: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-05-09T02:27:34.847 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:34 smithi059 ceph-mon[34781]: pgmap v511: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-05-09T02:27:35.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:34 smithi196 ceph-mon[34718]: pgmap v511: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-05-09T02:27:36.519 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-09T02:27:36.519 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T02:27:36.519 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 407, 2024-05-09T02:27:36.519 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 359, 2024-05-09T02:27:36.519 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 2014434, 2024-05-09T02:27:36.519 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 208452, 2024-05-09T02:27:36.519 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 6, 2024-05-09T02:27:36.519 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 1, 2024-05-09T02:27:36.520 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.067600893, 2024-05-09T02:27:36.520 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.018965902, 2024-05-09T02:27:36.520 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.054013631, 2024-05-09T02:27:36.520 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.007820877, 2024-05-09T02:27:36.520 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:36.520 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 359, 2024-05-09T02:27:36.520 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.014915466, 2024-05-09T02:27:36.520 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000041547 2024-05-09T02:27:36.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.520 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:36.520 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.520 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.521 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.521 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 2013600, 2024-05-09T02:27:36.521 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 208128 2024-05-09T02:27:36.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.521 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T02:27:36.521 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 1059, 2024-05-09T02:27:36.521 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 1308, 2024-05-09T02:27:36.521 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 260364, 2024-05-09T02:27:36.521 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 931011, 2024-05-09T02:27:36.522 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 7, 2024-05-09T02:27:36.522 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 5, 2024-05-09T02:27:36.522 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.107452242, 2024-05-09T02:27:36.522 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.033169784, 2024-05-09T02:27:36.522 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.023793070, 2024-05-09T02:27:36.522 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.038155081, 2024-05-09T02:27:36.522 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:36.522 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1305, 2024-05-09T02:27:36.522 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.080225772, 2024-05-09T02:27:36.522 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000061475 2024-05-09T02:27:36.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.522 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:36.523 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.523 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.523 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.523 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T02:27:36.523 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 594880 2024-05-09T02:27:36.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.523 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T02:27:36.523 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 2533, 2024-05-09T02:27:36.523 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 2535, 2024-05-09T02:27:36.523 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 2790228, 2024-05-09T02:27:36.523 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 805552, 2024-05-09T02:27:36.523 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2024-05-09T02:27:36.524 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 3, 2024-05-09T02:27:36.524 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.240301909, 2024-05-09T02:27:36.524 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.076703223, 2024-05-09T02:27:36.524 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.064371894, 2024-05-09T02:27:36.524 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.078812909, 2024-05-09T02:27:36.524 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:36.524 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2533, 2024-05-09T02:27:36.524 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.133027397, 2024-05-09T02:27:36.524 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000052517 2024-05-09T02:27:36.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.524 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:36.524 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.525 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.525 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.525 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T02:27:36.525 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T02:27:36.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.525 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-05-09T02:27:36.525 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 1, 2024-05-09T02:27:36.525 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-05-09T02:27:36.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.525 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-MDSRank": { 2024-05-09T02:27:36.525 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T02:27:36.525 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T02:27:36.526 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 6062, 2024-05-09T02:27:36.526 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.177339607, 2024-05-09T02:27:36.526 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000029254 2024-05-09T02:27:36.526 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:36.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.526 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-PurgeQueue": { 2024-05-09T02:27:36.526 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-09T02:27:36.526 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-09T02:27:36.526 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.526 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.526 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.527 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:36.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.527 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": { 2024-05-09T02:27:36.527 INFO:teuthology.orchestra.run.smithi059.stdout: "request": 0, 2024-05-09T02:27:36.527 INFO:teuthology.orchestra.run.smithi059.stdout: "reply": 0, 2024-05-09T02:27:36.527 INFO:teuthology.orchestra.run.smithi059.stdout: "reply_latency": { 2024-05-09T02:27:36.527 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.527 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.527 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.527 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_reply": 0, 2024-05-09T02:27:36.527 INFO:teuthology.orchestra.run.smithi059.stdout: "forward": 0, 2024-05-09T02:27:36.527 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_complete": 0, 2024-05-09T02:27:36.528 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_keys": 0, 2024-05-09T02:27:36.528 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_commit": 0, 2024-05-09T02:27:36.528 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_split": 0, 2024-05-09T02:27:36.528 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_merge": 0, 2024-05-09T02:27:36.528 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes": 521, 2024-05-09T02:27:36.528 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_top": 9, 2024-05-09T02:27:36.528 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_bottom": 512, 2024-05-09T02:27:36.528 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pin_tail": 0, 2024-05-09T02:27:36.528 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pinned": 508, 2024-05-09T02:27:36.528 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_expired": 0, 2024-05-09T02:27:36.528 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_with_caps": 0, 2024-05-09T02:27:36.528 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": 0, 2024-05-09T02:27:36.529 INFO:teuthology.orchestra.run.smithi059.stdout: "subtrees": 21, 2024-05-09T02:27:36.529 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse": 0, 2024-05-09T02:27:36.529 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_hit": 0, 2024-05-09T02:27:36.529 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_forward": 0, 2024-05-09T02:27:36.529 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_discover": 0, 2024-05-09T02:27:36.529 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_dir_fetch": 0, 2024-05-09T02:27:36.529 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_remote_ino": 0, 2024-05-09T02:27:36.529 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_lock": 0, 2024-05-09T02:27:36.529 INFO:teuthology.orchestra.run.smithi059.stdout: "load_cent": 0, 2024-05-09T02:27:36.529 INFO:teuthology.orchestra.run.smithi059.stdout: "q": 0, 2024-05-09T02:27:36.529 INFO:teuthology.orchestra.run.smithi059.stdout: "exported": 0, 2024-05-09T02:27:36.529 INFO:teuthology.orchestra.run.smithi059.stdout: "exported_inodes": 0, 2024-05-09T02:27:36.529 INFO:teuthology.orchestra.run.smithi059.stdout: "imported": 0, 2024-05-09T02:27:36.530 INFO:teuthology.orchestra.run.smithi059.stdout: "imported_inodes": 0, 2024-05-09T02:27:36.530 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_dir_fetch": 0, 2024-05-09T02:27:36.530 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_backtrace_fetch": 0, 2024-05-09T02:27:36.530 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_peer_discover": 0, 2024-05-09T02:27:36.530 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rfiles": 0, 2024-05-09T02:27:36.530 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rbytes": 0, 2024-05-09T02:27:36.530 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rsnaps": 0, 2024-05-09T02:27:36.530 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T02:27:36.530 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_set_tag": 0, 2024-05-09T02:27:36.530 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T02:27:36.530 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_inotable_repaired": 0, 2024-05-09T02:27:36.530 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_inodes": 0, 2024-05-09T02:27:36.531 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T02:27:36.531 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T02:27:36.531 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_file_inodes": 0, 2024-05-09T02:27:36.531 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_inode_file_caps": 0, 2024-05-09T02:27:36.531 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T02:27:36.531 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_grant": 0, 2024-05-09T02:27:36.531 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T02:27:36.531 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T02:27:36.531 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T02:27:36.531 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps": 0, 2024-05-09T02:27:36.531 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps_dirty": 0, 2024-05-09T02:27:36.531 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_cap_release": 0, 2024-05-09T02:27:36.532 INFO:teuthology.orchestra.run.smithi059.stdout: "process_request_cap_release": 0 2024-05-09T02:27:36.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.532 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache": { 2024-05-09T02:27:36.532 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update": 0, 2024-05-09T02:27:36.532 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update_receipt": 0, 2024-05-09T02:27:36.532 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_try_discover": 0, 2024-05-09T02:27:36.532 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_send_discover": 0, 2024-05-09T02:27:36.532 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_handle_discover": 0, 2024-05-09T02:27:36.532 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays": 0, 2024-05-09T02:27:36.532 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_delayed": 0, 2024-05-09T02:27:36.532 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_enqueuing": 0, 2024-05-09T02:27:36.532 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_created": 0, 2024-05-09T02:27:36.532 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_enqueued": 0, 2024-05-09T02:27:36.533 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_reintegrated": 0, 2024-05-09T02:27:36.533 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_migrated": 0, 2024-05-09T02:27:36.533 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_processing": 0, 2024-05-09T02:27:36.533 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_enqueued": 0, 2024-05-09T02:27:36.533 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_prioritized": 0, 2024-05-09T02:27:36.533 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_started": 0, 2024-05-09T02:27:36.533 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_completed": 0, 2024-05-09T02:27:36.533 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_path": 0, 2024-05-09T02:27:36.533 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_inode": 0, 2024-05-09T02:27:36.533 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T02:27:36.533 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_exportdir": 0, 2024-05-09T02:27:36.533 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_flush": 0, 2024-05-09T02:27:36.534 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragmentdir": 0, 2024-05-09T02:27:36.534 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragstats": 0, 2024-05-09T02:27:36.534 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_inodestats": 0 2024-05-09T02:27:36.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.534 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log": { 2024-05-09T02:27:36.534 INFO:teuthology.orchestra.run.smithi059.stdout: "evlrg": 0, 2024-05-09T02:27:36.534 INFO:teuthology.orchestra.run.smithi059.stdout: "evadd": 0, 2024-05-09T02:27:36.534 INFO:teuthology.orchestra.run.smithi059.stdout: "evex": 0, 2024-05-09T02:27:36.535 INFO:teuthology.orchestra.run.smithi059.stdout: "evtrm": 0, 2024-05-09T02:27:36.535 INFO:teuthology.orchestra.run.smithi059.stdout: "ev": 1590, 2024-05-09T02:27:36.535 INFO:teuthology.orchestra.run.smithi059.stdout: "evexg": 0, 2024-05-09T02:27:36.535 INFO:teuthology.orchestra.run.smithi059.stdout: "evexd": 0, 2024-05-09T02:27:36.535 INFO:teuthology.orchestra.run.smithi059.stdout: "segadd": 0, 2024-05-09T02:27:36.535 INFO:teuthology.orchestra.run.smithi059.stdout: "segex": 0, 2024-05-09T02:27:36.535 INFO:teuthology.orchestra.run.smithi059.stdout: "segtrm": 0, 2024-05-09T02:27:36.535 INFO:teuthology.orchestra.run.smithi059.stdout: "seg": 3, 2024-05-09T02:27:36.535 INFO:teuthology.orchestra.run.smithi059.stdout: "segmjr": 2, 2024-05-09T02:27:36.535 INFO:teuthology.orchestra.run.smithi059.stdout: "segexg": 0, 2024-05-09T02:27:36.535 INFO:teuthology.orchestra.run.smithi059.stdout: "segexd": 0, 2024-05-09T02:27:36.535 INFO:teuthology.orchestra.run.smithi059.stdout: "expos": 4194304, 2024-05-09T02:27:36.535 INFO:teuthology.orchestra.run.smithi059.stdout: "wrpos": 6238249, 2024-05-09T02:27:36.536 INFO:teuthology.orchestra.run.smithi059.stdout: "rdpos": 6235442, 2024-05-09T02:27:36.536 INFO:teuthology.orchestra.run.smithi059.stdout: "jlat": { 2024-05-09T02:27:36.536 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.536 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.536 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.536 INFO:teuthology.orchestra.run.smithi059.stdout: "replayed": 1590 2024-05-09T02:27:36.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.536 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mem": { 2024-05-09T02:27:36.536 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 447, 2024-05-09T02:27:36.536 INFO:teuthology.orchestra.run.smithi059.stdout: "ino+": 466, 2024-05-09T02:27:36.536 INFO:teuthology.orchestra.run.smithi059.stdout: "ino-": 19, 2024-05-09T02:27:36.537 INFO:teuthology.orchestra.run.smithi059.stdout: "dir": 30, 2024-05-09T02:27:36.537 INFO:teuthology.orchestra.run.smithi059.stdout: "dir+": 32, 2024-05-09T02:27:36.537 INFO:teuthology.orchestra.run.smithi059.stdout: "dir-": 2, 2024-05-09T02:27:36.537 INFO:teuthology.orchestra.run.smithi059.stdout: "dn": 521, 2024-05-09T02:27:36.537 INFO:teuthology.orchestra.run.smithi059.stdout: "dn+": 521, 2024-05-09T02:27:36.537 INFO:teuthology.orchestra.run.smithi059.stdout: "dn-": 0, 2024-05-09T02:27:36.537 INFO:teuthology.orchestra.run.smithi059.stdout: "cap": 0, 2024-05-09T02:27:36.537 INFO:teuthology.orchestra.run.smithi059.stdout: "cap+": 0, 2024-05-09T02:27:36.537 INFO:teuthology.orchestra.run.smithi059.stdout: "cap-": 0, 2024-05-09T02:27:36.537 INFO:teuthology.orchestra.run.smithi059.stdout: "rss": 126272, 2024-05-09T02:27:36.537 INFO:teuthology.orchestra.run.smithi059.stdout: "heap": 215440 2024-05-09T02:27:36.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.537 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_server": { 2024-05-09T02:27:36.538 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_client_request": 0, 2024-05-09T02:27:36.538 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_server_request": 0, 2024-05-09T02:27:36.538 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_request": 0, 2024-05-09T02:27:36.538 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_session": 0, 2024-05-09T02:27:36.538 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_peer_request": 0, 2024-05-09T02:27:36.538 INFO:teuthology.orchestra.run.smithi059.stdout: "req_create_latency": { 2024-05-09T02:27:36.538 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.538 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.538 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.538 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getattr_latency": { 2024-05-09T02:27:36.538 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.538 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.539 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.539 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getfilelock_latency": { 2024-05-09T02:27:36.539 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.539 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.539 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.539 INFO:teuthology.orchestra.run.smithi059.stdout: "req_link_latency": { 2024-05-09T02:27:36.539 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.540 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.540 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.540 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookup_latency": { 2024-05-09T02:27:36.540 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.540 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.540 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.540 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookuphash_latency": { 2024-05-09T02:27:36.540 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.540 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.540 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.541 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupino_latency": { 2024-05-09T02:27:36.541 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.541 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.541 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.541 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupname_latency": { 2024-05-09T02:27:36.541 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.541 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.541 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.541 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupparent_latency": { 2024-05-09T02:27:36.541 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.542 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.542 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.542 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupsnap_latency": { 2024-05-09T02:27:36.542 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.542 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.542 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.542 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lssnap_latency": { 2024-05-09T02:27:36.542 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.542 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.542 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.543 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mkdir_latency": { 2024-05-09T02:27:36.543 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.544 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.544 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.544 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mknod_latency": { 2024-05-09T02:27:36.544 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.544 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.544 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.544 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mksnap_latency": { 2024-05-09T02:27:36.544 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.544 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.544 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.545 INFO:teuthology.orchestra.run.smithi059.stdout: "req_open_latency": { 2024-05-09T02:27:36.545 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.545 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.545 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.545 INFO:teuthology.orchestra.run.smithi059.stdout: "req_readdir_latency": { 2024-05-09T02:27:36.545 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.545 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.545 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.545 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rename_latency": { 2024-05-09T02:27:36.545 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.546 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.546 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.546 INFO:teuthology.orchestra.run.smithi059.stdout: "req_renamesnap_latency": { 2024-05-09T02:27:36.546 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.546 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.546 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.546 INFO:teuthology.orchestra.run.smithi059.stdout: "req_snapdiff_latency": { 2024-05-09T02:27:36.546 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.546 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.546 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.547 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmdir_latency": { 2024-05-09T02:27:36.547 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.547 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.547 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.547 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmsnap_latency": { 2024-05-09T02:27:36.547 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.547 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.547 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.547 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmxattr_latency": { 2024-05-09T02:27:36.548 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.548 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.548 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.548 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setattr_latency": { 2024-05-09T02:27:36.548 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.548 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.548 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.548 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setdirlayout_latency": { 2024-05-09T02:27:36.548 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.548 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.548 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.549 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setfilelock_latency": { 2024-05-09T02:27:36.549 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.549 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.549 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.549 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setlayout_latency": { 2024-05-09T02:27:36.549 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.549 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.549 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.549 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setxattr_latency": { 2024-05-09T02:27:36.549 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.550 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.550 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.550 INFO:teuthology.orchestra.run.smithi059.stdout: "req_symlink_latency": { 2024-05-09T02:27:36.550 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.550 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.550 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.550 INFO:teuthology.orchestra.run.smithi059.stdout: "req_unlink_latency": { 2024-05-09T02:27:36.550 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.550 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.550 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.551 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_revoke_eviction": 0, 2024-05-09T02:27:36.551 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_acquisition_throttle": 0, 2024-05-09T02:27:36.551 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getvxattr_latency": { 2024-05-09T02:27:36.551 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.551 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.551 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.551 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:36.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessions": { 2024-05-09T02:27:36.551 INFO:teuthology.orchestra.run.smithi059.stdout: "session_count": 2, 2024-05-09T02:27:36.551 INFO:teuthology.orchestra.run.smithi059.stdout: "session_add": 4, 2024-05-09T02:27:36.552 INFO:teuthology.orchestra.run.smithi059.stdout: "session_remove": 2, 2024-05-09T02:27:36.552 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_open": 3, 2024-05-09T02:27:36.552 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_stale": 0, 2024-05-09T02:27:36.552 INFO:teuthology.orchestra.run.smithi059.stdout: "total_load": 0, 2024-05-09T02:27:36.552 INFO:teuthology.orchestra.run.smithi059.stdout: "average_load": 0, 2024-05-09T02:27:36.552 INFO:teuthology.orchestra.run.smithi059.stdout: "avg_session_uptime": 3120, 2024-05-09T02:27:36.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mdthresh_evicted": 0 2024-05-09T02:27:36.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-05-09T02:27:36.552 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-05-09T02:27:36.552 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-05-09T02:27:36.552 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T02:27:36.553 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-05-09T02:27:36.553 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T02:27:36.553 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-05-09T02:27:36.553 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T02:27:36.553 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T02:27:36.553 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T02:27:36.553 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T02:27:36.553 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T02:27:36.553 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-05-09T02:27:36.553 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T02:27:36.553 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T02:27:36.553 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-05-09T02:27:36.553 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-05-09T02:27:36.554 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-05-09T02:27:36.554 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-05-09T02:27:36.554 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T02:27:36.554 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T02:27:36.554 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T02:27:36.554 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T02:27:36.554 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T02:27:36.554 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-05-09T02:27:36.554 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-05-09T02:27:36.554 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-05-09T02:27:36.554 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T02:27:36.555 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T02:27:36.555 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-05-09T02:27:36.555 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-05-09T02:27:36.555 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-05-09T02:27:36.555 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-05-09T02:27:36.555 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T02:27:36.555 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-05-09T02:27:36.555 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T02:27:36.555 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-05-09T02:27:36.555 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 207955, 2024-05-09T02:27:36.555 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 513, 2024-05-09T02:27:36.555 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-05-09T02:27:36.556 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-05-09T02:27:36.556 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-05-09T02:27:36.556 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-05-09T02:27:36.556 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-05-09T02:27:36.556 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-05-09T02:27:36.556 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-05-09T02:27:36.556 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-05-09T02:27:36.556 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 8224, 2024-05-09T02:27:36.556 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 171, 2024-05-09T02:27:36.556 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T02:27:36.556 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-05-09T02:27:36.556 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-05-09T02:27:36.556 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-05-09T02:27:36.557 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 1480628, 2024-05-09T02:27:36.557 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 18581, 2024-05-09T02:27:36.557 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-05-09T02:27:36.557 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-05-09T02:27:36.557 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-05-09T02:27:36.557 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-05-09T02:27:36.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.557 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter": { 2024-05-09T02:27:36.557 INFO:teuthology.orchestra.run.smithi059.stdout: "op_active": 0, 2024-05-09T02:27:36.557 INFO:teuthology.orchestra.run.smithi059.stdout: "op_laggy": 0, 2024-05-09T02:27:36.557 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send": 3591, 2024-05-09T02:27:36.557 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send_bytes": 21, 2024-05-09T02:27:36.558 INFO:teuthology.orchestra.run.smithi059.stdout: "op_resend": 3, 2024-05-09T02:27:36.558 INFO:teuthology.orchestra.run.smithi059.stdout: "op_reply": 3591, 2024-05-09T02:27:36.558 INFO:teuthology.orchestra.run.smithi059.stdout: "op_latency": { 2024-05-09T02:27:36.558 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3591, 2024-05-09T02:27:36.558 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 7.887904466, 2024-05-09T02:27:36.558 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.002196576 2024-05-09T02:27:36.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.558 INFO:teuthology.orchestra.run.smithi059.stdout: "op_inflight": 0, 2024-05-09T02:27:36.558 INFO:teuthology.orchestra.run.smithi059.stdout: "oplen_avg": { 2024-05-09T02:27:36.558 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3591, 2024-05-09T02:27:36.558 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 3592 2024-05-09T02:27:36.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.559 INFO:teuthology.orchestra.run.smithi059.stdout: "op": 3591, 2024-05-09T02:27:36.559 INFO:teuthology.orchestra.run.smithi059.stdout: "op_r": 3591, 2024-05-09T02:27:36.559 INFO:teuthology.orchestra.run.smithi059.stdout: "op_w": 0, 2024-05-09T02:27:36.559 INFO:teuthology.orchestra.run.smithi059.stdout: "op_rmw": 0, 2024-05-09T02:27:36.559 INFO:teuthology.orchestra.run.smithi059.stdout: "op_pg": 0, 2024-05-09T02:27:36.559 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_stat": 2216, 2024-05-09T02:27:36.559 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_create": 0, 2024-05-09T02:27:36.559 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_read": 1373, 2024-05-09T02:27:36.559 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_write": 0, 2024-05-09T02:27:36.559 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writefull": 0, 2024-05-09T02:27:36.559 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writesame": 0, 2024-05-09T02:27:36.559 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_append": 0, 2024-05-09T02:27:36.559 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_zero": 0, 2024-05-09T02:27:36.560 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_truncate": 0, 2024-05-09T02:27:36.560 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_delete": 0, 2024-05-09T02:27:36.560 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_mapext": 0, 2024-05-09T02:27:36.560 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_sparse_read": 0, 2024-05-09T02:27:36.560 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_clonerange": 0, 2024-05-09T02:27:36.560 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_getxattr": 1, 2024-05-09T02:27:36.560 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_setxattr": 0, 2024-05-09T02:27:36.560 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_cmpxattr": 0, 2024-05-09T02:27:36.560 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_rmxattr": 0, 2024-05-09T02:27:36.560 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_resetxattrs": 0, 2024-05-09T02:27:36.560 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_call": 0, 2024-05-09T02:27:36.560 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_watch": 0, 2024-05-09T02:27:36.560 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_notify": 0, 2024-05-09T02:27:36.561 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T02:27:36.561 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls": 0, 2024-05-09T02:27:36.561 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls_filter": 0, 2024-05-09T02:27:36.561 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_other": 0, 2024-05-09T02:27:36.561 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_active": 0, 2024-05-09T02:27:36.561 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_send": 0, 2024-05-09T02:27:36.561 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_resend": 0, 2024-05-09T02:27:36.561 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_ping": 0, 2024-05-09T02:27:36.561 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_active": 0, 2024-05-09T02:27:36.561 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_send": 0, 2024-05-09T02:27:36.562 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_resend": 0, 2024-05-09T02:27:36.562 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_active": 0, 2024-05-09T02:27:36.562 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_send": 0, 2024-05-09T02:27:36.562 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_resend": 0, 2024-05-09T02:27:36.562 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_active": 0, 2024-05-09T02:27:36.562 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_send": 0, 2024-05-09T02:27:36.562 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_resend": 0, 2024-05-09T02:27:36.562 INFO:teuthology.orchestra.run.smithi059.stdout: "command_active": 0, 2024-05-09T02:27:36.562 INFO:teuthology.orchestra.run.smithi059.stdout: "command_send": 0, 2024-05-09T02:27:36.563 INFO:teuthology.orchestra.run.smithi059.stdout: "command_resend": 0, 2024-05-09T02:27:36.563 INFO:teuthology.orchestra.run.smithi059.stdout: "map_epoch": 102, 2024-05-09T02:27:36.563 INFO:teuthology.orchestra.run.smithi059.stdout: "map_full": 0, 2024-05-09T02:27:36.563 INFO:teuthology.orchestra.run.smithi059.stdout: "map_inc": 23, 2024-05-09T02:27:36.563 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_sessions": 5, 2024-05-09T02:27:36.563 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_open": 6, 2024-05-09T02:27:36.563 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_close": 1, 2024-05-09T02:27:36.563 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_laggy": 0, 2024-05-09T02:27:36.563 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_wr": 0, 2024-05-09T02:27:36.563 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_rd": 2, 2024-05-09T02:27:36.563 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_del": 0 2024-05-09T02:27:36.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.564 INFO:teuthology.orchestra.run.smithi059.stdout: "oft": { 2024-05-09T02:27:36.564 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_objs": 0, 2024-05-09T02:27:36.564 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_kv_pairs": 0, 2024-05-09T02:27:36.564 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_updates": 0, 2024-05-09T02:27:36.564 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_removes": 0 2024-05-09T02:27:36.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.564 INFO:teuthology.orchestra.run.smithi059.stdout: "osd-slow-ops": { 2024-05-09T02:27:36.564 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_ops_count": 0 2024-05-09T02:27:36.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.564 INFO:teuthology.orchestra.run.smithi059.stdout: "purge_queue": { 2024-05-09T02:27:36.564 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops": 0, 2024-05-09T02:27:36.564 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T02:27:36.564 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing": 0, 2024-05-09T02:27:36.565 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_high_water": 0, 2024-05-09T02:27:36.565 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed_ops": 0, 2024-05-09T02:27:36.565 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed": 0, 2024-05-09T02:27:36.565 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_item_in_journal": 0 2024-05-09T02:27:36.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.565 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T02:27:36.565 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:36.565 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T02:27:36.565 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:36.565 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 3995, 2024-05-09T02:27:36.565 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 4682050, 2024-05-09T02:27:36.565 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:36.565 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 3995, 2024-05-09T02:27:36.566 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T02:27:36.566 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T02:27:36.566 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 3995, 2024-05-09T02:27:36.566 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 4682050, 2024-05-09T02:27:36.566 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:36.566 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.566 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.566 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.566 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:36.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.566 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_bytes": { 2024-05-09T02:27:36.566 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:36.567 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-09T02:27:36.567 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:36.567 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T02:27:36.567 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T02:27:36.567 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:36.567 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:36.567 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 3591, 2024-05-09T02:27:36.567 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 2043950, 2024-05-09T02:27:36.567 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 3591, 2024-05-09T02:27:36.567 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 2043950, 2024-05-09T02:27:36.567 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:36.567 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.568 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.568 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.568 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:36.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.568 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_ops": { 2024-05-09T02:27:36.568 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:36.568 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 1024, 2024-05-09T02:27:36.568 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:36.568 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T02:27:36.569 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T02:27:36.569 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:36.569 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:36.569 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 3591, 2024-05-09T02:27:36.569 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 3591, 2024-05-09T02:27:36.569 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 3591, 2024-05-09T02:27:36.569 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 3591, 2024-05-09T02:27:36.569 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:36.569 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.569 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.569 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.569 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:36.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.570 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle": { 2024-05-09T02:27:36.570 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:36.570 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T02:27:36.570 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:36.570 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T02:27:36.570 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T02:27:36.570 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:36.570 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:36.570 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T02:27:36.570 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T02:27:36.570 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-05-09T02:27:36.570 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-05-09T02:27:36.570 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:36.571 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.571 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.571 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:36.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-09T02:27:36.571 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle-0x55bc839ce8f0": { 2024-05-09T02:27:36.571 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-09T02:27:36.571 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-09T02:27:36.571 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-09T02:27:36.571 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-09T02:27:36.571 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-09T02:27:36.571 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:36.572 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:36.572 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-09T02:27:36.572 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-09T02:27:36.572 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-05-09T02:27:36.572 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-05-09T02:27:36.572 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-09T02:27:36.572 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-09T02:27:36.572 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-09T02:27:36.572 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-09T02:27:36.572 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:36.572 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-09T02:27:36.572 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-09T02:27:36.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:36 smithi059 ceph-mon[34781]: pgmap v512: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-09T02:27:36.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:36 smithi019 ceph-mon[30151]: pgmap v512: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-09T02:27:36.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:36 smithi196 ceph-mon[34718]: pgmap v512: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-09T02:27:37.292 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-09T02:27:37.292 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-09T02:27:37.292 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-05-09T02:27:37.292 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-05-09T02:27:37.292 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T02:27:37.292 DEBUG:tasks.check_counter:Getting stats from c 2024-05-09T02:27:37.293 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-09T02:27:37.754 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:27:38.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:38 smithi019 ceph-mon[30151]: pgmap v513: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-05-09T02:27:38.962 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:38 smithi196 ceph-mon[34718]: pgmap v513: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-05-09T02:27:38.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:38 smithi059 ceph-mon[34781]: pgmap v513: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-05-09T02:27:39.754 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-05-09T02:27:39.754 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T02:27:39.754 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 2745, 2024-05-09T02:27:39.754 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 2865, 2024-05-09T02:27:39.754 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 982250, 2024-05-09T02:27:39.754 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 3453606, 2024-05-09T02:27:39.754 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 20, 2024-05-09T02:27:39.754 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-05-09T02:27:39.755 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.204923428, 2024-05-09T02:27:39.755 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.069071144, 2024-05-09T02:27:39.755 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.085456330, 2024-05-09T02:27:39.755 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.041804333, 2024-05-09T02:27:39.755 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:39.755 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2863, 2024-05-09T02:27:39.755 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.196363341, 2024-05-09T02:27:39.755 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000068586 2024-05-09T02:27:39.755 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.755 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:39.755 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2589, 2024-05-09T02:27:39.755 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000593673, 2024-05-09T02:27:39.756 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000229 2024-05-09T02:27:39.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.756 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1744, 2024-05-09T02:27:39.756 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 443344 2024-05-09T02:27:39.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.756 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T02:27:39.756 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 3282, 2024-05-09T02:27:39.756 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 3610, 2024-05-09T02:27:39.757 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 3034759, 2024-05-09T02:27:39.757 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 1791739, 2024-05-09T02:27:39.757 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 9, 2024-05-09T02:27:39.757 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 7, 2024-05-09T02:27:39.757 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.280265453, 2024-05-09T02:27:39.757 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.103854439, 2024-05-09T02:27:39.757 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.166319072, 2024-05-09T02:27:39.757 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.032802019, 2024-05-09T02:27:39.757 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:39.757 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3609, 2024-05-09T02:27:39.757 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.212894777, 2024-05-09T02:27:39.757 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000058989 2024-05-09T02:27:39.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.758 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:39.758 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3623, 2024-05-09T02:27:39.758 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000771938, 2024-05-09T02:27:39.758 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000213 2024-05-09T02:27:39.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.758 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 2038384, 2024-05-09T02:27:39.758 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 225408 2024-05-09T02:27:39.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.758 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T02:27:39.758 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 4373, 2024-05-09T02:27:39.759 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 3766, 2024-05-09T02:27:39.759 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 2382588, 2024-05-09T02:27:39.759 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 2314651, 2024-05-09T02:27:39.759 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 40, 2024-05-09T02:27:39.759 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 8, 2024-05-09T02:27:39.759 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.341807696, 2024-05-09T02:27:39.759 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.137741152, 2024-05-09T02:27:39.759 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.202379701, 2024-05-09T02:27:39.759 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.025381147, 2024-05-09T02:27:39.759 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:39.759 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3764, 2024-05-09T02:27:39.759 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.198016498, 2024-05-09T02:27:39.760 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000052607 2024-05-09T02:27:39.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.760 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:39.760 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 6444, 2024-05-09T02:27:39.760 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001533757, 2024-05-09T02:27:39.760 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000238 2024-05-09T02:27:39.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.760 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 8720, 2024-05-09T02:27:39.760 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 176208 2024-05-09T02:27:39.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.761 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-05-09T02:27:39.761 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-05-09T02:27:39.761 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-05-09T02:27:39.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.761 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-05-09T02:27:39.761 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T02:27:39.761 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T02:27:39.761 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2622, 2024-05-09T02:27:39.761 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.662188530, 2024-05-09T02:27:39.761 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000252550 2024-05-09T02:27:39.761 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:39.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.762 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-05-09T02:27:39.762 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T02:27:39.762 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T02:27:39.762 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 16, 2024-05-09T02:27:39.762 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000456732, 2024-05-09T02:27:39.762 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000028545 2024-05-09T02:27:39.762 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:39.762 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.762 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-05-09T02:27:39.763 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 410, 2024-05-09T02:27:39.763 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 349, 2024-05-09T02:27:39.763 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-05-09T02:27:39.763 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 349, 2024-05-09T02:27:39.763 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 13.737805209, 2024-05-09T02:27:39.763 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.039363338 2024-05-09T02:27:39.763 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.763 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-05-09T02:27:39.763 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 61, 2024-05-09T02:27:39.763 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-05-09T02:27:39.764 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-05-09T02:27:39.764 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 27, 2024-05-09T02:27:39.764 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 2, 2024-05-09T02:27:39.764 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-05-09T02:27:39.764 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 541, 2024-05-09T02:27:39.764 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 98, 2024-05-09T02:27:39.764 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 443, 2024-05-09T02:27:39.764 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-05-09T02:27:39.764 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 399, 2024-05-09T02:27:39.764 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 68, 2024-05-09T02:27:39.765 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 173, 2024-05-09T02:27:39.765 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 174, 2024-05-09T02:27:39.765 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 32, 2024-05-09T02:27:39.765 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 1237, 2024-05-09T02:27:39.765 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 1011, 2024-05-09T02:27:39.765 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 56, 2024-05-09T02:27:39.765 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 50, 2024-05-09T02:27:39.765 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-05-09T02:27:39.765 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-05-09T02:27:39.765 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 25, 2024-05-09T02:27:39.765 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 173, 2024-05-09T02:27:39.766 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-05-09T02:27:39.766 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-05-09T02:27:39.766 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-05-09T02:27:39.766 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 5, 2024-05-09T02:27:39.766 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 198, 2024-05-09T02:27:39.766 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-05-09T02:27:39.766 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 2414, 2024-05-09T02:27:39.766 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 1, 2024-05-09T02:27:39.766 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 998, 2024-05-09T02:27:39.766 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 1205583148, 2024-05-09T02:27:39.766 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 6, 2024-05-09T02:27:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 2414, 2024-05-09T02:27:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-05-09T02:27:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T02:27:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-05-09T02:27:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-05-09T02:27:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T02:27:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T02:27:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 2414, 2024-05-09T02:27:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 31, 2024-05-09T02:27:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 217, 2024-05-09T02:27:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 272, 2024-05-09T02:27:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T02:27:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T02:27:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 1, 2024-05-09T02:27:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 477, 2024-05-09T02:27:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 195, 2024-05-09T02:27:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 24, 2024-05-09T02:27:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 70 2024-05-09T02:27:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-05-09T02:27:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-05-09T02:27:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-05-09T02:27:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-05-09T02:27:39.769 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 56, 2024-05-09T02:27:39.769 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 39, 2024-05-09T02:27:39.769 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 3, 2024-05-09T02:27:39.769 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-05-09T02:27:39.769 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-05-09T02:27:39.769 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 24, 2024-05-09T02:27:39.769 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 2, 2024-05-09T02:27:39.769 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 16, 2024-05-09T02:27:39.769 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 17, 2024-05-09T02:27:39.769 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-05-09T02:27:39.770 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-05-09T02:27:39.770 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-05-09T02:27:39.770 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-05-09T02:27:39.770 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-05-09T02:27:39.770 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-05-09T02:27:39.770 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-05-09T02:27:39.770 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T02:27:39.770 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-05-09T02:27:39.770 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-05-09T02:27:39.770 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 2, 2024-05-09T02:27:39.771 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-05-09T02:27:39.771 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-05-09T02:27:39.771 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.774 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-05-09T02:27:39.774 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-05-09T02:27:39.774 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 1591, 2024-05-09T02:27:39.774 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-05-09T02:27:39.774 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-05-09T02:27:39.774 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 1591, 2024-05-09T02:27:39.774 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-05-09T02:27:39.774 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-05-09T02:27:39.774 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 3, 2024-05-09T02:27:39.774 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-05-09T02:27:39.774 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-05-09T02:27:39.774 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 3, 2024-05-09T02:27:39.775 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-05-09T02:27:39.775 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-05-09T02:27:39.775 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-05-09T02:27:39.775 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-05-09T02:27:39.775 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 6238823, 2024-05-09T02:27:39.775 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-05-09T02:27:39.775 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-05-09T02:27:39.775 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 419, 2024-05-09T02:27:39.775 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 3.329020164, 2024-05-09T02:27:39.775 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.007945155 2024-05-09T02:27:39.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.775 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-05-09T02:27:39.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.775 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-05-09T02:27:39.775 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 473, 2024-05-09T02:27:39.776 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 488, 2024-05-09T02:27:39.776 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 15, 2024-05-09T02:27:39.776 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 59, 2024-05-09T02:27:39.776 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 62, 2024-05-09T02:27:39.776 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 3, 2024-05-09T02:27:39.776 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 542, 2024-05-09T02:27:39.776 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 620, 2024-05-09T02:27:39.776 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 78, 2024-05-09T02:27:39.776 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 174, 2024-05-09T02:27:39.776 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 389, 2024-05-09T02:27:39.776 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 215, 2024-05-09T02:27:39.776 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 129084, 2024-05-09T02:27:39.776 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223632 2024-05-09T02:27:39.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.777 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-05-09T02:27:39.777 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 879, 2024-05-09T02:27:39.777 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 460, 2024-05-09T02:27:39.777 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 410, 2024-05-09T02:27:39.777 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 68, 2024-05-09T02:27:39.777 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 668, 2024-05-09T02:27:39.777 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-05-09T02:27:39.777 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-05-09T02:27:39.777 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.003930461, 2024-05-09T02:27:39.777 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003930461 2024-05-09T02:27:39.777 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.777 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-05-09T02:27:39.777 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 27, 2024-05-09T02:27:39.777 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.011596563, 2024-05-09T02:27:39.777 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000429502 2024-05-09T02:27:39.778 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.778 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-05-09T02:27:39.778 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.778 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.778 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.778 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.778 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-05-09T02:27:39.778 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 17, 2024-05-09T02:27:39.778 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.224166175, 2024-05-09T02:27:39.778 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.013186245 2024-05-09T02:27:39.778 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.778 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-05-09T02:27:39.779 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 194, 2024-05-09T02:27:39.779 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 4.239326216, 2024-05-09T02:27:39.779 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.021852196 2024-05-09T02:27:39.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.779 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-05-09T02:27:39.779 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.779 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.779 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.779 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-05-09T02:27:39.779 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-05-09T02:27:39.779 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000090586, 2024-05-09T02:27:39.779 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000090586 2024-05-09T02:27:39.779 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.780 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-05-09T02:27:39.780 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.780 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.780 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.780 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-05-09T02:27:39.780 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.780 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.780 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.780 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-05-09T02:27:39.780 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.780 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.780 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.780 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.781 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-05-09T02:27:39.781 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.781 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.782 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.782 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.782 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-05-09T02:27:39.782 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.782 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.782 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.782 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.782 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-05-09T02:27:39.782 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.782 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.782 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.782 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.783 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-05-09T02:27:39.783 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.783 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.783 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.783 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.783 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-05-09T02:27:39.783 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 32, 2024-05-09T02:27:39.783 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.019065656, 2024-05-09T02:27:39.783 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000595801 2024-05-09T02:27:39.783 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.783 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-05-09T02:27:39.783 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 23, 2024-05-09T02:27:39.783 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.014420832, 2024-05-09T02:27:39.784 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000626992 2024-05-09T02:27:39.784 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.784 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-05-09T02:27:39.784 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 30, 2024-05-09T02:27:39.784 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 9.137453525, 2024-05-09T02:27:39.784 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.304581784 2024-05-09T02:27:39.784 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.784 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-05-09T02:27:39.784 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.784 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.784 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.784 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.784 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-05-09T02:27:39.784 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.785 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.785 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.785 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.785 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-05-09T02:27:39.785 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.785 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.785 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.785 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.785 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-05-09T02:27:39.785 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.785 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.785 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.786 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-05-09T02:27:39.786 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.786 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.786 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.786 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-05-09T02:27:39.786 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.786 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.786 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.786 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-05-09T02:27:39.786 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.786 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.787 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.787 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-05-09T02:27:39.787 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.787 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.787 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.787 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-05-09T02:27:39.787 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.787 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.787 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.787 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-05-09T02:27:39.788 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.788 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.788 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.788 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.788 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-05-09T02:27:39.788 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.788 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.788 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.788 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.788 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-05-09T02:27:39.788 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 24, 2024-05-09T02:27:39.788 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.087755195, 2024-05-09T02:27:39.788 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003656466 2024-05-09T02:27:39.789 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.789 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-05-09T02:27:39.789 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-05-09T02:27:39.789 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-05-09T02:27:39.789 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.789 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.789 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.789 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:39.789 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.789 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-05-09T02:27:39.789 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 2, 2024-05-09T02:27:39.789 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 4, 2024-05-09T02:27:39.789 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 2, 2024-05-09T02:27:39.789 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 2, 2024-05-09T02:27:39.789 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-05-09T02:27:39.790 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 60, 2024-05-09T02:27:39.790 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 30, 2024-05-09T02:27:39.790 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 950, 2024-05-09T02:27:39.790 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-05-09T02:27:39.790 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.790 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-05-09T02:27:39.790 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-05-09T02:27:39.790 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-05-09T02:27:39.790 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T02:27:39.790 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-05-09T02:27:39.790 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T02:27:39.790 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-05-09T02:27:39.790 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T02:27:39.791 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T02:27:39.791 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T02:27:39.791 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T02:27:39.791 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T02:27:39.791 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-05-09T02:27:39.791 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T02:27:39.791 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T02:27:39.791 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-05-09T02:27:39.791 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-05-09T02:27:39.791 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-05-09T02:27:39.791 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-05-09T02:27:39.791 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T02:27:39.791 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T02:27:39.791 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T02:27:39.791 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T02:27:39.792 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T02:27:39.792 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-05-09T02:27:39.792 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-05-09T02:27:39.792 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-05-09T02:27:39.792 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T02:27:39.792 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T02:27:39.792 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-05-09T02:27:39.792 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-05-09T02:27:39.792 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-05-09T02:27:39.792 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-05-09T02:27:39.792 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T02:27:39.792 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-05-09T02:27:39.792 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T02:27:39.792 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-05-09T02:27:39.793 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 579493, 2024-05-09T02:27:39.793 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 427, 2024-05-09T02:27:39.793 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-05-09T02:27:39.793 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-05-09T02:27:39.793 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-05-09T02:27:39.793 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-05-09T02:27:39.793 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-05-09T02:27:39.793 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-05-09T02:27:39.793 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-05-09T02:27:39.793 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-05-09T02:27:39.793 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 8224, 2024-05-09T02:27:39.793 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 171, 2024-05-09T02:27:39.793 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T02:27:39.793 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-05-09T02:27:39.794 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-05-09T02:27:39.794 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-05-09T02:27:39.794 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 1710824, 2024-05-09T02:27:39.794 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 20805, 2024-05-09T02:27:39.794 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-05-09T02:27:39.794 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-05-09T02:27:39.794 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-05-09T02:27:39.794 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-05-09T02:27:39.794 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.794 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-05-09T02:27:39.794 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-05-09T02:27:39.794 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-05-09T02:27:39.794 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 3030, 2024-05-09T02:27:39.794 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 2223897, 2024-05-09T02:27:39.795 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 33, 2024-05-09T02:27:39.795 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 3030, 2024-05-09T02:27:39.795 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-05-09T02:27:39.795 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3030, 2024-05-09T02:27:39.795 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 7.373851108, 2024-05-09T02:27:39.795 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002433614 2024-05-09T02:27:39.795 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.795 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-05-09T02:27:39.795 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-05-09T02:27:39.795 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3030, 2024-05-09T02:27:39.795 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 3151 2024-05-09T02:27:39.795 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.795 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 3030, 2024-05-09T02:27:39.795 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 2415, 2024-05-09T02:27:39.795 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 615, 2024-05-09T02:27:39.796 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-05-09T02:27:39.796 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-05-09T02:27:39.796 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-05-09T02:27:39.796 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-05-09T02:27:39.796 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 1, 2024-05-09T02:27:39.796 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 420, 2024-05-09T02:27:39.796 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 67, 2024-05-09T02:27:39.796 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-05-09T02:27:39.796 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-05-09T02:27:39.796 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-05-09T02:27:39.796 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-05-09T02:27:39.796 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 16, 2024-05-09T02:27:39.796 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-05-09T02:27:39.796 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-05-09T02:27:39.796 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 2414, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-05-09T02:27:39.797 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-05-09T02:27:39.798 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-05-09T02:27:39.798 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-05-09T02:27:39.798 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-05-09T02:27:39.798 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-05-09T02:27:39.798 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-05-09T02:27:39.798 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-05-09T02:27:39.798 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-05-09T02:27:39.798 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-05-09T02:27:39.798 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-05-09T02:27:39.798 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-05-09T02:27:39.798 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-05-09T02:27:39.798 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-05-09T02:27:39.798 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 102, 2024-05-09T02:27:39.798 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-05-09T02:27:39.799 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 23, 2024-05-09T02:27:39.799 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 12, 2024-05-09T02:27:39.799 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 12, 2024-05-09T02:27:39.799 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-05-09T02:27:39.799 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-05-09T02:27:39.799 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 163, 2024-05-09T02:27:39.799 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-05-09T02:27:39.799 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 40 2024-05-09T02:27:39.799 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.799 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-05-09T02:27:39.799 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-05-09T02:27:39.799 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-05-09T02:27:39.799 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-05-09T02:27:39.799 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-05-09T02:27:39.799 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-05-09T02:27:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 1 2024-05-09T02:27:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-05-09T02:27:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-05-09T02:27:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 2, 2024-05-09T02:27:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-05-09T02:27:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 2, 2024-05-09T02:27:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 2, 2024-05-09T02:27:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 2, 2024-05-09T02:27:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-05-09T02:27:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T02:27:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T02:27:39.801 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:39.801 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 10396, 2024-05-09T02:27:39.801 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 5436557, 2024-05-09T02:27:39.801 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:39.801 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 10396, 2024-05-09T02:27:39.801 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T02:27:39.801 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T02:27:39.801 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 10396, 2024-05-09T02:27:39.801 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 5436557, 2024-05-09T02:27:39.801 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:39.801 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.801 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.801 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.801 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:39.801 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.802 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-05-09T02:27:39.802 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:39.802 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T02:27:39.802 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:39.802 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T02:27:39.802 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T02:27:39.802 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:39.802 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:39.802 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 3030, 2024-05-09T02:27:39.802 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 2224163, 2024-05-09T02:27:39.802 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3030, 2024-05-09T02:27:39.802 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2224163, 2024-05-09T02:27:39.802 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:39.802 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.803 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.803 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.803 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:39.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.803 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-05-09T02:27:39.803 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:39.803 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-05-09T02:27:39.803 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:39.803 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T02:27:39.803 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T02:27:39.803 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:39.803 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:39.803 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 3030, 2024-05-09T02:27:39.803 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 3030, 2024-05-09T02:27:39.804 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3030, 2024-05-09T02:27:39.804 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3030, 2024-05-09T02:27:39.804 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:39.804 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.804 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.804 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.804 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:39.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.804 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-05-09T02:27:39.804 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:39.804 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T02:27:39.804 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:39.804 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 2, 2024-05-09T02:27:39.804 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 266, 2024-05-09T02:27:39.805 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:39.805 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 2, 2024-05-09T02:27:39.805 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T02:27:39.805 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T02:27:39.805 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 1, 2024-05-09T02:27:39.805 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 266, 2024-05-09T02:27:39.805 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:39.805 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.805 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.805 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.805 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:39.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:39.805 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55d42a3dc410": { 2024-05-09T02:27:39.805 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 574, 2024-05-09T02:27:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T02:27:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 1591, 2024-05-09T02:27:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 2044519, 2024-05-09T02:27:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 1591, 2024-05-09T02:27:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T02:27:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T02:27:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 419, 2024-05-09T02:27:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2043945, 2024-05-09T02:27:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:39.807 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:39.807 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-05-09T02:27:40.596 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-09T02:27:40.597 INFO:tasks.check_counter:Daemon mds.c mds.imported=5 2024-05-09T02:27:40.597 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-05-09T02:27:40.597 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-09T02:27:40.597 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T02:27:40.597 DEBUG:tasks.check_counter:Getting stats from f 2024-05-09T02:27:40.597 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-09T02:27:40.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:40 smithi019 ceph-mon[30151]: pgmap v514: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-05-09T02:27:40.846 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:27:40.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:40 smithi059 ceph-mon[34781]: pgmap v514: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-05-09T02:27:40.967 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:40 smithi196 ceph-mon[34718]: pgmap v514: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-05-09T02:27:42.646 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:42 smithi196 ceph-mon[34718]: pgmap v515: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-05-09T02:27:42.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:42 smithi019 ceph-mon[30151]: pgmap v515: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-05-09T02:27:42.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:42 smithi059 ceph-mon[34781]: pgmap v515: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-05-09T02:27:43.010 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-05-09T02:27:43.010 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T02:27:43.010 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 6061, 2024-05-09T02:27:43.010 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 6749, 2024-05-09T02:27:43.010 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 3954838, 2024-05-09T02:27:43.010 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 6501909, 2024-05-09T02:27:43.010 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 15, 2024-05-09T02:27:43.010 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-05-09T02:27:43.010 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.440237671, 2024-05-09T02:27:43.010 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.157168385, 2024-05-09T02:27:43.011 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.266143021, 2024-05-09T02:27:43.011 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.059406072, 2024-05-09T02:27:43.011 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:43.011 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 6748, 2024-05-09T02:27:43.011 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.298095075, 2024-05-09T02:27:43.011 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000044175 2024-05-09T02:27:43.011 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.011 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:43.011 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 8334, 2024-05-09T02:27:43.011 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001993109, 2024-05-09T02:27:43.011 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000239 2024-05-09T02:27:43.011 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.011 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 2033888, 2024-05-09T02:27:43.012 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 226896 2024-05-09T02:27:43.012 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.012 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T02:27:43.012 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 1829, 2024-05-09T02:27:43.012 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 2155, 2024-05-09T02:27:43.012 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 668867, 2024-05-09T02:27:43.012 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 1473021, 2024-05-09T02:27:43.012 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 12, 2024-05-09T02:27:43.012 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 6, 2024-05-09T02:27:43.012 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.156332808, 2024-05-09T02:27:43.012 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.061201377, 2024-05-09T02:27:43.012 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.058064743, 2024-05-09T02:27:43.013 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.028487667, 2024-05-09T02:27:43.013 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:43.013 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2152, 2024-05-09T02:27:43.013 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.148370686, 2024-05-09T02:27:43.013 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000068945 2024-05-09T02:27:43.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.013 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:43.013 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1541, 2024-05-09T02:27:43.013 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000350150, 2024-05-09T02:27:43.013 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000227 2024-05-09T02:27:43.013 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.013 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T02:27:43.014 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 598304 2024-05-09T02:27:43.014 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.014 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T02:27:43.014 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 5426, 2024-05-09T02:27:43.014 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 4857, 2024-05-09T02:27:43.014 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 2745788, 2024-05-09T02:27:43.014 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 2546299, 2024-05-09T02:27:43.014 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 39, 2024-05-09T02:27:43.014 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 6, 2024-05-09T02:27:43.014 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.401969617, 2024-05-09T02:27:43.014 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.161645483, 2024-05-09T02:27:43.014 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.235773214, 2024-05-09T02:27:43.015 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.036643439, 2024-05-09T02:27:43.015 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:43.015 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 4856, 2024-05-09T02:27:43.015 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.256538798, 2024-05-09T02:27:43.015 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000052829 2024-05-09T02:27:43.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.015 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:43.015 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 7814, 2024-05-09T02:27:43.015 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001673677, 2024-05-09T02:27:43.015 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000214 2024-05-09T02:27:43.015 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.015 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T02:27:43.015 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T02:27:43.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.016 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-05-09T02:27:43.016 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-05-09T02:27:43.016 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-05-09T02:27:43.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.016 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-05-09T02:27:43.016 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T02:27:43.016 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T02:27:43.016 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3276, 2024-05-09T02:27:43.016 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.929151269, 2024-05-09T02:27:43.016 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000283623 2024-05-09T02:27:43.016 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:43.016 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.017 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-05-09T02:27:43.017 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T02:27:43.017 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T02:27:43.017 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 17, 2024-05-09T02:27:43.017 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000418425, 2024-05-09T02:27:43.017 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000024613 2024-05-09T02:27:43.017 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:43.017 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.017 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-05-09T02:27:43.017 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 674, 2024-05-09T02:27:43.017 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 611, 2024-05-09T02:27:43.017 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-05-09T02:27:43.017 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 611, 2024-05-09T02:27:43.017 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 7.212016155, 2024-05-09T02:27:43.018 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.011803627 2024-05-09T02:27:43.018 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.018 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-05-09T02:27:43.018 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 63, 2024-05-09T02:27:43.018 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-05-09T02:27:43.018 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-05-09T02:27:43.018 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 35, 2024-05-09T02:27:43.018 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 3, 2024-05-09T02:27:43.018 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-05-09T02:27:43.019 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 791, 2024-05-09T02:27:43.019 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 176, 2024-05-09T02:27:43.019 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 615, 2024-05-09T02:27:43.019 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-05-09T02:27:43.019 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 539, 2024-05-09T02:27:43.019 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 98, 2024-05-09T02:27:43.019 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 177, 2024-05-09T02:27:43.019 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 177, 2024-05-09T02:27:43.019 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 40, 2024-05-09T02:27:43.019 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 2320, 2024-05-09T02:27:43.019 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 1932, 2024-05-09T02:27:43.020 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 61, 2024-05-09T02:27:43.020 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 66, 2024-05-09T02:27:43.020 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-05-09T02:27:43.020 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-05-09T02:27:43.020 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 54, 2024-05-09T02:27:43.020 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 235, 2024-05-09T02:27:43.020 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-05-09T02:27:43.020 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-05-09T02:27:43.020 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-05-09T02:27:43.020 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 7, 2024-05-09T02:27:43.020 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 242, 2024-05-09T02:27:43.020 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-05-09T02:27:43.020 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 2543, 2024-05-09T02:27:43.020 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-05-09T02:27:43.021 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 998, 2024-05-09T02:27:43.021 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 1205583148, 2024-05-09T02:27:43.021 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 6, 2024-05-09T02:27:43.021 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 2543, 2024-05-09T02:27:43.021 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-05-09T02:27:43.021 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T02:27:43.021 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-05-09T02:27:43.021 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-05-09T02:27:43.021 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T02:27:43.021 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T02:27:43.021 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 2543, 2024-05-09T02:27:43.021 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 55, 2024-05-09T02:27:43.021 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 381, 2024-05-09T02:27:43.022 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 502, 2024-05-09T02:27:43.022 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 2, 2024-05-09T02:27:43.022 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T02:27:43.022 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 2, 2024-05-09T02:27:43.022 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 887, 2024-05-09T02:27:43.022 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 422, 2024-05-09T02:27:43.022 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 51, 2024-05-09T02:27:43.022 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 151 2024-05-09T02:27:43.022 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.022 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-05-09T02:27:43.022 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-05-09T02:27:43.022 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-05-09T02:27:43.022 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-05-09T02:27:43.022 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 71, 2024-05-09T02:27:43.023 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 74, 2024-05-09T02:27:43.023 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 11, 2024-05-09T02:27:43.023 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-05-09T02:27:43.023 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-05-09T02:27:43.023 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 49, 2024-05-09T02:27:43.023 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 3, 2024-05-09T02:27:43.023 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 41, 2024-05-09T02:27:43.023 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 23, 2024-05-09T02:27:43.023 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-05-09T02:27:43.023 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-05-09T02:27:43.023 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-05-09T02:27:43.023 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-05-09T02:27:43.023 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-05-09T02:27:43.023 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-05-09T02:27:43.024 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-05-09T02:27:43.024 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T02:27:43.024 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 1, 2024-05-09T02:27:43.024 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-05-09T02:27:43.024 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 3, 2024-05-09T02:27:43.024 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-05-09T02:27:43.024 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-05-09T02:27:43.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.024 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-05-09T02:27:43.024 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-05-09T02:27:43.024 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 2666, 2024-05-09T02:27:43.024 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-05-09T02:27:43.024 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-05-09T02:27:43.024 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 2666, 2024-05-09T02:27:43.025 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-05-09T02:27:43.025 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-05-09T02:27:43.025 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 4, 2024-05-09T02:27:43.025 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-05-09T02:27:43.025 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-05-09T02:27:43.025 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 4, 2024-05-09T02:27:43.025 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-05-09T02:27:43.025 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-05-09T02:27:43.025 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-05-09T02:27:43.025 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-05-09T02:27:43.025 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 7709897, 2024-05-09T02:27:43.025 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-05-09T02:27:43.025 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-05-09T02:27:43.025 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 691, 2024-05-09T02:27:43.025 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 5.877212114, 2024-05-09T02:27:43.026 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.008505372 2024-05-09T02:27:43.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.026 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-05-09T02:27:43.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.026 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-05-09T02:27:43.026 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 643, 2024-05-09T02:27:43.026 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 660, 2024-05-09T02:27:43.026 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 17, 2024-05-09T02:27:43.026 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 68, 2024-05-09T02:27:43.026 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 72, 2024-05-09T02:27:43.026 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 4, 2024-05-09T02:27:43.026 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 795, 2024-05-09T02:27:43.026 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 904, 2024-05-09T02:27:43.027 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 109, 2024-05-09T02:27:43.027 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 177, 2024-05-09T02:27:43.027 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 453, 2024-05-09T02:27:43.027 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 276, 2024-05-09T02:27:43.027 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 131496, 2024-05-09T02:27:43.027 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223632 2024-05-09T02:27:43.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.027 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-05-09T02:27:43.027 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 1712, 2024-05-09T02:27:43.027 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 689, 2024-05-09T02:27:43.027 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 674, 2024-05-09T02:27:43.027 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 88, 2024-05-09T02:27:43.027 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 1215, 2024-05-09T02:27:43.027 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-05-09T02:27:43.028 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-05-09T02:27:43.028 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.004801902, 2024-05-09T02:27:43.028 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002400951 2024-05-09T02:27:43.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.028 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-05-09T02:27:43.028 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 39, 2024-05-09T02:27:43.028 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.027929111, 2024-05-09T02:27:43.028 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000716131 2024-05-09T02:27:43.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.028 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-05-09T02:27:43.028 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.028 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.028 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.029 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-05-09T02:27:43.029 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 45, 2024-05-09T02:27:43.029 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.528662136, 2024-05-09T02:27:43.029 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.011748047 2024-05-09T02:27:43.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.029 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-05-09T02:27:43.029 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 328, 2024-05-09T02:27:43.029 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 5.033830521, 2024-05-09T02:27:43.029 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.015347044 2024-05-09T02:27:43.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.030 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-05-09T02:27:43.030 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.030 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.030 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.030 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-05-09T02:27:43.030 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.030 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.030 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.030 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.031 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-05-09T02:27:43.031 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.031 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.031 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.031 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-05-09T02:27:43.031 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.031 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.031 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.031 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-05-09T02:27:43.031 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.031 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.031 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.031 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.032 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-05-09T02:27:43.032 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.032 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.032 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.033 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-05-09T02:27:43.033 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.033 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.033 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.033 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-05-09T02:27:43.033 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.033 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.033 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.033 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.033 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-05-09T02:27:43.033 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.033 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.034 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.034 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-05-09T02:27:43.034 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 40, 2024-05-09T02:27:43.034 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.065051371, 2024-05-09T02:27:43.034 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001626284 2024-05-09T02:27:43.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.034 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-05-09T02:27:43.034 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 36, 2024-05-09T02:27:43.034 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.018714677, 2024-05-09T02:27:43.034 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000519852 2024-05-09T02:27:43.034 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.034 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-05-09T02:27:43.035 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 70, 2024-05-09T02:27:43.035 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 1.280673845, 2024-05-09T02:27:43.035 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.018295340 2024-05-09T02:27:43.035 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.035 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-05-09T02:27:43.038 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.038 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.038 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.039 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-05-09T02:27:43.039 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.039 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.039 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.039 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-05-09T02:27:43.039 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.039 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.039 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.039 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.039 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-05-09T02:27:43.039 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.039 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.040 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.040 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-05-09T02:27:43.040 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.040 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.040 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.040 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-05-09T02:27:43.040 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-05-09T02:27:43.041 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.012437663, 2024-05-09T02:27:43.041 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.006218831 2024-05-09T02:27:43.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.041 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-05-09T02:27:43.041 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.041 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.041 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.041 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-05-09T02:27:43.041 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.041 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.041 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.041 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-05-09T02:27:43.042 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.042 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.042 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.042 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-05-09T02:27:43.042 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.042 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.042 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.042 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-05-09T02:27:43.042 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.042 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.042 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.042 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.043 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-05-09T02:27:43.043 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 49, 2024-05-09T02:27:43.043 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.239914929, 2024-05-09T02:27:43.043 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.004896223 2024-05-09T02:27:43.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.043 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-05-09T02:27:43.043 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-05-09T02:27:43.043 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-05-09T02:27:43.043 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.043 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.043 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.043 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:43.043 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.043 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-05-09T02:27:43.043 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-05-09T02:27:43.044 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 1, 2024-05-09T02:27:43.044 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 0, 2024-05-09T02:27:43.044 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-05-09T02:27:43.044 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-05-09T02:27:43.044 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 145, 2024-05-09T02:27:43.044 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 145, 2024-05-09T02:27:43.044 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 2407, 2024-05-09T02:27:43.044 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-05-09T02:27:43.044 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.044 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-05-09T02:27:43.044 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-05-09T02:27:43.044 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-05-09T02:27:43.044 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T02:27:43.044 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-05-09T02:27:43.044 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T02:27:43.045 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-05-09T02:27:43.045 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T02:27:43.045 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T02:27:43.045 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T02:27:43.045 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T02:27:43.045 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T02:27:43.045 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-05-09T02:27:43.045 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T02:27:43.045 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T02:27:43.045 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-05-09T02:27:43.045 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-05-09T02:27:43.045 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-05-09T02:27:43.045 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-05-09T02:27:43.045 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T02:27:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T02:27:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T02:27:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T02:27:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T02:27:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-05-09T02:27:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-05-09T02:27:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-05-09T02:27:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T02:27:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T02:27:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-05-09T02:27:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-05-09T02:27:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-05-09T02:27:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-05-09T02:27:43.046 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T02:27:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-05-09T02:27:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T02:27:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-05-09T02:27:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 702578, 2024-05-09T02:27:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 540, 2024-05-09T02:27:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-05-09T02:27:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-05-09T02:27:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-05-09T02:27:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-05-09T02:27:43.047 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-05-09T02:27:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-05-09T02:27:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-05-09T02:27:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-05-09T02:27:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 8224, 2024-05-09T02:27:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 171, 2024-05-09T02:27:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T02:27:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-05-09T02:27:43.048 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-05-09T02:27:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-05-09T02:27:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 2345680, 2024-05-09T02:27:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 39033, 2024-05-09T02:27:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-05-09T02:27:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-05-09T02:27:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-05-09T02:27:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-05-09T02:27:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-05-09T02:27:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-05-09T02:27:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-05-09T02:27:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 3464, 2024-05-09T02:27:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 3763666, 2024-05-09T02:27:43.049 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 33, 2024-05-09T02:27:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 3464, 2024-05-09T02:27:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-05-09T02:27:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3464, 2024-05-09T02:27:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 11.205763698, 2024-05-09T02:27:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003234920 2024-05-09T02:27:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-05-09T02:27:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-05-09T02:27:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3464, 2024-05-09T02:27:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 3625 2024-05-09T02:27:43.050 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 3464, 2024-05-09T02:27:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 2544, 2024-05-09T02:27:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 920, 2024-05-09T02:27:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-05-09T02:27:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-05-09T02:27:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-05-09T02:27:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-05-09T02:27:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 1, 2024-05-09T02:27:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 692, 2024-05-09T02:27:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 65, 2024-05-09T02:27:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-05-09T02:27:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-05-09T02:27:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-05-09T02:27:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-05-09T02:27:43.051 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 18, 2024-05-09T02:27:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-05-09T02:27:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-05-09T02:27:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-05-09T02:27:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 2543, 2024-05-09T02:27:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-05-09T02:27:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-05-09T02:27:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-05-09T02:27:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-05-09T02:27:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-05-09T02:27:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-05-09T02:27:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-05-09T02:27:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T02:27:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-05-09T02:27:43.052 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-05-09T02:27:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 2, 2024-05-09T02:27:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-05-09T02:27:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-05-09T02:27:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-05-09T02:27:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-05-09T02:27:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-05-09T02:27:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-05-09T02:27:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-05-09T02:27:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-05-09T02:27:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-05-09T02:27:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-05-09T02:27:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-05-09T02:27:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-05-09T02:27:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-05-09T02:27:43.053 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-05-09T02:27:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-05-09T02:27:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-05-09T02:27:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 102, 2024-05-09T02:27:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-05-09T02:27:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 23, 2024-05-09T02:27:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 12, 2024-05-09T02:27:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 12, 2024-05-09T02:27:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-05-09T02:27:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-05-09T02:27:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 220, 2024-05-09T02:27:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-05-09T02:27:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 54 2024-05-09T02:27:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.054 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-05-09T02:27:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-05-09T02:27:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-05-09T02:27:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-05-09T02:27:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-05-09T02:27:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-05-09T02:27:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 1 2024-05-09T02:27:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-05-09T02:27:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-05-09T02:27:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 3, 2024-05-09T02:27:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-05-09T02:27:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 3, 2024-05-09T02:27:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 3, 2024-05-09T02:27:43.055 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 3, 2024-05-09T02:27:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-05-09T02:27:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T02:27:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T02:27:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 13312, 2024-05-09T02:27:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 6145263, 2024-05-09T02:27:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 13312, 2024-05-09T02:27:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T02:27:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T02:27:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 13312, 2024-05-09T02:27:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 6145263, 2024-05-09T02:27:43.056 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-05-09T02:27:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T02:27:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T02:27:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T02:27:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:43.057 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 3464, 2024-05-09T02:27:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 3764065, 2024-05-09T02:27:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3464, 2024-05-09T02:27:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3764065, 2024-05-09T02:27:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-05-09T02:27:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-05-09T02:27:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T02:27:43.058 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T02:27:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 3464, 2024-05-09T02:27:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 3464, 2024-05-09T02:27:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3464, 2024-05-09T02:27:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3464, 2024-05-09T02:27:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-05-09T02:27:43.059 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T02:27:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 3, 2024-05-09T02:27:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 399, 2024-05-09T02:27:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 3, 2024-05-09T02:27:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T02:27:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T02:27:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 1, 2024-05-09T02:27:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 399, 2024-05-09T02:27:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:43.060 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55f3dc2fa5b0": { 2024-05-09T02:27:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T02:27:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 2666, 2024-05-09T02:27:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 3515593, 2024-05-09T02:27:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 2666, 2024-05-09T02:27:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T02:27:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T02:27:43.061 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 691, 2024-05-09T02:27:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3515593, 2024-05-09T02:27:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:43.062 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:43.062 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-05-09T02:27:43.879 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-09T02:27:43.920 INFO:tasks.check_counter:Daemon mds.f mds.imported=7 2024-05-09T02:27:43.920 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-05-09T02:27:43.920 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-09T02:27:43.920 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T02:27:43.920 DEBUG:tasks.check_counter:Getting stats from i 2024-05-09T02:27:43.921 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-09T02:27:44.169 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:27:44.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:44 smithi019 ceph-mon[30151]: pgmap v516: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-05-09T02:27:44.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:44 smithi059 ceph-mon[34781]: pgmap v516: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-05-09T02:27:45.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:44 smithi196 ceph-mon[34718]: pgmap v516: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-05-09T02:27:46.113 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-05-09T02:27:46.113 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T02:27:46.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 9419, 2024-05-09T02:27:46.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 8851, 2024-05-09T02:27:46.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 3402047, 2024-05-09T02:27:46.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 16846054, 2024-05-09T02:27:46.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 127, 2024-05-09T02:27:46.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-05-09T02:27:46.113 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.742037735, 2024-05-09T02:27:46.114 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.233738474, 2024-05-09T02:27:46.114 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.356119027, 2024-05-09T02:27:46.114 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.157258747, 2024-05-09T02:27:46.114 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:46.114 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 8851, 2024-05-09T02:27:46.114 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.571040179, 2024-05-09T02:27:46.114 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000064517 2024-05-09T02:27:46.114 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.114 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:46.114 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 9017, 2024-05-09T02:27:46.114 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002047400, 2024-05-09T02:27:46.114 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000227 2024-05-09T02:27:46.114 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.114 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 605728, 2024-05-09T02:27:46.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 186176 2024-05-09T02:27:46.115 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.115 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T02:27:46.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 12867, 2024-05-09T02:27:46.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 15603, 2024-05-09T02:27:46.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 4309294, 2024-05-09T02:27:46.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 20217327, 2024-05-09T02:27:46.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 18, 2024-05-09T02:27:46.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 6, 2024-05-09T02:27:46.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.923326506, 2024-05-09T02:27:46.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.358833166, 2024-05-09T02:27:46.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.843639186, 2024-05-09T02:27:46.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.156087101, 2024-05-09T02:27:46.115 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:46.116 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 15604, 2024-05-09T02:27:46.116 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.621722855, 2024-05-09T02:27:46.116 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000039843 2024-05-09T02:27:46.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.116 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:46.116 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 18484, 2024-05-09T02:27:46.116 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.003370603, 2024-05-09T02:27:46.116 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000182 2024-05-09T02:27:46.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.116 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T02:27:46.116 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 159920 2024-05-09T02:27:46.116 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.116 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T02:27:46.116 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 9376, 2024-05-09T02:27:46.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 10271, 2024-05-09T02:27:46.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 5506131, 2024-05-09T02:27:46.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 10116436, 2024-05-09T02:27:46.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 11, 2024-05-09T02:27:46.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 9, 2024-05-09T02:27:46.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.689714759, 2024-05-09T02:27:46.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.274785795, 2024-05-09T02:27:46.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.439312892, 2024-05-09T02:27:46.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.058992717, 2024-05-09T02:27:46.117 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:46.117 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 10265, 2024-05-09T02:27:46.117 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.629697462, 2024-05-09T02:27:46.117 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000061344 2024-05-09T02:27:46.117 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.118 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:46.118 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 13581, 2024-05-09T02:27:46.118 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002851819, 2024-05-09T02:27:46.118 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000209 2024-05-09T02:27:46.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.118 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 2053360, 2024-05-09T02:27:46.118 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 681824 2024-05-09T02:27:46.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.118 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-05-09T02:27:46.118 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-05-09T02:27:46.118 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-05-09T02:27:46.118 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.118 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-05-09T02:27:46.118 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T02:27:46.119 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T02:27:46.119 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 9530, 2024-05-09T02:27:46.119 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 3.671112793, 2024-05-09T02:27:46.119 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000385216 2024-05-09T02:27:46.119 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:46.119 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.119 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-05-09T02:27:46.119 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T02:27:46.120 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T02:27:46.120 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 277, 2024-05-09T02:27:46.120 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.020388203, 2024-05-09T02:27:46.120 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000073603 2024-05-09T02:27:46.120 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:46.120 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.120 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-05-09T02:27:46.120 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 2553, 2024-05-09T02:27:46.120 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 2340, 2024-05-09T02:27:46.120 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-05-09T02:27:46.120 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2340, 2024-05-09T02:27:46.120 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 35.614544090, 2024-05-09T02:27:46.120 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.015219890 2024-05-09T02:27:46.121 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.121 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-05-09T02:27:46.121 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 213, 2024-05-09T02:27:46.121 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 3, 2024-05-09T02:27:46.121 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-05-09T02:27:46.121 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 47, 2024-05-09T02:27:46.121 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 4, 2024-05-09T02:27:46.121 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-05-09T02:27:46.121 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 801, 2024-05-09T02:27:46.121 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 22, 2024-05-09T02:27:46.121 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 516, 2024-05-09T02:27:46.121 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 263, 2024-05-09T02:27:46.121 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 768, 2024-05-09T02:27:46.121 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 1203, 2024-05-09T02:27:46.121 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 411, 2024-05-09T02:27:46.122 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 411, 2024-05-09T02:27:46.122 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 61, 2024-05-09T02:27:46.122 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 3908, 2024-05-09T02:27:46.122 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 2997, 2024-05-09T02:27:46.122 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 182, 2024-05-09T02:27:46.122 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 42, 2024-05-09T02:27:46.122 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-05-09T02:27:46.122 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 2, 2024-05-09T02:27:46.122 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 95, 2024-05-09T02:27:46.122 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 215, 2024-05-09T02:27:46.122 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-05-09T02:27:46.122 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 22, 2024-05-09T02:27:46.122 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 1065, 2024-05-09T02:27:46.122 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 7, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 358, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 8269, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 11, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 1001, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 1291982165, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 6, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 8269, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 497, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 15, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 482, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 7772, 2024-05-09T02:27:46.123 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 29, 2024-05-09T02:27:46.124 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 352, 2024-05-09T02:27:46.124 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 1769, 2024-05-09T02:27:46.124 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T02:27:46.124 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T02:27:46.124 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T02:27:46.124 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 2079, 2024-05-09T02:27:46.124 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 1225, 2024-05-09T02:27:46.124 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 74, 2024-05-09T02:27:46.124 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 387 2024-05-09T02:27:46.124 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.124 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-05-09T02:27:46.124 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-05-09T02:27:46.124 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-05-09T02:27:46.124 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-05-09T02:27:46.124 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 51, 2024-05-09T02:27:46.125 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 93, 2024-05-09T02:27:46.125 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 4, 2024-05-09T02:27:46.125 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-05-09T02:27:46.125 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-05-09T02:27:46.125 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 128, 2024-05-09T02:27:46.125 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 70, 2024-05-09T02:27:46.125 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 56, 2024-05-09T02:27:46.125 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 21, 2024-05-09T02:27:46.125 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-05-09T02:27:46.125 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-05-09T02:27:46.125 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-05-09T02:27:46.125 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-05-09T02:27:46.125 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-05-09T02:27:46.125 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-05-09T02:27:46.125 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-05-09T02:27:46.126 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 15, 2024-05-09T02:27:46.126 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 22, 2024-05-09T02:27:46.126 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-05-09T02:27:46.126 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 4, 2024-05-09T02:27:46.126 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-05-09T02:27:46.126 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-05-09T02:27:46.126 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.126 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-05-09T02:27:46.126 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-05-09T02:27:46.126 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 4490, 2024-05-09T02:27:46.126 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-05-09T02:27:46.127 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-05-09T02:27:46.127 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 4490, 2024-05-09T02:27:46.127 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-05-09T02:27:46.127 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-05-09T02:27:46.127 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 9, 2024-05-09T02:27:46.127 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-05-09T02:27:46.127 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-05-09T02:27:46.127 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 9, 2024-05-09T02:27:46.127 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-05-09T02:27:46.127 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-05-09T02:27:46.127 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-05-09T02:27:46.127 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-05-09T02:27:46.127 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 33333032, 2024-05-09T02:27:46.127 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-05-09T02:27:46.127 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-05-09T02:27:46.128 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1913, 2024-05-09T02:27:46.128 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 18.610752470, 2024-05-09T02:27:46.128 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.009728568 2024-05-09T02:27:46.128 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.128 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-05-09T02:27:46.128 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.128 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-05-09T02:27:46.128 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 706, 2024-05-09T02:27:46.128 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 1776, 2024-05-09T02:27:46.128 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 1070, 2024-05-09T02:27:46.128 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 120, 2024-05-09T02:27:46.128 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 142, 2024-05-09T02:27:46.128 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 22, 2024-05-09T02:27:46.128 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 806, 2024-05-09T02:27:46.128 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 2333, 2024-05-09T02:27:46.129 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 1527, 2024-05-09T02:27:46.129 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 411, 2024-05-09T02:27:46.129 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 1688, 2024-05-09T02:27:46.129 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 1277, 2024-05-09T02:27:46.129 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 141080, 2024-05-09T02:27:46.129 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 231824 2024-05-09T02:27:46.129 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.129 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-05-09T02:27:46.129 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 3446, 2024-05-09T02:27:46.129 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 1493, 2024-05-09T02:27:46.129 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 2553, 2024-05-09T02:27:46.129 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 328, 2024-05-09T02:27:46.129 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 2692, 2024-05-09T02:27:46.129 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-05-09T02:27:46.129 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1067, 2024-05-09T02:27:46.130 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 2.953564913, 2024-05-09T02:27:46.130 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002768102 2024-05-09T02:27:46.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.130 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-05-09T02:27:46.130 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 199, 2024-05-09T02:27:46.130 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.063559288, 2024-05-09T02:27:46.130 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000319393 2024-05-09T02:27:46.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.130 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-05-09T02:27:46.130 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.130 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.130 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.130 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.130 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-05-09T02:27:46.131 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 54, 2024-05-09T02:27:46.131 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.301056975, 2024-05-09T02:27:46.131 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.005575129 2024-05-09T02:27:46.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.131 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-05-09T02:27:46.131 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 211, 2024-05-09T02:27:46.131 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 7.238295597, 2024-05-09T02:27:46.131 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.034304718 2024-05-09T02:27:46.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.131 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-05-09T02:27:46.131 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.131 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.131 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.131 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.131 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-05-09T02:27:46.132 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-05-09T02:27:46.132 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000200397, 2024-05-09T02:27:46.132 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000200397 2024-05-09T02:27:46.132 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.132 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-05-09T02:27:46.132 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.132 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.132 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.132 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.132 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-05-09T02:27:46.132 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.132 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.132 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.132 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.132 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-05-09T02:27:46.133 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.133 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.133 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.133 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.133 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-05-09T02:27:46.134 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 16, 2024-05-09T02:27:46.134 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002904400, 2024-05-09T02:27:46.134 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000181525 2024-05-09T02:27:46.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.134 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-05-09T02:27:46.134 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 37, 2024-05-09T02:27:46.134 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.297750857, 2024-05-09T02:27:46.134 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.008047320 2024-05-09T02:27:46.134 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.134 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-05-09T02:27:46.134 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.134 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.134 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.135 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-05-09T02:27:46.135 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 16, 2024-05-09T02:27:46.135 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.204994108, 2024-05-09T02:27:46.135 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.012812131 2024-05-09T02:27:46.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.135 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-05-09T02:27:46.135 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 416, 2024-05-09T02:27:46.135 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.243072573, 2024-05-09T02:27:46.135 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000584309 2024-05-09T02:27:46.135 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.135 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-05-09T02:27:46.135 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 84, 2024-05-09T02:27:46.135 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.180666891, 2024-05-09T02:27:46.135 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002150796 2024-05-09T02:27:46.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.136 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-05-09T02:27:46.136 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 123, 2024-05-09T02:27:46.136 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 23.696608104, 2024-05-09T02:27:46.136 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.192655350 2024-05-09T02:27:46.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.136 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-05-09T02:27:46.136 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.136 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.136 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.136 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.136 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-05-09T02:27:46.136 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.136 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.136 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.010351507, 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.005175753 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 10, 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.056143475, 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.005614347 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.138 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-05-09T02:27:46.138 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-05-09T02:27:46.138 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001448172, 2024-05-09T02:27:46.138 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001448172 2024-05-09T02:27:46.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.138 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-05-09T02:27:46.138 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.138 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.138 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.138 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-05-09T02:27:46.138 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.138 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.138 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.139 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-05-09T02:27:46.139 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.139 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.139 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.139 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-05-09T02:27:46.139 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 7, 2024-05-09T02:27:46.139 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.021299040, 2024-05-09T02:27:46.139 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003042720 2024-05-09T02:27:46.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.139 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-05-09T02:27:46.139 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.139 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.139 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.140 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-05-09T02:27:46.140 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 96, 2024-05-09T02:27:46.140 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.342627793, 2024-05-09T02:27:46.140 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003569039 2024-05-09T02:27:46.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.140 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-05-09T02:27:46.140 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-05-09T02:27:46.140 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-05-09T02:27:46.140 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.140 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.140 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.140 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:46.140 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.140 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 2, 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 4, 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 2, 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 2, 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 151, 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 75, 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 968, 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-05-09T02:27:46.141 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T02:27:46.142 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-05-09T02:27:46.142 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T02:27:46.142 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T02:27:46.142 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T02:27:46.142 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T02:27:46.142 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T02:27:46.142 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-05-09T02:27:46.142 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T02:27:46.142 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T02:27:46.142 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-05-09T02:27:46.142 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-05-09T02:27:46.142 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-05-09T02:27:46.142 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-05-09T02:27:46.142 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T02:27:46.142 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T02:27:46.143 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T02:27:46.143 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T02:27:46.143 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T02:27:46.143 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-05-09T02:27:46.143 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-05-09T02:27:46.143 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-05-09T02:27:46.143 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T02:27:46.143 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T02:27:46.143 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-05-09T02:27:46.143 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-05-09T02:27:46.143 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-05-09T02:27:46.143 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-05-09T02:27:46.143 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T02:27:46.143 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-05-09T02:27:46.143 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 616312, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 783, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 264, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 3, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 8224, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 171, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-05-09T02:27:46.144 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-05-09T02:27:46.145 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-05-09T02:27:46.145 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 2862551, 2024-05-09T02:27:46.145 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 39941, 2024-05-09T02:27:46.145 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-05-09T02:27:46.145 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-05-09T02:27:46.145 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-05-09T02:27:46.145 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-05-09T02:27:46.145 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.145 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-05-09T02:27:46.145 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-05-09T02:27:46.145 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-05-09T02:27:46.145 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 11216, 2024-05-09T02:27:46.145 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 29688506, 2024-05-09T02:27:46.145 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 32, 2024-05-09T02:27:46.145 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 11216, 2024-05-09T02:27:46.146 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-05-09T02:27:46.146 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 11216, 2024-05-09T02:27:46.146 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 49.248995481, 2024-05-09T02:27:46.146 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.004390958 2024-05-09T02:27:46.146 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.146 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-05-09T02:27:46.146 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-05-09T02:27:46.146 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 11216, 2024-05-09T02:27:46.146 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 11474 2024-05-09T02:27:46.146 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.146 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 11216, 2024-05-09T02:27:46.146 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 8336, 2024-05-09T02:27:46.146 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 2880, 2024-05-09T02:27:46.146 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-05-09T02:27:46.146 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 11, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 46, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 1944, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 96, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 3, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 603, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 8284, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 22, 2024-05-09T02:27:46.147 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-05-09T02:27:46.148 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-05-09T02:27:46.148 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-05-09T02:27:46.148 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-05-09T02:27:46.148 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-05-09T02:27:46.148 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-05-09T02:27:46.148 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T02:27:46.148 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-05-09T02:27:46.148 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-05-09T02:27:46.148 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-05-09T02:27:46.148 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-05-09T02:27:46.148 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-05-09T02:27:46.148 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-05-09T02:27:46.148 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-05-09T02:27:46.148 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-05-09T02:27:46.148 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 102, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 23, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 12, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 12, 2024-05-09T02:27:46.149 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-05-09T02:27:46.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-05-09T02:27:46.150 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 369, 2024-05-09T02:27:46.150 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 9, 2024-05-09T02:27:46.150 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 87 2024-05-09T02:27:46.150 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.150 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-05-09T02:27:46.150 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-05-09T02:27:46.150 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-05-09T02:27:46.150 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-05-09T02:27:46.150 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-05-09T02:27:46.150 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.150 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-05-09T02:27:46.150 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-05-09T02:27:46.150 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.150 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 13, 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 6, 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 84, 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 70, 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 31658, 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 10277690, 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:46.151 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 31658, 2024-05-09T02:27:46.152 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T02:27:46.152 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T02:27:46.152 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 31658, 2024-05-09T02:27:46.152 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 10277690, 2024-05-09T02:27:46.152 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:46.152 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.152 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.152 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.152 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:46.152 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.152 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-05-09T02:27:46.152 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:46.152 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T02:27:46.152 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:46.152 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T02:27:46.153 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T02:27:46.153 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:46.153 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:46.153 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 11216, 2024-05-09T02:27:46.153 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 29696118, 2024-05-09T02:27:46.153 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 11216, 2024-05-09T02:27:46.153 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 29696118, 2024-05-09T02:27:46.153 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:46.153 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.153 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.153 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.153 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:46.153 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.153 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-05-09T02:27:46.153 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 11216, 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 11216, 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 11216, 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 11216, 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:46.154 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.155 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-05-09T02:27:46.155 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:46.155 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T02:27:46.155 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:46.155 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 70, 2024-05-09T02:27:46.155 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 7726, 2024-05-09T02:27:46.155 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:46.155 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 70, 2024-05-09T02:27:46.155 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T02:27:46.155 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T02:27:46.155 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 31, 2024-05-09T02:27:46.155 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 7726, 2024-05-09T02:27:46.155 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:46.155 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.155 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.156 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.156 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:46.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:46.156 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55f5d33d04e0": { 2024-05-09T02:27:46.156 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:46.156 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T02:27:46.156 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:46.156 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 4490, 2024-05-09T02:27:46.156 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 29138728, 2024-05-09T02:27:46.156 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:46.156 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 4490, 2024-05-09T02:27:46.156 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T02:27:46.156 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T02:27:46.156 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 1913, 2024-05-09T02:27:46.156 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 29138728, 2024-05-09T02:27:46.157 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:46.157 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:46.157 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:46.157 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:46.157 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:46.157 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:46.157 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-05-09T02:27:46.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:46 smithi019 ceph-mon[30151]: pgmap v517: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-05-09T02:27:46.913 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:46 smithi196 ceph-mon[34718]: pgmap v517: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-05-09T02:27:46.941 INFO:tasks.check_counter:Daemon mds.i mds.exported=22 2024-05-09T02:27:46.941 INFO:tasks.check_counter:Daemon mds.i mds.imported=7 2024-05-09T02:27:46.941 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-09T02:27:46.941 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=16 2024-05-09T02:27:46.941 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=10 2024-05-09T02:27:46.941 DEBUG:tasks.check_counter:Getting stats from l 2024-05-09T02:27:46.942 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-09T02:27:46.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:46 smithi059 ceph-mon[34781]: pgmap v517: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-05-09T02:27:47.180 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T02:27:48.641 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:48 smithi196 ceph-mon[34718]: pgmap v518: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-05-09T02:27:48.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:48 smithi019 ceph-mon[30151]: pgmap v518: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-05-09T02:27:48.931 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-05-09T02:27:48.931 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T02:27:48.931 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 3565, 2024-05-09T02:27:48.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 3565, 2024-05-09T02:27:48.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 4213281, 2024-05-09T02:27:48.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 1132853, 2024-05-09T02:27:48.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-05-09T02:27:48.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-05-09T02:27:48.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.285118062, 2024-05-09T02:27:48.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.082452309, 2024-05-09T02:27:48.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.074118093, 2024-05-09T02:27:48.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.109698659, 2024-05-09T02:27:48.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:48.932 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3564, 2024-05-09T02:27:48.932 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.159525293, 2024-05-09T02:27:48.932 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000044760 2024-05-09T02:27:48.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.932 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:48.933 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.933 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.933 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.933 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T02:27:48.933 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T02:27:48.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.933 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T02:27:48.933 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 5, 2024-05-09T02:27:48.933 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 258, 2024-05-09T02:27:48.933 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1036, 2024-05-09T02:27:48.933 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 691082, 2024-05-09T02:27:48.933 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 7, 2024-05-09T02:27:48.933 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-05-09T02:27:48.934 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.013683611, 2024-05-09T02:27:48.934 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.008841299, 2024-05-09T02:27:48.934 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.002367384, 2024-05-09T02:27:48.934 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000048409, 2024-05-09T02:27:48.934 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:48.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 255, 2024-05-09T02:27:48.934 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.039698152, 2024-05-09T02:27:48.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000155679 2024-05-09T02:27:48.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.934 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:48.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.935 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.935 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.935 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-09T02:27:48.935 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 690384 2024-05-09T02:27:48.935 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.936 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T02:27:48.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 407, 2024-05-09T02:27:48.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 361, 2024-05-09T02:27:48.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 2012012, 2024-05-09T02:27:48.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 210558, 2024-05-09T02:27:48.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 4, 2024-05-09T02:27:48.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-05-09T02:27:48.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.069008193, 2024-05-09T02:27:48.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.018410898, 2024-05-09T02:27:48.936 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.055915035, 2024-05-09T02:27:48.937 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.008057742, 2024-05-09T02:27:48.937 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T02:27:48.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 360, 2024-05-09T02:27:48.937 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.026742435, 2024-05-09T02:27:48.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000074284 2024-05-09T02:27:48.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.938 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-05-09T02:27:48.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.938 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.939 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 2011744, 2024-05-09T02:27:48.939 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 210240 2024-05-09T02:27:48.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.939 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-05-09T02:27:48.939 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-05-09T02:27:48.939 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-05-09T02:27:48.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.940 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-05-09T02:27:48.940 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T02:27:48.940 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T02:27:48.940 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 6050, 2024-05-09T02:27:48.940 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.154437143, 2024-05-09T02:27:48.940 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000025526 2024-05-09T02:27:48.940 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:48.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.940 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-05-09T02:27:48.940 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-05-09T02:27:48.940 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-05-09T02:27:48.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.941 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.941 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:48.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.941 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-05-09T02:27:48.941 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-05-09T02:27:48.941 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-05-09T02:27:48.941 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-05-09T02:27:48.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.941 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.941 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-05-09T02:27:48.941 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-05-09T02:27:48.942 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-05-09T02:27:48.942 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-05-09T02:27:48.942 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 0, 2024-05-09T02:27:48.942 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-05-09T02:27:48.942 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-05-09T02:27:48.942 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 461, 2024-05-09T02:27:48.942 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 7, 2024-05-09T02:27:48.942 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 454, 2024-05-09T02:27:48.942 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-05-09T02:27:48.942 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 451, 2024-05-09T02:27:48.942 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-05-09T02:27:48.942 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-05-09T02:27:48.942 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-05-09T02:27:48.942 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 7, 2024-05-09T02:27:48.943 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-05-09T02:27:48.943 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-05-09T02:27:48.943 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-05-09T02:27:48.943 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-05-09T02:27:48.943 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-05-09T02:27:48.943 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-05-09T02:27:48.943 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-05-09T02:27:48.943 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-05-09T02:27:48.943 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-05-09T02:27:48.943 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-05-09T02:27:48.943 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-05-09T02:27:48.943 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-05-09T02:27:48.944 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-05-09T02:27:48.944 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-05-09T02:27:48.944 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-05-09T02:27:48.944 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-05-09T02:27:48.944 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 0, 2024-05-09T02:27:48.944 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 0, 2024-05-09T02:27:48.944 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-05-09T02:27:48.944 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T02:27:48.944 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-05-09T02:27:48.944 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T02:27:48.944 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-05-09T02:27:48.944 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-05-09T02:27:48.944 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T02:27:48.944 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T02:27:48.944 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-05-09T02:27:48.945 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-05-09T02:27:48.945 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T02:27:48.945 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-05-09T02:27:48.945 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T02:27:48.945 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T02:27:48.945 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T02:27:48.945 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-05-09T02:27:48.945 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-05-09T02:27:48.945 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-05-09T02:27:48.945 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-05-09T02:27:48.945 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.945 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-05-09T02:27:48.945 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-05-09T02:27:48.945 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-05-09T02:27:48.946 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-05-09T02:27:48.946 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 0, 2024-05-09T02:27:48.946 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-05-09T02:27:48.946 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-05-09T02:27:48.946 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-05-09T02:27:48.946 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-05-09T02:27:48.946 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-05-09T02:27:48.946 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-05-09T02:27:48.946 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-05-09T02:27:48.947 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-05-09T02:27:48.947 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-05-09T02:27:48.947 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-05-09T02:27:48.947 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-05-09T02:27:48.947 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-05-09T02:27:48.947 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-05-09T02:27:48.947 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-05-09T02:27:48.947 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-05-09T02:27:48.947 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T02:27:48.948 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-05-09T02:27:48.948 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-05-09T02:27:48.948 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-05-09T02:27:48.948 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-05-09T02:27:48.948 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-05-09T02:27:48.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.948 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-05-09T02:27:48.948 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-05-09T02:27:48.948 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 0, 2024-05-09T02:27:48.948 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-05-09T02:27:48.948 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-05-09T02:27:48.948 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 1576, 2024-05-09T02:27:48.948 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-05-09T02:27:48.949 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-05-09T02:27:48.949 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 0, 2024-05-09T02:27:48.949 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-05-09T02:27:48.949 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-05-09T02:27:48.949 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 3, 2024-05-09T02:27:48.949 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-05-09T02:27:48.949 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-05-09T02:27:48.949 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-05-09T02:27:48.949 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-05-09T02:27:48.949 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 7403144, 2024-05-09T02:27:48.949 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 7402570, 2024-05-09T02:27:48.949 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-05-09T02:27:48.950 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.950 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.950 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.950 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 1576 2024-05-09T02:27:48.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.950 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-05-09T02:27:48.950 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 198, 2024-05-09T02:27:48.950 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 496, 2024-05-09T02:27:48.950 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 298, 2024-05-09T02:27:48.950 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 17, 2024-05-09T02:27:48.951 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 27, 2024-05-09T02:27:48.951 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 10, 2024-05-09T02:27:48.951 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 461, 2024-05-09T02:27:48.951 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 778, 2024-05-09T02:27:48.951 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 317, 2024-05-09T02:27:48.951 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-05-09T02:27:48.951 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-05-09T02:27:48.951 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-05-09T02:27:48.951 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 121188, 2024-05-09T02:27:48.951 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 215440 2024-05-09T02:27:48.951 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.951 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-05-09T02:27:48.951 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-05-09T02:27:48.951 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-05-09T02:27:48.952 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-05-09T02:27:48.952 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-05-09T02:27:48.952 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-05-09T02:27:48.952 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-05-09T02:27:48.952 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.952 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.952 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.953 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-05-09T02:27:48.953 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.953 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.953 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.953 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-05-09T02:27:48.953 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.953 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.953 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.953 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.953 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-05-09T02:27:48.953 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.953 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.954 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-05-09T02:27:48.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.954 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.954 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-05-09T02:27:48.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.954 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.954 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-05-09T02:27:48.954 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.954 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.955 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-05-09T02:27:48.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.955 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.955 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-05-09T02:27:48.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.955 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.955 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.955 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-05-09T02:27:48.956 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.956 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.956 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.956 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-05-09T02:27:48.956 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.956 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.956 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.956 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-05-09T02:27:48.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.957 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.957 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-05-09T02:27:48.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.957 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.958 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-05-09T02:27:48.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.958 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.958 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-05-09T02:27:48.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.958 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.958 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-05-09T02:27:48.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.958 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.958 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.959 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-05-09T02:27:48.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.959 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.959 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-05-09T02:27:48.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.959 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.959 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-05-09T02:27:48.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.959 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.959 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.960 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-05-09T02:27:48.960 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.960 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.960 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.960 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-05-09T02:27:48.960 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.960 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.960 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.961 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-05-09T02:27:48.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.961 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.961 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-05-09T02:27:48.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.961 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.961 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-05-09T02:27:48.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.961 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.961 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.962 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-05-09T02:27:48.962 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.962 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.962 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.962 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-05-09T02:27:48.962 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.962 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.962 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.962 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-05-09T02:27:48.962 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.962 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.962 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.962 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.963 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-05-09T02:27:48.963 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.963 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.963 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.963 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-05-09T02:27:48.963 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.963 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.963 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.963 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.963 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-05-09T02:27:48.963 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-05-09T02:27:48.963 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-05-09T02:27:48.963 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.964 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.964 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.964 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:48.964 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.964 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-05-09T02:27:48.964 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 2, 2024-05-09T02:27:48.964 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 4, 2024-05-09T02:27:48.964 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 2, 2024-05-09T02:27:48.964 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 4, 2024-05-09T02:27:48.964 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-05-09T02:27:48.964 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-05-09T02:27:48.964 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-05-09T02:27:48.964 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 3831, 2024-05-09T02:27:48.965 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-05-09T02:27:48.965 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.965 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-05-09T02:27:48.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-05-09T02:27:48.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-05-09T02:27:48.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T02:27:48.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-05-09T02:27:48.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T02:27:48.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-05-09T02:27:48.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T02:27:48.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T02:27:48.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T02:27:48.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T02:27:48.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T02:27:48.965 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-05-09T02:27:48.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T02:27:48.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T02:27:48.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-05-09T02:27:48.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-05-09T02:27:48.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-05-09T02:27:48.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-05-09T02:27:48.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T02:27:48.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T02:27:48.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T02:27:48.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T02:27:48.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T02:27:48.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-05-09T02:27:48.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-05-09T02:27:48.966 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-05-09T02:27:48.967 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T02:27:48.967 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T02:27:48.967 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-05-09T02:27:48.967 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-05-09T02:27:48.967 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-05-09T02:27:48.967 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-05-09T02:27:48.967 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T02:27:48.967 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-05-09T02:27:48.967 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T02:27:48.968 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-05-09T02:27:48.969 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 191358, 2024-05-09T02:27:48.970 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 298, 2024-05-09T02:27:48.970 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-05-09T02:27:48.970 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-05-09T02:27:48.970 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-05-09T02:27:48.970 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-05-09T02:27:48.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-05-09T02:27:48.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-05-09T02:27:48.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-05-09T02:27:48.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-05-09T02:27:48.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 8224, 2024-05-09T02:27:48.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 171, 2024-05-09T02:27:48.971 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T02:27:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-05-09T02:27:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-05-09T02:27:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-05-09T02:27:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 898230, 2024-05-09T02:27:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 28718, 2024-05-09T02:27:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-05-09T02:27:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-05-09T02:27:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-05-09T02:27:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-05-09T02:27:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-05-09T02:27:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-05-09T02:27:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-05-09T02:27:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 3564, 2024-05-09T02:27:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 21, 2024-05-09T02:27:48.973 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 3, 2024-05-09T02:27:48.973 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 3564, 2024-05-09T02:27:48.973 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-05-09T02:27:48.973 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3564, 2024-05-09T02:27:48.973 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 3.899920883, 2024-05-09T02:27:48.973 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001094253 2024-05-09T02:27:48.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.973 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-05-09T02:27:48.973 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-05-09T02:27:48.973 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3564, 2024-05-09T02:27:48.973 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 3565 2024-05-09T02:27:48.973 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.973 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 3564, 2024-05-09T02:27:48.973 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 3564, 2024-05-09T02:27:48.973 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 0, 2024-05-09T02:27:48.974 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-05-09T02:27:48.974 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-05-09T02:27:48.974 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 2167, 2024-05-09T02:27:48.974 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 0, 2024-05-09T02:27:48.974 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 1395, 2024-05-09T02:27:48.974 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 0, 2024-05-09T02:27:48.974 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 0, 2024-05-09T02:27:48.974 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-05-09T02:27:48.974 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-05-09T02:27:48.974 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-05-09T02:27:48.974 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-05-09T02:27:48.974 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 0, 2024-05-09T02:27:48.974 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-05-09T02:27:48.974 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-05-09T02:27:48.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-05-09T02:27:48.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 1, 2024-05-09T02:27:48.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 0, 2024-05-09T02:27:48.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-05-09T02:27:48.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-05-09T02:27:48.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-05-09T02:27:48.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-05-09T02:27:48.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-05-09T02:27:48.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-05-09T02:27:48.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T02:27:48.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-05-09T02:27:48.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-05-09T02:27:48.975 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-05-09T02:27:48.975 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-05-09T02:27:48.975 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-05-09T02:27:48.976 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-05-09T02:27:48.976 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-05-09T02:27:48.976 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-05-09T02:27:48.976 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-05-09T02:27:48.976 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-05-09T02:27:48.976 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-05-09T02:27:48.976 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-05-09T02:27:48.976 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-05-09T02:27:48.976 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-05-09T02:27:48.976 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-05-09T02:27:48.976 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-05-09T02:27:48.977 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-05-09T02:27:48.977 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-05-09T02:27:48.977 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-05-09T02:27:48.977 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 102, 2024-05-09T02:27:48.977 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-05-09T02:27:48.978 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 23, 2024-05-09T02:27:48.978 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 4, 2024-05-09T02:27:48.978 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 5, 2024-05-09T02:27:48.978 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 1, 2024-05-09T02:27:48.978 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-05-09T02:27:48.978 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 0, 2024-05-09T02:27:48.978 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 2, 2024-05-09T02:27:48.978 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 0 2024-05-09T02:27:48.978 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.978 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-05-09T02:27:48.978 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 0, 2024-05-09T02:27:48.978 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-05-09T02:27:48.979 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-05-09T02:27:48.979 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-05-09T02:27:48.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.979 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-05-09T02:27:48.979 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-05-09T02:27:48.979 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.979 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-05-09T02:27:48.979 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-05-09T02:27:48.979 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T02:27:48.979 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-05-09T02:27:48.979 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-05-09T02:27:48.979 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 3973, 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 5845082, 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 3973, 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3973, 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 5845082, 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:48.980 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.981 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.981 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.981 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:48.981 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.981 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-05-09T02:27:48.981 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:48.981 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-05-09T02:27:48.981 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:48.982 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T02:27:48.982 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T02:27:48.982 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:48.982 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:48.982 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 3564, 2024-05-09T02:27:48.982 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 3208845, 2024-05-09T02:27:48.982 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3564, 2024-05-09T02:27:48.982 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3208845, 2024-05-09T02:27:48.982 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:48.983 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.983 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.983 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.983 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:48.983 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.983 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-05-09T02:27:48.983 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:48.983 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-05-09T02:27:48.984 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:48.984 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T02:27:48.984 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T02:27:48.984 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:48.984 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:48.984 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 3564, 2024-05-09T02:27:48.985 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 3564, 2024-05-09T02:27:48.985 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3564, 2024-05-09T02:27:48.985 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3564, 2024-05-09T02:27:48.985 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:48.986 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.986 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.986 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.986 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:48.986 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.986 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-05-09T02:27:48.986 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:48.986 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T02:27:48.987 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:48.987 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T02:27:48.987 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T02:27:48.987 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:48.987 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:48.987 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T02:27:48.987 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T02:27:48.987 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-05-09T02:27:48.987 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-05-09T02:27:48.987 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:48.987 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.987 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.987 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.987 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:48.987 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-05-09T02:27:48.988 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x561058ace8f0": { 2024-05-09T02:27:48.988 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-05-09T02:27:48.988 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-05-09T02:27:48.988 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-05-09T02:27:48.988 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-05-09T02:27:48.988 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-05-09T02:27:48.988 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-05-09T02:27:48.988 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-05-09T02:27:48.988 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-05-09T02:27:48.988 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-05-09T02:27:48.988 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-05-09T02:27:48.988 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-05-09T02:27:48.988 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-05-09T02:27:48.988 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-05-09T02:27:48.988 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-05-09T02:27:48.989 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-05-09T02:27:48.989 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:48.989 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-05-09T02:27:48.989 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-05-09T02:27:48.989 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:48 smithi059 ceph-mon[34781]: pgmap v518: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-05-09T02:27:49.726 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-09T02:27:49.727 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-09T02:27:49.727 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-05-09T02:27:49.727 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-05-09T02:27:49.727 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T02:27:49.727 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-09T02:27:49.739 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-09T02:27:49.740 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T02:27:49.740 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T02:27:49.773 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-09T02:27:49.773 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T02:27:49.835 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-09T02:27:50.142 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:27:50.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:50 smithi019 ceph-mon[30151]: pgmap v519: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-05-09T02:27:50.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:50 smithi059 ceph-mon[34781]: pgmap v519: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-05-09T02:27:51.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:50 smithi196 ceph-mon[34718]: pgmap v519: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-05-09T02:27:52.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.028+0000 7f495349e700 1 -- 172.21.15.19:0/3414032896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c108f00 msgr2=0x7f494c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:52.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.028+0000 7f495349e700 1 --2- 172.21.15.19:0/3414032896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c108f00 0x7f494c109310 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f493c0099c0 tx=0x7f493c009cd0 comp rx=0 tx=0).stop 2024-05-09T02:27:52.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.028+0000 7f495349e700 1 -- 172.21.15.19:0/3414032896 shutdown_connections 2024-05-09T02:27:52.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.028+0000 7f495349e700 1 --2- 172.21.15.19:0/3414032896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f494c109850 0x7f494c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:52.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.028+0000 7f495349e700 1 --2- 172.21.15.19:0/3414032896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f494c1045d0 0x7f494c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.028+0000 7f495349e700 1 --2- 172.21.15.19:0/3414032896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c108f00 0x7f494c109310 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.028+0000 7f495349e700 1 -- 172.21.15.19:0/3414032896 >> 172.21.15.19:0/3414032896 conn(0x7f494c0ffea0 msgr2=0x7f494c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.028+0000 7f495349e700 1 -- 172.21.15.19:0/3414032896 shutdown_connections 2024-05-09T02:27:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.028+0000 7f495349e700 1 -- 172.21.15.19:0/3414032896 wait complete. 2024-05-09T02:27:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.029+0000 7f495349e700 1 Processor -- start 2024-05-09T02:27:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.029+0000 7f495349e700 1 -- start start 2024-05-09T02:27:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.029+0000 7f495349e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f494c1045d0 0x7f494c19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.029+0000 7f495349e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f494c108f00 0x7f494c19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.029+0000 7f495349e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c109850 0x7f494c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.029+0000 7f495349e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f494c114460 con 0x7f494c109850 2024-05-09T02:27:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.029+0000 7f495349e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f494c1142e0 con 0x7f494c108f00 2024-05-09T02:27:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.029+0000 7f495349e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f494c1145e0 con 0x7f494c1045d0 2024-05-09T02:27:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.029+0000 7f4951a3b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c109850 0x7f494c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.029+0000 7f4951a3b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c109850 0x7f494c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59374/0 (socket says 172.21.15.19:59374) 2024-05-09T02:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.029+0000 7f4951a3b700 1 -- 172.21.15.19:0/2661986813 learned_addr learned my addr 172.21.15.19:0/2661986813 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.029+0000 7f495123a700 1 --2- 172.21.15.19:0/2661986813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f494c1045d0 0x7f494c19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.029+0000 7f4950a39700 1 --2- 172.21.15.19:0/2661986813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f494c108f00 0x7f494c19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.030+0000 7f4951a3b700 1 -- 172.21.15.19:0/2661986813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f494c1045d0 msgr2=0x7f494c19aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.030+0000 7f4951a3b700 1 --2- 172.21.15.19:0/2661986813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f494c1045d0 0x7f494c19aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.030+0000 7f4951a3b700 1 -- 172.21.15.19:0/2661986813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f494c108f00 msgr2=0x7f494c19b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.030+0000 7f4951a3b700 1 --2- 172.21.15.19:0/2661986813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f494c108f00 0x7f494c19b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.030+0000 7f4951a3b700 1 -- 172.21.15.19:0/2661986813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494c19fda0 con 0x7f494c109850 2024-05-09T02:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.030+0000 7f4951a3b700 1 --2- 172.21.15.19:0/2661986813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c109850 0x7f494c19f700 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f494400ada0 tx=0x7f494400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.030+0000 7f49427fc700 1 -- 172.21.15.19:0/2661986813 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944017020 con 0x7f494c109850 2024-05-09T02:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.030+0000 7f49427fc700 1 -- 172.21.15.19:0/2661986813 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f494400ce30 con 0x7f494c109850 2024-05-09T02:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.030+0000 7f495349e700 1 -- 172.21.15.19:0/2661986813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494c1a0090 con 0x7f494c109850 2024-05-09T02:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.030+0000 7f49427fc700 1 -- 172.21.15.19:0/2661986813 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944005350 con 0x7f494c109850 2024-05-09T02:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.030+0000 7f495349e700 1 -- 172.21.15.19:0/2661986813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f494c1a79b0 con 0x7f494c109850 2024-05-09T02:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.031+0000 7f495349e700 1 -- 172.21.15.19:0/2661986813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f494c105a80 con 0x7f494c109850 2024-05-09T02:27:52.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.033+0000 7f49427fc700 1 -- 172.21.15.19:0/2661986813 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f494400c980 con 0x7f494c109850 2024-05-09T02:27:52.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.033+0000 7f49427fc700 1 --2- 172.21.15.19:0/2661986813 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f49380b2a70 0x7f49380b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:52.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.033+0000 7f495123a700 1 --2- 172.21.15.19:0/2661986813 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f49380b2a70 0x7f49380b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:52.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.034+0000 7f49427fc700 1 -- 172.21.15.19:0/2661986813 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f49440e2270 con 0x7f494c109850 2024-05-09T02:27:52.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.034+0000 7f495123a700 1 --2- 172.21.15.19:0/2661986813 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f49380b2a70 0x7f49380b4f80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4948005950 tx=0x7f494800b410 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:52.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.037+0000 7f49427fc700 1 -- 172.21.15.19:0/2661986813 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4944021020 con 0x7f494c109850 2024-05-09T02:27:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.266+0000 7f495349e700 1 -- 172.21.15.19:0/2661986813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f494c02e5f0 con 0x7f494c109850 2024-05-09T02:27:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.267+0000 7f49427fc700 1 -- 172.21.15.19:0/2661986813 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v102) ==== 81+0+941 (secure 0 0 0) 0x7f49440a9cb0 con 0x7f494c109850 2024-05-09T02:27:52.270 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3660764754 2024-05-10T02:11:17.084815+0000 2024-05-09T02:27:52.270 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3573608809 2024-05-10T02:11:17.084815+0000 2024-05-09T02:27:52.270 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3291838961 2024-05-10T02:00:23.920230+0000 2024-05-09T02:27:52.270 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1024133854 2024-05-10T02:00:23.920230+0000 2024-05-09T02:27:52.270 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1176455101 2024-05-10T02:00:23.920230+0000 2024-05-09T02:27:52.270 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2101052000 2024-05-10T02:00:23.920230+0000 2024-05-09T02:27:52.270 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/972543870 2024-05-10T02:11:17.084815+0000 2024-05-09T02:27:52.270 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/4018987404 2024-05-10T01:59:56.604998+0000 2024-05-09T02:27:52.270 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3079777324 2024-05-10T01:59:56.604998+0000 2024-05-09T02:27:52.270 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/972543870 2024-05-10T02:11:17.084815+0000 2024-05-09T02:27:52.270 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/1829553843 2024-05-10T01:59:56.604998+0000 2024-05-09T02:27:52.270 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/557070897 2024-05-10T02:11:17.084815+0000 2024-05-09T02:27:52.271 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/758334732 2024-05-10T02:11:17.084815+0000 2024-05-09T02:27:52.271 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/1829553843 2024-05-10T01:59:56.604998+0000 2024-05-09T02:27:52.271 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/179703528 2024-05-10T01:59:56.604998+0000 2024-05-09T02:27:52.271 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2101052000 2024-05-10T02:00:23.920230+0000 2024-05-09T02:27:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.270+0000 7f495349e700 1 -- 172.21.15.19:0/2661986813 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f49380b2a70 msgr2=0x7f49380b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.270+0000 7f495349e700 1 --2- 172.21.15.19:0/2661986813 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f49380b2a70 0x7f49380b4f80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4948005950 tx=0x7f494800b410 comp rx=0 tx=0).stop 2024-05-09T02:27:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.270+0000 7f495349e700 1 -- 172.21.15.19:0/2661986813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c109850 msgr2=0x7f494c19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.270+0000 7f495349e700 1 --2- 172.21.15.19:0/2661986813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c109850 0x7f494c19f700 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f494400ada0 tx=0x7f494400c330 comp rx=0 tx=0).stop 2024-05-09T02:27:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.270+0000 7f495349e700 1 -- 172.21.15.19:0/2661986813 shutdown_connections 2024-05-09T02:27:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.270+0000 7f495349e700 1 --2- 172.21.15.19:0/2661986813 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f49380b2a70 0x7f49380b4f80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.270+0000 7f495349e700 1 --2- 172.21.15.19:0/2661986813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c109850 0x7f494c19f700 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.270+0000 7f495349e700 1 --2- 172.21.15.19:0/2661986813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f494c1045d0 0x7f494c19aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.270+0000 7f495349e700 1 --2- 172.21.15.19:0/2661986813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f494c108f00 0x7f494c19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.270+0000 7f495349e700 1 -- 172.21.15.19:0/2661986813 >> 172.21.15.19:0/2661986813 conn(0x7f494c0ffea0 msgr2=0x7f494c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.270+0000 7f495349e700 1 -- 172.21.15.19:0/2661986813 shutdown_connections 2024-05-09T02:27:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:52.270+0000 7f495349e700 1 -- 172.21.15.19:0/2661986813 wait complete. 2024-05-09T02:27:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:listed 16 entries 2024-05-09T02:27:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:52 smithi059 ceph-mon[34781]: pgmap v520: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-05-09T02:27:52.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:52 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2661986813' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T02:27:52.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:52 smithi019 ceph-mon[30151]: pgmap v520: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-05-09T02:27:52.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:52 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2661986813' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T02:27:53.019 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-09T02:27:53.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T02:27:53.019 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /tmp/cephfs 2024-05-09T02:27:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:52 smithi196 ceph-mon[34718]: pgmap v520: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-05-09T02:27:53.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:52 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2661986813' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T02:27:53.140 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T02:27:53.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T02:27:53.140 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-05-09T02:27:53.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T02:27:53.211 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-05-09T02:27:53.271 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-05-09T02:27:53.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T02:27:53.272 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-05-09T02:27:53.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T02:27:53.341 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-05-09T02:27:53.454 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-09T02:27:53.465 INFO:tasks.cephadm:Teardown begin 2024-05-09T02:27:53.465 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T02:27:53.465 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:27:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:27:54.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:54 smithi019 ceph-mon[30151]: pgmap v521: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-09T02:27:54.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:54 smithi059 ceph-mon[34781]: pgmap v521: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-09T02:27:55.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:54 smithi196 ceph-mon[34718]: pgmap v521: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-09T02:27:55.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.843+0000 7fd0673fd700 1 -- 172.21.15.19:0/809902926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd060107df0 msgr2=0x7fd060108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.843+0000 7fd0673fd700 1 --2- 172.21.15.19:0/809902926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd060107df0 0x7fd060108200 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd0500099c0 tx=0x7fd050009cd0 comp rx=0 tx=0).stop 2024-05-09T02:27:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.843+0000 7fd0673fd700 1 -- 172.21.15.19:0/809902926 shutdown_connections 2024-05-09T02:27:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.843+0000 7fd0673fd700 1 --2- 172.21.15.19:0/809902926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd060108740 0x7fd0601100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.843+0000 7fd0673fd700 1 --2- 172.21.15.19:0/809902926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0601034c0 0x7fd0601038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.843+0000 7fd0673fd700 1 --2- 172.21.15.19:0/809902926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd060107df0 0x7fd060108200 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.843+0000 7fd0673fd700 1 -- 172.21.15.19:0/809902926 >> 172.21.15.19:0/809902926 conn(0x7fd0600fedc0 msgr2=0x7fd0601011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.843+0000 7fd0673fd700 1 -- 172.21.15.19:0/809902926 shutdown_connections 2024-05-09T02:27:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.843+0000 7fd0673fd700 1 -- 172.21.15.19:0/809902926 wait complete. 2024-05-09T02:27:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.844+0000 7fd0673fd700 1 Processor -- start 2024-05-09T02:27:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.844+0000 7fd0673fd700 1 -- start start 2024-05-09T02:27:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.844+0000 7fd0673fd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0601034c0 0x7fd06019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.844+0000 7fd0673fd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd060107df0 0x7fd06019c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.844+0000 7fd0673fd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd060108740 0x7fd06019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.844+0000 7fd0673fd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd060113350 con 0x7fd060108740 2024-05-09T02:27:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.844+0000 7fd0673fd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601131d0 con 0x7fd0601034c0 2024-05-09T02:27:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.844+0000 7fd0673fd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601134d0 con 0x7fd060107df0 2024-05-09T02:27:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd06599a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd060108740 0x7fd06019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd065199700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0601034c0 0x7fd06019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd06599a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd060108740 0x7fd06019c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59390/0 (socket says 172.21.15.19:59390) 2024-05-09T02:27:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd065199700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0601034c0 0x7fd06019bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.19:36158/0 (socket says 172.21.15.19:36158) 2024-05-09T02:27:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd06599a700 1 -- 172.21.15.19:0/277841748 learned_addr learned my addr 172.21.15.19:0/277841748 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd064998700 1 --2- 172.21.15.19:0/277841748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd060107df0 0x7fd06019c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd06599a700 1 -- 172.21.15.19:0/277841748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd060107df0 msgr2=0x7fd06019c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd06599a700 1 --2- 172.21.15.19:0/277841748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd060107df0 0x7fd06019c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd06599a700 1 -- 172.21.15.19:0/277841748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0601034c0 msgr2=0x7fd06019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd06599a700 1 --2- 172.21.15.19:0/277841748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0601034c0 0x7fd06019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd06599a700 1 -- 172.21.15.19:0/277841748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0601a7000 con 0x7fd060108740 2024-05-09T02:27:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd065199700 1 --2- 172.21.15.19:0/277841748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0601034c0 0x7fd06019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T02:27:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd06599a700 1 --2- 172.21.15.19:0/277841748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd060108740 0x7fd06019c750 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd058007ee0 tx=0x7fd05800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd0567fc700 1 -- 172.21.15.19:0/277841748 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05800dd20 con 0x7fd060108740 2024-05-09T02:27:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd0673fd700 1 -- 172.21.15.19:0/277841748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0601a72f0 con 0x7fd060108740 2024-05-09T02:27:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd0567fc700 1 -- 172.21.15.19:0/277841748 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd058004020 con 0x7fd060108740 2024-05-09T02:27:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.845+0000 7fd0567fc700 1 -- 172.21.15.19:0/277841748 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058015660 con 0x7fd060108740 2024-05-09T02:27:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.846+0000 7fd0673fd700 1 -- 172.21.15.19:0/277841748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0601a7800 con 0x7fd060108740 2024-05-09T02:27:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.847+0000 7fd0673fd700 1 -- 172.21.15.19:0/277841748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0600512c0 con 0x7fd060108740 2024-05-09T02:27:55.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.849+0000 7fd0567fc700 1 -- 172.21.15.19:0/277841748 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd0580041d0 con 0x7fd060108740 2024-05-09T02:27:55.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.850+0000 7fd0567fc700 1 --2- 172.21.15.19:0/277841748 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fd04c0b29f0 0x7fd04c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:55.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.850+0000 7fd0567fc700 1 -- 172.21.15.19:0/277841748 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7fd0580e27a0 con 0x7fd060108740 2024-05-09T02:27:55.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.850+0000 7fd065199700 1 --2- 172.21.15.19:0/277841748 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fd04c0b29f0 0x7fd04c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:55.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.851+0000 7fd065199700 1 --2- 172.21.15.19:0/277841748 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fd04c0b29f0 0x7fd04c0b4f00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd05c011490 tx=0x7fd05c009260 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:55.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:55.854+0000 7fd0567fc700 1 -- 172.21.15.19:0/277841748 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd058010070 con 0x7fd060108740 2024-05-09T02:27:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:56.121+0000 7fd0673fd700 1 -- 172.21.15.19:0/277841748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd06002e5f0 con 0x7fd060108740 2024-05-09T02:27:56.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:56.121+0000 7fd0567fc700 1 -- 172.21.15.19:0/277841748 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+11928 (secure 0 0 0) 0x7fd0580aa0f0 con 0x7fd060108740 2024-05-09T02:27:56.124 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:27:56.124 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":62,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:27:21.082970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24457,24455,24473,24463,24484]},"id":1}]} 2024-05-09T02:27:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:56.124+0000 7fd0673fd700 1 -- 172.21.15.19:0/277841748 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fd04c0b29f0 msgr2=0x7fd04c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:56.124+0000 7fd0673fd700 1 --2- 172.21.15.19:0/277841748 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fd04c0b29f0 0x7fd04c0b4f00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd05c011490 tx=0x7fd05c009260 comp rx=0 tx=0).stop 2024-05-09T02:27:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:56.124+0000 7fd0673fd700 1 -- 172.21.15.19:0/277841748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd060108740 msgr2=0x7fd06019c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:56.124+0000 7fd0673fd700 1 --2- 172.21.15.19:0/277841748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd060108740 0x7fd06019c750 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd058007ee0 tx=0x7fd05800d3b0 comp rx=0 tx=0).stop 2024-05-09T02:27:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:56.124+0000 7fd0673fd700 1 -- 172.21.15.19:0/277841748 shutdown_connections 2024-05-09T02:27:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:56.124+0000 7fd0673fd700 1 --2- 172.21.15.19:0/277841748 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fd04c0b29f0 0x7fd04c0b4f00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:56.124+0000 7fd0673fd700 1 --2- 172.21.15.19:0/277841748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd060108740 0x7fd06019c750 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:56.124+0000 7fd0673fd700 1 --2- 172.21.15.19:0/277841748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0601034c0 0x7fd06019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:56.124+0000 7fd0673fd700 1 --2- 172.21.15.19:0/277841748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd060107df0 0x7fd06019c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:56.124+0000 7fd0673fd700 1 -- 172.21.15.19:0/277841748 >> 172.21.15.19:0/277841748 conn(0x7fd0600fedc0 msgr2=0x7fd0601075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:56.124+0000 7fd0673fd700 1 -- 172.21.15.19:0/277841748 shutdown_connections 2024-05-09T02:27:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:56.124+0000 7fd0673fd700 1 -- 172.21.15.19:0/277841748 wait complete. 2024-05-09T02:27:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 62 2024-05-09T02:27:56.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:56 smithi019 ceph-mon[30151]: pgmap v522: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-05-09T02:27:56.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:56 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:27:56.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:56 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/277841748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:27:56.758 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T02:27:56.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:56 smithi059 ceph-mon[34781]: pgmap v522: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-05-09T02:27:56.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:56 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:27:56.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:56 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/277841748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:27:57.021 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:27:57.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:56 smithi196 ceph-mon[34718]: pgmap v522: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-05-09T02:27:57.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:56 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T02:27:57.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:56 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/277841748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:27:57.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:57 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:27:57.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:57 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:27:57.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:57 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:27:57.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:57 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:27:57.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:57 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:27:57.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:57 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:27:58.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:57 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T02:27:58.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:57 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T02:27:58.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:57 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' 2024-05-09T02:27:58.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.753+0000 7f4541adf700 1 -- 172.21.15.19:0/1835517495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c108760 msgr2=0x7f453c1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:58.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.753+0000 7f4541adf700 1 --2- 172.21.15.19:0/1835517495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c108760 0x7f453c1100c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f453400b0d0 tx=0x7f453400b3e0 comp rx=0 tx=0).stop 2024-05-09T02:27:58.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.753+0000 7f4541adf700 1 -- 172.21.15.19:0/1835517495 shutdown_connections 2024-05-09T02:27:58.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.753+0000 7f4541adf700 1 --2- 172.21.15.19:0/1835517495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c108760 0x7f453c1100c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:58.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.753+0000 7f4541adf700 1 --2- 172.21.15.19:0/1835517495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f453c1034e0 0x7f453c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:58.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.753+0000 7f4541adf700 1 --2- 172.21.15.19:0/1835517495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f453c107e10 0x7f453c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:58.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.753+0000 7f4541adf700 1 -- 172.21.15.19:0/1835517495 >> 172.21.15.19:0/1835517495 conn(0x7f453c0fef30 msgr2=0x7f453c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:58.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.753+0000 7f4541adf700 1 -- 172.21.15.19:0/1835517495 shutdown_connections 2024-05-09T02:27:58.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.753+0000 7f4541adf700 1 -- 172.21.15.19:0/1835517495 wait complete. 2024-05-09T02:27:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.754+0000 7f4541adf700 1 Processor -- start 2024-05-09T02:27:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.754+0000 7f4541adf700 1 -- start start 2024-05-09T02:27:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f4541adf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f453c1034e0 0x7f453c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f4541adf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c107e10 0x7f453c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f4541adf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f453c108760 0x7f453c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f4541adf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f453c113390 con 0x7f453c107e10 2024-05-09T02:27:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f4541adf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f453c113210 con 0x7f453c108760 2024-05-09T02:27:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f4541adf700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f453c113510 con 0x7f453c1034e0 2024-05-09T02:27:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f453affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c107e10 0x7f453c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f453affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c107e10 0x7f453c19b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59066/0 (socket says 172.21.15.19:59066) 2024-05-09T02:27:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f453affd700 1 -- 172.21.15.19:0/2799211595 learned_addr learned my addr 172.21.15.19:0/2799211595 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:27:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f453b7fe700 1 --2- 172.21.15.19:0/2799211595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f453c1034e0 0x7f453c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f453bfff700 1 --2- 172.21.15.19:0/2799211595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f453c108760 0x7f453c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f453affd700 1 -- 172.21.15.19:0/2799211595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f453c1034e0 msgr2=0x7f453c19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f453affd700 1 --2- 172.21.15.19:0/2799211595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f453c1034e0 0x7f453c19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f453affd700 1 -- 172.21.15.19:0/2799211595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f453c108760 msgr2=0x7f453c19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f453affd700 1 --2- 172.21.15.19:0/2799211595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f453c108760 0x7f453c19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.755+0000 7f453affd700 1 -- 172.21.15.19:0/2799211595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453c19fca0 con 0x7f453c107e10 2024-05-09T02:27:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.756+0000 7f453affd700 1 --2- 172.21.15.19:0/2799211595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c107e10 0x7f453c19b370 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f453000e970 tx=0x7f453000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.756+0000 7f4538ff9700 1 -- 172.21.15.19:0/2799211595 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530004a30 con 0x7f453c107e10 2024-05-09T02:27:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.756+0000 7f4538ff9700 1 -- 172.21.15.19:0/2799211595 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f453000cc90 con 0x7f453c107e10 2024-05-09T02:27:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.756+0000 7f4541adf700 1 -- 172.21.15.19:0/2799211595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453c19ff90 con 0x7f453c107e10 2024-05-09T02:27:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.756+0000 7f4538ff9700 1 -- 172.21.15.19:0/2799211595 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530005610 con 0x7f453c107e10 2024-05-09T02:27:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.756+0000 7f4541adf700 1 -- 172.21.15.19:0/2799211595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f453c1a78b0 con 0x7f453c107e10 2024-05-09T02:27:58.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.757+0000 7f4541adf700 1 -- 172.21.15.19:0/2799211595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f453c0512c0 con 0x7f453c107e10 2024-05-09T02:27:58.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.759+0000 7f4538ff9700 1 -- 172.21.15.19:0/2799211595 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f453000c7e0 con 0x7f453c107e10 2024-05-09T02:27:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.759+0000 7f4538ff9700 1 --2- 172.21.15.19:0/2799211595 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45240b2a70 0x7f45240b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:27:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.759+0000 7f453b7fe700 1 --2- 172.21.15.19:0/2799211595 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45240b2a70 0x7f45240b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:27:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.759+0000 7f4538ff9700 1 -- 172.21.15.19:0/2799211595 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f45300e2ef0 con 0x7f453c107e10 2024-05-09T02:27:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.760+0000 7f453b7fe700 1 --2- 172.21.15.19:0/2799211595 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45240b2a70 0x7f45240b4f80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f452c007be0 tx=0x7f452c0073d0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:27:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:58.763+0000 7f4538ff9700 1 -- 172.21.15.19:0/2799211595 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45300aa840 con 0x7f453c107e10 2024-05-09T02:27:58.821 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:58 smithi019 ceph-mon[30151]: pgmap v523: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-05-09T02:27:58.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:58 smithi059 ceph-mon[34781]: pgmap v523: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-05-09T02:27:59.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:59.030+0000 7f4541adf700 1 -- 172.21.15.19:0/2799211595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f453c19c130 con 0x7f453c107e10 2024-05-09T02:27:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:59.030+0000 7f4538ff9700 1 -- 172.21.15.19:0/2799211595 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+11928 (secure 0 0 0) 0x7f45300aa140 con 0x7f453c107e10 2024-05-09T02:27:59.033 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:27:59.033 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121765116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":4121765116},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121765116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2298435247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2298435247},{"type":"v1","addr":"172.21.15.19:6837","nonce":2298435247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":62,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T02:07:50.316306+0000","modified":"2024-05-09T02:27:21.082970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24473,"mds_2":24463,"mds_3":24457,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/810391856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":810391856},{"type":"v1","addr":"172.21.15.19:6839","nonce":810391856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/3162604494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3162604494},{"type":"v1","addr":"172.21.15.19:6841","nonce":3162604494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6833/2040289281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2040289281},{"type":"v1","addr":"172.21.15.59:6833","nonce":2040289281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/416853517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":416853517},{"type":"v1","addr":"172.21.15.196:6833","nonce":416853517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/978263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":978263217},{"type":"v1","addr":"172.21.15.59:6835","nonce":978263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/2268419175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2268419175},{"type":"v1","addr":"172.21.15.59:6837","nonce":2268419175}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.196:6835/442694937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":442694937},{"type":"v1","addr":"172.21.15.196:6835","nonce":442694937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.196:6837/1033177921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1033177921},{"type":"v1","addr":"172.21.15.196:6837","nonce":1033177921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.59:6839/1206568409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":1206568409},{"type":"v1","addr":"172.21.15.59:6839","nonce":1206568409}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/89436664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":89436664},{"type":"v1","addr":"172.21.15.196:6839","nonce":89436664}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24484,"qdb_cluster":[24457,24455,24473,24463,24484]},"id":1}]} 2024-05-09T02:27:59.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:59.033+0000 7f4541adf700 1 -- 172.21.15.19:0/2799211595 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45240b2a70 msgr2=0x7f45240b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:59.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:59.033+0000 7f4541adf700 1 --2- 172.21.15.19:0/2799211595 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45240b2a70 0x7f45240b4f80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f452c007be0 tx=0x7f452c0073d0 comp rx=0 tx=0).stop 2024-05-09T02:27:59.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:59.033+0000 7f4541adf700 1 -- 172.21.15.19:0/2799211595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c107e10 msgr2=0x7f453c19b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:27:59.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:59.033+0000 7f4541adf700 1 --2- 172.21.15.19:0/2799211595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c107e10 0x7f453c19b370 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f453000e970 tx=0x7f453000ec80 comp rx=0 tx=0).stop 2024-05-09T02:27:59.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:59.033+0000 7f4541adf700 1 -- 172.21.15.19:0/2799211595 shutdown_connections 2024-05-09T02:27:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:59.033+0000 7f4541adf700 1 --2- 172.21.15.19:0/2799211595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f453c108760 0x7f453c19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:59.033+0000 7f4541adf700 1 --2- 172.21.15.19:0/2799211595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f453c1034e0 0x7f453c19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:59.033+0000 7f4541adf700 1 --2- 172.21.15.19:0/2799211595 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f45240b2a70 0x7f45240b4f80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:59.033+0000 7f4541adf700 1 --2- 172.21.15.19:0/2799211595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c107e10 0x7f453c19b370 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:27:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:59.033+0000 7f4541adf700 1 -- 172.21.15.19:0/2799211595 >> 172.21.15.19:0/2799211595 conn(0x7f453c0fef30 msgr2=0x7f453c06c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:27:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:59.033+0000 7f4541adf700 1 -- 172.21.15.19:0/2799211595 shutdown_connections 2024-05-09T02:27:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:27:59.033+0000 7f4541adf700 1 -- 172.21.15.19:0/2799211595 wait complete. 2024-05-09T02:27:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 62 2024-05-09T02:27:59.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:58 smithi196 ceph-mon[34718]: pgmap v523: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-05-09T02:27:59.748 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T02:27:59.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:27:59 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/2799211595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:28:00.008 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:28:00.058 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:27:59 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/2799211595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:28:00.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:27:59 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/2799211595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T02:28:00.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:00 smithi019 ceph-mon[30151]: pgmap v524: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-05-09T02:28:00.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:00 smithi059 ceph-mon[34781]: pgmap v524: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-05-09T02:28:01.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:00 smithi196 ceph-mon[34718]: pgmap v524: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-05-09T02:28:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.706+0000 7fead8248700 1 -- 172.21.15.19:0/2682857422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead0104610 msgr2=0x7fead0104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.706+0000 7fead8248700 1 --2- 172.21.15.19:0/2682857422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead0104610 0x7fead0104a20 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7feacc00b0d0 tx=0x7feacc00b3e0 comp rx=0 tx=0).stop 2024-05-09T02:28:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.706+0000 7fead8248700 1 -- 172.21.15.19:0/2682857422 shutdown_connections 2024-05-09T02:28:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.706+0000 7fead8248700 1 --2- 172.21.15.19:0/2682857422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fead0109890 0x7fead01111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.706+0000 7fead8248700 1 --2- 172.21.15.19:0/2682857422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead0104610 0x7fead0104a20 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.706+0000 7fead8248700 1 --2- 172.21.15.19:0/2682857422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead0108f40 0x7fead0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.706+0000 7fead8248700 1 -- 172.21.15.19:0/2682857422 >> 172.21.15.19:0/2682857422 conn(0x7fead007af70 msgr2=0x7fead007b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:28:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.706+0000 7fead8248700 1 -- 172.21.15.19:0/2682857422 shutdown_connections 2024-05-09T02:28:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.706+0000 7fead8248700 1 -- 172.21.15.19:0/2682857422 wait complete. 2024-05-09T02:28:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.706+0000 7fead8248700 1 Processor -- start 2024-05-09T02:28:01.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.706+0000 7fead8248700 1 -- start start 2024-05-09T02:28:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead8248700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead0104610 0x7fead019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead8248700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead0108f40 0x7fead019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead8248700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fead0109890 0x7fead019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead8248700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fead0114540 con 0x7fead0104610 2024-05-09T02:28:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead8248700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fead01143c0 con 0x7fead0109890 2024-05-09T02:28:01.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead8248700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fead01146c0 con 0x7fead0108f40 2024-05-09T02:28:01.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead5fe4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead0104610 0x7fead019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:01.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead5fe4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead0104610 0x7fead019af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59088/0 (socket says 172.21.15.19:59088) 2024-05-09T02:28:01.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead5fe4700 1 -- 172.21.15.19:0/472477109 learned_addr learned my addr 172.21.15.19:0/472477109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:28:01.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead67e5700 1 --2- 172.21.15.19:0/472477109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fead0109890 0x7fead019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:01.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead57e3700 1 --2- 172.21.15.19:0/472477109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead0108f40 0x7fead019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:01.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead5fe4700 1 -- 172.21.15.19:0/472477109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead0108f40 msgr2=0x7fead019b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:01.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead5fe4700 1 --2- 172.21.15.19:0/472477109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead0108f40 0x7fead019b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:01.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead5fe4700 1 -- 172.21.15.19:0/472477109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fead0109890 msgr2=0x7fead019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:01.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead5fe4700 1 --2- 172.21.15.19:0/472477109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fead0109890 0x7fead019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:01.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.707+0000 7fead5fe4700 1 -- 172.21.15.19:0/472477109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead019fd30 con 0x7fead0104610 2024-05-09T02:28:01.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.708+0000 7fead5fe4700 1 --2- 172.21.15.19:0/472477109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead0104610 0x7fead019af10 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7feacc007a20 tx=0x7feacc007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:28:01.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.708+0000 7feac6ffd700 1 -- 172.21.15.19:0/472477109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc01f070 con 0x7fead0104610 2024-05-09T02:28:01.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.708+0000 7fead8248700 1 -- 172.21.15.19:0/472477109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead019ffc0 con 0x7fead0104610 2024-05-09T02:28:01.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.708+0000 7feac6ffd700 1 -- 172.21.15.19:0/472477109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feacc003c00 con 0x7fead0104610 2024-05-09T02:28:01.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.708+0000 7feac6ffd700 1 -- 172.21.15.19:0/472477109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc01b6a0 con 0x7fead0104610 2024-05-09T02:28:01.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.708+0000 7fead8248700 1 -- 172.21.15.19:0/472477109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead01a04a0 con 0x7fead0104610 2024-05-09T02:28:01.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.709+0000 7fead8248700 1 -- 172.21.15.19:0/472477109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead00512c0 con 0x7fead0104610 2024-05-09T02:28:01.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.711+0000 7feac6ffd700 1 -- 172.21.15.19:0/472477109 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7feacc022070 con 0x7fead0104610 2024-05-09T02:28:01.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.712+0000 7feac6ffd700 1 --2- 172.21.15.19:0/472477109 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7feabc0b29f0 0x7feabc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:01.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.712+0000 7fead57e3700 1 --2- 172.21.15.19:0/472477109 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7feabc0b29f0 0x7feabc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:01.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.712+0000 7feac6ffd700 1 -- 172.21.15.19:0/472477109 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7feacc00f020 con 0x7fead0104610 2024-05-09T02:28:01.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.713+0000 7fead57e3700 1 --2- 172.21.15.19:0/472477109 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7feabc0b29f0 0x7feabc0b4f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fead0100f10 tx=0x7feac000a380 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:28:01.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.716+0000 7feac6ffd700 1 -- 172.21.15.19:0/472477109 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feacc029020 con 0x7fead0104610 2024-05-09T02:28:01.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.934+0000 7fead8248700 1 -- 172.21.15.19:0/472477109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fead002e5f0 con 0x7fead0104610 2024-05-09T02:28:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.936+0000 7feac6ffd700 1 -- 172.21.15.19:0/472477109 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+23166 (secure 0 0 0) 0x7feacc0ab6f0 con 0x7fead0104610 2024-05-09T02:28:01.939 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-09T02:28:01.939 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":102,"fsid":"a0611f3e-0da7-11ef-bc97-c7b262605968","created":"2024-05-09T01:59:27.591343+0000","modified":"2024-05-09T02:27:14.615737+0000","last_up_change":"2024-05-09T02:06:45.565065+0000","last_in_change":"2024-05-09T02:06:24.967186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T02:02:50.018959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T02:07:43.283868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T02:07:47.110614+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"996c6a3d-3e3a-451c-8789-36006e0b3dce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6803","nonce":3002373576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6805","nonce":3002373576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6809","nonce":3002373576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3002373576},{"type":"v1","addr":"172.21.15.19:6807","nonce":3002373576}]},"public_addr":"172.21.15.19:6803/3002373576","cluster_addr":"172.21.15.19:6805/3002373576","heartbeat_back_addr":"172.21.15.19:6809/3002373576","heartbeat_front_addr":"172.21.15.19:6807/3002373576","state":["exists","up"]},{"osd":1,"uuid":"1ac68d09-1198-4e81-86c7-e37f7c41aa5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6801","nonce":827458353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6803","nonce":827458353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6807","nonce":827458353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":827458353},{"type":"v1","addr":"172.21.15.59:6805","nonce":827458353}]},"public_addr":"172.21.15.59:6801/827458353","cluster_addr":"172.21.15.59:6803/827458353","heartbeat_back_addr":"172.21.15.59:6807/827458353","heartbeat_front_addr":"172.21.15.59:6805/827458353","state":["exists","up"]},{"osd":2,"uuid":"818a0de1-04e6-471d-83df-a68833011b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6801","nonce":1659504639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6803","nonce":1659504639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6807","nonce":1659504639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1659504639},{"type":"v1","addr":"172.21.15.196:6805","nonce":1659504639}]},"public_addr":"172.21.15.196:6801/1659504639","cluster_addr":"172.21.15.196:6803/1659504639","heartbeat_back_addr":"172.21.15.196:6807/1659504639","heartbeat_front_addr":"172.21.15.196:6805/1659504639","state":["exists","up"]},{"osd":3,"uuid":"7b204854-86b8-454d-aee0-8bb5775ce828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6811","nonce":2707671599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6813","nonce":2707671599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6817","nonce":2707671599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2707671599},{"type":"v1","addr":"172.21.15.19:6815","nonce":2707671599}]},"public_addr":"172.21.15.19:6811/2707671599","cluster_addr":"172.21.15.19:6813/2707671599","heartbeat_back_addr":"172.21.15.19:6817/2707671599","heartbeat_front_addr":"172.21.15.19:6815/2707671599","state":["exists","up"]},{"osd":4,"uuid":"091667f1-e887-4a94-8723-6f85b676fef8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6809","nonce":223037330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6811","nonce":223037330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6815","nonce":223037330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":223037330},{"type":"v1","addr":"172.21.15.59:6813","nonce":223037330}]},"public_addr":"172.21.15.59:6809/223037330","cluster_addr":"172.21.15.59:6811/223037330","heartbeat_back_addr":"172.21.15.59:6815/223037330","heartbeat_front_addr":"172.21.15.59:6813/223037330","state":["exists","up"]},{"osd":5,"uuid":"2d787f0c-bbfb-4ef0-a9fe-2e1b22ae08ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6809","nonce":2516161898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6811","nonce":2516161898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6815","nonce":2516161898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2516161898},{"type":"v1","addr":"172.21.15.196:6813","nonce":2516161898}]},"public_addr":"172.21.15.196:6809/2516161898","cluster_addr":"172.21.15.196:6811/2516161898","heartbeat_back_addr":"172.21.15.196:6815/2516161898","heartbeat_front_addr":"172.21.15.196:6813/2516161898","state":["exists","up"]},{"osd":6,"uuid":"4e352851-1279-475f-ba8b-eaf9ca73df74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6819","nonce":2150451637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6821","nonce":2150451637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6825","nonce":2150451637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2150451637},{"type":"v1","addr":"172.21.15.19:6823","nonce":2150451637}]},"public_addr":"172.21.15.19:6819/2150451637","cluster_addr":"172.21.15.19:6821/2150451637","heartbeat_back_addr":"172.21.15.19:6825/2150451637","heartbeat_front_addr":"172.21.15.19:6823/2150451637","state":["exists","up"]},{"osd":7,"uuid":"83f53f59-9a60-4d87-9a8a-67e5c6f57004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6817","nonce":967902053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6819","nonce":967902053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6823","nonce":967902053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":967902053},{"type":"v1","addr":"172.21.15.59:6821","nonce":967902053}]},"public_addr":"172.21.15.59:6817/967902053","cluster_addr":"172.21.15.59:6819/967902053","heartbeat_back_addr":"172.21.15.59:6823/967902053","heartbeat_front_addr":"172.21.15.59:6821/967902053","state":["exists","up"]},{"osd":8,"uuid":"a8f1bfb5-1cc8-4a65-8396-239bd781e1e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6817","nonce":2946064259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6819","nonce":2946064259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6823","nonce":2946064259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2946064259},{"type":"v1","addr":"172.21.15.196:6821","nonce":2946064259}]},"public_addr":"172.21.15.196:6817/2946064259","cluster_addr":"172.21.15.196:6819/2946064259","heartbeat_back_addr":"172.21.15.196:6823/2946064259","heartbeat_front_addr":"172.21.15.196:6821/2946064259","state":["exists","up"]},{"osd":9,"uuid":"73c58dc3-fcb0-4440-92b3-bf41a2e58fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6827","nonce":1735017886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6829","nonce":1735017886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6833","nonce":1735017886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":1735017886},{"type":"v1","addr":"172.21.15.19:6831","nonce":1735017886}]},"public_addr":"172.21.15.19:6827/1735017886","cluster_addr":"172.21.15.19:6829/1735017886","heartbeat_back_addr":"172.21.15.19:6833/1735017886","heartbeat_front_addr":"172.21.15.19:6831/1735017886","state":["exists","up"]},{"osd":10,"uuid":"8325282f-7325-4880-ab7d-b972717be9dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6825","nonce":2932899995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6827","nonce":2932899995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6831","nonce":2932899995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2932899995},{"type":"v1","addr":"172.21.15.59:6829","nonce":2932899995}]},"public_addr":"172.21.15.59:6825/2932899995","cluster_addr":"172.21.15.59:6827/2932899995","heartbeat_back_addr":"172.21.15.59:6831/2932899995","heartbeat_front_addr":"172.21.15.59:6829/2932899995","state":["exists","up"]},{"osd":11,"uuid":"8b19228c-b286-43c8-a408-169782f13075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6825","nonce":2608023001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6827","nonce":2608023001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6831","nonce":2608023001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2608023001},{"type":"v1","addr":"172.21.15.196:6829","nonce":2608023001}]},"public_addr":"172.21.15.196:6825/2608023001","cluster_addr":"172.21.15.196:6827/2608023001","heartbeat_back_addr":"172.21.15.196:6831/2608023001","heartbeat_front_addr":"172.21.15.196:6829/2608023001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:01:59.457748+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:23.640666+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:02:47.768243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:13.016746+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:03:39.060898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:04.508313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:31.366217+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:04:57.579758+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:23.177396+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:05:50.966399+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:16.838666+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T02:06:43.959387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3660764754":"2024-05-10T02:11:17.084815+0000","172.21.15.19:0/3573608809":"2024-05-10T02:11:17.084815+0000","172.21.15.19:0/3291838961":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1024133854":"2024-05-10T02:00:23.920230+0000","172.21.15.19:0/1176455101":"2024-05-10T02:00:23.920230+0000","172.21.15.19:6801/2101052000":"2024-05-10T02:00:23.920230+0000","172.21.15.19:6801/972543870":"2024-05-10T02:11:17.084815+0000","172.21.15.19:0/4018987404":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/3079777324":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/972543870":"2024-05-10T02:11:17.084815+0000","172.21.15.19:6800/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/557070897":"2024-05-10T02:11:17.084815+0000","172.21.15.19:0/758334732":"2024-05-10T02:11:17.084815+0000","172.21.15.19:6801/1829553843":"2024-05-10T01:59:56.604998+0000","172.21.15.19:0/179703528":"2024-05-10T01:59:56.604998+0000","172.21.15.19:6800/2101052000":"2024-05-10T02:00:23.920230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T02:28:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.938+0000 7fead8248700 1 -- 172.21.15.19:0/472477109 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7feabc0b29f0 msgr2=0x7feabc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.938+0000 7fead8248700 1 --2- 172.21.15.19:0/472477109 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7feabc0b29f0 0x7feabc0b4f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fead0100f10 tx=0x7feac000a380 comp rx=0 tx=0).stop 2024-05-09T02:28:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.938+0000 7fead8248700 1 -- 172.21.15.19:0/472477109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead0104610 msgr2=0x7fead019af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.938+0000 7fead8248700 1 --2- 172.21.15.19:0/472477109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead0104610 0x7fead019af10 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7feacc007a20 tx=0x7feacc007a50 comp rx=0 tx=0).stop 2024-05-09T02:28:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.938+0000 7fead8248700 1 -- 172.21.15.19:0/472477109 shutdown_connections 2024-05-09T02:28:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.938+0000 7fead8248700 1 --2- 172.21.15.19:0/472477109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fead0109890 0x7fead019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.938+0000 7fead8248700 1 --2- 172.21.15.19:0/472477109 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7feabc0b29f0 0x7feabc0b4f00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.938+0000 7fead8248700 1 --2- 172.21.15.19:0/472477109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fead0104610 0x7fead019af10 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.938+0000 7fead8248700 1 --2- 172.21.15.19:0/472477109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead0108f40 0x7fead019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.938+0000 7fead8248700 1 -- 172.21.15.19:0/472477109 >> 172.21.15.19:0/472477109 conn(0x7fead007af70 msgr2=0x7fead01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:28:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.938+0000 7fead8248700 1 -- 172.21.15.19:0/472477109 shutdown_connections 2024-05-09T02:28:01.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:01.938+0000 7fead8248700 1 -- 172.21.15.19:0/472477109 wait complete. 2024-05-09T02:28:02.629 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-09T02:28:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:28:02.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:02 smithi059 ceph-mon[34781]: pgmap v525: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-05-09T02:28:02.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:02 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/472477109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:28:02.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:02 smithi019 ceph-mon[30151]: pgmap v525: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-05-09T02:28:02.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:02 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/472477109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:28:03.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:02 smithi196 ceph-mon[34718]: pgmap v525: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-05-09T02:28:03.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:02 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/472477109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T02:28:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.808+0000 7f2b87749700 1 -- 172.21.15.19:0/1955760763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b801030e0 msgr2=0x7f2b80111170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.808+0000 7f2b87749700 1 --2- 172.21.15.19:0/1955760763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b801030e0 0x7f2b80111170 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800b0a0 tx=0x7f2b7800b3b0 comp rx=0 tx=0).stop 2024-05-09T02:28:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.808+0000 7f2b87749700 1 -- 172.21.15.19:0/1955760763 shutdown_connections 2024-05-09T02:28:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.808+0000 7f2b87749700 1 --2- 172.21.15.19:0/1955760763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b801030e0 0x7f2b80111170 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.808+0000 7f2b87749700 1 --2- 172.21.15.19:0/1955760763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b80109300 0x7f2b80109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.808+0000 7f2b87749700 1 --2- 172.21.15.19:0/1955760763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b80102790 0x7f2b80102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.808+0000 7f2b87749700 1 -- 172.21.15.19:0/1955760763 >> 172.21.15.19:0/1955760763 conn(0x7f2b800fe4e0 msgr2=0x7f2b80100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:28:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.808+0000 7f2b87749700 1 -- 172.21.15.19:0/1955760763 shutdown_connections 2024-05-09T02:28:04.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.808+0000 7f2b87749700 1 -- 172.21.15.19:0/1955760763 wait complete. 2024-05-09T02:28:04.812 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:04 smithi019 ceph-mon[30151]: pgmap v526: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-05-09T02:28:04.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.809+0000 7f2b87749700 1 Processor -- start 2024-05-09T02:28:04.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.809+0000 7f2b87749700 1 -- start start 2024-05-09T02:28:04.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.809+0000 7f2b87749700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b80102790 0x7f2b801a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.809+0000 7f2b87749700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b801030e0 0x7f2b801a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.809+0000 7f2b87749700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b80109300 0x7f2b801ad300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.809+0000 7f2b87749700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b801145a0 con 0x7f2b80109300 2024-05-09T02:28:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.809+0000 7f2b87749700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b80114420 con 0x7f2b80102790 2024-05-09T02:28:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.809+0000 7f2b87749700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b80114720 con 0x7f2b801030e0 2024-05-09T02:28:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.809+0000 7f2b85ce6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b80109300 0x7f2b801ad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.809+0000 7f2b85ce6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b80109300 0x7f2b801ad300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59096/0 (socket says 172.21.15.19:59096) 2024-05-09T02:28:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.809+0000 7f2b84ce4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b801030e0 0x7f2b801a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.809+0000 7f2b85ce6700 1 -- 172.21.15.19:0/3522194950 learned_addr learned my addr 172.21.15.19:0/3522194950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:28:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.809+0000 7f2b854e5700 1 --2- 172.21.15.19:0/3522194950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b80102790 0x7f2b801a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.810+0000 7f2b85ce6700 1 -- 172.21.15.19:0/3522194950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b801030e0 msgr2=0x7f2b801a27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.810+0000 7f2b85ce6700 1 --2- 172.21.15.19:0/3522194950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b801030e0 0x7f2b801a27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.810+0000 7f2b85ce6700 1 -- 172.21.15.19:0/3522194950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b80102790 msgr2=0x7f2b801a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.810+0000 7f2b85ce6700 1 --2- 172.21.15.19:0/3522194950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b80102790 0x7f2b801a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.810+0000 7f2b85ce6700 1 -- 172.21.15.19:0/3522194950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b801a2ff0 con 0x7f2b80109300 2024-05-09T02:28:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.810+0000 7f2b85ce6700 1 --2- 172.21.15.19:0/3522194950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b80109300 0x7f2b801ad300 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800b830 tx=0x7f2b78004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:28:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.810+0000 7f2b727fc700 1 -- 172.21.15.19:0/3522194950 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78021070 con 0x7f2b80109300 2024-05-09T02:28:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.810+0000 7f2b727fc700 1 -- 172.21.15.19:0/3522194950 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b78007e30 con 0x7f2b80109300 2024-05-09T02:28:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.810+0000 7f2b727fc700 1 -- 172.21.15.19:0/3522194950 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78004700 con 0x7f2b80109300 2024-05-09T02:28:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.810+0000 7f2b87749700 1 -- 172.21.15.19:0/3522194950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b801ad840 con 0x7f2b80109300 2024-05-09T02:28:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.810+0000 7f2b87749700 1 -- 172.21.15.19:0/3522194950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b801add00 con 0x7f2b80109300 2024-05-09T02:28:04.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.811+0000 7f2b87749700 1 -- 172.21.15.19:0/3522194950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b800512c0 con 0x7f2b80109300 2024-05-09T02:28:04.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.813+0000 7f2b727fc700 1 -- 172.21.15.19:0/3522194950 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2b78003d70 con 0x7f2b80109300 2024-05-09T02:28:04.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.814+0000 7f2b727fc700 1 --2- 172.21.15.19:0/3522194950 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2b6c0b2a70 0x7f2b6c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:04.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.814+0000 7f2b854e5700 1 --2- 172.21.15.19:0/3522194950 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2b6c0b2a70 0x7f2b6c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:04.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.814+0000 7f2b727fc700 1 -- 172.21.15.19:0/3522194950 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f2b7800f020 con 0x7f2b80109300 2024-05-09T02:28:04.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.814+0000 7f2b854e5700 1 --2- 172.21.15.19:0/3522194950 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2b6c0b2a70 0x7f2b6c0b4f80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2b8006ca90 tx=0x7f2b7400a3b0 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:28:04.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:04.818+0000 7f2b727fc700 1 -- 172.21.15.19:0/3522194950 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2b7802a020 con 0x7f2b80109300 2024-05-09T02:28:04.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:04 smithi059 ceph-mon[34781]: pgmap v526: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-05-09T02:28:05.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:04 smithi196 ceph-mon[34718]: pgmap v526: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-05-09T02:28:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:05.081+0000 7f2b87749700 1 -- 172.21.15.19:0/3522194950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2b8002e5f0 con 0x7f2b80109300 2024-05-09T02:28:05.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:05.083+0000 7f2b727fc700 1 -- 172.21.15.19:0/3522194950 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) ==== 161+0+0 (secure 0 0 0) 0x7f2b780ab910 con 0x7f2b80109300 2024-05-09T02:28:05.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:05.087+0000 7f2b87749700 1 -- 172.21.15.19:0/3522194950 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2b6c0b2a70 msgr2=0x7f2b6c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:05.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:05.087+0000 7f2b87749700 1 --2- 172.21.15.19:0/3522194950 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2b6c0b2a70 0x7f2b6c0b4f80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2b8006ca90 tx=0x7f2b7400a3b0 comp rx=0 tx=0).stop 2024-05-09T02:28:05.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:05.087+0000 7f2b87749700 1 -- 172.21.15.19:0/3522194950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b80109300 msgr2=0x7f2b801ad300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:05.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:05.087+0000 7f2b87749700 1 --2- 172.21.15.19:0/3522194950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b80109300 0x7f2b801ad300 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800b830 tx=0x7f2b78004ce0 comp rx=0 tx=0).stop 2024-05-09T02:28:05.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:05.087+0000 7f2b87749700 1 -- 172.21.15.19:0/3522194950 shutdown_connections 2024-05-09T02:28:05.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:05.087+0000 7f2b87749700 1 --2- 172.21.15.19:0/3522194950 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2b6c0b2a70 0x7f2b6c0b4f80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:05.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:05.087+0000 7f2b87749700 1 --2- 172.21.15.19:0/3522194950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b80109300 0x7f2b801ad300 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:05.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:05.087+0000 7f2b87749700 1 --2- 172.21.15.19:0/3522194950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b80102790 0x7f2b801a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:05.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:05.087+0000 7f2b87749700 1 --2- 172.21.15.19:0/3522194950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b801030e0 0x7f2b801a27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:05.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:05.087+0000 7f2b87749700 1 -- 172.21.15.19:0/3522194950 >> 172.21.15.19:0/3522194950 conn(0x7f2b800fe4e0 msgr2=0x7f2b80104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:28:05.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:05.087+0000 7f2b87749700 1 -- 172.21.15.19:0/3522194950 shutdown_connections 2024-05-09T02:28:05.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:05.087+0000 7f2b87749700 1 -- 172.21.15.19:0/3522194950 wait complete. 2024-05-09T02:28:05.091 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T02:28:05.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:05 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a[30147]: 2024-05-09T02:28:05.082+0000 7efd76ead700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T02:28:05.423 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:28:05 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-a[70699]: ignoring --setuser ceph since I am not root 2024-05-09T02:28:05.423 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:28:05 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-a[70699]: ignoring --setgroup ceph since I am not root 2024-05-09T02:28:05.423 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:28:05 smithi019 ceph-mds[70703]: -- 172.21.15.19:0/1244769450 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56333fdf5a40 con 0x56333fe3da80 2024-05-09T02:28:05.423 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:28:05 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-a[70699]: starting mds.a at 2024-05-09T02:28:05.423 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:28:05 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-g[70241]: ignoring --setuser ceph since I am not root 2024-05-09T02:28:05.423 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:28:05 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-g[70241]: ignoring --setgroup ceph since I am not root 2024-05-09T02:28:05.423 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:28:05 smithi019 ceph-mds[70255]: -- 172.21.15.19:0/65064139 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56417e3cda40 con 0x56417e415a80 2024-05-09T02:28:05.423 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:28:05 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-g[70241]: starting mds.g at 2024-05-09T02:28:05.466 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:28:05 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-b[64759]: ignoring --setuser ceph since I am not root 2024-05-09T02:28:05.466 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:28:05 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-b[64759]: ignoring --setgroup ceph since I am not root 2024-05-09T02:28:05.466 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:28:05 smithi059 ceph-mds[64763]: -- 172.21.15.59:0/530795443 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5562d1603a40 con 0x5562d164ba80 2024-05-09T02:28:05.467 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:28:05 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-b[64759]: starting mds.b at 2024-05-09T02:28:05.467 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:28:05 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-e[64149]: ignoring --setuser ceph since I am not root 2024-05-09T02:28:05.467 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:28:05 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-e[64149]: ignoring --setgroup ceph since I am not root 2024-05-09T02:28:05.467 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:28:05 smithi059 ceph-mds[64153]: -- 172.21.15.59:0/657741318 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555cb366ba40 con 0x555cb36b3180 2024-05-09T02:28:05.467 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:28:05 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-e[64149]: starting mds.e at 2024-05-09T02:28:05.467 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:28:05 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-k[63830]: ignoring --setuser ceph since I am not root 2024-05-09T02:28:05.467 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:28:05 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-k[63830]: ignoring --setgroup ceph since I am not root 2024-05-09T02:28:05.467 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:28:05 smithi059 ceph-mds[63835]: -- 172.21.15.59:0/2871603937 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a8e5481a40 con 0x55a8e54c9180 2024-05-09T02:28:05.468 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:28:05 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-k[63830]: starting mds.k at 2024-05-09T02:28:05.468 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:28:05 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-h[64452]: ignoring --setuser ceph since I am not root 2024-05-09T02:28:05.468 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:28:05 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-h[64452]: ignoring --setgroup ceph since I am not root 2024-05-09T02:28:05.468 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:28:05 smithi059 ceph-mds[64456]: -- 172.21.15.59:0/4031567645 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ffc00e9a40 con 0x55ffc0131180 2024-05-09T02:28:05.468 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:28:05 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-h[64452]: starting mds.h at 2024-05-09T02:28:05.545 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:28:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-c[65291]: ignoring --setuser ceph since I am not root 2024-05-09T02:28:05.545 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:28:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-c[65291]: ignoring --setgroup ceph since I am not root 2024-05-09T02:28:05.545 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:28:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-c[65291]: starting mds.c at 2024-05-09T02:28:05.545 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:28:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i[65599]: ignoring --setuser ceph since I am not root 2024-05-09T02:28:05.545 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:28:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i[65599]: ignoring --setgroup ceph since I am not root 2024-05-09T02:28:05.545 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:28:05 smithi196 ceph-mds[65603]: -- 172.21.15.196:0/1440786139 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565283855a40 con 0x56528389da80 2024-05-09T02:28:05.545 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:28:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i[65599]: starting mds.i at 2024-05-09T02:28:05.546 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:28:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-l[64679]: ignoring --setuser ceph since I am not root 2024-05-09T02:28:05.546 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:28:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-l[64679]: ignoring --setgroup ceph since I am not root 2024-05-09T02:28:05.546 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:28:05 smithi196 ceph-mds[64683]: -- 172.21.15.196:0/465956743 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564202d07a40 con 0x564202d4f180 2024-05-09T02:28:05.546 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:28:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-l[64679]: starting mds.l at 2024-05-09T02:28:05.546 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 02:28:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-f[64986]: ignoring --setuser ceph since I am not root 2024-05-09T02:28:05.546 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 02:28:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-f[64986]: ignoring --setgroup ceph since I am not root 2024-05-09T02:28:05.547 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 02:28:05 smithi196 ceph-mds[64990]: -- 172.21.15.196:0/3672977834 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d4bb87a40 con 0x556d4bbcf600 2024-05-09T02:28:05.547 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 02:28:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-f[64986]: starting mds.f at 2024-05-09T02:28:05.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:05 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3522194950' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T02:28:05.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:05 smithi019 ceph-mon[30151]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T02:28:05.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:05 smithi019 ceph-mon[30151]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T02:28:05.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:05 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/3522194950' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T02:28:05.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:05 smithi019 ceph-mon[30151]: osdmap e103: 12 total, 12 up, 12 in 2024-05-09T02:28:05.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:05 smithi019 ceph-mon[30151]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-09T02:28:05.762 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T02:28:05.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:05 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3522194950' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T02:28:05.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:05 smithi059 ceph-mon[34781]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T02:28:05.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:05 smithi059 ceph-mon[34781]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T02:28:05.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:05 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/3522194950' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T02:28:05.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:05 smithi059 ceph-mon[34781]: osdmap e103: 12 total, 12 up, 12 in 2024-05-09T02:28:05.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:05 smithi059 ceph-mon[34781]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-09T02:28:06.019 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:28:06.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:05 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3522194950' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T02:28:06.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:05 smithi196 ceph-mon[34718]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T02:28:06.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:05 smithi196 ceph-mon[34718]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T02:28:06.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:05 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/3522194950' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T02:28:06.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:05 smithi196 ceph-mon[34718]: osdmap e103: 12 total, 12 up, 12 in 2024-05-09T02:28:06.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:05 smithi196 ceph-mon[34718]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-09T02:28:06.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: pgmap v528: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 93 KiB/s wr, 33 op/s 2024-05-09T02:28:06.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.59:6833/1646250079,v1:172.21.15.59:6835/1646250079] up:boot 2024-05-09T02:28:06.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.59:6832/368545077,v1:172.21.15.59:6834/368545077] up:boot 2024-05-09T02:28:06.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.19:6838/2054781309,v1:172.21.15.19:6839/2054781309] up:boot 2024-05-09T02:28:06.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.19:6840/842119285,v1:172.21.15.19:6841/842119285] up:boot 2024-05-09T02:28:06.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.196:6832/3057395410,v1:172.21.15.196:6836/3057395410] up:boot 2024-05-09T02:28:06.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.196:6835/2455158599,v1:172.21.15.196:6838/2455158599] up:boot 2024-05-09T02:28:06.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.196:6834/1043591381,v1:172.21.15.196:6839/1043591381] up:boot 2024-05-09T02:28:06.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.196:6833/1361990060,v1:172.21.15.196:6837/1361990060] up:boot 2024-05-09T02:28:06.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.59:6836/4288879540,v1:172.21.15.59:6838/4288879540] up:boot 2024-05-09T02:28:06.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: mds.? [v2:172.21.15.59:6837/3970806341,v1:172.21.15.59:6839/3970806341] up:boot 2024-05-09T02:28:06.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-09T02:28:06.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T02:28:06.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T02:28:06.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T02:28:06.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T02:28:06.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T02:28:06.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T02:28:06.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T02:28:06.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T02:28:06.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T02:28:06.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:06 smithi019 ceph-mon[30151]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T02:28:06.868 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: pgmap v528: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 93 KiB/s wr, 33 op/s 2024-05-09T02:28:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.59:6833/1646250079,v1:172.21.15.59:6835/1646250079] up:boot 2024-05-09T02:28:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.59:6832/368545077,v1:172.21.15.59:6834/368545077] up:boot 2024-05-09T02:28:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.19:6838/2054781309,v1:172.21.15.19:6839/2054781309] up:boot 2024-05-09T02:28:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.19:6840/842119285,v1:172.21.15.19:6841/842119285] up:boot 2024-05-09T02:28:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.196:6832/3057395410,v1:172.21.15.196:6836/3057395410] up:boot 2024-05-09T02:28:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.196:6835/2455158599,v1:172.21.15.196:6838/2455158599] up:boot 2024-05-09T02:28:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.196:6834/1043591381,v1:172.21.15.196:6839/1043591381] up:boot 2024-05-09T02:28:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.196:6833/1361990060,v1:172.21.15.196:6837/1361990060] up:boot 2024-05-09T02:28:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.59:6836/4288879540,v1:172.21.15.59:6838/4288879540] up:boot 2024-05-09T02:28:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: mds.? [v2:172.21.15.59:6837/3970806341,v1:172.21.15.59:6839/3970806341] up:boot 2024-05-09T02:28:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-09T02:28:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T02:28:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T02:28:06.869 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T02:28:06.870 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T02:28:06.870 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T02:28:06.870 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T02:28:06.870 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T02:28:06.870 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T02:28:06.870 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T02:28:06.870 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:06 smithi059 ceph-mon[34781]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T02:28:07.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: pgmap v528: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 93 KiB/s wr, 33 op/s 2024-05-09T02:28:07.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.59:6833/1646250079,v1:172.21.15.59:6835/1646250079] up:boot 2024-05-09T02:28:07.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.59:6832/368545077,v1:172.21.15.59:6834/368545077] up:boot 2024-05-09T02:28:07.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.19:6838/2054781309,v1:172.21.15.19:6839/2054781309] up:boot 2024-05-09T02:28:07.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.19:6840/842119285,v1:172.21.15.19:6841/842119285] up:boot 2024-05-09T02:28:07.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.196:6832/3057395410,v1:172.21.15.196:6836/3057395410] up:boot 2024-05-09T02:28:07.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.196:6835/2455158599,v1:172.21.15.196:6838/2455158599] up:boot 2024-05-09T02:28:07.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.196:6834/1043591381,v1:172.21.15.196:6839/1043591381] up:boot 2024-05-09T02:28:07.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.196:6833/1361990060,v1:172.21.15.196:6837/1361990060] up:boot 2024-05-09T02:28:07.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.59:6836/4288879540,v1:172.21.15.59:6838/4288879540] up:boot 2024-05-09T02:28:07.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: mds.? [v2:172.21.15.59:6837/3970806341,v1:172.21.15.59:6839/3970806341] up:boot 2024-05-09T02:28:07.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-09T02:28:07.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T02:28:07.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T02:28:07.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T02:28:07.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T02:28:07.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T02:28:07.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T02:28:07.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T02:28:07.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T02:28:07.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T02:28:07.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:06 smithi196 ceph-mon[34718]: from='mgr.15222 172.21.15.19:0/2555012957' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T02:28:07.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.735+0000 7f2a5e437700 1 -- 172.21.15.19:0/1913648772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a58108f00 msgr2=0x7f2a58109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:07.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.735+0000 7f2a5e437700 1 --2- 172.21.15.19:0/1913648772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a58108f00 0x7f2a58109310 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2a400099c0 tx=0x7f2a40009cd0 comp rx=0 tx=0).stop 2024-05-09T02:28:07.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.735+0000 7f2a5e437700 1 -- 172.21.15.19:0/1913648772 shutdown_connections 2024-05-09T02:28:07.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.735+0000 7f2a5e437700 1 --2- 172.21.15.19:0/1913648772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a58109850 0x7f2a581111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:07.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.735+0000 7f2a5e437700 1 --2- 172.21.15.19:0/1913648772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a581045d0 0x7f2a581049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:07.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.735+0000 7f2a5e437700 1 --2- 172.21.15.19:0/1913648772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a58108f00 0x7f2a58109310 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:07.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.735+0000 7f2a5e437700 1 -- 172.21.15.19:0/1913648772 >> 172.21.15.19:0/1913648772 conn(0x7f2a580ffea0 msgr2=0x7f2a581022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:28:07.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.735+0000 7f2a5e437700 1 -- 172.21.15.19:0/1913648772 shutdown_connections 2024-05-09T02:28:07.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.735+0000 7f2a5e437700 1 -- 172.21.15.19:0/1913648772 wait complete. 2024-05-09T02:28:07.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.736+0000 7f2a5e437700 1 Processor -- start 2024-05-09T02:28:07.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.736+0000 7f2a5e437700 1 -- start start 2024-05-09T02:28:07.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.736+0000 7f2a5e437700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a581045d0 0x7f2a5819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:07.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.736+0000 7f2a5e437700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a58108f00 0x7f2a5819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:07.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.736+0000 7f2a5e437700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a58109850 0x7f2a5819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:07.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.736+0000 7f2a5e437700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a58114480 con 0x7f2a58108f00 2024-05-09T02:28:07.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.736+0000 7f2a5e437700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2a58114300 con 0x7f2a58109850 2024-05-09T02:28:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.736+0000 7f2a5e437700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a58114600 con 0x7f2a581045d0 2024-05-09T02:28:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.736+0000 7f2a4ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a58108f00 0x7f2a5819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.736+0000 7f2a4ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a58108f00 0x7f2a5819b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37578/0 (socket says 172.21.15.19:37578) 2024-05-09T02:28:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.736+0000 7f2a4ffff700 1 -- 172.21.15.19:0/910488047 learned_addr learned my addr 172.21.15.19:0/910488047 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:28:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.736+0000 7f2a57fff700 1 --2- 172.21.15.19:0/910488047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a581045d0 0x7f2a5819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.736+0000 7f2a5c9d4700 1 --2- 172.21.15.19:0/910488047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a58109850 0x7f2a5819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.737+0000 7f2a57fff700 1 -- 172.21.15.19:0/910488047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a58109850 msgr2=0x7f2a5819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.737+0000 7f2a57fff700 1 --2- 172.21.15.19:0/910488047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a58109850 0x7f2a5819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.737+0000 7f2a57fff700 1 -- 172.21.15.19:0/910488047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a58108f00 msgr2=0x7f2a5819b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.737+0000 7f2a57fff700 1 --2- 172.21.15.19:0/910488047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a58108f00 0x7f2a5819b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.737+0000 7f2a57fff700 1 -- 172.21.15.19:0/910488047 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5819fdc0 con 0x7f2a581045d0 2024-05-09T02:28:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.737+0000 7f2a57fff700 1 --2- 172.21.15.19:0/910488047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a581045d0 0x7f2a5819aef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800e9f0 tx=0x7f2a4800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:28:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.737+0000 7f2a55ffb700 1 -- 172.21.15.19:0/910488047 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48004220 con 0x7f2a581045d0 2024-05-09T02:28:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.737+0000 7f2a55ffb700 1 -- 172.21.15.19:0/910488047 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a480043c0 con 0x7f2a581045d0 2024-05-09T02:28:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.737+0000 7f2a55ffb700 1 -- 172.21.15.19:0/910488047 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48005020 con 0x7f2a581045d0 2024-05-09T02:28:07.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.737+0000 7f2a5e437700 1 -- 172.21.15.19:0/910488047 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a581a00b0 con 0x7f2a581045d0 2024-05-09T02:28:07.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.737+0000 7f2a5e437700 1 -- 172.21.15.19:0/910488047 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a581a79d0 con 0x7f2a581045d0 2024-05-09T02:28:07.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.738+0000 7f2a5e437700 1 -- 172.21.15.19:0/910488047 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a580512c0 con 0x7f2a581045d0 2024-05-09T02:28:07.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.740+0000 7f2a55ffb700 1 -- 172.21.15.19:0/910488047 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2a48004750 con 0x7f2a581045d0 2024-05-09T02:28:07.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.741+0000 7f2a55ffb700 1 --2- 172.21.15.19:0/910488047 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2a440b2a70 0x7f2a440b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:07.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.741+0000 7f2a4ffff700 1 --2- 172.21.15.19:0/910488047 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2a440b2a70 0x7f2a440b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:07.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.741+0000 7f2a55ffb700 1 -- 172.21.15.19:0/910488047 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8957+0+0 (secure 0 0 0) 0x7f2a480e2760 con 0x7f2a581045d0 2024-05-09T02:28:07.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.741+0000 7f2a4ffff700 1 --2- 172.21.15.19:0/910488047 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2a440b2a70 0x7f2a440b4f80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2a581034b0 tx=0x7f2a4001a040 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:28:07.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:07.744+0000 7f2a55ffb700 1 -- 172.21.15.19:0/910488047 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a480aa020 con 0x7f2a581045d0 2024-05-09T02:28:08.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:08.005+0000 7f2a5e437700 1 -- 172.21.15.19:0/910488047 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2a5819c250 con 0x7f2a581045d0 2024-05-09T02:28:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:08.609+0000 7f2a55ffb700 1 -- 172.21.15.19:0/910488047 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) ==== 104+0+0 (secure 0 0 0) 0x7f2a480a9920 con 0x7f2a581045d0 2024-05-09T02:28:08.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:08.613+0000 7f2a5e437700 1 -- 172.21.15.19:0/910488047 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2a440b2a70 msgr2=0x7f2a440b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:08.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:08.613+0000 7f2a5e437700 1 --2- 172.21.15.19:0/910488047 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2a440b2a70 0x7f2a440b4f80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2a581034b0 tx=0x7f2a4001a040 comp rx=0 tx=0).stop 2024-05-09T02:28:08.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:08.613+0000 7f2a5e437700 1 -- 172.21.15.19:0/910488047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a581045d0 msgr2=0x7f2a5819aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:08.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:08.613+0000 7f2a5e437700 1 --2- 172.21.15.19:0/910488047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a581045d0 0x7f2a5819aef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800e9f0 tx=0x7f2a4800ed00 comp rx=0 tx=0).stop 2024-05-09T02:28:08.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:08.613+0000 7f2a5e437700 1 -- 172.21.15.19:0/910488047 shutdown_connections 2024-05-09T02:28:08.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:08.613+0000 7f2a5e437700 1 --2- 172.21.15.19:0/910488047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a58109850 0x7f2a5819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:08.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:08.613+0000 7f2a5e437700 1 --2- 172.21.15.19:0/910488047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a581045d0 0x7f2a5819aef0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:08.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:08.613+0000 7f2a5e437700 1 --2- 172.21.15.19:0/910488047 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7f2a440b2a70 0x7f2a440b4f80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:08.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:08.613+0000 7f2a5e437700 1 --2- 172.21.15.19:0/910488047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a58108f00 0x7f2a5819b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:08.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:08.613+0000 7f2a5e437700 1 -- 172.21.15.19:0/910488047 >> 172.21.15.19:0/910488047 conn(0x7f2a580ffea0 msgr2=0x7f2a5810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:28:08.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:08.613+0000 7f2a5e437700 1 -- 172.21.15.19:0/910488047 shutdown_connections 2024-05-09T02:28:08.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:08.614+0000 7f2a5e437700 1 -- 172.21.15.19:0/910488047 wait complete. 2024-05-09T02:28:08.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:08 smithi059 ceph-mon[34781]: pgmap v529: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 92 KiB/s wr, 32 op/s 2024-05-09T02:28:08.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:08 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/910488047' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T02:28:08.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:08 smithi059 ceph-mon[34781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T02:28:08.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:08 smithi019 ceph-mon[30151]: pgmap v529: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 92 KiB/s wr, 32 op/s 2024-05-09T02:28:08.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:08 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/910488047' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T02:28:08.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:08 smithi019 ceph-mon[30151]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T02:28:09.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:08 smithi196 ceph-mon[34718]: pgmap v529: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 92 KiB/s wr, 32 op/s 2024-05-09T02:28:09.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:08 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/910488047' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T02:28:09.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:08 smithi196 ceph-mon[34718]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T02:28:09.341 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T02:28:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:28:09.869 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:09 smithi019 ceph-mon[30151]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T02:28:09.869 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:09 smithi019 ceph-mon[30151]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T02:28:09.869 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:09 smithi019 ceph-mon[30151]: Cluster is now healthy 2024-05-09T02:28:09.869 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:09 smithi019 ceph-mon[30151]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T02:28:09.869 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:09 smithi019 ceph-mon[30151]: fsmap 12 up:standby 2024-05-09T02:28:09.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:09 smithi059 ceph-mon[34781]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T02:28:09.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:09 smithi059 ceph-mon[34781]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T02:28:09.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:09 smithi059 ceph-mon[34781]: Cluster is now healthy 2024-05-09T02:28:09.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:09 smithi059 ceph-mon[34781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T02:28:09.967 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:09 smithi059 ceph-mon[34781]: fsmap 12 up:standby 2024-05-09T02:28:10.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:09 smithi196 ceph-mon[34718]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T02:28:10.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:09 smithi196 ceph-mon[34718]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T02:28:10.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:09 smithi196 ceph-mon[34718]: Cluster is now healthy 2024-05-09T02:28:10.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:09 smithi196 ceph-mon[34718]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T02:28:10.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:09 smithi196 ceph-mon[34718]: fsmap 12 up:standby 2024-05-09T02:28:10.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:10 smithi019 ceph-mon[30151]: pgmap v530: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 75 KiB/s wr, 30 op/s 2024-05-09T02:28:10.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:10 smithi059 ceph-mon[34781]: pgmap v530: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 75 KiB/s wr, 30 op/s 2024-05-09T02:28:11.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:10 smithi196 ceph-mon[34718]: pgmap v530: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 75 KiB/s wr, 30 op/s 2024-05-09T02:28:11.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.322+0000 7fa0c64ae700 1 -- 172.21.15.19:0/1086436461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c0107df0 msgr2=0x7fa0c0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.322+0000 7fa0c64ae700 1 --2- 172.21.15.19:0/1086436461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c0107df0 0x7fa0c0108200 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80099c0 tx=0x7fa0a8009cd0 comp rx=0 tx=0).stop 2024-05-09T02:28:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.322+0000 7fa0c64ae700 1 -- 172.21.15.19:0/1086436461 shutdown_connections 2024-05-09T02:28:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.322+0000 7fa0c64ae700 1 --2- 172.21.15.19:0/1086436461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0c0108740 0x7fa0c01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.322+0000 7fa0c64ae700 1 --2- 172.21.15.19:0/1086436461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c01034c0 0x7fa0c01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.322+0000 7fa0c64ae700 1 --2- 172.21.15.19:0/1086436461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c0107df0 0x7fa0c0108200 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.322+0000 7fa0c64ae700 1 -- 172.21.15.19:0/1086436461 >> 172.21.15.19:0/1086436461 conn(0x7fa0c00fedc0 msgr2=0x7fa0c01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:28:11.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.322+0000 7fa0c64ae700 1 -- 172.21.15.19:0/1086436461 shutdown_connections 2024-05-09T02:28:11.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.322+0000 7fa0c64ae700 1 -- 172.21.15.19:0/1086436461 wait complete. 2024-05-09T02:28:11.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.323+0000 7fa0c64ae700 1 Processor -- start 2024-05-09T02:28:11.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.323+0000 7fa0c64ae700 1 -- start start 2024-05-09T02:28:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.323+0000 7fa0c64ae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0c01034c0 0x7fa0c019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.323+0000 7fa0c64ae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c0107df0 0x7fa0c019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.323+0000 7fa0c64ae700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c0108740 0x7fa0c019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0c64ae700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c0113350 con 0x7fa0c0107df0 2024-05-09T02:28:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0c64ae700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c01131d0 con 0x7fa0c01034c0 2024-05-09T02:28:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0c64ae700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c01134d0 con 0x7fa0c0108740 2024-05-09T02:28:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0b7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c0107df0 0x7fa0c019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0bffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0c01034c0 0x7fa0c019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0b7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c0107df0 0x7fa0c019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37592/0 (socket says 172.21.15.19:37592) 2024-05-09T02:28:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0b7fff700 1 -- 172.21.15.19:0/1914526102 learned_addr learned my addr 172.21.15.19:0/1914526102 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:28:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0c4a4b700 1 --2- 172.21.15.19:0/1914526102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c0108740 0x7fa0c019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0b7fff700 1 -- 172.21.15.19:0/1914526102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c0108740 msgr2=0x7fa0c019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0b7fff700 1 --2- 172.21.15.19:0/1914526102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c0108740 0x7fa0c019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0b7fff700 1 -- 172.21.15.19:0/1914526102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0c01034c0 msgr2=0x7fa0c019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0b7fff700 1 --2- 172.21.15.19:0/1914526102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0c01034c0 0x7fa0c019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0b7fff700 1 -- 172.21.15.19:0/1914526102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c019fbf0 con 0x7fa0c0107df0 2024-05-09T02:28:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0b7fff700 1 --2- 172.21.15.19:0/1914526102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c0107df0 0x7fa0c019b370 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80098e0 tx=0x7fa0a80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:28:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0bdffb700 1 -- 172.21.15.19:0/1914526102 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8027070 con 0x7fa0c0107df0 2024-05-09T02:28:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0bdffb700 1 -- 172.21.15.19:0/1914526102 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a8003f20 con 0x7fa0c0107df0 2024-05-09T02:28:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0bdffb700 1 -- 172.21.15.19:0/1914526102 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8022d70 con 0x7fa0c0107df0 2024-05-09T02:28:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.324+0000 7fa0c64ae700 1 -- 172.21.15.19:0/1914526102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c019fe80 con 0x7fa0c0107df0 2024-05-09T02:28:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.325+0000 7fa0c64ae700 1 -- 172.21.15.19:0/1914526102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c01a0310 con 0x7fa0c0107df0 2024-05-09T02:28:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.325+0000 7fa0c64ae700 1 -- 172.21.15.19:0/1914526102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c00512c0 con 0x7fa0c0107df0 2024-05-09T02:28:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.327+0000 7fa0bdffb700 1 -- 172.21.15.19:0/1914526102 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa0a802a070 con 0x7fa0c0107df0 2024-05-09T02:28:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.328+0000 7fa0bdffb700 1 --2- 172.21.15.19:0/1914526102 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0ac0b29f0 0x7fa0ac0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.328+0000 7fa0bffff700 1 --2- 172.21.15.19:0/1914526102 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0ac0b29f0 0x7fa0ac0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.328+0000 7fa0bdffb700 1 -- 172.21.15.19:0/1914526102 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8957+0+0 (secure 0 0 0) 0x7fa0a80e3e50 con 0x7fa0c0107df0 2024-05-09T02:28:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.328+0000 7fa0bffff700 1 --2- 172.21.15.19:0/1914526102 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0ac0b29f0 0x7fa0ac0b4f00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000ac50 tx=0x7fa0b000a380 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:28:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.331+0000 7fa0bdffb700 1 -- 172.21.15.19:0/1914526102 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa0a8031080 con 0x7fa0c0107df0 2024-05-09T02:28:11.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.544+0000 7fa0c64ae700 1 -- 172.21.15.19:0/1914526102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa0c01a78b0 con 0x7fa0c0107df0 2024-05-09T02:28:11.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.625+0000 7fa0bdffb700 1 -- 172.21.15.19:0/1914526102 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) ==== 181+0+0 (secure 0 0 0) 0x7fa0a8013070 con 0x7fa0c0107df0 2024-05-09T02:28:11.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.628+0000 7fa0c64ae700 1 -- 172.21.15.19:0/1914526102 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0ac0b29f0 msgr2=0x7fa0ac0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:11.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.628+0000 7fa0c64ae700 1 --2- 172.21.15.19:0/1914526102 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0ac0b29f0 0x7fa0ac0b4f00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000ac50 tx=0x7fa0b000a380 comp rx=0 tx=0).stop 2024-05-09T02:28:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.628+0000 7fa0c64ae700 1 -- 172.21.15.19:0/1914526102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c0107df0 msgr2=0x7fa0c019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.628+0000 7fa0c64ae700 1 --2- 172.21.15.19:0/1914526102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c0107df0 0x7fa0c019b370 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80098e0 tx=0x7fa0a80057c0 comp rx=0 tx=0).stop 2024-05-09T02:28:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.629+0000 7fa0c64ae700 1 -- 172.21.15.19:0/1914526102 shutdown_connections 2024-05-09T02:28:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.629+0000 7fa0c64ae700 1 --2- 172.21.15.19:0/1914526102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c0108740 0x7fa0c019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.629+0000 7fa0c64ae700 1 --2- 172.21.15.19:0/1914526102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0c01034c0 0x7fa0c019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.629+0000 7fa0c64ae700 1 --2- 172.21.15.19:0/1914526102 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa0ac0b29f0 0x7fa0ac0b4f00 secure :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000ac50 tx=0x7fa0b000a380 comp rx=0 tx=0).stop 2024-05-09T02:28:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.629+0000 7fa0c64ae700 1 --2- 172.21.15.19:0/1914526102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c0107df0 0x7fa0c019b370 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.629+0000 7fa0c64ae700 1 -- 172.21.15.19:0/1914526102 >> 172.21.15.19:0/1914526102 conn(0x7fa0c00fedc0 msgr2=0x7fa0c01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:28:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.629+0000 7fa0c64ae700 1 -- 172.21.15.19:0/1914526102 shutdown_connections 2024-05-09T02:28:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:11.629+0000 7fa0c64ae700 1 -- 172.21.15.19:0/1914526102 wait complete. 2024-05-09T02:28:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-05-09T02:28:11.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:11 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1914526102' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T02:28:12.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:11 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1914526102' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T02:28:12.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:11 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1914526102' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T02:28:12.281 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T02:28:12.541 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T02:28:12.918 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:12 smithi019 ceph-mon[30151]: pgmap v531: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 75 KiB/s wr, 28 op/s 2024-05-09T02:28:12.921 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:12 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1914526102' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T02:28:12.921 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:12 smithi019 ceph-mon[30151]: osdmap e104: 12 total, 12 up, 12 in 2024-05-09T02:28:12.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:12 smithi059 ceph-mon[34781]: pgmap v531: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 75 KiB/s wr, 28 op/s 2024-05-09T02:28:12.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:12 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1914526102' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T02:28:12.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:12 smithi059 ceph-mon[34781]: osdmap e104: 12 total, 12 up, 12 in 2024-05-09T02:28:13.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:12 smithi196 ceph-mon[34718]: pgmap v531: 97 pgs: 97 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 75 KiB/s wr, 28 op/s 2024-05-09T02:28:13.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:12 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1914526102' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T02:28:13.045 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:12 smithi196 ceph-mon[34718]: osdmap e104: 12 total, 12 up, 12 in 2024-05-09T02:28:13.925 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:13 smithi019 ceph-mon[30151]: pgmap v533: 65 pgs: 65 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:28:13.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:13 smithi059 ceph-mon[34781]: pgmap v533: 65 pgs: 65 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:28:14.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:13 smithi196 ceph-mon[34718]: pgmap v533: 65 pgs: 65 active+clean; 4.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:28:14.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.223+0000 7fa9ea8de700 1 -- 172.21.15.19:0/2163640762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9e4108f20 msgr2=0x7fa9e4109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:14.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.223+0000 7fa9ea8de700 1 --2- 172.21.15.19:0/2163640762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9e4108f20 0x7fa9e4109330 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0099c0 tx=0x7fa9cc009cd0 comp rx=0 tx=0).stop 2024-05-09T02:28:14.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.223+0000 7fa9ea8de700 1 -- 172.21.15.19:0/2163640762 shutdown_connections 2024-05-09T02:28:14.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.223+0000 7fa9ea8de700 1 --2- 172.21.15.19:0/2163640762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e4109870 0x7fa9e41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:14.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.223+0000 7fa9ea8de700 1 --2- 172.21.15.19:0/2163640762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41045f0 0x7fa9e4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.223+0000 7fa9ea8de700 1 --2- 172.21.15.19:0/2163640762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9e4108f20 0x7fa9e4109330 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.223+0000 7fa9ea8de700 1 -- 172.21.15.19:0/2163640762 >> 172.21.15.19:0/2163640762 conn(0x7fa9e4100040 msgr2=0x7fa9e4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:28:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.223+0000 7fa9ea8de700 1 -- 172.21.15.19:0/2163640762 shutdown_connections 2024-05-09T02:28:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.223+0000 7fa9ea8de700 1 -- 172.21.15.19:0/2163640762 wait complete. 2024-05-09T02:28:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.224+0000 7fa9ea8de700 1 Processor -- start 2024-05-09T02:28:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.224+0000 7fa9ea8de700 1 -- start start 2024-05-09T02:28:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.224+0000 7fa9ea8de700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41045f0 0x7fa9e419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.224+0000 7fa9ea8de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9e4108f20 0x7fa9e419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.224+0000 7fa9ea8de700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e4109870 0x7fa9e419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:14.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.224+0000 7fa9ea8de700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4114480 con 0x7fa9e4108f20 2024-05-09T02:28:14.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.224+0000 7fa9ea8de700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4114300 con 0x7fa9e41045f0 2024-05-09T02:28:14.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.224+0000 7fa9ea8de700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4114600 con 0x7fa9e4109870 2024-05-09T02:28:14.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.224+0000 7fa9dbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9e4108f20 0x7fa9e419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:14.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.224+0000 7fa9dbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9e4108f20 0x7fa9e419b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37616/0 (socket says 172.21.15.19:37616) 2024-05-09T02:28:14.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.224+0000 7fa9dbfff700 1 -- 172.21.15.19:0/1709568674 learned_addr learned my addr 172.21.15.19:0/1709568674 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-09T02:28:14.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.224+0000 7fa9e3fff700 1 --2- 172.21.15.19:0/1709568674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41045f0 0x7fa9e419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:14.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.224+0000 7fa9e8e7b700 1 --2- 172.21.15.19:0/1709568674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e4109870 0x7fa9e419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:14.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.225+0000 7fa9dbfff700 1 -- 172.21.15.19:0/1709568674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e4109870 msgr2=0x7fa9e419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:14.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.225+0000 7fa9dbfff700 1 --2- 172.21.15.19:0/1709568674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e4109870 0x7fa9e419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:14.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.225+0000 7fa9dbfff700 1 -- 172.21.15.19:0/1709568674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41045f0 msgr2=0x7fa9e419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.225+0000 7fa9dbfff700 1 --2- 172.21.15.19:0/1709568674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41045f0 0x7fa9e419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.225+0000 7fa9dbfff700 1 -- 172.21.15.19:0/1709568674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e419fc50 con 0x7fa9e4108f20 2024-05-09T02:28:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.225+0000 7fa9dbfff700 1 --2- 172.21.15.19:0/1709568674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9e4108f20 0x7fa9e419b3d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc006010 tx=0x7fa9cc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:28:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.225+0000 7fa9e1ffb700 1 -- 172.21.15.19:0/1709568674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc027070 con 0x7fa9e4108f20 2024-05-09T02:28:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.225+0000 7fa9e1ffb700 1 -- 172.21.15.19:0/1709568674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9cc0040e0 con 0x7fa9e4108f20 2024-05-09T02:28:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.225+0000 7fa9ea8de700 1 -- 172.21.15.19:0/1709568674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e419fee0 con 0x7fa9e4108f20 2024-05-09T02:28:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.225+0000 7fa9e1ffb700 1 -- 172.21.15.19:0/1709568674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc022e50 con 0x7fa9e4108f20 2024-05-09T02:28:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.225+0000 7fa9ea8de700 1 -- 172.21.15.19:0/1709568674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e41a03c0 con 0x7fa9e4108f20 2024-05-09T02:28:14.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.226+0000 7fa9ea8de700 1 -- 172.21.15.19:0/1709568674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e40512c0 con 0x7fa9e4108f20 2024-05-09T02:28:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.229+0000 7fa9e1ffb700 1 -- 172.21.15.19:0/1709568674 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa9cc02a070 con 0x7fa9e4108f20 2024-05-09T02:28:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.230+0000 7fa9e1ffb700 1 --2- 172.21.15.19:0/1709568674 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa9d00b2b90 0x7fa9d00b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T02:28:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.230+0000 7fa9e3fff700 1 --2- 172.21.15.19:0/1709568674 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa9d00b2b90 0x7fa9d00b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T02:28:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.230+0000 7fa9e1ffb700 1 -- 172.21.15.19:0/1709568674 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8403+0+0 (secure 0 0 0) 0x7fa9cc0e3960 con 0x7fa9e4108f20 2024-05-09T02:28:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.230+0000 7fa9e3fff700 1 --2- 172.21.15.19:0/1709568674 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa9d00b2b90 0x7fa9d00b50a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400ac50 tx=0x7fa9d400a380 comp rx=0 tx=0).ready entity=mgr.15222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T02:28:14.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.233+0000 7fa9e1ffb700 1 -- 172.21.15.19:0/1709568674 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9cc013070 con 0x7fa9e4108f20 2024-05-09T02:28:14.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.447+0000 7fa9ea8de700 1 -- 172.21.15.19:0/1709568674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa9e402e5f0 con 0x7fa9e4108f20 2024-05-09T02:28:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.640+0000 7fa9e1ffb700 1 -- 172.21.15.19:0/1709568674 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) ==== 169+0+0 (secure 0 0 0) 0x7fa9cc0ab500 con 0x7fa9e4108f20 2024-05-09T02:28:14.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.645+0000 7fa9ea8de700 1 -- 172.21.15.19:0/1709568674 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa9d00b2b90 msgr2=0x7fa9d00b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:14.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.645+0000 7fa9ea8de700 1 --2- 172.21.15.19:0/1709568674 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa9d00b2b90 0x7fa9d00b50a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400ac50 tx=0x7fa9d400a380 comp rx=0 tx=0).stop 2024-05-09T02:28:14.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.645+0000 7fa9ea8de700 1 -- 172.21.15.19:0/1709568674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9e4108f20 msgr2=0x7fa9e419b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T02:28:14.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.645+0000 7fa9ea8de700 1 --2- 172.21.15.19:0/1709568674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9e4108f20 0x7fa9e419b3d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc006010 tx=0x7fa9cc0047b0 comp rx=0 tx=0).stop 2024-05-09T02:28:14.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.645+0000 7fa9ea8de700 1 -- 172.21.15.19:0/1709568674 shutdown_connections 2024-05-09T02:28:14.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.645+0000 7fa9ea8de700 1 --2- 172.21.15.19:0/1709568674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e4109870 0x7fa9e419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:14.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.645+0000 7fa9ea8de700 1 --2- 172.21.15.19:0/1709568674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41045f0 0x7fa9e419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:14.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.645+0000 7fa9ea8de700 1 --2- 172.21.15.19:0/1709568674 >> [v2:172.21.15.19:6800/2638922161,v1:172.21.15.19:6801/2638922161] conn(0x7fa9d00b2b90 0x7fa9d00b50a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:14.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.645+0000 7fa9ea8de700 1 --2- 172.21.15.19:0/1709568674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9e4108f20 0x7fa9e419b3d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T02:28:14.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.645+0000 7fa9ea8de700 1 -- 172.21.15.19:0/1709568674 >> 172.21.15.19:0/1709568674 conn(0x7fa9e4100040 msgr2=0x7fa9e410a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T02:28:14.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.646+0000 7fa9ea8de700 1 -- 172.21.15.19:0/1709568674 shutdown_connections 2024-05-09T02:28:14.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-09T02:28:14.647+0000 7fa9ea8de700 1 -- 172.21.15.19:0/1709568674 wait complete. 2024-05-09T02:28:14.651 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-05-09T02:28:14.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:14 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1709568674' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T02:28:15.026 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:14 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1709568674' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T02:28:15.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:14 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1709568674' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T02:28:15.357 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T02:28:15.395 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T02:28:15.432 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T02:28:15.468 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-09T02:28:15.468 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T02:28:15.484 DEBUG:teuthology.orchestra.run.smithi059:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T02:28:15.515 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T02:28:15.533 INFO:tasks.cephadm:Stopping all daemons... 2024-05-09T02:28:15.533 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-09T02:28:15.533 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.a 2024-05-09T02:28:15.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:15 smithi019 systemd[1]: Stopping Ceph mon.a for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:28:15.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:15 smithi059 ceph-mon[34781]: from='client.? 172.21.15.19:0/1709568674' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T02:28:15.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:15 smithi059 ceph-mon[34781]: osdmap e105: 12 total, 12 up, 12 in 2024-05-09T02:28:15.716 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:15 smithi059 ceph-mon[34781]: pgmap v535: 1 pgs: 1 active+clean; 577 KiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:28:15.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:15 smithi196 ceph-mon[34718]: from='client.? 172.21.15.19:0/1709568674' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T02:28:15.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:15 smithi196 ceph-mon[34718]: osdmap e105: 12 total, 12 up, 12 in 2024-05-09T02:28:15.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:15 smithi196 ceph-mon[34718]: pgmap v535: 1 pgs: 1 active+clean; 577 KiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:28:15.926 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:15 smithi019 ceph-mon[30151]: from='client.? 172.21.15.19:0/1709568674' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T02:28:15.926 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:15 smithi019 ceph-mon[30151]: osdmap e105: 12 total, 12 up, 12 in 2024-05-09T02:28:15.926 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:15 smithi019 ceph-mon[30151]: pgmap v535: 1 pgs: 1 active+clean; 577 KiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T02:28:16.308 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a[30147]: 2024-05-09T02:28:15.922+0000 7efd7c6b8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:28:16.308 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a[30147]: 2024-05-09T02:28:15.922+0000 7efd7c6b8700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-09T02:28:16.308 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:16 smithi019 podman[117879]: 2024-05-09 02:28:16.083074913 +0000 UTC m=+0.296312719 container died 4d8bf39103f3413dd54b9f9d782cc5312527d3e29548bf4a2826a12a87e3bcc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5) 2024-05-09T02:28:16.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:16 smithi019 podman[117879]: 2024-05-09 02:28:16.514134013 +0000 UTC m=+0.727371798 container cleanup 4d8bf39103f3413dd54b9f9d782cc5312527d3e29548bf4a2826a12a87e3bcc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-09T02:28:16.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:16 smithi019 bash[117879]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a 2024-05-09T02:28:17.228 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 09 02:28:16 smithi019 podman[117893]: 2024-05-09 02:28:16.857764776 +0000 UTC m=+0.772699163 container remove 4d8bf39103f3413dd54b9f9d782cc5312527d3e29548bf4a2826a12a87e3bcc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-09T02:28:17.245 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.a.service' 2024-05-09T02:28:17.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:28:17.321 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-09T02:28:17.321 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-09T02:28:17.321 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.b 2024-05-09T02:28:17.700 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:17 smithi059 systemd[1]: Stopping Ceph mon.b for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:28:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:17 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-b[34777]: 2024-05-09T02:28:17.697+0000 7f20003b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:28:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:17 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-b[34777]: 2024-05-09T02:28:17.697+0000 7f20003b0700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-09T02:28:17.966 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:17 smithi059 podman[72334]: 2024-05-09 02:28:17.852987625 +0000 UTC m=+0.277035493 container died e28aece55a381b4e8ebc446b21c52484a127984ae57b1b5baf548d803f017e2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-b, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T02:28:18.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:18 smithi059 podman[72334]: 2024-05-09 02:28:18.240052033 +0000 UTC m=+0.664099901 container cleanup e28aece55a381b4e8ebc446b21c52484a127984ae57b1b5baf548d803f017e2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T02:28:18.466 INFO:journalctl@ceph.mon.b.smithi059.stdout:May 09 02:28:18 smithi059 bash[72334]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-b 2024-05-09T02:28:18.620 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.b.service' 2024-05-09T02:28:18.692 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:28:18.692 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-09T02:28:18.692 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-09T02:28:18.692 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.c 2024-05-09T02:28:19.027 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:18 smithi196 systemd[1]: Stopping Ceph mon.c for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:28:19.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:19 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-c[34714]: 2024-05-09T02:28:19.024+0000 7fdd7e780700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:28:19.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:19 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-c[34714]: 2024-05-09T02:28:19.024+0000 7fdd7e780700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-09T02:28:19.295 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:19 smithi196 podman[72773]: 2024-05-09 02:28:19.09659348 +0000 UTC m=+0.183149690 container died 228fb8b256953b0444daa6354ad01d748ede0140ce1ebefe2e53e43b8e9648d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T02:28:19.787 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:19 smithi196 podman[72773]: 2024-05-09 02:28:19.534036414 +0000 UTC m=+0.620592641 container cleanup 228fb8b256953b0444daa6354ad01d748ede0140ce1ebefe2e53e43b8e9648d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-c, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-09T02:28:19.787 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:19 smithi196 bash[72773]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-c 2024-05-09T02:28:20.007 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.c.service' 2024-05-09T02:28:20.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:19 smithi196 podman[72787]: 2024-05-09 02:28:19.785788715 +0000 UTC m=+0.678793395 container remove 228fb8b256953b0444daa6354ad01d748ede0140ce1ebefe2e53e43b8e9648d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T02:28:20.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:20 smithi196 systemd[1]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.c.service: Deactivated successfully. 2024-05-09T02:28:20.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:20 smithi196 systemd[1]: Stopped Ceph mon.c for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:28:20.044 INFO:journalctl@ceph.mon.c.smithi196.stdout:May 09 02:28:20 smithi196 systemd[1]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mon.c.service: Consumed 16.970s CPU time. 2024-05-09T02:28:20.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:28:20.056 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-09T02:28:20.057 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-09T02:28:20.057 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mgr.x 2024-05-09T02:28:20.379 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:28:20 smithi019 systemd[1]: Stopping Ceph mgr.x for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:28:20.706 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:28:20 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x[30385]: 2024-05-09T02:28:20.376+0000 7f9c80238700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:28:20.706 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:28:20 smithi019 podman[117996]: 2024-05-09 02:28:20.418473235 +0000 UTC m=+0.158612010 container died fb9ba98e5788fb69ea0f7a33100fd486a6d11f027435800bed710edb562aac05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2) 2024-05-09T02:28:21.071 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:28:20 smithi019 podman[117996]: 2024-05-09 02:28:20.803281832 +0000 UTC m=+0.543420596 container cleanup fb9ba98e5788fb69ea0f7a33100fd486a6d11f027435800bed710edb562aac05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T02:28:21.071 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:28:20 smithi019 bash[117996]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x 2024-05-09T02:28:21.788 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-09T02:28:21.812 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:28:21 smithi019 podman[118010]: 2024-05-09 02:28:21.555770608 +0000 UTC m=+1.133428504 container remove fb9ba98e5788fb69ea0f7a33100fd486a6d11f027435800bed710edb562aac05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-x, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-09T02:28:21.812 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:28:21 smithi019 systemd[1]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mgr.x.service: Deactivated successfully. 2024-05-09T02:28:21.813 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:28:21 smithi019 systemd[1]: Stopped Ceph mgr.x for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:28:21.813 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 09 02:28:21 smithi019 systemd[1]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mgr.x.service: Consumed 1min 44.517s CPU time. 2024-05-09T02:28:21.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:28:21.827 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-09T02:28:21.827 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-09T02:28:21.827 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mgr.y 2024-05-09T02:28:22.185 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:28:21 smithi059 systemd[1]: Stopping Ceph mgr.y for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:28:22.466 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:28:22 smithi059 podman[72457]: 2024-05-09 02:28:22.183702483 +0000 UTC m=+0.153619585 container died f0c89ea92ac19a9194b88ffd7ebf2ba59460317c77b0d07ba111b3ca48ed50fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T02:28:22.966 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:28:22 smithi059 podman[72457]: 2024-05-09 02:28:22.761445575 +0000 UTC m=+0.731362676 container cleanup f0c89ea92ac19a9194b88ffd7ebf2ba59460317c77b0d07ba111b3ca48ed50fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5) 2024-05-09T02:28:22.966 INFO:journalctl@ceph.mgr.y.smithi059.stdout:May 09 02:28:22 smithi059 bash[72457]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-y 2024-05-09T02:28:23.200 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-09T02:28:23.234 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:28:23.235 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-09T02:28:23.235 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-09T02:28:23.235 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mgr.z 2024-05-09T02:28:23.544 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:28:23 smithi196 systemd[1]: Stopping Ceph mgr.z for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:28:25.010 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:28:24 smithi196 podman[72895]: 2024-05-09 02:28:24.63418416 +0000 UTC m=+1.204737369 container died 8fab2740a03d5be50c56a03b68ef5ab470fe8701c1d9bcec28d4c306ceecb81e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T02:28:25.294 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:28:25 smithi196 podman[72895]: 2024-05-09 02:28:25.106813682 +0000 UTC m=+1.677366888 container cleanup 8fab2740a03d5be50c56a03b68ef5ab470fe8701c1d9bcec28d4c306ceecb81e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, ceph=True) 2024-05-09T02:28:25.294 INFO:journalctl@ceph.mgr.z.smithi196.stdout:May 09 02:28:25 smithi196 bash[72895]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mgr-z 2024-05-09T02:28:25.500 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-09T02:28:25.533 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:28:25.533 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-09T02:28:25.533 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-09T02:28:25.533 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.0 2024-05-09T02:28:25.821 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:28:25 smithi019 systemd[1]: Stopping Ceph osd.0 for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:28:26.321 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:28:25 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-0[42587]: 2024-05-09T02:28:25.958+0000 7fc59391e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:28:26.321 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:28:25 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-0[42587]: 2024-05-09T02:28:25.958+0000 7fc59391e700 -1 osd.0 105 *** Got signal Terminated *** 2024-05-09T02:28:26.321 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:28:25 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-0[42587]: 2024-05-09T02:28:25.958+0000 7fc59391e700 -1 osd.0 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T02:28:31.970 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:28:31 smithi019 podman[118117]: 2024-05-09 02:28:31.717008997 +0000 UTC m=+5.984915623 container died e195ade32bd5a8d41c26e21b626f409b676df1dedc17cbec35389b4f5df30814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-09T02:28:32.321 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:28:32 smithi019 podman[118117]: 2024-05-09 02:28:32.176330876 +0000 UTC m=+6.444237550 container cleanup e195ade32bd5a8d41c26e21b626f409b676df1dedc17cbec35389b4f5df30814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-09T02:28:32.321 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:28:32 smithi019 bash[118117]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-0 2024-05-09T02:28:32.813 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:28:32 smithi019 podman[118135]: 2024-05-09 02:28:32.503030424 +0000 UTC m=+0.781111430 container remove e195ade32bd5a8d41c26e21b626f409b676df1dedc17cbec35389b4f5df30814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-0, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-05-09T02:28:33.071 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:28:32 smithi019 podman[118227]: 2024-05-09 02:28:32.715097874 +0000 UTC m=+0.018737675 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:28:33.071 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:28:32 smithi019 podman[118227]: 2024-05-09 02:28:32.834613936 +0000 UTC m=+0.138253786 container create 1f16072b97390250a4462ee25fca20a1e562e89346fe67073d2460948102ce86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-0-deactivate, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-09T02:28:33.428 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:28:33 smithi019 podman[118227]: 2024-05-09 02:28:33.093422823 +0000 UTC m=+0.397062620 container init 1f16072b97390250a4462ee25fca20a1e562e89346fe67073d2460948102ce86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-09T02:28:33.428 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:28:33 smithi019 podman[118227]: 2024-05-09 02:28:33.0968727 +0000 UTC m=+0.400512484 container start 1f16072b97390250a4462ee25fca20a1e562e89346fe67073d2460948102ce86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-05-09T02:28:33.428 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:28:33 smithi019 podman[118227]: 2024-05-09 02:28:33.198612344 +0000 UTC m=+0.502252147 container attach 1f16072b97390250a4462ee25fca20a1e562e89346fe67073d2460948102ce86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T02:28:33.717 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 09 02:28:33 smithi019 podman[118227]: 2024-05-09 02:28:33.426854115 +0000 UTC m=+0.730493900 container died 1f16072b97390250a4462ee25fca20a1e562e89346fe67073d2460948102ce86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-09T02:28:34.131 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.0.service' 2024-05-09T02:28:34.172 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:28:34.172 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-09T02:28:34.172 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-09T02:28:34.172 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.1 2024-05-09T02:28:34.466 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:28:34 smithi059 systemd[1]: Stopping Ceph osd.1 for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:28:34.966 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:28:34 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-1[39386]: 2024-05-09T02:28:34.490+0000 7fd174fd0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:28:34.966 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:28:34 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-1[39386]: 2024-05-09T02:28:34.490+0000 7fd174fd0700 -1 osd.1 105 *** Got signal Terminated *** 2024-05-09T02:28:34.967 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:28:34 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-1[39386]: 2024-05-09T02:28:34.490+0000 7fd174fd0700 -1 osd.1 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T02:28:40.373 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:28:40 smithi059 podman[72584]: 2024-05-09 02:28:40.115746848 +0000 UTC m=+5.754435355 container died 59eb3914594b1ac14f9dbbfaab5ed11923be403b556f71b3ade2efef02bc1578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-09T02:28:40.716 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:28:40 smithi059 podman[72584]: 2024-05-09 02:28:40.474147747 +0000 UTC m=+6.112836269 container cleanup 59eb3914594b1ac14f9dbbfaab5ed11923be403b556f71b3ade2efef02bc1578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-1, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T02:28:40.716 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:28:40 smithi059 bash[72584]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-1 2024-05-09T02:28:41.117 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:28:40 smithi059 podman[72601]: 2024-05-09 02:28:40.817574486 +0000 UTC m=+0.697021586 container remove 59eb3914594b1ac14f9dbbfaab5ed11923be403b556f71b3ade2efef02bc1578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-1, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-05-09T02:28:41.439 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:28:41 smithi059 podman[72691]: 2024-05-09 02:28:41.019017646 +0000 UTC m=+0.018473133 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:28:41.439 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:28:41 smithi059 podman[72691]: 2024-05-09 02:28:41.143379814 +0000 UTC m=+0.142835317 container create 9914713a1dfefca1b0605fdb862d0e684741d3ed2eb5eda17b03c7b333456c0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T02:28:41.439 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:28:41 smithi059 podman[72691]: 2024-05-09 02:28:41.35530578 +0000 UTC m=+0.354761238 container init 9914713a1dfefca1b0605fdb862d0e684741d3ed2eb5eda17b03c7b333456c0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T02:28:41.440 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:28:41 smithi059 podman[72691]: 2024-05-09 02:28:41.358660811 +0000 UTC m=+0.358116269 container start 9914713a1dfefca1b0605fdb862d0e684741d3ed2eb5eda17b03c7b333456c0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-09T02:28:41.440 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:28:41 smithi059 podman[72691]: 2024-05-09 02:28:41.437832345 +0000 UTC m=+0.437287823 container attach 9914713a1dfefca1b0605fdb862d0e684741d3ed2eb5eda17b03c7b333456c0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-09T02:28:41.966 INFO:journalctl@ceph.osd.1.smithi059.stdout:May 09 02:28:41 smithi059 podman[72691]: 2024-05-09 02:28:41.711350089 +0000 UTC m=+0.710805559 container died 9914713a1dfefca1b0605fdb862d0e684741d3ed2eb5eda17b03c7b333456c0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T02:28:42.342 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.1.service' 2024-05-09T02:28:42.383 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:28:42.383 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-09T02:28:42.383 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-09T02:28:42.383 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.2 2024-05-09T02:28:42.687 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:28:42 smithi196 systemd[1]: Stopping Ceph osd.2 for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:28:43.020 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:28:42 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-2[39882]: 2024-05-09T02:28:42.683+0000 7f1d38069700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:28:43.020 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:28:42 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-2[39882]: 2024-05-09T02:28:42.683+0000 7f1d38069700 -1 osd.2 105 *** Got signal Terminated *** 2024-05-09T02:28:43.020 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:28:42 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-2[39882]: 2024-05-09T02:28:42.683+0000 7f1d38069700 -1 osd.2 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T02:28:48.680 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:28:48 smithi196 podman[73016]: 2024-05-09 02:28:48.383208833 +0000 UTC m=+5.816512617 container died d874e54741ad11cfbe2f206f367aaab2b2cd0fca269c608a6c74995588f458db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-09T02:28:49.045 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:28:48 smithi196 podman[73016]: 2024-05-09 02:28:48.777306943 +0000 UTC m=+6.210610728 container cleanup d874e54741ad11cfbe2f206f367aaab2b2cd0fca269c608a6c74995588f458db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T02:28:49.045 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:28:48 smithi196 bash[73016]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-2 2024-05-09T02:28:49.402 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:28:49 smithi196 podman[73045]: 2024-05-09 02:28:49.049333724 +0000 UTC m=+0.660651972 container remove d874e54741ad11cfbe2f206f367aaab2b2cd0fca269c608a6c74995588f458db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-09T02:28:49.709 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:28:49 smithi196 podman[73133]: 2024-05-09 02:28:49.304029826 +0000 UTC m=+0.018105158 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:28:49.709 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:28:49 smithi196 podman[73133]: 2024-05-09 02:28:49.44674261 +0000 UTC m=+0.160817967 container create 97bc5a46147929ed4e5e01e003fd99a6c8a00064bbb42a39235b95472d44a62a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T02:28:49.709 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:28:49 smithi196 podman[73133]: 2024-05-09 02:28:49.707884412 +0000 UTC m=+0.421959727 container init 97bc5a46147929ed4e5e01e003fd99a6c8a00064bbb42a39235b95472d44a62a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-09T02:28:50.044 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:28:49 smithi196 podman[73133]: 2024-05-09 02:28:49.711173914 +0000 UTC m=+0.425249232 container start 97bc5a46147929ed4e5e01e003fd99a6c8a00064bbb42a39235b95472d44a62a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-09T02:28:50.044 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:28:49 smithi196 podman[73133]: 2024-05-09 02:28:49.816419088 +0000 UTC m=+0.530494409 container attach 97bc5a46147929ed4e5e01e003fd99a6c8a00064bbb42a39235b95472d44a62a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-09T02:28:50.363 INFO:journalctl@ceph.osd.2.smithi196.stdout:May 09 02:28:50 smithi196 podman[73133]: 2024-05-09 02:28:50.064071623 +0000 UTC m=+0.778146939 container died 97bc5a46147929ed4e5e01e003fd99a6c8a00064bbb42a39235b95472d44a62a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-09T02:28:50.725 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.2.service' 2024-05-09T02:28:50.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:28:50.764 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-09T02:28:50.764 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-09T02:28:50.764 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.3 2024-05-09T02:28:51.066 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:28:50 smithi019 systemd[1]: Stopping Ceph osd.3 for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:28:51.321 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:28:51 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-3[49562]: 2024-05-09T02:28:51.064+0000 7f85f19f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:28:51.321 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:28:51 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-3[49562]: 2024-05-09T02:28:51.064+0000 7f85f19f4700 -1 osd.3 105 *** Got signal Terminated *** 2024-05-09T02:28:51.321 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:28:51 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-3[49562]: 2024-05-09T02:28:51.064+0000 7f85f19f4700 -1 osd.3 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T02:28:53.821 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:28:53 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:28:53.422+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:28:54.821 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:28:54 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:28:54.408+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:28:55.821 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:28:55 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:28:55.435+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:28:55.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:28:55 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:28:55.561+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:28:56.478 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:28:56 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:28:56.213+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:28:56.821 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:28:56 smithi019 podman[118345]: 2024-05-09 02:28:56.725051639 +0000 UTC m=+5.761069827 container died fbc073cc682969c9bdffbb1f19fb57fd661e063f1ac3a82c389e9ca7371de360 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, ceph=True) 2024-05-09T02:28:56.821 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:28:56 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:28:56.475+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:28:56.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:28:56 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:28:56.605+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:28:57.321 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:28:57 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:28:57.226+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:28:57.321 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:28:57 smithi019 podman[118345]: 2024-05-09 02:28:57.264595112 +0000 UTC m=+6.300613300 container cleanup fbc073cc682969c9bdffbb1f19fb57fd661e063f1ac3a82c389e9ca7371de360 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T02:28:57.321 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:28:57 smithi019 bash[118345]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-3 2024-05-09T02:28:57.439 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:28:57 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:28:57.185+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:28:57.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:28:57 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:28:57.436+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:28:57.821 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:28:57 smithi019 podman[118363]: 2024-05-09 02:28:57.508067909 +0000 UTC m=+0.777595372 container remove fbc073cc682969c9bdffbb1f19fb57fd661e063f1ac3a82c389e9ca7371de360 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T02:28:57.822 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:28:57 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:28:57.514+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:28:57.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:28:57 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:28:57.648+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:28:58.200 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:28:57 smithi019 podman[118453]: 2024-05-09 02:28:57.84816616 +0000 UTC m=+0.018818176 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:28:58.200 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:28:57 smithi019 podman[118453]: 2024-05-09 02:28:57.967649173 +0000 UTC m=+0.138301176 container create a4faebdb16e3867b66901d75b0048a459e5b605f7e6520d7cd836a0388a1f17e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS) 2024-05-09T02:28:58.200 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:28:58 smithi019 podman[118453]: 2024-05-09 02:28:58.195023337 +0000 UTC m=+0.365675353 container init a4faebdb16e3867b66901d75b0048a459e5b605f7e6520d7cd836a0388a1f17e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-09T02:28:58.200 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:28:58 smithi019 podman[118453]: 2024-05-09 02:28:58.198934613 +0000 UTC m=+0.369586617 container start a4faebdb16e3867b66901d75b0048a459e5b605f7e6520d7cd836a0388a1f17e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T02:28:58.411 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:28:58 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:28:58.159+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:28:58.471 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:28:58 smithi019 podman[118453]: 2024-05-09 02:28:58.286860208 +0000 UTC m=+0.457512214 container attach a4faebdb16e3867b66901d75b0048a459e5b605f7e6520d7cd836a0388a1f17e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-09T02:28:58.471 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:28:58 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:28:58.213+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:28:58.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:28:58 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:28:58.409+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:28:58.821 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:28:58 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:28:58.468+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:28:58.821 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 09 02:28:58 smithi019 podman[118453]: 2024-05-09 02:28:58.511013272 +0000 UTC m=+0.681665276 container died a4faebdb16e3867b66901d75b0048a459e5b605f7e6520d7cd836a0388a1f17e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-09T02:28:58.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:28:58 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:28:58.636+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:28:59.199 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.3.service' 2024-05-09T02:28:59.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:28:59.244 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-09T02:28:59.244 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-09T02:28:59.244 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.4 2024-05-09T02:28:59.493 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:28:59 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:28:59.187+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:28:59.544 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:28:59 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:28:59.203+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:28:59.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:28:59 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:28:59.453+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:28:59.554 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:28:59 smithi059 systemd[1]: Stopping Ceph osd.4 for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:28:59.821 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:28:59 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:28:59.492+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:28:59.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:28:59 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:28:59.680+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:28:59.966 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:28:59 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4[45904]: 2024-05-09T02:28:59.552+0000 7fad36fdf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:28:59.967 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:28:59 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4[45904]: 2024-05-09T02:28:59.552+0000 7fad36fdf700 -1 osd.4 105 *** Got signal Terminated *** 2024-05-09T02:28:59.967 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:28:59 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4[45904]: 2024-05-09T02:28:59.552+0000 7fad36fdf700 -1 osd.4 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T02:29:00.406 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:00 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:00.213+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:29:00.407 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:00 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:00.404+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:00.446 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:00 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:00.156+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:00.821 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:00 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:00.444+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:00.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:00 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:00.646+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:01.407 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:01.149+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:01.407 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:01.149+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:01.544 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:01 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:01.174+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:29:01.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:01 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:01.367+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:01.821 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:01.404+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:01.821 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:01 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:01.404+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:01.966 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:29:01 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4[45904]: 2024-05-09T02:29:01.536+0000 7fad2f58d700 -1 osd.4 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.914345+0000 front 2024-05-09T02:28:35.914386+0000 (oldest deadline 2024-05-09T02:29:00.614535+0000) 2024-05-09T02:29:01.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:01 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:01.636+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:02.544 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:02 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:02.208+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:29:02.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:02 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:02.331+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:02.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:02 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:02.186+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:02.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:02 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:02.186+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:02.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:02 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:02.365+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:02.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:02 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:02.365+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:02.966 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:29:02 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4[45904]: 2024-05-09T02:29:02.567+0000 7fad2f58d700 -1 osd.4 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.914345+0000 front 2024-05-09T02:28:35.914386+0000 (oldest deadline 2024-05-09T02:29:00.614535+0000) 2024-05-09T02:29:02.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:02 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:02.609+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:02.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:02 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:02.609+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:03.490 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:03 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:03.175+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:29:03.490 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:03 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:03.345+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:03.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:03 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:03.157+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:03.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:03 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:03.157+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:03.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:03 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:03.387+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:03.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:03 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:03.387+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:03.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:03 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:03.488+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:03.970 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:29:03 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4[45904]: 2024-05-09T02:29:03.614+0000 7fad2f58d700 -1 osd.4 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.914345+0000 front 2024-05-09T02:28:35.914386+0000 (oldest deadline 2024-05-09T02:29:00.614535+0000) 2024-05-09T02:29:03.970 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:03 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:03.560+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:03.970 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:03 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:03.560+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:03.970 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:03 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:03.565+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:04.495 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:04 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:04.155+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:29:04.495 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:04 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:04.155+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.653634+0000 front 2024-05-09T02:28:37.653696+0000 (oldest deadline 2024-05-09T02:29:03.553736+0000) 2024-05-09T02:29:04.495 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:04 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:04.389+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:04.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:04 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:04.201+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:04.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:04 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:04.201+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:04.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:04 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:04.397+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:04.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:04 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:04.397+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:04.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:04 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:04.492+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:04.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:04 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:04.589+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:04.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:04 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:04.533+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:04.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:04 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:04.533+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:05.480 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:05.371+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:05.480 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:05.205+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:29:05.480 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:05.205+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.653634+0000 front 2024-05-09T02:28:37.653696+0000 (oldest deadline 2024-05-09T02:29:03.553736+0000) 2024-05-09T02:29:05.558 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:29:05 smithi059 podman[72808]: 2024-05-09 02:29:05.272085881 +0000 UTC m=+5.827648973 container died 7311b3fc9fb23e154b447f792b65b9eabe57129ef8d263ff1abb7d289612a05a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T02:29:05.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:05 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:05.180+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:05.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:05 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:05.180+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:05.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:05 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:05.367+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:05.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:05 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:05.367+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:05.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:05 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:05.478+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:05.947 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:05 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:05.560+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:05.947 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:29:05 smithi059 podman[72808]: 2024-05-09 02:29:05.66931232 +0000 UTC m=+6.224875403 container cleanup 7311b3fc9fb23e154b447f792b65b9eabe57129ef8d263ff1abb7d289612a05a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-05-09T02:29:05.947 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:29:05 smithi059 bash[72808]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4 2024-05-09T02:29:05.947 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:05 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:05.556+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:05.947 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:05 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:05.556+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:06.216 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:29:05 smithi059 podman[72827]: 2024-05-09 02:29:05.945611161 +0000 UTC m=+0.676190551 container remove 7311b3fc9fb23e154b447f792b65b9eabe57129ef8d263ff1abb7d289612a05a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-09T02:29:06.528 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:06 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:06.217+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:29:06.528 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:06 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:06.217+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.653634+0000 front 2024-05-09T02:28:37.653696+0000 (oldest deadline 2024-05-09T02:29:03.553736+0000) 2024-05-09T02:29:06.528 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:06 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:06.399+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:06.529 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:06 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:06.399+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:06.559 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:29:06 smithi059 podman[72921]: 2024-05-09 02:29:06.194170129 +0000 UTC m=+0.018888081 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:29:06.559 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:29:06 smithi059 podman[72921]: 2024-05-09 02:29:06.330022328 +0000 UTC m=+0.154740318 container create be164a70f1d6a40de56f2ea40ff3189010cb34b054498763a5ad881d7deac35e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=) 2024-05-09T02:29:06.559 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:29:06 smithi059 podman[72921]: 2024-05-09 02:29:06.55744686 +0000 UTC m=+0.382164819 container init be164a70f1d6a40de56f2ea40ff3189010cb34b054498763a5ad881d7deac35e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T02:29:06.559 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:06 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:06.529+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:06.559 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:06 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:06.529+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:06.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:06 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:06.176+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:06.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:06 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:06.176+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:06.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:06 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:06.337+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:06.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:06 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:06.337+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:06.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:06 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:06.526+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:06.867 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:06 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:06.588+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:06.867 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:29:06 smithi059 podman[72921]: 2024-05-09 02:29:06.560987192 +0000 UTC m=+0.385705147 container start be164a70f1d6a40de56f2ea40ff3189010cb34b054498763a5ad881d7deac35e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T02:29:06.867 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:29:06 smithi059 podman[72921]: 2024-05-09 02:29:06.649042676 +0000 UTC m=+0.473760637 container attach be164a70f1d6a40de56f2ea40ff3189010cb34b054498763a5ad881d7deac35e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-09T02:29:07.178 INFO:journalctl@ceph.osd.4.smithi059.stdout:May 09 02:29:06 smithi059 podman[72921]: 2024-05-09 02:29:06.86599486 +0000 UTC m=+0.690712814 container died be164a70f1d6a40de56f2ea40ff3189010cb34b054498763a5ad881d7deac35e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-09T02:29:07.504 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.4.service' 2024-05-09T02:29:07.540 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:07 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:07.182+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:29:07.540 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:07 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:07.182+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.653634+0000 front 2024-05-09T02:28:37.653696+0000 (oldest deadline 2024-05-09T02:29:03.553736+0000) 2024-05-09T02:29:07.540 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:07 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:07.389+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:07.540 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:07 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:07.389+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:07.550 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:29:07.550 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-09T02:29:07.550 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-09T02:29:07.550 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.5 2024-05-09T02:29:07.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:07 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:07.182+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:07.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:07 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:07.182+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:07.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:07 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:07.359+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:07.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:07 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:07.359+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:07.794 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:07 smithi196 systemd[1]: Stopping Ceph osd.5 for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:29:07.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:07 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:07.537+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:07.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:07 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:07.620+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:07.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:07 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:07.489+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:07.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:07 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:07.489+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:08.220 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:07 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:07.864+0000 7fcfd49be700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:29:08.220 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:07 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:07.864+0000 7fcfd49be700 -1 osd.5 105 *** Got signal Terminated *** 2024-05-09T02:29:08.220 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:07 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:07.864+0000 7fcfd49be700 -1 osd.5 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T02:29:08.220 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:08 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:08.218+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:29:08.220 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:08 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:08.218+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.653634+0000 front 2024-05-09T02:28:37.653696+0000 (oldest deadline 2024-05-09T02:29:03.553736+0000) 2024-05-09T02:29:08.489 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:08 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:08.404+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:08.490 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:08 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:08.404+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:08.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:08 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:08.162+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:08.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:08 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:08.162+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:08.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:08 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:08.336+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:08.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:08 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:08.336+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:08.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:08 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:08.487+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:08.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:08 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:08.616+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:08.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:08 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:08.491+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:08.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:08 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:08.491+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:09.544 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:09 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:09.453+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:09.545 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:09 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:09.236+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:29:09.545 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:09 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:09.236+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.653634+0000 front 2024-05-09T02:28:37.653696+0000 (oldest deadline 2024-05-09T02:29:03.553736+0000) 2024-05-09T02:29:09.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:09 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:09.361+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:09.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:09 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:09.361+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:09.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:09 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:09.361+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:09.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:09 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:09.137+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:09.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:09 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:09.137+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:09.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:09 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:09.137+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:43.196500+0000 front 2024-05-09T02:28:43.196484+0000 (oldest deadline 2024-05-09T02:29:09.096418+0000) 2024-05-09T02:29:09.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:09 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:09.304+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:09.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:09 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:09.304+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:09.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:09 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:09.304+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:09.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:09 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:09.610+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:09.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:09 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:09.474+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:09.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:09 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:09.474+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:10.544 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:10 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:10.213+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:29:10.545 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:10 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:10.213+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.653634+0000 front 2024-05-09T02:28:37.653696+0000 (oldest deadline 2024-05-09T02:29:03.553736+0000) 2024-05-09T02:29:10.545 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:10 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:10.425+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:10.545 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:10 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:10.425+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:10.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:10 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:10.369+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:10.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:10 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:10.369+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:10.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:10 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:10.369+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:10.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:10 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:10.119+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:10.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:10 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:10.119+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:10.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:10 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:10.119+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:43.196500+0000 front 2024-05-09T02:28:43.196484+0000 (oldest deadline 2024-05-09T02:29:09.096418+0000) 2024-05-09T02:29:10.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:10 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:10.279+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:10.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:10 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:10.279+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:10.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:10 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:10.279+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:10.867 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:10 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:10.627+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:10.868 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:10 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:10.477+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:10.868 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:10 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:10.477+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:11.544 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:11 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:11.235+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:29:11.545 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:11 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:11.235+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.653634+0000 front 2024-05-09T02:28:37.653696+0000 (oldest deadline 2024-05-09T02:29:03.553736+0000) 2024-05-09T02:29:11.545 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:11 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:11.474+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:11.545 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:11 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:11.474+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:11.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:11 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:11.346+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:11.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:11 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:11.346+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:11.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:11 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:11.346+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:11.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:11 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:11.102+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:11.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:11 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:11.102+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:11.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:11 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:11.102+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:43.196500+0000 front 2024-05-09T02:28:43.196484+0000 (oldest deadline 2024-05-09T02:29:09.096418+0000) 2024-05-09T02:29:11.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:11 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:11.235+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:11.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:11 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:11.235+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:11.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:11 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:11.235+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:11.716 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:11 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:11.657+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:11.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:11 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:11.428+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:11.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:11 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:11.428+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:12.544 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:12 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:12.243+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.652694+0000 front 2024-05-09T02:28:30.652718+0000 (oldest deadline 2024-05-09T02:28:56.552473+0000) 2024-05-09T02:29:12.545 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:12 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5[46419]: 2024-05-09T02:29:12.243+0000 7fcfccf6c700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.653634+0000 front 2024-05-09T02:28:37.653696+0000 (oldest deadline 2024-05-09T02:29:03.553736+0000) 2024-05-09T02:29:12.545 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:12 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:12.463+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:12.545 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:12 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:12.463+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:12.546 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:12 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:12.334+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:12.546 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:12 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:12.334+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:12.546 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:12 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:12.334+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:12.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:12 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:12.134+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:12.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:12 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:12.134+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:12.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:12 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:12.134+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:43.196500+0000 front 2024-05-09T02:28:43.196484+0000 (oldest deadline 2024-05-09T02:29:09.096418+0000) 2024-05-09T02:29:12.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:12 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:12.194+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:12.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:12 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:12.194+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:12.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:12 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:12.194+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:12.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:12 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:12.643+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:12.967 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:12 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:12.643+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:12.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:12 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:12.475+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:12.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:12 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:12.475+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:12.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:12 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:12.475+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:13.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:13 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:13.156+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:13.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:13 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:13.156+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:13.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:13 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:13.156+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:43.196500+0000 front 2024-05-09T02:28:43.196484+0000 (oldest deadline 2024-05-09T02:29:09.096418+0000) 2024-05-09T02:29:13.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:13 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:13.182+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:13.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:13 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:13.182+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:13.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:13 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:13.182+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:13.742 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:13 smithi196 podman[73240]: 2024-05-09 02:29:13.482561915 +0000 UTC m=+5.736820824 container died dc8ebbcaf9397bda3e727c55b5cd7f4e37da86f391e02bf31edf2ff09f2a3a57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-09T02:29:13.743 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:13 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:13.478+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:13.743 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:13 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:13.478+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:13.743 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:13 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:13.354+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:13.743 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:13 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:13.354+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:13.743 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:13 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:13.354+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:13.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:13 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:13.642+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:13.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:13 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:13.642+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:13.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:13 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:13.525+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:13.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:13 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:13.525+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:13.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:13 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:13.525+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:14.044 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:13 smithi196 podman[73240]: 2024-05-09 02:29:13.881422237 +0000 UTC m=+6.135681129 container cleanup dc8ebbcaf9397bda3e727c55b5cd7f4e37da86f391e02bf31edf2ff09f2a3a57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T02:29:14.044 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:13 smithi196 bash[73240]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5 2024-05-09T02:29:14.197 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:14 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:14.162+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:14.198 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:14 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:14.162+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:14.198 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:14 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:14.162+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:43.196500+0000 front 2024-05-09T02:28:43.196484+0000 (oldest deadline 2024-05-09T02:29:09.096418+0000) 2024-05-09T02:29:14.500 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:14 smithi196 podman[73258]: 2024-05-09 02:29:14.157950012 +0000 UTC m=+0.676487771 container remove dc8ebbcaf9397bda3e727c55b5cd7f4e37da86f391e02bf31edf2ff09f2a3a57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-09T02:29:14.501 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:14 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:14.337+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:14.501 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:14 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:14.337+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:14.501 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:14 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:14.337+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:14.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:14 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:14.196+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:14.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:14 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:14.196+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:14.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:14 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:14.196+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:14.794 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:14 smithi196 podman[73356]: 2024-05-09 02:29:14.434860343 +0000 UTC m=+0.018804066 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:29:14.795 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:14 smithi196 podman[73356]: 2024-05-09 02:29:14.550873336 +0000 UTC m=+0.134817058 container create 75016fc6ccc37583eaa542ef84dfdcf92a8bf08c288c8c33b9f704096e9df178 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T02:29:14.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:14 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:14.498+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:14.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:14 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:14.498+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:14.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:14 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:14.611+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:14.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:14 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:14.611+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:14.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:14 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:14.503+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:14.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:14 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:14.503+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:14.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:14 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:14.503+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:15.119 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:14 smithi196 podman[73356]: 2024-05-09 02:29:14.820795738 +0000 UTC m=+0.404739460 container init 75016fc6ccc37583eaa542ef84dfdcf92a8bf08c288c8c33b9f704096e9df178 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T02:29:15.119 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:14 smithi196 podman[73356]: 2024-05-09 02:29:14.82405009 +0000 UTC m=+0.407993813 container start 75016fc6ccc37583eaa542ef84dfdcf92a8bf08c288c8c33b9f704096e9df178 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-09T02:29:15.119 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:14 smithi196 podman[73356]: 2024-05-09 02:29:14.904030946 +0000 UTC m=+0.487974668 container attach 75016fc6ccc37583eaa542ef84dfdcf92a8bf08c288c8c33b9f704096e9df178 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-05-09T02:29:15.119 INFO:journalctl@ceph.osd.5.smithi196.stdout:May 09 02:29:15 smithi196 podman[73356]: 2024-05-09 02:29:15.117238136 +0000 UTC m=+0.701181860 container died 75016fc6ccc37583eaa542ef84dfdcf92a8bf08c288c8c33b9f704096e9df178 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T02:29:15.400 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:15 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:15.295+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:15.400 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:15 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:15.295+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:15.400 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:15 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:15.295+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:15.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:15.149+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:15.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:15.149+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:15.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:15.149+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:43.196500+0000 front 2024-05-09T02:28:43.196484+0000 (oldest deadline 2024-05-09T02:29:09.096418+0000) 2024-05-09T02:29:15.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:15.207+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:15.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:15.207+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:15.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:15.207+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:15.667 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:15 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:15.536+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:15.667 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:15 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:15.536+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:15.698 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.5.service' 2024-05-09T02:29:15.738 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:29:15.739 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-09T02:29:15.739 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-09T02:29:15.739 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.6 2024-05-09T02:29:15.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:15 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:15.647+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:15.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:15 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:15.647+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:15.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:15 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:15.544+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:15.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:15 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:15.544+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:15.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:15 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:15.544+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:16.062 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:15 smithi019 systemd[1]: Stopping Ceph osd.6 for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:29:16.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:16.222+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:16.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:16.222+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:16.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:16.222+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:16.321 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:16.059+0000 7f7f32b3f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:29:16.321 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:16.059+0000 7f7f32b3f700 -1 osd.6 105 *** Got signal Terminated *** 2024-05-09T02:29:16.321 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:16.059+0000 7f7f32b3f700 -1 osd.6 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T02:29:16.322 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:16.145+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:16.322 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:16.145+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:16.322 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:16 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:16.145+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:43.196500+0000 front 2024-05-09T02:28:43.196484+0000 (oldest deadline 2024-05-09T02:29:09.096418+0000) 2024-05-09T02:29:16.542 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:16 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:16.267+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:16.542 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:16 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:16.267+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:16.542 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:16 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:16.267+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:16.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:16 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:16.540+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:16.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:16 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:16.540+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:16.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:16 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:16.681+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:16.967 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:16 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:16.681+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:16.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:16 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:16.513+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:16.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:16 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:16.513+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:16.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:16 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:16.513+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:17.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:17 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:17.260+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:17.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:17 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:17.260+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:17.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:17 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:17.260+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:17.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:17.170+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:17.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:17.170+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:17.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:17.170+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:43.196500+0000 front 2024-05-09T02:28:43.196484+0000 (oldest deadline 2024-05-09T02:29:09.096418+0000) 2024-05-09T02:29:17.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:17.264+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:17.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:17.264+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:17.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:17.264+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:17.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:17 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:17.264+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:17.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:17 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:17.665+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:17.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:17 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:17.665+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:17.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:17 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:17.535+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:17.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:17 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:17.535+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:17.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:17 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:17.535+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:17.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:17 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:17.535+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:18.045 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:17 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:17.550+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:18.045 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:17 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:17.550+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:18.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:18 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:18.240+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:18.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:18 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:18.240+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:18.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:18 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:18.240+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:18.573 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:18 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:18.149+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:18.573 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:18 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:18.149+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:18.573 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:18 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:18.149+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:43.196500+0000 front 2024-05-09T02:28:43.196484+0000 (oldest deadline 2024-05-09T02:29:09.096418+0000) 2024-05-09T02:29:18.573 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:18 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:18.149+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.297825+0000 front 2024-05-09T02:28:54.297869+0000 (oldest deadline 2024-05-09T02:29:17.797838+0000) 2024-05-09T02:29:18.574 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:18 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:18.238+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:18.574 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:18 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:18.238+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:18.574 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:18 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:18.238+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:18.574 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:18 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:18.238+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:18.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:18 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:18.704+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:18.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:18 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:18.704+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:18.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:18 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:18.494+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:18.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:18 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:18.494+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:18.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:18 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:18.494+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:18.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:18 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:18.494+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:19.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:18 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:18.565+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:19.045 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:18 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:18.565+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:19.045 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:18 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:18.565+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:19.535 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:19 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:19.215+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:19.535 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:19 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:19.215+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:19.535 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:19 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:19.215+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:19.535 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:19 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:19.215+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:19.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:19 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:19.159+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:19.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:19 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:19.159+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:19.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:19 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:19.159+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:43.196500+0000 front 2024-05-09T02:28:43.196484+0000 (oldest deadline 2024-05-09T02:29:09.096418+0000) 2024-05-09T02:29:19.572 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:19 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:19.159+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.297825+0000 front 2024-05-09T02:28:54.297869+0000 (oldest deadline 2024-05-09T02:29:17.797838+0000) 2024-05-09T02:29:19.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:19 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:19.211+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:19.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:19 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:19.211+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:19.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:19 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:19.211+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:19.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:19 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:19.211+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:19.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:19 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:19.533+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:19.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:19 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:19.533+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:19.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:19 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:19.533+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:19.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:19 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:19.658+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:19.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:19 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:19.658+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:19.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:19 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:19.503+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:19.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:19 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:19.503+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:19.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:19 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:19.503+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:19.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:19 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:19.503+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:20.489 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:20 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:20.220+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:20.489 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:20 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:20.220+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:20.489 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:20 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:20.220+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:20.489 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:20 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:20.220+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:20.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:20 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:20.205+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:30.294589+0000 front 2024-05-09T02:28:30.294663+0000 (oldest deadline 2024-05-09T02:28:55.594701+0000) 2024-05-09T02:29:20.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:20 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:20.205+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:35.595292+0000 front 2024-05-09T02:28:35.595454+0000 (oldest deadline 2024-05-09T02:29:00.895250+0000) 2024-05-09T02:29:20.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:20 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:20.205+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:43.196500+0000 front 2024-05-09T02:28:43.196484+0000 (oldest deadline 2024-05-09T02:29:09.096418+0000) 2024-05-09T02:29:20.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:20 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6[56522]: 2024-05-09T02:29:20.205+0000 7f7f2f0f5700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.297825+0000 front 2024-05-09T02:28:54.297869+0000 (oldest deadline 2024-05-09T02:29:17.797838+0000) 2024-05-09T02:29:20.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:20 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:20.216+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:20.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:20 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:20.216+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:20.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:20 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:20.216+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:20.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:20 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:20.216+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:20.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:20 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:20.487+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:20.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:20 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:20.487+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:20.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:20 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:20.487+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:20.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:20 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:20.653+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:20.967 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:20 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:20.653+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:20.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:20 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:20.492+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:20.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:20 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:20.492+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:20.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:20 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:20.492+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:20.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:20 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:20.492+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:21.489 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:21 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:21.191+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:21.489 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:21 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:21.191+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:21.489 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:21 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:21.191+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:21.489 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:21 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:21.191+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:21.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:21 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:21.179+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:21.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:21 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:21.179+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:21.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:21 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:21.179+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:21.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:21 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:21.179+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:21.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:21 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:21.487+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:21.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:21 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:21.487+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:21.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:21 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:21.487+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:21.912 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:21 smithi019 podman[118574]: 2024-05-09 02:29:21.645311056 +0000 UTC m=+5.705880766 container died 03802a5580d4d7aa7d8f7d34f0715bb0084c7c481aa9f097a49a9ca8af04fe65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T02:29:21.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:21 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:21.644+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:21.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:21 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:21.644+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:21.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:21 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:21.503+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:21.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:21 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:21.503+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:21.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:21 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:21.503+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:21.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:21 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:21.503+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:22.313 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:22 smithi019 podman[118574]: 2024-05-09 02:29:22.009927423 +0000 UTC m=+6.070497125 container cleanup 03802a5580d4d7aa7d8f7d34f0715bb0084c7c481aa9f097a49a9ca8af04fe65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T02:29:22.313 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:22 smithi019 bash[118574]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6 2024-05-09T02:29:22.313 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:22 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:22.151+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:22.313 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:22 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:22.151+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:22.313 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:22 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:22.151+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:22.314 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:22 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:22.151+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:22.493 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:22 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:22.182+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:22.493 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:22 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:22.182+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:22.493 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:22 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:22.182+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:22.493 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:22 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:22.182+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:22.571 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:22 smithi019 podman[118591]: 2024-05-09 02:29:22.311356072 +0000 UTC m=+0.658428944 container remove 03802a5580d4d7aa7d8f7d34f0715bb0084c7c481aa9f097a49a9ca8af04fe65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T02:29:22.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:22 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:22.491+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:22.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:22 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:22.491+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:22.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:22 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:22.491+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:22.958 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:22 smithi019 podman[118684]: 2024-05-09 02:29:22.544134586 +0000 UTC m=+0.018685447 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:29:22.958 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:22 smithi019 podman[118684]: 2024-05-09 02:29:22.653882598 +0000 UTC m=+0.128433505 container create a7bdb64027a2141775655fe6ec21d4faa1b593d47b20db5735d2f3a2930247dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True) 2024-05-09T02:29:22.958 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:22 smithi019 podman[118684]: 2024-05-09 02:29:22.872719213 +0000 UTC m=+0.347270075 container init a7bdb64027a2141775655fe6ec21d4faa1b593d47b20db5735d2f3a2930247dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6-deactivate, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-09T02:29:22.958 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:22 smithi019 podman[118684]: 2024-05-09 02:29:22.876719665 +0000 UTC m=+0.351270535 container start a7bdb64027a2141775655fe6ec21d4faa1b593d47b20db5735d2f3a2930247dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-09T02:29:22.958 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:22 smithi019 podman[118684]: 2024-05-09 02:29:22.956196912 +0000 UTC m=+0.430747789 container attach a7bdb64027a2141775655fe6ec21d4faa1b593d47b20db5735d2f3a2930247dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-09T02:29:22.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:22 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:22.658+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:22.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:22 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:22.658+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:22.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:22 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:22.658+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:56.023616+0000 front 2024-05-09T02:28:56.023654+0000 (oldest deadline 2024-05-09T02:29:21.923785+0000) 2024-05-09T02:29:22.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:22 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:22.535+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:22.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:22 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:22.535+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:22.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:22 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:22.535+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:22.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:22 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:22.535+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:23.321 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 09 02:29:23 smithi019 podman[118684]: 2024-05-09 02:29:23.183480056 +0000 UTC m=+0.658030918 container died a7bdb64027a2141775655fe6ec21d4faa1b593d47b20db5735d2f3a2930247dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-09T02:29:23.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:23 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:23.133+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:23.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:23 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:23.133+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:23.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:23 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:23.133+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:23.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:23 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:23.133+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:23.545 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:23 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:23.448+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:23.545 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:23 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:23.448+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:23.545 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:23 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:23.448+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:23.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:23 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:23.214+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:23.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:23 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:23.214+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:23.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:23 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:23.214+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:23.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:23 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:23.214+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:23.964 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.6.service' 2024-05-09T02:29:23.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:23 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:23.672+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:23.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:23 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:23.672+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:23.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:23 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:23.672+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:56.023616+0000 front 2024-05-09T02:28:56.023654+0000 (oldest deadline 2024-05-09T02:29:21.923785+0000) 2024-05-09T02:29:23.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:23 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:23.506+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:23.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:23 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:23.506+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:23.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:23 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:23.506+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:23.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:23 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:23.506+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:24.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:29:24.011 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-09T02:29:24.012 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-09T02:29:24.012 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.7 2024-05-09T02:29:24.322 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:24 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:24.112+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:24.322 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:24 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:24.113+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:24.322 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:24 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:24.113+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:24.322 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:24 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:24.113+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:24.332 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:24 smithi059 systemd[1]: Stopping Ceph osd.7 for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:29:24.333 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:24 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:24.330+0000 7fba4a08d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:29:24.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:24 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:24.256+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:24.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:24 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:24.256+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:24.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:24 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:24.256+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:24.445 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:24 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:24.256+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:24.648 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:24 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:24.330+0000 7fba4a08d700 -1 osd.7 105 *** Got signal Terminated *** 2024-05-09T02:29:24.648 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:24 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:24.330+0000 7fba4a08d700 -1 osd.7 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T02:29:24.648 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:24 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:24.484+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:24.648 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:24 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:24.484+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:24.648 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:24 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:24.484+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:24.648 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:24 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:24.484+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:24.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:24 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:24.442+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:24.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:24 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:24.442+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:24.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:24 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:24.442+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:24.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:24 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:24.645+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:24.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:24 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:24.646+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:24.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:24 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:24.646+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:56.023616+0000 front 2024-05-09T02:28:56.023654+0000 (oldest deadline 2024-05-09T02:29:21.923785+0000) 2024-05-09T02:29:25.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:25 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:25.099+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:25.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:25 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:25.099+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:25.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:25 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:25.099+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:25.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:25 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:25.099+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:25.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:25 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:25.466+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:25.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:25 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:25.466+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:25.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:25 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:25.466+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:25.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:25 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:25.298+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:25.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:25 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:25.298+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:25.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:25 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:25.298+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:25.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:25 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:25.298+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:25.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:25 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:25.614+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:25.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:25 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:25.614+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:25.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:25 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:25.614+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:56.023616+0000 front 2024-05-09T02:28:56.023654+0000 (oldest deadline 2024-05-09T02:29:21.923785+0000) 2024-05-09T02:29:25.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:25 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:25.487+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:25.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:25 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:25.487+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:25.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:25 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:25.487+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:25.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:25 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:25.487+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:26.544 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:26 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:26.500+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:26.545 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:26 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:26.500+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:26.545 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:26 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:26.500+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:26.545 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:26 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:26.500+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.202637+0000 front 2024-05-09T02:29:02.202619+0000 (oldest deadline 2024-05-09T02:29:25.702720+0000) 2024-05-09T02:29:26.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:26 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:26.280+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:26.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:26 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:26.280+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:26.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:26 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:26.280+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:26.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:26 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:26.280+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:26.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:26 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:26.077+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:26.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:26 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:26.077+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:26.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:26 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:26.077+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:26.572 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:26 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:26.077+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:26.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:26 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:26.612+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:26.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:26 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:26.612+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:26.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:26 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:26.612+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:56.023616+0000 front 2024-05-09T02:28:56.023654+0000 (oldest deadline 2024-05-09T02:29:21.923785+0000) 2024-05-09T02:29:26.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:26 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:26.612+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.924176+0000 front 2024-05-09T02:29:01.924146+0000 (oldest deadline 2024-05-09T02:29:26.024308+0000) 2024-05-09T02:29:26.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:26 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:26.473+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:26.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:26 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:26.473+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:26.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:26 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:26.473+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:26.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:26 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:26.473+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:26.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:26 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:26.473+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:27.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:27 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:27.072+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:27.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:27 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:27.072+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:27.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:27 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:27.072+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:27.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:27 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:27.072+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:27.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:27 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:27.072+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:27.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:27 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:27.536+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:27.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:27 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:27.536+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:27.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:27 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:27.536+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:27.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:27 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:27.536+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.202637+0000 front 2024-05-09T02:29:02.202619+0000 (oldest deadline 2024-05-09T02:29:25.702720+0000) 2024-05-09T02:29:27.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:27 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:27.312+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:27.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:27 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:27.312+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:27.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:27 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:27.312+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:27.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:27 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:27.312+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:27.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:27 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:27.633+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:27.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:27 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:27.633+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:27.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:27 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:27.633+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:56.023616+0000 front 2024-05-09T02:28:56.023654+0000 (oldest deadline 2024-05-09T02:29:21.923785+0000) 2024-05-09T02:29:27.967 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:27 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:27.633+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.924176+0000 front 2024-05-09T02:29:01.924146+0000 (oldest deadline 2024-05-09T02:29:26.024308+0000) 2024-05-09T02:29:27.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:27 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:27.484+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:27.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:27 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:27.484+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:27.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:27 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:27.484+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:27.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:27 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:27.484+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:27.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:27 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:27.484+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:28.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:28 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:28.085+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:28.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:28 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:28.085+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:28.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:28 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:28.085+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:28.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:28 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:28.085+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:28.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:28 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:28.085+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:28.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:28 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:28.317+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:28.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:28 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:28.317+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:28.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:28 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:28.317+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:28.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:28 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:28.317+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:28.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:28 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:28.519+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:28.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:28 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:28.519+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:28.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:28 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:28.519+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:28.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:28 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:28.519+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.202637+0000 front 2024-05-09T02:29:02.202619+0000 (oldest deadline 2024-05-09T02:29:25.702720+0000) 2024-05-09T02:29:28.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:28 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:28.586+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.719327+0000 front 2024-05-09T02:28:38.719399+0000 (oldest deadline 2024-05-09T02:29:02.819505+0000) 2024-05-09T02:29:28.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:28 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:28.586+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:47.321962+0000 front 2024-05-09T02:28:47.321729+0000 (oldest deadline 2024-05-09T02:29:12.621682+0000) 2024-05-09T02:29:28.967 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:28 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:28.586+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:56.023616+0000 front 2024-05-09T02:28:56.023654+0000 (oldest deadline 2024-05-09T02:29:21.923785+0000) 2024-05-09T02:29:28.967 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:28 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7[52918]: 2024-05-09T02:29:28.586+0000 7fba4263b700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.924176+0000 front 2024-05-09T02:29:01.924146+0000 (oldest deadline 2024-05-09T02:29:26.024308+0000) 2024-05-09T02:29:28.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:28 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:28.479+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:28.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:28 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:28.479+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:28.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:28 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:28.479+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:28.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:28 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:28.479+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:28.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:28 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:28.479+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:29.539 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:29 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:29.278+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:29.539 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:29 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:29.278+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:29.539 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:29 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:29.278+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:29.539 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:29 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:29.278+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:29.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:29 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:29.097+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:29.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:29 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:29.097+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:29.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:29 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:29.097+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:29.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:29 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:29.097+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:29.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:29 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:29.097+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:29.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:29 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:29.537+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:29.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:29 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:29.537+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:29.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:29 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:29.537+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:29.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:29 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:29.537+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.202637+0000 front 2024-05-09T02:29:02.202619+0000 (oldest deadline 2024-05-09T02:29:25.702720+0000) 2024-05-09T02:29:29.895 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:29 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:29.524+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:29.896 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:29 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:29.524+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:29.896 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:29 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:29.524+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:29.896 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:29 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:29.524+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:29.896 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:29 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:29.524+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:30.216 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:29 smithi059 podman[73038]: 2024-05-09 02:29:29.894211193 +0000 UTC m=+5.682976542 container died 1f63947673b00ab13c7d13c59c5e227e0a7907c9fdd994a07318226d994eaf2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS) 2024-05-09T02:29:30.516 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:30 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:30.254+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:30.516 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:30 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:30.254+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:30.516 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:30 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:30.254+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:30.516 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:30 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:30.254+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:30.516 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:30 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:30.254+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:30.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:30 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:30.071+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:30.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:30 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:30.071+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:30.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:30 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:30.071+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:30.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:30 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:30.071+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:30.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:30 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:30.071+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:30.633 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:30 smithi059 podman[73038]: 2024-05-09 02:29:30.430452209 +0000 UTC m=+6.219217557 container cleanup 1f63947673b00ab13c7d13c59c5e227e0a7907c9fdd994a07318226d994eaf2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-09T02:29:30.633 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:30 smithi059 bash[73038]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7 2024-05-09T02:29:30.634 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:30 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:30.497+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:30.634 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:30 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:30.497+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:30.634 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:30 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:30.497+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:30.634 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:30 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:30.497+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:30.634 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:30 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:30.497+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:30.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:30 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:30.512+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:30.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:30 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:30.512+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:30.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:30 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:30.512+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:30.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:30 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:30.512+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.202637+0000 front 2024-05-09T02:29:02.202619+0000 (oldest deadline 2024-05-09T02:29:25.702720+0000) 2024-05-09T02:29:30.966 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:30 smithi059 podman[73055]: 2024-05-09 02:29:30.631918034 +0000 UTC m=+0.739364290 container remove 1f63947673b00ab13c7d13c59c5e227e0a7907c9fdd994a07318226d994eaf2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-09T02:29:31.319 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:31 smithi059 podman[73146]: 2024-05-09 02:29:30.965662972 +0000 UTC m=+0.018716377 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:29:31.320 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:31 smithi059 podman[73146]: 2024-05-09 02:29:31.066061271 +0000 UTC m=+0.119114678 container create 82c772038aa843147ade4a8cf239b56c83d6d3e9c385a84982c4f848c7a94cc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T02:29:31.320 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:31 smithi059 podman[73146]: 2024-05-09 02:29:31.251591343 +0000 UTC m=+0.304644750 container init 82c772038aa843147ade4a8cf239b56c83d6d3e9c385a84982c4f848c7a94cc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T02:29:31.320 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:31 smithi059 podman[73146]: 2024-05-09 02:29:31.255172653 +0000 UTC m=+0.308226064 container start 82c772038aa843147ade4a8cf239b56c83d6d3e9c385a84982c4f848c7a94cc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T02:29:31.320 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:31 smithi059 podman[73146]: 2024-05-09 02:29:31.318116534 +0000 UTC m=+0.371169957 container attach 82c772038aa843147ade4a8cf239b56c83d6d3e9c385a84982c4f848c7a94cc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T02:29:31.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:31 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:31.221+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:31.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:31 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:31.221+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:31.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:31 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:31.221+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:31.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:31 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:31.221+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:31.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:31 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:31.221+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:31.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:31 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:31.070+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:31.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:31 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:31.070+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:31.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:31 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:31.070+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:31.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:31 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:31.070+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:31.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:31 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:31.070+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:31.716 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:31 smithi059 podman[73146]: 2024-05-09 02:29:31.559779255 +0000 UTC m=+0.612832653 container died 82c772038aa843147ade4a8cf239b56c83d6d3e9c385a84982c4f848c7a94cc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-09T02:29:31.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:31 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:31.530+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:31.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:31 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:31.530+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:31.717 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:31 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:31.530+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:31.717 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:31 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:31.530+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:31.717 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:31 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:31.530+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:32.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:31 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:31.558+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:32.045 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:31 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:31.558+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:32.045 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:31 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:31.558+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:32.045 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:31 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:31.558+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.202637+0000 front 2024-05-09T02:29:02.202619+0000 (oldest deadline 2024-05-09T02:29:25.702720+0000) 2024-05-09T02:29:32.067 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.7.service' 2024-05-09T02:29:32.092 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:32 smithi059 podman[73172]: 2024-05-09 02:29:32.046714547 +0000 UTC m=+0.479507613 container remove 82c772038aa843147ade4a8cf239b56c83d6d3e9c385a84982c4f848c7a94cc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-7-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True) 2024-05-09T02:29:32.092 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:32 smithi059 systemd[1]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.7.service: Deactivated successfully. 2024-05-09T02:29:32.092 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:32 smithi059 systemd[1]: Stopped Ceph osd.7 for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:29:32.093 INFO:journalctl@ceph.osd.7.smithi059.stdout:May 09 02:29:32 smithi059 systemd[1]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.7.service: Consumed 11min 42.623s CPU time. 2024-05-09T02:29:32.111 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:29:32.111 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-09T02:29:32.111 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-09T02:29:32.111 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.8 2024-05-09T02:29:32.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:32 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:32.045+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:32.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:32 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:32.045+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:32.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:32 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:32.045+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:32.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:32 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:32.045+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:32.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:32 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:32.045+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:32.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:32 smithi196 systemd[1]: Stopping Ceph osd.8 for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:29:32.437 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:32 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:32.260+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:32.437 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:32 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:32.260+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:32.437 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:32 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:32.260+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:32.437 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:32 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:32.260+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:32.437 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:32 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:32.260+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:32.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:32 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:32.573+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:32.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:32 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:32.573+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:32.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:32 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:32.573+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:32.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:32 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:32.573+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:32.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:32 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:32.573+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:32.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:32 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:32.434+0000 7f406bfe4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:29:32.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:32 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:32.434+0000 7f406bfe4700 -1 osd.8 105 *** Got signal Terminated *** 2024-05-09T02:29:32.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:32 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:32.434+0000 7f406bfe4700 -1 osd.8 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T02:29:32.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:32 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:32.566+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:32.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:32 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:32.566+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:32.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:32 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:32.566+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:32.795 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:32 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:32.567+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.202637+0000 front 2024-05-09T02:29:02.202619+0000 (oldest deadline 2024-05-09T02:29:25.702720+0000) 2024-05-09T02:29:33.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:33 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:33.060+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:33.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:33 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:33.060+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:33.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:33 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:33.060+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:33.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:33 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:33.060+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:33.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:33 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:33.060+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:33.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:33 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:33.211+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:33.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:33 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:33.211+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:33.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:33 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:33.211+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:33.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:33 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:33.211+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:33.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:33 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:33.211+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:33.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:33 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:33.607+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:33.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:33 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:33.607+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:33.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:33 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:33.607+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:33.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:33 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:33.607+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:33.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:33 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:33.607+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:34.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:33 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:33.614+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:34.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:33 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:33.614+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:34.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:33 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:33.614+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:34.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:33 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:33.614+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.202637+0000 front 2024-05-09T02:29:02.202619+0000 (oldest deadline 2024-05-09T02:29:25.702720+0000) 2024-05-09T02:29:34.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:34 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:34.014+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:34.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:34 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:34.014+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:34.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:34 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:34.014+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:34.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:34 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:34.014+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:34.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:34 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:34.014+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:34.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:34 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:34.233+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:34.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:34 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:34.233+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:34.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:34 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:34.233+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:34.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:34 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:34.233+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:34.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:34 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:34.233+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:34.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:34 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:34.578+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:34.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:34 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:34.578+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:34.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:34 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:34.578+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:34.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:34 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:34.578+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:34.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:34 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:34.578+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:34.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:34 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:34.578+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:35.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:34 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:34.610+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:35.045 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:34 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:34.610+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:35.045 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:34 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:34.610+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:35.045 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:34 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:34.610+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.202637+0000 front 2024-05-09T02:29:02.202619+0000 (oldest deadline 2024-05-09T02:29:25.702720+0000) 2024-05-09T02:29:35.045 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:34 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:34.610+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.904383+0000 front 2024-05-09T02:29:10.904348+0000 (oldest deadline 2024-05-09T02:29:33.704906+0000) 2024-05-09T02:29:35.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:35 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:35.024+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:35.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:35 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:35.024+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:35.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:35 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:35.024+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:35.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:35 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:35.024+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:35.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:35 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:35.024+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:35.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:35 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:35.251+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:35.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:35 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:35.251+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:35.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:35 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:35.251+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:35.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:35 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:35.251+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:35.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:35 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:35.251+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:35.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:35 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:35.252+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:35.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:35 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:35.591+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:35.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:35 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:35.591+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:35.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:35 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:35.591+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:35.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:35 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:35.591+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:35.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:35 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:35.591+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:35.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:35 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:35.591+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:36.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:35 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:35.609+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:36.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:35 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:35.609+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:36.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:35 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:35.609+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:36.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:35 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:35.609+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.202637+0000 front 2024-05-09T02:29:02.202619+0000 (oldest deadline 2024-05-09T02:29:25.702720+0000) 2024-05-09T02:29:36.045 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:35 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:35.609+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.904383+0000 front 2024-05-09T02:29:10.904348+0000 (oldest deadline 2024-05-09T02:29:33.704906+0000) 2024-05-09T02:29:36.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:35 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:35.998+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:36.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:35 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:35.998+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:36.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:35 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:35.998+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:36.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:35 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:35.998+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:36.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:35 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:35.998+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:36.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:36 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:36.218+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:36.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:36 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:36.218+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:36.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:36 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:36.218+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:36.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:36 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:36.218+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:36.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:36 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:36.218+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:36.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:36 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:36.218+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:36.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:36 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:36.594+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:36.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:36 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:36.594+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:36.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:36 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:36.594+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:36.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:36 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:36.594+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:36.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:36 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:36.594+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:36.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:36 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:36.594+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:37.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:36 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:36.573+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:37.296912+0000 front 2024-05-09T02:28:37.296897+0000 (oldest deadline 2024-05-09T02:29:02.596872+0000) 2024-05-09T02:29:37.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:36 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:36.573+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:46.098219+0000 front 2024-05-09T02:28:46.098250+0000 (oldest deadline 2024-05-09T02:29:09.598341+0000) 2024-05-09T02:29:37.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:36 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:36.573+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:55.300856+0000 front 2024-05-09T02:28:55.300633+0000 (oldest deadline 2024-05-09T02:29:18.101287+0000) 2024-05-09T02:29:37.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:36 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:36.573+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.202637+0000 front 2024-05-09T02:29:02.202619+0000 (oldest deadline 2024-05-09T02:29:25.702720+0000) 2024-05-09T02:29:37.045 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:36 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8[52947]: 2024-05-09T02:29:36.573+0000 7f406859a700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.904383+0000 front 2024-05-09T02:29:10.904348+0000 (oldest deadline 2024-05-09T02:29:33.704906+0000) 2024-05-09T02:29:37.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:37 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:37.043+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:37.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:37 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:37.043+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:37.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:37 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:37.043+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:37.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:37 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:37.043+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:37.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:37 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:37.043+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:37.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:37 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:37.043+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:11.620657+0000 front 2024-05-09T02:29:11.620752+0000 (oldest deadline 2024-05-09T02:29:36.320531+0000) 2024-05-09T02:29:37.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:37 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:37.230+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:37.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:37 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:37.230+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:37.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:37 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:37.230+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:37.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:37 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:37.230+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:37.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:37 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:37.230+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:37.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:37 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:37.230+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:37.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:37 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:37.550+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:37.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:37 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:37.550+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:37.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:37 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:37.550+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:37.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:37 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:37.550+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:37.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:37 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:37.550+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:37.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:37 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:37.550+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:38.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:38.044+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:38.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:38.044+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:38.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:38.044+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:38.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:38.044+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:38.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:38.044+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:38.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:38.044+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:11.620657+0000 front 2024-05-09T02:29:11.620752+0000 (oldest deadline 2024-05-09T02:29:36.320531+0000) 2024-05-09T02:29:38.419 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:38 smithi196 podman[73463]: 2024-05-09 02:29:38.086535643 +0000 UTC m=+5.775127609 container died 2352e114b9ce35327535da1bd96f04179d5f3a800acaa27126f7964c2a8882d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-09T02:29:38.419 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:38 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:38.273+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:38.419 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:38 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:38.273+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:38.419 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:38 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:38.273+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:38.419 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:38 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:38.273+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:38.419 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:38 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:38.273+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:38.419 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:38 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:38.273+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:38.679 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:38 smithi196 podman[73463]: 2024-05-09 02:29:38.417877167 +0000 UTC m=+6.106469132 container cleanup 2352e114b9ce35327535da1bd96f04179d5f3a800acaa27126f7964c2a8882d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T02:29:38.679 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:38 smithi196 bash[73463]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8 2024-05-09T02:29:38.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:38 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:38.551+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:38.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:38 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:38.551+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:38.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:38 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:38.551+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:38.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:38 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:38.551+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:38.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:38 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:38.551+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:38.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:38 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:38.551+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:39.044 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:38 smithi196 podman[73481]: 2024-05-09 02:29:38.677734239 +0000 UTC m=+0.591578234 container remove 2352e114b9ce35327535da1bd96f04179d5f3a800acaa27126f7964c2a8882d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-09T02:29:39.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:38.998+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:39.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:38.998+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:39.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:38.998+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:39.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:38.998+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:39.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:38.998+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:39.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:38 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:38.998+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:11.620657+0000 front 2024-05-09T02:29:11.620752+0000 (oldest deadline 2024-05-09T02:29:36.320531+0000) 2024-05-09T02:29:39.382 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:39 smithi196 podman[73573]: 2024-05-09 02:29:38.980902619 +0000 UTC m=+0.017578929 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:29:39.382 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:39 smithi196 podman[73573]: 2024-05-09 02:29:39.0954675 +0000 UTC m=+0.132143857 container create c3a3f6b5231b00b6b6b485cc58ce8cbde67ab241ea5bb5bc21f8abf695a2faa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-05-09T02:29:39.383 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:39 smithi196 podman[73573]: 2024-05-09 02:29:39.297723873 +0000 UTC m=+0.334400183 container init c3a3f6b5231b00b6b6b485cc58ce8cbde67ab241ea5bb5bc21f8abf695a2faa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True) 2024-05-09T02:29:39.383 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:39 smithi196 podman[73573]: 2024-05-09 02:29:39.300929146 +0000 UTC m=+0.337605450 container start c3a3f6b5231b00b6b6b485cc58ce8cbde67ab241ea5bb5bc21f8abf695a2faa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T02:29:39.383 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:39 smithi196 podman[73573]: 2024-05-09 02:29:39.381051862 +0000 UTC m=+0.417728176 container attach c3a3f6b5231b00b6b6b485cc58ce8cbde67ab241ea5bb5bc21f8abf695a2faa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-09T02:29:39.383 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:39 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:39.243+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:39.383 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:39 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:39.243+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:39.383 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:39 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:39.243+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:39.383 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:39 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:39.243+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:39.383 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:39 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:39.243+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:39.383 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:39 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:39.243+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:39.794 INFO:journalctl@ceph.osd.8.smithi196.stdout:May 09 02:29:39 smithi196 podman[73573]: 2024-05-09 02:29:39.597000695 +0000 UTC m=+0.633676999 container died c3a3f6b5231b00b6b6b485cc58ce8cbde67ab241ea5bb5bc21f8abf695a2faa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-09T02:29:39.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:39 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:39.592+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:39.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:39 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:39.592+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:39.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:39 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:39.592+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:39.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:39 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:39.592+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:39.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:39 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:39.592+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:39.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:39 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:39.592+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:40.189 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.8.service' 2024-05-09T02:29:40.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:29:40.227 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-09T02:29:40.227 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-09T02:29:40.227 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.9 2024-05-09T02:29:40.257 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:39 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:39.996+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:40.257 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:39 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:39.996+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:40.257 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:39 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:39.996+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:40.257 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:39 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:39.996+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:40.257 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:39 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:39.996+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:40.257 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:39 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:39.996+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:11.620657+0000 front 2024-05-09T02:29:11.620752+0000 (oldest deadline 2024-05-09T02:29:36.320531+0000) 2024-05-09T02:29:40.443 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:40 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:40.291+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:40.443 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:40 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:40.291+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:40.443 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:40 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:40.291+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:40.443 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:40 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:40.291+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:40.443 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:40 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:40.291+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:40.443 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:40 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:40.291+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:40.547 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:40 smithi019 systemd[1]: Stopping Ceph osd.9 for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:29:40.548 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:40 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:40.545+0000 7f99787c0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:29:40.821 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:40 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:40.545+0000 7f99787c0700 -1 osd.9 105 *** Got signal Terminated *** 2024-05-09T02:29:40.821 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:40 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:40.545+0000 7f99787c0700 -1 osd.9 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T02:29:40.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:40 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:40.617+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:40.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:40 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:40.617+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:40.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:40 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:40.617+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:40.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:40 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:40.617+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:40.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:40 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:40.617+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:40.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:40 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:40.617+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:41.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:40 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:40.970+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:41.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:40 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:40.970+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:41.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:40 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:40.970+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:41.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:40 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:40.970+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:41.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:40 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:40.970+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:41.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:40 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:40.970+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:11.620657+0000 front 2024-05-09T02:29:11.620752+0000 (oldest deadline 2024-05-09T02:29:36.320531+0000) 2024-05-09T02:29:41.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:41 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:41.244+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:41.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:41 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:41.244+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:41.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:41 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:41.244+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:41.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:41 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:41.244+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:41.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:41 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:41.244+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:41.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:41 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:41.244+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:41.880 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:41 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:41.615+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:41.880 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:41 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:41.615+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:41.880 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:41 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:41.615+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:41.880 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:41 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:41.615+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:41.880 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:41 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:41.615+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:41.881 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:41 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:41.615+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:42.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:42 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:42.017+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:42.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:42 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:42.017+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:42.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:42 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:42.017+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:42.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:42 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:42.017+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:42.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:42 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:42.017+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:42.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:42 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:42.017+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:11.620657+0000 front 2024-05-09T02:29:11.620752+0000 (oldest deadline 2024-05-09T02:29:36.320531+0000) 2024-05-09T02:29:42.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:42 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:42.279+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:42.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:42 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:42.279+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:42.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:42 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:42.279+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:42.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:42 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:42.279+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:42.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:42 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:42.279+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:42.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:42 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:42.279+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:42.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:42 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:42.655+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:42.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:42 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:42.655+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:42.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:42 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:42.655+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:42.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:42 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:42.655+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:42.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:42 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:42.655+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:42.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:42 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:42.656+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:43.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:43 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:43.003+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:43.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:43 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:43.003+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:43.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:43 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:43.003+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:43.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:43 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:43.003+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:43.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:43 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:43.003+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:43.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:43 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:43.003+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:11.620657+0000 front 2024-05-09T02:29:11.620752+0000 (oldest deadline 2024-05-09T02:29:36.320531+0000) 2024-05-09T02:29:43.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:43 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:43.244+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:43.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:43 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:43.244+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:43.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:43 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:43.244+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:43.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:43 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:43.244+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:43.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:43 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:43.244+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:43.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:43 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:43.244+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:43.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:43 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:43.652+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:43.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:43 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:43.652+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:43.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:43 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:43.652+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:43.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:43 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:43.652+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:43.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:43 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:43.652+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:43.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:43 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:43.652+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:44.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:43 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:43.997+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:44.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:43 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:43.997+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:44.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:43 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:43.997+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:44.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:43 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:43.997+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:44.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:43 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:43.997+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:44.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:43 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:43.997+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:11.620657+0000 front 2024-05-09T02:29:11.620752+0000 (oldest deadline 2024-05-09T02:29:36.320531+0000) 2024-05-09T02:29:44.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:44 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:44.283+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:44.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:44 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:44.283+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:44.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:44 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:44.283+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:44.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:44 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:44.283+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:44.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:44 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:44.283+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:44.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:44 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:44.283+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:44.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:44 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:44.621+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:44.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:44 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:44.621+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:44.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:44 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:44.621+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:44.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:44 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:44.621+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:44.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:44 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:44.621+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:44.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:44 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:44.621+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:44.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:44 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:44.621+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T02:29:19.381723+0000 front 2024-05-09T02:29:19.381766+0000 (oldest deadline 2024-05-09T02:29:44.581926+0000) 2024-05-09T02:29:45.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:45 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:45.017+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:28.512885+0000 front 2024-05-09T02:28:28.512819+0000 (oldest deadline 2024-05-09T02:28:52.613024+0000) 2024-05-09T02:29:45.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:45 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:45.017+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:38.514120+0000 front 2024-05-09T02:28:38.514258+0000 (oldest deadline 2024-05-09T02:29:00.814276+0000) 2024-05-09T02:29:45.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:45 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:45.017+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.416000+0000 front 2024-05-09T02:28:45.416139+0000 (oldest deadline 2024-05-09T02:29:08.816673+0000) 2024-05-09T02:29:45.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:45 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:45.017+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:52.917785+0000 front 2024-05-09T02:28:52.917718+0000 (oldest deadline 2024-05-09T02:29:16.918409+0000) 2024-05-09T02:29:45.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:45 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:45.017+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:02.819807+0000 front 2024-05-09T02:29:02.819820+0000 (oldest deadline 2024-05-09T02:29:26.919529+0000) 2024-05-09T02:29:45.321 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:45 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9[63416]: 2024-05-09T02:29:45.017+0000 7f9970d6e700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:11.620657+0000 front 2024-05-09T02:29:11.620752+0000 (oldest deadline 2024-05-09T02:29:36.320531+0000) 2024-05-09T02:29:45.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:45 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:45.316+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:45.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:45 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:45.316+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:45.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:45 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:45.316+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:45.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:45 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:45.316+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:45.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:45 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:45.316+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:45.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:45 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:45.316+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:45.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:45 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:45.632+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:45.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:45 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:45.632+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:45.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:45 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:45.632+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:45.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:45 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:45.632+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:45.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:45 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:45.632+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:45.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:45 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:45.632+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:45.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:45 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:45.632+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T02:29:19.381723+0000 front 2024-05-09T02:29:19.381766+0000 (oldest deadline 2024-05-09T02:29:44.581926+0000) 2024-05-09T02:29:46.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:46 smithi019 podman[118803]: 2024-05-09 02:29:46.24835858 +0000 UTC m=+5.815345616 container died 47857d55cbe09a820a4ab77b673c88b2f6b8444574a82982092179386f6f2d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-05-09T02:29:46.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:46 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:46.335+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:46.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:46 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:46.335+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:46.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:46 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:46.335+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:46.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:46 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:46.335+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:46.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:46 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:46.335+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:46.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:46 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:46.335+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:46.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:46 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:46.617+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:46.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:46 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:46.617+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:46.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:46 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:46.617+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:46.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:46 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:46.617+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:46.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:46 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:46.617+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:46.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:46 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:46.617+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:46.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:46 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:46.617+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T02:29:19.381723+0000 front 2024-05-09T02:29:19.381766+0000 (oldest deadline 2024-05-09T02:29:44.581926+0000) 2024-05-09T02:29:46.998 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:46 smithi019 podman[118803]: 2024-05-09 02:29:46.711976353 +0000 UTC m=+6.278963386 container cleanup 47857d55cbe09a820a4ab77b673c88b2f6b8444574a82982092179386f6f2d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T02:29:46.998 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:46 smithi019 bash[118803]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9 2024-05-09T02:29:47.320 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:46 smithi019 podman[118820]: 2024-05-09 02:29:46.996529108 +0000 UTC m=+0.746918973 container remove 47857d55cbe09a820a4ab77b673c88b2f6b8444574a82982092179386f6f2d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T02:29:47.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:47 smithi019 podman[118912]: 2024-05-09 02:29:47.222365696 +0000 UTC m=+0.018396059 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:29:47.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:47 smithi019 podman[118912]: 2024-05-09 02:29:47.322278523 +0000 UTC m=+0.118308915 container create cb6b68cfd639306247a2c4bfb7a2b7f448088c629a81eb1d19388ff20187053a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-09T02:29:47.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:47 smithi019 podman[118912]: 2024-05-09 02:29:47.516214018 +0000 UTC m=+0.312244369 container init cb6b68cfd639306247a2c4bfb7a2b7f448088c629a81eb1d19388ff20187053a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T02:29:47.571 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:47 smithi019 podman[118912]: 2024-05-09 02:29:47.519604504 +0000 UTC m=+0.315634854 container start cb6b68cfd639306247a2c4bfb7a2b7f448088c629a81eb1d19388ff20187053a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-09T02:29:47.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:47 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:47.332+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:47.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:47 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:47.332+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:47.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:47 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:47.332+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:47.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:47 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:47.332+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:47.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:47 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:47.332+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:47.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:47 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:47.332+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:47.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:47 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:47.626+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:47.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:47 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:47.626+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:47.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:47 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:47.626+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:47.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:47 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:47.626+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:47.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:47 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:47.626+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:47.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:47 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:47.626+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:47.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:47 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:47.626+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T02:29:19.381723+0000 front 2024-05-09T02:29:19.381766+0000 (oldest deadline 2024-05-09T02:29:44.581926+0000) 2024-05-09T02:29:48.071 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:47 smithi019 podman[118912]: 2024-05-09 02:29:47.574420666 +0000 UTC m=+0.370451030 container attach cb6b68cfd639306247a2c4bfb7a2b7f448088c629a81eb1d19388ff20187053a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T02:29:48.071 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 09 02:29:47 smithi019 podman[118912]: 2024-05-09 02:29:47.824020874 +0000 UTC m=+0.620051224 container died cb6b68cfd639306247a2c4bfb7a2b7f448088c629a81eb1d19388ff20187053a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-09T02:29:48.544 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:48 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:48.289+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:48.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:48 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:48.289+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:48.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:48 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:48.289+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:48.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:48 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:48.289+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:48.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:48 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:48.289+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:48.545 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:48 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:48.289+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:48.563 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.9.service' 2024-05-09T02:29:48.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:29:48.649 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-09T02:29:48.649 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-09T02:29:48.649 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.10 2024-05-09T02:29:48.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:48 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:48.654+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:48.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:48 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:48.654+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:48.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:48 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:48.654+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:48.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:48 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:48.654+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:48.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:48 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:48.654+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:48.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:48 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:48.654+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:48.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:48 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:48.654+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T02:29:19.381723+0000 front 2024-05-09T02:29:19.381766+0000 (oldest deadline 2024-05-09T02:29:44.581926+0000) 2024-05-09T02:29:48.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:48 smithi059 systemd[1]: Stopping Ceph osd.10 for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:29:49.466 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:48 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:48.974+0000 7fc22e831700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:29:49.466 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:48 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:48.974+0000 7fc22e831700 -1 osd.10 105 *** Got signal Terminated *** 2024-05-09T02:29:49.466 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:48 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:48.974+0000 7fc22e831700 -1 osd.10 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T02:29:49.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:49 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:49.309+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:49.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:49 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:49.309+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:49.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:49 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:49.309+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:49.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:49 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:49.309+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:49.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:49 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:49.309+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:49.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:49 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:49.309+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:49.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:49 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:49.656+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:49.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:49 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:49.656+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:49.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:49 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:49.656+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:49.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:49 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:49.656+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:49.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:49 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:49.656+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:49.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:49 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:49.656+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:49.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:49 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:49.656+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T02:29:19.381723+0000 front 2024-05-09T02:29:19.381766+0000 (oldest deadline 2024-05-09T02:29:44.581926+0000) 2024-05-09T02:29:50.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:50 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:50.348+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:50.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:50 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:50.348+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:50.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:50 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:50.348+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:50.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:50 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:50.348+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:50.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:50 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:50.348+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:50.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:50 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:50.348+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:50.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:50 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:50.675+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:50.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:50 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:50.675+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:50.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:50 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:50.675+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:50.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:50 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:50.675+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:50.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:50 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:50.675+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:50.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:50 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:50.675+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:50.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:50 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:50.675+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T02:29:19.381723+0000 front 2024-05-09T02:29:19.381766+0000 (oldest deadline 2024-05-09T02:29:44.581926+0000) 2024-05-09T02:29:51.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:51 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:51.346+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:51.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:51 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:51.346+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:51.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:51 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:51.346+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:51.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:51 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:51.346+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:51.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:51 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:51.346+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:51.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:51 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:51.346+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:51.959 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:51 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:51.697+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:51.959 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:51 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:51.697+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:51.959 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:51 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:51.697+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:51.959 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:51 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:51.697+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:51.959 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:51 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:51.697+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:51.959 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:51 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:51.697+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:51.959 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:51 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:51.697+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T02:29:19.381723+0000 front 2024-05-09T02:29:19.381766+0000 (oldest deadline 2024-05-09T02:29:44.581926+0000) 2024-05-09T02:29:52.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:52 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:52.304+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:52.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:52 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:52.304+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:52.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:52 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:52.304+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:52.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:52 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:52.304+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:52.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:52 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:52.304+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:52.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:52 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:52.304+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:52.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:52 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:52.304+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T02:29:28.675951+0000 front 2024-05-09T02:29:28.675966+0000 (oldest deadline 2024-05-09T02:29:51.575831+0000) 2024-05-09T02:29:52.966 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:52 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:52.707+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:52.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:52 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:52.707+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:52.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:52 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:52.707+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:52.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:52 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:52.707+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:52.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:52 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:52.707+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:52.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:52 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:52.707+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:52.967 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:52 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:52.707+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T02:29:19.381723+0000 front 2024-05-09T02:29:19.381766+0000 (oldest deadline 2024-05-09T02:29:44.581926+0000) 2024-05-09T02:29:53.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:53 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:53.350+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:53.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:53 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:53.350+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:53.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:53 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:53.350+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:53.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:53 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:53.350+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:53.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:53 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:53.350+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:53.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:53 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:53.350+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:53.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:53 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:53.350+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T02:29:28.675951+0000 front 2024-05-09T02:29:28.675966+0000 (oldest deadline 2024-05-09T02:29:51.575831+0000) 2024-05-09T02:29:54.216 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:53 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:53.743+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.373261+0000 front 2024-05-09T02:28:26.373500+0000 (oldest deadline 2024-05-09T02:28:55.173969+0000) 2024-05-09T02:29:54.248 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:53 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:53.743+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:39.275033+0000 front 2024-05-09T02:28:39.274978+0000 (oldest deadline 2024-05-09T02:29:02.175188+0000) 2024-05-09T02:29:54.248 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:53 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:53.743+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:42.176194+0000 front 2024-05-09T02:28:42.176086+0000 (oldest deadline 2024-05-09T02:29:11.576318+0000) 2024-05-09T02:29:54.248 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:53 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:53.743+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:51.576705+0000 front 2024-05-09T02:28:51.576809+0000 (oldest deadline 2024-05-09T02:29:17.476850+0000) 2024-05-09T02:29:54.248 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:53 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:53.743+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:01.577869+0000 front 2024-05-09T02:29:01.577889+0000 (oldest deadline 2024-05-09T02:29:25.578570+0000) 2024-05-09T02:29:54.248 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:53 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:53.743+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:10.780206+0000 front 2024-05-09T02:29:10.780186+0000 (oldest deadline 2024-05-09T02:29:33.679899+0000) 2024-05-09T02:29:54.248 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:53 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10[59559]: 2024-05-09T02:29:53.743+0000 7fc22ade7700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-09T02:29:19.381723+0000 front 2024-05-09T02:29:19.381766+0000 (oldest deadline 2024-05-09T02:29:44.581926+0000) 2024-05-09T02:29:54.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:54 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:54.309+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:54.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:54 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:54.309+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:54.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:54 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:54.309+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:54.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:54 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:54.309+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:54.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:54 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:54.309+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:54.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:54 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:54.309+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:54.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:54 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:54.309+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T02:29:28.675951+0000 front 2024-05-09T02:29:28.675966+0000 (oldest deadline 2024-05-09T02:29:51.575831+0000) 2024-05-09T02:29:54.900 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:54 smithi059 podman[73263]: 2024-05-09 02:29:54.60802266 +0000 UTC m=+5.763378395 container died 7f08b240316dfadd5c8956f2bf33508569c7717c3859fe2867c8117d5db321fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-09T02:29:55.216 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:54 smithi059 podman[73263]: 2024-05-09 02:29:54.898721998 +0000 UTC m=+6.054077762 container cleanup 7f08b240316dfadd5c8956f2bf33508569c7717c3859fe2867c8117d5db321fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T02:29:55.216 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:54 smithi059 bash[73263]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10 2024-05-09T02:29:55.216 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:55 smithi059 podman[73280]: 2024-05-09 02:29:55.141371647 +0000 UTC m=+0.533523890 container remove 7f08b240316dfadd5c8956f2bf33508569c7717c3859fe2867c8117d5db321fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T02:29:55.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:55 smithi059 podman[73369]: 2024-05-09 02:29:55.458743988 +0000 UTC m=+0.106535162 container create 7db13b3de1d8a0663cdeacbccc65e106a300a3c6fa44bd2825bb305cc62a1940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-09T02:29:55.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:55 smithi059 podman[73369]: 2024-05-09 02:29:55.371031252 +0000 UTC m=+0.018822452 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:29:55.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:55 smithi059 podman[73369]: 2024-05-09 02:29:55.661073126 +0000 UTC m=+0.308864294 container init 7db13b3de1d8a0663cdeacbccc65e106a300a3c6fa44bd2825bb305cc62a1940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-09T02:29:55.716 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:55 smithi059 podman[73369]: 2024-05-09 02:29:55.664347711 +0000 UTC m=+0.312139108 container start 7db13b3de1d8a0663cdeacbccc65e106a300a3c6fa44bd2825bb305cc62a1940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-09T02:29:55.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:55 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:55.359+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:55.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:55 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:55.359+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:55.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:55 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:55.359+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:55.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:55 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:55.359+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:55.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:55 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:55.359+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:55.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:55 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:55.359+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:55.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:55 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:55.359+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T02:29:28.675951+0000 front 2024-05-09T02:29:28.675966+0000 (oldest deadline 2024-05-09T02:29:51.575831+0000) 2024-05-09T02:29:55.979 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:55 smithi059 podman[73369]: 2024-05-09 02:29:55.719052146 +0000 UTC m=+0.366843329 container attach 7db13b3de1d8a0663cdeacbccc65e106a300a3c6fa44bd2825bb305cc62a1940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-09T02:29:56.290 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:55 smithi059 podman[73369]: 2024-05-09 02:29:55.978565635 +0000 UTC m=+0.626356827 container died 7db13b3de1d8a0663cdeacbccc65e106a300a3c6fa44bd2825bb305cc62a1940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-09T02:29:56.540 INFO:journalctl@ceph.osd.10.smithi059.stdout:May 09 02:29:56 smithi059 podman[73395]: 2024-05-09 02:29:56.522320858 +0000 UTC m=+0.533701387 container remove 7db13b3de1d8a0663cdeacbccc65e106a300a3c6fa44bd2825bb305cc62a1940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T02:29:56.544 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.10.service' 2024-05-09T02:29:56.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:29:56.584 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-09T02:29:56.584 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-09T02:29:56.584 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.11 2024-05-09T02:29:56.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:56 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:56.387+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:56.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:56 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:56.387+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:56.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:56 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:56.387+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:56.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:56 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:56.387+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:56.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:56 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:56.387+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:56.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:56 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:56.387+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:56.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:56 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:56.387+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T02:29:28.675951+0000 front 2024-05-09T02:29:28.675966+0000 (oldest deadline 2024-05-09T02:29:51.575831+0000) 2024-05-09T02:29:56.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:56 smithi196 systemd[1]: Stopping Ceph osd.11 for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:29:57.294 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:56 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:56.869+0000 7fea63d67700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:29:57.294 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:56 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:56.869+0000 7fea63d67700 -1 osd.11 105 *** Got signal Terminated *** 2024-05-09T02:29:57.294 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:56 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:56.869+0000 7fea63d67700 -1 osd.11 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T02:29:57.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:57 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:57.357+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:57.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:57 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:57.357+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:57.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:57 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:57.357+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:57.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:57 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:57.357+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:57.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:57 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:57.357+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:57.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:57 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:57.357+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:57.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:57 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:57.357+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T02:29:28.675951+0000 front 2024-05-09T02:29:28.675966+0000 (oldest deadline 2024-05-09T02:29:51.575831+0000) 2024-05-09T02:29:58.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:58 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:58.402+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:58.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:58 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:58.402+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:58.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:58 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:58.402+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:58.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:58 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:58.402+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:58.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:58 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:58.402+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:58.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:58 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:58.402+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:58.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:58 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:58.402+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T02:29:28.675951+0000 front 2024-05-09T02:29:28.675966+0000 (oldest deadline 2024-05-09T02:29:51.575831+0000) 2024-05-09T02:29:59.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:59 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:59.357+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:29:59.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:59 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:59.357+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:29:59.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:59 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:59.357+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:29:59.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:59 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:59.357+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:29:59.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:59 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:59.357+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:29:59.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:59 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:59.357+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:29:59.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:29:59 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:29:59.357+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T02:29:28.675951+0000 front 2024-05-09T02:29:28.675966+0000 (oldest deadline 2024-05-09T02:29:51.575831+0000) 2024-05-09T02:30:00.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:00 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:30:00.403+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:30:00.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:00 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:30:00.403+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:30:00.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:00 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:30:00.403+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:30:00.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:00 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:30:00.403+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:30:00.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:00 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:30:00.403+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:30:00.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:00 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:30:00.403+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:30:00.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:00 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:30:00.403+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T02:29:28.675951+0000 front 2024-05-09T02:29:28.675966+0000 (oldest deadline 2024-05-09T02:29:51.575831+0000) 2024-05-09T02:30:01.794 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:01 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:30:01.355+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-09T02:28:26.866248+0000 front 2024-05-09T02:28:26.866334+0000 (oldest deadline 2024-05-09T02:28:56.866988+0000) 2024-05-09T02:30:01.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:01 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:30:01.355+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-05-09T02:28:36.867741+0000 front 2024-05-09T02:28:36.867707+0000 (oldest deadline 2024-05-09T02:29:05.668300+0000) 2024-05-09T02:30:01.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:01 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:30:01.355+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-05-09T02:28:45.668745+0000 front 2024-05-09T02:28:45.668803+0000 (oldest deadline 2024-05-09T02:29:09.168968+0000) 2024-05-09T02:30:01.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:01 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:30:01.355+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-09T02:28:54.370360+0000 front 2024-05-09T02:28:54.370468+0000 (oldest deadline 2024-05-09T02:29:18.470386+0000) 2024-05-09T02:30:01.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:01 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:30:01.355+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-05-09T02:29:04.272071+0000 front 2024-05-09T02:29:04.271931+0000 (oldest deadline 2024-05-09T02:29:29.571726+0000) 2024-05-09T02:30:01.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:01 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:30:01.355+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-05-09T02:29:12.472502+0000 front 2024-05-09T02:29:12.472438+0000 (oldest deadline 2024-05-09T02:29:34.772519+0000) 2024-05-09T02:30:01.795 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:01 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11[59881]: 2024-05-09T02:30:01.355+0000 7fea5c315700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-05-09T02:29:28.675951+0000 front 2024-05-09T02:29:28.675966+0000 (oldest deadline 2024-05-09T02:29:51.575831+0000) 2024-05-09T02:30:02.747 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:02 smithi196 podman[73689]: 2024-05-09 02:30:02.485837598 +0000 UTC m=+5.707405861 container died d91b37888210c211c3e648d3529be0a453a9264b3a2d31c0891425888b2287d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-09T02:30:03.014 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:02 smithi196 podman[73689]: 2024-05-09 02:30:02.819538203 +0000 UTC m=+6.041106461 container cleanup d91b37888210c211c3e648d3529be0a453a9264b3a2d31c0891425888b2287d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-09T02:30:03.014 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:02 smithi196 bash[73689]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11 2024-05-09T02:30:03.294 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:03 smithi196 podman[73707]: 2024-05-09 02:30:03.012349511 +0000 UTC m=+0.527290657 container remove d91b37888210c211c3e648d3529be0a453a9264b3a2d31c0891425888b2287d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-09T02:30:03.692 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:03 smithi196 podman[73797]: 2024-05-09 02:30:03.342088108 +0000 UTC m=+0.017727140 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T02:30:03.692 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:03 smithi196 podman[73797]: 2024-05-09 02:30:03.463133207 +0000 UTC m=+0.138772206 container create 92bfc65cfca3a17593c7cddd36eef64a6e024600f5fa26af9d0e2f9f31eb701f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-09T02:30:03.987 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:03 smithi196 podman[73797]: 2024-05-09 02:30:03.690791616 +0000 UTC m=+0.366430620 container init 92bfc65cfca3a17593c7cddd36eef64a6e024600f5fa26af9d0e2f9f31eb701f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2) 2024-05-09T02:30:03.987 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:03 smithi196 podman[73797]: 2024-05-09 02:30:03.694576875 +0000 UTC m=+0.370215877 container start 92bfc65cfca3a17593c7cddd36eef64a6e024600f5fa26af9d0e2f9f31eb701f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T02:30:03.987 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:03 smithi196 podman[73797]: 2024-05-09 02:30:03.782118231 +0000 UTC m=+0.457757233 container attach 92bfc65cfca3a17593c7cddd36eef64a6e024600f5fa26af9d0e2f9f31eb701f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-09T02:30:04.286 INFO:journalctl@ceph.osd.11.smithi196.stdout:May 09 02:30:03 smithi196 podman[73797]: 2024-05-09 02:30:03.985848216 +0000 UTC m=+0.661487228 container died 92bfc65cfca3a17593c7cddd36eef64a6e024600f5fa26af9d0e2f9f31eb701f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T02:30:04.559 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@osd.11.service' 2024-05-09T02:30:04.595 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:30:04.595 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-09T02:30:04.595 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-09T02:30:04.595 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.a 2024-05-09T02:30:04.890 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:30:04 smithi019 systemd[1]: Stopping Ceph mds.a for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:30:05.165 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:30:04 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-a[70699]: 2024-05-09T02:30:04.887+0000 7fb5a9a68700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:30:05.165 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:30:04 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-a[70699]: 2024-05-09T02:30:04.887+0000 7fb5a9a68700 -1 mds.a *** got signal Terminated *** 2024-05-09T02:30:09.173 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:30:08 smithi019 podman[119019]: 2024-05-09 02:30:08.907677561 +0000 UTC m=+4.115984209 container died 3dc436e93cf4dec1ddd220b0cd739038c6d5d2a25dfed94fc48c1c4f5f515cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-09T02:30:09.571 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:30:09 smithi019 podman[119019]: 2024-05-09 02:30:09.362520396 +0000 UTC m=+4.570827031 container cleanup 3dc436e93cf4dec1ddd220b0cd739038c6d5d2a25dfed94fc48c1c4f5f515cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T02:30:09.571 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 09 02:30:09 smithi019 bash[119019]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-a 2024-05-09T02:30:09.780 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.a.service' 2024-05-09T02:30:09.817 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:30:09.818 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-09T02:30:09.818 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-09T02:30:09.818 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.d 2024-05-09T02:30:10.071 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 02:30:09 smithi019 systemd[1]: Stopping Ceph mds.d for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:30:10.571 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 02:30:10 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-d[69440]: 2024-05-09T02:30:10.174+0000 7f2e30389700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:30:10.571 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 02:30:10 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-d[69440]: 2024-05-09T02:30:10.174+0000 7f2e30389700 -1 mds.d *** got signal Terminated *** 2024-05-09T02:30:14.503 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 02:30:14 smithi019 podman[119151]: 2024-05-09 02:30:14.190792334 +0000 UTC m=+4.149540991 container died 40e01ca93a14effc5b8c6abb05e611581eab598d776d02a95be6a9ba14404449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T02:30:14.821 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 02:30:14 smithi019 podman[119151]: 2024-05-09 02:30:14.599636983 +0000 UTC m=+4.558385645 container cleanup 40e01ca93a14effc5b8c6abb05e611581eab598d776d02a95be6a9ba14404449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-d, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T02:30:14.821 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 09 02:30:14 smithi019 bash[119151]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-d 2024-05-09T02:30:15.100 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.d.service' 2024-05-09T02:30:15.136 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:30:15.136 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-09T02:30:15.136 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-09T02:30:15.137 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.g 2024-05-09T02:30:15.571 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:30:15 smithi019 systemd[1]: Stopping Ceph mds.g for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:30:15.571 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:30:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-g[70241]: 2024-05-09T02:30:15.445+0000 7f2375da8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:30:15.571 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:30:15 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-g[70241]: 2024-05-09T02:30:15.445+0000 7f2375da8700 -1 mds.g *** got signal Terminated *** 2024-05-09T02:30:17.456 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:30:17 smithi019 podman[119271]: 2024-05-09 02:30:17.188907359 +0000 UTC m=+1.827282122 container died 39eab8a6e5710a2b455524e879dce33337911ce1d1a185238b2c6b0e0d05e822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-g, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-05-09T02:30:17.821 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:30:17 smithi019 podman[119271]: 2024-05-09 02:30:17.553639917 +0000 UTC m=+2.192014679 container cleanup 39eab8a6e5710a2b455524e879dce33337911ce1d1a185238b2c6b0e0d05e822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-g, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-09T02:30:17.821 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 09 02:30:17 smithi019 bash[119271]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-g 2024-05-09T02:30:17.970 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.g.service' 2024-05-09T02:30:18.004 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:30:18.004 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-09T02:30:18.004 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-09T02:30:18.004 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.j 2024-05-09T02:30:18.317 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 02:30:18 smithi019 systemd[1]: Stopping Ceph mds.j for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:30:18.571 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 02:30:18 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-j[68974]: 2024-05-09T02:30:18.315+0000 7fd76eedb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:30:18.571 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 02:30:18 smithi019 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-j[68974]: 2024-05-09T02:30:18.315+0000 7fd76eedb700 -1 mds.j *** got signal Terminated *** 2024-05-09T02:30:22.766 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 02:30:22 smithi019 podman[119393]: 2024-05-09 02:30:22.34244352 +0000 UTC m=+4.119784344 container died 49737d7ffb42c9daeb5a0b2c7c05c5aeeb3285ea536a5032e0a877105bf49c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0) 2024-05-09T02:30:22.766 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 02:30:22 smithi019 podman[119393]: 2024-05-09 02:30:22.764929649 +0000 UTC m=+4.542270475 container cleanup 49737d7ffb42c9daeb5a0b2c7c05c5aeeb3285ea536a5032e0a877105bf49c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-09T02:30:23.071 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 02:30:22 smithi019 bash[119393]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-j 2024-05-09T02:30:23.071 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 09 02:30:22 smithi019 podman[119407]: 2024-05-09 02:30:22.982959947 +0000 UTC m=+0.635516778 container remove 49737d7ffb42c9daeb5a0b2c7c05c5aeeb3285ea536a5032e0a877105bf49c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-j, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-09T02:30:23.218 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.j.service' 2024-05-09T02:30:23.251 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:30:23.252 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-09T02:30:23.252 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-09T02:30:23.252 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.b 2024-05-09T02:30:23.558 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:30:23 smithi059 systemd[1]: Stopping Ceph mds.b for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:30:23.840 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:30:23 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-b[64759]: 2024-05-09T02:30:23.555+0000 7f4ce3da1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:30:23.840 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:30:23 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-b[64759]: 2024-05-09T02:30:23.555+0000 7f4ce3da1700 -1 mds.b *** got signal Terminated *** 2024-05-09T02:30:25.466 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:30:25 smithi059 podman[73475]: 2024-05-09 02:30:25.1852163 +0000 UTC m=+1.738553132 container died de12d771cb4367d534f6990c4294529011a99daa99ea3e84c05666456b75de69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-05-09T02:30:25.816 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:30:25 smithi059 podman[73475]: 2024-05-09 02:30:25.605907507 +0000 UTC m=+2.159244339 container cleanup de12d771cb4367d534f6990c4294529011a99daa99ea3e84c05666456b75de69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-09T02:30:25.816 INFO:journalctl@ceph.mds.b.smithi059.stdout:May 09 02:30:25 smithi059 bash[73475]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-b 2024-05-09T02:30:25.958 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.b.service' 2024-05-09T02:30:25.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:30:25.995 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-09T02:30:25.995 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-09T02:30:25.996 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.e 2024-05-09T02:30:26.328 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:30:26 smithi059 systemd[1]: Stopping Ceph mds.e for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:30:26.716 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:30:26 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-e[64149]: 2024-05-09T02:30:26.325+0000 7f2f32321700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:30:26.716 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:30:26 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-e[64149]: 2024-05-09T02:30:26.325+0000 7f2f32321700 -1 mds.e *** got signal Terminated *** 2024-05-09T02:30:29.466 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:30:29 smithi059 podman[73609]: 2024-05-09 02:30:29.211381466 +0000 UTC m=+2.995097632 container died f355fbdce2f62126f5f7495fb28b168c9967a5f3a3cf867e798bf6fded3b7175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-09T02:30:29.966 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:30:29 smithi059 podman[73609]: 2024-05-09 02:30:29.5509249 +0000 UTC m=+3.334641063 container cleanup f355fbdce2f62126f5f7495fb28b168c9967a5f3a3cf867e798bf6fded3b7175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-09T02:30:29.966 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:30:29 smithi059 bash[73609]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-e 2024-05-09T02:30:29.966 INFO:journalctl@ceph.mds.e.smithi059.stdout:May 09 02:30:29 smithi059 podman[73623]: 2024-05-09 02:30:29.752094506 +0000 UTC m=+0.537453348 container remove f355fbdce2f62126f5f7495fb28b168c9967a5f3a3cf867e798bf6fded3b7175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-e, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-09T02:30:29.986 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.e.service' 2024-05-09T02:30:30.020 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:30:30.020 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-09T02:30:30.020 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-09T02:30:30.020 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.h 2024-05-09T02:30:30.357 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:30:30 smithi059 systemd[1]: Stopping Ceph mds.h for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:30:30.716 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:30:30 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-h[64452]: 2024-05-09T02:30:30.354+0000 7fa359126700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:30:30.716 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:30:30 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-h[64452]: 2024-05-09T02:30:30.354+0000 7fa359126700 -1 mds.h *** got signal Terminated *** 2024-05-09T02:30:33.466 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:30:33 smithi059 podman[73729]: 2024-05-09 02:30:33.209039573 +0000 UTC m=+2.966386521 container died 3c406c8ba9fd450955bf3ac46406466a9b922b359fe64526a6c349a9f9df7a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-09T02:30:33.814 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:30:33 smithi059 podman[73729]: 2024-05-09 02:30:33.586289212 +0000 UTC m=+3.343636149 container cleanup 3c406c8ba9fd450955bf3ac46406466a9b922b359fe64526a6c349a9f9df7a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-h, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507) 2024-05-09T02:30:33.815 INFO:journalctl@ceph.mds.h.smithi059.stdout:May 09 02:30:33 smithi059 bash[73729]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-h 2024-05-09T02:30:33.961 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.h.service' 2024-05-09T02:30:33.994 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:30:33.994 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-09T02:30:33.994 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-09T02:30:33.994 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.k 2024-05-09T02:30:34.466 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:30:34 smithi059 systemd[1]: Stopping Ceph mds.k for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:30:34.466 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:30:34 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-k[63830]: 2024-05-09T02:30:34.298+0000 7fd50e777700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:30:34.466 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:30:34 smithi059 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-k[63830]: 2024-05-09T02:30:34.298+0000 7fd50e777700 -1 mds.k *** got signal Terminated *** 2024-05-09T02:30:37.466 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:30:37 smithi059 podman[73851]: 2024-05-09 02:30:37.188498956 +0000 UTC m=+2.966616508 container died 23ac5e3c1872106d001bfadb096566002393d4c9d961fffeb01435e74a40b4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T02:30:37.926 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:30:37 smithi059 podman[73851]: 2024-05-09 02:30:37.665976038 +0000 UTC m=+3.444093598 container cleanup 23ac5e3c1872106d001bfadb096566002393d4c9d961fffeb01435e74a40b4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-k, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T02:30:37.926 INFO:journalctl@ceph.mds.k.smithi059.stdout:May 09 02:30:37 smithi059 bash[73851]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-k 2024-05-09T02:30:38.054 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.k.service' 2024-05-09T02:30:38.086 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:30:38.086 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-09T02:30:38.086 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-09T02:30:38.086 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.c 2024-05-09T02:30:38.410 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:30:38 smithi196 systemd[1]: Stopping Ceph mds.c for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:30:38.794 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:30:38 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-c[65291]: 2024-05-09T02:30:38.551+0000 7efcaec98700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:30:38.794 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:30:38 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-c[65291]: 2024-05-09T02:30:38.551+0000 7efcaec98700 -1 mds.c *** got signal Terminated *** 2024-05-09T02:30:41.544 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:30:41 smithi196 podman[73903]: 2024-05-09 02:30:41.189631154 +0000 UTC m=+2.902716229 container stop 1b60ffd11ad0658943e02e8d038327b8d19b4ad7b89915d6e6a0ccae7f962f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T02:30:41.544 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:30:41 smithi196 podman[73903]: 2024-05-09 02:30:41.216783766 +0000 UTC m=+2.929868833 container died 1b60ffd11ad0658943e02e8d038327b8d19b4ad7b89915d6e6a0ccae7f962f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-c, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T02:30:41.990 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:30:41 smithi196 podman[73903]: 2024-05-09 02:30:41.633428601 +0000 UTC m=+3.346513665 container cleanup 1b60ffd11ad0658943e02e8d038327b8d19b4ad7b89915d6e6a0ccae7f962f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T02:30:41.990 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:30:41 smithi196 bash[73903]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-c 2024-05-09T02:30:41.990 INFO:journalctl@ceph.mds.c.smithi196.stdout:May 09 02:30:41 smithi196 podman[73928]: 2024-05-09 02:30:41.785103142 +0000 UTC m=+0.587925730 container remove 1b60ffd11ad0658943e02e8d038327b8d19b4ad7b89915d6e6a0ccae7f962f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-c, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T02:30:42.004 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.c.service' 2024-05-09T02:30:42.038 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:30:42.039 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-09T02:30:42.039 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-09T02:30:42.039 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.f 2024-05-09T02:30:42.294 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 02:30:42 smithi196 systemd[1]: Stopping Ceph mds.f for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:30:42.794 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 02:30:42 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-f[64986]: 2024-05-09T02:30:42.363+0000 7fcc54bfd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:30:42.794 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 02:30:42 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-f[64986]: 2024-05-09T02:30:42.363+0000 7fcc54bfd700 -1 mds.f *** got signal Terminated *** 2024-05-09T02:30:45.450 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 02:30:45 smithi196 podman[74035]: 2024-05-09 02:30:45.190493852 +0000 UTC m=+2.935343011 container died c43f3f422142342f7ee77348ac684739274650239e1e4efcaf03fe445df32a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-09T02:30:45.794 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 02:30:45 smithi196 podman[74035]: 2024-05-09 02:30:45.538100463 +0000 UTC m=+3.282949628 container cleanup c43f3f422142342f7ee77348ac684739274650239e1e4efcaf03fe445df32a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507) 2024-05-09T02:30:45.794 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 02:30:45 smithi196 bash[74035]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-f 2024-05-09T02:30:45.794 INFO:journalctl@ceph.mds.f.smithi196.stdout:May 09 02:30:45 smithi196 podman[74049]: 2024-05-09 02:30:45.689282126 +0000 UTC m=+0.493369636 container remove c43f3f422142342f7ee77348ac684739274650239e1e4efcaf03fe445df32a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-f, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T02:30:45.902 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.f.service' 2024-05-09T02:30:45.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:30:45.936 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-09T02:30:45.936 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-09T02:30:45.936 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.i 2024-05-09T02:30:46.260 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:30:46 smithi196 systemd[1]: Stopping Ceph mds.i for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:30:46.544 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:30:46 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i[65599]: 2024-05-09T02:30:46.258+0000 7fcd2ffdc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:30:46.544 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:30:46 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i[65599]: 2024-05-09T02:30:46.258+0000 7fcd2ffdc700 -1 mds.i *** got signal Terminated *** 2024-05-09T02:30:49.543 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:30:49 smithi196 podman[74155]: 2024-05-09 02:30:49.198326005 +0000 UTC m=+3.047030153 container died 6e8623f5a409b3f80c0fc0edee7d872ad4789c304ca3c3e8a528f1357e7609c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-09T02:30:49.794 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:30:49 smithi196 podman[74155]: 2024-05-09 02:30:49.641366222 +0000 UTC m=+3.490070362 container cleanup 6e8623f5a409b3f80c0fc0edee7d872ad4789c304ca3c3e8a528f1357e7609c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0) 2024-05-09T02:30:49.794 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:30:49 smithi196 bash[74155]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i 2024-05-09T02:30:50.053 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.i.service' 2024-05-09T02:30:50.073 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:30:49 smithi196 podman[74170]: 2024-05-09 02:30:49.809163811 +0000 UTC m=+0.606765104 container remove 6e8623f5a409b3f80c0fc0edee7d872ad4789c304ca3c3e8a528f1357e7609c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-i, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T02:30:50.073 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:30:50 smithi196 systemd[1]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.i.service: Deactivated successfully. 2024-05-09T02:30:50.073 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:30:50 smithi196 systemd[1]: Stopped Ceph mds.i for a0611f3e-0da7-11ef-bc97-c7b262605968. 2024-05-09T02:30:50.074 INFO:journalctl@ceph.mds.i.smithi196.stdout:May 09 02:30:50 smithi196 systemd[1]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.i.service: Consumed 33.992s CPU time. 2024-05-09T02:30:50.084 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:30:50.085 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-09T02:30:50.085 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-09T02:30:50.085 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.l 2024-05-09T02:30:50.397 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:30:50 smithi196 systemd[1]: Stopping Ceph mds.l for a0611f3e-0da7-11ef-bc97-c7b262605968... 2024-05-09T02:30:50.794 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:30:50 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-l[64679]: 2024-05-09T02:30:50.395+0000 7fa1b28d3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T02:30:50.794 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:30:50 smithi196 ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-l[64679]: 2024-05-09T02:30:50.395+0000 7fa1b28d3700 -1 mds.l *** got signal Terminated *** 2024-05-09T02:30:53.480 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:30:53 smithi196 podman[74276]: 2024-05-09 02:30:53.202807355 +0000 UTC m=+2.907435902 container died c0f206047bfa50a15c733dbdc90c585bfc07fcb6894e07d58a19871eed5164bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T02:30:53.794 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:30:53 smithi196 podman[74276]: 2024-05-09 02:30:53.570323901 +0000 UTC m=+3.274952447 container cleanup c0f206047bfa50a15c733dbdc90c585bfc07fcb6894e07d58a19871eed5164bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-l, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-09T02:30:53.794 INFO:journalctl@ceph.mds.l.smithi196.stdout:May 09 02:30:53 smithi196 bash[74276]: ceph-a0611f3e-0da7-11ef-bc97-c7b262605968-mds-l 2024-05-09T02:30:53.961 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0611f3e-0da7-11ef-bc97-c7b262605968@mds.l.service' 2024-05-09T02:30:53.993 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T02:30:53.993 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-09T02:30:53.994 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T02:30:54.195 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:30:56.928 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T02:30:57.128 INFO:teuthology.orchestra.run.smithi059.stdout:Deleting cluster with fsid: a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:30:59.688 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T02:30:59.876 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:31:02.298 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T02:31:02.335 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T02:31:02.369 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T02:31:02.399 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-09T02:31:02.403 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698978/remote/smithi019/crash 2024-05-09T02:31:02.404 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/crash -- . 2024-05-09T02:31:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:tar: /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T02:31:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:tar: Error is not recoverable: exiting now 2024-05-09T02:31:02.449 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698978/remote/smithi059/crash 2024-05-09T02:31:02.450 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/crash -- . 2024-05-09T02:31:02.485 INFO:teuthology.orchestra.run.smithi059.stderr:tar: /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T02:31:02.486 INFO:teuthology.orchestra.run.smithi059.stderr:tar: Error is not recoverable: exiting now 2024-05-09T02:31:02.488 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698978/remote/smithi196/crash 2024-05-09T02:31:02.490 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/crash -- . 2024-05-09T02:31:02.531 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T02:31:02.532 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-05-09T02:31:02.533 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-09T02:31:02.533 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-09T02:31:02.597 INFO:tasks.cephadm:Compressing logs... 2024-05-09T02:31:02.597 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T02:31:02.600 DEBUG:teuthology.orchestra.run.smithi059:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T02:31:02.603 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T02:31:02.630 INFO:teuthology.orchestra.run.smithi059.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T02:31:02.631 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T02:31:02.631 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.log 2024-05-09T02:31:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T02:31:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-09T02:31:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-09T02:31:02.635 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-09T02:31:02.636 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.7.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-09T02:31:02.636 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-09T02:31:02.636 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-09T02:31:02.636 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-09T02:31:02.636 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T02:31:02.636 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-09T02:31:02.636 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T02:31:02.636 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-09T02:31:02.636 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-09T02:31:02.637 INFO:teuthology.orchestra.run.smithi019.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T02:31:02.638 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T02:31:02.638 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.log 2024-05-09T02:31:02.638 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T02:31:02.638 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-09T02:31:02.638 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-09T02:31:02.641 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-09T02:31:02.641 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-09T02:31:02.641 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-09T02:31:02.642 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-09T02:31:02.642 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.0.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-09T02:31:02.642 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T02:31:02.642 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mon.a.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-09T02:31:02.642 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T02:31:02.642 INFO:teuthology.orchestra.run.smithi019.stderr: 87.3% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T02:31:02.642 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-09T02:31:02.642 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-09T02:31:02.642 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.d.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-09T02:31:02.647 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T02:31:02.647 INFO:teuthology.orchestra.run.smithi196.stderr:: No such file or directory 2024-05-09T02:31:02.647 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.log 2024-05-09T02:31:02.647 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-09T02:31:02.647 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-09T02:31:02.648 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T02:31:02.648 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-09T02:31:02.648 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-09T02:31:02.648 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-09T02:31:02.648 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-09T02:31:02.648 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-09T02:31:02.648 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T02:31:02.648 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-09T02:31:02.648 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-09T02:31:02.648 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T02:31:02.649 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-09T02:31:02.649 INFO:teuthology.orchestra.run.smithi019.stderr: 94.5% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-09T02:31:02.655 INFO:teuthology.orchestra.run.smithi019.stderr: 84.2% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T02:31:02.674 INFO:teuthology.orchestra.run.smithi019.stderr: 90.3% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T02:31:02.686 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mon.b.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.k.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T02:31:02.689 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.2.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.f.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.audit.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-volume.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.5.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.c.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mon.c.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mgr.z.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.i.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.11.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.8.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.cephadm.log: /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.l.log: 83.7% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T02:31:02.777 INFO:teuthology.orchestra.run.smithi196.stderr: 90.5% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T02:31:02.778 INFO:teuthology.orchestra.run.smithi196.stderr: 87.3% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T02:31:02.781 INFO:teuthology.orchestra.run.smithi196.stderr: 93.6% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-09T02:31:02.790 INFO:teuthology.orchestra.run.smithi019.stderr: 93.3% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-09T02:31:02.791 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T02:31:02.802 INFO:teuthology.orchestra.run.smithi059.stderr: 92.2% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-09T02:31:02.815 INFO:teuthology.orchestra.run.smithi059.stderr: 83.7% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T02:31:02.819 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T02:31:02.824 INFO:teuthology.orchestra.run.smithi059.stderr: 90.5% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T02:31:02.831 INFO:teuthology.orchestra.run.smithi019.stderr: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T02:31:02.884 INFO:teuthology.orchestra.run.smithi196.stderr: 91.0% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T02:31:02.889 INFO:teuthology.orchestra.run.smithi196.stderr: 92.3% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-09T02:31:02.936 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T02:31:03.069 INFO:teuthology.orchestra.run.smithi196.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T02:31:03.366 INFO:teuthology.orchestra.run.smithi019.stderr: 90.3% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-09T02:31:03.442 INFO:teuthology.orchestra.run.smithi196.stderr: 93.7% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-09T02:31:03.583 INFO:teuthology.orchestra.run.smithi059.stderr: 93.3% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-09T02:31:03.761 INFO:teuthology.orchestra.run.smithi196.stderr: 91.6% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-09T02:31:03.861 INFO:teuthology.orchestra.run.smithi059.stderr: 93.7% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-09T02:31:04.277 INFO:teuthology.orchestra.run.smithi059.stderr: 91.8% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-09T02:31:04.349 INFO:teuthology.orchestra.run.smithi019.stderr: 93.5% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-09T02:31:04.456 INFO:teuthology.orchestra.run.smithi019.stderr: 90.1% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-09T02:31:04.792 INFO:teuthology.orchestra.run.smithi059.stderr: 93.9% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-09T02:31:06.811 INFO:teuthology.orchestra.run.smithi196.stderr: 93.9% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-09T02:31:07.994 INFO:teuthology.orchestra.run.smithi196.stderr: 93.7% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-09T02:31:11.744 INFO:teuthology.orchestra.run.smithi059.stderr: 93.6% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-09T02:31:54.800 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-09T02:31:55.976 INFO:teuthology.orchestra.run.smithi059.stderr: 91.6% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-09T02:31:57.648 INFO:teuthology.orchestra.run.smithi059.stderr: 91.2% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-09T02:32:06.010 INFO:teuthology.orchestra.run.smithi196.stderr: 91.3% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-09T02:32:09.677 INFO:teuthology.orchestra.run.smithi196.stderr: 91.2% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-09T02:32:16.905 INFO:teuthology.orchestra.run.smithi059.stderr: 91.3% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-09T02:32:17.405 INFO:teuthology.orchestra.run.smithi196.stderr: 91.4% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-09T02:32:20.180 INFO:teuthology.orchestra.run.smithi019.stderr: 91.6% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-09T02:32:20.780 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-09T02:32:20.781 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-05-09T02:32:20.782 INFO:teuthology.orchestra.run.smithi059.stderr:real 1m18.165s 2024-05-09T02:32:20.782 INFO:teuthology.orchestra.run.smithi059.stderr:user 3m50.185s 2024-05-09T02:32:20.782 INFO:teuthology.orchestra.run.smithi059.stderr:sys 0m6.940s 2024-05-09T02:32:29.050 INFO:teuthology.orchestra.run.smithi196.stderr: 91.2% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-09T02:32:29.052 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-05-09T02:32:29.052 INFO:teuthology.orchestra.run.smithi196.stderr:real 1m26.435s 2024-05-09T02:32:29.052 INFO:teuthology.orchestra.run.smithi196.stderr:user 3m39.917s 2024-05-09T02:32:29.052 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m7.320s 2024-05-09T02:32:45.810 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-09T02:32:54.074 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/a0611f3e-0da7-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-09T02:32:54.076 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-05-09T02:32:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:real 1m51.462s 2024-05-09T02:32:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:user 3m49.460s 2024-05-09T02:32:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m7.070s 2024-05-09T02:32:54.077 INFO:tasks.cephadm:Archiving logs... 2024-05-09T02:32:54.077 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698978/remote/smithi019/log 2024-05-09T02:32:54.078 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T02:33:33.831 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698978/remote/smithi059/log 2024-05-09T02:33:33.832 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T02:34:14.522 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698978/remote/smithi196/log 2024-05-09T02:34:14.523 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T02:34:53.983 INFO:tasks.cephadm:Removing cluster... 2024-05-09T02:34:53.983 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 --force 2024-05-09T02:34:54.187 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:34:54.770 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 --force 2024-05-09T02:34:54.972 INFO:teuthology.orchestra.run.smithi059.stdout:Deleting cluster with fsid: a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:34:55.515 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0611f3e-0da7-11ef-bc97-c7b262605968 --force 2024-05-09T02:34:55.709 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: a0611f3e-0da7-11ef-bc97-c7b262605968 2024-05-09T02:34:56.233 INFO:tasks.cephadm:Removing cephadm ... 2024-05-09T02:34:56.234 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T02:34:56.255 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T02:34:56.270 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T02:34:56.286 INFO:tasks.cephadm:Teardown complete 2024-05-09T02:34:56.286 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T02:34:56.297 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T02:34:56.297 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T02:34:56.300 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T02:34:56.312 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T02:34:56.447 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T02:34:56.458 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T02:34:56.458 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T02:34:56.461 DEBUG:teuthology.orchestra.run.smithi059:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T02:34:56.463 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T02:34:56.475 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-05-09T02:34:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-05-09T02:34:56.477 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-05-09T02:34:56.481 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T02:34:56.481 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-05-09T02:34:56.481 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv01.front.sepia.ceph.com 3 6 377 39 +3060us[+3067us] +/- 58ms 2024-05-09T02:34:56.481 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 -2221us[-2221us] +/- 74ms 2024-05-09T02:34:56.481 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 -2124us[-2124us] +/- 91ms 2024-05-09T02:34:56.481 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T02:34:56.488 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T02:34:56.489 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-05-09T02:34:56.489 INFO:teuthology.orchestra.run.smithi059.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 +3050us[+3043us] +/- 58ms 2024-05-09T02:34:56.489 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 -2246us[-2252us] +/- 74ms 2024-05-09T02:34:56.489 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 -2117us[-2124us] +/- 91ms 2024-05-09T02:34:56.489 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T02:34:56.491 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T02:34:56.492 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-05-09T02:34:56.492 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 +3036us[+3047us] +/- 58ms 2024-05-09T02:34:56.492 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 -2240us[-2228us] +/- 74ms 2024-05-09T02:34:56.492 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -2139us[-2127us] +/- 91ms 2024-05-09T02:34:56.492 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T02:34:56.493 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T02:34:56.503 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T02:34:56.504 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T02:34:56.585 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T02:34:56.758 DEBUG:teuthology.orchestra.run.smithi059:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T02:34:56.843 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T02:34:56.954 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T02:34:56.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T02:34:56.988 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T02:34:57.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T02:34:57.016 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T02:34:57.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T02:34:57.043 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T02:34:57.056 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T02:34:57.095 INFO:teuthology.task.internal:Duration was 2579.833854 seconds 2024-05-09T02:34:57.095 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T02:34:57.129 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T02:34:57.129 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T02:34:57.132 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T02:34:57.134 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T02:34:57.187 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T02:34:57.196 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T02:34:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T02:34:57.592 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T02:34:57.593 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-05-09T02:34:57.593 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T02:34:57.619 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi059.front.sepia.ceph.com 2024-05-09T02:34:57.619 DEBUG:teuthology.orchestra.run.smithi059:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T02:34:57.649 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-05-09T02:34:57.649 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T02:34:57.680 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T02:34:57.681 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T02:34:57.683 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T02:34:57.692 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T02:34:57.747 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T02:34:57.747 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T02:34:57.758 DEBUG:teuthology.orchestra.run.smithi059:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T02:34:57.765 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T02:34:59.000 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T02:34:59.012 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T02:34:59.012 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T02:34:59.055 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T02:34:59.106 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T02:34:59.139 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T02:34:59.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T02:34:59.153 DEBUG:teuthology.orchestra.run.smithi059:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T02:34:59.155 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T02:34:59.177 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-05-09T02:34:59.186 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = core 2024-05-09T02:34:59.208 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-05-09T02:34:59.240 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T02:34:59.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T02:34:59.255 DEBUG:teuthology.orchestra.run.smithi059:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T02:34:59.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T02:34:59.271 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T02:34:59.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T02:34:59.295 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T02:34:59.306 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T02:34:59.307 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698978/remote/smithi019 2024-05-09T02:34:59.307 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T02:34:59.373 DEBUG:teuthology.misc:Transferring archived files from smithi059:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698978/remote/smithi059 2024-05-09T02:34:59.374 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T02:34:59.414 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698978/remote/smithi196 2024-05-09T02:34:59.414 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T02:34:59.450 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T02:34:59.450 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T02:34:59.453 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T02:34:59.455 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T02:34:59.500 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T02:34:59.511 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T02:34:59.511 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T02:34:59.578 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T02:34:59.578 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T02:34:59.581 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T02:34:59.583 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T02:34:59.594 INFO:teuthology.orchestra.run.smithi019.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 02:34 /home/ubuntu/cephtest 2024-05-09T02:34:59.597 INFO:teuthology.orchestra.run.smithi196.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 02:34 /home/ubuntu/cephtest 2024-05-09T02:34:59.598 INFO:teuthology.orchestra.run.smithi059.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 02:34 /home/ubuntu/cephtest 2024-05-09T02:34:59.599 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T02:34:59.615 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T02:34:59.699 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/postgres}} duration: 2579.8338539600372 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-09T02:34:59.699 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T02:34:59.791 INFO:teuthology.run:pass